What
is Data Warehouse?
·
Known as an enterprise data warehouse (EDW).
·
A data
warehouse is a centralized repository that stores data from multiple
information sources and transforms them into a common multidimensional
data model for efficient querying and analysis.
Is used to collect
and combine data from other databases it updates this data on regular basis depending on a
company's information needs .
Examples: Annual reports.
What are the
specifications of DWs?
· Subject-oriented : Meaning that the data in the database is
organized so that all the data elements relating to the same real-world
event or object are linked together .
· Integrated : meaning that
the database contains data from most or all of an organization's operational
applications, and that this data is made consistent.
· Time varying : meaning that
the changes to the data in the database are tracked and recorded so that
reports can be produced showing changes over time .
· Non-volatile : meaning that
data in the database is never over-written or deleted, once committed, the data
is static, read-only, but retained for future reporting.
What
is Data Mart?
· A database, or collection of databases, designed to help
managers make strategic decisions about their business.
· Data marts are usually smaller than data warehouses and focus
on a particular subject or department.
Why Do We Need a
Data Mart?
· To partition data in order to Force access control
strategies.
· To speed up the queries by reducing the size of data to be
scanned.
·
To structure data in a suitable form for a user
access tool.
Data warehouse or
Data Mart?
Data Warehouse
|
Data Mart
|
Holds multiple
subject areas
|
Often holds one
subject area EX:(Finance,Sales)
|
Holds very
detailed information
|
May holds more
summarized data
|
Works to
integrate all data sources
|
Concentrates on
integrating information from set of source
|
Does not
necessarily use a dimensional model but feeds dimensional models
|
Is built focused
on a dimensional model
|
What is Data Mining?
· Data Mining is defined as
extracting information from huge sets of data.
· we can say that data mining is the
procedure of mining knowledge from data.
What is
the Business intelligence (BI)?
· Business intelligence
(BI) technology infrastructure for gaining maximum information from
available data for the purpose of improving business processes.
· Business
Intelligence produces analysis and provides information to help
making effective and high quality business decisions.
· The most common
kinds of Business Intelligence systems are:
· EIS - Executive
Information Systems
·
MIS
- Management
Information Systems
·
GIS
- Geographic
Information Systems
·
DSS
- Decision
Support Systems
·
OLAP
- Online
Analytical Processing .
· CRM - Customer
Relationship Management
· Business
Intelligence tools:
· Oracle - Siebel
Business Analytics Applications
· SAP - Business
Objects XI
· IBM - Cognos 8
BI
OLTP
vs. OLAP:
|
· OLTP (On-line Transaction Processing)
is characterized by a large number of short online transactions (INSERT,
UPDATE, and DELETE).
· OLTP very fast query processing,
maintaining data integrity in multi-access environments .
· OLAP (On-line Analytical Processing) is
characterized by relatively low size of transactions.
· Queries are often very complex and involve
aggregations. (magmo3at)
· OLAP applications are widely used by Data Mining techniques.
· In OLAP database there is aggregated And Historical data ,
stored in multi-dimensional schemas (usually star schema).
What is Big Data?
· It is a collection of large datasets that
cannot be processed using traditional computing techniques.
Benefits of Big Data:
·
Using the information existing in the social network like
Face book,
· Using the data about the previous medical
history of patients, hospitals are providing better and quick service .
// Big Data Technologies
There are various technologies in the market
from different vendors including Amazon, IBM, Microsoft, etc., to handle big
data. While looking into the technologies that handle big data, we examine the
following two classes of technology.
What is No SQL?
Is an approach
to data management and database design that's useful for very
large sets of distributed data.
// Where to use
No SQL?
When project
has unstructured big data that require real-time or offline analysis or
web/mobile application.
Example: Social
Network app, Analytics app.
Advantages and
Disadvantages of NoSQL DB.
Advantages of
NoSQL:
- Big Data
- Economics
- Flexible
data models
//Disadvantages
of No SQL:
- Maturity
- Support
- Analytics
and business intelligence
- Administration
- Expertise
//What is Hadoop?
Hadoop is a free, Java-based
programming framework that supports the processing of large data sets in a
distributed computing environment. It is part of the Apache project sponsored
by the Apache Software Foundation.
What is the Map Reduce?
Map Reduce is a software
framework that allows developers to write programs that process massive amounts
of unstructured data in parallel across a distributed cluster of processors or
stand-alone computers.
What is PL/SQL?
PL/SQL is a
combination of SQL along with the procedural features of programming languages.
It was developed by Oracle Corporation in the early 90's to enhance the
capabilities of SQL.
·
PL/SQL is one of three key programming
languages embedded in the Oracle Database, along with SQL itself and Java.
·
Short for Procedural
Language/SQL,
Oracle procedural
extension to the industry-standard SQL. By adding procedural constructs to SQL, such as encapsulation, function overloading, information hiding, block structure, conditional statements, loop statements, variable types, structured data and customized error handling,
the PL/SQL language takes on characteristics of object-oriented programming languages.
What
is the Meta Data ?
What is Metadata?
Metadata is simply defined as data about data.
The data that is used to represent other data is known as
metadata.
For example, the index of a book serves as a metadata for the contents in the
book. In other words, we can say that metadata is the summarized data that
leads us to detailed data. In terms of data warehouse, we can define metadata
as follows.
·
Metadata is the road-map to a data warehouse.
·
Metadata in a data warehouse defines the warehouse
objects.
·
Metadata acts as a directory. This directory helps the
decision support system to locate the contents of a data warehouse.
Note: In a data warehouse, we create metadata for the
data names and definitions of a given data warehouse. Along with this metadata,
additional metadata is also created for time-stamping any extracted data, the
source of extracted data.