Business Intelligence:

Current Status
What do I do??? How do I increase sales???? How do I make my product better???

Business Users

Mountains of Data

Mountains of Data


From Operational Systems



ERP (Enterprise Resource Planning)

Sales/Order Inventory

 

Customer Relationship Management (CRM) Web Sites

Orders Click-stream

Mountains of Data
 

Organizations have lots of data Data is not in a form that is useful to decision-makers

Not easy to review Not informative nor insightful

Todays Information Flow



Business in 90s invested in transactional systems:

Supply Chain Management (SCM) Customer Relationship Management (CRM) Enterprise Resource Planning (ERP) Manufacturing Resource Planning (MRP) Finance (budget, forecasting and reporting)

Proliferation of Data

Sales Reporting Layer

Procurement

Operations

Finance

Transaction Layer CRM

SCM

MRP

Finance

Silos of data by functional area

Data from Disparate Sources

Sales Reporting Layer Sales Sales Sales

Transaction Layer Div 2 Region: B Region: A Div 1

Silos of data within large organizations

Business Intelligence


Business is now investing in Business Intelligence Business Intelligence is about making effective business decisions

What is BI?
The process by which an organization manages large amounts of data, extracting pertinent information, and turning that information into knowledge upon which actions can be taken.

What is BI?
Business intelligence (BI) is a broad category of application programs and technologies for gathering, storing, analyzing, and providing access to data to help enterprise users make better business decisions.

BI
 

Involves PEOPLE and Technology Involves using a rational approach to management Involves a continuous cycle of measurement, adjustment & re-measurement

The BI Cycle

BI
start

Reasons for BI


BI enables organizations to make well informed business decisions and gain competitive advantage. BI enables organizations to use information to quickly and constantly respond to changes.

Benefits of BI
   

Improved performance based upon timely and accurate information Elimination of guesswork Expedited decision making Early visibility of changes:

Customer buying patterns Supply chain activity Financial arrangements

Benefits of BI
 

Single Version of the truth Accurate, timely data available to all levels of the organization

To Note:
Although we call it Business Intelligence, the concepts and techniques are applicable to almost any organization including those in health care, biotech, education, government

BI Activities
BI applications include the activities of: decision support, query and reporting, online analytical processing (OLAP), statistical analysis, forecasting, and data mining.

BI Users


There are many different users who can benefit from business intelligence

Executives Business Decision Makers Information Workers Line Workers Analysts

BI SolutionsHow to make it happen



Two main components:

Data Consolidation and Storage Data Retrieval, Analysis and Presentation

BI Curriculum
    

Multi-Dimensional Analysis Data Warehousing Data Mining Dimensional Modeling Data Visualization

The Problem
ow do I retain customers? How do I increase sales???? GAP How do I make my product better???

Business People Mountains of Data

Bridging the Gap



Need data storage structures to facilitate fast analysis of huge volumes of data Need software to provide access to the data, allow flexible manipulation, and provide meaningful presentation

Data Storage Structures



Multi-Dimensional Databases
 Cubes

Multi-Dimensional Databases


Measures

Any quantitative expression Some are designated as Key Performance Indicators (KPI) Appropriate to the business process. How we describe the measures: Product/Customer/Region/Time These are the Bys What were our Customer Sales by Product Line by Region by Quarter for the past two years?.

Dimensions

Logical Structure

Multi-Dimensional Databases (Cubes)

Multi-Dimensional Database (Cube)

Business Intelligence Programs

Data Warehouse

ODS

ODS

ODS

Relational Database Programs

* ODS = Operational Data Store

Multi-Dimensional Databases

Multi-Dimensional Database (Cube)

Software Applications

Business Person

Reporting Applications Multi-Dimensional Database Analytic Applications (Cube)

Business Person

Score Cards Business Person Dashboards

Analytics


Reporting Applications

Limited user interaction Fulfill a significant portion of an organizations information needs Allow users to visualize and explore data following their train of thought Extensive interactivity

Analytic Applications

Data Warehousing

Designing a Data Warehouse

Data Warehouse Topics



Decision Support Systems

history Where data located, owners, definition, how often updated Determine for table structures

Requirements Gathering

Data Analysis

Data Warehouse


ETL Processes & Deliverables

Cleaning & Conforming

 

Valid, missing Address, gender Dimension Tables Fact Tables

Schemas
 

Data Consolidation & Storage

Customers Sales Procurement Suppliers Operations Finance

Shared Reporting Shared Data Layer

Data Warehouse

Transaction Layer SCM



CRM

MRP

Finance

Operations and financial information is shared across the organization from same core data

Data Warehouses

Multi-Dimensional Database (Cube)

Data Warehouse

ODS*

ODS

ODS

*ODS = Operational Data Store

How is data consolidated?



This is difficult!!!!!

Data is often spread across multiple systems, stored in different formats, and may even be localized for different countries

Transforming Data


Data must be transformed for consistency and meaning

Transformations may be as simple as copying columns or may be incredibly complex Common transformations include:
 

Hard-coded changes (T to 1) Looking up values in a table (mapping a customer number across disparate systems) Inserting dummy records and mapping them to unknowns (inserting an Unknown customer)

Cleansing Data


Data must be cleansed to be meaningful

All companies have bad data in their systems Data may be missing Data may be inconsistent Data may be wrong

Data Warehouses


ETL (extract, transform and load) processes are needed to create data warehouses

This is an arduous and technical process that can account for a large percentage of a BI project cost!!!!

Data Mining

Data Mining
  

The process of identifying patterns in data Goes beyond simple querying of the database Goes beyond multi-dimensional database queries as well

Data Mining


Data Mining works for problems like:

Develop a general profile for credit card customers Differentiate individuals who are poor credit risks Determine what characteristics differentiate male & female investors.

Data Mining vs. Data Query



Use data query if you already almost know what you are looking for. Use data mining to find regularities in data that are not obvious.

Data Mining Applications

   

Fraud detection Targeted Marketing Risk Management Business Analysis

Origins of Data Mining



Mathematics

Statistics Numerical Analysis

 

Artificial Intelligence/Machine Learning Computer Science

Data Storage and Manipulation

How does Data Mining work?



Uses induction-based learning:

The process of forming general concept definitions by observing specific examples of concepts to be learned.

How does Data Mining work?

What-Cha-Ma-Call-Its NOT What-Cha-Ma-Call-Its

How does Data Mining work?

Which of these are What-Cha-Ma-Call-Its?

Data Mining Process

List of Customers: -some bicycle buyers -some not

Data Mining Software

Model

List of Prospective Buyers

Model

List of Likely Buyers

Overview of Mining Strategies

Note: This representation is over-simplified and data mining strategies are continually being invented.