Académique Documents
Professionnel Documents
Culture Documents
Data warehousing is the act of extracting, transferring, transforming, storing and retrieval of data for reporting and analytical purposes.
Business Intelligence (BI) is a terminology for applications that uses data stores for analytical purposes.
BI applications are not required to run on top of data warehouses, but the majority does
Source Data
Extract
Transform
Data Warehouse
BI Applications
Custom Developed Applications Data Mining
Statistical Programs
Source: Mike Schroeck, David Zinn and Bjarne Berg, Integrated Analytics Getting Increased Value from Enterprise Resource Planning Systems, Data Management Review, May, 2002; Adapted: Bjarne Berg How to Manage a BW Project, BW & Portals Conference, 2007, Miami
ERP
DW
Dividing Line
For the first go-live, keep the scope as small as possible I.e. Accounts Payable, Accounts Receivable, General ledger, Sales, Distribution
Scope
Resources
(people, technology and money)
Time
Warning
If one of these dimensions changes, you have to adjust at least one of the others 6
Business team
Business analyst
Technical team
SAP BI Architect ETL developer
Presentation developer
4-5 team members and normally 3-6 months duration depending on scope
8
E.g., Global Cost and Profitability, international cross organization or consolidated billing
Project sponsor/ Steering Committee Project Manager SAP BW Architect Business Analyst(s) Extract, Transforms and Loads Data Management (InfoCubes & ODS) Presentation Developer(s) Sr. Presentation developer Presentation developer
Note: These are roles, not positions (sometimes one team member can fill more than one role)
Basis and functional R/3 support 8-10 team members and normally 2-4 months duration depending on scope 9
Note: These are roles, not positions (sometimes one team member can fill more than one role)
Sales Team Business analyst/(sub-team lead) SAP BI developer Presentation developer(s) ETL developer
Finance Team Business analyst/(sub-team lead) SAP BI developer Presentation developer(s) ETL developer
Material Mgmt. Team Business analyst/(sub-team lead) SAP BI developer Presentation developer(s) ETL developer
10
Independent (38%)
30% Successful
11
Size the SAP BI effort based on the scope Prioritize the effort
3.
4.
Tip
Create the Milestone Plan and Scope Statement first, before attacking the budgeting process!!
Start the budgeting process by estimating the workload in terms of the development effort. Refine based on the teams skill experience and skill level 12
L M L M L M L L L L L L L L L M M M M
Financials General ledger line item (ODS) COPA Prod cost planning released cost estimates (COPC_C09) Exploded itemization standard product cost (COPC_C10) Cost and allocations (COOM_C02) Cost object controlling (0PC_C01) Order Billing Sales order Acct. Rec. (0FIAR_C03) Deliver Shipment cost details (0LES_C02) Shipment header (0LES_C11) Stages of shipment (0LES_C12) Delivery data of shipment stages (0LES_C13) Delivery service (0SD_C05) Planning and Scheduling Material Movements (0IC_C03) APO Planning SNP Integration Manufacturing Processes Production Orders Cross Applications Total Hours
216 158 216 238 216 238 216 216 216 216 216 216 216 180 216 277 277 277 277 4,298
229 286 229 286 1144 286 229 229 229 229 228 228 228 229 457 832 832 832 832 8,074
188 153 188 216 188 216 187 187 187 187 187 187 187 133 132 216 216 216 216 3,587
101 127 101 126 101 137 101 101 101 101 101 101 101 101 101 127 127 127 127 2,110
132 153 133 153 132 153 132 132 132 132 132 132 132 132 132 153 153 153 153 2,656
134 152 135 152 135 152 135 135 135 135 135 135 135 134 134 152 152 152 152 2,681
100 120 100 120 100 120 100 100 100 100 100 100 100 100 100 120 120 120 120 2,040
79 94 79 94 79 94 79 79 79 79 79 79 79 79 79 94 94 94 94 1,606
150 180 150 180 150 180 150 150 150 150 150 150 150 150 150 180 180 180 180 3,060
403 470 403 470 403 470 403 403 403 403 403 403 403 403 403 470 470 470 470 8,126
1,732 1,893 1,734 2,035 2,648 2,046 1,732 1,732 1,732 1,732 1,731 1,731 1,731 1,641 1,904 2,621 2,621 2,621 2,621 38,238
Remember that your sizing also has to be based on the teams experience and skill level.
13
The next step is to prioritize and outline the effort on a strategic timeline
Make sure your sponsor and the business community agree with your delivery schedule
14
3. Use Project Estimates & the Timeline to Create Project Load Plan
2005 qtr 1 Financials General ledger line item (ODS) 866 COPA 946.5 Prod cost planning released cost estimates (COPC_C09) Exploded itemization standard product cost (COPC_C10) Cost and allocations (COOM_C02) Cost object controlling (0PC_C01) Order Billing Sales order Accounts receivables (0FIAR_C03) Deliver Shipment cost details (0LES_C02) Shipment header (0LES_C11) Stages of shipment (0LES_C12) Delivery data of shipment stages (0LES_C13) Delivery service (0SD_C05) Planning and Scheduling Material Movements (0IC_C03) APO Planning SNP Integration Manufacturing Processes Production Orders Cross Applications Total 1,813 qtr 2 qtr 3 qtr 4 qtr 1 qtr 2 2006 qtr 3 qtr 4 qtr 1 2007 qtr 2 qtr 3
866 947 867 1017.5 1324 1023 867 1017.5 1324 1023 866 866 866 866 866 866 866 865.5 865.5 865.5 820.5 866 865.5 865.5 865.5 820.5 952 1310.5 1310.5 952 1311 1311 1311 1311 1,311 1,311
1,732 1,893 1,734 2,035 2,648 2,046 1,732 1,732 1,732 1,732 1,731 1,731 1,731 1,641 1,904 2,621 2,621 2,621 2,621
1,813
4,232
4,232
2,598
2,598
4,283
4,283
3,573
6,195
2,622
38,238
Note
There are 480 available work hours per project member per quarter. Knowing this, we can plan the number of team members we need NOTE: Remember to plan for different vacation schedules (i.e. in the USA a 1-2 weeks vacation is most common).
15
This spike in resource needs is due to an overlap in the delivery schedule Now might be a good time to review that decision
Tip
Many companies plan a 60%- 40% mix of internal and external resources for a first go-live. Also, most use $50-$90 per hr for internal budgeting and $90-$170 per hr for external resources.
16
Source: Planning and improving global software development process by Setamanit, Wakeland, Raffo, May 2006, international workshop on Global software development
Recent research have demonstrated that global projects that spends more days (duration) on similar tasks, have less defects and less re-work. Since team members are more likely to work on multiple tasks not related to the project, longer durations on developing the SAP BI system does not mean more effort (i.e. work hours).
17
System Testing
Complete?
Unit Testing
Yes
Peer Review
No
Approved?
Complete? No
Yes
Structured walkthrough
No Complete? Yes
Structured walkthrough
18
19
The OLAP software market is growing fast.. This year it is expected to exceed 6 billion dollars (over $500 million per month in sales)
OLAP Spending in Billions Source: OLAP Report
6 5 4 3 2 1 0 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007
20
On-Line Analytical Processing (OLAP) Market Market shares are changing ever faster. ERP vendors are the only companies with significant market share increases in the last 2-3 yrs.
Others have virtually vanished as market leaders i.e. Acuity, Acumate, Gentia, Holos, Information Advantage, Media, MetaCube, Mineshare, Pilot, Prodea Beacon, WhiteLight etc
Microsoft ecosystem Hyperion solution (incl. Brio) Cognos (incl. Adaytum & Frango) Business Objects (incl. Crystal) Microstrategy
25
20
15
SAP
Note: Since the overall market is growing, some companies are still growing despite loosing market share
*Cartesis was bought by PwC in 1999, Sold to APAX partners in 2004, then to Business Objects in 2007 and finally to SAP in October 2007
10
Cartesis
5
System Union
Oracle
21
OLAP Market Vendor overview Big Mergers in the BI and OLAP space is back. The number of mergers are fairly constant, but the sizes are increasing
Amount in 2007 $millions (adjusted for inflation) - Estimated 11,039 232 6,001 464 2,137 1,010 1,428 5,430 5,519 929 111 175 302 86
10,000
Year 2007 2006 2005 2004 2003 2002 2001 2000 1999 1998 1997 1996 1995 1994
8,000
6,000
4,000
2,000
0 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007
There has been a dramatic increase in merger activities, that we have not seen since 1998-1999 22
Products
Value
$19m Accrue $10m Broadbase $15m Business Objects ~$4bn CA
Buyer
2000 Pilot Software 2000 Decisionism 2000 OLAP@Work 2000 Sterling 1999 SQRIBE 1999 Next Action Technology 1999 Platinum Technology 1999 Sapling 1999 Cartesis 1999 Information Advantage 1998 Hyperion Software 1998 IQ Software 1997 AppSource 1997 Andyne 1997 Pilot Software 1996 Sinper 1996 Panorama 1996 Holistic Systems 1995 IOC 1995 STG 1995 Soft Systems 1995 IRI Software 1995 Prodea 1994 Pilot 1994 Info-Innov
Deal over $250 mill. Deal between $25million and $250million Deal under $25million
$250m Brio Technology, subsequently renamed to Brio Software $8m Business Objects ~3.5bn CA $15.5m Hyperion Solutions Unknown PwC $168m Sterling $600m Arbor Software --> Hyperion Solutions $36m Information Advantage $6.7m Arbor Software ~$60m Hummingbird ~$5m? Platinum Equity Holdings $11m Applix ~$15m? Microsoft $84m Seagate Software Unknown DecisionWorks $16.5m Informix $5.2m IQ Software $100m Oracle $36m Platinum Technology ~$28m Dun & Bradstreet Unknown Speedware
23
Seller
Magnitude
Products
Controller, Consolidator Alphablox IntelligentApps Analysis, Holos Executive Suite e.Planning MPC and Decision Intelligence DecisionWare, Alea, onVision, Plain, DeltaMiner Gentia Pilot WhiteLight MetaCube Max (later renamed to Data Analyzer) Strategy
Value
$52m Cognos Unknown IBM Unknown Sage Group
Buyer
$1.2bn Business Objects Unknown China Development Corporation $157m Cognos $52m Geac $142m Hyperion Solutions $42m Systems Union Unknown Open Ratings $1.5m Pilot Software Acquisition Corp. Unknown SymphonyRPM $1bn IBM ~$15m Microsoft $94m SPSS
Source: OLAP Report & Dr. Berg 2007.
Deal over $250 mill. Deal between $25million and $250million Deal under $25million
24
Yr
Seller
PilotWorks
Products
BusinessObjects, Crystal, Cartesis
Value
$6.8bn ($5.3bn net) SAP Unknown SAP
Buyer
2007 Business Objects 2007 Pilot Software 2007 Cartesis 2007 OutlookSoft 2007 Hyperion Solutions 2007 Applix 2006 Temtec 2006 ALG 2006 Geac 2006 MIS 2006 Extensity 2006 ProClarity Corporation 2005 SRC 2005 INEA 2005 Siebel
Cartesis Finance, Planning and Analytics OutlookSoft 5 (formerly Everest) Essbase, Hyperion Planning, HFM, former Brio TM1 and Executive Viewer Executive Viewer EPO MPC DecisionWare (including Alea) MPC and DecisionWare ProClarity SRC INEA Siebel Analytics
225m (~$300m) Business Objects $375m? SAP $3.3bn Oracle $339m, $306m net Cognos $14.5m Applix $56m Business Objects Unknown Golden Gate Capital (Extensity) Unknown Infor (Golden Gate Capital) Unknown Infor (Golden Gate Capital) ~$50m Microsoft $100m Business Objects Unknown Cartesis $5.85bn Oracle
Source: OLAP Report & Dr. Berg 2007.
Deal over $250 mill. Deal between $25million and $250million Deal under $25million
25
Definition: Multi dimensional On-Line Analytical Processing (MOLAP) Let us look at those remaining vendors
Server or client resident, proprietary data bases, architected to quickly manipulate and display data in different combinations, i.e., to provide for fast, adhoc analysis. Also known as OLAP engines, OLAP servers, data-cubes, and hyper-cubes.
Note: this category includes only those products that employ an actual physical MOLAP cube database
26
Market leaders in MOLAP Cognos PowerPlay The Canadian company Cognos is a market leader in MOLAP technology and boosts support for instances with over 190,000 users through their web services tools (server based scalability). PowerPlay is in version 8.0 now and is now over 15 years old stable technology. PowerPlay achieve some scalability by partitioning the data cubes by filters (i.e. by year). This creates many cubes, but balances the time it takes to update the cubes. PowerPlay MOLAP cubes are easy to navigate and are extremely fast due to the fact that results sets are precalculated.
27
Market leaders in MOLAP Oracle Express Oracle have taken full advantage of their Express 10g tool. You can buy either Express Server or Personal Express for single users (desktop MOLAP).
Tools for Express development and access
Express Analyzer Express Objects Express Web Publisher Express Spreadsheet Add-in - Reporting and analysis tool - Object-oriented development environment - Develop Briefings for Web Deployment - MS-Excell interface to the Express Server
- analyze sales, marketing and other corporate data - financial reporting, analysis, budgeting & planning
28
Other MOLAP Vendors There are many other MOLAP vendors. The differentiator between them is the support for building the cubes on the both the server and the client side (both is better)
Vendor Name Cognos, Inc. Microsoft Corp. Open Ratings (bought Gentia 2002) Oracle (bought Hyperion 2007) Oracle Corp. Cognos (bought Applix 2007) SAS Institute Business Objects (Seagate) Golden Gate Capital (bought Comshare 2006) Hummingbird Oracle (bought Hyperion 2007) Product Powerplay Analysis Services (OLAP Services) Gentia Essbase Express iTM/1 OLAP Server / MDDB HOLOS (discontinued in 2005) Commander BI/Analyze Brio Enterprise Where is the cube built Both Both Both Both Both Server Server Server Client Client Client
29
Definition: Relational On-Line Analytical Processing (ROLAP) ROLAP Tool Category Definition: Products that provide OLAP functionality by building multidimensional views on the fly from data stored in relational database-based data warehouses. To achieve this, ROLAP vendors employ sophisticated SQL generation engines & multidimensional oriented metadata.
Conceptual Multi-dimensional Layer
Quarter Month Week Week 1-2 January First Week 3-4 Quarter Week 5-6 February Week 7-8 Time
Data Sources
SQL
Operational Systems
All else held constant, ROLAP tools are more scalable than MOLAP tools, but is also slower
Geography
Business Users
Time
Data Warehouse
Product
U.S.
South
West
Los Angeles
Queries
ODS
30
OLAP Market leaders in ROLAP MicroStrategy A leading vendor in the ROLAP market is MicroStrategy and their ROLAP tools that is currently in version 8. Their tool architecture is highly scalable (based on underlying database) and also include 50+ new statistical functions for advanced analytics.
31
OLAP Market leaders in ROLAP MicroStrategy The tool also have improved event monitoring and broadcast features as well as structured formatting capabilities and better charting for dashboards then previous releases. You can also develop custom analytical applications more quickly using the new version.
32
OLAP Market leaders in ROLAP SAS SAS can also build ROLAP cubes using their Cube Studio in their new SAS-9 OLAP Server (you can choose between ROLAP and MOLAP) This is a powerful tool that allows quick data mart build capabilities in MOLAP, and the power of ROLAP for the Enterprise data warehouse.
33
Other ROLAP Vendors There are many other ROLAP vendors. The differentiator between them is the native support for the underlying database of your company (i.e. Microstrategy just got certified for Oracle 11g)
ROLAP Vendor Name SAP SAS CA - (Bought Platinum in 1999) CA - (bought Sterling (IA) in 2000) IBM (bought Informix 2001) Microsoft Corp. Microstrategy, Inc. SymphonyRPM (bought WhiteLight in 2002) Product BW / BI / NetWeaver OLAP Server DecisionBase (InfoBeacon) Eureka Metacube Analysis Services (OLAP Services) Microstrategy 8 WhiteLight
34
Definition: Managed Query Environments (MQE) Definition: A software package that can access one or more data sources via a simplified and optimized semantic layer.
In the semantic layer no data is stored, but instead it contains access methods and metadata to simplify and the query access interface.
Data Sources
Operational Systems
SQL
Business Users Queries
Semantic Layer
Data Warehouse
ODS
35
Market leaders in MQE Cognos ReportNet In 2003, Cognos renamed its old and trusted MQE tool known as Impromptu. The new name was ReportNet. ReportNet is part of Cognos BI 8.0. It can do complex formatting in via a web based architecture and provide query optimization in the SQL generation (faster queries). Today ReportNet is among the top-5 most used query tool and has an world-wide install and support base.
36
Market leaders in MQE Oracle Discoverer Oracle Discoverer is a market leader that has been around since the mid-1990s. It was originally known as Discoverer-2000.
While it can be used against most database types, it leverages Oracles statistical features for better query management (i.e. predictive run times).
It is a simple tool to learn, relatively low cost, and have many customizations options.
Product Alphablox, SpreadsheetBlox Brio Enterprise Business Objects Eureka ReportNet (Impromptu) BI/Query Focus Reporter Q+E Discoverer InfoSet Query SAS Query, Enterprise Reporter Crystal Reports
Class of Tool Development Environment MQE MQE Query/Reporting/MQE MQE MQE MQE Query/Reporting Query/Reporting/MQE Query/Reporting Query/Reporting/MQE Query/Reporting
39
Report Servers Tools that creates batch reports that can be pixel
formatted for statements and high-volume access.
The market leader in this segment is Crystal Reports in its new release Crystal-2008.
functions, i.e. clustering, neural networks, decision trees, discrimminant analysis, analysis of variance for complex decision making. The market leader in this segment is SAS Institute and their SAS/Stat tool
40
41
SAP NetWeaver
People Integration
Multi-Channel Access Portal Collaboration
Information Integration
Business Intelligence Knowledge Management
Process Integration
Integration Broker Business Process Management
3. Data Mining (inside BW) 4. Advanced Planning and Optimization (APO-BI) 5. Supply Chain Event Manager (SCEM-BI) 6. Customer Relationship Manager (CRM-BI)
Application Platform
J2EE ABAP
.NET
WebSphere
The DW is the source of the data for all these integrated analytical applications (iAnalytics)
42
Flat Reporting
Formatted Print Form based Static Predictable access
43
44
SAP Strategy Management (formerly PilotWorks) SAP Business Planning and Consolidation (formerly OutlookSoft 5) SAP Business Profitability Management by Acorn
KEY POINT
What we see is a temporary strategy of integrating previously acquired software under a shared marketing umbrella until SAP can digest the Business Objects acquisition (2008) and build coherent front-end tools. In the interim, SAP will have a complex number of tools with overlapping capabilities.
45
Oracle Business Intelligence Suite Business Intelligence Suite is intended for executives, managers, and front line workers , and include ad hoc query and analysis, proactive intelligence and alerts, advanced reporting, and predictive analytics.
Oracle Data Warehousing The Data Warehousing products include a graphical environment that supports design, deployment, and management of data warehouses. It include parallel database technology, and a suite of data access and management tools.
46
Source: Oracle
47
48
The daily business intelligence is presented in 46 overview pages, 287 KPIs and 769 delivered reports
Marketing Campaign Telemarketing Sales Forecast Pipeline Sales revenues Order capture Sales contracts
The interactive dashboards and embedded analytics covers a variety of scenarios and many of them allows drill down to the supporting transactions in the data warehouse or the transaction system.
49
HCM Warehouse Compensation mart Learning & development mart Recruiting mart Workforce profiling mart Supply Chain Warehouse Fulfillment & billing mart Inventory mart Manufacturing mart Procurement mart Spend mart Supply chain planning mart 50
These are the 20+ pre-delivered data marts that comes with ETL programs as well
Workforce Analytics Workforce planning Workforce rewards Workforce scorecard Industry Specific Analytics Funds transfer pricing Healthcare scorecard Manufacturing scorecard Risk Weighted capital
Supply Chain Analytics Activity based management Manufacturing scorecard Supplier rating system
These analytical applications runs on top of the 20 pre-delivered datamarts from PeopleSoft.
51
In 2007 Oracle added more content for higher education, pharmaceuticals and financial services, but is still weak on industry solutions in most other areas.
The 2008 challenge for Oracle is to expand the Fusion analytics platform to create the next generation of integrated BI applications. This will include a combination of the Business Intelligence Suite and Oracle applications
52
KEY POINTS
What we see is two tracks: One with continued development of legacy and heritance ERP BI applications, and on track with the to-be vision of integrated analytics based on the Oracle fusion as the platform, and business intelligence suite as the integrated BI applications
53
PeopleSoft EPM
SAP BW
Siebel BI
55
Virtual OLAP is a trend of merging operational and analytical data in a logical manner. Since the merger is virtual, it can have zero lag time and therefore provide real-time analytics. Many companies such as the GL company is providing tieins to Oracle applications and databases that can logically merge the data as well as transform the data to external indexing engines for extremely fast query accessing.
57
SAP launched the BI accelerator at the end of June 2006. This tool takes BI and/or operational data and creates a proprietary indexing system that allows users to access the data very quickly.
58
In Data warehousing v2.0 (Bill Immons new terminology), we are using BI applications to create new data that needs to be fed to the data warehouse, and /or the transactional system Tools are now available to build association data sets that creates updates in the transaction systems based on BI findings (i.e. customer segmentation, risk, profitability flagging etc).
59
Images: Bhanu Gupta, ASUG 2005 Illinois chapter
Time to Delivery
Low Low
High
Impact of Failure
Source: Dr. Bjarne Berg, SAP Project Management Conference - Oct. 2005, Las 60 Vegas How to pick JAD, RAD, XP or a SDLC Methodology for your IT project
61
Your Turn!
COMERIT (presentations, articles, accellerators) www.comerit.net
62