Managed Self-Service BI & Data As A Service

PASS DW/BI Virtual Chapter 7/12/2012 Melissa Coates

About Melissa
Business Intelligence developer

based in Charlotte, NC Sr. Consultant with Intellinet Specialize in BI and Data Warehousing solutions using the Microsoft platform
Melissa Coates Blog: http://www.sqlchick.com Twitter: @sqlchick

About Intellinet
Strategy
IT Strategy & Operations

Intellinet is a management consulting and Microsoft-centric technology services firm.

Business
Cost Reduction
Cycle Time Reduction Supply Chain, Systems, & Software Optimization Business Productivity Solutions

Process
Business Process Optimization

Revenue Enablement
E-Commerce Solutions Commercial Software Development Merger & Acquisition Assimilation

IT Strategy Roadmaps & Benchmarks Assessment, Planning, & Governance Portfolio Management Continuous Improvement Programs

Project & Program Management Service Desk & IT Operations Business Analysis & Quality Control Change Management

Technology
Portals & Collaboration
Intranets & Extranets
SharePoint

Business Intelligence
Data Warehouses, Reports & Analytics Front-end Tools
PerformancePoint / Excel

Application Development
Custom Development
.NET

Cloud-based Solutions
Office 365 / Azure

Application Lifecycle Management

Visual Studio / TFS

Corporate Social Media Solutions

Yammer / Twitter Facebook / LinkedIn

Data Integration & Management

SQL / SSIS / SSAS / SSRS

Business Process & Integration

BizTalk / SOA

Infrastructure
Server Platform AD / Exchange / Windows Server Virtualization Hyper-V Cloud Computing Office 365

http://www.intellinet.com

Agenda
Introduction to Managed Self-Service BI Overview of Microsoft Self-Service components Demo: PowerPivot | Power View | PowerPivot Gallery Techniques to Monitor, Secure, & Manage SSBI

environment
Demo: PowerPivot Management Dashboard
Introduction to Data As A Service (DaaS) Wrap-up: Keys to Success with SSBI
Not in scope for our discussion: Installation & configuration of PowerPivot for SharePoint. See MSDN + this TechEd 2012 recording: http://channel9.msdn.com/Events/TechEd/NorthAmerica/2012/DBI402

INTRODUCTION TO MANAGED SELF-SERVICE BI

Self-Service BI what do we really mean?

Commonly thought of: Empower users to create their own reports so users get what they want without having to ask IT. Generally 2 groups of users:
Data Analysts (Power Users)
True ad-hoc needs Direct data access Small % of users Producers of data

Data Consumers (Casual Users)

Guided ad-hoc needs Parameterized reports Big % of users Consumers of data

Corporate BI
Business Users

Data Warehouse, Cubes ETL

Source Data

Corporate Reports, Dashboards, Scorecards

IT pro

Corporate BI + Self-Service BI
Data Feeds, PowerPivot Models, Excel Services, Misc Files Data Warehouse, Cubes ETL Source Data
Producers

Business Users

Data Feeds, PowerPivot Models, Misc DBs, Misc Files

Corporate Reports, Dashboards, Scorecards

Business Reports, Dashboards, Scorecards, PowerPivot Models

Consumers

IT pro

A single managed environment

(SharePoint Portal)
Business Users

Needs driving SSBI

1. Productivity / time Long dev cycles for IT to develop ETL, DW, OLAP IT backlog of requests Business decision may need to be made quickly with whatever information is readily available 2. Data exploration / unpredictable ad-hoc analysis Requirements arent always known or predictable A one-time analysis may not justify augmenting the existing BI solution 3. Prototyping for a Corporate BI solution Convey requirements Reduce development cycles

Challenges with SSBI

1. Training of producers on data & tools Need to understand the data and how to use the tools Recognition when SSBI works vs. Corporate BI 2. IT support Some IT staff have the users cant handle it syndrome Instead of an IT backlog for reports; SSBI creates different demands: training, support help, documentation
3. Access of questionable data sources 4. Chaotic, unorganized environment 5. Lack of testing & validation 6. Lack of governance & change management

Managed Self-Service BI
Self-Service BI Tools Corporate BI Tools

Producer: Data Analysts -or- IT

PowerPivot Excel 2010 with Excel Services Power View Report Builder

Producer: IT
PerformancePoint Reporting Services Visio Services

Delivery:

SharePoint 2010 Portal

Managed, Monitored, Secured by:

IT Staff

OVERVIEW OF MICROSOFT SELF-SERVICE BI TOOLS

PowerPivot
Add-in to Excel 2010 In-memory solution Based on xVelocity (Vertipaq)

column-store indexes
Large volumes of data
Create mashups of data Data is embedded

PowerPivot for Excel

Introduces DAX
Schedule data refreshes in

SharePoint
Can do visualization in familiar

PowerPivot for SharePoint

Excel environment, or another tool

Excel with Excel Services

Displays workbook

on Web
Share an entire

workbook or sections
Not all Excel

functionality is supported
Integrated with

PowerPivot for SharePoint

Power View
Visual, interactive

reporting Unique data discovery Presentation-ready (like PPT slides) Silverlight-based Requires a Tabular source (either PowerPivot for SharePoint or a Tabular SSAS model) Requires Reporting Services Add-In

Report Builder
ClickOnce application

Pixel-perfect, fully

formatted reports
Export capabilities Subscription delivery Extremely powerful tool if

report developer is clever with expressions (Visual Basic)

PowerPivot Gallery
Specialized

SharePoint document library

Thumbnail

previews
Manage

data refreshes for PowerPivot workbooks

Demo
Create Model with PowerPivot Publish Model to PowerPivot Gallery Visualize Data with Power View

TECHNIQUES TO MONITOR, SECURE, & MANAGE THE SELF-SERVICE BI ENVIRONMENT

PowerPivot for SharePoint

Source: http://msdn.microsoft.com/en-us/library/ee210692.aspx

xVelocity Engine & Data Cache

SSAS Instance for PowerPivot

If displayed in SSMS: The data is actively loaded to memory

User uploads workbook Added to Content DB Query issued Data is loaded into memory
Kept in memory for 48 hours (if no other memory pressure) at which time data is unloaded to the cache (kept for another 72 hours in cache). If 5 days pass with no activity, cache file is physically deleted. Cache files stored: \Microsoft SQL Server\MSAS11.PowerPivot\OLAP\Backup

Data Refreshes 2 Approaches

PowerPivot for Excel: manually refreshed PowerPivot for SharePoint: 1. User refreshes while the workbook is open.
**Does not refresh data in the PowerPivot database.**
2. User sets up scheduled data refreshes.

This method does refresh both the Excel workbook + the PowerPivot database. (It actually sets the Refresh on Open flag in the Excel workbook to make the Excel file update itself when user opens it.)

Managing PowerPivot in SharePoint

Workload on Monitor server health incl. query durations the server Teach authors not to retrieve every field in from the model (its all in memory after all!) PowerPivot Manage memory availability: all dbs in use models must be in memory concurrently (leaving a ~10%-20% buffer)

1/2

Data Refresh Monitor refresh times & durations Schedule Ensure users must use the data refresh account configured by the administrator & not their own credentials Users have individual access to misc dbs Disk space Monitor disk space (files are cached to avoid round trips to content db)

Managing PowerPivot in SharePoint

2/2

Naming & Teach publishers to continue using same Versioning of name (i.e., avoid habit of V1, V2, or dates at PowerPivot end of file names) Models Consider minimal versioning in library (content db size issues since data is embedded in the workbook) Monitor file uploads for storage requirements (since data is embedded)
Information about the Model Requiring certain metadata fields for reports & models will aid with Search (cant see custom fields in Gallery view though)

Securing the PowerPivot Gallery

Permissions to publish
Access to PowerPivot models (aka the workbook-as-adata-source) Access to the PowerPivot Gallery document library Separate SPS document libraries Based on content Based on security Permissions on individual models

1/2

Consider using workflows for approval of new models being published

Access to reports Same as above plus (Power View, Permissions on reports are preferably the Excel Services, same as the underlying model Report Builder)

Securing the PowerPivot Gallery

Thumbnail previews possibility of showing a preview for data the user cannot see Set permissions on report the same as the model (possible security hole) Use a regular document library (instead of PowerPivot Gallery) if preview is of concern Library
Model Report

2/2

Access to data contained within the PowerPivot workbook

View Only permissions: user will get a snapshot only; no data is exposed Contribute permissions: user can download full workbook & access all data stored within the PowerPivot model

Managing PowerPivot on Desktops

RAM RAM upgrades may be needed on user machines (min 4GB give 6GB or 8GB if possible) 64-bit vs. 32- 32-bit version cannot handle data volumes bit version >1M rows 64-bit version not compatible with all Office add-ins (although the Data Mining add-in is now supported with the latest update) Query syntax IT staff may need to initially develop & dynamic PowerPivot models & have business users data maintain them (a pretty common scenario refreshes currently)

Reporting on PowerPivot Usage

PowerPivot Management Data:
SQL Server database PowerPivot Service Application

SSAS Tabular Model

Management Data Sandbox

Ad-hoc reporting

PowerPivot Management Dashboard

Customizable dashboard (need Central Admin access)

PowerPivot Management Dashboard

PowerPivot Management - Reports

Visibility into:

Queries CPU Memory Connections Workbook Sizes Users & Authors Usage Increase & Decrease Data Refresh

PowerPivot Management - Refreshing

Relies on 3 timer jobs:

http://msdn.microsoft.com/en-us/library/ee210657.aspx

Reporting on PowerPivot Usage

1

Demo
PowerPivot Management Dashboard
Ad-Hoc Reporting on PowerPivot Management Data

Managing Report Builder

Reuse & centralization Shared Data Sources (connection string) Shared Datasets (queries) Report Parts (reusable objects like charts, tables, parameters) Database views Cube perspectives Report Models are deprecated Separate SPS document libraries Based on content Based on security Global report timeout setting

1/2

Semantic layer Deployment location Workload on server

Managing Report Builder

Starter reports Deploy starter reports with connections already in place Logging

2/2

Enable Execution Logging (default: 60 days probably want to extract & store historically) Enable Client Printing - An ActiveX control is required for the print button on the toolbar

Printing

Managing Excel Services

Trusted locations Trusted Data Connection Library Trusted File Locations Trusted Data Providers Office Data Connection (ODC) files Starter reports House within a Data Connection Library Use consistent, friendly names Consider linked connections, instead of embedded Deploy starter reports with connections already in place (helps too if users are PivotTable-challenged) Data connection timeouts Allocation of memory

Workload on server

Securing Excel Services

Access to data contained within the Excel Services workbook View Only permissions: user can open, interact, refresh workbook; a snapshot may be downloaded; no data is exposed same as PowerPivot Read permissions: user can download full workbook & access all data published via Excel Services different than PowerPivot Contribute permissions: in addition to Read, user can update & delete Design permissions: in addition to Contribute, user can approve

Managing Power View

Silverlight Data source required BI Semantic Model (BISM) connection files Requires Silverlight 5 Requires a tabular model PowerPivot, or Analysis Services in Tabular Mode PowerPivot for SharePoint HTTP link to model, or BISM data connection Analysis Services Tabular Model BISM data connection Although BISM files are similar to ODC files, they dont have the same trust settings required by Excel Services

INTRODUCTION TO DATA AS A SERVICE IN SUPPORT OF SELF-SERVICE BI

Why DaaS?
You might have created a centralized report catalog

but how about a centralized data catalog? A centralized data abstraction layer allows users to explore and consume data (and perhaps publish) Microsoft examples of DaaS:
Windows Azure Marketplace (DataMarket) An internal Silverlight-based application called DSL (Data Services

Layer) as part of their CBI (Consolidated BI) environment Data Feeds library SQL Azure Labs Data Hub (not a released product yet)

Corporate BI + Self-Service BI + DaaS

Data Feeds, PowerPivot Models, Excel Services, Misc Files
Data Warehouse, Cubes ETL Centralized Data Catalog
Power Users

Business Users

Data Feeds, PowerPivot Models, Misc DBs, Misc Files

Corporate Reports, Dashboards, Scorecards

Source Data

Business Reports, Dashboards, Scorecards, PowerPivot Models

Consumers

Enterprise Report Catalog

IT pro

(SharePoint)
Business Users

How Microsoft does DaaS

1/2

Source: http://technet.microsoft.com/en-us/library/hh499043.aspx

How Microsoft does DaaS

2/2

Data Services Layer: SQL APIs, Web Services, and UI controls

Source: http://technet.microsoft.com/en-us/library/hh499043.aspx

Centralized Data Catalog

Consider including: Description of data SQL statements Business justification Version info re: changes Owner name Sample data Security model (i.e., unsecured or limited) Availability dates (i.e., if limited time)

Data Explorer
A self-service tool to Discover, Enrich & Publish data

Web-based client is Cloud Service preview (SQL Azure Labs) Desktop client has an Excel add-in; not as full-featured as Cloud

Data Hub
A Windows Azure service to create & manage a private data marketplace for your enterprise data.

In a Cloud Service preview (SQL Azure Labs). In the 1st milestone of the Data Hub roadmap.

WRAP-UP: KEYS TO SUCCESS WITH SELF-SERVICE BI

Keys to Success
Work with strengths & limitations of Self-Service BI

Find where it complements Corporate BI in your org

Training & documentation User support

Executive support & evangelists

Governance Change management

Further Reading
Increasing Productivity by Empowering Business Users with Self-Serve BI

Choose the Right Business Intelligence Technology to Suit Your Style

Self Service Key to Creating Enterprise Business Intelligence Heroes A Primer on PowerPivot Topologies and Configurations Self-Service BI: Remember to Plan for the Back-End Infrastructure Enabling Data as a Service for Self-Service Business Intelligence PowerPivot Management Dashboard TechEd 2012 Presentation: Deploying and Managing a PowerPivot for SharePoint Infrastructure Using Microsoft SQL Server 2012

Thanks for attending!

Melissa Coates Blog: http://www.sqlchick.com Twitter: @sqlchick

Creative Commons License: Attribution-NonCommercial-NoDerivative Works 3.0