Académique Documents
Professionnel Documents
Culture Documents
Big Data BI and analytics: Tips and best practices for managing bigdata
When organizations are ready to analyze big data they often have big challenges. How do you prioritize and organize exponentially increasing data volumes to get accurate, current and useful analysis? How do you deliver big-data BI in a timely fashion to those who need it to make critical strategic and operational decisions? This eGuide discusses the emerging BI and analytics best practices for big-data environments, with practical advice on how to scope, plan and prioritize projects within your organization. Readers will get: Tips on best practices for planning and managing big-data analytics and BI programs A heads-up on common challenges and pitfalls to watch out for, with advice on overcoming them
Sponsored By:
SearchDataManagement.com E-Guide Big Data BI and analytics: Tips for best practices for managing big-data
E-Guide
Big Data BI and analytics: Tips for best practices for managing big-data
Table of Contents
Best Practices for Big Data Analytics Rely on Familiar Disciplines Skills Shortage, Training Present Pitfalls to Big Data Analytics Resources from SAP and Hewlett-Packard
Sponsored By:
Page 2 of 10
SearchDataManagement.com E-Guide Big Data BI and analytics: Tips for best practices for managing big-data
Sponsored By:
Page 3 of 10
SearchDataManagement.com E-Guide Big Data BI and analytics: Tips for best practices for managing big-data
process of capturing data can also make you better prepared. "Even if you don't know what you're going to use it for, start capturing the information; otherwise there is a missed opportunity, because you won't have that rich history of information [to draw on]," he said. Start small with big data Analyzing big data sets is yet another instance where it makes sense to define small, highvalue opportunities and use those as a starting point. As companies expand their data sources and create the all-important analytical models that will uncover patterns, they need to be vigilant about homing in on those patterns that are most important to their stated business objectives. "If you end up in a place where all you're doing is looking for new patterns and you can't do anything with them, you've hit a dead spot," said Yvonne Genovese, a vice president and distinguished analyst at Gartner Inc. ComScore Inc., which provides services around digital marketing intelligence, knew it would need some sort of big data strategy, yet it picked very targeted spots and built out its data analytics program over time. "We started with small bites -- taking individual flows and migrating them into different systems," said Will Duckworth, comScore's vice president of software engineering. "If you're working with any kind of scale, you can't roll something like this out overnight." Scale is something comScore is very conscious of, given the amount of data the Reston, Va.-based company processes. Back in 2009 when it started collecting 300 million records a day, Duckworth began searching in earnest for a new set of systems and infrastructure that could more efficiency handle its processing needs -- now over 600 billion records a month (23 billion a day) and still growing -- in a far more cost-efficient fashion. Leveraging open source Hadoop technologies and emerging packaged analytics tools, Duckworth has been able to make the open source environment more familiar to business analysts who are trained in SQL. He says companies need to consider scale as a primary factor when mapping out a big data analytics roadmap. "You have to consider what the
Sponsored By:
Page 4 of 10
SearchDataManagement.com E-Guide Big Data BI and analytics: Tips for best practices for managing big-data
ramp-up will look like -- how much data will you be putting in six months from now, how many more servers will you need to handle that, is the software up to the task," he explained. "People don't think about how much it is going to grow or how popular the solution might be once it's rolled into production." The other thing companies commonly lose sight of as they get enveloped in the new normal that is big data is that the "old normal" rules around data management still apply. "Information governance practices are just as important today with the notion of big data as they were yesterday with data warehousing," said Marcus Collins, research director at Gartner. "Even though companies want flexibility in terms of processing, remember that information is a corporate asset and should be treated as such."
Sponsored By:
Page 5 of 10
SearchDataManagement.com E-Guide Big Data BI and analytics: Tips for best practices for managing big-data
Sponsored By:
Page 6 of 10
SearchDataManagement.com E-Guide Big Data BI and analytics: Tips for best practices for managing big-data
fully shrink-wrapped product where you open the box, install it on servers and it runs fine. You need a good set of system administrators and solid practices around how to build out these environments." Bring on the PhDs Because much of what big data analytics brings to the table is predictive modeling or a look into future trends, the discipline of developing the models is not within the skill set of the average business user or even the traditional business intelligence (BI) data analyst. A lot of the data is in a raw form, be it in Web logs or from sensor data, thus companies need access to a cadre of experts who are versed in statistical and mathematical principles to build the advanced models that uncover patterns and actually make big data useful. "Not only do you need the IT operational skills to be able to realize value, the biggest shortage we see around big data is data scientists -- people with PhDs in statistics," said Brian Hopkins, a principal analyst at Forrester Research Inc. "Most of the data is raw -- it's not something you can read and get value out of. There will always be a need for a skill set of people who know what to do with the raw information ... and you have to build the acquisition of talent into the business case." At Reston, Va.-based comScore, where the business model is predicated on crunching through volumes of Web data to unearth trends for customers, many users are trained in predictive modeling and are also technically savvy enough to understand the impact of a particular query on overall system performance. Others, however, don't possess this level of expertise, so comScore has invested time and money in re-educating users, to orient them to think about the scale of the data and to spend time considering such details as data partitioning and load size when they are building out models and queries. At the same time, comScore has designed its big data system with checks and balances, so for example, if someone issues a query that could potentially crash the cluster, the system pops up a note to ensure the user is fully aware of the ramifications of that job. "At scale, things break pretty fast," Duckworth said. ComScore has also brought in a packaged application that adds a SQL-like environment to its Hadoop big data analytics offering, so it feels more familiar to traditional users.
Sponsored By:
Page 7 of 10
SearchDataManagement.com E-Guide Big Data BI and analytics: Tips for best practices for managing big-data
Training was also an integral part of the big data analytics strategy for Zions Bancorporation, a commercial bank holding company in Salt Lake City that has deployed the technology to do modeling and risk management for various loan portfolios. Yet the training wasn't just about learning Hadoop skills or serving as a crash course in statistical science. Rather, a considerable amount of time and energy went into acclimating the technical team so they were able to comfortably transition to a totally new way of managing data. "This is new technology that traditional and very conservative IT shops may be reluctant to implement," said Clint Johnson, senior vice president of data warehousing, BI and analytics at Zions. "You have systems administrators or database administrators who've built an entire career around a particular skill set, and then you thrust some new technology at them and say they have to learn it. There are cultural challenges you have to deal with in terms of supporting the new model." About the Author: Beth Stackpole is a freelance writer who has been covering the intersection of technology and business for 25-plus years for a variety of trade and business publications and websites.
Sponsored By:
Page 8 of 10
SearchDataManagement.com E-Guide Big Data BI and analytics: Tips for best practices for managing big-data
High Performance Organizations Empower Employees with Real-Time Analytics SAP HANA Appliance Software: Innovation Through Partnership with HP Transactional Data: Driving Real-Time Business
About SAP
SAP's vision is for companies of all sizes to become best-run businesses. Best-run businesses transform rigid value chains into dynamic business networks of customers, partners, and suppliers. They close the loop between strategy and execution, help individuals work more productively, and leverage technology for sustainable, profitable growth. This vision is in keeping with SAP's mission to accelerate business innovation for companies and industries worldwide - contributing to economic development on a grand scale.
About Hewlett-Packard
Hewlett-Packard is one of the world's largest computer companies and the foremost producer of test and measurement instruments. The company's more than 29,000 products are used by people for personal use and in industry, business, engineering, science, medicine and education.
Sponsored By:
Page 9 of 10
SearchDataManagement.com E-Guide Big Data BI and analytics: Tips for best practices for managing big-data
In addition, the company makes networking products, medical electronic equipment, instruments and systems for chemical analysis, handheld calculators and electronic components. HP is among the top 20 on the Fortune 500 list. The company had net revenue of $42.9 billion in its 1997 fiscal year. More than 56 percent of its business comes from outside the United States, and more than two-thirds of that is from Europe. Other principal markets are Japan, Canada, Australasia, the Far East and Latin America. HP ranks among the top 10 U.S. exporters. HP is No. 5 among Fortune's Most Admired Companies and No. 10 among Fortune's Best Companies to Work for in America.
Sponsored By:
Page 10 of 10