Trendwise Analytics

Copyright Trendwise Analytics

Big Data/Hadoop
Introduction
Trendwise Analytics
Agenda
1.Introduction to Big Data and Hadoop
2.Big Data Business cases
3.Technology
4.!A " #ind up

Trendwise Analytics
#hat is Big Data
Three $%s

Volume

Variety

Velocity
Trendwise Analytics
Copyright Trendwise Analytics
$olu&e o' Data

A co&&ercial aircra't
generates 3(B o'
'light sensor data in 1
hour
An )*+ syste& 'or an &id
si,e co&pany grows -y
1"2TB annually
A $ideo .u/eillance
Ca&era generates 1"
3TB data in 3 &onths
Airtel or $oda'one
generates 3TB o'
Call Details
*ecords 0CD*1
e/ery day
)/ery day 2.2 3uintillion
02.24156171 -ytes o' data
is created
i.e.8 282558555TB
Trendwise Analytics
Copyright Trendwise Analytics
Internet 9inute ....
Trendwise Analytics
9ar:et opportunity
IDC, a research firm8 predicts that
the &ar:et 'or Big Data technology and ser/ices will reach ;1<.= -illion -y 25128
up 'ro& ;3.2 -illion in 2515. That is a 45 percent"a"year growth rate >
a-out se/en ti&es the esti&ated growth rate 'or the o/erall in'or&ation
technology and co&&unications -usiness8 according to IDC.
Billions and billions: big data
becomes a big deal :
Deloitte predicts that in 25128 ?-ig
data@ will li:ely eAperience
accelerating growth and &ar:et
penetration.
Trendwise Analytics
How Companies are
using Big Data?
Trendwise Analytics
Common Big Data Customer Scenarios
in your industry
Web app
optimization
Smart meter
monitoring
Equipment
monitoring
Advertising
analysis
Life sciences
research
raud
detection
!ealthcare
outcomes
Weather
forecasting
"atural
resource
e#ploration
Social net$or%
analysis
Churn
analysis
&raffic flo$
optimization
'&
infrastructure
optimization
Legal
discovery
Trendwise Analytics
Copyright Trendwise Analytics
9
16 Apr 2013
#atson wins BeopardyC

Feb 14
th
2011 atson wins !eopardy"
beating its h#$an opponents%

atson is &'()s s#per co$p#ter b#ilt

#sing 'ig *ata Technology%
Trendwise Analytics
Big data Applications

Social media analytics ( )*eople +ou ,ay -no$) at

Lin%ed'n

.oice analytics ( Call center

&e#t analytics ( .oice of customer/

sentiment analysis/ $arranty analysis

.ideo analytics ( 'ntelligence/ policing/ retail

applications

&elecom 0 customer churn

Trendwise Analytics
Big Data at 1E
22

New $1B corporate center for software and analytics

!iring 344 data scientists

Includes financial and marketing applications,

but with special focus on industrial uses of big data

When $ill this gas turbine need maintenance5

!o$ can $e optimize the performance of a locomotive5

What is the best $ay to ma%e decisions about energy finance5

Trendwise Analytics
Copyright Trendwise Analytics
Dord (ets .&arter A-out 9ar:eting and
Design

Ford collects and aggregates data +ro$ the 4 million ,ehicles that #se
in-car sensing and re$ote app $anage$ent so+tware

The data allows to glean in+or$ation on a range o+ iss#es. +ro$ how

dri,ers are #sing their ,ehicles. to the dri,ing en,iron$ent that co#ld
help the$ i$pro,e the /#ality o+ the ,ehicle

0artnered with (icroso+t to de,elop 123C

Trendwise Analytics
Copyright Trendwise Analytics
How A&a,on Eses Big Data To 9a:e Fou
Go/e The&

A$a4on has been collecting c#sto$er in+or$ation +or years--not 5#st

addresses and pay$ent in+or$ation b#t the identity o+ e,erything that a
c#sto$er had e,er bo#ght or e,en loo6ed at%

They)re #sing that data to b#ild c#sto$er relationship

Trendwise Analytics
Copyright Trendwise Analytics
How Gin:edIn is *iding a #a/e o' Big Data
All the #ay to the Ban:

7in6ed&n is a tro,e o+ data not 5#st abo#t people. b#t how people are
$a6ing their $oney and what ind#stries they are wor6ing in and how
they connect to each other%
Trendwise Analytics
Copyright Trendwise Analytics
How AT!T is using cell phone to watch
user &o/e&entsH

AT8T has 300 million c#sto$ers

A tea$ o+ researchers is wor6ing to t#rn data collected thro#gh the

co$pany)s cell#lar networ6 into a tro,e o+ in+or$ation +or
policy$a6ers. #rban planners and tra++ic engineers%

The researchers want to see how the city changes ho#rly by loo6ing at
calls and te9t $essages relayed thro#gh cell towers aro#nd the region.
noting that certain towers see $ore acti,ity at di++erent ti$es
Trendwise Analytics
Copyright Trendwise Analytics
(o/t o' India

Aadhar pro5ect by :o,t% o+ &ndia #ses ;adoop

Trendwise Analytics
Copyright Trendwise Analytics
T)CHIJGJ(F
Hadoop Ion Hadoop
Trendwise Analytics
Copyright Trendwise Analytics
Hadoop Co&ponents
Trendwise Analytics
Copyright Trendwise Analytics
Hadoop HDD. and 9ap*educe

;adoop r#ns on ;*F1.

;adoop *istrib#ted Filesyste$

Any data stored is con,erted to bloc6s

and distrib#ted across the cl#ster nodes
Trendwise Analytics
Copyright Trendwise Analytics
Jther co&ponents
Hive
K Data Warehouse infrastructure
that provides data
summarization and ad hoc
querying on top of !adoop
PI
K A high"le/el data"'low language and
eAecution 'ra&ewor: 'or parallel
co&putation
Soop
K Sqoop is a tool designed to
help users of large data
import e#isting relational
databases into their !adoop
clusters
!ookeeper
K 6oo%eeper is a centralized
service for maintaining
configuration information/
naming/ providing
distributed synchronization/
and providing group services
Trendwise Analytics
Bene'its o' Hadoop
K
Hadoop is designed to run on cheap co&&odity
hardware
K
It auto&atically handles data replication and node
'ailure
K
Handles large /olu&es o' unstructured data easily
K
Gast -ut not least L its 'reeC 0 Jpen source1
Trendwise Analytics
Copyright Trendwise Analytics
Co&&ercial Hadoop Distri-utions

Clo#dera

;ortonwor6s

:reenpl#$. A *i,ision o+ <(C

&'( &n+o1phere 'ig&nsights

Trendwise Analytics
Technology L Ion Hadoop
KHPCC " H+CC .yste&s 'ro& GeAisIeAis *is: .olutions o''ers a pro/en8
K open"source8 data"intensi/e superco&puting plat'or&
K designed 'or the enterprise to sol/e -ig data pro-le&s.
K !"P H"#" is .A+ A(Ms i&ple&entation o' in"&e&ory data-ase technology.
K#o!$% Databases
Ney"$alues .tores L *edis8 *ia:
Colu&n Da&ily .tores L Cassandra8 HBase
Docu&ent Data-ases L CouchDB8 9ongoDB
(raph Data-ase L In'o(rid8 In'inite (raph
Trendwise Analytics
.A+ HAIA In"&e&ory Data-ase .yste&

Hana is an in"&e&ory data-ase syste&

de/eloped -y .A+ A(.

It ta:es the ad/antage o' L

low"cost o' &ain &e&ory

Dast data access o' solid state dri/es.

Data processing a-ilities

o' &ulti"core processors.

It supports -oth row"oriented

and colu&n"oriented data storage.

It incorporates power'ul graph and

teAt processing capa-ilities to wor:
with se&i and 'ull unstructured data.

.A+ has positioned HAIA as its solution

to -ig data challenges at the low end o' this
scale.
Trendwise Analytics
Copyright Trendwise Analytics
Than: FouC
Trendwise Analytics
Additional resources
Hadoop:
http:&&hadoop'apache'org&
http:&&bigdatauni(ersity'com&
)thers:
http:&&www'cloudera'com&content&cloudera&en&home'html
http:&&hortonwor*s'com&