flattening the deep learning

time to value curve

Ashish Agarwal Vikas Bhardwaj

ashish.agarwal@in.ibm.com vbhardwaj@in.ibm.com
Globally Integrated Systems Team (GIST) Globally Integrated Systems Team (GIST)

Follow us on
https://twitter.com/search?q=%23IBMGIST&src=typd
The deeper you go, the more value you gain,
and the more you know

Artificial
Intelligence Machine Deep
and Learning
Learning
Cognitive (Neural Networks)

Applications
 deep
learning

simple
machine
learning
 see hear feel
talk learn write
read find discover
 There was a time when a
3 year old could spot a
bird better than a
computer… 165 212 177
112 129 222 255 01
18 04 33 97 32 16 02
05 19 16 85 92 62 25 11 00 01 11 06 45 01
18 04 33
03 12 19 55 12 32 65 22 77 12
01
21 29 76 25 19 58 82 32 42 255 09
181 99 62 02 71 08 29 11 31 02 155
03 112 19 155 12 32 165 22 77 19 15 11
14 92 03 59 11 43 87 78 19 22 13 121 03
IBM’s new PowerAI tools automate image recognition
18 204 33 97 32 216 102 128 55 32 68
05 19 16 85 92 62 25 11 00 01 11 133
New AI Vision software will make image recognition
81 119 06 215 19 16 12 22 19 01 67
easier and faster for developers
108 09 226 52 111 255 221 123
By Agham Shah, 08 Service
26U.S. Correspondent, IDG News
Shape
Boundary

Attenuation Technology
understands save
morphology.
91% accuracy
cancerous
time
determination.
money
Holy grail?
Premalignant lesions lives
 DATA
PREPARATION DEPLOY & INFER
most time requires different
spent here skills

up and 9 days
Iterate
running work
faster
over a becomes
and do it
quick time spent 4 hours Assisted again
lunch drops from … more parameter
80% to models selection
30% and tuning

UP & RUNNING automation is the MAINTAIN

weeks to months ‘elixir’ for those BUILD, TRAIN, ACCURACY
looking to create OPTIMIZE experience all
new models, not that pain again
very iterative
maintain old ones
IBM POWER SYSTEMS

AC922
An Acceleration Superhighway
Unleash accelerated computing potential in
the post CPU-only era

Designed for the AI Era

Architected for the modern analytics and
AI workloads that fuel insights

Delivering Enterprise-Class AI
Cutting-edge AI innovation data scientists desire,
with dependability IT requires

8
Newell – Model GTH Air Cooled Version
Realize unprecedented performance and application gains with POWER9 and NVLink 2.0
• 2 POWER9 CPUs and up to 4 “Volta” NVLink 2.0 @ 25Gb/s GPUs in a versatile 2U Linux server
• PCIe Gen4 bus has double I/O Bandwidth vs. PCIe Gen3
• CPU (Turbo)/GPU (Boost) enabled for improved data center efficiency and performance to be
maintained at high levels
High level System Overview
▪ 2-Socket, 2U Packaging
▪ 40 POWER9 Processor cores
▪ 4 NVIDIA Volta 2.0 GPUs
▪ 4 PCIe Gen4 Slots
New for Model GTH vs. GTG
▪ 2x SFF (HDD/SSD) SATA, Up to 7.7 TB storage
• Increased POWER9 Processor performance
▪ Supports 1.6TB and 3.2TB NVMe Adapters
• DD2.2 – 16 Core freq. up to 2.4GHZ
▪ Redundant Hot Swap Power Supplies and Fans
• DD2.2 – 20 Core Freq up to 2.7GHZ
▪ Default 3 year 9x5 warranty, 100% CRU • Expanded Memory offerings
• Max Memory increase to 2 TB Max
• Expanded Adapter card offerings
Newell – Model GTX – Water cooled version
Realize unprecedented performance and application gains with POWER9 and NVLink 2.0
• 2 POWER9 CPUs and up to 6 “Volta” NVLink 2.0 @ 25Gb/s GPUs in a versatile 2U Linux server
• PCIe Gen4 bus has double I/O Bandwidth vs. PCIe Gen3
• Water cooled option improving data center efficiency and enabling CPU (Turbo)/GPU (Boost)
performance to be maintained at high levels

▪ 2-socket, 2U Packaging
▪ Processors - up to 44 POWER9 cores
▪ Up to 6 NVIDIA Volta GPUs
▪ 2 TB Memory (16 DIMMs)
▪ 4 PCIe Gen4 Slots
▪ 2x SFF (HDD/SSD), SATA, 7.7 TB storage
▪ Supports 1.6TB and 3.2TB NVMe Adapters
▪ Redundant Hot Swap Power Supplies and Fans
▪ Water cooled Processor and GPU’s for maximum performance
▪ Default 3 year 9x5 warranty, 100% CRU
 Two AC922 GPU configurations available
4 GPUs - Air (4Q’17)/Water Cooled (2Q’18) 6 GPUs - Water Cooled (2Q’18)

• Up to 4 GPUs, air/water cooled options • Up to 6 GPUs, water cooled only

• 150GB/s of bandwidth from CPU-GPU • 100 GB/s of bandwidth from CPU-GPU

• Coherent access to system memory

• PCIe Gen 4 and CAPI 2.0 to InfiniBand
• Water cooled options available in 2Q’18
 Performance…
Faster Training
faster training times
and Inferencing for data scientists

Distributed Deep Learning Traditional Model Support → Large Model Support (LMS)

(Competitors) (PowerAI)
Limited memory on GPU forces Use system memory and GPU
trade-off in model size / data to support more complex models
resolution and higher resolution data

DDR4 CPU DDR4 POWER

CPU

NVLink
PCIe
Graphics
GPU Graphics
Memory GPU
Memory

12
 Deep Learning Has Revolutionized Machine Learning
# of Searches for Deep Learning from 2011 - 2017 Accuracy

100
Deep Learning
80

60

40 Traditional
Machine Learning
20

Source: Google Trends. Search term “Deep Learning” Data

 Data Science is a Team Sport

Data Engineer Data Scientist Biz Analyst Dev Ops App Developer Dev Ops

Extract Prepare Build Train Use Monetize

Evaluate Deploy Monitor
Data Data models Models models $$$

Building cognitive apps using deep learning requires multiple skillsets.

14 14
 PowerAI: Open-Source AI Offering
Ease of Use & Performance

Developer Ease-of-Use Tools

Open Source Frameworks: Supported Distribution

Faster Training Times via

HW & SW Performance Optimizations

15 15
 Enterprise Ready Build on Open Source

16
 AI Enterprise Frameworks and Models

17
 POWER9
An acceleration superhighway.
The only processor specifically designed for the AI era.

4x 9.5x 2.6x 1st

Threads per Up to 9.5x more I/O More RAM CPU to deliver
core vs x86 bandwidth than x86 possible vs. x86 PCIe gen 4

18
 Faster Training Time with Distributed Deep Learning

Recognition
9Days
What will you do?

4 Hours
4 Hours
4 Hours
4 Hours
4 Hours
4 Hours
4 Hours
4 Hours
4 Hours
4 Hours
4 Hours
4 Hours
Iterate more and create more accurate models?
Create more models?
Both?
Recognition

54x
Learning
runs with
Power 8
19
 2 Stages Deep Learning for Cognitive Solution Build

20
 Steps for Deep Learning Development

21
 AI Vision enables enterprise level DNN easier

22
 PowerAI Vision Framework
The Deep Learning Development Platform for image/video analysis

23
 Image Classification
I’m Aethopyga I’m Pycnonotus

Need to engage Deep Neural

Networks to identify Scientific
names.

Result on public cloud API : Result on public cloud API :

white, red, yellow and teal bird white and black short beak bird

User defines categories in PowerAI Vision

Acridotheres Acrocephalus Aethopyga

Train Aethopyga: 0.90708

Butorides Corvus
… >20 categories
24
Pycnonotus: 0. 99988
 Object Detection

Developers intend to create applications that monitor

compliance and safety of workers

Define regulation
requirements through data
labeling

Trained models detect

objects for insights*.

25
 Intuitive interface from data labeling to deploying trained models

1. User will define the categories and upload 2. Start model training
data set for new model training

3. Dash boarding with insights on time to train and accuracies

4. Deploy trained model for integration via REST APIS.

26
 Labeling datasets for classification and object detection

▪ Grouping datasets into categories ▪ Marking objects for training*

27
 Inbuilt models for data preprocessing

Faces: Cars: Pedestrians:

▪ Rapid labeling of data sets with pre-built feature detectors

▪ Extendable architecture to incorporate customized feature extractors.

28
 Smarter & Safer Cities
Near miss at intersections Monitor and Impose regulations

29
Car Count Demo

30
PowerAI Vision Demo

31