Vous êtes sur la page 1sur 47

DEEP LEARNING IN PYTHON

Introduction to
deep learning
Deep Learning in Python

Imagine you work for a bank


You need to predict how many transactions each customer
will make next year
Deep Learning in Python

Example as seen by linear regression


Age

Bank Balance Number of


Transactions

Retirement Status


Deep Learning in Python

Example as seen by linear regression


Model with no interactions Model with interactions

Predicted Predicted
Transactions Not Retired Transactions Not Retired

Retired Retired

Bank Balance Bank Balance


Deep Learning in Python

Interactions
Neural networks account for interactions really well
Deep learning uses especially powerful neural networks
Text
Images
Videos
Audio
Source code
Deep Learning in Python

Course structure
First two chapters focus on conceptual knowledge
Debug and tune deep learning models on conventional
prediction problems
Lay the foundation for progressing towards modern
applications
This will pay o in the third and fourth chapters
Deep Learning in Python

Build deep learning models with keras


In [1]: import numpy as np

In [2]: from keras.layers import Dense

In [3]: from keras.models import Sequential

In [4]: predictors = np.loadtxt('predictors_data.csv', delimiter=',')

In [5]: n_cols = predictors.shape[1]

In [6]: model = Sequential()

In [7]: model.add(Dense(100, activation='relu', input_shape = (n_cols,)))

In [8]: model.add(Dense(100, activation='relu')

In [9]: model.add(Dense(1))
Deep Learning in Python

Deep learning models capture interactions


Age

Number of
Bank Balance Transactions

Retirement Status


Deep Learning in Python

Interactions in neural network


Input Layer
Hidden Layer
Age
Output Layer
Income
Number of
# Accounts Transactions
DEEP LEARNING IN PYTHON

Lets practice!
DEEP LEARNING IN PYTHON

Forward
propagation
Course Title

Bank transactions example


Make predictions based on:
Number of children
Number of existing accounts
Deep Learning in Python

Forward propagation
Input Hidden Layer
1
# Children 2 5
1 2
Output
9 # Transactions
-1
-1
3 1 1
# Accounts
Deep Learning in Python

Forward propagation
Input Hidden Layer
1
# Children 2 5
2
1 Output
9 # Transactions
-1
-1

3 1 1
# Accounts
Deep Learning in Python

Forward propagation
Input Hidden Layer
1
# Children 2 5
2
1 Output
9 # Transactions
-1
-1

3 1 1
# Accounts
Deep Learning in Python

Forward propagation
Input Hidden Layer
1
# Children 2 5
2
1 Output
9 # Transactions
-1
-1

3 1 1
# Accounts
Course Title

Forward propagation
Multiply - add process
Dot product
Forward propagation for one data point at a time
Output is the prediction for that data point
Deep Learning in Python

Forward propagation code


In [1]: import numpy as np

In [2]: input_data = np.array([2, 3])

In [3]: weights = {'node_0': np.array([1, 1]),


...: 'node_1': np.array([-1, 1]),
...: 'output': np.array([2, -1])}

In [4]: node_0_value = (input_data * weights['node_0']).sum()

In [5]: node_1_value = (input_data * weights['node_1']).sum()

Input Hidden Layer Output


1
2 5 2
1

3 -1
1 -1
1
Deep Learning in Python

Forward propagation code


In [6]: hidden_layer_values = np.array([node_0_value, node_1_value])

In [7]: print(hidden_layer_values)
[5, 1]

In [8]: output = (hidden_layer_values * weights['output']).sum()

In [9]: print(output)
9

Input Hidden Layer Output


1
2 5 2
1
9
3 -1
1 -1
1
DEEP LEARNING IN PYTHON

Lets practice!
DEEP LEARNING IN PYTHON

Activation
functions
Deep Learning in Python

Linear vs Nonlinear Functions

Linear Functions Nonlinear Functions


Deep Learning in Python

Activation functions
Applied to node inputs to produce node output
Deep Learning in Python

Improving our neural network


Input Hidden Layer
1
2 5
1 2
Output
9
-1
-1

3 1 1
Deep Learning in Python

Activation functions
Input Hidden Layer

1 tanh(2+3)
2
2
1 Output

9
-1
-1

3 1
tanh(-2+3)
Deep Learning in Python

ReLU (Rectified Linear Activation)


Rectifier
Deep Learning in Python

Activation functions
In [1]: import numpy as np

In [2]: input_data = np.array([-1, 2])

In [3]: weights = {'node_0': np.array([3, 3]),


...: 'node_1': np.array([1, 5]),
...: 'output': np.array([2, -1])}

In [4]: node_0_input = (input_data * weights['node_0']).sum()

In [5]: node_0_output = np.tanh(node_0_input)

In [6]: node_1_input = (input_data * weights['node_1']).sum()

In [7]: node_1_output = np.tanh(node_1_input)

In [8]: hidden_layer_outputs = np.array([node_0_output, node_1_output])

In [9]: output = (hidden_layer_output * weights['output']).sum()

In [10]: print(output)
1.2382242525694254
DEEP LEARNING IN PYTHON

Lets practice!
DEEP LEARNING IN PYTHON

Deeper networks
Deep Learning in Python

Multiple hidden layers


2 -1 Age
3
-3
4 1

4
2 7
-5 2
55

Calculate with ReLU Activation Function


Deep Learning in Python

Multiple hidden layers


2 -1 Age
3
-3
4 1

4
2 7
-5 2
55

Calculate with ReLU Activation Function


Deep Learning in Python

Multiple hidden layers


2 -1 Age
3
-3
4 1

4
2 7
-5 2
55

Calculate with ReLU Activation Function


Deep Learning in Python

Multiple hidden layers


2 -1 Age
3
-3
4 1

4
2 7
-5 2
55

Calculate with ReLU Activation Function


Deep Learning in Python

Multiple hidden layers


2 -1 Age
3
-3
4 1

4
2 7
-5 2
55

Calculate with ReLU Activation Function


Deep Learning in Python

Multiple hidden layers


2 Age
3

55

Calculate with ReLU Activation Function


Deep Learning in Python

Multiple hidden layers


2 Age
3

55

Calculate with ReLU Activation Function


Deep Learning in Python

Multiple hidden layers


2 Age
3

55

Calculate with ReLU Activation Function


Deep Learning in Python

Multiple hidden layers


2 Age
3 26

55

Calculate with ReLU Activation Function


Deep Learning in Python

Multiple hidden layers


Age
3 26

-5
55

Calculate with ReLU Activation Function


Deep Learning in Python

Multiple hidden layers


Age
3 26

-5
55

Calculate with ReLU Activation Function


Deep Learning in Python

Multiple hidden layers


Age
3 26

-5
55

Calculate with ReLU Activation Function


Deep Learning in Python

Multiple hidden layers


2 -1 Age
3 26
-3
4 1

4
2 7
-5 2
55 0

Calculate with ReLU Activation Function


Deep Learning in Python

Multiple hidden layers


2 -1 Age
3 26 0
-3
4 1

364
4
2 7
-5 2
55 0 52

Calculate with ReLU Activation Function


Deep Learning in Python

Representation learning
Deep networks internally build representations of pa"erns in
the data
Partially replace the need for feature engineering
Subsequent layers build increasingly sophisticated
representations of raw data
Deep Learning in Python

Representation learning
Deep Learning in Python

Deep learning
Modeler doesnt need to specify the interactions
When you train the model, the neural network gets weights
that find the relevant pa"erns to make be"er predictions
DEEP LEARNING IN PYTHON

Lets practice!