National University of San Agustin

System Engineer School

AI - Lab 02 - Linear Regression

Christian E. Portugal-Zambrano

April 08, 2019

1 Introduction
1.1 Objectives

• Understand the supervised learning setting.

• Understand linear regression (aka least squares)

• Understand how to apply linear regression models to make predictions.

1.2 Pre-requisites

You need some linear algebra concepts and as we mentioned before you must had choosed
some language and tools to performs some exercises. We provide Python or Octave code
according to the requirements of the practice.

2 Machine Learning and Linear Regression

In this practice we would starting trying to represent a model to perform some ma-
chine learning algorithms, we are going to model some linear functions and show how a
regression can predict or classify some data.

1
 2.1 What is Machine Learning?

Until now we dened some Articial intelligent denitions, machine learning is an area
which can be used through a supervised or unsupervised learning, tow denitions of
machine learning are oered: Arthur Samuel described it as: the eld of study that gives
computer the ability to learn without being explicitly programmed, then Tom Mitchell
provides a more modern denition: A computer program is said to learn from experience
E with respect to some class of task T and performance measure P, if its performance at
tasks in T, as measured by P, improves with experience E, in [?] is dened as An agent
is learning if it improves its performance on future tasks after making observations about
the world.

2.1.1 Supervised Learning

In supervised learning, we are given a dataset and already know what our correct output
should look like, having the idea that there is a relationship between the input (Inputs
are also often referred to as covariates, predictors and features) and the output (variates,
targets and labels). Supervised learning problems are categorized into regression and
classication.

2.1.2 Unsupervised Learning

Unsupervised learning allow us to approach problems with little or no idea what our
results should look like. We can derive structure from data where we don't necessarily
know the eect of the variables. With this technique there is no feedback based on the
prediction results.

Clustering Classication

2.2 Model representation

To establish notation for future use, we will use x(i) to denote an input variable, and y (i)
to denote the output or target variable, so a pair (x(i) , y (i) ) is called a training example

2
where i = 1, . . . , m is called a training set.
Our goal is, given a training set, to learn a function h : X → Y so that h(x) is a good
predictor for y , Figure 2.1 shows a common structure for supervised machine learning
problems:

Figure 2.1: Structure of a supervised machine learning problem

This way we start with this representation:

y = 4x + 3

This is a linear equation that we can also represent as:

y(θ0 , θ1 ) = θ1 x + θ0

then for every model where we have a training dataset, we must consider the next:

3
 2.3 Cost Function

As we showed before, we can measure the accuracy of our hypothesis function by using
a cost function, dened as:
m m
1 X 1 X
J(θ0 , θ1 ) = (y
i − yi )2 = (hθ (x) − yi )2
2m 2m
i=1 i=1

This function is called Squared error function or Mean squared error, the 12 term
is halved as a convenience for the posterior computation of gradient descent. Finally
the idea is to choose some θ0 , θ1 so that hθ (x) is close to y for our dataset (xi , yi ) of
m samples, in other words try to minimize J(θ0 , θ1 ) We are going to analyze this cost
function:

3 To do
After this explanation you have to work with the Salary_data.csv dataset which consist
of the income of an employee according to his years of experience, for this data you have
to resolve this tasks:
• Write a code that plots the points of the data.

• Make some manual predictions with the parameters you choose, write the code to
plots the lines generated.

• Write a code that plots the costs for each prediction.

• After doing some predictions, nd a method to perform a linear regression with the
data, to do this rst make a division of the data into a training set and a test set,
e.g 80% of the data to training and the 20% for test.

• Write a code to nd the cost function for the training and test set. What will be
the error for each one?

4
 • According to the theory, what do you think, Linear regression is a prediction or for
classication tasks? Please explain.

• There is another le called Data.csv, which consist of some more features to predict
if a person buy something according to his age, nationality and salary, what will
be the problems of this if we want to perform a linear regression?.

4 Deep inside
This section is just for anyone who wants to make a deep inside into the theory and
like challenges1 , you will have to prepare a presentation of just 5 minutes to explain the
article given, for this week the article is "Crédito para todos", then you will have to
defend yourself another ve minutes of questions. I want that all students benet from
your presentation, remember that we are here to learn. If you want to do this please
email to cportugalz@unsa.edu.pe

5 Deadline
The deadline to make your report is 10 minutes before the next class arrive for each group.
The report must be sent out to the instructor's email. You can use the TEX model
from here All question and doubts must be done to the same email. For the report
pay attention to the orthography and references used, be careful with all the ways of
plagiarism, you must include the code into the report with all of its graphics generated,
the only le to be sent is a pdf, no other les will be considered for the note.

References
[1] S. J. Russell and P. Norvig, Articial intelligence: a modern approach. Malaysia;
Pearson Education Limited 2016.

1
This must not be included into the report