UNIVERSITY OF EXETER COLLEGE OF ENGINEERING, MATHEMATICS AND PHYSICAL SCIENCES

ECMM412 Machine Learning and Optimization

Continuous Assessment III

Date set: Hand-in date: Return date: 15 March 2012 29 March 2012 11 May 2012

This CA comprises 40% of the overall module assessment. This is an individual exercise, and your attention is drawn to the guidelines on collaboration and plagiarism in the College handbook.

The objective of this CA is to test your skills and knowledge in Linear Programming, Nonlinear Optimization and PCA. You are not expected to waste time typesetting the answers: legibly handwritten answers are quite sufcient.

Please Turn Over 1

(1) Linear Programming Let an optimization problem be dened as follows: PROBLEM 1 Maximize the function f (x, y) = 4x + 3y subject to the following constrains C.1 : C.2 : C.3 : C.4 : x + y x + 2y x y 4 5 0 0

(i) Solve Problem 1 by the Simplex Method and by using the tableau notation. (4 marks) (ii) Solve Problem 1 by using the graphical method (described in the lectures). (4 marks) (iii) Solve the following minimization problem: min f (x, y) = 4x + 3y subject to the following constrains C.1 : C.2 : C.3 : C.4 : C.5 : x + y x + 2y x y 4x + 3y 4 5 0 0 4 (2 marks) (iv) Solve the following minimization problem: min f (x, y) = 4x + 3y subject to the following constrains C.1 : C.2 : C.4 : x + y 4 x + 2y 5 y 0 (2 marks) Please Turn Over 2

(2) Nonlinear Optimization

Consider the function f (x) = 3 x4 4 x3 12 x + 12 on the interval x [1, 2].

(i) Prove that f (x) = 0 has at least one solution on the interval [1, 2]. (2 marks) (ii) Find a solution x of f (x) = 0 within an error of 1/100 by narrowing a bracket. (3 marks) (iii) Find a local optimum of f (x) within an error of 1/50 by narrowing a bracket. (3 marks) (iv) Approximate a local extreme of f (x) by using the Newton method. Use ve iterations of the method. (4 marks)

Please Turn Over 3

(3) Principal Component Analysis

On the module web page http://vle.exeter.ac.uk/course/view.php?id=829 you will nd the le ecmm412-ca3.m which includes 3-dimensional feature vectors vk = (xk1 , xk2 , xk3 ) (k = 1, 2, . . . , 104, i.e. 104 vectors in total).

You should do following. (i) Subtract the mean from each of the data dimensions and produce the new data set D consisting of 104 vectors vk = (k1 , xk2 , xk3 ) (the mean subtracted is the average x across each dimension). (1 mark) (ii) Provide the following plots: For each coordinate axis (three of them), plot the projections of all the vectors vk (k = 1, 2, ..., 104) onto this coordinate axis. Plot the projections of all the vectors vk (k = 1, 2, ..., 104) onto the planes deter mined by x1 -axis and x2 -axis; x1 -axis and x3 -axis; x2 -axis and x3 -axis. (2 marks) (iii) Calculate the covariance matrix C(D) corresponding to the new feature vectors set D = {(k1 , xk2 , xk3 ) | k = 1, 2, ..., 104}. x (2 marks) (iv) Calculate the eigenvalues and eigenvectors of the covariance matrix C(D). (2 marks)

Please Turn Over 4

(v) Let 1 , 2 , and 3 , be eigenvalues of C(D) given in a non decreasing order (i.e 1 2 3 ) and let u1 , u2 , and u3 , be eigenvectors corresponding to such determined eigenvalues, respectively. Consider the coordinate system whose coordinate axes 1 , 2 , and 3 , are parallel to the eigenvectors u1 , u2 , and u3 . Now, do the following. For each of the new coordinate axes 1 , 2 , and 3 , plot the projections of all the vectors vk , from D, onto this coordinate axis. Plot the projections of all the vectors vk , from D, onto the planes determined by 1 -axis and 2 -axis; 1 -axis and 3 -axis; 2 -axis and 3 -axis. (4 marks) (vi) What can be concluded from the plots based on the use of the PCA? Compare these plots with the plots from the item (ii). Comment the results obtained and give some theoretical explanation. (5 marks)

Note. You should also submit:

(1) all the plots required in the items above and (2) a printed copy of all the source codes used for the computation (these can be Matlab codes or your personal codes).

Please Turn Over 5

Submission

This coursework must be submitted to the Education Ofce using BART. Submission must be made by 12pm (noon) on the data indicated on the front page of this document. You submission should include detailed explanations about work that has been done, results obtained, and your conclusions and comments abut the results. You have to submit a commented listing of your programs used to obtain the results required.

End of CA