L.

Vandenberghe
EE133A

10/15/2014

Homework 1 solutions
1. (a) Not linear or affine. Choose
"

x=

1
0

"

y=

0
1

1
== .
2

We have
f (x) = f (y) = 1,
and

"

x + y =

1/2
1/2

f (x) + f (y) = 1,
#

f (x + y) = 0.

Therefore f (x + y) 6= f (x) + f (y). This example shows that f is not affine,

and therefore also not linear.
(b) Linear. We can write f as an inner product: f (x) = xn x1 = aT x with
a = (1, 0, . . . , 0, 1).
(c) Affine. Working out the squares gives
f (x) =
=
=
=

(x c)T (x c) (x d)T (x d)
(xT x cT x xT c + cT c) (xT x dT x xT d + dT d)
xT x 2cT x + cT c xT x + 2dT x dT d
2(d c)T x + kck2 kdk2 .

This shows that f can be expressed as f (x) = aT x + b with

b = kck2 kdk2 .

a = 2(d c),
The function is linear if kck = kdk.
(d) Not linear or affine. Choose

x= 1
,
4

y= 1
,
0

1
== .
2

We have
f (x) = f (y) = 1,
1

f (x) + f (y) = 1,

and

x + y = 1 ,
2

f (x + y) = 2.

Therefore f (x + y) 6= f (x) + f (y). The example shows that f is not affine,

and therefore also not linear.
2. First we express T as T = a1 P1 + a2 P2 + a3 P3 + b. The four data points give the
following information:
10a1 + 10a2 + 10a3 + b
100a1 + 10a2 + 10a3 + b
10a1 + 100a2 + 10a3 + b
10a1 + 10a2 + 100a3 + b

=
=
=
=

30
60
70
65.

Subtracting the first equation from the second gives 90a1 = 30. Hence, a1 = 1/3. Subtracting the first equation from the third gives 90a2 = 40, Hence a2 = 4/9. Subtracting
the first equation from the fourth gives 90a3 = 35, Hence a3 = 35/90. Plugging in these
values in any of the four equations and solving for b gives b = 55/3.
If all processors operate at the same power P1 = P2 = P3 = Q, then T 85 if
55
7
85,
(a1 + a2 + a3 )Q + b = Q +
6
3
i.e., Q 57.1.
3. (a) Take ai = |xi | and bi = 1/n, i = 1, . . . , n, in the Cauchy-Schwarz inequality
aT b kakkbk.
With this choice of a, b we have
aT b =

n
1X
|xi |,
n i=1

kak = kxk,

1
kbk = ,
n

and the Cauchy-Schwarz inequality reduces to

n
1X
1
|xi | kxk.
n i=1
n

The upper bound in the Cauchy-Schwarz inequality holds with equality if and
only if a and b are aligned. Applying this to b = (1/n, 1/n, . . . , 1/n), we can
conclude that
n
1X
1
|xi | = kxk
n i=1
n
if the absolute values of the elements of x are all equal. Therefore x must be of
the form x = (, , . . . , ) for some 0.
2

(b) The inequality can be written as

n
X

xk

k=1

and
n

n
X

!1/2

xk

k=1

n
X

1
k=1 xk
n
X

1
k=1 xk

!1/2

This is the Cauchy-Schwarz inequality aT b kakkbk with ak =

1/ xk .

xk and bk =

4. The MATLAB code is as follows. Note that there are many other correct ways to get
the same result in MATLAB.
load mnist;
I = find(labels < 5);
digits = digits(:, I);
[n, M] = size(digits);
K = 5;
% cluster centers are stored as columns of Z
Z = rand(n, K);
D = zeros(K, M);
Jprev = NaN;
for iter = 1:100
% assign vectors to clusters
% D is a K x M matrix with D(k,j) the distance of point j to center k
for k = 1:K
D(k,:) = sqrt( sum( (digits - Z(:, k*ones(1,M))).^2) );
end;
% d and class are 1 x M arrays
% d(k) is the minimum value in column k of D
% class(k) is the row index of the minimum
[d, class] = min(D);
% the cost J is the average of the sum of squared elements of d
if iter > 1
if J == Jprev, break; end;
Jprev = J;
end;
J = (1/M) * norm(d)^2;
3

disp(sprintf(J = %e., J));

% update clusters
for k = 1:K
% I contains the indices of the digits assigned to cluster k
I = find(class == k);
if size(I,2)
% new cluster center k is the average of the points in cluster k
Z(:,k) = sum(digits(:,I), 2) / size(I,2);
else
disp(sprintf(Cluster %d is empty., k)); return;
end
end
end;
for k=1:K
subplot(1,K,k)
imshow(reshape(1-Z(:,k), 28, 28));
end
A typical result is shown below.