ECE 534

Fall 2016

Homework Assignment 3
Due Date: Wednesday, March 2 (in class)
Reading: Sections 2.3-2.5, Sections 3.1-3.4
1. Problem 2.17.
2. Applications of Jensens Inequality. Prove the following:
(a) If E[|X|n ] < for some positive integer n, then E[|X|m ] < for all positive integers m n.
(b) If a non-negative random variable X has pdf with mean , then
Z
p
Q( 2x)fX (x)dx Q( 2).
0

(Here Q is the tail of a unit Gaussian distribution.)

(c) If f and g are positive pdfs on a set A (i.e., they have common support A), then
Z
f (x)

f (x) ln
D(f kg) =
dx 0
g(x)
A
(Note: D(f kg) is called the Kullback-Leibler divergence between the pdfs f and g.)
3. (Modification of Problem 2.19 of text.) A gambler repeatedly plays the following game: She bets
one dollar and then there are three possible outcomes: she wins two dollars back with probability
0.4, she gets just the one dollar back with probability 0.1, and otherwise she gets nothing back.
We are interested in computing the probability of the following event:
A = {Gamblers net gain is positive after playing 100 times}
(a) Compute the exact value of P(A) using Matlab or similar program.
(b) Find the CLT approximation for P(A).
(c) Find the Chernoff bound for P(A).
4. Problem 2.35 of text.
5. Let X1 , X2 , . . . be i.i.d. Poisson () random variables, i.e., each of the Xi s has p.m.f.
pX (k) = P{X = k} =

e k
k!

k = 0, 1, . . .

ju

(a) Show that X (u) = e(e 1) .

P
(b) Suppose Sn = ni=1 Xi . Show that Sn Poisson(n).
(c) Now set = 1, and write an expression for P{Sn 2n} in terms of the p.m.f. of Sn .
(d) Use the Chernoff Bound to show that
P{Sn 2n} en(2 ln 21)
c
V.
V. Veeravalli, 2016

6. Problem 3.3. Hint for (b): Can E[X|Y ] be negative.

7. Suppose X, Y have joint pdf
2
fX,Y (x, y) =
3



|y|
x+
11{0x1} 11{|y|1}
2

(a) Find E[X|Y ].

(b) Find E[X|Y

]. (Hint: What is the integral of an odd function over interval [a, a]?)
8. Problem 3.7. Note: Regarding the uniqueness in part (b), the answer is non-unique. You may
want to think of an example to show that the answer is non-unique, but you dont have to do it
as part of the homework.
9. Problem 3.9.
10. Problem 3.19.

c
V.
V. Veeravalli, 2016