Académique Documents
Professionnel Documents
Culture Documents
[1]
Solution:
[2]
Solution:
1
[3]
Solution:
[4]
Solution:
[5]
2
Solution:
[6]
Solution:
[7]
3
Solution:
[8]
Solution:
4
[9]
Solution:
[10]
Solution:
[11]
5
Solution:
[12]
If = + , prove that,
() = () + ||
Solution:
6
[13]
Solution:
[14]
Solution:
7
[15]
8
[16]
Solution:
9
[17]
Solution:
10
[18]
Solution:
11
[19]
Solution:
12
[20]
Solution:
[21]
13
Solution:
[22]
Solution:
14
[23]
Solution:
15
[24]
Solution:
[25]
16
Solution:
17
[26]
Solution:
18
[27]
Solution:
19
20
[28]
Solution:
21
[29]
22
Solution:
[30]
23
Solution:
[31]
Solution:
[32]
Solution:
24
[33]
Solution:
25
[34]
Solution:
26
[35]
27
Solution:
[36]
28
Solution:
[37]
Solution:
29
[38]
Solution:
30
[39]
Solution:
[40]
31
Solution:
32
[41]
Solution:
33
[42]
Solution:
[43]
The binary erasure channel has four inputs and five outputs as described in figure, and
the corresponding transition matrix is described in next figure. The inputs are labeled
0, 1, 2, and 3 with probabilities , , , and respectively, and the outputs are
labeled 0, 1, 2, 3, and E.
[ ]
34
Solution:
() = () ( ) ( )
Now, lets define the horseshoe function or entropy function, by definition, the
entropy of binary source with probability, is (), sometimes referred to
as (, ), or simply(),
() = (, ) = () ( ) ( )
And therefore,
(, ) = () ( ) ( )
And hence, () = (, )
(, )
() = = ( )
And therefore, if the probabilities are not multiplied by any constants, the derivative
would always be the second probability over the first one. If both probabilities are
shifted by the same constant of curse the [ ] term would be multiplied by that
constant,
(, )
= ( )
(|) = ( )
(; ) = () (|) = (, ) ( )
35
(; )
= ( ) + = ( ) +
( ) =
( ) = = .
= (; )|@=. = .
[44]
36
Solution:
We notice that the overall channel is just a binary symmetric channel with crossover
probability of [e/2], which leaves us with two points,
The output Y is the sum of the input X and an independent noise random variable Z
with,
() = { ,
,
(/) = { , 2
,
37
Solution:
(i)
() = ( + ) = () ()
After Convolution,
, < 1
() = ,
{ , <
38
[46] Three binary symmetric channels are connected in cascade, as shown in the
figure below. Assuming that all channels have the same crossover probability, p,
(i) Find the transition matrix for the overall channel in terms of p.
(iii) Compare the capacity of the overall channel to the capacity of channel
1, and comment on the results.
Solution:
(i) To find the overall transition matrix, we find all paths from one input to
one output,
39
2nd path: ()()( ) = ( )
= ( ) + ( )
= + ( )
40
So, the overall transition matrix is,
( ) + ( ) + ( )
=[ ]
+ ( ) ( ) + ( )
(ii) for =
. .
=[ ]
. .
And the channel capacity for any BSC is,
= () = (. ) = .
Which means this channel is almost useless.
(iii) For a single BSC,
= ( ) = .
The capacity is much better than cascaded channels but still bad.
[47] State whether the following are true or false by circling the correct choice
(Justification NOT required).
(a) T / F H(X|X) = 0.
If X and Y are independent, then H(X|Y) = H(X) and H(Y|X) = H(Y), which are
not equal in general.
41