INFORMATION THEORY AND

CODING
5th SEM E&C

JAYANTHDWIJESH H P M.tech (DECS)

Assistant Professor Dept of E&C

B.G.S INSTITUTE OF TECHNOLOGY (B.G.S.I.T)

B.G Nagara, Nagamangala Tq, Mandya District- 571448
 FORMULAS FOR REFERENCE

MODULE 3 ( INFORMATION CHANNELS)

The entropy of input symbol is given by

H (A) = = ( )( ( ) ) bits/ message-symbol

The entropy of output symbol is given by

H (B) = = ( )( ) bits/ message-symbol
( )

The joint entropy is given by

H (A, B) = = = ( , )( ) bits/ message-symbol
( , )

The equivocation H(A/B) is given by

H (A / B) = = = ( , )( ) bits/ message-symbol
( / )

( , )
Since P( / ) =
( )
And another formula
H (A/B) = H (A, B) H (B) bits/ message-symbol

The equivocation H(B/A) is given by

H (B / A) = = = ( , )( ) bits/ message-symbol
( / )

( , )
Since P( / ) =
( )
And another formula
H (B/A) = H (A, B) H (A) bits/ message-symbol

The mutual information is given by

I (A, B) = H (A) H (A/B) bits/ message-symbol

The channel capacity is given by

C = (log s h) bits/ sec

Since h= = = ( ) bits/ message-symbol

The capacity of the channel is given by

C=B [1 + S/N]
Source efficiency is given by
()
= x 100%
 Source efficiency is given by

=
Channel efficiency is given by
(,)
= x 100%

Channel redundancy is given by
Channel redundancy = (1- ) x 100%
Estimation of channel capacity by MUROGAS method