Chng 3: M HA NGUN

1. TIn Iur
hguon tin la nJi hat ra tin tuc. Tin tu c duJc hat ra la
mot qua trinh ngau nhien.
Mot ban tin cu the la mot the hien cua qua trinh ngau
nhien do.
Tin tuc co hai loai:
Tin tuc lien tuc
Tin tuc rJi rac.
Mot ban tin lie n tuc hoan toan tuJng duJng vJi mot
ban tin rJi rac qua dinh ly lay mau shannon
vi vay, de dJn gian, ta chi xet tin tuc rJi rac.
TIn Iur
Tat ca cac ban tin cua mot nguon tin deu duJc thanh
la tu mot bo ky hieu (symbol) cua nguon tin do.
0oi bo ky hieu nay la A, va gia su bo ky hieu co m ky
hieu: a
1
, a
2
, , a
m
. Ta co:
A = a
i
, i=1..m}
Mot ban tin X la day cac ky hieu ai duJc xa xe theo
mot thu tu nao do:
X= (x
1
, x
2
, , x
n
) x
i
A
TIn Iur
vi du 1: Thiet bi hat tin hieu Morse su dung trong die n
bao:
hguon tin A co 8 ky hieu la: ., _, (khoang trang)
Tu c la a
1
= ., a
2
= _, va a
8
=
0ia su nguon tin nay hat ra ban tin:
X = (. _) (chu A).
Ba n tin na y co : x
1
= ., x
2
= _
X= (_ . . .) (chu B).
Ba n tin na y co : x
1
= _, x
2
= ., x
8
= ., x
4
= .
X= (_ . . . . _) (chu BA).
Ba n tin na y co : x
1
= _, x
2
= ., x
8
= ., x
4
= .
x
5
= , x
G
= ., x
7
= _
TIn Iur
vi du 2: May die n thoai di dong nhan tin cac tin
tieng Anh:
hguon tin A gom ca c ky hieu la: a, b, c, , /, O, 1, ,
O va ca c dau.
Tuc la a
1
= a, a
2
= b, a
8
= c , a
2G
= /
0ia su nguon tin na y hat ra ba n tin:
X = (di uong cae)
Ban tin nay co: x
1
= d, x
2
= i, x
8
= , x
4
= u,
TIn Iur
Boi vJi nguJi nhan tin, khi doi cho cac ky hieu vJi
nhau, noi dung tin thay doi.
vJi he thong truyen tin, khi doi cho, so lan xuat hien
cua cac ky hieu la khong doi va khong anh huJng
tJi qua trinh truyen tin.
Xet tre n quan diem truyen tin, ta chi quan tam tJi kha
nang xuat cua cac ky hieu trong ban tin ma khong
quan tam tJi vi tri cua tung ky hieu.
TIn Iur
hhu vay, mot nguon tin co the duJc mo ta bang bo ky
hieu A=a
i
, i=1..m} va xac suat xuat hien cac ky hieu
do (a
i
).
Ky hieu nguon: A, (a
i
), i=1..m}
Mot he bien doi tong quat trong he thong truyen tin
la he biet doi tu bo ky hieu nay sang bo ky hieu
khac:
A, (a
i
), i=1..m} > B, (b
j
), j=1..n}
TIn Iur
vi du 1: Fhat tin hieu Morse thuc chat la he bien
Tu bo ky hieu: A, B, . . . , Z, O, 1,. . ., O}
Sang bo ky hieu: ., _, }
Thu tin hieu Morse la he bien doi nguJc lai:
Tu bo ky hieu: ., _, }
Sang bo ky hieu:A, B, . . . , Z, O, 1,. . ., O}
}nh ngha Iuong IIn
LuJng tin cua mot ky hieu ai la mot ham (a
i
), hu
thuoc vao kha nang xuat hien cua ky hieu do:
heu cJ so la e thi thu nguye n la nat
heu cJ so la 1O thi thu nguye n la harley
heu cJ so la 2 thi thu nguyen la bit
) ( log
) (
1
log ) (
i
i
i
a p
a p
a I = =
Luong IIn rIang
Cho nguon tin A,F(a
i
), i=1..m}
LuJng tin chua trong moi moi tin a
i
duJc goi la luJng
tin rie ng cua tin do:
) ( log
) (
1
log ) (
i
i
i
a p
a p
a I = =
Luong IIn rIang
vi du 1: Mot nguon tin gom co bon ky hieu (tin) la
OO, O1, 1O, 11} dang xac suat.
Tuc la F(a
1
) = F(a
2
) = F(a
8
) = F(a
4
)=1/4
vay (a
1
) = (a
2
) = (a
8
) = (a
4
)= log
2
1/(1/4)=2 bit
vi du 2: Mot nguon tin gom co 8 ky hieu (tin) la
OOO, OO1, , 111} dang xac suat.
Tuc la F(a
i
) = 1/8 i=1..8
vay (a
i
) = log
2
1/(1/8)=8 bit i=1..8
Luong IIn Iuong ho
Trong truyen tin, neu kenh truyen la ly tuJng, ta se
co su bien doi ly tuJng tu nguon A, (ai), i=1..n}
sang nguon B,(bi), i=1..n} la 11
Tuc la:
Bau hat hat tin a
i
co luJng tin (a
i
)= log(a
i
)
Bau thu thu tin b
i
co luJng tin (b
i
)= log(b
i
)
Thi (a
i
) = (b
i
)
Luong IIn Iuong ho
Kenh truyen lytng
a
1
b
1
P(b
1
/a
1
)=1
a
2
b
2
P(b
2
/a
2
)=1
a
3
b
3
P(b
3
/a
3
)=1
a
n
b
n
P(b
n
/a
n
)=1
.

.

.
.

.

.

.
.

Luong IIn Iuong ho
Trong thuc te, cac kenh truyen thuJng xuyen co
nhieu, lam cho xac suat thu duJc cac tin b
i
khi
hat tin a
i
giam xuong.
Fhan con lai se la xac suat thu duJc cac tin b
k
, k i
va k=1..n.
Luong IIn Iuong ho
hhu vay, khi hat di nhieu tin a
j
, ta deu co the nhan
duJc tin b
i
vJi xac suat xuat hien khac nhau.
Trong cac tin a
j
do co 1 tin a
i
dung la tuJng ung vJi b
i
.
Cac tin con lai do nhieu tac dong ma chuyen thanh
b
i
.
van de dat ra la hai xac dinh duJc tin a
i
tuJng ung
vJi b
i
thu duJc.
Luong IIn Iuong ho
Kenh truyen conhieu
a
1
b
1
a
2
b
2
a
3
b
3
a
n
b
n
.

.

.
.

.

.

.
.

Luong IIn Iuong ho
van de duJc giai quyet bang cach tim luJng tin cua
cac tin a
j
chua trong tin b
i
thu duJc goi la luJng
tin tuJng ho giua a
j
va b
i
.
Tin a
j
nao co luJng tin tuJng ho vJi b
i
lJn nhat la tin a
i
can tim.
Luong IIn Iuong ho
Binh nghia: Luci tii tuci |c iua a
j
va |
i
|a |uci
tii vc a
j
c|ua trci tii |
i
Cach tinh: LuJng tin tuJng ho giua a
j
va b
i
la luJng
chua trong tin a
j
tru di luJng tin con lai cua a
j
sau
khi da nhan duJc tin b
i
Ky hieu: (a
j
; b
i
)
Luong IIn Iuong ho
vay:
(a
j
; b
i
) = (a
j
) (a
j
/ b
i
)
= log (a
j
/ b
i
) / (a
j
) (1)
Trong do: (a
j
/ b
i
) la luJng tin co dieu kien, do la
luJng tin cua a
j
bi ta nhieu ha huy.
Luong IIn Iuong ho
Theo cong thuc Bayes:
Thay va o (1) ta duJ c:
vay:
Trong do: (b
i
/a
j
) la xa c suat nhan duJc tin b
i
khi a
j
duJ c truyen
di (dac tinh nhieu cu a kenh).

=
=
n
j
Aj B P Aj P
Aj B P Aj P
B Aj P
1
) / ( ). (
) / ( ). (
) / (
(
(

=
) (
1
.
) / ( ). (
) / ( ). (
log ) / (
1
j
n
j
j i j
j i j
i j
a p
a b P a P
a b P a P
b a I

=
=
n
j
j i j
j i
i j
a b P a P
a b P
b a I
1
) / ( ). (
) / (
log ) ; (
Luong IIn Irung bnh
Trong thuc te, ta khong quan tam tJi luJng tin chua
trong tung tin dJn le ma quan tam tJi luJng tin chua
trong mot ta hJ tin.
vi vay, nguJi ta dua ra khai niem luJng tin trung binh
cua mot nguon tin.
Binh nghia: Luci tii trui |i| cua mct iuci tii |a
tii tuc trui |i| c|ua trci mct ky |icu |at ky
cua iuci tii Jc.
Ky hieu: (A)
Luong IIn Irung bnh
0ia su co nguon tin A,F(a
i
), i=1..n)}
LuJng tin trung binh duJc tinh:
Thu nguyen: |bit/ky hieuj

= =
= =
n
i
i i
n
i
i
i
a p a p
a p
a p A I
1 1
) ( log ). (
) (
1
log ). ( ) (
Luong IIn Irung bnh
vi du 1: 0ia su co nguon tin O,1} dang xac sua t. Tim
luJng tin rie ng cua tung ky hieu va luJng tin trung binh cu a
nguon
0iai
LuJng rieng cua tung tin:
(O) = (1) = log
2
1/(1/2) = 1 |bitj
LuJng tin trung binh cu a nguon:
(X) = (O).(O) + (1).(1)
= .1 + .1 = 1 |bit/ky hie uj
Vay k|i iuci Jai xac suat t| |uci tii rici cua tu i
tii |a i |uci tii trui |i| cu a iuci.
Luong IIn Irung bnh
vi du 2: 0ia su co nguon tin O,1} vJi xa c suat xuat hien
cac ky hie u O va 1 lan luJt la O,OO va O,O1. Tim luJng tin
rieng cu a tung ky hieu va luJng tin trung binh cua nguon
0iai
(O) = log
2
1/(O,OO) =
(1) = log
2
1/(O,O1) = G,5 |bitj
(X) = (O).(O) + (1).(1)
= O,OO.log
2
1/O,OO + O,O1.log
2
1/O,O1
= O,O81 |bit/ky hieuj
||ai xct. |uci tii ric i cac cua 1 i|ui |uci tii trui
|i| rat t|ap.
Luong IIn Iuong ho Irung bnh
TuJng tu nhu luJng tin rie ng, (a
j
;b
i
)chi cho biet luJng
tin cua a
j
chua trong b
i
.
Trong thuc te, ta quan tam tJi luJng tin trung binh cua
nguon A chua trong mot tin bat ky cua nguon B
duJc goi la luJng tin tuJng ho trung binh.
Ky hieu: (A;B)
Luong IIn Iuong ho Irung bnh
LuJng tin tuJng ho trung binh duJc tinh:

= =
= =
n
i
m
j
i
j i
j i
B A
a p
b a p
b a p
a p
b a p
b a p B A I
1 1 ,
) (
) / (
log ). . (
) (
) / (
log ). . ( ) ; (
Luong IIn Irung bnh ro dIau kIan.
Khi truyen tin tu nguon A sang nguon B, mot ha n cua A bi tam
nhieu ha huy va khong tJi duJc B.
Be danh gia muc do ha hu y nay, ta co khai nie m luJng tin
trung binh co dieu kie n:
Bay chinh la luJng tin trung binh cua nguo n A bi mat di do ta
nhieu trong qua trinh truyen sang B.
vay: (A,B) = (A) (A/B)

= =
=
n
i
m
j
j i j i
b a p b a p B A I
1 1
) / ( log ). . ( ) / (
o baI d}nh rua nguon IIn
Mot tho ng so cJ ba n cu a tin tuc la do bat dinh cu a tin, ky hieu:
h(a
i
)
Bo bat dinh cu a mot tin la luJng tin cu a tin do nhung xet tai
thJi diem chua nhan duJc tin nay.
hhu va y, ve mat gia tri:
h(a
i
) = (a
i
) = log
2
(a
i
) |bitj
Bo bat dinh cu a mot nguon tin co gia tri bang luJ ng tin trung
binh cua nguon tin do:

= =
= =
n
i
i i
n
i
i
i
a p a p
a p
a p A H
1 1
) ( log ). (
) (
1
log ). ( ) (
o baI d}nh rua nguon IIn
Tinh chat cua do bat dinh:
h(A) O
h(A) = O khi A co mot ky hieu cha c chan xuat hien va
cac ky hie u co n la i chac chan khong xuat hien.
h(A) = h(A)
max
khi ca c ky hieu cua nguon la dang xac
suat.
o baI d}nh dong IhoI
Bo bat dinh dong thJi la do bat dinh trung binh cua
mot ca (a
i
,b
j
) bat ky trong he bien doi tu nguJn A
sang nguon B.
Ky hieu: h(AB)
vay:

= =
=
n
i
m
j
i
j i
j i
a p
b a p
b a p AB H
1 1
) (
) / (
log ). . ( ) (
o baI d}nh ro dIau kIan
Bo bat dinh co die u kien la do bat dinh trung binh cua
mot ky hieu a
i
thuoc ta A khi da biet mot ky hieu bat
ky b
j
thuoc ta B.
Ky hieu: h(A/B)
vay:

= =
=
n
i
m
j
j i j i
b a p b a p B A H
1 1
) / ( log ). , ( ) / (
Tor do Iap IIn
Be danh gia muc do nhanh cham cua qua trinh hat
tin tu mot nguon tin, ta su dung khai niem toc do la
tin cua nguon tin.
Binh nghia: Tcc Jc |ap tii cua mct iuci tii |a |uci
tii ma iuci tii Jc p|at ra trci mct Jci vi t|ci
iai.
Ky hieu: R(A)
Tor do Iap IIn
Cach tinh:
R(A) = n
O
. h(A) = n
O
. (A)
Thu nguyen: |bit/sj
Trong do:
n
O
la so ky hieu do nguon hat ra trong mot dJn vi thJi
gian.
h(A) la do bat dinh cu a nguon tin.
(A) la luJng tin trung binh cua nguon tin.
Tor do Iap IIn
vi du 1: 0ia su co nguon tin O,1} dang xac sua t. hguon
tin na y hat ra G4.OOO ky hieu trong mo t giay. Tim toc do
la tin cu a nguon?
0iai
LuJng tin trung binh cu a nguon:
h(A) = (O).(O) + (1).(1)
= .1 + .1 = 1 |bit/ky hie uj
n
O
= G4.OOO ky hieu / s
Vay. F(A) = i
0
x |(A) = 64.000 |it/s = 64 ||it/s
Thong Iuong kanh
Binh nghia: T|ci |uci kci| |a |uci tii tci Ja cc
t|c di qua kenh trci mct Jci vi t|ci iai ma k|ci
cc |ci.
Ky hieu: C |bit/sj
Moi loai moi truJng truyen dan khac nhau se co thong
luJng kenh khac nhau
hoi chung, thong luJng kenh thuJng la rat lJn so vJi
toc do la tin cua nguon.
Thong Iuong kanh
Binh nghia: T|ci |uci kci| |a |uci tii tci Ja cc
t|c di qua kenh trci mct Jci vi t|ci iai ma k|ci
cc |ci.
Ky hieu: C |bit/sj
Moi loai moi truJng truyen dan khac nhau se co thong
luJng kenh khac nhau
hoi chung, thong luJng kenh thuJng la rat lJn so vJi
toc do la tin cua nguon.
Thong Iuong kanh
Be su dung co hieu qua ke nh truyen, ta can tang toc do la tin
cua nguon.
ThuJng co 2 ca ch:
Tang n
O
: ky thuat ghe kenh theo thJi gian, su dung ma hat hien
loi, ma sua sai (ma chong nhieu).
Tang h(A): cac ky thuat dieu che so
C > R(A)
R(A)
V d: Tnh lng tin
vi du : Fhe bien doi A/0 tin hieu thoai: tn s ly
mu: Fs = 8kh/, luJng tu hoa 25G muc. Tinh toc
do la tin cua bo bien doi A/0.
0iai:
R(A) = n
O
x h(A) (1)
n
O
= Fs = 8OOO mau / s
hguon dang xac suat nen h(A) = (A) = (a
i
) =
log
2
|1/(ai)j = log
2
|25G)j = log
2
|2
8
j = 8 bit /mau
R(A) = n
O
x h(A) = 8OOO x 8 = G4OOO bit/s.
ThoaI
N0 = Fs = 8000
H(A) = I(A) = 8
A/D
Thoai
So
8000x8
VoIra sIarao
N0 = Fs = 32 Khz
H(A) = I(A) = 12
A/D
L
So
32*10^ 3 * 12*2
R