CE00036-3-Data Communication Systems Individual Assignment Page 1 of 8

A) Explain Measurement of Information-Entropy?

Information: - Communication is the process of establishing connection or link between two

points for information exchange. Or communication is simply the basics process of
exchanging information. Communication system serves to communicate a message or
information. This information originates in the information source. In general, there can be
various message in the form of word, code, symbols, sound signal etc. however, out of these
message, only the desired message is selected and conveyed or communicated.

The amount of information contained in an event is closely related to its uncertainty. Message
containing knowledge of high probability of occurrence convey relatively little information.
We note that if an event is certain ( that is, the event occurs with probability 1 ), it conveys
zero information. Thus, a mathematical measure of information should be a function of the
probability of the outcome.

Information Equation

Let us consider a discrete memoryless source denoted by X and having alphabet (x1, x2,
xm). The information content of a symbol xi, denoted by I(xi) is defined by

1
I(xi) = log ( )= log ( )

Where P(xi) is the probability of occurrence of symbol xi.

Example of Calculating Information

Coin Toss
There are two probabilities in fair coin, which are head(0.5) and tail(0.5).
So, if we get either head or tail we will get 1 bit of information through following formula.

I(head) = -log (.5) = 1 bit

Entropy: - In a practical communication system, we usually transmit long sequence of

symbol from an information source. Thus, we are more interested in the average information
that a source produces than the information content of a single symbol. In order to get the
information content of the symbol, we take notice of the fact that the flow of information in a
system can fluctuate widely because of randomness involved into the selection of the
symbols. But to measure information using entropy we make the following assumption.

Level 3 Asia Pacific Institute of Information Technology 2017

CE00036-3-Data Communication Systems Individual Assignment Page 2 of 8

(a) The source is stationary so, the probability may remain constant with time.
(b) The symbols are generated from information source must follow rate of information.

Mathematical expression

The mean value of I(xi) over the alphabet of source X with m different symbols is represented
by H(X) is given by

H(X) = E[I(xi)] =
=1 ( )( ) = =1 ( ) log 2 ( ) bits/symbol

The quantity H(X) is known as the entropy of source X. it is a measure of the average
information content per source symbol.

Example of Calculating source entropy

Suppose a DMS X has four symbol x1, x2, x3, x4 with probabilities 0.4, 0.3, 0.2, 0.1
respectively.
H(X) = 4=1 ( ) log 2 ( )
H(X) = -0.4log 2 0.4-0.3log 2 0.3-0.2log 2 0.2-0.1log 2 0.1
H(X) = 1.85 bit/symbol
B) What do you understand by Channel model & Channel capacity?
Channel model: - Other than continuous and discrete channel, there are some special type of
channels with their own channel matrices. A DMC with an input X and output Y during each
time interval the channel except a input symbol from source alphabet and generate a output
symbol from Y. The channel is said to be discrete when alphabets of X and Y are both finite.
It is said to be memoryless when the current outputs depend on the current inputs. Each
possible input to output path is indicated along with a conditional probability and it is called
channel transition probability.
Types of channel
I. Lossless channel: - A channel describe by a channel matrix with only one non-zero
element in each column is called a lossless channel.

Level 3 Asia Pacific Institute of Information Technology 2017

CE00036-3-Data Communication Systems Individual Assignment Page 3 of 8

Lossless matrix is to be

II. Deterministic channel: - A channel describe by a channel matrix with only one non-
zero in each row is called deterministic channel.

Matrix of deterministic channel would be

Level 3 Asia Pacific Institute of Information Technology 2017

CE00036-3-Data Communication Systems Individual Assignment Page 4 of 8

III. Noiseless channel: -A channel is called noiseless if it is both lossless and

deterministic. The channel matrix has only one element in each row and each column
and this element is unity in this input and output alphabet are of the same size.
Channel capacity: - Channel capacity is concerned with the information handling capacity of
a given channel. It is affected by:

The attenuation of a channel which varies with frequency as well as channel length.
The noise induced into the channel which increaseswith distance.
Non-linear effects such as clipping on the signal.

Some of the effects may change with time e.g. the frequency response of a copper cable
changes with temperature and age. Obviously, we need a way to model achannel in order to
estimate how much information can be passed through it. Although we can compensate for
non-linear effects and attenuation it is extremely difficult to remove noise. The highest rate of
information that can be transmit- ted through a channel is called the channel capacity, C.
We know that bandwidth and the noise power place a restriction upon the rate of the
information that can be transmitted by a channel. It may be shown that in a channel which is
disturbed by a white Gaussian noise, one can transmit at rate of C bits per second, where C is
the channel capacity and is expressed as
C = B log2 (1+S/N)
B = channel bandwidth in Hz, S = Signal power, N = Noise power

C) How analog modulations differ from digital modulation techniques?

Modulation is the process of modifying one signal based on another, and it is used mostly in
the transmission of data from one point to another. Although there are many types of
modulation, there are two basic types; analog modulation and digital modulation. The main
difference between analog modulation and digital modulation is in the manner that they
transmit data. With analog modulation, the input needs to be in the analog format, while
digital modulation needs the data in a digital format.

Because of the differences in the input signal, the output signal is also quite different. In
analog modulation, any value between the maximum and minimum is considered to be valid.
It is not so with digital modulation as only two values are considered valid; one value to

Level 3 Asia Pacific Institute of Information Technology 2017

CE00036-3-Data Communication Systems Individual Assignment Page 5 of 8

represent 1 and another to represent 0. All other values are considered noise and are
rejected.

Because most signals that we transmit are analog in nature, like ones voice, it is far simpler
to do analog modulation than digital. If you want to transmit a voice using digital modulation,
youd need to pass it through an analog-to-digital converter before transmission and a digital-
to-analog converter at the receiver to recover the original signal. The additional stages needed
for transmitting digital modulation increases both the cost and complexity of the transmitter
and receiver.

The major advantage that digital modulation has over analog transmission is how it achieves
greater fidelity. With analog modulation, any noise or interference that falls in the given
frequency bandwidth gets mixed with the actual signal. Although there are a number of ways
to mitigate noise, it will still cause some amount of degradation. Because digital modulation
only recognizes 0s and 1s, any noise is virtually eliminated once the receiver discerns
whether a 0 or a 1 was transmitted. Unless the signal is very badly distorted, the output
signal will be literally identical to what was transmitted.

Under both analog modulation and digital modulation, there are a number of other
modulation techniques each with its own strengths and weaknesses. But each technique has
its basic commonalities of transmitting either a digital or analog signal.

D) What do you understand by channel coding systems? Explain its types?

Channel coding: - Channel coding deals with error control techniques. If the data at the
output of a communications system has errors that are too frequent for the desired use, the
errors can often be reduced by the use of a number of techniques. Coding permits an
increased rate of information transfer at a fixed error rate, or a reduced error rate for a fixed
transfer rate.
Now, it is interesting to note that the purpose of error control can be achieved in some
situations even without accomplishing the complete process of decoding. Accordingly, the
process of error control can be categorized into the following:

a) Forward Error Correction (FEC): - Complete process of decoding is applied on the

received sequence to detect error positions in the sequence and correct the erroneous
symbols. However, the process of error correction is not fool-proof and occasionally the

Level 3 Asia Pacific Institute of Information Technology 2017

CE00036-3-Data Communication Systems Individual Assignment Page 6 of 8

decoder may either fail to detect presence of errors in a received sequence or, may detect
errors at wrong locations, resulting in a few more erroneous symbols. This happens if, for
example, too much noise gets added to the signal during transmission through a wireless
channel.
b) Auto Repeat reQuest (ARQ): - In some applications (such as in data communications) it is
important to receive only error-free information, even if it means more than usual delay in
transmission and reception. A conceptually simple method of error detection and
retransmission is useful in such situations. The error control decoder, at the receiver, only
checks the presence of any error in a received sequence (this is a relatively easy task
compared to full error correction). In case any error is detected, a request is sent back to the
transmitter (via return channel, which must be available for this purpose) for re-transmitting
the sequence (or packet) once again. The process ideally continues till an error-free sequence
is received and, this may involve considerable delay in receipt and may result in delay for
subsequent sequences.
Another aspect of this scheme is that the transmitter should have enough provision for storing
new sequences while a packet is repeated several times. One may think of several interesting
variations of the basic scheme of ARQ. Three important and popular variations are: i) Stop
and Wait ARQ, ii) Continuous ARQ and iii) Selective Repeat ARQ.

The two main categories of channel codes are:

Block codes a block of k information bits is encodedto give a codeword of n bits (n >
k). For each sequence of k information bits, there is a a distinctcodeword of n bits.
Examples of block codes includeHamming Codes and Cyclic Codes. A
CyclicRedundancy Check (CRC) code can detect anyerror burst up to the length of
the CRC code itself.
Convolutional Codes the coded sequence of n bitsdepends not only on the present k
information bitsbut also on the previous information bits.
E) Explain theapplications where digital modulation techniques are used with the help
of diagram.

Some important applications of digital modulation techniques are

Voice grade MODEMs: The channel bandwidth of telephone is 300 to 3400 Hz i.e, 3100
Hz. The telephone line is used as medium for the widely-spread composition of network. But
it is necessary to convert the digital data into an analog signal so as to transmit it over the

Level 3 Asia Pacific Institute of Information Technology 2017

CE00036-3-Data Communication Systems Individual Assignment Page 7 of 8

telephone line. Therefore, it is necessary to use a data transmitter to modulate a voice

frequency carrier and a data receiver to demodulate this signal. A circuit which performs both
these actions is called as a Modem it is basically a trans receiver. In order to convert the
digital data into an analog signal which is compatible to the telephone line, some type of
modulation is used. The type of modulation commonly used are:

a)Binary FSK

b)Binary PSK

c)Four phase DPSK

d)16-QAM

Digital radio: - The ratio transmission known to us is analog it can be A.M or F.m however,
in digital radio, the information obtained from a source is transmitted to the receiver by using
some kind of digital modulation technique. The microwave links are used for the
communication and the type of propagation used is line of sight propagation.

Digital communication by satellite: - The broadcast capability of a channel can be exploited

by using the multiple access techniques. There are three multiple access techniques that can
be used.

a)Frequency division multiple access (FDMA)

b)Time division multiple access (TDMA)

c)code division multiple access (CDMA)

Out of these TDMA is ideally suitable for working with the digital communication systems.

TDMA: - This method of multiple access is suitable for the digitally modulated signals. In
TDMA, a number of ground stations are enabled to access a satellite in different time slots.

Level 3 Asia Pacific Institute of Information Technology 2017

CE00036-3-Data Communication Systems Individual Assignment Page 8 of 8

Fig: - Structure of a TDMA frame

In TDMA frame structure, one frame consists of N number of bursts. There is a lot of
variation in distance between satellite and different earth stations. To compensate for it a
guard band is inserted between sucessive bits. This also protects the system against overlap.
One burst per frame is used as a reference and remaining (N-1) bursts are allocated to
different ground stations. Thus each ground station is allowed to transmit once per frame.

Level 3 Asia Pacific Institute of Information Technology 2017