a). i). Determine the self-information and the entropy of the following four symbol probability distribution that originates from a discrete memory less source.
p(A) = 0.1, p(B) = 0.2, p(C) = 0.305, p(D) = 0.395.
Suggest a set of codes for this distribution and determine the coding efficiency for your codes.
ii). A data source generates 100 (equiprobable) discrete memory-less symbols. It is to be source encoded using a size ten alphabet. The 100 code words in figure 1. are suggested to identify each symbol. Comment on this choice of coding alphabet and code words including calculations based on related information theory concepts. .
A BA CA DA EA FA GA HA IA JA B BB CB DB EB FB GB HB IB JB C BC CC DC EC FC GC HC IC JC D BD CD DD ED FD GD HD ID JD E BE CE DE EE FE GE HE IE JE F BF CF DF EF FF GF HF IF JF G BG CG DG EG FG GG HG IG JG H BH CH DH EH FH GH HH IH JH I BI CI DI EI FI GI HI II JI J BJ CJ DJ EJ FJ GJ HJ IJ JJ
iii). An alphabet gave the following probability distribution
0.5, 0.4 and 0.1.
Encode the alphabet into binary words using the Huffman coding method and determine the coding efficiency.
Compare with fixed length encoding.
Explain how the efficiency could be increased even further using a method that is still based on Huffman coding. Discuss the effect on relevant communication system parameters when using these codes. [25 marks]
2). a). A binary symmetric channel is shown in figure 3.1.
x0, p(x0) y0
1 - p
1 - p
Figure 3.1 binary symmetric channel.
Explain the meaning of each of the variables the following expression and describe how you would use this to determine a value for Z that relates to figure 3.1. You don't need to know what is the purpose of Z.
∑∑ − = − = âŽŸ âŽŸ
âŽœ âŽœ âŽ âŽ›
1 0 ) |( 1log)|()(M m m n mn N n m x yp xypxpZ
Then given for a binary source p(x0) = 0.3, p = 0.15 calculate Z.
b). Plot a suitable graph that depicts the entropy (as the dependent variable) of a binary symmetric channel and considers all relevant parameters. [25 marks]
3). a). i). An analogue signal with a range of 0 to 5 V is to be pulse code modulated and is to have a dynamic range of at least 45 dB. Calculate: the number of bits used for quantisation the step size the increase in dynamic range, as a dB, if the number of bits used for quantisation is subsequently doubled.