Information Theory and Source Coding

The course provides basic mathematical tools and concepts of information theory, which are used to define amount of information, amount of transmitted information, capacity of a channel and other criteria used in the evaluation and planning of communication channels.  

The main requirements and properties of coding are introduced and some typical examples of lossless coding are shown, such as the uniform code, the Huffman code, arithmetic code and Lempel-Ziv code. 

The transmission of analogue signals over digital communication paths is examined, and implications of time discretization and the effect of quantization on the analog signal are analysed. 

In the last part, a few different ways of audio signal compression are shown, ranging from logarithmic compression (A- and μ-law) and differential coding (DPCM and ADPCM), to compression taking into account the psychoacoustic characteristics of hearing, typical of MP2, MP3, AAC, etc., and the basics of speech source coding (LPC, CELP).

Basic principles of information transmission and related backgrounds. Entropy as the basic measure of information. Source coding and basic data compaction algorithms. Fundamental limits of reliable communication over noisy channel. Properties of analogue signals that are important for coding schemes.

Distinction between redundancy and irrelevance.  Redundancy removal in advanced speech coding. Basic principles of perceptual coding of audio signals.

Lectures, tutorial, homeworks