Information Theory and Coding

In the introductory part, students become familiar with the definitions of information, coding, and the communication system. 

They then learn about the definition of the measure of entropy and information of random variables, which are used to model sources and recipients of information. Special attention is given to modeling stationary ergodic discrete information sources, both with and without memory. In the context of information source encoding, students are introduced to various systems for efficient and secret encoding of information sources. Special focus is placed on cryptographic systems with aprivate and public keys, enabling the digital signing of electronic messages. When learning about continuous and discrete communication channels, students become acquainted with the definition of their capacity measure. For the transmission of information over noisy channels, they also learn about optimal decoding procedures for detecting and correcting errors. In the concluding part, they are introduced to secure coding for the transmission of information over noisy channels. 

The objective of the course is to provide the student with the knowledge of the most important concepts and methods of information theory, information source coding, cryptography and communication-channel coding that form the basis for the design and development of communication interfaces that are essential components of intelligent systems in automation. The acquired knowledge forms the basis for the understanding and development of the technologies that involve obtaining and processing information from the environment.

The lectures provide a theoretical background of all the considered models and methods together with simple computational examples that illustrate the key characteristics of all the presented methods. A textbook and other study material, such as lecture notes with solved example problems and lecture slides, are available to the students. As part of the lectures, the students receive optional homework assignments including theoretical questions as well as computational exercises that enable the students to promptly verify the acquired knowledge. Practical work is carried out as part of the laboratory exercises, where students solve given programming problems. As part of the laboratory exercises, students also carry out additional elective projects within which the selected methods of information theory and coding should be implemented. The results of the elective projects are reported in written reports.