Course description
- Introduction: definition of information, code, coding and communication system.
- Entropy: discrete random systems, entropy of a discrete random variable, entropy of a continuous random variable.
- Information: average information, mutual information of two discrete variables, mutual information of two continuous variables.
- Discrete information sources: entropy of a stationary source, ergodic stationary sources, memoryless sources, sources with memory, and source redundancy.
- Information source coding: fixed-length and variable-length coding, Kraft – McMillan inequality, Huffman code, arithmetic code, LZW code.
- Secrecy coding: cryptosystems with a secret key, DES and AES cryptosystems, cryptosystems with a public key, RSA cryptosystem, digital signature.
- Communication channels I: continuous communication channels, the capacity of a continuous communication channel, discrete communication channels, the capacity of a discrete communication channel.
- Communication channels II: error-detecting codes, error-correcting codes, optimal decoding, Shannon theorem of secure coding, inversion of Shannon theorem.
- Channel coding: linear block codes, cyclic codes, Hamming codes, Golay codes, the Reed-Muller codes, convolutional codes, the Viterbi algorithm, Turbo codes.
Course is carried out on study programme
2nd Cycle Postgraduate Study Programme in Electrical Engineering
Objectives and competences
The objective of the course is to provide the student with the knowledge of the most important concepts and methods of information theory, information source coding, cryptography and communication-channel coding that form the basis for the design and development of communication interfaces that are essential components of intelligent systems in automation. The acquired knowledge forms the basis for the understanding and development of the technologies that involve obtaining and processing information from the environment.
Learning and teaching methods
The lectures provide a theoretical background of all the considered models and methods together with simple computational examples that illustrate the key characteristics of all the presented methods. A textbook and other study material, such as lecture notes with solved example problems and lecture slides, are available to the students. As part of the lectures, the students receive optional homework assignments including theoretical questions as well as computational exercises that enable the students to promptly verify the acquired knowledge. Practical work is carried out as part of the laboratory exercises, where students solve given programming problems. As part of the laboratory exercises, students also carry out additional elective projects within which the selected methods of information theory and coding should be implemented. The results of the elective projects are reported in written reports.
Intended learning outcomes
After successful completion of the course, students should be able to:
- present the basic mathematical models describing the main phenomena in technical communication systems,
- explain the measures of entropy, information and capacity of channels that are used for the quantitative descriptions of communication systems,
- use encryption methods to ensure the security and confidentiality of communications,
- use digital certificates to verify the identity of people and other information sources in different communication systems,
- analyse the advantages and disadvantages of communication systems, where secure and confidential communication must be ensured,
- build examples of communication systems that provide secure and confidential communication between information sources and receivers, and
- to evaluate the level of security and confidentiality of communications in the given communication systems.
Reference nosilca
- KRIŽAJ, Janez, DOBRIŠEK, Simon, ŠTRUC, Vitomir. Making the most of single sensor information : a novel fusion approach for 3D face recognition using region covariance descriptors and Gaussian mixture models. Sensors, ISSN 1424-8220, Mar.-2 2022, iss. 6, 2388, str. 1-26.
- BATAGELJ, Borut, PEER, Peter, ŠTRUC, Vitomir, DOBRIŠEK, Simon. How to correctly detect face-masks for COVID-19 from visual information?. Applied sciences, ISSN 2076-3417, Feb. 2021, vol. 11, iss. 5, str. 1-24.
- GAJŠEK, Rok, MIHELIČ, France, DOBRIŠEK, Simon. Speaker state recognition using an HMM-based feature extraction method. Computer speech & language, ISSN 0885-2308, Jan. 2013, vol. 27, no. 1, str. 135-150.
- ŠTRUC, Vitomir, DOBRIŠEK, Simon. Informacija in kodi : dopolnilni učbenik z vajami. izd. Ljubljana: Založba FE, 2016
- GOLOB, Žiga, ŽGANEC GROS, Jerneja, ŽGANEC, Mario, VESNICER, Boštjan, DOBRIŠEK, Simon. FST-based pronunciation lexicon compression for speech engines. International journal of advanced robotic systems, ISSN 1729-8814, 2012, vol. 9, no. 211, str. 1-9.
Study materials
- N. Pavešić: Informacija in kodi, (2. izdaja), Založba FE in FRI, 2010.
- V. Štruc, S. Dobrišek: Informacija in kodi : dopolnilni učbenik z vajami. 1. izd., Založba FE, 2016
- T. M. Cover, J. A. Thomas: Elements of Information Theory, Wiley-Interscience, New York, 2006.
- Roberto Togneri, Christopher J. S. deSilva: Fundamentals of information Theory and Coding Design, Chapman & Hall / CRC, 2002.