Multimedia content processing

Subject description

Modern concept of signals in ICT (signals related to user and the environment, bio-signals, social signals, ambient signals);

Perception of sound, images and video. Physiology and psychology of content perception (understanding of limitations of human perception, the concept of lossy notation of multimedia content);

Overview of single- and multi-dimensional discrete algorithms and transformations transformations that are commonly used in processing of audio and video signals (digital filters, unitary transforms, discrete Fourier transform, discrete cosine transform, wavelet transform, eigenvalue analysis, PCA, ICA);

Methods of coding and notation of multimedia signals (lossless compression, redundancy, packetisation, encapsulation);

Standardisation and commercial notation of audio-visual materials, commercial codecs and containers (MPEG, …);

Identification of objects and states in multimedia systems for implicit interactivity in communication services involving electronic devices;

Algorithms and procedures for synthesis and integration of multimedia content;

Content management systems for multimedia;

Content adaptation in audio-visual services to characteristics of the communication channel and terminal equipment;

Digital rights management and conditional access to multimedia content;

Algorithms and methods for evaluation of quality of services in electronic media (Qualitative and quantitative evaluation of quality, standardization, quality of user experience).

The subject is taught in programs

Objectives and competences

The objective of the course Multimedia content processing is to equip students with knowledge, required to use principles, procedures and software for multimedia signal processing and production. The course covers modern signal processing, modelling and encoding procedures and algorithms for multimedia content.

Students are taught theoretical background of methods for signal processing, as well as their implementation within available tools.

Students are capable of selection of the most convenient tool for specific problem, understanding the background and are capable to use the selected tool for efficient problem solution. Students are taught theoretical background of modern techniques for perceptual based compression of multimedia signals. Acquired knowledge is basis for efficient selection of appropriate standard for digital multimedia content encoding.

Students are skilled with usage of processes for digital distribution of content with encoding, encapsulation of data, content protection and evaluation of quality of services.

Teaching and learning methods

The lectures provide a theoretical background on particular subjects together with presentation of simple practical examples. A complete study material is available to the students.

Practical work is being performed in the laboratory environment, and is accomplished in steps acquainting students with practical problems and tools.

Students are required to finish practical individual project. Project group consists of two or three students who accomplish the task from list of available tasks such as: selection and usage of watermarking procedure and analysis of the watermark resilience to attacks.

At the end of semester, students report on their results together with comparison to the results from the literature.

Expected study results

After successful completion of the course, students should be able to:

– Choose for a problem appropriate algorithms and procedures for analysis and reconstruction of digital multimedia signals,

– select and use programing tools related to multimedia signal processing on a various platforms and problem-specific areas

– list and describe dominant signal modalities in multimedia services

– describe physiological perception of sound, images and video from standpoint of human communication channel capacity

– describe and use basic algorithms for lossy compression of sound, images and video, and select appropriate standard for effective coding of multimedia signals

– list, describe and use systems for digital rights management

– list, describe and use of methods for evaluation of multimedia service quality.

Basic sources and literature

1. Saeed V. Vaseghi. Multimedia Signal Processing: Theory and Applications in Speech, Music and Communications. (Wiley) Nov 12, 2007.

2. Murat Tekalp. Digital Video Processing (2nd Edition). (Prentice Hall Signal Processing Series) Jun 28, 2015.

3. John W. Woods. Multidimensional Signal, Image, and Video Processing and Coding, Second Edition. (Elsevier) Jul 1, 2011.

4. Bose, T., Digital signal and image processing, John Wiley and Sons, 2010 (izbrana poglavja).

5. Mandal, M., Multimedia Signals and Systems, Springer, 2012.