Multimodal interactive 3D technologies

Subject description

The course introduces the psychophysiology of human perception of visual, auditory and kinaesthetic/tactile stimuli. It focuses on methods of modelling and rendering three-dimensional synthetic stimuli for visual, auditory and haptic modalities. Spatial display technologies such as stereoscopic and hologram displays, spatial sound generation and haptic robots enable realistic display of visual, auditory and tactile information. The course further explores technologies and methods for tracking the user's movement to enable interaction and navigation in virtual environments. Augmented reality and telepresence extend the possibilities of interaction with real or remote environments. The analysis of the user's psychophysiological responses and the adaptation of the virtual environment to the user's condition allow for a personalised user experience. The technologies have applications in design, manufacturing, marketing, architecture, education, medicine and research and development.

The subject is taught in programs

Objectives and competences

The course enables the student to understand interactive 3D presentations, to design interactive 3D applications and to select adequate presentation technologies. It deals with methods and technologies for interactive presentations of three-dimensional computer generated environments by conveying visual, auditory and haptic stimuli. Analysed are effects of multimodal virtual environments on human psychophysiological state (immersion, presence) and presented are methods for assessment of psychophysiological responses in real-time. Students learn to design and implement interactive 3D simulations and interactive presentations used for functional product analysis, 3D simulation based learning (medicine), marketing applications, entertainment and other applications.

Teaching and learning methods

Consultations and individual project based work.

Expected study results

After successful completion of the course, students should be able to:

– understand psychophysiology of human perception of environmental stimuli,

– develop methods for affecting psychophysiological human responses,

– explain multi-modal communication between human and machine (robot, computer),

– synthesise visual, auditory and haptic stimuli for the purpose of human-machine interaction,

– evaluate the effects of artificially generated stimuli on user,

– implement interactive 3D applications in industry, medicine and science.

Basic sources and literature

Mihelj M, Podobnik J (2012) Haptics for Virtual Reality and Teleoperation, Springer, Berlin.
Mihelj M, Novak D, Beguš S (2013) Virtual Reality Technology and Applications, Springer, Berlin.
Burdea G, Coiffet P (2003) Virtual reality technology, John Wilwy & Sons.
Sherman WR, Craig AB (2003) Understanding virtual reality : interface, application and design, Morgan Kaufmann Publishers.
Ong SK, Nee, AYC (2004) Virtual and augmented reality applications in manufacturing, Springer.