Multimodal interactive 3D technologies

Course description

Psychophysiology of human visual, auditory and haptic sensing; Methods for modelling and rendering of three-dimensional stimuli of all three modalities (visual, auditory and haptic); Collision detection between complex virtual objects; Technologies for spatial presentation of synthesized 3D stimuli (stereoscopic and holographic displays, spatial sound generation, haptic robots for presentation of kinaesthetic and tactile stimuli); Technologies and methods for user movement tracking that enable interaction with the environment and navigation within the environment; Augmented reality as interaction with real and telepresence as interaction with remote environment through digital medium; Analysis of users’ psychophysiological responses and strategies for real time adaptation of virtual environment based on users’ psychophysiological state; Use of interactive 3D technologies in areas such as product design, sales and marketing, architecture and design, education, medicine, research and development.

Course is carried out on study programme

Objectives and competences

The course enables the student to understand interactive 3D presentations, to design interactive 3D applications and to select adequate presentation technologies. It deals with methods and technologies for interactive presentations of three-dimensional computer generated environments by conveying visual, auditory and haptic stimuli. Analysed are effects of multimodal virtual environments on human psychophysiological state (immersion, presence) and presented are methods for assessment of psychophysiological responses in real-time. Students learn to design and implement interactive 3D simulations and interactive presentations used for functional product analysis, 3D simulation based learning (medicine), marketing applications, entertainment and other applications.

Learning and teaching methods

Consultations and individual project based work.

Intended learning outcomes

After successful completion of the course, students should be able to:

– understand psychophysiology of human perception of environmental stimuli,

– develop methods for affecting psychophysiological human responses,

– explain multi-modal communication between human and machine (robot, computer),

– synthesise visual, auditory and haptic stimuli for the purpose of human-machine interaction,

– evaluate the effects of artificially generated stimuli on user,

– implement interactive 3D applications in industry, medicine and science.

Reference nosilca

  1. MIHELJ, Matjaž, PODOBNIK, Janez. Haptics for virtual reality and teleoperation, Springer, 2012.
  2. M. Mihelj, D. Novak, S. Beguš, Virtual Reality Technology and Applications, Springer, 2013.
  3. MIHELJ, Matjaž, PODOBNIK, Janez, MUNIH, Marko. Sensory fusion of magnetoinertial data based on kinematic model with Jacobian weighted-left-pseudoinverse and Kalman-adaptive gains. IEEE transactions on instrumentation and measurement. Jul. 2019, vol. 68, no. 7, str. 2610-2620.
  4. ŠLAJPAH, Sebastjan, ČEBAŠEK, Eva, MUNIH, Marko, MIHELJ, Matjaž. Time-based and path-based analysis of upper-limb movements during activities of daily living. Sensors. Feb.-1 2023, vol. 23, iss. 3, 1289, str. 1-20.
  5. JAKOPIN, Blaž, MIHELJ, Matjaž, MUNIH, Marko. An unobtrusive measurement method for assessing physiological response in physical human-robot interaction. IEEE transactions on human-machine systems, 2017, vol. 47, no. 7, str. 474-485.

Study materials

1. Mihelj M, Podobnik J (2012) Haptics for Virtual Reality and Teleoperation, Springer, Berlin.

2. Mihelj M, Novak D, Beguš S (2013) Virtual Reality Technology and Applications, Springer, Berlin.

3. Furht B (2011) Handbook of Augmented Reality, Springer, Berlin.

4. Grasnick A (2021), Basics of Virtual Reality, Springer.

5. Schmalstieg D, Höllerer T (2016), Augmented Reality, Addison-Wesley.

Bodi na tekočem

Univerza v Ljubljani, Fakulteta za elektrotehniko, Tržaška cesta 25, 1000 Ljubljana

E: T:  01 4768 411