Multimedia User Interfaces
(Master's Degree Program Multimedia Management - 3rd quarter)
This course is obligatory for all students having their focus on Media Informatics or Communication Design.
- Multimedia Projects
- Interactive Multimedia: Synchronized Multimedia Integration Language (SMIL)
- Speech and Sound: Speech synthesis and recognition (VoiceXML)
- Haptics: Force feedback, gestures
An overview on multimedia human computer interaction is given in this course. As an introduction the structure of the mark-up language, SMIL is investigated. SMIL is used used for broadcasting purposes (Real Server) and supported by MS Internet Explorer.
Interactive systems however require integration of other human capabilities than vision. This course provides a basic understanding of acoustic data processing for speech generation and for speech recognition. Practical experiences will be gained through the Java Media Foundation Classes (Java MFC) as well as the Java Speech API.
Kinaesthetic and haptic sensations allow for multimedia applications which utilise gesture recognition and force feedback. Several case studies will be presented to illustrate multimodal systems.
A learning unit on digital TV with its MHP technology as well as the
development of systems for mobile devices finishes the presentation of
multimedia user interfaces.
Credits: 4+2 ECTS.
Exercises include the use of SMIL for multimedia web pages, the development of spoken applications using the phone for speech input and speech output, and the design of a haptic scene using a force feedback device. It is planned to design MHP applications in student projects.
Software packages used: JBuilder, Internet Explorer, Realplayer, Bevocal, Cardinal, Multireader, Java Speech API, Java Media Foundation Classes.
Examination: oral exam.
- Adissono, L. et.al.: Personalized Digitial Television , Dordrecht: Kluwer, 2004.
- Burdea: Force and Touch Feedback for Virtual Reality , Wiley, .
- Dale, R.; Moisl, H.; Somery, H.: Handbook of Natural Language Processing , Marcel Dekker Inc., 2000.
- DeCarmo, L.: Core Java Media Framework , Prentice Hall, 1999.
- Dix, A.; Finlay, J.; Abowd, G.; Beale, R.: Human-Computer Interaction , Prentice Hall,1998.
- Raman, T. V.: Auditory User Interfaces , Kluwer Verlag, 1997.
- Rutledge, L.; Bulterman, D.: SMIL , Addison Wesley, 2003.
- Schroeder, M. R.: Computer Speech , Springer, 1999.
- Sharma, C.; Kunins, J.: VoiceXML , Wiley, 2002.
- Sutcliffe, A.: Multimedia and Virtual Reality , Lawrence Erlbaum, 2003.