Audio-Visual Speech Recognition
Outline
Purpose of the lecture
Contents
Concepts and terminology
Overview of speech production - physical aspects
Bimodality of human speech
Speechreading by humans
Computer speechreading systems
Strategies for combining audio and visual modalities of speech
The visual speech recognition (lipreading) subsystem
Types of facial speech features
Mouth region localization
Mouth/ lip contour extraction and tracking
Classification of visual speech features
Experimental frameworks
Examples of audio-visual speech recognition systems
Challenges in audio-visual speech recognition area
Conclusions
References
Demo - McGurk effect
Email: ssip@inf.u-szeged.hu
Home Page: http://www.inf.u-szeged.hu/~ssip