Bimodality of human speech
Department of Informatics
Aristotle University of Thessaloniki
The 2 modalities are:
- Audio speech signal = the acoustic waveform produced by the speaker;
- Visual speech signal = the movements of lips, tongue, other facial muscles, teeth
Speech is bimodal in:
- Production ? related to physical aspects of speech production, since some of the articulations responsible for speech production are visible
- Perception ? the McGurk effect: when a person is presented with conflicting audio & visual stimuli, the person may perceive a sound which is not present in either modality,
- e.g.: “hear” /ba/ + “see” /ga/ ? perceives /da/
“hear” /ma/ + “see” /ga/ ? perceives /na/