76 Understanding Speech

Difficulty in segmenting phonemes: where does one stop and another start? Anyone who has learned a second language knows how difficult this is to figure out! People who study speech call this the segmentation problem.

Variability due to co-articulation: phonemes look/sound slightly different in different contexts. In a spectrogram, phonemes are characterized by formants (bands of constant sound during vowels) and transitions (onsets and offsets of formants, usually associated with consonants). A single phoneme will look very different on a spectrogram, depending on what syllable, word or phrase it is part of.

Variation in speaker styles: we all speak at different speeds, slur words together, etc. Human listeners employ a lot of social and contextual cues (e.g., visual cues) to figure out what people are saying. Computers do not have access to this information.

The McGurk effect shows how we use visual cues (as part of cue combination, which uses auditory plus visual cues) to ignore the variability and figure out which phoneme is which. In other words, what we see affects what we hear.

To learn more about the McGurk Effect, watch the video linked here and included below.