Chapter 3 endnote 2, from Lisa Feldman Barrett.
Some context is:
In each [photograph you viewed], your emotion concepts helped you make meaning from the image. [...] A similar phenomenon occurs with the McGurk Effect, in which when someone speaks to you, what you see (mouth movements) influences what you hear (the sounds you perceive)
The McGurk Effect is an example of how our senses are not really independent of one another. If you watch a video of person speaking “Ba ba, ba” (lips pressed together to make the “ba” sound) and at the same time as the soundtrack is playing the sounds “Ba ba, ba” you will hear “Ba ba, ba.” But if the person begins speaking “Fa, fa, fa” (top teeth pressed atop bottom lip to make the “fa” sound) but the soundtrack is playing “Ba, ba, ba,” you will actually hear “Fa, fa, fa.” The sound has not changed, but your perception has. If you look away or close your eyes, you will hear “Ba, ba, ba.” The mouth movements that you see influence the sounds that you hear.
Notes on the Notes
- McGurk, Harry, and John MacDonald. 1976. "Hearing lips and seeing voices." Nature 264 (5588): 746–748.