top

Auditory Pattern Perception

Demonstrations (Direct Links)

Demonstration 11.1 A Mysterious Melody
Demonstration 11.2 Consonance and Dissonance
Demonstration 11.3 Shepard Tones
Demonstration 11.4 The Octave Illusion
Demonstration 11.5 Consonants
Demonstration 11.6 Backwards Speech
Demonstration 11.7 Phonemic Restoration Effect
Demonstration 11.8 McGurk Effect


Before You Start

• Although a great deal of research has used simple sine waves as stimuli, the sounds you typically encounter in everyday life are complex sounds. Thus, humans have evolved to hear complex sounds and we're quite good at interpreting such sounds. What are the adaptive advantages to hearing complex sounds such as music and speech?

• Music is a common complex auditory stimulus. You probably listen to a fair amount of music every day. Even if you're not a musician, you probably understand a lot about music. Why might you enjoy listening to music? Why do you prefer particular kinds of music? Why are some pieces of music able to elicit emotional responses?

• Imagine that you couldn't speak or hear speech. How would you communicate with other humans? Perhaps you're imagining a life of text messaging! Speech is certainly an important human ability. Have you given any thought to how you developed the ability to speak? (Check out Chapter 14!) When you listen to someone speak, do you hear a series of words? When you speak, do you utter a series of words? As you learn more about speech, you may be surprised to learn that you're hearing boundaries between words that aren't so readily apparent.

 


Perception of Complex Auditory Patterns

The Nature of Complex Sound Stimuli

Making Sense of Complex Sounds


Music Perception

The Musical Stimulus

Demonstration 11.1 A Mysterious Melody Diana Deutsch (University of California, San Diego) has provided a clear illustration of the importance of tone height. The mysterious melody has all the correct tone chroma, but the heights are randomized over three octaves. Note how difficult it is to determine the melody based solely on tone chroma. Once you've listened to the original form of the melody, listen to the scrambled version again. Why might you be able to better identify the melody now?

Link - Do you have perfect (absolute) pitch? You can find out at the University of California, San Francisco Perfect Pitch Site.

Demonstration 11.2 Consonance and Dissonance Locate a piano.  You don’t need to know how to play it, but you do need to be able to find middle C.  The figure below illustrates the notes included in this octave.

keyboard
Illustration by R. Oldmixon

Play each of the first seven tone combinations and judge the pleasantness of each of these pairs.  Now try the three dissonant tones whose combinations form complex fractions.  You may want to experiment with other possible combinations.

Name Of Interval

Notes (In C-Major Key)

Frequency Ratio

Octave

C-C

2/1

Major Third

C-E

5/4

Minor Third

E-G

6/5

Fourth

C-F

4/3

Fifth

C-G

3/2

Major Sixth

C-A

5/3

Minor Sixth

E-C

8/5

(Dissonance)

C-C#

262/277

(Dissonance)

C-D

262/294

(Dissonance)

C-D#

262/311

Music and the Brain

Link - You can watch a lecture by Aniruddh Patel on Music and the Mind.

Link - Daniel Levitin has written a very interesting book (This Is Your Brain on Music). You'll find his web site an interesting amalgam of music and research.

The Role of Cognition in Music Perception

Illusions in Music

Demonstration 11.3 Shepard Tones Roger Shepard developed a demonstration in which tone height is confusing. A series of tones will be heard as continually rising (or falling), even though they begin to repeat. You will find many variants on the web if you search for Shepard Tones. Here's one nice demonstration of Shepard Tones from Scott Flinn. The Exploratorium (San Francisco) also has a nice demonstration of Shepard Tones (both discrete and continuous) as well as the Tritone Paradox.

Demonstration 11.4 The Octave Illusion Diana Deutsch (University of California, San Diego) discovered the remarkable Octave Illusion. To experience the demonstration, you'll need stereo headphones. Although the tones presented to your two ears are rarely heard correctly, they can be misperceived in a number of different ways.


Speech Perception

The Speech Stimulus

Link - Listen to samples of emotion in speech.

Demonstration 11.5 Consonants Refer to the figure below to help you locate several anatomical structures that are essential in pronouncing consonants.

face

The place of articulation in the table below shows the portion of the vocal tract involved in making a particular sound.  The rows in the table show the manner of articulation (e.g., complete closure vs. narrowing of the passageway).  Finally, the top member of each pair of words is voiceless, and the bottom member is voiced.  This distinction illustrates the third dimension of consonant production: voicing.  Pronounce each of these consonants to appreciate the variety of ways in which sounds can be produced.

consonants

 

 

Theories of Speech Perception

Speech and the Brain

Link - YouTube video of a person with Broca's aphasia

Link - YouTube video of a person with Wernicke's aphasia

The Role of Cognition in Speech Perception

Demonstration 11.6 Backwards Speech At least since the 1960s, people have been fascinated with music and speech played backwards (also called--somewhat confusingly--backmasking). The Beatles' album Revolver contains examples, and some people reported that a Beatles' song contained evidence (when played backwards) that "Paul (McCartney) is dead." As John Vokey and Don Read noted (in their 1985 article in American Psychologist, Subliminal messages: Between the Devil and the Media), people's expectations play a primary role in what they ultimately hear in the backwards messages.

Sound processing programs for your computer (such as Audacity) now make it simple to create your own reverse speech or music. Below are a couple of examples. You will be able to find many examples by searching the web for backmasking, such as this list.

• First, listen to this example and try to determine what the normal (forward) speech said. (It's actually a paragraph from your textbook.) Second, determine what "hidden messages" might be found in the backward speech example. Did you hear anything? With a bit of effort, I was able to extract something that was vaguely interpretable. Here's that segment from the original example. Do you hear any message in that segment? Now, let me give you a hint. So that it doesn't pop out immediately as you read, I'll give you the hint backwards: [pohs eht ni zihw relaeh reniew a ereht sI] I'm fairly certain that's not a phrase you use often! Now, listen to the segment (or the complete example if you wish) and notice how easily you hear the phrase I've suggested.

• Second, listen to this example of a common song. Can you tell what it is when played forward? Do you hear any "hidden messages" in the song?

Demonstration 11.7 Phonemic Restoration Effect It's often the case that sounds occur simultaneously. For example, while you are listening to someone speak, another person may cough or slam a nearby door. In spite of the competing noise, you'll often experience little difficulty in hearing all that the speaker is saying. In fact, you may even have difficulty determining what portion of the speech was affected by the noise. That is the phonemic restoration effect, in which noise may overlap a person's speech, yet you perceive the speech as complete and unaffected by the noise. Below are some examples of speech in which a cough has obscured a portion of the speech. Do you have any difficulty hearing the speech in its entirety?

Phonemic Restoration Example 1.

OK, could you hear that sentence clearly, even with the superimposed cough? You might think that there's sufficient information in the original sentence to allow you to hear it correctly, even with the noise superimposed. To eliminate that possibility, for the remaining three examples, I've removed the information from the original sentence (i.e., replaced with silence) in addition to inserting the cough. Note that in each case, you should be able to "hear" the entire sentence. Note, also, that you can only "fill in" or restore the missing phonemic information after you've heard the ending to the sentence. What would happen if you couldn't hear the rest of the sentence? Note how ambiguous the auditory information becomes when I remove the last part of the sentence (silence + no ending).


Phonemic Restoration Example 2. Phonemic Restoration Example 2 (silence). Phonemic Restoration Example 2 (silence + no ending).
Phonemic Restoration Example 3. Phonemic Restoration Example 3 (silence). Phonemic Restoration Example 3 (silence + no ending).
Phonemic Restoration Example 4. Phonemic Restoration Example 4 (silence). Phonemic Restoration Example 4 (silence + no ending).

 


Interactions between Complex Auditory and Visual Stimuli

Demonstration 11.8 The McGurk Effect The McGurk Effect is an example of the interaction between our senses--specifically audition and vision. Your ultimate auditory experience is different from both the auditory and the visual information. For example, when you hear a person saying "ba" while watching that person's lips form "ga," you'll actually hear "da." This YouTube video provides one example of the McGurk Effect. Lawrence Rosenblum (University of California, Riverside) has conducted a great deal of research on multimodal speech perception, so he's got a demonstration of the McGurk Effect. Dominic Massaro (University of California, Santa Cruz) has done research on the McGurk Effect using a computer-animated face (Baldi). In this demo, you can experience a fairly complex stimulus.


Test Yourself

1. Contrast complex auditory stimuli with the simple auditory stimuli we often discussed in Chapters 9 and 10. Perception of complex auditory stimuli shares many similarities with perception of complex visual stimuli. Illustrate as many similarities between auditory and visual pattern perception as you can. Pay particular attention to cognitive factors. Why do you think these similarities might have emerged?


2. We discussed pitch perception in this chapter and in Chapter 10. Why is pitch more complicated than a simple arrangement of notes from low to high? Be certain to mention the concepts of the octave and tonality. Then discuss how notes with similar tone chromas from different octaves are similar to each other. How might you use pitch to contrast the physical stimulus and the psychological stimulus?


3. Imagine yourself at a concert of your choice. Discuss how the material in the music perception portion of the chapter might make you more aware of perceptual qualities of the music. Be certain to mention pitch, tonality, loudness, timbre, and tone combinations.


4. Speech perception seems so easy and effortless. Nonetheless, theories of speech perception are complex. Why? Describe the distinctions among the Special Mechanism and the General Mechanism accounts of speech perception. In spite of the evidence against Special Mechanisms, why might one still believe that speech is special?


5. Illusions play important roles in helping us to understand the operation of perceptual systems. How might the musical and speech illusions (e.g., phonemic restoration) in this chapter help inform our understanding of the auditory system? Can you make any links to visual illusions?


6. Integrate the information from the previous two chapters with information from this chapter to describe what happens in the brain as you listen to music and to speech. How might the complexity of brain activity argue against areas of the brain being specialized for music or speech?


7. Imagine that you have just heard a reporter on television deliver a report. As you watched the reporter, you heard one sentence distinctly except for a part of one word. Use the information from this chapter to illustrate how you would be able to understand the utterance in spite of incomplete auditory information.


8. In earlier chapters, we often commented on the importance of context for visual perception. In this chapter, you have seen a number of examples in which context influences auditory perception. Use several examples from this chapter to clarify the role of context in perception. Does the role of context seem to differ when dealing with auditory stimuli?


9. Top-down (or conceptually driven) processing becomes especially important in the perception of complex stimuli. Describe the operation of top-down processes in the perception of complex auditory stimuli. How do bottom-up (data-driven) processes function in the perception of complex auditory stimuli? Which class of processes do you think is dominant in the perception of complex stimuli?


10. After learning about visual perception, you’ve now completed three chapters about auditory perception. Given what you know now, how would you characterize the relationship between vision and audition? Make use of the information in this chapter (e.g., the McGurk effect) to support your position.

 


Teaching Materials


Philomel Records

Diana Deutsch (University of California San Diego) has produced these two CDs, which demonstrate a number of phenomena that Dr. Deutsch has researched extensively. On the Musical Illusions CD you'll find the octave illusion and the tritone paradox, both of which are discussed in the textbook. On the Phantom Words CD you'll find additional clever demonstrations, including phantom words and "music" emerging from repetitive speech. You'll also find useful information and auditory demonstrations on her web page.

• Subliminal Messages

John Vokey and Don Read have written an excellent article and produced an enlightening audio cassette illustrating backwards speech. If you've ever listened to a Beatles album backwards in an effort to hear hidden messages (or even if you haven't) you'll really enjoy this tape. Information on acquiring the tape is in the article:

Vokey, J. R. & Read, J. D. (1985). Subliminal messages: Between the Devil and the media. American Psychologist, 40, 1231-1239.

And, of course, you can create your own examples of reverse speech/music using sound editing software (examples below). If you want to teach your students to be critical thinkers, you might want to make use of the ReverseSpeech site, wherein David John Oates makes a number of "humble" claims ("It has been called the discovery of the 7th sense. The research into this phenomenon has been described as being of 'Nobel calibre'.") :-) Unfortunately, the pseudoscience that characterizes the site appears to be seductive to some people.

• Sound Editing Software

You can create a number of interesting demonstrations with some relatively cheap and easy-to-use software. In the Mac world, Amadeus is a great choice. Martin Hairer develops the software, which you can obtain from his website Hairersoft.com. A cross-platform option is Audacity, which is free. You can download the software from the Audacity website.

Link - The UCLA Phonetics Lab Data site provides a number of sounds from different languages. Peter Ladefoged will surely be missed! Some additional demonstrations from the lab are to be found here. In addition, you can access a number of teaching aids.

Sensimetrics is a company that has developed a number of tools that can be used for teaching about speech (including a PC-based CD called Speech Production and Perception, a CD for both PCs and non-Intel Macs called Seeing and Hearing Speech, and a hearing loss simulator).

Link - Rob Hagiwara (University of Manitoba) produces a Monthly Mystery Spectrogram. Can you figure out the speech shown graphically?

Link - Musical Acoustics pages at University of New South Wales (John R. Smith) including a virtual flute.

Link - Music Cognition Resource Center (Ohio State) has a number of organized links, etc.

Link - The Music Cognition Lab (Queens University) also has helpful links regarding music.

Link - Music Research Site (University of Miami) maintains a wealth of useful information.

Link - Stefan Koelsch (Max Planck Institute) does interesting work on music perception. Not only does his page have access to files of his published work, but also he's placed appropriate sound files on the page.

Link - Vinod Menon (Stanford University) and his colleagues have published an interesting paper (in Neuron) about neuroimaging in the processing of music. Their web page also has a movie showing how the brain processing changes with changes in the music.

Link - Andrew Oxenham (University of Minnesota) heads the Auditory Perception and Cognition Group, providing a set of interesting sound samples related to publications from the group.

Link - Robert Zatorre (McGill University) maintains a very useful site with publications and some supporting sound files (look at Supplements).

Link - Isabelle Peretz (University of Montreal) also maintains a very useful site with publications and supporting sound files (e.g., emotion stimuli).

Link - The primary outlet for research on music perception is a journal called...Music Perception. :-) You might also want to check out the pages of the Society for Music Perception and Cognition.

Link - The Exploratorium has a few demonstrations of hearing phenomena.

Link - Dale Purves (Duke), who has done a lot of very interesting work in visual perception, has turned his attention to sound and music. Hear some demonstrations at his site.

Link - If you'd like to help out with some research on music perception, please go to Josh McDermott's web study (while it's still available).

Link - Robert Dooling (University of Maryland) does interesting work on communication in (small) birds, so you'll learn a lot by visiting his pages.

Link - Isabelle Charrier has a page that supports her study of vocal communication in seals (pinnipeds). You'll find both pictures and recordings of the vocal communications.

Link - The Exploratorium has an online exhibit about Listening.

Link - Lori Holt (Carnegie Mellon) has provided a page of auditory demonstrations, including the McGurk effect.

Link - Various links to sources in Acoustics, Ultrasound, and Vibration.


Recommended Readings

Butler, D. (1992). The musician's guide to perception and cognition. New York: Schirmer. (This book is an excellent resource, but the accompanying CD is particularly useful for classroom demonstrations.)

Cook, P. R. (Ed.) (1999) Music, cognition, and computerized sound: An introduction to psychoacoustics. Cambridge, MA: MIT. (With CD of demos)

Diehl, R. L., Lotto, A. J. & Holt, L. L. (2004). Speech perception. Annual Review of Psychology, 55, 149-179.

Levitin, D. J. (2006). This is your brain on music: The science of a human obsession. New York: Dutton.

Patel, A. D. (2008). Music, language, and the brain. New York: Oxford.

Peretz, I. & Zatorre, R. J. (2005). Brain organization for music processing. Annual Review of Psychology, 56, 89-114.

Radocy, R. E. & Boyle, J. D. (Eds.) (2003). Psychological foundations of musical behavior, 4th Ed.. Springfield, IL: Charles C. Thomas.

Sacks, O. (2008). Musicophilia: Tales of music and the brain. New York: Vintage.

Sloboda, J. (2005). Exploring the musical mind. Oxford.