Hearing What You See || Seeing What You Hear

Eyes vs. Ears

I have previously thought that visual information almost always predominates over all other senses. In fact when you think about it, we constantly receive salient visual information as long as our eyes are open, which is technically almost all our waking hours. We depend on our eyes so much and rightfully so because it us one of our more developed sense organ. Nonetheless our eyes are not foolproof. They can be easily fooled by the simplest optical illusion. But another proof that our eyes are susceptible to errors, and perhaps something relatively unknown to most people, is that our eyes can be fooled by our ears. This is known as the Sound Induced Visual Motion Perception.

When I first heard/read of this, my initial reaction was “what is that, too many big words!” But then dissecting the term showed me that it is actually fairly simple. When we hear certain arrangements of sounds (sound induced) we sometimes see visual stimulus as moving despite the lack of actual movement (visual motion perception).

Two notable studies on this topic include one entitled “Auditory Motion Information Drives Visual Motion Perception” published online in 2011, and a 2013 study, “Effect of Pitch-Space Correspondence on Sound-Induced Visual Motion Perception”. Both papers were written mostly by Japanese authors and major contributions on both papers were made by Souta Hidaka and Wataru Teramoto. 

Visual Stimulus + "Moving" Sound = Mind Tricked
Amazing, right? (haha!)

Most people will be creeped out to know that they sometimes see movement where there is none. If this happens, some may even think that they are hallucinating or that something is haunting them. Little do they know that simply adding the right combination of sounds to a particular visual stimulus will trick your mind into seeing movement.

Let us now look at the two studies mentioned above in greater detail. The earlier research published in 2011 found out that a smooth sound moving either from the left or the right to the opposite direction induced movement perception of a non-moving object located outside the focus of the eye. This effect also increases as the object is moved to a more peripheral area. So generally, it means that an object seen using your peripheral vision will appear to move when accompanied by a sound moving from left to right or right to left. Furthermore this effect is so strong that people cannot distinguish whether movement of the object in their periphery was just an illusion brought about by the introduction of the sound or if there was actual movement (Hidaka, Teramoto, Sugita, Manaka, Sakamoto, & Suzuki, 2011).

Did it move?

The second study is somewhat similar to the first except that it took into consideration pitch and alternating up/down located sounds.  In this experiment, the researchers determined that hearing up/down sounds can give the impression that an object located in the periphery is moving (similar to the findings for left/right sounds). In the case of pitch, they found out that it did not affect visual motion perception before exposure however, after associating a pitch with a visual apparent motion, a change in pitch resulted to seeing the object move(Hidaka, Teramoto, Keetels, & Vroomen, 2013).

I know this is hard to believe. Your eyes and ears working together to play tricks on you? No way! Besides, where would you even encounter something like that in real life. Outside of the laboratory setting, most sounds accompany motion and most movements are accompanied by sounds. They are not isolated especially when looking at it using physics. Nonetheless these two researches show how we integrate the information coming from our senses, particularly vision and hearing, and it proves that our senses indeed work together (for us, or against us) in order to interpret what we perceive. 

So now the question is,

Better yet, let me ask

Because sometimes, they sure are playing tricks with our brains...   

References:

- Hidaka, S., Teramoto, W., Sugita, Y., Manaka, Y., Sakamoto, S., & Suzuki, Y. (2011). Auditory Motion Information Drives Visual Motion. PLoS ONE , 6 (3), 1-9.

- Hidaka, S., Teramoto, W., Keetels, M., & Vroomen, J. (2013, September 11). Effect of Pitch-Space Correspondence on Sound-Induced Visual Motion Perception. Experimental Brain Research.

Image Sources:

1. Minions

http://orig15.deviantart.net/8575/f/2013/340/a/1/see_no_evil_hear_no_evil_speak_no_evil_by_dincenzo-d6x0848.jpg

2. Candace

https://s3.amazonaws.com/rapgenius/candace_hallucinating_by_osopod2-d3gnl79.gif

3. Confused

http://www.revelationtv.com/images/uploads/thumbnails/news/confused-by-credit.jpg

4. Trust Eyes

http://i.ytimg.com/vi/ZflIMBxyIak/maxresdefault.jpg

5. Trust Ears

 http://i.ytimg.com/vi/kzo45hWXRWU/maxresdefault.jpg