Hearing What You See || Seeing What You Hear

Eyes vs. Ears

I have previously thought that visual information almost always predominates over all other senses. In fact when you think about it, we constantly receive salient visual information as long as our eyes are open, which is technically almost all our waking hours. We depend on our eyes so much and rightfully so because it us one of our more developed sense organ. Nonetheless our eyes are not foolproof. They can be easily fooled by the simplest optical illusion. But another proof that our eyes are susceptible to errors, and perhaps something relatively unknown to most people, is that our eyes can be fooled by our ears. This is known as the Sound Induced Visual Motion Perception.

When I first heard/read of this, my initial reaction was “what is that, too many big words!” But then dissecting the term showed me that it is actually fairly simple. When we hear certain arrangements of sounds (sound induced) we sometimes see visual stimulus as moving despite the lack of actual movement (visual motion perception).

Two notable studies on this topic include one entitled “Auditory Motion Information Drives Visual Motion Perception” published online in 2011, and a 2013 study, “Effect of Pitch-Space Correspondence on Sound-Induced Visual Motion Perception”. Both papers were written mostly by Japanese authors and major contributions on both papers were made by Souta Hidaka and Wataru Teramoto. 

Visual Stimulus + "Moving" Sound = Mind Tricked
Amazing, right? (haha!)

Most people will be creeped out to know that they sometimes see movement where there is none. If this happens, some may even think that they are hallucinating or that something is haunting them. Little do they know that simply adding the right combination of sounds to a particular visual stimulus will trick your mind into seeing movement.

Let us now look at the two studies mentioned above in greater detail. The earlier research published in 2011 found out that a smooth sound moving either from the left or the right to the opposite direction induced movement perception of a non-moving object located outside the focus of the eye. This effect also increases as the object is moved to a more peripheral area. So generally, it means that an object seen using your peripheral vision will appear to move when accompanied by a sound moving from left to right or right to left. Furthermore this effect is so strong that people cannot distinguish whether movement of the object in their periphery was just an illusion brought about by the introduction of the sound or if there was actual movement (Hidaka, Teramoto, Sugita, Manaka, Sakamoto, & Suzuki, 2011).

Did it move?

The second study is somewhat similar to the first except that it took into consideration pitch and alternating up/down located sounds.  In this experiment, the researchers determined that hearing up/down sounds can give the impression that an object located in the periphery is moving (similar to the findings for left/right sounds). In the case of pitch, they found out that it did not affect visual motion perception before exposure however, after associating a pitch with a visual apparent motion, a change in pitch resulted to seeing the object move(Hidaka, Teramoto, Keetels, & Vroomen, 2013).

I know this is hard to believe. Your eyes and ears working together to play tricks on you? No way! Besides, where would you even encounter something like that in real life. Outside of the laboratory setting, most sounds accompany motion and most movements are accompanied by sounds. They are not isolated especially when looking at it using physics. Nonetheless these two researches show how we integrate the information coming from our senses, particularly vision and hearing, and it proves that our senses indeed work together (for us, or against us) in order to interpret what we perceive. 

So now the question is,

Better yet, let me ask

Because sometimes, they sure are playing tricks with our brains...   

References:

- Hidaka, S., Teramoto, W., Sugita, Y., Manaka, Y., Sakamoto, S., & Suzuki, Y. (2011). Auditory Motion Information Drives Visual Motion. PLoS ONE , 6 (3), 1-9.

- Hidaka, S., Teramoto, W., Keetels, M., & Vroomen, J. (2013, September 11). Effect of Pitch-Space Correspondence on Sound-Induced Visual Motion Perception. Experimental Brain Research.

Image Sources:

1. Minions

http://orig15.deviantart.net/8575/f/2013/340/a/1/see_no_evil_hear_no_evil_speak_no_evil_by_dincenzo-d6x0848.jpg

2. Candace

https://s3.amazonaws.com/rapgenius/candace_hallucinating_by_osopod2-d3gnl79.gif

3. Confused

http://www.revelationtv.com/images/uploads/thumbnails/news/confused-by-credit.jpg

4. Trust Eyes

http://i.ytimg.com/vi/ZflIMBxyIak/maxresdefault.jpg

5. Trust Ears

 http://i.ytimg.com/vi/kzo45hWXRWU/maxresdefault.jpg

The Algorithm to Emotions: Music Manipulation and its Application

Music always plays a huge part in our life. From the lullabies we hear as children, the music we listen to as teenagers, and the songs we use for special events. It’s impossible not to hear music and not feel something at the same time. So, the question is, why is it that some music makes us happy? Why is it that some make us sad? Why do we even listen to sad music if it makes us sad? And why do we need to answer these questions anyway?

But before anything else, researchers have debated amongst themselves on the definition of emotions. (Hunter and Schellenberg, 2010) What exactly constitutes emotions? For the layman, emotions are the usual: happy, sad, angry, disgust, excitement, fear, etc. However, when taken into research terms, it becomes much more complicated than that. For example, when we listen to music, are we actually feeling emotions that we feel in everyday life or is music just inducing temporary feelings that are different from the emotions we feel everyday?

Another matter of fact is, how exactly does music affect emotions? Psychologists have to separate emotions, feelings, and moods. While there are various theories chosen to explain how these came to be, it has been mostly agreed upon that music expresses emotion and these things affect us. Researchers argue whether the emotions we feel are real emotions, aesthetic emotions, or just moods induced by the music. It’s also been debated whether we just perceive the emotion expressed by the music or if we’re really feeling this emotion. (Hunter and Schellenberg, 2010)

There have been various studies debating on the many properties of music and emotions (citation), but if there’s one thing that all the researches have agreed on, it’s that certain chords are associated with certain emotions. Major chords are associated with happiness, minor chords are associated with sadness, and dissonance chords are associated with unpleasantness (Scherer, 2004). An example would be playing a song in either minor or major chords and seeing the difference of the emotions evoked in the chords played.

As seen in the video, switching from major to minor chords can really change how songs feel, don't they?

            It’s incredible how just changing the set of chords can make us feel very different things. However, it’s not just the chords that could change how music can portray emotions. Other musical characteristics can actually also change how music can feel.  For example, tempo and mode are the strongest determinants of emotion in music in a series of studies created before (citation). Fast tempos and major modes are associated with happiness and slow tempos and minor modes are associated with sadness. However, even as music can make us feel emotions, the most consistently judged emotions are happiness and sadness, compared to other emotions like anger and fear.

            Some evidence has also been found that timbre affects emotions in music. Soft timbres attenuated with high frequencies are associated with sadness, where as sharp timbres are associated with anger. Loudness of the sound has also been found to be a universal cue to anger according to a cross-cultural study by Balkwill and coworkers (2004)

As seen in this video, one can really feel the anger and unpleasantness of the song. It doesn't matter if you don't understand the lyrics, as long as you hear that melody, it feels quite angry.

Compared to this very soft and calming music, the disparity between the two really is huge. The other makes you feel anger while this one calms you down and makes you feel at peace with yourself.

            Although while it has been found that music portrays emotion, interestingly enough, children have difficulty identifying the emotions in music, at least until they reach the age of 11 years old, where they reach the adult-like accuracy (Hunter et. al, 2008)

            So why should we learn about emotions and music and why they’re related? Music is always a huge part of our lives. The music industry wants to sell more, wants people to buy their product. Obviously, if you’re a customer and you realize you like a product, the next question you usually ask is:

Is there anything else like this?

            And so, these connections of music and emotions have been used in the music industry in order to recommend songs or group playlists according to the emotions that each one portrays. Using a complicated formula, theories, and models that predict the type of emotion music portrays and recommends music according to the similarity of the songs. This includes Spotify, MusicCat, and other music sharing programs (Han, B. J., Rho, S., Jun, S., & Hwang, E.,2010 ;Shan, M. K., Kuo, F. F., Chiang, M. F., & Lee, S. Y., 2009)!

            So there you have it, music’s algorithm and its application in real life.

REFERENCES:

Balkwill L-L, Thompson WF, Matsunaga R (2004) Recognition of emotion in Japanese Western and Hindustani music by Japanese listeners. Jpn Psychol Res 46:337–349. 
Han, B. J., Rho, S., Jun, S., & Hwang, E. (2010). Music emotion classification and context-based music recommendation. Multimedia Tools and Applications,47(3), 433-460.Hunter, P. G., & Schellenberg, E. G. (2010). Music and emotion. In Music perception (pp. 129-164). Springer New York.Pallesen, K. J., Brattico, E., Bailey, C., Korvenoja, A., Koivisto, J., Gjedde, A., & Carlson, S. (2005). Emotion processing of major, minor, and dissonant chords.Annals of the New York Academy of Sciences, 1060(1), 450-453.Shan, M. K., Kuo, F. F., Chiang, M. F., & Lee, S. Y. (2009). Emotion-based music recommendation by affinity discovery from film music. Expert Systems with Applications, 36(4), 7666-7674.Scherer, K. R. (2004). Which emotions can be induced by music? What are the underlying mechanisms? And how can we measure them?. Journal of new music research, 33(3), 239-251.

The Voice- #HALALAN2016

 May 2016 elections is fast approaching and as an excited first time voter, I have been caught up with who is going to run for which position, researching for their political records and alliances. It is very important for me to know which one is credible and worthy enough to get seated as an administrator and public servant. Being interested in politics and voting behaviours right now, I want to explore on the mechanisms that might influence people to vote for a certain candidates. Among other factors, I think that a politician’s voice- biologically and metaphorically- is one important characteristic to influence voting behaviour.

                Public appearances such as press conferences, interviews, media advertisements and debates are essential for a politician who is running for an office whether it may be national or regional. More than what they want to tell and convey, first of all, they must first catch the mass’s attention and maintain that attention until the end. Human voice pitch, which is distinct between the sexes, conveys important information about its owner (Borkowski & Pawlowski,2011). Our auditory perception has helped us in making sense in our environment (Goldstein, 2013) which makes us vulnerable to being bias in judging a person’s attractiveness (Feinber, et al., 2006;  Zuckerman & Miyake, 1993; Hughes, Moailski, & Harrison, 2014) and dominance (Borkowska & Pawlowski, 2011; Klofstad, Anderson & Nowicki, 2015;  Klofstad, Anderson, & Peters, 2012; Hodges-Simeon, Gaulin and Puts 2010). Generally, studies have found out that politicians and candidates with lower pitch voices are favoured and preferred. But why?

PERCEIVED ATTRACTIVENESS

                In a study conducted by Hodges-Simeon, Gaulin and Puts (2010) on predicting perceptions of dominance and attractiveness by examining the different vocal parameters, they have found out that the low mean fundamental frequency in men’s voices has been found to positively influence perceptions on dominance by men and attractiveness by women using standardized speech. Since frequency is a physical measure of pitch (Goldstein, 2010), the study implies that lower pitch for males makes them perceived to be dominant.

                Women with lower voices were also found out to be perceived as more dominant. In a study  by Borkowska and Pawlowski (2011) wherein voices of women that were classified as  low (184.6 Hz), medium (223.7 Hz), high (261.9 Hz) or very high (310.3 Hz) have been recorded and assessed of their dominance and attractiveness, it was found out that low voices are more dominant and the relationship is linear. It was also found out that female voices above 280 Hz are judged to be too high that sounds babyish which is related to being immature.

AGE

                Investigating the influence of perception on competence, strength and age on voters in selecting leaders,  Klofstad, Anderson and Nowicki (2015) found out that lower-pitch voices are generally preferred. Male and female candidates who are in their 40s and 50s, a time in human life cycle when voice pitch is at its lowest, are chosen over candidates on their 30s, 60s and 70s. This is an antithesis to their hypothesis that through age comes wisdom and credibility which might favour older candidates.  It was concluded that speakers with lower voices are stronger, more competent, and older.

LIKABILITY

Lastly, lower-pitched voices were associated with favourable personality traits more often than were higher-pitched voices and that people preferred to vote politicians with lower pitched than higher pitched voices (Tigue and colleagues, 2012). 

References:

Borkowska, B., Pawlowski, B. (2011). Female voice frequency in the context of dominance and attractiveness perception. Animal Behavior. 82(1): 55-59

Feinber, D.R., Jones, B.C, Law SMih, M.J., Moore, F.R., DeBruine, L.M>, Cornwell, R.E., Hillier, S.G., Perrett., D.I. (2006). Menstrual cycle, trait estrogen level, masculinity preferences in the human voice. Hormones and Behavior, 49(2): 215-222

Goldstein, E.B. (2010, 2013) Sensation and Perception. Cengage Learning: Wadsworth

Hodges-Simeon, C.R., Gaulin, S.J.C, Puts, D.A. (2010). Different vocal parameters predict perceptions of dominance and attractiveness. Human Nature 21: 406-420. DOI: 10.1007/s12110-010-9101-5

Hughes, S.M., Moailski, J.K., Harrison, M.A (2014). The Perception and Parameters of Intentional Voice Manipulation. Journal of Nonverbal Behavior. 38(1): 107-127

Klofstad, C.A., Anderson, R.C, Peters, S. (2012). Sounds like winner: voice pitch influences perception of leadership capacity in both men and women. Retrieved from: rspb.royalsocietypublishing.org

Klofstad, C.A., Anderson, R.C., Nowicki, S. (2015). Perceptions of competence, strength and age influence voters to select leaders with lower-pitched voices. Retrieved from: journals.plos.org

Tigue, C.C., Borak, D.J, O’Connor, J.J.M, Schandl,C., Feinberg, D.R. (2012). Voice pitch influences voting behaviour. Evolution and Human Behavior 33(3): 210-216

Zuckerman, M., Miyake, K. (1993). The attractive voice: What makes it so?

Pitch Perfect 2: How to Get an Earworm

I remember the first time I watched Pitch Perfect 2. When the optimistic freshman Emily started crooning the lyrics to her self-composed song “Flashlight”, I cringed. I just plain hated it, especially it’s repetitive used of the word “Flashlight”. When (spoiler alert!) they start singing the song along with past bellas and confetti fluttering everywhere for the

You're my Flashlight-light-liSHHHHHHUSHH YOU

championship (end of spoiler), I still loathed it. Even when I found out that Jessie J composed the song, I still couldn’t bring myself to care for it. But as fate (and any musically-inspired and popular movie) would have it, the song seemed almost omnipresent on the radio, television, and in any local mall or department store near you. For a time, it almost seemed impossible to avoid it so I just had to learn to live with “Flashlight”. One day, I found myself just strolling about minding my own business, when I started absentmindedly humming the lyrics.

got all I need when I got you and I
I look around me, and see sweet life
I'm stuck in the dark but you're my flashlight
You're gettin’ me, gettin’ me through the night

I stopped in my tracks. For a moment, I was totally mortified. When I got over my initial shock, I began a period of self-loathing. How could I, started humming along to its deplorable tune? As I engaged in a battle between hating the song, Pitch Perfect 2, Jessie J, and myself, the song “Flashlight” continued to loop in the background of my mind.

We’ve all had our own experiences of a song abruptly popping into our head and constantly repeating itself like a broken record, but what is this particular phenomenon called? In Germany, the phenomenon was coined Ohrwurm (Williams, 2015). From this point, different names were spawnedsuch as Korvamato (Finnish), Oorworm (Dutch), and amusingly enough, Canzone tormente (Italian) which- if I had to guess- means song of torment.

It’s currently  known as “earworm” in English, but has also been called a brainworm and stuck song syndrome, among others. An earworm happens when a person experiences a short melody which gets stuck in the mind and just keeps repeating beyond a person’s control. However, just because you are experiencing an earworm doesn’t mean that you have a psychological disorder. Earworms are different from hallucinations, since hallucinations are seen as coming from outside and not within your mind. Current research links it to the term ‘Involuntary Musical Imagery’ (INMI). Depending on who you ask, it is synonymous  to INMI(Floridou, Williamson, & Mullensiefen, 2015) or is seen as a type of INMI (Williams, 2015). The INMI has been coined as the process of accessing perceptual information from memory, which brings about involuntary hearing of a song in the ‘mind’s ear’. Seeing the two definitions alongside each other, I can see why current researchers are in an epic struggle to differentiate the two (if that was ever really necessary).

Not really sure if you’ve experienced an earworm? Let’s try describing some general conditions for you, dear reader. Earworms usually last minutes to hours, but don’t usually last beyond 24 hours (Halpern & Bartlett, 2011).It also usually isn’t on constant repeat. Much like a dog who enjoys running away and coming home to eat every day, the song sometimes leaves one’s conscious awareness, only to come back later. Sometimes, one earworm would replace another and continue to co-exist in such a manner. Unless you are just plain crazy for that song, this is a good thing; having one song replay for the rest of my life sounds like a real nightmare. Another notable manifestation of an earworm is the fact that only a small part of the song repeats in a person’s mind, with the brain often favoring a certain part of the song (Beaman & Williams, 2010; Hyman et al.,2015). Puzzled? Take a look at my story again, specifically the lyrics. These lyrics are a part of the song called the chorus, and it is these lines that often repeat over and over again. Don’t get me wrong, it’s still possible to have a full song mentally replaying, but most people find themselves unable to complete it. This is because we often find it difficult remembering the next lines following the chorus, so we ending up with a chorus that just keeps on looping.

Research looking into earworms and INMI is relatively young and the causes of earworms are still pretty unclear, although some researchers have tried giving some possible causes. Floridou, Williamson, & Mullensiefen (2015) narrowed it down into three factors: the music, the situation, and the person. Let’s try scrutinizing some movie scenes and my own “earworm” experience and see if they check out.

Certain song qualities increase the chances of turning the latest pop song into an ear worm. Let’s look back at the earlier example. According to Kellaris (2001; as cited by Floridou, Williamson, & Mullensiefen, 2015), INMI is kind of like a cognitive itch that happens when the “music mosquito” with the right qualities comes along- namely repetition, musical simplicity, and incongruity. Repetition is certainly straightforward, if you repeat a phrase or motif often enough, it becomes a lot easier to remember. The song “Flashlight” is a perfect example, they pretty much repeat the song title several times in the chorus. For Pete’s sake, the chorus ends with the phrase “You're my flash light, you're my flash light” repeating over and over again. Musical simplicity ( a quality that many children’s songs have), is pretty much a staple of pop music. In the end, “Flashlight” possesses all the right ingredients- enough of a unique flavor in terms of rhythm (incongruity), repetition of words, especially flashlight, and musical simplicity- to create the perfect earworm. On to the next step!

"I know that song-Santa's Super Sleigh!"
kid Spock-like Nicholas Hoult> "heartthrob" Nicholas Hoult

As always, context is important. What we might be experiencing or environmental events may be enough to trigger an earworm. In fact, Beaman & Williams (2010), claimed that recent musical exposure carried more weight than the qualities of musical structure like repetition. In fact, recent exposure can be used to explain why people often report playing Christmas songs in their heads during the holidays (Halpern & Bartlett, 2011). Let’s use About A Boy as an example for this case. In the movie, Will is able to live off royalties to his father’s jingle Santa’s Super Sleigh due to its ubiquitous presence in every supermarket, mall, or any random store playing it during the holiday season. In fact, Will just mentioning that Christmas jingle was enough to set Marcus and his mum on a jolly singing frenzy , serenading Will with Santa’s Super Sleigh.  Thus, just hearing more Christmas songs as the Yuletide comes close is enough to increase the chances of a Christmas-related earworm.

The amount of thinking you’re doing is important too. Most people report experiencing earworms while walking, driving, and doing household chores- think we tend to do automatically and without thinking (Hyman, et al., 2013; as cited by Hyman, et al., 2015). On the opposite end of the spectrum, people also experience earworms when doing an especially difficult task which requires a lot of thinking. This may be a case of mind wandering, which depends on cognitive load. So if you want to avoid having an earworm, do an appropriately difficult task- not too hard or too easy- this way, the song won’t have any space to creep back into you consciousness.

Neuroticism=me everyday

The list of situations that can trigger an earworm is actually quite extensive. These include how often you are exposed to the song, or music exposure, which was seen in the case of “Flashlight” being played virtually everywhere. Just being able to relate the song to something in your environment (holding a flashlight-light-light), a memory (remembering Pitch Perfect 2), or event (again, like Christmas) may also cause an earworm.  How you are feeling or affective state may also influence the likelihood of getting an earworm.

The type of person you are can also increase your susceptibility for earworms. In a study conducted by Floridou, Williamson, & Mullensiefen (2015) conducted a study on how personality and musicality affects contracting an earworm. They found that there was a strong positive relationship between musical engagement (which can be exemplified in emotional response to music and creativity) and neuroticism (or how susceptible a person is to anxiety, stress, and other negative moods). In fact, neuroticism also influenced other characteristics related to earworm, such as degree of concert, length, and controllability.

So it looks like my story checks out. While I am not exactly “musically engaged”, I tend to score high on the Neuroticism portion of personality tests. I was engaged in an activity that is automatic to most people (walking), and I remembered a “catchy” pop song from a movie that was just newly released and being played virtually everywhere (Flashlight). I didn’t even stand a chance.

TripleDent Gum, will make you smile!
TripleDent Gum, it lasts a while!

Before I end this article, I want to go back to one of Inside Out’s scenes (yes, we are going back to that movie) wherein Riley recalls the Triple Dent Gum jingle. As soon as it plays, Anger, Fear, and Disgust react with, well, disgust to the constantly reoccurring song.   Contrary to popular belief, aren’t as obnoxious as most people think. In fact, while disliked songs sometimes get stuck, most people report actually liking the songs stuck in their heads and finding it quite pleasant (Beaty et al., 2013 as cited by Hyman, 2015). However, when songs are disliked, they are usually due to constant exposure to it in the environment- something that is beyond their control. An example would be hearing music over the department store sound system while shopping. Lesson here is: avoid the department store unless you want to form bad memories of a song and make it feel intrusive. Or not. It really depends on how you feel about the song in the end, it may just as well elicit feelings of nostalgia or happiness depending on the associations you make with the song.

References:

Floridou, G. A., Williamson, V. J., & Müllensiefen, D. (2012). Contracting earworms: The roles of personality and musicality. Proceedings of ICMPC-ESCOM, 12, 302-310.

Beaman,C.P.,&Williams,T.I.(2010).Earworms (stuck song syndrome): Towards a natural history of intrusive thoughts. British Journal of Psychology, 101, 637–653. http://dx.doi.org/10.1348/000712609X479636

Beaty, R. E., Burgin, C. J., Nusbaum, E. C., Kwapil, T. R., Hodges, D. A., & Silvia, P. J. (2013). Music to the inner ears: Exploring individual differences in musical imagery. Consciousness and Cognition: An International Journal, 22, 1163–1173. http://dx.doi.org/10.1016/j.concog .2013.07.006

Halpern, A. R., & Bartlett, J. C. (2011). The persistence of musical memories: A descriptive study of earworms. Music Perception, 28, 425–432. http://dx.doi.org/10.1525/mp.2011.28.4.425.

Hyman, I. E., Jr., Burland, N. K., Duskin, H. M., Cook, M. C., Roy, C. M., McGrath, J. C., & Roundhill, R. F. (2013). Going gaga: Investigating, creating, and manipulating the song stuck in my head. Applied Cognitive Psychology, 27, 204–215. http://dx.doi.org/10.1002/acp.2897

Hyman Jr, I. E., Cutshaw, K. I., Hall, C. M., Snyders, M. E., Masters, S. A., Au, V. S., & Graham, J. M. (2015). Involuntary to Intrusive: Using Involuntary Musical Imagery to Explore Individual Differences and the Nature of Intrusive Thoughts.

Kellaris, J. J. (2001). Identifying properties of tunes that get ‘stuck-in-your-head’: Toward a theory of cognitive itch. Proceedings of the Society for Consumer Psychology Winter 2001 Conference, 66-67. Scottsdale, AZ: American Psychological Society

Williams, T. I. (2015). The classification of involuntary musical imagery: The case for earworms. Psychomusicology: Music, Mind, and Brain, 25(1), 5.