To place this blog entry into context, I will first kindly ask you to watch two videos which features two highly accomplished pianists from two different centuries. The first one shows Arthur Rubinstein (1887-1982) and the second one Lang Lang (1982 – ). Both will be playing the same piece: Chopin’s Polonaise op. 53 in A flat major. Please, watch Rubinstein’s video at least up to minute 2.14, and Lang Lang’s video up to minute 1.44.
Arthur Rubinstein playing Chopin
Although they both play the same piece, it is possible to perceive differences among them in tempo, dynamics, phrasing, etc. However, I would like to draw your attention to their body movements. Wanderley, Vines, Middleton, McKay and Hatch (2005) made a distinction between movements that are directly involved in producing sound in an instrument, and movements which are not. The first kind were called “instrumental gestures”, while the second kind were called “ancillary gestures”. In pianists, the most obvious instrumental gestures is the movement of the fingers on the keys. Meanwhile, the ancillary gestures is the movement of the shoulders and head and facial expressions.
Rubinstein performs the piece with just the minimal instrumental gestures. So, why is Lang Lang moving so much to play the same notes? Do ancillary gestures possess any relevant role in music perception, particularly in the musical expressive intention of the performer and the audience’s perceived quality of a performance?
The expressive intentions of the performer
Does ancillary gestures have any function? In order to answer this question, different researchers have set up a similar experimental paradigm: they used different kind of movements of varying expressive intensity, ranging from a flat, inexpressive and invariant performance up to a performance in which all dynamic, tempo indications and other musical features have to be overstated. Alternatively, players can be asked to perform following different instructions: focusing on technique, expression or emotion (Van Zijl, Toiviainen, Lartillot & Luck, 2014). For example, Thompson and Luck (2012) asked pianists to play a Chopin’s prelude in four possible movement conditions: deadpan, normal, exaggerated and immobile. In a similar fashion, Wanderley et al. (2005) also instructed their clarinetists’ participants to play under three conditions: standard, expressive and immobile.
Movements are usually recorded with some sort of motion capture system (the same kind of technology that allowed the creation of digital characters such as Gollum, from Lord of the Rings) which enables to register subtle changes in different movement parameters. Additionally, music is recorded with audio or MIDI technology. Both, movement and audio information, are then analyzed with impressive software designed by very clever people. So, what have we learned out of this technology about the expressive intentions of performers? Movements do affect sound properties.
Davidson’s (1993) seminal work showed that playing pieces in deadpan, projected and exaggerated manners generates clear audible differences which allow people to rate them as different in terms of expressiveness. Van Zijn et al. (2014) found that sad performances differed significantly from expressive ones in several musical features such as loudness, dynamic range, tempo, etc. These recent findings have been supported by previous ones. For instance, in terms of duration of the pieces, Wanderley et al. (2005) found that performances such as standard or expressive were significantly slower than immobile or non-expressive ones. This result was echoed by Thompson and Luck (2012) who reported that, on average, deadpan performances were the shortest, while exaggerated the longest.
But what about movements themselves? Do they change according to the performance manner? There is evidence that suggest so. Wanderley et al. (2005), Davidson (2007), and Thompson and Luck (2012) arrived to similar conclusions on different studies, with different instrument players and different pieces: regardless of the expressive manner, musicians show consistency in their ancillary movements, although they varied in their intensity according to the degree of expressiveness. What is more, movements coincided with highly musically tense sections and musical phrasing boundaries (although some recent evidence has posit alternative explanations. See Demos, Chaffin and Logan, 2017). Taken together, ancillary movements are a building part of a performer’s representation of the piece, pointing at relevant sections in it. But, is the audience perception of the performance affected by ancillary movements?
The audience’s perceived quality of a performance
Audience’s rating of expressiveness are predominantly influenced by the visual channel. Davidson (1993) showed that experienced audience can detect changes in expressivity with greater accuracy if they watch a performer than if they simply listen to an audio recording. Moreover, Tsay (2013) found that both novice and expert musical judges rely heavily on visual information to raise a judgment on a performance, and not on auditory means. Moreover, novice judges can accurately predict the winner of a music competition just by watching a silent performance, thus implying that visual information bias their decisions. Thus, ancillary movements are a source of information about expressiveness levels and even performance quality.
So… why is Lang Lang moving so much?
We can now partly try to answer that question. Playing an instrument involves both instrumental and ancillary gestures. According to the described studies, ancillary gestures are a rich source of interpretative and aesthetic information for both the player and the audience. Thus, Lang Lang might be a clever (conscious or unconscious) operator of our psychological functioning. Some of us, however, resist to play his game.
Summary
- Different performance manners have an influence in musician’s body movement.
- These movements are consistent across performing conditions and are closely related to structurally relevant musical sections.
- Audience are influenced on their judgment about a performance (in terms of expressivity and quality) based on the perceived body movements.
References
Davidson, J. W. (1993). Visual perception of performance manner in the movement of solo musicians. Psychology of Music, 21, 103-113. doi: 10.1177/030573569302100201
Davidson, J. W. (2007). Qualitative insights into the use of expressive body movement in solo piano performance: A case study approach. Psychology of Music, 35(3), 381-401. doi:10.1177/0305735607072652
Demos, A. P., Chaffin, R., & Logan, T. (2017). Musicians body sway embodies musical structure and expression: A recurrence-based approach. Musicae Scientiae, 1-20. https://doi.org/10.1177/1029864916685928
Thompson, M. R., & Luck, G. (2012). Exploring relationships between pianists’ body movements, their expressive intentions, and structural elements of the music. Musicae Scientiae, 16(1), 19-40. doi:10.1177/1029864911423457
Tsay, C. J. (2013). Sight over sound in the judgment of music performance. Proceedings of the National Academy of Sciences of the United States of America, 110(36), 14580–14585. doi: 10.1073/pnas.1221454110
Van Zijl, A. G., Toiviainen, P., Lartillot, O., & Luck, G. (2014). The sound of emotion. Music Perception: An Interdisciplinary Journal, 32(1), 33-50. doi:10.1525/mp.2014.32.1.33
Wanderley, M. M., Vines, B. W., Middleton, N., McKay, C., & Hatch, W. (2005). The musical significance of clarinetists’ ancillary gestures: An exploration of the field. Journal of New Music Research, 34(1), 97-113. doi:10.1080/09298210500124208
Music, Mind & Technology 
Nice post, Alvaro. I don’t quite agree with your statement that the literature tells us that musicians’ expressive movements affect the sound properties of their music. You seem to cite Davidson (1993) and Van Zijl et al.,( 2014) as evidence that musician’s movements affect the sound they produce, but the experimental manipulation in these studies was not the movement, it was the expressive intention. Although movements did change according to the expressive intention, I would say that it was the intention which affected the sound differences, not necessarily the movement. You didn’t mention that Thompson and Luck (2012) actually found that expressive timing was not affected by the immobile condition, although it was for the deadpan, suggesting that it isn’t the movement itself that causes changes to the sound, but the expressive intention. This idea is supported by Palmer et al. (2009) who looked at changes in acoustic features dependent on clarinet bell movements but found nothing significant.
Actually, I wonder if I misunderstood what you meant to say? I thought I would chip in with that anyway.
LikeLike
Hey Emma! Thanks for your interest and observations. Indeed, the studies you quoted were measuring the effect of expressive intentions in different kinds of music-related variables. Here is my reply to your commentary.
Although it might be fair to imply a difference between expressive movements and expressive intentions, there might not be a more direct way of measuring “expressive intentions” in performance other than through body movements. When you say “although movements did change according to the expressive intention, I would say that it was the intention which affected the sound differences, not necessarily the movement” I cannot imagine how else can you generate differences in sound other than executing some sort of movement. Having an intention to move a table does not have the same effect as if I actually go and push it.
Your commentary allowed me to point out at one limitation in psychological research. Unfortunately, since we cannot measure our intended constructs (personality, motivation, anxiety, etc.) in the same way as we can measure the length of a table, then we must come up with creative ways to study psychological entities (through psychological tests, electrophysiological measures, functional analyses, etc.). Consequently, expressive intentions (at least in the quoted studies) can only be measured through expressive movements. In other words, for measuring purposes, I conceive expressive movements as a way of measuring expressive intentions.
Perhaps the last two studies you mentioned did not report any effect of expressive intentions in sound properties for different reasons. Instructions could not have been relevant for obtaining the intended response, or maybe players executed movements that had no real effect in the sound generation. If the latter is true, then these movements could still have an impact on a observing audience: the ancillary gestures could communicate the performer’s emotional state (even if the music does not reflect the communicative intention of the performer).
LikeLike
You make some interesting points. Of course sound is produced from movement, but the distinction is made between ‘sound producing’ and ‘sound accompanying’ gestures. My point is that there is no evidence, so far, that the sound accompanying gestures affect the sound produced, which is the basis for my masters’ thesis. If you want to make an expressive crescendo on a violin you alter the speed, weight and/ or position of the bow on the string. Of course this involves movement of the muscles in your fingers and arms, but whether or not you sway your body, flick your hair, raise your eyebrows or dance a little polka may or may not change the actual sound produced by the violin. I am talking about these extra gestures, and I don’t think they can be considered an indirect measure of expressive intention yet, as there is no evidence that they directly affect the sound. The research tells us that gestures do generally reflect expressive intention, but I have seen performers play who move very little, and that doesn’t necessarily mean they have no expressive intentions.
Also, I would say that in the cited studies, expressive intention was determined by the instruction given to the performer, not measured through gesture. It was decided that the performer’s intention was ‘deadpan’ because they were told to play ‘deadpan’. Then, the gestures were examined, to see what was happening, but the gestures weren’t a measurement for the expressive intention. Expressive intention could be measured by asking the performer what their intention was, by measuring listener perceptions, or by analysing acoustic features of the sound produced. Expressivity in music can be defined as variations in intensity, tempo, sound quality and even pitch, that is often not written in the score, so measuring these things would be a better way to measure intention than through gesture.
Of course the gestures have an impact on the visual perception of the audience, as you say, but I still believe you are mistaken to say that they have an impact on the sound.
LikeLike
Nice post, Alvaro. I have a potentially boring conceptual question for you. If ancillary movements impact the music produced as well as perception of a piece of music, do you think this distinction between ancillary and instrumental gestures serves a useful purpose in music cognition research? I gather from the research on this topic that if you take ancillary gestures away then you change important qualities of the music. If we are looking at the impact of music on a listener or the full range of music that can be produced from someone and their instrument, aren’t ancillary gestures essential to the music, much like instrumental gestures? This could just be a semantic game that researchers are playing, but maybe you’ll play along here, too 🙂
LikeLike
I meant to include this! One of my favourite examples of ancillary gestures that I think displays rhythm maintenance.
LikeLike
Hey Shannon, I just wanted to chip in because as I commented above, I believe there is not evidence that if you ‘take ancillary gesture away’ (e.g ask a musician to be still while they play) it changes the quality of the SOUND, although it does affect visual perception. (If you have some evidence of this, please let me know!) There are certain musical tasks that can be executed with as little movement as possible, or with these extra movements, and I think the separation of the two is useful because it highlights the phenomenon of musicians making these visually expressive movements. The question, for me (and my thesis) is; are these gestures important somehow in the cognitive process of shaping expressive ideas? in other words, are these seemingly extra gestures evidence of the embodied nature of creating expression in music? If they are, then it is interesting because they will, in a way become sound producing gestures.
LikeLike
I am not sure if it is correct to make this comparison between two artists from two different century, during a hundred years many things have changed, like culture and technology and many others. Internet somehow changes the concept of art, or it is better to say it makes many new artistic concept, so performers try not to be just a sound producer, they also try to be visually attractive.
LikeLike