studiohead logo

Musical Performance Simulator

The Royal College of Music, one of the world’s leading conservatoires, approached us to create a Musical Performance Simulation. Professor Aaron Williamon heads up the RCM’s Centre for Performance Science, and he had identified a large gap in the training of our leading musicians. When musicians learn and practice they usually do so in a small practice room, alone or with fellow musicians and without any of the pressure of an audience.

Stage freight is widely understood issue, and effects people in walks of life from occasional speech givers to the world’s finest pianists, after all why should there be any correlation between expertise with an instrument and comfort under the scrutiny of thousands of eyes and ears?

Aaron’s challenge to us was to recreate the feelings of performing to a live audience, without having a real audience so that musicians could develop their stage presence and performance skills. We needed to fill the gap between practice room and performing on stage.

We undertook user research, interviewing musicians from the RCM, and watched performances from backstage with the musicians perspective in our minds and came up with guiding principles that lead to our resulting design.

1. When performing, the timing of events is out of the control of the musician and therefore contributes to the anxiety and stress of the occasion. The start time is dictated by the venue, preceding artists, and the readiness of the audience.

2. By having an audience that can react is a variety of ways, demonstrating variance in their level of approval, we can recreate some of the pressures of performing to real audience. Audiences also interrupt performances with coughing, fidgeting, tapping (sometimes out of time), and their un-silenced mobile phones.

3. The size of the audience is not necessarily an important factor. An intimate setting full of familiar people can be as daunting as a packed stadium.

The resulting design has two key components:

RCM-1

1. A green room – this is the holding area where the muscians wait before going out on stage. Although there is not audience, the simulation makes the musicians wait until the audience is settled, and in place. While they wait, they can see ‘CCTV monitor’ footage of an audience assembling for a concert.

RCM-sim-4

2. An interactive, projected audience. A technician can control the audience to get them to watch, applaud , give a standing ovation, and even ‘boo” at the press of a button. Addition distractions include coughing noises and the sound of a mobile phone ringing. We also included curtains and stage lights to add to the sense of performance and frame the projected image against the studio wall.

How we created the simulation

The projection and interface for the audience and audition panel simulations are built in Adobe Flash, and use simple keyboard commands to move to different points in a movie loop to control the projected image. In order to keep it as simple to use as possible we decided that the computer screen and projector image would be mirrored – what you see on the computer is what is seen on the projected image. Although most computers can handle two screens, this is simpler to set up, easier to use on new computer / projector arrangements, and the user can be sure that what they see on the screen in front of them is what is projected to the performer. This is why we used keys to control the interface, as it allowed us to create easy-to-remember key combinations (e.g. ‘A, B , C’ for a excellent, average, poor reactions respectively, like exam grades) and keep the image free of visual cursor based ‘controls’ .

RCM-greenscreening

We needed to create a projection of a medium sized audience on a limited budget so we filmed one person at a time in front of a ‘green screen’. The green screen then allowed us to superimpose all the individual audience members in to a single composition. The theatre backdrop image was created digitally. This technique enabled us to use multiple versions of the same actor to increase the perceived size of the audience, while staying within budget.

In order to achieve the desired outcome we needed footage of all the individual actors performing the same tasks on the same timeline even though we were filming them separately. We did this by asking the actors to copy the strictly timed actions of an pre-flimed lead actor displayed on an iPad screen mounted beside the video camera. This meant that when we came to compile the composite image, all the actors performed appropriately at the same time, and where looking in the right direction.

We also filmed footage of an audience assembling for a concert from the perspective and in the style of a cctv camera to play to the performers in the green room, prior to them going out to perform. The idea behind this was that a significant cause of anxiety and nervousness in performance is being made to wait as the audience settles.

This short video explains some of the thinking behind the project and shows the audience in action.

Related Links

This project was followed up with a second commission for a simulated audition panel.

Read about this project on the RCM website: www.rcm.ac.uk/cps/simulator