Researchers at the University of Southern California (USC) have designed an interface for non-musicians to play music. This interface, part of the Expression Synthesis Project (ESP), is based on the fact that more people know how to drive a car than an orchestra. In "Baby, you can drive my song," the researchers explain how they converted real musical scores into digital virtual roads. Then using a steering wheel and foot pedals, you 'drive' on this road to interpret the piece of music, becoming a real maestro. Such a system should be demonstrated in a public exhibit by 2008 and become available to everyone in the same time frame. Read more.
Here are some details about the ESP project, devised by a team led by Elaine Chew of the USC Viterbi School of Engineering.
ESP "attempts to provide a driving interface for musical expression," according to Chew’s published description. "The premise of ESP is that driving serves as an effective metaphor for expressive music performance. Not everyone can play an instrument but almost anyone can drive a car. By using a familiar interface, ESP aims to provide a compelling metaphor for expressive performance so as to make high-level expressive decisions accessible to non-experts."
Created by Chew, Alexandre R. J. François, a research professor in the Viterbi School, and graduate students Jie Liu and Aaron Yang, ESP starts with a piece of music in the Musical Instrument Digital Interface (MIDI) format, one that has been converted from the printed score.
Below is a diagram showing how the system works, from a real musical score to a virtual digital road, and then from this road to real music played by you (Credit: USC Viterbi School of Engineering).
This image comes from this document about the Expression Synthesis Project (PDF format, 2 pages, 658 KB).
Of course, the difficult part is to convert a real musical score into a digital road.
My aversion from music rests on political grounds.
—Thomas Mann (1875–1955)
The group is building tools to automate the process of creating such roads, applying artificial intelligence techniques to the analysis of the score. "Having the road build itself will be the most difficult part," says François.
The road’s turns suggest to the driver when to slow down and speed up. however, the ultimate decision on what to do at each turn is entirely in the driver's hands (or foot). The foot pedals control both the tempo and the volume of the music. Additionally, buttons mounted on the wheel act as the equivalent of the pedals on the piano, making the notes either sustain or cut off crisply.
This research work was presented at the 2005 International Conference on New Interfaces for Musical Expression (NIME), held on May 26-28 in Vancouver, Canada.
Here is a link to the paper which was presented at this conference, "ESP: A Driving Interface for Expression Synthesis" (PDF format, 4 pages, 289 KB).
You can also find more information about this project by visiting the Music Computation and Cognition website (but it appears that some links are broken right now) or the USC Integrated Media Systems Center (IMSC).
Finally, on this poster about the project (PDF format, 1 page, 439 KB), you'll read that the goal is to have an interactive public exhibit in 2008.
Ready to drive an orchestra?
I, who had heard of music in the spheres,
But not of speech in stars, began to muse:
But turning to my God, whose ministers
The stars and all things are; If I refuse,
Dread Lord, said I, so oft my good;
Then I refuse not ev’n with blood
To wash away my stubborn thought:
For I will do or suffer what I ought.
—George Herbert (1593–1633)
Sources: USC Viterbi School of Engineering news release, May 30, 2005; and various websites
Related stories can be found in the following categories.
Engineering
Human Computer Interface
Innovation
Music.