A Robot Conductor
Takes the Stage

The baton came up gracefully for the upbeats and descended forcefully for the downbeats, while the other hand danced along in time, coaxing the musicians from one crescendo to another, the singers from one emotion to another. The musicians were members of the Lucca Philharmonic orchestra;the singers included famed tenor Andrea Bocelli.

The conductor was a robot.

YuMi is its name and conducting Verdi in Pisa was its debut on the big stage. In fact, it was the first time any robot had directed such an acclaimed orchestra. But YuMi wasn’t originally designed to be the first Robotscanini. A product of ABB, aSwitzerland-based engineering firm focused on control systems and other digital technologies, the machine was created to learn gestures and tasks quickly, without the need for the user to know a programming language. It can alsobe put to use in almost any kind of industry. In front of an orchestra, YuMidemonstrated both the range of its potential uses and the ease with which it can be taught.

While even a mediocre conductor spends decades working in music, YuMi learned the three pieces it was to conduct, along with every gesture to every measure, in a matter of days.

“In the first step, the robot was guided through the motions of the conductor,” says Andrea Cassoni, ABB’s robotics manager in Italy. “This is an actual type of robot programming used in industry called ‘lead-through programming,’ where the robot is taught the required movements by demonstration rather than programming each individual point of motion in software.”

The person doing the leading in this case was actual flesh-and-blood maestroAndrea Colombini. Listening to the orchestra, he guided YuMi through each section. As if leading a tango, Colombini took YuMi in a near embrace, guiding its baton-holding hand through the music in one pass and then doing the same for the left hand in another pass.

Once YuMi had learned all the necessary moves, they were imported into an ABB program called RobotStudio. The software allows users—typically manufacturers—to refine the robots’ movements by tweaking a virtual model. In this case, RobotStudio was used to smooth out and finesse every movement, as well as to synchronize those movements with the score. This was, by far, the simpler part of the project.

“The biggest challenge was translating the subtle but wide range of movements necessary to conduct an orchestra into the robot’s programming,” Cassoni says. “This took some practice and close partnership between the conductor and robot engineer. But no out-of-the-ordinary software or tools were required.”

The orchestra musicians followed YuMi perfectly and were awed by its grace and seeming musicality. After the performance, Colombini raved about the machine’s fluidity. Arguably, a conducting robot could someday put people like him out of a job, or at least free them from attending every rehearsal. On the other hand, perhaps the conducting robot will increase demand for human conductors. Amateur, robot-run orchestras will need someone to program all the pieces the machines might conduct. Currently, though, an amateur orchestra hoping to follow an automaton, and anyone else who wants to put YuMi to use, will have to scrape together $40,000.

The YuMi that performed was no more sensitive than an animatronic character at Disneyland. From the first beat to the last note, the robot ran through its program and the musicians followed along. But there’s no reason that it couldn’t some day be outfitted with a sound sensor so that it could exaggerate a movement if the musicians weren’t reaching a certain decibelor hitting a predetermined tempo. At the moment,ABB has no intentions of refining YuMi’s conducting capabilities.

“Robots continue to capture the imagination of artists, the media, and the general public,” says Cassoni. “So you never know.”

Michael Abrams is an independent writer.

For Further Discussion