Laurent and Larry Bourgeois look more like cyborgs than humans when they dance. Their movements are impossible—their upper bodies gyrating independently of their lower extremities. Their moves are sporadic, easily mistaken as a video editing trick of starting, freezing, speeding up, and slowing down footage—a flow so mindboggling, it’s sometimes uncomfortable to watch. But the homemade videos populating YouTube are real-time. Their performance is an illusion they create both separately and together. Unnatural movements, one after the other, happen so quickly, and overlap so calculatedly, so fluidly. It’s difficult to make sense of them. It’s a magic trick and a synergy of the human body and mind that appears an absurdity of anatomy and physics. And they create these illusions together, playing off each other, improvising seamlessly.
The Parisian identical twin Hip-Hop duo, collectively known as Les Twins, became an international sensation in 2008 after dancing on the French talent show, Incroyable Talent. Like most good dancers, they are natural physicists, which makes them effective illusionists. Without labeling it as such, they calculate the forces of their extremities, and the potential of their muscles to create movement. Their choreography is an elegant equation—one that isn’t yet described in ones and zeroes.
Decoding this equation might be the key, scientists believe, to unraveling the mysteries of the human body. “Dance is the ultimate abstract movement,” said Dr. Susan Wiesner, a dancer and Digital Innovation Fellow at the University of Virginia, “if you can figure it out with dance, you can figure out anything.” Indeed, a recent surge in interest in the biomechanics of dance is already advancing our knowledge about the intricacies of our musculoskeletal systems, and enhancing our understanding of how we interpret movement. The number of potential applications are staggering—from medicine, to movies, to national defense.
The early days of dance biomechanics began in 1937 with Disney’s cartoon, Snow White and the Seven Dwarfs. The animators then lacked the ability to fabricate believable human movement, so they turned to dancers. Using real dance footage, they replicated and adapted each frame to their character, inspiring more than a half-century of animation.
About 50 years later, the late Merce Cunningham, a pioneering avant garde dancer and choreographer, incorporated motion capture into his choreography, both as a means for instruction, and as a way to explore how to make seemingly impossible movements possible. Using an early version of the software Life Forms, choreography software developed at Simon Fraser University in 1986, he choreographed his first piece—Trackers—using motion capture, and never looked back. The software allowed choreographers to compose movement sequences in space and time with single dancers or multiple dancers at one time, and experiment with each figure’s physical movement. “Life Forms is not revolutionizing dance, but expanding it,” he said, “because you see movement in a way that was always there, but wasn’t visible to the naked eye.” Since then, the software—now called Dance Forms—is used by choreographers worldwide.
But the dance landscape is changing as technology becomes more sophisticated. “In the last 15 years, dance has become more daring,” said Dr. Margaret Wilson, a dancer and dance biomechanist at the University of Wyoming. “The growing interest in using technology in dance has drawn technologists to dance, and then the technologists began asking their own questions.”
What Les Twins know instinctively is what biomechanics now seeks to define: how do you quantify aesthetics in a meaningful way? Wilson has been tackling this question for nearly a decade. Though dance has long embraced motion capture technology, motion capture technologists—those that operate and perform research using the technology—have long bypassed dancers to focus on sports medicine. Dance is just not that straight forward. In sports, the goals are simple—run faster, kick harder, throw farther—but there are more abstractions in dance.
“Dance analyzes movement in ways that human eyes can’t,” said Wilson, “very skilled dancers create the illusion that the body isn’t moving, but in fact their body is moving a lot more.” Balance alone is a dynamic thing. Every subtle movement requires a countermovement, which is difficult if not impossible to visualize on a TV screen. Furthermore, these movements and counter movements are happening in three-dimensional space, which requires multiple coordinated cameras to capture. Most sports only require one. But with multiples, a technique that was previously reserved for animation, video can be slowed to a snail’s pace, and an entire three-dimensional space created. Paired with sensors that measure muscle movement and wireless accelerometers the size of quarters, a scientist can visualize which muscles are used and exactly how much they engaged.
With such a setup, biomechanists are essentially “taking away the skin and just looking at the muscles,” said Wilson. “We are looking inside the body.” This informs dance instructors and dancers, as well as physical therapists, who now reference these studies to fine tune therapy. A point dancer, for instance, can now see exactly where pressure is being absorbed on the foot, and can tailor their movements to prevent injury.
This is exactly the kind of work biomechanist and physical therapist Dr. Shaw Bronner does at Northeastern University, in Boston, Massachusetts. Bronner found physical therapy after suffering a career-ending injury in the early 1990s as a modern dancer. She began treating dancers from Alvin Ailey and other contemporary dance companies that didn’t have budgets the size of wealthy ballet companies. Dance medicine was in its infancy then, and career-ending injuries like hers were commonplace—particularly in contemporary dance where dancers had little access to therapy. Bronner knew there had to be a better way to treat these dancers, and in 1998, suggested an in-house therapy program at Alvin Ailey American Dance Center. The program proved effective. Dancers in the company began to be treated earlier, and their injuries kept them sidelined for less time.
Around then, she developed a biomechanics lab with a motion capture system, and started looking at the motion patterns of the dancers she treated. What she found broke ground for new therapies for dancers, as well as inspired a new generation of dance biomechanists, including Margaret Wilson.
During her stint at Ailey, Bronner was approached by a group of hip-hop dancers concerned about the incidence of injury among them. Upon further investigation, she was appalled by the catastrophic injuries she was seeing, and the rates at which it was happening. In a web-based survey of 232 dancers, 738 injuries were reported that year—a 237% annual injury incidence. These rates were more like those of gymnasts, not dancers. And unlike other forms of dance, there were an inordinate number inflicted on the upper body—a testament to the style, which involves head spins, irregular balancing on arms and shoulders, and generally much stronger forces on joints—but there was no body of research documenting it. Most dancers lacked health insurance, which presented the challenge of funding physical therapy, and there was little to no education about injury prevention. Bronner took on the project. Utilizing the motion capture technology in her biomechanics lab, she identified problem areas, and techniques for injury prevention. Among those adopted by dancers: the use of dance-appropriate sneakers, padding in the baseball cap for head spins, and stretching before performances.
“Biomechanics is changing the way dancers are taught,” said Bronner. Dancers aim to create visually interesting illusions of forms that might look impossible to the average viewer—and without anatomically-friendly instruction, some of the movements the body is really not meant to do. Dance biomechanics shows them how, but safely. Her guiding question is, “how do we do this in a human-friendly way?”
But by unlocking the secrets to fine muscle movement, biomechanists like Bronner and Wilson have also invited a slew of questions from other fields. In orthopedics, for instance, motion capture has experienced a similar renaissance over the last decade. In both fields, a patient or dancer is decorated with an assortment of sensors while performing a task. Data from the sensors is then poured into a computer, and an image is generated that reveals a variety of factors occurring during movement—joint angles, accelerations, axel symmetries, joint torque or stress—and a diagnosis can be made to identify a patient’s weak points. For years, this technology has been used to assess and develop therapies for cerebral palsy patients with gait issues, but it’s all been done in a lab where patients can’t be expected to move naturally. That’s all changing as the technology improves. Even now, Wilson’s equipment is portable enough to bring to the studio. “More portable devices allow us to look at dance in its natural environment,” she said. It’s no longer a requirement for the dancer to even come to the lab.
Beyond medicine though, other researchers have sought out totally new, even unsettling applications of this technology, like Drs. Susan Weisner and Brad Bennett at the University of Virginia. Weisner, another former dancer and dance instructor as well as a computational linguist and librarian, studied text mining for her Ph.D. Sifting through hours of dance footage one day, she began to wonder if it might be possible to mine video for movement just as she mined text for words. She approached Bennett, the Research Director of the Motion Analysis and Motor Performance Laboratory at UVA, and they decided to give it a shot. Their collaboration became the ARTeFact project.
For the project, they decided to start with ballet, which has so-called codified movements—clearly defined movements like the plié, arabesque, and jeté—that are performed almost exactly the same way by every dancer. They plugged a dancer into the motion capture lab and began taking data, effectively turning each codified ballet movement into syntax, and plugging that into an algorithm that could recognize these movements in dance video footage just as one might search for a word on a webpage. At the very least, this saves the dance instructor loads of time sifting through footage to use in class.
But then Wiesner took it even further: dance, like writing, uses various combinations of syntax to create semantics—meaning behind movement. Dance can be angry, combative, happy, melancholy, all of which are expressed by combinations of codified movement. Wiesner wanted to expand the algorithm to sense emotion, but how? Recruiting a group of dancers, Wiesner had them watch hours of dance footage to define patterns of movement meant to convey an emotion. “In conflict,” she said, “when you surrender, your hands are facing up. When you attack, your hands are down.” The algorithm she is developing now will be able to identify not only specific movements, but interpret and categorize whole grammatical sentences of movements as emotions.
The same idea applied to orthopedics could be revolutionary—any video footage of a patient could be interpreted by a computer, without needing either cumbersome sensors or a lab. Pair that with ever-improving motion capture technology, and that gives a pretty clear picture of what exactly is going on in the musculoskeletal system of a patient. And it doesn’t stop there. Imagine having that technology following the Boston Marathon bombings. If a computer was capable of narrowing down video footage to a handful of people moving suspiciously, the bombers might have been identified much sooner.
But still, no computer can replicate the elegance of a dancer, or the intricacies of human movement. Technology is only beginning to interpret the fine movements more effectively, and even though it’s already making healthier dancers, still much remains a mystery. The most realistic-looking computer animation available—Avatar, the most innovative video games—still relies on real humans and motion capture technology to make movements believable. And of course, technology is still no match for the Bourgeois boys, whose appearance in a recent video game, Just Dance 2 for the Nintendo Wii, had to be copied down to the last detail, from their eye-popping moves, to their matching afros.