A range of remarkable prostheses are now available to give function back to people who have had hands, arms, feet, or legs amputated, but for all their capabilities, these devices are missing a critical feature: a real sense of touch. Without it, a patient has no tactile sensory feedback on whether they have stepped off a curb or onto a misplaced child’s toy, or are gripping a Styrofoam coffee cup or a toddler’s hand too tightly or too loosely. Research projects today, however, are coming closer to restoring the sense of touch, which will help prostheses evolve from amazing tools to true replacement body parts.
Using different tactics, including a new surgical approach to amputations, researchers are beginning to take steps forward to restore the tactile sensation to amputees and give them vastly improved mastery of their prostheses. “My dream is a person with a below-the-elbow amputation can play the piano at normal speeds, and a person with a leg amputation can dance in a completely normal way. I think we’re about a decade away from those extraordinary, very exciting demonstrations,” predicted Hugh Herr, Ph.D., who heads the Biomechatronics Group in the Center for Extreme Bionics at the Massachusetts Institute of Technology’s Media Lab. “There’s so much development and so much innovation going on now. It is indeed a very, very exciting time.”
Sensors, interfaces, and better control
Several research groups are working to incorporate touch into prostheses through the use of sophisticated machine–body interfaces that can bridge the gap between the body and the prosthesis, so that the body’s normal network of brain, nerves, muscles, and tendons can again control the extremity, even if it is artificial.
One is a University of Utah group that is part of a Defense Advanced Research Projects Agency (DARPA) initiative to help veterans who have lost an extremity during military service. Led by biomedical engineering associate professor Gregory Clark, the Utah team has the role of adding neural interface capability to the LUKE Arm [1], a high-tech prosthesis equipped with sophisticated motors and sensors, and a hand with individual digits. This is in contrast to the claw seen in more traditional prostheses, which the user typically operates with the feet: taps with one foot toggle through a menu to select an action, such as rotating the wrist or gripping the hand, and taps with the other foot to start or stop the action, according to Jacob George, National Science Foundation graduate research fellow of biomedical engineering and member of the Utah group (Figure 1). “It’s a nonintuitive and difficult process, so the person literally cannot walk and use his or her hands at the same time.”
To make the prosthesis easier to use, the Utah group spent hours mapping out the role of upper-arm nerves and muscles in hand movement and sensation. They did it by testing 200–300 electrodes (developed by Blackrock Microsystems, Salt Lake City, UT) implanted into individual nerves, and 32 electrodes (developed by Ripple Neuro LLC, Salt Lake City, UT) implanted into individual muscles. They asked subjects to report what they felt when each electrode stimulated the nerve. “A lot of people think that touch is one sense, but it’s actually comprised of many different submodalities, such as pressure, vibration, pain, and temperature. And there is also proprioception, which is perception of where your hand is in space,” George said. “We tapped into all of the individual nerve fibers that are conveying all of this information.”
Once they had that exhaustive map, the researchers were able to generate sophisticated artificial-intelligence algorithms in a computer interface that serve as a translator between the upper-arm nerves and muscles and the motors and sensors of the prosthesis. As a result, a patient can think about making a movement, the brain sends the directions down the nerves and muscles as usual, but instead of the signals going directly to the hand, they route to the external computer interface, which instantaneously interprets them and passes the instructions to the prosthesis, which then performs the desired action, George explained. “We are not quite to the point where the prosthesis moves like the hand does naturally, but what we’re able to do is give the patient more intuitive control over the prosthesis.”
The group has tested the system in seven amputees, including 17 months with the most recent participant, and found that it has both functional [2] and psychological benefits [3]. Functional advantages range from improved dexterity and proportional control to the ability to discriminate objects by size, softness, or hardness (Figure 2), he said. “On the psychological or emotional side, benefits include things like reductions in phantom pain because we’re able to tap into the nerves and recreate the signal that you would have normally had before the amputation; and also embodiment, which is the ability of the person to think that the prosthesis as an actual part of his or her body and meaningfully incorporate it into their body image.” As an example, he recounted that patients switch from talking about “my prosthetic hand” to “my hand” when they are afforded the tactile sensation. He remarked, “That resonates as a personal embodiment of the device—it’s no longer just a tool but it’s actually become their hand.”
The Utah research group is now starting at-home trials of the system [4]. “This is by far the biggest challenge for our group, because it’s taking this system that is normally run in the lab by three graduate-level engineers, and trying to simplify it to a single program that can not only run on an iPad, but that can also be run by a lay individual, typically one who doesn’t have hands.” In addition, the researchers are finalizing designs for wireless versions of the system.
Another group that is using machine–body interfaces to connect the body’s communication network to a prosthesis is that of Staniša Raspopović, Ph.D. assistant professor of neuroengineering at the Swiss Federal Institute of Technology (or Eidgenössische Technische Hochschule, ETH) in Zurich, Switzerland. One of his group’s main contributions has been in restoring sensory feedback to people who have had above-the-knee or transfemoral amputations (Figure 3).
Raspopović and his team implanted minute transversal intraneural electrodes in nerves of the residual limb and gathered patient feedback to inform computer algorithms and an interface to allow the nerves and the prosthesis to communicate. Raspopović’s solution had several components. One was a sensor-studded insole, which he and his ETH group designed to pick up measures, especially pressure, in the foot. An important aspect of the insole, which is placed beneath the prosthetic foot, is that it can be added to a wide range of prosthetic models, he said. “This was really one of the constraints for our design and our implementation: We wanted to make something that was available to the users having super-fancy prostheses as well as those having basic prostheses.”
To link the insole to the residual nerve network, the researchers developed an interface—a wireless system controller—to capture pressure information from the insole. They also worked with a prosthesis manufacturer to encode data from already-existing inertial sensors in the knee of the prosthesis and send that to the system controller as well. To connect with the residual nerves, the controller, who receives signals from the prosthetic sensors, sends commands to a tiny neurostimulator that then transmits signals to the residual nerves.
Lab and clinical tests of the system (Figure 4) showed that the sensory-enabled system provided such enhanced sensation from the prosthetic foot and knee that patients experienced greatly diminished or eliminated phantom pain, and better mobility, including much improved walking, even in challenging situations such as on stairs, in deep sand, and over obstacles [5], [6]. “And embodiment was remarkably better when the sensory feedback was restored,” Raspopović added.
A spinoff company, called SensArs Neuroprosthetics of Lausanne, Switzerland, is now working to eliminate the last remaining nonwireless component of the system, the external cable between the neurostimulator and electrodes, and is planning to have a fully wireless system in place within two to four years, according to Raspopović. Clinical trials are slated to begin in 2021, with commercial production following shortly thereafter.
Reflecting on the advances the Utah group and others are making toward restoring touch, George remarked, “Everyone relates this work back to the Star Wars movie where Luke Skywalker’s arm gets cut off in a light-saber duel, but he’s able to have a bionic arm attached to him and it functions just like he has his own hand back again. It’s always been kind of a sci-fi dream to be able to do something like that, and now we’re at the point where it’s actually becoming a reality.”
Advanced surgery and the natural feedback loop
For Herr (Figure 5), the path toward the restoration of touch should begin with the amputation surgery itself. “You have this precipitous improvement in technology of the limb prostheses with microprocessors, sensors, materials, and actuators, but largely the way limbs are amputated—the standard of care—hasn’t changed since the Civil War. It’s outrageous,” he remarked. “What we are doing is modernizing how limbs are amputated using state-of-the-art surgical and regenerative techniques.”
To understand the need for the surgery, he began by explaining a few basics. When moving an intact limb, the brain sends signals that contract one muscle and simultaneously stretch another, and this agonist–antagonist arrangement pulls tendons that control the limb action. At the same time, natural sensors or mechanoreceptors in the muscles and tendons report on the limb’s activity, and the brain responds by sending additional signals to refine that activity.
In traditional amputation surgery, Herr said, all the tissue below the amputation site is discarded and a technique called myodesis is used to secure the residual muscle (above the amputation site) to the bone in the remaining limb. This severs the agonist–antagonist interplay, so muscles can no longer communicate normal proprioceptive information to the central nervous system.
Herr worked closely with surgeon Matthew Carty of Brigham and Women’s Hospital to devise a new surgical strategy [7] that would reconnect the agonist–antagonist muscles in the residual upper limb in such a way that they would get the same push–pull sensation they did when they were connected to the lower leg (specifically the ankle and the subtalar foot joint). The new surgery, called the “Ewing amputation,” also preserves the wealth of mechanoreceptors in the muscles and tendons as well as the cutaneous mechanoreceptors, in the residual limb. In other words, he said, the Ewing amputation
technique retains the natural feedback loop that existed prior to the amputation, providing linkup points for prostheses Herr’s lab have specially designed for that purpose. “So instead of artificially stimulating nerves for a feedback to the brain for sensation, we’re putting natural biological sensors in the control loop,” he explained, noting that this provides the brain with information about force, length, and speed of each of the muscles engaged in the agonist–antagonist interaction occurring during a limb movement.
As of early January 2020, Herr reported that more than 20 patients at Brigham and Women’s Hospital had undergone the Ewing technique for below- and above-the-knee amputations, and all were showing “superior control” over their bionic prostheses. By using the natural feedback loop, he said, patients have volitional multidegree of freedom of control to move several joints at once (such as leg, knee, and ankle during stair-climbing) and fine-tune capability, so patients can move when and how they want, and feel as if they have their normal limb back. “We are continuing to introduce more patients, while continuing also to test the patients that we have,” he said. “Currently, Walter Reed Army Medical Center (in Bethesda, MD) has begun exploring these surgical techniques as well, which is very, very exciting.”
As that proceeds, Herr’s group is working on prostheses that interface more effectively with the new surgical paradigm and give the user even better control, and anticipates seeing some of those products becoming available in as little as five years. In addition, his group is now implanting small magnetic beads into muscles, and using an array of external sensors on the skin to track the movement of those beads as a way to obtain robust measures of muscle length and speed, which can then be used to control synthetic motors in the prosthetic limb. By combining this with the novel surgical strategy, he said, “a person will have a direct link-up to the bionic limb, so they’ll be able to think and with high fidelity directly control the synthetic joint as if it’s their own limb, and do it at normal speeds and with normal levels of accuracy.”
In another project, Herr’s group is growing cutaneous nerves in skin cells, wrapping the skin cells with a muscle, and then artificially stimulating and controlling the muscle to restore a sense of touch. “In a robotic finger, for instance, as it touches the coffee mug, we would sense that pressure using artificial sensors and then proportionally stimulate the muscle inside the body to press on the skin cells so the person feels the cup. So again, it’s a take on the same theme: We’re using biological mechanoreceptors in the feedback loop, as opposed to directly stimulating the nerves artificially and trying to create sensation,” Herr said, noting that his group hopes to publish this work later in 2020, and has already submitted a grant proposal to implement the technique in humans within the next three years.
All of these projects share the same core concept, Herr said, “We’re doing neural embodied design, and coadapting the biology with the synthetics to enhance neural communication.” He added, “I believe strongly that this is the winning strategy that will give patients extraordinary capabilities.”
References
- “LUKE arm details,” Mobius Bionics, Manchester, NH, USA. Accessed: Jan. 13, 2020. [Online]. Available: https://www.mobiusbionics.com/luke-arm/
- J. A. George et al., “Biomimetic sensory feedback through peripheral nerve stimulation improves dexterous use of a bionic hand,” Sci. Robot., vol. 4, no. 32, p. eaax2352, Jul. 24, 2019.
- D. M. Page et al., “Motor control and sensory feedback enhance prosthesis embodiment and reduce phantom pain after long-term hand amputation,” Front. Hum. Neurosci., vol. 12, no. 352, Sep. 21, 2018. [Online]. Available: https://www.frontiersin.org/articles/10.3389/fnhum.2018.00352/full
- J. A. George, T. S. Davis, M. R. Brinton, and G. A. Clark, “Intuitive neuromyoelectric control of a dexterous bionic arm using a modified Kalman filter,” J. Neurosci. Methods, vol. 330, no. 32, Jan. 15, 2020. [Online]. Available: https://www.sciencedirect.com/journal/journal-of-neuroscience-methods/vol/330/suppl/C
- F. M. Petrini et al., “Enhancing functional abilities and cognitive integration of the lower limb prosthesis,” Sci. Transl. Med., vol. 11, no. 512, p. eaav8939, Oct. 2, 2019.
- F. M. Petrini et al., “Sensory feedback restoration in leg amputees improves walking speed, metabolic cost and phantom pain,” Nature Med., vol. 25, no. 9, pp. 1356–1363, Sep. 2019.
- T. R. Clites, H. M. Herr, S. S. Srinivasan, A. N. Zorzos, and M. J. Carty, “The Ewing amputation: The first human implementation of the agonist–antagonist myoneural interface,” Plast. Reconstr. Surg.: Global Open, vol. 6, no. 11, p. e1997, Nov. 16, 2018.
Leslie Mertz (lmertz@nasw.org) is a freelance science, medical, and technical writer, author, and educator living in Northern Michigan.