Above: RAVEN on the set of Ender’s Game, portraying a teleoperated robot on an orbiting station.
In the recent movie adaptation of Orson Scott Card’s book Ender’s Game, a key character undergoes emergency brain surgery in an attempt to save his life after suffering a traumatic head injury in a space station. In this orbiting operating room of the future, no human surgeons are in sight. Instead, a four-armed surgical robot executes the delicate procedure, controlled remotely by ground-based surgeons. By the time the movie was released, NASA and Applied Dexterity, a medical robotics company spun out from the University of Washington, were already exploring sending a modified RAVEN™, the same robot that played a cameo in the movie, up to the ISS. So, what is RAVEN? How will it be used in the ISS? And, how will these developments impact healthcare and engineering progress on Earth?
The Birth of RAVEN
RAVEN is a third generation surgical robotics research platform, originally developed with DOD funding to demonstrate a lightweight field deployable surgical robot that could be teleoperated by surgeons remote from the battlefield. RAVEN’s genesis was in a decade of research at the University of Washington aimed at the application of mathematical modeling and technology to surgery. A series of experiments were designed to teach a computer model to recognize the motions of expert laparoscopic surgeons. Such a software system can be used in surgical training to recognize levels of surgical proficiency. Tools for minimally invasive surgery (MIS) were modified to incorporate a range of measurement instruments. Then these tools, along with systems including Blue and Red Dragon, were used to record a stream of dynamic and kinematic data from cadaver and animal surgeries. These data were then encapsulated in multi-state statistical model, using a Hidden Markov Model (HMM) approach. HMMs are used frequently in speech recognition software. In this case the software was used to recognize the “language of surgery” with a vocabulary and grammar of its own.
A valuable side benefit of this work was an extremely detailed database of force, torque, and range of motion values from laparoscopic surgeries. From the start, RAVEN’s mechanical design was optimized to deliver those same forces, torques, and range of motion, while minimizing size and weight of the mechanism and maximizing stiffness. Furthermore, the arm links and the base configuration were designed to incorporate either two or four arms, all sharing a common surgical site, while eliminating self-collisions. In that way, two surgeons can collaborate seamlessly.
RAVEN’s Challenging Childhood
Conceived as a small, light, field-deployable surgical robot, the next stage in RAVEN’s development was to not only to demonstrate its suitability for surgery, but also to challenge the very definition of “operating room.” RAVEN’s ruggedness, portability, and reliability have been demonstrated in live porcine surgery (cholecystectomy) as well as a series of tests in remote challenging environments.
In the US Army sponsored HAPs/MRT demonstration (High Altitude Platform/Mobile Telesurgery, 2005), RAVEN was set up in a tent in the desert (Simi Valley, CA) powered by a gasoline generator. A surgeon performed simulated surgery by teleoperation from a distant tent. Control signals were relayed via an autonomous drone circling above the surgical site. To accommodate limited communication bandwidth, the frequency of sending position updates from the controller to the robot was reduced from 1000 times per second to 100 Hz. The video signal was compressed to 800 kb/s, giving some increases in pixilation, but surgical performance was not hampered.
In a separate test, as part of NASA’s NEEMO-12 mission (NASA Extreme Environment Mission Operations, 2006), sections of RAVEN were transported in dive bags to the Aquarius underwater habitat, 3.5 Km off Key Largo on the ocean bed at a depth of 19 meters. Following two days of training on system assembly, startup, and disassembly tasks, NASA aquanauts successfully assembled and commissioned RAVEN in the habitat. The system was then teleoperated from Seattle, 3000 miles away, to perform a variety of surgical skills tasks. These tasks were completed while experiencing communication delays of about one second, approximating communication delays to the ISS. In the vision of the future presented in Ender’s Game, communication delays aren’t an issue; the ansible allows signals to travel instantaneously, not constrained by the speed of light.
RAVEN Goes to College
Recognizing RAVEN’s value as a test bed for surgical robotics advances, units are now in use at twelve leading medical robotics research institutions. With funding from the NSF, the initial RAVEN design was updated for more compact volume, modernized electronics, and improved software and software development environment. RAVEN IIs were initially placed at seven centers. Since that time, five additional sites have purchased RAVENs to join the growing community of RAVEN users.
RAVEN users are advancing the state of the art in surgical robotics with work in a range of fields including cooperative teleoperation, haptics, virtual fixtures, computer vision, machine learning, and autonomy. The RAVEN II Surgical Research Platform, currently marketed by Applied Dexterity, is controlled by open source software released through the Limited Gnu Public License (LGPL), providing researchers with full access to the source code. The software environment is further enhanced through integration with the widely adopted Robotics Operating System (ROS) open-source robotics middleware layer.
Overview of CITRIS collaboration to provide increases in surgical robot autonomy.
RAVEN Gets a Job
Now, with commissioning of the Rodent Habitat (see “Of Mice and Men” by Michael S. Roberts), NASA is embarking on a series of rodent experiments that will require the recovery of specific tissues from euthanized animals. Initially, astronauts will carry out these dissections manually. However, crew time on the ISS is an extremely precious resource, so a modified RAVEN, installed on the ISS and teleoperated from the ground, will be used to perform the rodent dissections as a way to better leverage that resource. In addition to buying back crew time, the installation will provide dramatic demonstration of long distance surgical teleoperation and enable a range of complex procedures.
Once an astronaut has snapped the relevant tools to the robotic arms and placed a euthanized rodent within reach, experts, seated in ergonomic control stations, will perform the remainder of the dissection tasks from ground control. In addition to dissections, the system can also be used for other experimental procedures requiring fine dexterity.
RAVEN Comes Home
The use of RAVEN on the ISS will stimulate wide-ranging benefits back at home, both in advances in technology and in improving the delivery of healthcare. This type of long distance teleoperation demonstration highlights the possibility of using robotic technology to assist in delivering medical care in a range of remote environments ranging from the battlefield to other underserved areas. The key point being that the surgeon does not need to be in the same room as the patient for successful outcomes. An orbital system also stimulates further improvements in reliability and additional reductions in size and weight. The system is already providing a proving ground for advances in computer-aided surgery.
The presence of a RAVEN on the ISS, that is software compatible with systems located at a large number of key research institutions, provides an inspiring development target for ongoing advancements in both software and hardware. The existing RAVEN centers, collaboratively sharing results, will be well equipped to provide practical advances, directly increasing the utility of the RAVEN onboard the ISS, ultimately translating into increasingly sophisticated systems for use in the clinic. This open innovation model, facilitated by open source software, also stimulates dialog challenging current FDA guidelines for the approval of medical device software.
Open Source Software for Safety Critical and Medical Devices
Open source software is developed in a very different way than the traditional “Waterfall” manner typical of medical device software. In “Waterfall” software development, a very detailed requirements document is crafted for the end product and software development is managed against a series of code development and testing milestones. The regulatory process places an emphasis on documenting all steps in this process as well as all changes to the initial requirements or specifications. In contrast, open source software development is usually incremental, self documenting (for example through tools like Doxygen), and changes are vetted though developer discussion forums or on-line collaborative code review tools such as Gerrit.
Although the open source process can seem chaotic, the key potential advantage of open source from the point of view of safety and security is its transparency, allowing many eyes to visit the code, critiquing it from multiple perspectives. Another key advantage is cost. If designers of a safety critical system need a capability (for example an image processing library) then adopting a widely used and actively developed/maintained open source library is arguably less risky than writing one from scratch with limited resources.
Regulatory bodies have taken a traditional approach to the certification of software in safety critical systems. Rather than formally auditing software functionality, they try to ensure that software developers have thoroughly followed and documented their Waterfall process. Open source software has been used in mission critical non-medical projects such as the JPL Mars Rover (Norris, 2004).
In Medical software, it will take time for society to adapt its regulatory process to incorporate open source development practices with high confidence that the overall goal of safe medical devices is preserved. Nevertheless, several organizations are developing and deploying open source software for medical use. OsiriX is a medical image (DICOM) viewer for the Mac. OsiriX is open source (LGPL), but a version approved by the US FDA (Class II) is available for sale (Rosset et al., 2004). In a similar vein, Image Guided Surgery Tool Kit (IGSTK) is an open source toolkit produced via an agile, iterative and open source development process (Gary et al., 2011).
As software directly controls a variety of devices, many proprietary interfaces become involved. The Medical Device Dongle (Asare et al., 2012) is an open source hardware accessory, which connects proprietary device interfaces to an open network protocol. Lee at U. Pennsylvania, has developed open source software for a benchmark pulse oximeter design and coordinated their efforts with the FDA (King et al. 2010).
The RAVEN II Surgical Research Platform, marketed by Applied Dexterity, is controlled by open source software released through the LGPL. Full source code from the University of Washington is available and managed on github. Through efforts currently underway, code from eleven other Raven-II sites will be merged with this code-base to create a definitive Raven-II release. Documentation for this software version is created with Doxygen. An up-to-date version is available here.
Summary
A surgical robotics facility on the ISS, based on a widely used platform, supported by an open innovation community, provides a way to increase the efficiency of research on the ISS, better leveraging the efforts of flight crew, and enabling new types of research requiring small scale dexterous manipulation. In addition, this capability will drive advances in surgical robotics that will be directly relevant to improving healthcare on Earth.
For Further Reading
- Applied Dexterity Inc.
- Raven II Wiki
- Raven II Developer Blog
- Kinematic Model
- IEEE Trans BME paper: 2013
- IEEE Spectrum 2006
References
- Asare, Philip, et al. “The medical device dongle: an open-source standards-based platform for interoperable medical device connectivity.” Proceedings of the 2nd ACM SIGHIT International Health Informatics Symposium. ACM, 2012.
- Gary, Kevin, et al. “Agile methods for open source safety-critical software.” Software: Practice and Experience 41.9 (2011): 945-962.
- King, Andrew, et al. “Prototyping closed loop physiologic control with the medical device coordination framework.” Proceedings of the 2010 ICSE Workshop on Software Engineering in Health Care. ACM, 2010.
- Norris, Jeffrey S. “Mission-critical development with open source software: Lessons learned.” Software, IEEE 21.1 (2004): 42-49.
- Rosset, Antoine, Luca Spadola, and Osman Ratib. “OsiriX: an open-source software for navigating in multidimensional DICOM images.” Journal of Digital Imaging. 17.3 (2004): 205-216.