Self Driving and Self Diagnosing

Self Driving and Self Diagnosing 617 374 IEEE Pulse

Advances in automobile technology have made headlines over the last few years, and not always for good reasons. In March 2018, an Uber self-driving car ran over a pedestrian in Arizona—the latest in a handful of casualties tied to automated car systems. Despite such tragedies, however, cutting-edge technologies in cars may prove beneficial for people’s health overall. In 2016, a driver who suffered a pulmonary embolism while on the road credited the autopilot features of his Tesla Model X with saving his life: the car managed to navigate the 20 miles to the nearest hospital during the incident.
As urban sprawl increases and cities grow denser, people will likely spend much more time in their cars. But advances in self-driving vehicles and other assistive technologies, coupled with the growing field of personalized health monitoring, may one day help cars promote driver and passenger well-being. Car companies are getting closer to this holy grail, but a few obstacles remain.

Promoting a Healthy Lifestyle

David Agus, M.D., professor of medicine and engineering at the University of Southern California (USC), believes that cars are an ideal place to unobtrusively gather data in a more useful way than the typical smart watch or health fitness wristband (Figure 1).

David Agus
FIGURE 1 David Agus, professor of
medicine and engineering at USC

“Most Americans spend many hours a day in the car, so it’s an amazing opportunity to collect and use data to improve driving and our lives,” says Agus, who is also the founding director and chief executive officer of the Lawrence J. Ellison Institute for Transformative Medicine at USC. “Smart watches hit a small portion of the population. Cars interact with almost everyone. You have a captive audience and the ability to generate data that’s useful—it’s the perfect storm.”
Much of our healthcare data are collected only during a doctor’s visit, Agus points out. But a car could pair biometric readings with environmental context to alleviate health issues. Aside from gathering more data, integrated health technologies in cars could allow the collection of better data. “The beauty of a car is that we can put the data into context, whereas most health tracking devices are lacking context,” says Agus.
For example, if the car measures a high reading for heart rate variability (a surrogate for stress) during heavy traffic, it could play relaxing music or design a route via its global positioning system that may be longer but less congested. Conversely, if the integrated biometrics can detect when a driver is tired, the car could automatically change the lighting, airflow, or music to be more stimulating. These kinds of fatigue-detection technologies could also set off alerts or suggest a coffee shop nearby if the driver is drowsy.
While such seamless detection and implementation sound good in theory, is such an integrated health-car system actually implementable? After testing the waters of health and vehicle technologies for some time, companies finally seem to be picking up steam. One in three new cars will include a form of biometric feature by 2025, according to a 2016 report by global research and consulting organization Frost & Sullivan.
“Almost all leading OEMs [original equipment manufacturers] including Ford, Jaguar, Audi, Volvo, Toyota, and BMW are working toward incorporating health and wellness technologies in their cars,” says Joe Praveen Vijaykumar, a Frost & Sullivan intelligent mobility industry analyst. “Heart rate monitoring, stress detection, respiration monitoring, fatigue detection, and massage functions are some of the key areas of focus and are most likely to develop. I see maximum impact of these technologies in avoiding road accidents attributed to fatigue and stress, especially in drivers who frequently commute long distances and commercial long-haul drivers.”

Audi Fit Driver
FIGURE 2 Systems like the Audi Fit Driver could use a driver’s wearable device to take biometrics and integrate the data into the car. (Figure courtesy of Audi.)

Most efforts car companies are making in the healthcare realm have a similar approach: detect the driver’s mood and respond accordingly. In 2015, Jaguar announced a research project, Mind Sense, that explored adding sensors into the company’s cars to read brainwaves and assess whether drivers are drowsy or distracted. Similarly, Audi’s Fit Driver platform envisions a driver using a wearable device (like a watch or wristband) to take vitals, including heart rate and skin temperature (Figure 2). That information would be integrated into vehicle sensor data—such as driving style, breathing rate, and weather or traffic conditions—to evaluate and adjust to the driver’s mental and emotional state, as detailed in a 2016 release.
And it’s not just luxury vehicles taking the plunge. In January 2016, Ford opened its Automotive Wearables Experience Lab to integrate wearable devices into vehicles to promote driver-assist technologies in response to the driver’s state of being. At the 2017 IEEE Body Sensor Networks Conference, the managing director of the Ford Research Center Aachen, Pim van der Jagt, Ph.D., described the company’s efforts to add an electrocardiography reader into the driver’s seat to help detect heart irregularities. Ford has also explored adding a dashboard glucose monitor readout that can display information from a wearable or app, as well as using cameras and other sensors to perform full-body check-ups, measuring body temperature, respiration, and heart rate.
Also, at the 2017 Consumer Electronics Show (CES), Mercedes showcased new features of its Fit & Healthy initiative, which aims to improve the health of its users while driving. Its “vitality coach” can suggest healthy actions for the driver, whose pulse rate is detected through sensors in the steering wheel. To promote relaxation, for instance, the car could release a scent or show an animation on the dashboard screen. During a CES panel discussion, Ola Källenius, a member of the executive board of Daimler AG, Group Research & Mercedes-Benz Cars Development, explained, “Beyond safety, we want to put vitality at the center and make sure you feel better when you get out of the car than when you got in.”
A consistent theme is that many of these efforts are turning to consumer wearables and other portable devices for guidance or integration. “Wearables will play a key role in health monitoring in the cars of the future. Increasingly, OEMs are looking at wearables as a feasible way to bring health technologies into the car,” says Vijaykumar. “In the future, OEMs will focus on providing a robust platform that will facilitate seamless integration of brought-in devices and secure connectivity to the cloud.”

Saving Lives

Beyond combating driver stress or fatigue, some car companies are looking to detect or predict deadly health events. If they could do so, a car could potentially ask the driver to pull over and send out a distress message to emergency responders or family contacts. But, first, such a system would need to accurately sense an adverse event.
Toyota’s Collaborative Safety Research Center (CSRC) was established in 2011 with the aim of accelerating automotive safety research. Last year, the company launched the CSRC-Next initiative, one goal of which is to improve driver state detection by collaborating with researchers across the country. The center is partnering with the University of Nebraska Medical Center to detect hypoglycemic events in diabetic drivers and the University of Michigan Center for Investigative Research in Critical Care to recognize cardiac events. “We have just launched these projects,” says Pujitha Gunaratne, Ph.D., a principle scientist with Toyota’s CSRC, “and are exploring the potential for future applications as we progress with viable outcomes.”

Kayvan Najarian
FIGURE 3 Kayvan Najarian, associate
professor of computational medicine
and bioinformatics at the University
of Michigan’s School of Medicine

A lead researcher in one of the Toyota collaborations, Kayvan Najarian, Ph.D., associate professor of computational medicine and bioinformatics at the University of Michigan’s School of Medicine, is evaluating the ability to detect cardiac events inside a vehicle in hopes of preventing related car accidents (Figure 3). Najarian studies systems that monitor the physiological state of patients outside the hospital and attempt to predict the likelihood of a cardiac event using wearable devices and artificial intelligence. “Wearable devices are often not as sophisticated as the devices used by hospitals,” says Najarian, who is also an associate professor of emergency medicine. “Additionally, the motion artifact (which can cause false positives) is the main cause of noise in data, which is typically minimal in hospital settings.”
The Toyota collaboration is a two-part project: the first step is to test different combinations of off-the-shelf wearable products and analyze the signals they collect (e.g., electrocardiogram). From there, the University of Michigan’s team is designing signal processing to account for the motion artifacts, extract characteristic patterns using these signal processing methods, and design machine-learning algorithms to accurately predict if a patient is having a cardiovascular event. “We want to see if our algorithms are at least comparable when used with devices in hospitals versus when we use these wearable devices,” says Najarian. “So far I would say, based on preliminary results for a subset of cardiovascular complications, we have really promising results.”
Najarian points out that, even with autonomous driving systems, cars often carry occupants in addition to a driver, any of whom may have a variety of medical conditions. The more medical monitoring can be integrated with future intelligent cars, the more a system can intervene and react quickly and efficiently to the possibility of a medical event for any of its occupants. “We have a lot more to do, but for the new algorithms we have developed for this project so far, it is likely that we can predict at least a subset of these cardiac events with high accuracy,” Najarian continues.

A smartphone-connected electrocardiogram device
FIGURE 4 A smartphone-connected electrocardiogram device developed by AliveCor to detect abnormal heart rhythms. (Photo courtesy of Alivecor.)

A host of new wearable technologies can benefit drivers even if they aren’t integrated directly into cars. AliveCor, based in Mountain View, California, and cited by Fast Company as among the most innovative companies of 2018, developed the first smartphone-connected electrocardiogram, a tiny device that accurately detects abnormal heart rhythms (Figure 4). Although the company has not announced any plans to partner with car makers, Agus points to AliveCor’s work as an example of refined specialized portable sensors that could potentially be integrated into cars of the future.

Custom glasses developed by Optalert
FIGURE 5 Custom glasses developed by
Optalert to measure a driver’s blinks and
to determine, for example, fatigue.
(Images courtesy of Optalert.)

Other companies are developing car-specific biometrics for health and safety. Sober Steering, based in Ontario, Canada, offers steering wheels equipped with biosensors that detect alcohol blood levels through gas emissions in the palm. Optalert, based in Australia, developed technology (Figure 5) to measure and evaluate the speed and rate of a driver’s blinks to determine fatigue and incorporated this technology into custom glasses that can measure a user’s blinks 500 times a second. Recently, the company has applied its algorithm to video camera technology that may be integrated into vehicles.


In addition to the need to refine and optimize hardware and sensors—both wearable and integrated—into cars, automobile companies that are looking to add health features face other limitations.
For one, regulatory processes for vehicles can take a very long time. For another, consumer backlash against companies collecting personal data could add a barrier: car companies would need to ensure the privacy of any health data that are collected. “There are strict regulations across the world pertaining to data collection, storage, and sharing with third parties, and OEMs have to ensure that they adhere to these regulations when they monitor the vitals of drivers,” says Vijaykumar. “And with sensitive user data involved, OEMs have to invest in robust cybersecurity systems to safeguard user information.”
Cost is another obstacle. The investment required to develop the technologies (such as sensor and microchip prices) has limited the availability of such features to luxury cars. But Vijaykumar believes that health and biometric features will start to appear in the affordable segment of the automobile market over the next decade. “Overall, the future is exciting for car-based health technologies where the car becomes a health center, monitoring your vitals every day and alerting you and your healthcare provider to potential abnormalities, thereby paving the way for preemptive treatments,” he explains. “These technologies will also make mobility of the future safer through fatigue and stress detection.”
That, coupled with the rise of automated features and self-driving cars, could mean a very different driving experience is on the horizon.