Exploiting Spatial-redundancy of Image Sensor for Motion Robust rPPG
Wenjin Wang, Sander Stuijk, and Gerard de Haan, Eindhoven University of Technology, the Netherlands, Volume:62, Issue:2, Page(s): 415 – 425
(Video) Remote photoplethysmography (rPPG) techniques can measure cardiac activity by detecting pulse-induced color variations on human skin using an RGB camera. State-of-the-art rPPG methods are sensitive to subject body motions (e.g., motion-induced color distortions). This study proposes a novel framework to improve the motion robustness of rPPG. The basic idea of this work originates from the observation that a camera can simultaneously sample multiple skin regions in parallel, and each of them can be treated as an independent sensor for pulse measurement. The spatial-redundancy of an image sensor can thus be exploited to distinguish the pulse-signal from motion-induced noise. To this end, the pixel-based rPPG sensors are constructed to estimate a robust pulse-signal using motion-compensated pixel-to-pixel pulse extraction, spatial pruning, and temporal filtering. The evaluation of this strategy is not based on a full clinical trial, but on 36 challenging benchmark videos consisting of subjects that differ in gender, skin-types and performed motion-categories, i.e., subject motion challenge is emphasized in the dataset, which contains 6 types of head motion – stationary, translation, rotation, talking, scaling and mixture of all motions. Experimental results show that the proposed method improves the SNR of the state-of-the-art rPPG technique from 3.34dB to 6.76dB, and the Bland-Altman agreement with instantaneous reference pulse-rate from 55% to 80% correct. ANOVA with post-hoc comparison shows that the improvement on motion robustness is significant. The rPPG method developed in this study has a performance that is very close to that of the contact-based sensor under realistic situations, while its computational efficiency allows real-time processing on an off-the-shelf computer. Our research website link: http://www.es.ele.tue.nl/~dehaan/ http://www.es.ele.tue.nl/~sander/
Keywords: Biomedical monitoring, remote photoplethysmography, motion robustness.