Tenodesis Grasp Detection in Egocentric Video

Tenodesis Grasp Detection in Egocentric Video 1268 632 Journal of Biomedical and Health Informatics (JBHI)

Cervical spinal cord injury (cSCI) can impair motor function in the upper limbs. Video from wearable cameras (egocentric video) has the potential to provide monitoring of rehabilitation outcomes at home, but methods for automated analysis of this data are needed. Wrist flexion/extension is an essential element to track grasping strategies after cSCI, as it may reflect the use of the tenodesis grasp, a common compensatory strategy. However, there is no established method to evaluate wrist flexion/extension from egocentric video. Methods: We propose a machine learning-based approach consisting of three steps—hand detection, pose estimation, and arm orientation estimation—to estimate wrist angle data, leading to the detection of tenodesis grasp. Results: The hand detection in conjunction with the pose estimation algorithm correctly located wrist and index finger metacarpophalangeal coordinates in 63% and 76% of 15,319 annotated frames, respectively, extracted from egocentric videos of individuals with cSCI performing activities of daily living in a home simulation laboratory. The arm orientation algorithm had a mean absolute error of 2.76 +/- 0.39 degrees in 12,863 labeled frames. Using these estimates, the presence of a tenodesis grasp was correctly detected in 72% +/- 11% of frames in videos of 6 activities. Conclusion: The results provided a clear indication of which participants relied on tenodesis grasp and which did not. Significance: This paradigm provides the first method that can enable clinicians and researchers to monitor the use of the tenodesis grasp by individuals with cSCI at home, with implications for remote therapeutic guidance.