A fundamental challenge in soft-tissue surgery is that target tissue moves and deforms, becomes occluded by blood or other tissue, and is difficult to differentiate from surrounding tissue. To overcome these problems, we developed biocompatible near-infrared fluorescent (NIRF) markers with a novel 3D and NIR camera tracking system, enabling three-dimensional tracking of tools and target tissue while overcoming blood and tissue occlusion in the uncontrolled, rapidly changing surgical environment. These markers glow when excited by long-wavelength light, and are able to be easily seen by a special camera sensitive in this range. This allows us to easily differentiate our markers and track them, even when they are occluded by blood or tissue. This is highly useful for surgical applications where soft tissue structures need to be tracked to assist manual or robotic interventions. In this work, we present the tracking system and marker design and compare tracking accuracies to standard optical tracking methods using robotic experiments. We compare the performance to a commercially available Polaris optical tracker, which requires a line of sight to its special markers, and to a robot reference which knows its position at all times. When the Polaris markers are covered in blood or tissue, they are unable to be tracked. However, our special glowing markers are still able to be tracked even when occluded. At speeds of 1 mm/s, we observe tracking accuracies of 1.61 mm, degrading only to 1.71 mm when the markers are covered in blood and tissue. This technology has many applications, particularly in medicine, and can help overcome limitations when tracking tools in tissues in in-vivo settings.
Keywords: Imaging, biomarkers, plenoptic, measurement, robotics