Inverse Reinforcement Learning Intra-operative Path Planning for Steerable Needle

Inverse Reinforcement Learning Intra-operative Path Planning for Steerable Needle IEEE EMBS

Over the last two decades, different research groups have focused their efforts on the development of needles able to steer inside the tissue. These needles can perform curvilinear trajectories planned to maximize the distance from sensitive anatomical structures to be avoided and reach targets otherwise inaccessible via rectilinear insertion paths.
This work presents a safe and effective keyhole neurosurgery intra-operative planning framework for flexible neurosurgical robots. To achieve this goal, we present a system that integrates inverse reinforcement learning (IRL) path planning algorithm combined with 1) a pre-operative path planning framework for fast and intuitive user interaction, 2) a realistic, time-bounded simulator based on Position-based Dynamics (PBD) simulation that mocks brain deformations due to catheter insertion and 3) a simulated robotic system.

This study investigates IRL methods for intraoperative re-planning where the advantage is that IRL realises sequential decision making under uncertainties through end-to-end learning without explicitely for modelling the system uncertainty. With the introduction of manual demonstration with the IRL approach, human expertise is embedded in the planning system. As a case study, we integrate the proposed planning models into a simulated framework that consists of a Programmable Bevel Tip Needle. Simulation results performed on a human brain dataset show that the IRL intra-operative planning method can guide a steerable needle with bounded curvature to a predefined target pose with an average targeting error of 1.34 mm in position and 3.16 degrees in orientation under a deformable simulated environment, with a re-planning time of 0.02 sec. The fact our intra-operative path planning framework can be efficiently integrated into a robotic system for needle steering provides that the algorithm can work in a practical clinical scenario. This work can be used to develop a virtual training system and assist the surgeon in planning intra-operative paths.