Kamyar Abhari, John S.H. Baxter, Elvis C.S. Chen, Ali R. Khan, Terry M. Peters, Sandrine de Ribaupierre, and Roy Eagleson, Robarts Research Institute, Imaging Laboratory, London, ON, Canada,
Volume: 62, Issue: 6, Page(s): 1466 – 1477
Our paper describes the design and evaluation of an Augmented Virtual Reality system, to be used for training neurosurgical procedures that involve perception of anatomical targets (such as tumours) and control of targeting. Planning a surgical intervention based on imaging and prior anatomical knowledge also requires spatial reasoning to coordinate between the preoperatively acquired images and the patient’s head in the operating room. Our studies compared different visual display modalities (2D ortho-planar and cross-planar views, 3D graphics, and Augmented Reality) in order to investigate how image-based clinical targeting can be facilitated. The platform consists of a Polaris optical tracker, which tracks the coordinate frames, established by reflective fiducials, fixed with respect to three objects: the surgical tool, the Vuzix display glasses, and the model of the patient’s head. The view provided to the clinician through the Vuzix glasses display is a video stream, whose source is taken from the miniature cameras that are front-facing on the glasses. The video stream is augmented using 3D graphics to ‘paint’ pixels on the video stream in order to create an augmented reality scene, with compelling results: As the viewer moves their head, they perceive a virtual world in which the internal anatomical structures (rendered by the GPU on the basis of segmented 3D clinical scans) are visible ‘within’ the head. Our results demonstrate that, compared to conventional environments (2D images in the axial, coronal and sagittal planes), the proposed system greatly improves the novice users’ performance (p<0.01), as well as the time required for the experts to perform the task. Our paper details the derivation of our objective task performance metric which respects the tradeoff between speed and accuracy.