Graphics and Simulation in Laparoscopic Medicine
Companies and Products
- Simulab. Physical models used for training. Uses real instruments and physical (rubber and plastic) props.
- Haptica. ProMIS (minimally invasive surgery) uses real, tracked instruments (probably magnetic, based on video). Three setups, fully virtual, and using physical props with or without annotations on superimposed on display.
- Virtual system is used for early training (hold position, track target position, touch point w/o touching surrounding tissue). Nice modeling and rendering of tissue.
- Physical system is used as with classic prop training.
- Augmented display provides instructions and annotations super-imposed on the actual video. The camera appears to be fixed, allowing hard-coding of screen-space coordinates for data overlays. Bleeding appears to be handled as an overlay, just drawing on-top of video.
- Immersion's LapSim. (image) Fixed training workbench, not real tools. Haptic feedback. Appears to do (crude) 3D instruments and simulated suture overlayed on video.
- Simbionix's LAP Mentor. Entirely virtual. Not real instruments. Very nice art and rendering, good instructional focus. Good physical interaction (needle through tissue). Demo suturing video shows several nice tricks, like swapping in a fixed knot when tying conditions are met.
Papers - Physical Simulation of Surgery
- Toward Modeling of a Suturing Task. Matt LeDuc, Shahram Payandeh, John Dill. Simon Fraser University. 2003, Graphics Interface.
In this paper we present our initial work on simulating suture and suturing using mass-spring models. Various models for simulating suture were studied, and a simple linear mass-spring model of the suture was determined to give good performance. A novel model for pulling a suture through a deformable surface is presented. By connecting two separate surfaces through the suture, our model can simulate a suturing task. The results are shown using software we developed that runs on a standard PC and models the action of a suturing device used in minimally invasive Laparoscopic surgery. - A Suture Model for Surgical Simulation. J Lenoir, P Meseure, L Grisoni, C Chaillou - 2nd International Symposium on Medical Simulation (ISMS 2004).
In this paper, we propose a surgical thread model in order for surgeons to practice a suturing task. We first model the thread as a spline animated by continuous mechanics. The suture is simulated via so-called ldquosliding pointrdquo constraints, which allow the spline to move freely while constrained to pass through specific piercing points. The direction of the spline at these points can also be imposed. Moreover, to enhance realism, an adapted model of friction is proposed, which allows the thread to remain fixed at the piercing point or slides through it. Our model yields to good results showing realistic behavior, robust computation and interactive rates. - Suturing for Surface Meshes. Marshall, P. Payandeh, S. Dill, J. Control Applications 2005. (Project page at SFU)
We present a suturing simulator for surgery operating on spring-mass surface meshes. The rope model we use for suturing is a geometry-based model which includes the capability to form a knot. We extend a previous suture model to maintain a consistent flexibility when conflicting constraints cause the suture to stretch. The suture, which is attached to a needle, can be manipulated by the user to stitch a virtual cut on a surface. When the needle pierces the deformable model, the surface mesh is subdivided at the contact point using a novel subdivision algorithm. During suturing, the simulation of the pierced mesh differs depending on whether it is pierced by the needle, or has passed on to the suture. To aid the interactions between the deformable and the suture models, a novel quasispring tension method is presented. As an aid to creating tutorials for surgical students, we present a knot planner which can tie knots for demonstration.
Performance Metrics in Surgical Training
- What can motion derivatives tell us about skill performance. Vuong, L., Schwaitzberg, S., Cao, C.G.L. (2006). Society of American Gastrointestinal Endoscopic Surgeons Annual Scientific Session, Dallas, Texas, April 26-29, 2006.
"Subjects were evaluated on time to completion, errors, smoothness and total path length (used to calculate efficiency).
Results show that experience level is still distinguishable when using task-dependent parameters to evaluate performances during peg transfer (p= 0.035), pre-tied loop placement (p= 0.022), extracorporeal (p= 0.0006) and intracorporeal (p= 0.025) knot tying in this new simulator environment." - A computer-based laparoscopic skills assessment device differentiates experienced from novice laparoscopic surgeons. McNatt S., C. Smith. Surgical Endoscopy, pp 1085-1089, 2001.
- Time to complete task
- "Errors"
- Economy of movement for each hand
- Real-Time Objective Assessment of Knot Quality with a Portable Tensiometer Is Superior to Execution Time for Assessment of Laparoscopic Knot-Tying Performance. E. Ritter, et al. Sugical Innovation, vol 12, no 3, pp. 233-237. Sept 2005.
Argues that physically testing the knot (using a tensiometer) is essential to evaluate performance. "Time to complete a knot is a poor metric for the objective assessment of intracorporeal knot-tying performance in the absence of a measure of knot quality."
Great paper to help justify real-world/phsical knot tying. Using a tensiometer could justify claims that trace-measures are effective at judging performance (or not). - Bimodal assessment of laparoscopic suturing skills: Construct and concurrent validity. Surgical Endoscopy, vol 18, pp 1608-1612, 2004.
Justifies use of path length as a good measure of overall performance. Details the checklist used by human observers to judge performace-- some of the checklist's 29 items might be possible to automate using traces, e.g., needle driving with a single movement, angle of needle insertion, needle holder in view at all times. - Quantifying Risky Behavior in Surgical Simulation. Sewell C, Morris D, Blevins N, Barbagli F, Salisbury K. (Stanford) 2005 Medicine Meets Virtual Reality
Bone-drilling specific. Looks at a voxel-based approach to quatifying motions that are- Hidden from the view of the surgeon, esp bad when bone-drilling,
- Too fast, esp when handling the drill,
- Too strong.
Papers - Virtual and Mediated Reality in Laparoscopic Surgery
- Augmented Reality Visualization for Laparoscopic Surgery. Fuchs et al. (UNC Laparoscopic AR Research) Proc. 1st Int’l Conf. Medical Image
Computing and Computer-Assisted Intervention (MICCAI98).
Depth-extraction aided by projection of pattern into abdominal cavity.
We present the design and a prototype implementation of a three-dimensional visualization system to assist with laparoscopic surgi- cal procedures. The system uses 3D visualization, depth extraction from laparoscopic images, and six degree-of-freedom head and laparoscope tracking to display a merged real and synthetic image in the surgeon's video-see-through head-mounted display. We also introduce a custom design for this display. A digital light projector, a camera, and a conven- tional laparoscope create a prototype 3D laparoscope that can extract depth and video imagery. Such a system can restore the physician's natural point of view and head motion parallax that are used to understand the 3D structure during open surgery. These cues are not available in conventional laparoscopic surgery due to the displacement of the laparoscopic camera from the physician's viewpoint. The system can also display multiple laparoscopic range imaging data sets to widen the e ective eld of view of the device. These data sets can be displayed in true 3D and registered to the exterior anatomy of the patient. Much work remains to realize a clinically useful system, notably in the acquisition speed, reconstruction, and registration of the 3D imagery. - Augmented Reality for Enhancement of Endoscopic Interventions Bockholt, Bisler, Becker, Muller-Wittig, Voss. (Fraunhofer) IEEE VR 2003.
Computer assisted operation planning systems win more and more recognition in the field of surgery. These systems offer new possibilities to prepare an intervention with the goal to shorten the expansive time in the operation room required for the intervention. The safest and most effective surgical approach should be selected. But often, it is difficult to transfer the output of the planning system to the intra-operative situation and so to consider the planning results in the real intervention. At the Fraunhofer Institute for Computer Graphics (IGD) in Darmstadt and the Centre for Advanced Media Technology (CAMTech) in Singapore, methods are developed to bridge the gap between the external planning session and the intra-operative case: Augmented Reality (AR) techniques are used to overlap preoperative scanned image data as well as results of the planning session to the operation field. - Virtual endoscopy: development and evaluation using the Visible Human Datasets. R.A. Robb. Computerized Medical Imaging and Graphics 2000.
Virtual endoscopy (VE) is a new method of diagnosis using computer processing of 3D image datasets (such as CT or MRI scans) to provide simulated visualizations of patient specific organs similar or equivalent to those produced by standard endoseopic procedures. Conventional endoscopy is invasive and often uncomfortable for patients. It sometimes has serious side effects such as perforation, infection and hemorrhage. VE visualization avoids these risks and can minimize difficulties and decrease morbidity when used before actual endoseopic procedures. In addition, there are many body regions not compatible with real endoseopy that can be explored with VE. Eventually, VE may replace many forms of real endoscopy. There remains a critical need to refine and validate VE visualizations for routine clinical use. We have used the Visible Human Dataset from the National Library of Medicine to develop and test these procedures and to evaluate their use in a variety of clinical applications. We have developed specific clinical protocols to compare virtual endoscopy with real endoscopy. We have developed informative and dynamic on-screen navigation guides to help the surgeon or physician interactively determine body orientation and precise anatomical localization while performing the VE procedures. Additionally, the adjunctive value of full 3D imaging (e.g. looking "outside" of the normal field of view) during the VE exam is being evaluated. Quantitative analyses of local geometric and densitometric properties obtained from the virtual procedures ("virtual biopsy") are being developed and compared with other direct measures. Preliminary results suggest that these virtual procedures can provide accurate, reproducible and clinically useful visualizations and measurements. These studies will help drive improvements in and lend credibility to VE procedures and simulations as routine clinical tools. VE holds significant promise for optimizing endoscopic diagnostic procedures, minimizing patient risk and morbidity, and reducing health care costs.
Govt, Research Groups and Conferences
- Telemedicine and Advanced Technology Research Center (TATRC). (DoD)
- Surgical Simulation Laboratory (SSL). (part of National Capital Area Medical Simulation Center, Uniformed Services University)
- Medicine Meets Virtual Realtiy (MMVR).