Image-based Pose Estimation and Tracking of Surgical Instruments in Minimally Invasive Surgery

Yu Shin Wang, Kai Tai Song

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Scopus citations

Abstract

This study proposes a method for image-based surgical instrument pose estimation. In minimally invasive surgery (MIS), stereo vision can be used for surgical tool recognition and pose estimation. We use color markers to indicate the instrument's joints and detect them with a custom-made stereo camera to obtain their 3D coordinates. Kalman filter is adopted to attain the position of the given instrument's joints. We propose a kinematic algorithm to calculate the 6-Dof pose of the surgical instrument in the Cartesian coordinate system and the rotation angles of its joints. The proposed method is verified on a lab-built experimental platform, consisting of a 6-DoF HIWIN manipulator and a motorized surgical instrument. The experiment involves the instrument's tracking with a stereo camera in MIS pick-and-place routine. Experimental results show that the visual system has a maximum absolute error lower than 2.7 millimeters at the endpoint and the MAE of joint angles is less than 0.11 radians. The developed image-based pose estimation system can be used for the surgeon's training in MIS and may be applied to autonomous surgical operations.

Original languageEnglish
Title of host publication2020 International Automatic Control Conference, CACS 2020
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781728171982
DOIs
StatePublished - 4 Nov 2020
Event2020 International Automatic Control Conference, CACS 2020 - Hsinchu, Taiwan
Duration: 4 Nov 20207 Nov 2020

Publication series

Name2020 International Automatic Control Conference, CACS 2020

Conference

Conference2020 International Automatic Control Conference, CACS 2020
CountryTaiwan
CityHsinchu
Period4/11/207/11/20

Keywords

  • object pose estimation
  • stereo vision
  • surgical image recognition
  • surgical instrument tracking

Fingerprint Dive into the research topics of 'Image-based Pose Estimation and Tracking of Surgical Instruments in Minimally Invasive Surgery'. Together they form a unique fingerprint.

Cite this