Three dimensional panorama visualization for endoscopic video

Atul Kumar*, Yen Yu Wang, Kai Che Liu, Wan Chi Hung, Shih Wei Huang, Wen Nung Lie, Ching-Chun Huang

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Scopus citations


Introduction: Endoscopy is widely used in the surgical world. Reduced surgical injury during endoscopic surgery makes the patient’s life easy. However, the endoscopic surgery becomes a challenge for the novice surgeons because of the narrow field of view and the lack of 3D perception in the 2D endoscope image. Such limitations of the endoscope may be addressed with a 3D panorama created with the endoscopic 2D images. Few studies have been reported to create a 3D panorama from endoscopic images. However, to our knowledge, no study has reported a 3D panorama system which uses only a single camera image from the conventional endoscope to make a 3D shape of the organs, and further stitch those 3D shapes to create a 3D panorama of the surgical scene. Such system would enable surgeons to have an extended field of view of the surgical scene with a 3D perception. Method: Images from the endoscopic surgery video were used in this study. A shape from shading (SfS) algorithm was applied to create a 3D shape of the organs in the images. Characteristic feature points were identified on the images using a feature detection algorithm (SURF). The matching feature points in the consecutive images were found using a feature matching algorithm (BRIEF). An iterative closest point (ICP) algorithm was applied to stitch 3D shapes from the consecutive images to synthesize the 3D panorama. Results: The method was applied on 100 consecutive video frames from an endoscopic video of a patient. The root mean square error for the registration of the consecutive image feature points was <4 mm. The 3D panorama from the method helps in visualizing a larger area of the surgical anatomy. An improved version of the method may be applied to a real time video from the endoscope.

Original languageEnglish
Title of host publication7th WACBE World Congress on Bioengineering, 2015
EditorsChwee Teck Lim, James Goh
PublisherSpringer Verlag
Number of pages4
ISBN (Print)9783319194516
StatePublished - 1 Jan 2015
Event7th World Congress on Bioengineering, WACBE 2015 - Kent Ridge, Singapore
Duration: 6 Jul 20158 Jul 2015

Publication series

NameIFMBE Proceedings
ISSN (Print)1680-0737


Conference7th World Congress on Bioengineering, WACBE 2015
CityKent Ridge


  • 3D surface reconstruction
  • Mosaicing
  • Panorama

Fingerprint Dive into the research topics of 'Three dimensional panorama visualization for endoscopic video'. Together they form a unique fingerprint.

Cite this