360SD-Net: 360° Stereo Depth Estimation with Learnable Cost Volume

Ning Hsu Wang, Bolivar Solarte, Yi Hsuan Tsai, Wei Chen Chiu, Min Sun

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Recently, end-to-end trainable deep neural networks have significantly improved stereo depth estimation for perspective images. However, 360° images captured under equirectangular projection cannot benefit from directly adopting existing methods due to distortion introduced (i.e., lines in 3D are not projected onto lines in 2D). To tackle this issue, we present a novel architecture specifically designed for spherical disparity using the setting of top-bottom 360° camera pairs. Moreover, we propose to mitigate the distortion issue by (1) an additional input branch capturing the position and relation of each pixel in the spherical coordinate, and (2) a cost volume built upon a learnable shifting filter. Due to the lack of 360° stereo data, we collect two 360° stereo datasets from Matterport3D and Stanford3D for training and evaluation. Extensive experiments and ablation study are provided to validate our method against existing algorithms. Finally, we show promising results on real-world environments capturing images with two consumer-level cameras. Our project page is at https://albert100121.github.io/360SD-Net-Project-Page.

Original languageEnglish
Title of host publication2020 IEEE International Conference on Robotics and Automation, ICRA 2020
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages582-588
Number of pages7
ISBN (Electronic)9781728173955
DOIs
StatePublished - May 2020
Event2020 IEEE International Conference on Robotics and Automation, ICRA 2020 - Paris, France
Duration: 31 May 202031 Aug 2020

Publication series

NameProceedings - IEEE International Conference on Robotics and Automation
ISSN (Print)1050-4729

Conference

Conference2020 IEEE International Conference on Robotics and Automation, ICRA 2020
CountryFrance
CityParis
Period31/05/2031/08/20

Fingerprint Dive into the research topics of '360SD-Net: 360° Stereo Depth Estimation with Learnable Cost Volume'. Together they form a unique fingerprint.

Cite this