Alignment of deep features in 3D models for camera pose estimation

Jui Yuan Su*, Shyi Chyi Cheng, Chin Chun Chang, Jun-Wei Hsieh

*Corresponding author for this work

研究成果: Conference contribution同行評審


Using a set of semantically annotated RGB-D images with known camera poses, many existing 3D reconstruction algorithms can integrate these images into a single 3D model of the scene. The semantically annotated scene model facilitates the construction of a video surveillance system using a moving camera if we can efficiently compute the depth maps of the captured images and estimate the poses of the camera. The proposed model-based video surveillance consists of two phases, i.e. the modeling phase and the inspection phase. In the modeling phase, we carefully calibrate the parameters of the camera that captures the multi-view video for modeling the target 3D scene. However, in the inspection phase, the camera pose parameters and the depth maps of the captured RGB images are often unknown or noisy when we use a moving camera to inspect the completeness of the object. In this paper, the 3D model is first transformed into a colored point cloud, which is then indexed by clustering—with each cluster representing a surface fragment of the scene. The clustering results are then used to train a model-specific convolution neural network (CNN) that annotates each pixel of an input RGB image with a correct fragment class. The prestored camera parameters and depth information of fragment classes are then fused together to estimate the depth map and the camera pose of the current input RGB image. The experimental results show that the proposed approach outperforms the compared methods in terms of the accuracy of camera pose estimation.

主出版物標題MultiMedia Modeling - 25th International Conference, MMM 2019, Proceedings
編輯Benoit Huet, Ioannis Kompatsiaris, Stefanos Vrochidis, Vasileios Mezaris, Wen-Huang Cheng, Cathal Gurrin
發行者Springer Verlag
出版狀態Published - 1 一月 2019
事件25th International Conference on MultiMedia Modeling, MMM 2019 - Thessaloniki, Greece
持續時間: 8 一月 201911 一月 2019


名字Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
11296 LNCS


Conference25th International Conference on MultiMedia Modeling, MMM 2019

指紋 深入研究「Alignment of deep features in 3D models for camera pose estimation」主題。共同形成了獨特的指紋。