Robust 3D skeleton tracking based on openpose and a probabilistic tracking framework

Ching Chun Huang*, Manh Hung Nguyen

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

For human skeleton tracking, conventional methods may directly rely on the Kinect built-in function to extract skeleton joints and realize the tracking function. However, the left-right confusion problem and the self-occlusion problem would make the Kinect joints unstable and incorrect. In this paper, we aim to solve these crucial issues and realize a reliable 3D skeleton tracking system via multiple Kinect cameras. First, the proposed method corrected the Kinect skeleton by referring to the OpenPose-extracted joints. Since OpenPose extracts the joints by analyzing image content, we can differentiate the front side and the back side and thus can correct the left-right confusion. Second, to alleviate the selfocclusion problem, we back-project the 2D OpenPose joints from multiple cameras and form an alternative 3D skeleton. The OpenPose reconstructed joints are treated as robust 3D anchors for multiple skeleton fusion. Last but not least, we introduce the inter-joint constraints into our skeleton tracking framework so that we can trace all joints simultaneously, make sure the skeleton movement consistent, and well maintain the length between neighboring joints. We evaluate our method with challenging actions. A practical system is also built for demonstration. The experimental results show the system can track skeleton stably without error propagation and vibration. The average localization error is also smaller than conventional methods.

Original languageEnglish
Title of host publication2019 IEEE International Conference on Systems, Man and Cybernetics, SMC 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages4107-4112
Number of pages6
ISBN (Electronic)9781728145693
DOIs
StatePublished - Oct 2019
Event2019 IEEE International Conference on Systems, Man and Cybernetics, SMC 2019 - Bari, Italy
Duration: 6 Oct 20199 Oct 2019

Publication series

NameConference Proceedings - IEEE International Conference on Systems, Man and Cybernetics
Volume2019-October
ISSN (Print)1062-922X

Conference

Conference2019 IEEE International Conference on Systems, Man and Cybernetics, SMC 2019
CountryItaly
CityBari
Period6/10/199/10/19

Fingerprint Dive into the research topics of 'Robust 3D skeleton tracking based on openpose and a probabilistic tracking framework'. Together they form a unique fingerprint.

Cite this