A two-dimensional editing system based on hand gestures and real object manipulation using RGB-D sensor

Hsin Kai Chen, Sai-Keung Wong*, Po Han Huang, Yu An Chang

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Scopus citations

Abstract

This paper presents a two-dimensional editing system which compromises the advantages of using hand gestures and real blocks to manipulate virtual objects. The system uses a RGB-D sensor to capture the color and depth images of the hands and real blocks. The raw images of the hands and blocks are segmented. Then we employ a contour based algorithm to perform hand gesture and shape recognition. The contours are extracted and reduced for eliminating the redundant points. After that a set of feature points are collected and they can be used for recognizing the hand gesture and the shapes of the real blocks. The users have two ways to interact with the virtual objects: 1) manipulating the real blocks to control them; and 2) using hand gestures as editing operations to edit them. Experimental results indicate that our system achieves high recognition rate. Our system supports the basic editing operations to the virtual objects, such as 'move', 'scale', 'rotate', and 'copy'. We also conducted a user study which reveals that our system is intuitive and entertaining.

Original languageEnglish
Title of host publicationProceedings - VRCAI 2016
Subtitle of host publication15th ACM SIGGRAPH Conference on Virtual-Reality Continuum and Its Applications in Industry
EditorsStephen N. Spencer
PublisherAssociation for Computing Machinery, Inc
Pages119-128
Number of pages10
ISBN (Electronic)9781450346924
DOIs
StatePublished - 3 Dec 2016
Event15th ACM SIGGRAPH Conference on Virtual-Reality Continuum and Its Applications in Industry, VRCAI 2016 - Zhuhai, China
Duration: 3 Dec 20164 Dec 2016

Publication series

NameProceedings - VRCAI 2016: 15th ACM SIGGRAPH Conference on Virtual-Reality Continuum and Its Applications in Industry
Volume1

Conference

Conference15th ACM SIGGRAPH Conference on Virtual-Reality Continuum and Its Applications in Industry, VRCAI 2016
CountryChina
CityZhuhai
Period3/12/164/12/16

Keywords

  • Hand gesture recognition
  • Human and computer interaction
  • RGB-D sensor
  • Shape recognition

Fingerprint Dive into the research topics of 'A two-dimensional editing system based on hand gestures and real object manipulation using RGB-D sensor'. Together they form a unique fingerprint.

Cite this