Real-time fingerspelling recognition system design based on RGB-D image information

Kuan Yu Chou, Hui Chi Chuang, Meng Tzu Chiu, Yon-Ping Chen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Scopus citations

Abstract

This paper provides a fingerspelling recognition system with high accuracy rate based on RGB-D image. The system are separated into three parts, including hand region detection, hand feature extraction, and fingerspelling recognition. For the hand regions detection, the regions of hand and face are first obtained by skin color detection and connect component labeling (CCL), and then the hand, the region-of-interest (ROI), is determined by the feature point extraction based on distance transform. Followed is the hand feature extraction which consists of the hand structure and the hand texture. From the feature points of ROI, the locations of palm and fingertips, palm direction, and finger vectors are formed as the hand structure. In addition to the hand structure, this paper adopts the LBP operator to generate the hand texture to deal with the fingerspelling not recognizable by the hand structure. Finally, the extracted hand features are sent into the fingerspelling recognition system, which is built with several different neural network classifiers. The experimental results show that this system is an effective real-time recognition system whose accuracy is higher than 80% for most of the fingerspelling in American Sign Language (ASL).

Original languageEnglish
Title of host publicationCACS 2014 - 2014 International Automatic Control Conference, Conference Digest
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages75-80
Number of pages6
ISBN (Electronic)9781479945849
DOIs
StatePublished - 28 Apr 2014
Event2014 International Automatic Control Conference, CACS 2014 - Kaohsiung, Taiwan
Duration: 26 Nov 201428 Nov 2014

Publication series

NameCACS 2014 - 2014 International Automatic Control Conference, Conference Digest

Conference

Conference2014 International Automatic Control Conference, CACS 2014
CountryTaiwan
CityKaohsiung
Period26/11/1428/11/14

Keywords

  • fingerspelling recognition
  • gesture recognition

Fingerprint Dive into the research topics of 'Real-time fingerspelling recognition system design based on RGB-D image information'. Together they form a unique fingerprint.

  • Cite this

    Chou, K. Y., Chuang, H. C., Chiu, M. T., & Chen, Y-P. (2014). Real-time fingerspelling recognition system design based on RGB-D image information. In CACS 2014 - 2014 International Automatic Control Conference, Conference Digest (pp. 75-80). [7097165] (CACS 2014 - 2014 International Automatic Control Conference, Conference Digest). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/CACS.2014.7097165