Motion manipulation for characters in videos

Hung Kuo Chu, Che Hua Yeh, Nick C. Tang, Hsiao Rong Tyan, Yu-Shuen Wang, Hong Yuan Mark Liao

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Multimedia content creation and manipulation have attracted great attention in recent years due to the popularity of mobile image and video capturing devices. In our daily life, the most common subject appearing in the captured content are people. Hence, to create images and videos by manipulating appearances or motions of the human character inside these media becomes an important research issue. In this paper, we propose a novel idea that requires the fusion of video and audio intelligence. We develop a system whose inputs are a monocular video captured by a stationary RGBD sensor and a music clip. In the context of a human character performing a music conducting motion, the output is a re-targeted video where the motion of performer is manipulated according to the emotional cues extracted from the music clip. To achieve this goal, our system is decomposed into three major stages. First, one needs to access to the geometric and appearance information pertaining to meaningful and representative targets in the video sequence. Second, a systematic way is needed to reliably classify and identify important emotions from the music. Third, to complete the emotion transfer, one has to manipulate the video targets based on the extracted music emotions.

Original languageEnglish
Title of host publicationTAAI 2015 - 2015 Conference on Technologies and Applications of Artificial Intelligence
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages512-516
Number of pages5
ISBN (Electronic)9781467396066
DOIs
StatePublished - 12 Feb 2016
EventConference on Technologies and Applications of Artificial Intelligence, TAAI 2015 - Tainan, Taiwan
Duration: 20 Nov 201522 Nov 2015

Publication series

NameTAAI 2015 - 2015 Conference on Technologies and Applications of Artificial Intelligence

Conference

ConferenceConference on Technologies and Applications of Artificial Intelligence, TAAI 2015
CountryTaiwan
CityTainan
Period20/11/1522/11/15

Keywords

  • Motion characterization
  • Motion manipulation
  • Music Emotion

Fingerprint Dive into the research topics of 'Motion manipulation for characters in videos'. Together they form a unique fingerprint.

  • Cite this

    Chu, H. K., Yeh, C. H., Tang, N. C., Tyan, H. R., Wang, Y-S., & Liao, H. Y. M. (2016). Motion manipulation for characters in videos. In TAAI 2015 - 2015 Conference on Technologies and Applications of Artificial Intelligence (pp. 512-516). [7407085] (TAAI 2015 - 2015 Conference on Technologies and Applications of Artificial Intelligence). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/TAAI.2015.7407085