Image-text dual neural network with decision strategy for small-sample image classification

Fangyi Zhu, Zhanyu Ma*, Xiaoxu Li, Guang Chen, Jen-Tzung Chien, Jing Hao Xue, Jun Guo

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

28 Scopus citations

Abstract

Small-sample classification is a challenging problem in computer vision. In this work, we show how to efficiently and effectively utilize semantic information of the annotations to improve the performance of small-sample classification. First, we propose an image-text dual neural network to improve the classification performance on small-sample datasets. The proposed model consists of two sub-models, an image classification model and a text classification model. After training the sub-models separately, we design a novel method to fuse the two sub-models rather than simply combine their results. Our image-text dual neural network aims to utilize the text information to overcome the training problem of deep models on small-sample datasets. Then, we propose to incorporate a decision strategy into the image-text dual neural network to further improve the performance of our original model on few-shot datasets. To demonstrate the effectiveness of the proposed models, we conduct experiments on the LabelMe and UIUC-Sports datasets. Experimental results show that our method is superior to other models.

Original languageEnglish
Pages (from-to)182-188
Number of pages7
JournalNeurocomputing
Volume328
DOIs
StatePublished - 7 Feb 2019

Keywords

  • Deep convolutional neural network
  • Ensemble learning
  • Few-shot
  • Small-sample image classification

Fingerprint Dive into the research topics of 'Image-text dual neural network with decision strategy for small-sample image classification'. Together they form a unique fingerprint.

Cite this