Decoding and encoding of visual patterns using magnetoencephalographic data represented in manifolds

Po Chih Kuo, Yong-Sheng Chen*, Li Fen Chen, Jen Chuen Hsieh

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

5 Scopus citations

Abstract

Visual decoding and encoding are crucial aspects in investigating the representation of visual information in the human brain. This paper proposes a bidirectional model for decoding and encoding of visual stimulus based on manifold representation of the temporal and spatial information extracted from magnetoencephalographic data. In the proposed decoding process, principal component analysis is applied to extract temporal principal components (TPCs) from the visual cortical activity estimated by a beamforming method. The spatial distribution of each TPC is in a high-dimensional space and can be mapped to the corresponding spatiotemporal component (STC) on a low-dimensional manifold. Once the linear mapping between the STC and the wavelet coefficients of the stimulus image is determined, the decoding process can synthesize an image resembling the stimulus image. The encoding process is performed by reversing the mapping or transformation in the decoding model and can predict the spatiotemporal brain activity from a stimulus image. In our experiments using visual stimuli containing eleven combinations of checkerboard patches, the information of spatial layout in the stimulus image was revealed in the embedded manifold. The correlation between the reconstructed and original images was 0.71 and the correlation map between the predicted and original brain activity was highly correlated to the map between the original brain activity for different stimuli (r= 0.89). These results suggest that the temporal component is important in visual processing and manifolds can well represent the information related to visual perception.

Original languageEnglish
Pages (from-to)435-450
Number of pages16
JournalNeuroImage
Volume102
Issue numberP2
DOIs
StatePublished - 15 Nov 2014

Keywords

  • Magnetoencephalography
  • Manifold
  • Visual decoding
  • Visual encoding

Fingerprint Dive into the research topics of 'Decoding and encoding of visual patterns using magnetoencephalographic data represented in manifolds'. Together they form a unique fingerprint.

Cite this