Anomaly detection via online oversampling principal component analysis

Yuh-Jye Lee, Yi Ren Yeh, Yu Chiang Frank Wang

Research output: Contribution to journalArticle

108 Scopus citations

Abstract

Anomaly detection has been an important research topic in data mining and machine learning. Many real-world applications such as intrusion or credit card fraud detection require an effective and efficient framework to identify deviated data instances. However, most anomaly detection methods are typically implemented in batch mode, and thus cannot be easily extended to large-scale problems without sacrificing computation and memory requirements. In this paper, we propose an online oversampling principal component analysis (osPCA) algorithm to address this problem, and we aim at detecting the presence of outliers from a large amount of data via an online updating technique. Unlike prior principal component analysis (PCA)-based approaches, we do not store the entire data matrix or covariance matrix, and thus our approach is especially of interest in online or large-scale problems. By oversampling the target instance and extracting the principal direction of the data, the proposed osPCA allows us to determine the anomaly of the target instance according to the variation of the resulting dominant eigenvector. Since our osPCA need not perform eigen analysis explicitly, the proposed framework is favored for online applications which have computation or memory limitations. Compared with the well-known power method for PCA and other popular anomaly detection algorithms, our experimental results verify the feasibility of our proposed method in terms of both accuracy and efficiency.

Original languageEnglish
Article number6200273
Pages (from-to)1460-1470
Number of pages11
JournalIEEE Transactions on Knowledge and Data Engineering
Volume25
Issue number7
DOIs
StatePublished - 3 Jun 2013

Keywords

  • Anomaly detection
  • least squares
  • online updating
  • oversampling
  • principal component analysis

Fingerprint Dive into the research topics of 'Anomaly detection via online oversampling principal component analysis'. Together they form a unique fingerprint.

  • Cite this