A framework for quantitative evaluation of parallel control-flow obfuscation

Yu-Lun Huang*, Hsin Yi Tsai

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

3 Scopus citations

Abstract

Software obfuscation is intended to protect a program by thwarting reverse engineering. Several types of software obfuscation have been proposed, and control-flow obfuscation is a commonly adopted one. In this paper, we present a framework to evaluate parallel control-flow obfuscation, which raises difficulty of reverse engineering by increasing parallelism of a program. We also define a control flow graph of a program and some atomic operators for obfuscating transformations. The proposed framework comprises three phases: parsing, formalization and evaluation. A program is first parsed to a control flow graph. Then, we formalize a parallel control-flow obfuscating transformation based on our atomic operators. By selecting target code blocks in the control flow graph and applying obfuscating transformations to the target code blocks, the original program is then obfuscated. In the third phase, we define a measure to calculate the program complexity. The measure can be considered as a degree to which an obfuscating transformation can confuse a human trying to understand the obfuscated program. Such a measure can also be used as the base of the potency metric to estimate the capability of the obfuscated program against reverse engineering. Our novel framework helps efficiently examine a control-flow obfuscating transformation in a systematic manner and helps select an appropriate obfuscating transformation among a number of candidates to better protect a program.

Original languageEnglish
Pages (from-to)886-896
Number of pages11
JournalComputers and Security
Volume31
Issue number8
DOIs
StatePublished - 1 Nov 2012

Keywords

  • Control flow graph
  • Reverse engineering
  • Security evaluation
  • Software metrics
  • Software obfuscation

Fingerprint Dive into the research topics of 'A framework for quantitative evaluation of parallel control-flow obfuscation'. Together they form a unique fingerprint.

Cite this