On design of browser-oriented data extraction system and the plug-ins

Jui Yuan Su*, Der Johng Sun, I-Chen Wu, Lung Pin Chen

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

5 Scopus citations

Abstract

Web data extraction systems currently not only extract data on web pages but also need to navigate to the target correctly. Most traditional web data extraction systems extract URLs directly from web pages, and then access next pages using the extracted URLs. This data extraction approach is herein called the URL-oriented data extraction approach in this paper. However, currently, more and more web pages use script functions, such as JavaScript, to access next pages and may hide URLs inside these functions, making it difficult to extract URLs. In order to solve this problem, a new data extraction approach, named the browser-oriented data extraction (BODE) approach, is proposed to be built on top of browser objects access pages by simulating users' operations on browsers to invoke script functions. A data extraction system based on this approach is called the BODE system. Based on the BODE approach, this paper designed a BODE system with the following contributions: (a) Define a scripting language, named the BODED (Browser-Oriented Data Extraction Description) language, which instructs the BODE system to extract data, (b) Design a plug-ins that can be used to extend the functionalities of the BODE system, (c) Design a visualization tool to support the data extraction in the BODE system, (d) Illustrate the plug-in mechanism of the BODE system by automating the playing of the game Connect6 over an Internet game site.

Original languageEnglish
Pages (from-to)189-200
Number of pages12
JournalJournal of Marine Science and Technology
Volume18
Issue number2
StatePublished - 1 Apr 2010

Keywords

  • Browser-oriented data extraction
  • Data extraction model
  • Plug-ins
  • Url-oriented data extraction
  • Visualization tools

Fingerprint Dive into the research topics of 'On design of browser-oriented data extraction system and the plug-ins'. Together they form a unique fingerprint.

Cite this