Tracking the changes of dynamic web pages in the existence of URL rewriting

Ping Jer Yeh; Jie Tsung Li; Shyan-Ming Yuan

Tracking the changes of dynamic web pages in the existence of URL rewriting

Ping Jer Yeh^*, Jie Tsung Li, Shyan-Ming Yuan

^*Corresponding author for this work

Department of Computer Science

Research output: Contribution to journal › Conference article › peer-review

Abstract

Crawlers in a knowledge management system need to collect and archive documents from websites, and also track the change status of these documents. However, the existence of URL rewriting mechanism raises a page tracking problem since the URLs of a pair of dynamic page instances obtained during different sessions will no longer be the same. This paper proposes a series of algorithms in a bottom-up manner to find the corresponding pairs of dynamic page instances, and then to judge the change status of them. Experiments showed that the performance was very good and the outcome was 100% accurate.

Original language	English
Pages (from-to)	169-176
Number of pages	8
Journal	Conferences in Research and Practice in Information Technology Series
Volume	61
State	Published - 1 Dec 2006
Event	5th Australasian Data Mining Conference, AusDM 2006 - Sydney, NSW, Australia Duration: 29 Nov 2006 → 30 Nov 2006

Keywords

Crawler
HTTP session
String matching
URL rewriting

Fingerprint Dive into the research topics of 'Tracking the changes of dynamic web pages in the existence of URL rewriting'. Together they form a unique fingerprint.

Cite this

@article{538c3efa34d8475daaa360e2ce04578c,

title = "Tracking the changes of dynamic web pages in the existence of URL rewriting",

abstract = "Crawlers in a knowledge management system need to collect and archive documents from websites, and also track the change status of these documents. However, the existence of URL rewriting mechanism raises a page tracking problem since the URLs of a pair of dynamic page instances obtained during different sessions will no longer be the same. This paper proposes a series of algorithms in a bottom-up manner to find the corresponding pairs of dynamic page instances, and then to judge the change status of them. Experiments showed that the performance was very good and the outcome was 100% accurate.",

keywords = "Crawler, HTTP session, String matching, URL rewriting",

author = "Yeh, {Ping Jer} and Li, {Jie Tsung} and Shyan-Ming Yuan",

year = "2006",

month = dec,

day = "1",

language = "English",

volume = "61",

pages = "169--176",

journal = "Conferences in Research and Practice in Information Technology Series",

issn = "1445-1336",

note = "null ; Conference date: 29-11-2006 Through 30-11-2006",

}

TY - JOUR

T1 - Tracking the changes of dynamic web pages in the existence of URL rewriting

AU - Yeh, Ping Jer

AU - Li, Jie Tsung

AU - Yuan, Shyan-Ming

PY - 2006/12/1

Y1 - 2006/12/1

N2 - Crawlers in a knowledge management system need to collect and archive documents from websites, and also track the change status of these documents. However, the existence of URL rewriting mechanism raises a page tracking problem since the URLs of a pair of dynamic page instances obtained during different sessions will no longer be the same. This paper proposes a series of algorithms in a bottom-up manner to find the corresponding pairs of dynamic page instances, and then to judge the change status of them. Experiments showed that the performance was very good and the outcome was 100% accurate.

AB - Crawlers in a knowledge management system need to collect and archive documents from websites, and also track the change status of these documents. However, the existence of URL rewriting mechanism raises a page tracking problem since the URLs of a pair of dynamic page instances obtained during different sessions will no longer be the same. This paper proposes a series of algorithms in a bottom-up manner to find the corresponding pairs of dynamic page instances, and then to judge the change status of them. Experiments showed that the performance was very good and the outcome was 100% accurate.

KW - Crawler

KW - HTTP session

KW - String matching

KW - URL rewriting

UR - http://www.scopus.com/inward/record.url?scp=84870547488&partnerID=8YFLogxK

M3 - Conference article

AN - SCOPUS:84870547488

VL - 61

SP - 169

EP - 176

JO - Conferences in Research and Practice in Information Technology Series

JF - Conferences in Research and Practice in Information Technology Series

SN - 1445-1336

Y2 - 29 November 2006 through 30 November 2006

ER -

Tracking the changes of dynamic web pages in the existence of URL rewriting

Abstract

Keywords

Other files and links

Cite this