ReHRS: A Hybrid Redundant System for Improving MapReduce Reliability and Availability

Jia Chun Lin, Fang Yie Leu, Ying-ping Chen

Research output: Chapter in Book/Report/Conference proceedingChapterpeer-review

5 Scopus citations

Abstract

MapReduce is a parallel programming framework proposed by Google. Recently, it has become a popular technology for solving data-intensive applications. However, current MapReduce implementations provide insufficient redundant mechanisms for their master servers, consequently causing the fact that the master servers' services cannot continue and all jobs cannot proceed and complete when the master servers unexpectedly fail. To solve this problem, this chapter proposes a master server redundant mechanism called the Reliable Hybrid Redundant System (ReHRS for short), in which a hot-standby server is employed to maintain the latest metadata of the master sever so as to achieve a fast takeover, and a warm-standby server is employed to further enhance system reliability and extend the operation of MapReduce when both the master server and hot-standby server cannot work properly. We proposed a failure detection algorithm to detect the failure of the master server and hot-standby server, and provided appropriate takeover processes to continue their operations. Additionally, we introduced a dynamic warmup mechanism for the warm-standby server to warm itself up such that it can quickly act as the hot-standby server when necessary. The extensive simulation and experiment results show that the ReHRS significantly speeds up the takeover process as compared with three state-of-the-art schemes.

Original languageEnglish
Title of host publicationModeling and Optimization in Science and Technologies
PublisherSpringer Verlag
Pages187-209
Number of pages23
DOIs
StatePublished - 1 Jan 2015

Publication series

NameModeling and Optimization in Science and Technologies
Volume4
ISSN (Print)2196-7326
ISSN (Electronic)2196-7334

Keywords

  • MapReduce
  • availability
  • reliability
  • reliable hybrid redundant system
  • single-point-of-failure

Fingerprint Dive into the research topics of 'ReHRS: A Hybrid Redundant System for Improving MapReduce Reliability and Availability'. Together they form a unique fingerprint.

Cite this