LiPS: A cost-efficient data and task co-scheduler for MapReduce

Moussa Ehsan; Yao Chen; Hui Kang; Radu Sion; Jennifer Wong

doi:10.1109/HiPC.2013.6799103

LiPS: A cost-efficient data and task co-scheduler for MapReduce

Moussa Ehsan
, Yao Chen
, Hui Kang
, Radu Sion
, Jennifer Wong

Computer Science

Stony Brook University

Research output: Contribution to conference › Paper › peer-review

2 Scopus citations

Abstract

We introduce LiPS, a new cost-efficient data and task co-scheduler for MapReduce in a cloud environment. By using linear programming to simultaneously co-schedule data and tasks, LiPS helps to achieve minimized dollar cost globally. We evaluated LiPS both analytically and on Amazon EC2 in order to measure actual dollar charges. The results were significant; LiPS saved 62-81% of the dollar costs when compared with the Hadoop default scheduler and the delay scheduler, while also allowing users to fine-tune the cost-performance tradeoff.

Original language	English
Pages	49-58
Number of pages	10
DOIs	https://doi.org/10.1109/HiPC.2013.6799103
State	Published - 2013
Event	20th Annual International Conference on High Performance Computing, HiPC 2013 - Bangalore, India Duration: Dec 18 2013 → Dec 21 2013

Conference

Conference	20th Annual International Conference on High Performance Computing, HiPC 2013
Country/Territory	India
City	Bangalore
Period	12/18/13 → 12/21/13

Access to Document

10.1109/HiPC.2013.6799103

Cite this

@conference{88deba80b07b43729691b551d0363a7c,

title = "LiPS: A cost-efficient data and task co-scheduler for MapReduce",

abstract = "We introduce LiPS, a new cost-efficient data and task co-scheduler for MapReduce in a cloud environment. By using linear programming to simultaneously co-schedule data and tasks, LiPS helps to achieve minimized dollar cost globally. We evaluated LiPS both analytically and on Amazon EC2 in order to measure actual dollar charges. The results were significant; LiPS saved 62-81\% of the dollar costs when compared with the Hadoop default scheduler and the delay scheduler, while also allowing users to fine-tune the cost-performance tradeoff.",

author = "Moussa Ehsan and Yao Chen and Hui Kang and Radu Sion and Jennifer Wong",

year = "2013",

doi = "10.1109/HiPC.2013.6799103",

language = "English",

pages = "49--58",

note = "20th Annual International Conference on High Performance Computing, HiPC 2013 ; Conference date: 18-12-2013 Through 21-12-2013",

}

TY - CONF

T1 - LiPS

T2 - 20th Annual International Conference on High Performance Computing, HiPC 2013

AU - Ehsan, Moussa

AU - Chen, Yao

AU - Kang, Hui

AU - Sion, Radu

AU - Wong, Jennifer

PY - 2013

Y1 - 2013

N2 - We introduce LiPS, a new cost-efficient data and task co-scheduler for MapReduce in a cloud environment. By using linear programming to simultaneously co-schedule data and tasks, LiPS helps to achieve minimized dollar cost globally. We evaluated LiPS both analytically and on Amazon EC2 in order to measure actual dollar charges. The results were significant; LiPS saved 62-81% of the dollar costs when compared with the Hadoop default scheduler and the delay scheduler, while also allowing users to fine-tune the cost-performance tradeoff.

AB - We introduce LiPS, a new cost-efficient data and task co-scheduler for MapReduce in a cloud environment. By using linear programming to simultaneously co-schedule data and tasks, LiPS helps to achieve minimized dollar cost globally. We evaluated LiPS both analytically and on Amazon EC2 in order to measure actual dollar charges. The results were significant; LiPS saved 62-81% of the dollar costs when compared with the Hadoop default scheduler and the delay scheduler, while also allowing users to fine-tune the cost-performance tradeoff.

UR - https://www.scopus.com/pages/publications/84900302873

U2 - 10.1109/HiPC.2013.6799103

DO - 10.1109/HiPC.2013.6799103

M3 - Paper

AN - SCOPUS:84900302873

SP - 49

EP - 58

Y2 - 18 December 2013 through 21 December 2013

ER -

LiPS: A cost-efficient data and task co-scheduler for MapReduce

Abstract

Conference

Access to Document

Other files and links

Fingerprint

Cite this