TY - GEN
T1 - A run-time system for efficient execution of scientific workflows on distributed environments
AU - Teodoro, George
AU - Tavares, Tulio
AU - Ferreira, Renato
AU - Kurc, Tahsin
AU - Meira, Wagner
AU - Guedes, Dorgival
AU - Pan, Tony
AU - Saltz, Joel
PY - 2006
Y1 - 2006
N2 - Scientific workflow systems have been introduced in response to the demand of researchers from several domains of science who need to process and analyze increasingly larger datasets. The design of these systems is largely based on the observation that data analysis applications can be composed as pipelines or networks of computations on data. In this work we present a run-time support system that is designed to facilitate this type of computation in distributed computing environments. Our system is optimized for data-intensive workflows, in which efficient management and retrieval of data, coordination of data processing and data movement, and check-pointing of intermediate results are critical and challenging issues. Experimental evaluation of our system shows that linear speedups can be achieved for sophisticated applications, which are implemented as a network of multiple data processing components.
AB - Scientific workflow systems have been introduced in response to the demand of researchers from several domains of science who need to process and analyze increasingly larger datasets. The design of these systems is largely based on the observation that data analysis applications can be composed as pipelines or networks of computations on data. In this work we present a run-time support system that is designed to facilitate this type of computation in distributed computing environments. Our system is optimized for data-intensive workflows, in which efficient management and retrieval of data, coordination of data processing and data movement, and check-pointing of intermediate results are critical and challenging issues. Experimental evaluation of our system shows that linear speedups can be achieved for sophisticated applications, which are implemented as a network of multiple data processing components.
UR - https://www.scopus.com/pages/publications/38949177029
U2 - 10.1109/SBAC-PAD.2006.6
DO - 10.1109/SBAC-PAD.2006.6
M3 - Conference contribution
AN - SCOPUS:38949177029
SN - 0769527043
SN - 9780769527048
T3 - Proceedings - Symposium on Computer Architecture and High Performance Computing
SP - 81
EP - 88
BT - Proceedings - SBAC-PAD 2006
T2 - SBAC-PAD 2006: 18th International Symposium on Computer Architecture and High Performance Computing
Y2 - 17 October 2006 through 20 October 2006
ER -