TY - GEN
T1 - Efficient manipulation of large datasets on heterogeneous storage systems
AU - Beynon, M. D.
AU - Kurc, T.
AU - Catalyurek, U.
AU - Sussman, A.
AU - Saltz, J.
N1 - Publisher Copyright:
© 2002 IEEE.
PY - 2002
Y1 - 2002
N2 - In this paper we are concerned with the efficient use of a collection of disk-based storage systems and computing platforms in a heterogeneous setting for retrieving and processing large scientific datasets. We demonstrate, in the context of a data-intensive visualization application, how heterogeneity affects performance and show a set of optimization techniques that can be used to improve performance in a component-based framework. In particular, we examine the application of parallelism via transparent copies of application components in the pipelined processing of data.
AB - In this paper we are concerned with the efficient use of a collection of disk-based storage systems and computing platforms in a heterogeneous setting for retrieving and processing large scientific datasets. We demonstrate, in the context of a data-intensive visualization application, how heterogeneity affects performance and show a set of optimization techniques that can be used to improve performance in a component-based framework. In particular, we examine the application of parallelism via transparent copies of application components in the pipelined processing of data.
UR - https://www.scopus.com/pages/publications/84966649631
U2 - 10.1109/IPDPS.2002.1015655
DO - 10.1109/IPDPS.2002.1015655
M3 - Conference contribution
AN - SCOPUS:84966649631
T3 - Proceedings - International Parallel and Distributed Processing Symposium, IPDPS 2002
SP - 84
EP - 96
BT - Proceedings - International Parallel and Distributed Processing Symposium, IPDPS 2002
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 16th International Parallel and Distributed Processing Symposium, IPDPS 2002
Y2 - 15 April 2002 through 19 April 2002
ER -