TY - GEN
T1 - A Case Study in Application I/O on Linux Clusters
AU - Ross, Robert
AU - Nurmi, Daniel
AU - Cheng, Albert
AU - Zingale, Michael
N1 - Publisher Copyright:
© 2001 ACM.
PY - 2001/11/10
Y1 - 2001/11/10
N2 - A critical but often ignored component of system performance is the I/O system. Today's applications demand a great deal from underlying storage systems and software, and both high-performance distributed storage and high level interfaces have been developed to fill these needs.In this paper we discuss the I/O performance of a parallel scientific application on a Linux cluster, the FLASH astrophysics code. This application relies on three I/O software components to provide high-performance parallel I/O on Linux clusters: the Parallel Virtual File System, the ROMIO MPI-IO implementation, and the Hierarchical Data Format library. Through instrumentation of both the application and underlying system software code we discover the location of major software bottlenecks. We work around the most inhibiting of these bottlenecks, showing substantial performance improvement. We point out similarities between the inefficiencies found here and those found in message passing systems, indicating that research in the message passing field could be leveraged to solve similar problems in high-level I/O interfaces.
AB - A critical but often ignored component of system performance is the I/O system. Today's applications demand a great deal from underlying storage systems and software, and both high-performance distributed storage and high level interfaces have been developed to fill these needs.In this paper we discuss the I/O performance of a parallel scientific application on a Linux cluster, the FLASH astrophysics code. This application relies on three I/O software components to provide high-performance parallel I/O on Linux clusters: the Parallel Virtual File System, the ROMIO MPI-IO implementation, and the Hierarchical Data Format library. Through instrumentation of both the application and underlying system software code we discover the location of major software bottlenecks. We work around the most inhibiting of these bottlenecks, showing substantial performance improvement. We point out similarities between the inefficiencies found here and those found in message passing systems, indicating that research in the message passing field could be leveraged to solve similar problems in high-level I/O interfaces.
UR - https://www.scopus.com/pages/publications/85129642101
U2 - 10.1145/582034.582045
DO - 10.1145/582034.582045
M3 - Conference contribution
AN - SCOPUS:85129642101
T3 - Proceedings of the International Conference on Supercomputing
SP - 11
BT - Proceedings of the 2001 ACM/IEEE Conference on Supercomputing, SC 2001
PB - Association for Computing Machinery
T2 - 2001 ACM/IEEE Conference on Supercomputing, SC 2001
Y2 - 10 November 2001 through 16 November 2001
ER -