TY - GEN
T1 - Enhancing DataRaceBench for Evaluating Data Race Detection Tools
AU - Verma, Gaurav
AU - Shi, Yaying
AU - Liao, Chunhua
AU - Chapman, Barbara
AU - Yan, Yonghong
N1 - Publisher Copyright:
© 2020 IEEE.
PY - 2020/11
Y1 - 2020/11
N2 - DataRaceBench is a dedicated benchmark suite to evaluate tools aimed to find data race bugs in OpenMP programs. Since its initial release in 2017, DataRaceBench has been widely used by tool developers to find the strengths and limitations of their tools. The results also provide an apple-to-apple comparison of the state-of-the-art of data race detection tools. In this paper, we discuss our latest efforts to enhance DataRaceBench. In particular, we have added support for Fortran language and some of the newest OpenMP 5.0 language features. We have also added new kernels representing new patterns from literature and other benchmarks (e.g., NAS Parallel Benchmark). To reduce duplicated code patterns in the benchmark suite, we have designed a distance-based code similarity analysis, combining both static and dynamic code features. Finally, we dockerize tools and streamline the entire benchmarking process to quickly generate a dashboard showing the state-of-the-art of data race detection of OpenMP programs. The enhanced DataRaceBench is released as v 1.3.0, with 222 newly added benchmarks. 56 of them are in C, and the remaining 166 are in Fortran, reproducing the C programs' nature. Our experiments show that this new version can spot more limitations of the current data race detection tools, with significantly reduced user efforts needed to run experiments.
AB - DataRaceBench is a dedicated benchmark suite to evaluate tools aimed to find data race bugs in OpenMP programs. Since its initial release in 2017, DataRaceBench has been widely used by tool developers to find the strengths and limitations of their tools. The results also provide an apple-to-apple comparison of the state-of-the-art of data race detection tools. In this paper, we discuss our latest efforts to enhance DataRaceBench. In particular, we have added support for Fortran language and some of the newest OpenMP 5.0 language features. We have also added new kernels representing new patterns from literature and other benchmarks (e.g., NAS Parallel Benchmark). To reduce duplicated code patterns in the benchmark suite, we have designed a distance-based code similarity analysis, combining both static and dynamic code features. Finally, we dockerize tools and streamline the entire benchmarking process to quickly generate a dashboard showing the state-of-the-art of data race detection of OpenMP programs. The enhanced DataRaceBench is released as v 1.3.0, with 222 newly added benchmarks. 56 of them are in C, and the remaining 166 are in Fortran, reproducing the C programs' nature. Our experiments show that this new version can spot more limitations of the current data race detection tools, with significantly reduced user efforts needed to run experiments.
KW - Benchmarks
KW - Data Races
KW - OpenMP
KW - Tools
UR - https://www.scopus.com/pages/publications/85099385401
U2 - 10.1109/Correctness51934.2020.00008
DO - 10.1109/Correctness51934.2020.00008
M3 - Conference contribution
AN - SCOPUS:85099385401
T3 - Proceedings of Correctness 2020: 4th International Workshop on Software Correctness for HPC Applications, Held in conjunction with SC 2020: The International Conference for High Performance Computing, Networking, Storage and Analysis
SP - 20
EP - 30
BT - Proceedings of Correctness 2020
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 4th International Workshop on Software Correctness for HPC Applications, Correctness 2020
Y2 - 11 November 2020
ER -