The Template Task Graph (TTG) - An emerging practical dataflow programming paradigm for scientific simulation at extreme scale

G. Bosilca; R. J. Harrison; T. Herault; M. M. Javanmard; P. Nookala; E. F. Valeev

doi:10.1109/ESPM251964.2020.00011

The Template Task Graph (TTG) - An emerging practical dataflow programming paradigm for scientific simulation at extreme scale

G. Bosilca
, R. J. Harrison
, T. Herault
, M. M. Javanmard
, P. Nookala
, E. F. Valeev

Applied Mathematics and Statistics

University of Tennessee
Stony Brook University
Virginia Polytechnic Institute and State University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

14 Scopus citations

Abstract

We describe TESSE, an emerging general-purpose, open-source software ecosystem that attacks the twin challenges of programmer productivity and portable performance for advanced scientific applications on modern high-performance computers. TESSE builds upon and extends the ParsecDAG/-dataflow runtime with a new Domain Specific Languages (DSL) and new integration capabilities. Motivating this work is our belief that such a dataflow model, perhaps with applications composed in domain specific languages, can overcome many of the challenges faced by a wide variety of irregular applications that are poorly served by current programming and execution models. Two such applications from many-body physics and applied mathematics are briefly explored. This paper focuses upon the Template Task Graph (TTG), which is TESSE's main C++ Api that provides a powerful work/data-flow programming model. Algorithms on spatial trees, block-sparse tensors, and wave fronts are used to illustrate the API and associated concepts, as well as to compare with related approaches.

Original language	English
Title of host publication	Proceedings of ESPM2 2020
Subtitle of host publication	5th International IEEE Workshop on Extreme Scale Programming Models and Middleware, Held in conjunction with SC 2020: The International Conference for High Performance Computing, Networking, Storage and Analysis
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	48-54
Number of pages	7
ISBN (Electronic)	9781665422840
DOIs	https://doi.org/10.1109/ESPM251964.2020.00011
State	Published - Nov 2020
Event	5th IEEE/ACM International IEEE Workshop on Extreme Scale Programming Models and Middleware, ESPM2 2020 - Virtual, Atlanta, United States Duration: Nov 11 2020 → …

Publication series

Name	Proceedings of ESPM2 2020: 5th International IEEE Workshop on Extreme Scale Programming Models and Middleware, Held in conjunction with SC 2020: The International Conference for High Performance Computing, Networking, Storage and Analysis

Conference

Conference	5th IEEE/ACM International IEEE Workshop on Extreme Scale Programming Models and Middleware, ESPM2 2020
Country/Territory	United States
City	Virtual, Atlanta
Period	11/11/20 → …

Keywords

DAG
dataflow
exascale
graph
high-performance computing
workflow

Access to Document

10.1109/ESPM251964.2020.00011

Cite this

Bosilca, G., Harrison, R. J., Herault, T., Javanmard, M. M., Nookala, P., & Valeev, E. F. (2020). The Template Task Graph (TTG) - An emerging practical dataflow programming paradigm for scientific simulation at extreme scale. In Proceedings of ESPM2 2020: 5th International IEEE Workshop on Extreme Scale Programming Models and Middleware, Held in conjunction with SC 2020: The International Conference for High Performance Computing, Networking, Storage and Analysis (pp. 48-54). Article 9307054 (Proceedings of ESPM2 2020: 5th International IEEE Workshop on Extreme Scale Programming Models and Middleware, Held in conjunction with SC 2020: The International Conference for High Performance Computing, Networking, Storage and Analysis). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ESPM251964.2020.00011

Bosilca, G. ; Harrison, R. J. ; Herault, T. et al. / The Template Task Graph (TTG) - An emerging practical dataflow programming paradigm for scientific simulation at extreme scale. Proceedings of ESPM2 2020: 5th International IEEE Workshop on Extreme Scale Programming Models and Middleware, Held in conjunction with SC 2020: The International Conference for High Performance Computing, Networking, Storage and Analysis. Institute of Electrical and Electronics Engineers Inc., 2020. pp. 48-54 (Proceedings of ESPM2 2020: 5th International IEEE Workshop on Extreme Scale Programming Models and Middleware, Held in conjunction with SC 2020: The International Conference for High Performance Computing, Networking, Storage and Analysis).

@inproceedings{9db59224127f42f7ae5a2713f0fe08f4,

title = "The Template Task Graph (TTG) - An emerging practical dataflow programming paradigm for scientific simulation at extreme scale",

abstract = "We describe TESSE, an emerging general-purpose, open-source software ecosystem that attacks the twin challenges of programmer productivity and portable performance for advanced scientific applications on modern high-performance computers. TESSE builds upon and extends the ParsecDAG/-dataflow runtime with a new Domain Specific Languages (DSL) and new integration capabilities. Motivating this work is our belief that such a dataflow model, perhaps with applications composed in domain specific languages, can overcome many of the challenges faced by a wide variety of irregular applications that are poorly served by current programming and execution models. Two such applications from many-body physics and applied mathematics are briefly explored. This paper focuses upon the Template Task Graph (TTG), which is TESSE's main C++ Api that provides a powerful work/data-flow programming model. Algorithms on spatial trees, block-sparse tensors, and wave fronts are used to illustrate the API and associated concepts, as well as to compare with related approaches.",

keywords = "DAG, dataflow, exascale, graph, high-performance computing, workflow",

author = "G. Bosilca and Harrison, \{R. J.\} and T. Herault and Javanmard, \{M. M.\} and P. Nookala and Valeev, \{E. F.\}",

note = "Publisher Copyright: {\textcopyright} 2020 IEEE.; 5th IEEE/ACM International IEEE Workshop on Extreme Scale Programming Models and Middleware, ESPM2 2020 ; Conference date: 11-11-2020",

year = "2020",

month = nov,

doi = "10.1109/ESPM251964.2020.00011",

language = "English",

series = "Proceedings of ESPM2 2020: 5th International IEEE Workshop on Extreme Scale Programming Models and Middleware, Held in conjunction with SC 2020: The International Conference for High Performance Computing, Networking, Storage and Analysis",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "48--54",

booktitle = "Proceedings of ESPM2 2020",

}

Bosilca, G, Harrison, RJ, Herault, T, Javanmard, MM, Nookala, P & Valeev, EF 2020, The Template Task Graph (TTG) - An emerging practical dataflow programming paradigm for scientific simulation at extreme scale. in Proceedings of ESPM2 2020: 5th International IEEE Workshop on Extreme Scale Programming Models and Middleware, Held in conjunction with SC 2020: The International Conference for High Performance Computing, Networking, Storage and Analysis., 9307054, Proceedings of ESPM2 2020: 5th International IEEE Workshop on Extreme Scale Programming Models and Middleware, Held in conjunction with SC 2020: The International Conference for High Performance Computing, Networking, Storage and Analysis, Institute of Electrical and Electronics Engineers Inc., pp. 48-54, 5th IEEE/ACM International IEEE Workshop on Extreme Scale Programming Models and Middleware, ESPM2 2020, Virtual, Atlanta, United States, 11/11/20. https://doi.org/10.1109/ESPM251964.2020.00011

The Template Task Graph (TTG) - An emerging practical dataflow programming paradigm for scientific simulation at extreme scale. / Bosilca, G.; Harrison, R. J.; Herault, T. et al.
Proceedings of ESPM2 2020: 5th International IEEE Workshop on Extreme Scale Programming Models and Middleware, Held in conjunction with SC 2020: The International Conference for High Performance Computing, Networking, Storage and Analysis. Institute of Electrical and Electronics Engineers Inc., 2020. p. 48-54 9307054 (Proceedings of ESPM2 2020: 5th International IEEE Workshop on Extreme Scale Programming Models and Middleware, Held in conjunction with SC 2020: The International Conference for High Performance Computing, Networking, Storage and Analysis).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - The Template Task Graph (TTG) - An emerging practical dataflow programming paradigm for scientific simulation at extreme scale

AU - Bosilca, G.

AU - Harrison, R. J.

AU - Herault, T.

AU - Javanmard, M. M.

AU - Nookala, P.

AU - Valeev, E. F.

PY - 2020/11

Y1 - 2020/11

N2 - We describe TESSE, an emerging general-purpose, open-source software ecosystem that attacks the twin challenges of programmer productivity and portable performance for advanced scientific applications on modern high-performance computers. TESSE builds upon and extends the ParsecDAG/-dataflow runtime with a new Domain Specific Languages (DSL) and new integration capabilities. Motivating this work is our belief that such a dataflow model, perhaps with applications composed in domain specific languages, can overcome many of the challenges faced by a wide variety of irregular applications that are poorly served by current programming and execution models. Two such applications from many-body physics and applied mathematics are briefly explored. This paper focuses upon the Template Task Graph (TTG), which is TESSE's main C++ Api that provides a powerful work/data-flow programming model. Algorithms on spatial trees, block-sparse tensors, and wave fronts are used to illustrate the API and associated concepts, as well as to compare with related approaches.

AB - We describe TESSE, an emerging general-purpose, open-source software ecosystem that attacks the twin challenges of programmer productivity and portable performance for advanced scientific applications on modern high-performance computers. TESSE builds upon and extends the ParsecDAG/-dataflow runtime with a new Domain Specific Languages (DSL) and new integration capabilities. Motivating this work is our belief that such a dataflow model, perhaps with applications composed in domain specific languages, can overcome many of the challenges faced by a wide variety of irregular applications that are poorly served by current programming and execution models. Two such applications from many-body physics and applied mathematics are briefly explored. This paper focuses upon the Template Task Graph (TTG), which is TESSE's main C++ Api that provides a powerful work/data-flow programming model. Algorithms on spatial trees, block-sparse tensors, and wave fronts are used to illustrate the API and associated concepts, as well as to compare with related approaches.

KW - DAG

KW - dataflow

KW - exascale

KW - graph

KW - high-performance computing

KW - workflow

UR - https://www.scopus.com/pages/publications/85099591684

U2 - 10.1109/ESPM251964.2020.00011

DO - 10.1109/ESPM251964.2020.00011

M3 - Conference contribution

AN - SCOPUS:85099591684

T3 - Proceedings of ESPM2 2020: 5th International IEEE Workshop on Extreme Scale Programming Models and Middleware, Held in conjunction with SC 2020: The International Conference for High Performance Computing, Networking, Storage and Analysis

SP - 48

EP - 54

BT - Proceedings of ESPM2 2020

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 5th IEEE/ACM International IEEE Workshop on Extreme Scale Programming Models and Middleware, ESPM2 2020

Y2 - 11 November 2020

ER -

Bosilca G, Harrison RJ, Herault T, Javanmard MM, Nookala P, Valeev EF. The Template Task Graph (TTG) - An emerging practical dataflow programming paradigm for scientific simulation at extreme scale. In Proceedings of ESPM2 2020: 5th International IEEE Workshop on Extreme Scale Programming Models and Middleware, Held in conjunction with SC 2020: The International Conference for High Performance Computing, Networking, Storage and Analysis. Institute of Electrical and Electronics Engineers Inc. 2020. p. 48-54. 9307054. (Proceedings of ESPM2 2020: 5th International IEEE Workshop on Extreme Scale Programming Models and Middleware, Held in conjunction with SC 2020: The International Conference for High Performance Computing, Networking, Storage and Analysis). doi: 10.1109/ESPM251964.2020.00011

The Template Task Graph (TTG) - An emerging practical dataflow programming paradigm for scientific simulation at extreme scale

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this