Martin Berzins, Christopher Earl, Qingyu Meng, Matt Might, John Schmidt, James Sutherland, J. Davison de St. Germain, Alan Humphrey

Driving Problem

The driving problem for the use of Uintah under the PSAAP II grant is to simulate state-of-the-art electricity producing coal fired boilers. Figure 1 contains a visualization (showing the density of 90 micron particles) of a preliminary version of the simulation of an experimental boiler at the Alstom Boiler Simulation Facility (BSF).

UINTAH-X - Advancing Progress towards Exascale Computer Simulations

  1. UINTAH-X Runtime System (RTS) (Berzins, Meng, Schmidt, Humphrey) Uintah uses an asynchronous RTS (based on directed acyclic graphs (DAG)) to provide scalable, exascale simulations.
  2. Wasatch and the NEBO Domain Specific Language (DSL) (Might, Sutherland, Earl) This prototype DSL system shows great promise in generating efficient DAG extensible exascale simulation code from high-level complex physics expressions.
Figure 1. Particle density
90-micron 30-micron 60-micron
90 micron particles 30 micron particles 60 micron particles


Uintah-X Runtime System

runtimeUintah-X Architecture

  • Component-based design. The Uintah system allows multiple complex simulation components to co-exist and leverage the a common scheduling and communication framework. Components include ARCHES, ICE, MPM, and Wasatch.
  • Abstract task-graph. Uintah uses a directed acyclic task-graph based approach to manage the complexity of both processing and communication.
  • Runtime System. The Uintah RTS provides asynchronous out-of-order task execution and work stealing, along with overlapping communication and computation. Tasks may be executed on either CPU or GPU resources.
  • Scalable I/O via Visus PIDX. Exascale simulations produce a huge amount of data. Uintah will leverage Visus PIDX to provide for real-time data analysis and visualization.

"Exascale programming will require prioritization of critical-path and non-critical path tasks, adaptive directed acyclic graph scheduling of critical-path tasks, and adaptive rebalancing of all tasks......"
[Brown et al. Exascale Report]



Unified Heterogeneous Scheduler & Runtime




Exascale Computing Goal

  • Target: 100 million-way parallelism for one simulation
  • Largest machines in the world 10 years from now



Wasatch & Nebo EDSL

edsl-1 edsl-2


Embedded Domain Specific Language (EDSL) Goals

  • edsl-goalExpressive syntax (matlab-style array operations)
    • Programmer expresses intent (problem structure) - not implementation.
  • High performance
    • Should match hand-tuned code in performance
  • Extensible
    • Insulate programmer from architecture changes (e.g. multicore → GPU → ...)
    • EDSL "back-end" compiles into code for target architecture
  • "Plays well with others"
    • Allow programmer to write in C++ and inter-operate with EDSL
    • Not an "all-or-none" approach
    • Allows concurrent development of EDSL and application codes
    • Adapt to appropriate data structures without impacting application programmer.




Performance and Scalability

Speedup using DSL* relative to other Uintah codes Weak scaling on Titan
ice-arches titan
*Comparison to ICE and ARCHES, sister codes in Uintah, on a 3D Taylor-Green vortex problem. Run on a single processor. "1" indicates perfect weak scaling.