To extend Catalyst (1) with In-Transit capabilities, we are designing an implementation based on the ADIOS2 library (2).
Context
Catalyst is the favored tool to perform in-situ analysis with ParaView, running your pipeline on the fly without read/write on disk.
As of today, it does not allow In-Transit operations, i.e. moving data to dedicated MPI ranks for analysis.
ADIOS2 is “A Unified High-performance I/O Framework”. It supports different ways of communication, known as engine. The Sustainable Staging Transport (SST, 3) is the “Streaming-oriented” engine where “the number of reader ranks can differ from the number of writer ranks”, which is typically what we want here.
Design
This work is splitted in two parts.
On the first side, we should create an ADIOS2 implementation of the Catalyst API. The simulation will use it as any Catalyst implementation. This will run on the Simulation nodes and any code already Catalyst-capable will benefit from this catalyst-adios variant without any need of further development.
On the Analysis nodes, we will have some kind of “catalyst replay” executable, to read the ADIOS2 streamed data and run the ParaView analysis.
As ADIOS2 and Conduit are both data descriptor, our goal is to convert from Conduit to ADIOS2 in the library and from ADIOS2 to Conduit in the “replay” executable, delegating the actual VTK objects creation to the existent code (vtkConduitSource).