Can Catalyst stream simulation data to multiple independent pvserver ?

In Situ Support
1

I am working on an in-situ visualization pipeline using AdiosReplay/Catalyst2 and ParaView. My setup involves a running simulation that streams data via ADIOS2/SST to AdiosReplay, which then uses Catalyst to forward the data for visualization.

My question is: is it possible to have Catalyst stream the simulation data to multiple independent pvserver instances running on different nodes, and then connect a single ParaView GUI client to all of them simultaneously to visualize the simulation in real time?

If this is not directly supported, what would be the recommended architecture for distributing the visualization workload across multiple nodes while keeping interactive access from a ParaView GUI?

2

Hello @Ayyoub_Boulahri

is it possible to have Catalyst stream the simulation data to multiple independent pvserver instances running on different nodes

Yes, you need to first set the numbers of readers to a higher value in your ADIOS2 XML configuration file <parameter key="RendezvousReaderCount" value="2" />, then run multiple AdiosReplay instances, to which you connect different ParaView servers. There is such an example in the AdiosCatalyst repository: https://gitlab.kitware.com/paraview/adioscatalyst/-/blob/main/Examples/Multimesh/testing/test_multiple_reader.py?ref_type=heads

and then connect a single ParaView GUI client to all of them simultaneously to visualize the simulation in real time?

I don’t think a single ParaView client can connect simultaneously to multiple servers

If this is not directly supported, what would be the recommended architecture for distributing the visualization workload across multiple nodes while keeping interactive access from a ParaView GUI?

I would need more information about your use case, but it could be possible to run multiple AdiosRepaly endpoints using MPI, that launch multiple pvserver processes, which coordinate a single ParaView session, and have your datasets partitioned and split across multiple processes.

3

Actually a single client can connect to multiple pvserver using the –-multi-serversoption. However, if possible, I would recommend using pvserver with MPI to distribute the data rather than multiple pvserver instances with different adresses. The multiple reader example shows how to run a different Catalyst pipeline on the 2 endpoints.

4

Thank you for the clarification about --multi-servers, that’s very useful to know!

Regarding the architecture, even in the case of a very large simulation with massive datasets, would you still recommend a single pvserver with MPI over multiple independent pvserver instances?

5

yes, MPI is the way to go for large datasets or HPC

6

Thank you so much @Louis_Gombert for your time and explanations, it was extremely helpful!

7

No problem, let me know if you run into more issues with AdiosCatalyst. This is still an experimental technology.