Initializing a new pipeline during a run with ParaView Catalyst

In Situ Support
1

Dear all,

As far as I can tell, we cannot initialize a new Catalyst pipeline during a run. catalyst_initialize() can only be called once, and that’s the place where we tell ParaView about our scripts.

It would be useful for me to be able to add scripts after I have called catalyst_initialize(). From a quick glance at the vtkInSituInitializationHelper I don’t see any fundamental problem with adding a pipeline mid-run.

Any thoughts?

2

Would calling catalyst_finalize and then catalyst_initialize again with different scripts work? I haven’t tried though.

3

I would not mind doing that, but currently this does not work. The second initialize is still ignored by ParaView (and ParaView warns that initialize cannot be called again).

4

If we were to handle this case by changing ParaView, it would be better to call catalyst_finalize first and make everything clean (reset static variables basically) for the next catalyst_initialize call to work properly. Would you mind opening an issue on the GitLab tracker?

5

Louis,

Thanks for your quick responses. Here is the issue: https://gitlab.kitware.com/paraview/paraview/-/work_items/23273

6

I would be against allowing re-initialize unless a very clear use-case were presented. The design is that catalyst follows the adaptor/simulation lifecycle. There is nothing preventing registration of all the possible pipelines you would want to use up-front, and then selectively executing them

auto pipelines = state["pipelines"];
pipelines.append().set("forces");  // every step, cheap
if (timestep % 100 == 0) {
  pipelines.append().set("vorticity");  // every 100 steps, expensive
}
if (timestep % 1000 == 0) {
  pipelines.append().set("screenshots");  // every 1000 steps, very expensive
}

By default, if state/pipelines isnt populated catalyst executes all of the registered pipelines. Maybe I am missing the actual use-case you have in mind.

7

Right, all pipelines must be registered up front, so no pipelines can be added during the simulation. We can start registered pipelines later in the simulation but we cannot register them during the simulation.

A simple use case is “I forgot to register a pipeline.” A more realistic use case is something like “Looking at the slices being generated, this area seems interesting, I wish I could add a contour of vorticity here”. Indeed, the ability to gain insights during the simulation is one of the advantage of in-situ processing compared to post-processing.

That’s actually why I’m asking. We allow users to change inputs to our software during the simulation. So adding a slice during the simulation is in line with our simulation life cycle. I would like to offer that feature to our users, which is why I am asking for a matching Catalyst feature.

There are ways of doing this currently. We can register empty slots and fill them later with a real pipeline. We can also register a single pipeline that is just a daemon, and the daemon takes care of monitoring user inputs and adding and deleting pipelines. ParaView supports this, the question is what is the best way to do it through Catalyst?

I’m not necessarily asking for a re-initialize. I’m asking for an add/delete interface. The how is up to you. If we do it through a re-initialize, that works for me. (I don’t really need a delete, since as you say, you can simply stop executing an existing pipeline, but that would be cleaner).

I do think that the concept of finalize cleans up everything, and initialize can be called again is easy to understand for user-developers.