I am trying to perform live visualization using Catalyst. I was having some issues with my simulation code, so I decided to test the Catalyst grid example.
https://gitlab.kitware.com/paraview/paraview/-/tree/master/Examples/Catalyst2/CxxImageDataExample
I have slightly modify the example, adding a sleep after each “simulation” iteration. I have found that if I sleep for less than 100ms, I can’t perform Live Visualization because ParaView get completely stuck.
My catalyst_pipeline is also a modified version, here is the code of the python script.
# script-version: 2.0
# Catalyst state generated using paraview version 5.12.0-RC2
import paraview
paraview.compatibility.major = 5
paraview.compatibility.minor = 12
from paraview.simple import *
# Paraview pipeline, just do nothing
grid = TrivialProducer(registrationName='grid')
from paraview import catalyst
options = catalyst.Options()
options.GlobalTrigger = 'Time Step'
options.EnableCatalystLive = 1
options.CatalystLiveURL = 'localhost:22222'
options.CatalystLiveTrigger = 'Time Step'
def catalyst_execute(info):
global garments
grid.UpdatePipeline()
As you can see, I do nothing in the python script.
In ParaView, if I am simulating with a sleep of 80ms after each iteration, I can still receive data, but as soon as I try to visualiza the mesh, ParaView get stuck or it receives data less frequently, sometimes missing a lot of steps.
Both the C++ code and ParaView are runnig on the same machine, with a 5700X and a nvidia RTC 2070 Super. I would be surprised if ParaView can’t handle the example data (or even the example I was testing with my original soimulation, which was just 30 vertices in a square patch).
The issue seems to be related to rendering. If I add a CSV extractor in catalyst pipeline, it still goes fast, but if I render from the catalyst pipeline, adding a PNG extractor, then it goes very slow.
I have been doing some more testing, measuring time from the start of the simulation iteration to several points. The time is accumulative from the beginning of the simulation iteration
Simulate is some stuff I am doing to produce the data
Prepare is just the preparation of the conduit nodes
Catalyst execute is the execution of the python catalyst_execute method
Execute is the time it takes from the start of the simulation frame until after the catalyst_execute call in C++
Results is the same for catalyst_results
Final is the end of the simulation iteration. I sleep for 16ms just before this
catalyst_execute(conduit_cpp::c_node(&exec_params));
std::cout << "Execute: " << std::chrono::duration_cast<std::chrono::milliseconds>(std::chrono::high_resolution_clock::now() - start).count() << std::endl;
conduit_cpp::Node results_params;
catalyst_results(conduit_cpp::c_node(&results_params));
std::cout << "Results: " << std::chrono::duration_cast<std::chrono::milliseconds>(std::chrono::high_resolution_clock::now() - start).count() << std::endl;
std::this_thread::sleep_for(std::chrono::milliseconds(16));
std::cout << "Final: " << std::chrono::duration_cast<std::chrono::milliseconds>(std::chrono::high_resolution_clock::now() - start).count() << std::endl;
If I dont connect ParaView to Catalyst, then the script does nothing, and everything goes super fast
Simulate: 0
Prepare: 0
Catalyst execute: 0.0034
Execute: 0
Results: 0
Final: 16
If I connect ParaView to Catalyst, the execute call in C++ takes around 84ms.
Simulate: 0
Prepare: 0
Catalyst execute: 0.00848
Execute: 84
Results: 84
Final: 100
I understand that for a simulation that takes 1 hour for each simulation step, adding 84ms to perform Live Visualization is not a lot, but my simulations are real time, and adding 84ms for a simulation step that takes less than 40ms, is really a lot.
I have tested with a mesh example that occupies 600KB and another one with 2MB, with same results, around 80ms for the catalyst_execute method in C++.
Is this normal?
I have been doing some more testing around this issue. Apparently, its related to Problems with Catalyst2 and Paraview Live Visualization - #3 by Lucas_Givord
I produce data faster than ParaView can process it, and it freezes ParaView. Is there anyway to block the Catalyst pipeline until ParaView has finished processing the data?
I had completely forgotten sorry, I don’t think that ParaviewCatalyst supports such context with a “small” simulation (when the simulation calculation takes less time than the visualization computation) currently.
Could you please open an issue on ParaView about that ? https://gitlab.kitware.com/paraview/paraview/-/issues
Thanks
Sure. Ill open an issue on gitlab. Is there anyway to pause at each simulation step? That way, at least I would control the pace of the simulation and be able to still watch it live.
Other way would be to let ParaView GUI tell Catalyst when it has finished processing data, so I can sleep Catalyst until that happens. Is such thing possible?
Sadly not AFAIK, the only way to handle that would be to use the breakpoint feature in the GUI of ParaView but it currently doesn’t work until we found a way to fix it : https://gitlab.kitware.com/paraview/paraview/-/issues/21735
maybe @nicolas.vuaille have some ideas?
Even with the breakpoint functionality, I could only set ask to stop on frame X, but not at each timestep. For sure I can open the brakpoint dialog and keep updating the frame to stop, but that would be terribly slow from a user perspective.
I guess it would ideal to have some playback controls, that I can hit hit play/stop/step…
1 Like
Indeed, your right. Thanks for having open the issue
I guess for now I will create a Python plugin and will add the play/pause buttons to be able to handle this. Is there any tutorial on how to add buttons to Paraview GUI? I haven’t found any example adding buttons
I have already looked at the Steering parameters, but I am not sure if that its what I want. Are those parameters modified from the GUI or by data produced by the Catalyst script?
Would that allow me to pass a parameter from the GUI to the Catalyst script?
Steering is intended to send data from the ParaView side to the simulation.
In the example you have a plugin xml to load on client side (and on server side in the script), so you will have a pipeline proxy with a GUI to modify values. Then simulation can read that back.
I am not sure how I am suppose to run the example. I get a python error
./bin/CxxSteeringExample ../catalyst_steering_proxies.xml ../catalyst_pipeline.py IOT ✘ seddi@b57455101613 09:47:59
( 0.110s) [pvbatch ] ParaViewCatalyst.cxx:291 INFO| Catalyst plugin loaded: ../catalyst_steering_proxies.xml
executing catalyst_pipeline
( 0.590s) [pvbatch ]vtkSIProxyDefinitionMan:517 ERR| vtkSIProxyDefinitionManager (0x55eddaec8660): No proxy that matches: group=extract_writers and proxy=steering were found.
Traceback (most recent call last):
File "/workspace/ParaView/lib/python3.10/site-packages/paraview/catalyst/v2_internals.py", line 82, in import_and_validate
m = importlib.import_module(modulename)
File "/workspace/ParaView/lib/python3.10/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/home/seddi/paraview/Examples/Catalyst2/CxxSteeringExample/build/../catalyst_pipeline.py", line 13, in <module>
steering_extractor = CreateExtractor('steering', filterArray, registrationName='steerchannel')
File "/workspace/ParaView/lib/python3.10/site-packages/paraview/simple.py", line 486, in CreateExtractor
RegisterExtractor(extractor)
File "/workspace/ParaView/lib/python3.10/site-packages/paraview/catalyst/detail.py", line 80, in RegisterExtractor
_get_active_helper().RegisterExtractor(extractor.SMProxy)
AttributeError: 'NoneType' object has no attribute 'SMProxy'