Spatio-temporal parallelism 5.11

ParaView Support
1

Hello everyone.

I would to ask about the “spatio-temporal parallelism” discussed in these two posts:

https://www.kitware.com/spatio-temporal-parallelism-with-paraview/

https://www.kitware.com/paraview-spatio-temporal-parallelism-revised/

First of all, I explain my case. I am working in a project in which I have a big amount of heavy timesteps. Normally, Paraview process all these timesteps sequentially, taking around one minute for timestep. The result is that it takes approximately 8 or 10 hours to get all the render images I need for making animations later. The bottleneck here is only to read the timestep, since once it has been read, all the filters and and images exportation are done quickly. As I have numerous cases, the complete postprocessing can take several days and I started to think in a way to optimize this process. As each timestep is independent itself, I searched for a way to process the timesteps in a parallel way. I count on a cluster with 128 cores and 512 Gb of memory, so I thought I could reduce the postprocessing time.

By searching on the web, I found the two posts I have attached at the beginning. Basically, they do exactly what I want. It seems that a group created a plugin in 2013 that allows to do this, and later in 2018 (the second post), the plugin was integrated as part of the Paraview GUI.

I have successfully proofed the process with Paraview 5.7, which is the one where the plugin was implemented, and it works perfectly. For doing this, it is necessary to create your pipeline, specifying the necessary images exports and click on the option “Export temporal script” of the “Catalyst” submenu. Later, this script is executed with “pvbatch”. My first question is:

  • How can I do this in Paraview 5.11? I assume that the process should be similar but using the new way to create exports (“Extractors”) and saving the catalyst state, but the temporal option has disappeared from the GUI (or I cannot see it), so the state will not have this information. I have seen that the script used for this plugin in 5.7 (“spatiotemporalparallelism.py”) remains unmodified in the bin folder in 5.11, so it should exist a way to use it.
  • By assuming that the process could be translated to Paraview 5.11, I have a second problem. As I have said, in 5.7 it works, but not directly with my project. In the process you have to specify the base name of each timestep file as it is designed to read each timestep in a separated file. However, my simulation data is generated in ANSYS Fluent and it is exported in Ensight Gold format, so I have only one file (the .case) with the information of all timesteps which must be read. Perhaps I have to create a different .case for each timestep?

Any help would be appreciated. Thanks in advance.

2

Hi @luismhy

The “Export Temporal Script” from 2018 is indeed gone but the feature itself is still present in the python logic and can be used.
https://gitlab.kitware.com/paraview/paraview/-/blob/master/Wrapping/Python/paraview/spatiotemporalparallelism.py

TBH, this logic is only a helper and creating a python script that performs simple temporal parallelism should be fairly simple.

The idea is just to distribute the timesteps accros ranks and update the pipeline as many times as needed, while using --symmetric option.

I did not try it though, so if you try and have feedbacks I would be interested!

FYI @cory.quammen @nicolas.vuaille @jfausty @Dave_DeMarle @Andy_Bauer

3

Hi @mwestphal

Many thanks for the answer.

You are right, with the temporal script exported in Paraview 5.7, I have successfully done the temporal parallelization for a transient simulation in 5.11, using the format .pvti.

However, as commented in my original question, my goal is to do the same with Ensight Gold Cases from FLUENT, and this script is not working for this kind of files. Therefore, I think the best option is to create a simple script to do the parallelization, as you have commented.

I am not an expert in creating scripts for paraview, so any help would be appreciated. Basically, what I have doing right now is try to replicate the temporal parallelization with the same .pvti simulation previously mentioned (because it is clear that this is possible). If this works, I will continue with the Ensight Gold files.

I have done something like this:

“”””””””””””””””””””””””””””””””””””””””””””””””””””””””””””””””””””””””””””””””””””””””

from mpi4py import MPI

*from paraview.simple import **

from paraview import catalyst

from paraview.simple import SaveExtractsUsingCatalystOptions

if name == “main”:

comm = MPI.COMM_WORLD

rank = comm.Get_rank()

size = comm.Get_size()

total_timesteps = 720

timesteps_per_process = total_timesteps // size

# Calculate timesteps assigned to this process

start_timestep = rank * timesteps_per_process

end_timestep = start_timestep + timesteps_per_process - 1

for timestep in range(start_timestep, end_timestep + 1):

# Generate string with the timestep (6 digits)

timestep_str = f"{timestep:06d}"

# Generate filename to read

filename_timestep = f"filename_{timestep_str}.pvti"

filename_path=f"path_to_the cwd\filename_{timestep_str}.pvti"

output_image=f"image{timestep_str}.png"

.

.

.

[PARAVIEW PIPELINE]

“”””””””””””””””””””””””””””””””””””””””””””””””””””””””””””””””””””””””””””””””””””””””

So, I have created a simple paraview catalyst state reading the first timestep of the .pvti simulation and doing some slice and creating a view to extract an image. This pipeline is inserted in a for loop which is parallelized with MPI, as can be seen in the portion of code. Each iteration will read the correspondent timestep according to the distribution done (and will create an image file with the name of the timestep).

It is very simple, and I do not know if this is the correct way to proceed. When I execute this script with mpiexec and 1 processor it works. But if I use more than 1 processor, the parallelization does not work (symmetric option is enabled). For example, when I used the temporal script exported with the logic spatiotemporalparalellism.py using 10 cores, 10 images were generated at a time, later 10 more…etc. With my approach, the 10 processes are launched but the images are generated 1 by 1, and I do not know the reason.

Many thanks in advance.