Distribution of regular grids incoherent for 16 processes

ParaView Support
1

Hi all,

I am using Paraview 5.11.0 and MPI 4.0.3.

I have noticed a strange behavior on the distribution of regular grids.
In my example, i use the pvserver in mpi for 16 processes:

mpirun -n 16 pvserver

paraview

I then create a data set using the Wavelet filter. The bounds of the wavelet are: [0, 255, 0, 255, 0, 255]. This creates a cube. In the image below, I have shown the vtkProcessId field for points. What can be seen is that the distribution is not the same for all processes.

Screenshot from 2023-06-07 09-58-22
Screenshot from 2023-06-07 09-58-221335×684 28.2 KB

For example, for the two dark blue processes in the upper left corner, in the front, the division between the two processes is done according to the y direction, whereas for the two yellow blocks in the upper right corner, the division is done following the z direction.

Is this a bug, or is this normal? My understanding of the distribution algorithm is that the division is done following the direction for which the block has the biggest dimension. In this case, it is a cube therefore when going from 8 to 16 processes, all dimensions are of the same size (all are cubes for 8 processes). Is there a priority put in place so that when there is an equality in dimensions, some direction is chosen rather than another one?

Another thing that I noticed, when using the ResampleToImage filter on the previous data set, to create a 256^3 cube (so a cube of exactly the same dimensions as before), Paraview produces the following output:

image
image1364×729 35.7 KB

This is what I would like to always have for my application.

Is this a bug or is this normal? Is there a way to force the second distribution to always happen?

Thanks for any help.

2

It is down to the Wavelet source (vtkRTAnalyticSource) implementation to distribute itself. Are you actually using this source in your workflow ?

3

I am using the Wavelet filter available in Paraview.

However, my problem seems to be more related to the distribution of regular grids, because it appears with other data sets that I work on (for example: this one). The problematic characteristic seems to be the cubic dimensions of a power of two.

4

If memory serves (and it very well might not) the VTK executive uses this code to pick a decent partitioning for structured datasets like the two you describe to assign domains to different ranks in parallel.

https://gitlab.kitware.com/vtk/vtk/-/blob/master/Common/ExecutionModel/vtkExtentTranslator.cxx#L152

There were ways to provide your own partitioning scheme but those were rarely used and might no longer exist.

5

Thank you for your answer, it was very helpful. Here is what I’ve understood from looking into the vtkExtentTranslator:

By default, the splitting will happen according to the cells, and not the points. This means that my 256^3 cube is actually a 255^3 cube, and in this case, the partitioning indeed produces blocks that are not all equal, resulting in my first image.

For some reason, the ResampleToImage uses the points and not the cells, which creates the partitioning in the second image.

The solution to my problem would be to either create a new file reader that uses points and not cells or to redistribute the data set to split it using points.