Implications of 'Nvidia Index does not distribute data sets in vtk format with pvserver'

ParaView Support
1

I managed to set up a working reverse client-server connection using paraview 5.7.0 installed from the superbuild and MPI. I run on a Debian GNU/Linux 10 ‘buster’ and Cuda 10.1.168.

The server side has a GPU too, which I would harness for speeding up further the visualisation of bulky datasets. Therefore, I thought to include Nvidia Index plugin in the build of the current paraview/pvserver pair. So, the user’s guide for Nvidia Index is available from the Paraview download area. The most recent plugin version is 2.3.20190820 and is available from the Paraview dependencies repository. However, the readme file of that plugin warns me that

Known Limitations and Bugs in the BETA release
-------------------------------------------------------------------------------
Regular volume grids
-------------------------
…
Datasets in *.vtk format won’t distribute to the PV Server and cause errors. Please use *.pvti or any other format instead for distributed data
This issue is ParaView specific. Please contact Kitware for additional details.

Before I raise too much hopes on attempting hardware acceleration with my remote connection (forewarned is forearmed), a few questions arise:

  1. Does the statement above imply that the datasets will not distribute across multi-GPU systems? (Dataset not distributed inside the GPU would seem to defeat the purpose of it and all nice showcase examples around.)
  2. Is this issue confirmed from the Paraview development team as well?
    2.1 If so, has this been sorted out in releases later than 5.7?
    2.2 If not, is there a ‘lossless’ way to convert *vtk datasets into *ptvi ones? Scripting solutions would be particularly useful.

Thanks for answering them and for sharing tips and experiences.

2

I will also look at those MPI aspect in the future and I have taken a few notes to prepare the jump to distributed exectution.
First, it seems to me that the limitation is that the input must somehow be already split into parts and in order to split it, I would look up at https://vtk.org/doc/nightly/html/classvtkPDistributedDataFilter.html#details .
Hopefully this would help you.

3

Yohan,
I am pretty sure you are correct. With a small number of exceptions, ParaView will only read one file into one core/gpu. Thus, if you have 16 cores with a gpu each (or 8 gpus), one file will go to one core/gpu, and the rest will remain idle. This is also true of Exodus and CGNS datasets. The solution is to split/spread these datsets into numerous files (or leave them spread from the simulation).

4

Hi Xavier,

I was reading Paraview 5.8.0 release note, and what you probably want is the «Redistribute dataset filter» with the number of partitions equals to the number of GPUs, then save the result into a «.pvtk» file and reopen everything with the «nvidia index plugin».
But maybe just using 5.8.0 would do the trick.

Reference: https://blog.kitware.com/paraview-5-8-0-release-notes/#filter-changes