pvpython scripting in parallel

ParaView Support
1

Hi, I use pvpython scripting to convert a bunch of vtk files to png images in paraview.

The logic is to:
(i) define main function vtkToImg(), which reads one vtk file and save it as one screenshot (.png);
(ii) use for-loop to iteratively run the main function for all vtk files.

The script runs well for single core, but processing vtk file one by one is very time-cosuming. So I try to use multi-processing module to run the script in parallel, and the code looks like:

import multiprocessing as mp
from paraview.simple import *

def getvtkFile(path_vtk)    # get vtk files in path_vtk           
      ...

def vtkToImg(vtkFile, path_img)   # main function with 2 inputs
      ...

if __name__ == "__main__":        # parallel

   path_vtk = './sequencedVTK/'
   path_img = './sequencedIMG/'
   
   # get vtk files
   vtkFiles = getvtkFile(path_vtk)

   # build pool including 8 processors
   pool = mp.Pool(8)
   pool.starmap(vtkToImage, [(vtkFiles[i], path_img) for i in range(len(vtkFiles))])  
   pool.close()

But when running this s in the terminal, I received the error AttributeError: ‘Pool’ object has no attribute 'starmap’ I find this error appears due to I apply python2 instead of python3.

So my question is:
(1) how to apply python3 with pvpython?
(2) is it reasonable to use multi-processing module for pvpython parallel? Or pvpython has other better methods to run in parallel?

I would be very appreciated if anyone can give me some help!

2

Hi @Ying ,

I’m afraid pvpython is not threadsafe and cannot be used like this.

You should either go higher with subprocesses, each importing paraview simple, or go lower and use pvbatch (6. Remote and parallel visualization — ParaView Documentation 5.10.0 documentation)

Hth