pvpython and multiprocess

ParaView Support
1

Hi, I’m trying to use the ProcessThreadPool class of python but I get the following error: Some resources might leak . I’m trying to understand if it’s because I run my python script under pvpython or not.

What am I doing?

I have a grid of points from which I want to extract the streamTracer filter. Essentially I give to the python script the list of the points and he moves the filter around the script saving each time a data file. Since I have more that 5.000 points (I could have easily >30k points), it turns that I need to save a lot of files and it keeps >2 minutes. I was thinking on a multi process solution (multi thread doesn’t improve performance), but I cannot get rid of the following error:

pvpython export_streamlines.py receivers.geojson 0.5 3
/Applications/ParaView-5.11.0-RC2.app/Contents/Libraries/lib/python3.9/multiprocessing/resource_tracker.py:96: UserWarning: resource_tracker: process died unexpectedly, relaunching.  Some resources might leak.
  warnings.warn('resource_tracker: process died unexpectedly, '

Here my code:

from paraview.simple import *
import os
import json
import time
from concurrent.futures import ProcessPoolExecutor as pool


def extract_and_save(feature,streamTracer1):
	x = feature['geometry']['coordinates'][0]
	y = feature['geometry']['coordinates'][1]
	z = feature['geometry']['coordinates'][2]
	ID = feature['properties']['PK']
	Center = [x,y,z]
	streamTracer1.SeedType.Center = Center
	SaveData('../streamlines/'+str(ID)+'.txt', proxy=streamTracer1, ChooseArraysToWrite=1,
    PointDataArrays=['IntegrationTime', 'U [m/s]'],
    CellDataArrays=['ReasonForTermination', 'SeedIds'],
    FieldDataArrays=['CasePath'])

start_time = time.time()

if len(sys.argv)<4:
    print("Inputs: <geojson path> , <Radius> , <numberOfPoints> ")
    sys.exit(1)

with open(sys.argv[1], 'r') as json_file:
    data = json.load(json_file)

# get the time-keeper
timeKeeper1 = GetTimeKeeper()

# load state
LoadState('../streamlines.pvsm')

# find source
streamTracer1 = FindSource('StreamTracer1')

# set active source
SetActiveSource(streamTracer1)

#Inputs
streamTracer1.SeedType.Radius = float(sys.argv[2])
streamTracer1.SeedType.NumberOfPoints = int(sys.argv[3])

# save data
if not os.path.exists("../streamlines"):
	    os.makedirs("../streamlines")

#for feature in data['features']:
	 #extract_and_save(feature,streamTracer1)
with pool(max_workers=1) as exe:
	_ = exe.submit(extract_and_save, data['features'])


print('[Finished in: %s seconds ]' % (time.time() - start_time))
2

is there anybody here?

3

This looks not related to ParaView at all, did you try your scripts without using ParaView ?

4

Thank you for your attention. Sure, I tested a little multiprocess script outside Paraview and it works. I found this issue inside the paraview gitlab.
Of course I could not test the same script outside paraview since it depends from it

5

@jourdain