Cannot create Programmable Source using MPI

Development
,
1

Hello everyone! I use Paraview 5.10.1 with MPI on Windows Client and Linux Server.
I have a problem regarding MPI. My goal is to have a Python module that will be placed both on client and server machine. Python commands from the module will be inputted via client Python Shell, and then a Programmable Source with script that uses commands from the same module will be created on server side. My approach is working when I deal with only one instance of pvserver. But when I run several instances using mpiexec on Linux machine Programmable Source isn’t created. In fact, it executes only RequestInformation part and then throws a SIGSEGV error.

To sum it up, I have a file my_module.py that has command create_stuff_remote(filename) and create_stuff(filename). I connect to PV server that is run on Linux via mpiexec. I import my_module on client and type create_stuff_remote(filename) in client’s Python Shell and it imports my_module on server and makes Programmable Source there that has create_stuff(filename) in its Script. So in my understanding it is supposed to execute create_stuff on all instances of pvserver

I try to turn on parallel execution with following lines (taken from here: custom parallel source using pythonalgorithm - #2 by pavel):

in ScriptRequestInformation:

executive = self.GetExecutive()
outInfo = executive.GetOutputInformation(0)
outInfo.Set(vtk.vtkAlgorithm.CAN_HANDLE_PIECE_REQUEST(), 1)

in ScriptRequestData:

executive = self.GetExecutive()
outInfo = executive.GetOutputInformation(0)
part = outInfo.Get(executive.UPDATE_PIECE_NUMBER())
print('ACQUIRED PART NUMBER: ', part) #just to checkif MPI is running ok
#(here I run command from my module that takes filename as an argument, the filename is passed to function that sets ScriptRequestData)

(Clarifying: a Programmable Source ps_1 object is created in create_remote_stuff() and ps_1.Script and ps_1.ScriptRequestInformation is filled with lines above)

So the ScriptRequestInformation does work (checked via putting debug prints)
But I also tried to put Get(executive.UPDATE_PIECE_NUMBER()) part in ScriptRequestInformation and outputted the part, and I got the same output 1 in all n instances of pvserver

And rigt after that, supposedly before RequestData even started to execute (again, checked through debug prints) all server instances crash with tons of red text and errors such as:
Loguru caught a signal: SIGSEGV
error: exception occurred: Segmentation fault
EXIT CODE: 9

My questions are following: Is there anything that I did fundamentally wrong while trying to make this work? Is it possible to make parallel creation of Programmable Source between several servers and if so, what is a proper way to do that?

Thank you very much, any help will be appreciated!

2

Hi @lowered_priora

Working with MPI in prog filter is not recommended, it can be quite hard to debug.

If you need to work in python, then working with Python Plugins would be much more confortable for you.

And actual C++ would be even better if possible.

Did you consider that ?

Best,

3

Hello, Mathiue, thank you for your reply!
I’ve heard of plugins and that it is possible to implement MPI there, but my current goal requires to do all of that through Python Shell.
C++ is also not an option at the moment. I understand that debugging might be difficult, but I really need the MPI to work via Programmable Source.

4

You cant really implement a filter through python shell, so not sure what you mean.