Cannot import GhostCellsGenerator into Catalyst V2 pipeline

In Situ Support
1

Is it possible to import ghost cells generator into catalyst V2 scripts currently? I have a tentative solution for a parallel pipeline, that I’d like to run, but when I plug my script to Catalyst it breaks with the following message:

Traceback (most recent call last):                                                                                                                             
  File "/home/rsa/projects/sources/paraview/build-5.9.0/lib/python3.9/site-packages/paraview/__init__.py", line 161, in <module>                               
    from .modules import vtkRemotingCore                                                                                                                       
ImportError: cannot import name 'vtkRemotingCore' from 'paraview.modules' (/home/rsa/projects/sources/paraview/build-5.9.0/lib/python3.9/site-packages/paraview/modules/__init__.py)

I reproduce this with a minimal script that only imports GhostCellsGenerator.

It looks like a fairly low-level error so I wanted to ask before I dive into this. Please advise.

2

That error is a red herring, often indicating ParaView’s Python compiled modules have not been found correctly. Trying printing sys.path , LD_LIBRARY_PATH etc. to ensure they are setup to find ParaView/VTK compiled/wrapping Python modules.

3

Gosh you’re right. It seems that my installation somehow depended on the build files which I have since deleted and actually no Catalyst script now works. I have updated the paths, and recompiled catalyst function object, but now I am stuck with

malloc(): invalid size (unsorted)
Loguru caught a signal: SIGABRT

when calling catalyst_execute(conduit::c_node...). I can’t even print sys.path. I am recompiling ParaView as I think maybe python upgrade from 3.9.2 to 3.9.3 may have broken it.

4

uff, this seems to have gone awry :). Try LD_DEBUG=libs environment variable and see what libraries are getting loaded. That may illuminate the issues.

5

Thanks. That’s a very useful tip with LD_DEBUG. I am going to review this now. After recompiling ParaView it’s all back and the example from pvserver is working as expected.

6

I love LD_DEBUG. There’s DYLD_DEBUG on macOS which has the same effect. Very useful tools especially when debugging cryptic issues in dependency-heavy softwares.

1 Like
7

Thanks for the tip. This is awesome.

I debugged a little and I still have a dependency on my build directory. When catalyst_execute is called, I am seeing the following:

    240848:	calling init: /home/rsa/projects/sources/paraview/build-5.9.0/lib/python3.9/site-packages/paraview/modules/vtkRemotingCore.so
    240848:	
    240848:	
    240848:	calling init: /home/rsa/projects/sources/paraview/build-5.9.0/lib/python3.9/site-packages/paraview/modules/vtkRemotingClientServerStream.so
...
    240848:	calling init: /home/rsa/projects/sources/paraview/build-5.9.0/lib/python3.9/site-packages/paraview/modules/vtkRemotingLive.so

I am a great fan of Lmod and care about my environment dearly so I have added those paths to my LD_LIBRARY_PATH of the paraview/5.9.0 module and a similarly named so is being found when the catalyst armed simulation starts.

    240848:	find library=libvtkRemotingCore-pv5.9.so.1 [0]; searching
    240848:	 search path=/home/rsa/apps/compiler/gcc/10.2/openmpi/4.0.5/lib:/home/rsa/apps/mpi/gcc/10.2/openmpi/4.0/paraview/5.9.0/lib		(LD_LIBRARY_PATH)
    240848:	  trying file=/home/rsa/apps/compiler/gcc/10.2/openmpi/4.0.5/lib/libvtkRemotingCore-pv5.9.so.1
    240848:	  trying file=/home/rsa/apps/mpi/gcc/10.2/openmpi/4.0/paraview/5.9.0/lib/libvtkRemotingCore-pv5.9.so.1

I tried to link create a symbolic link to libvtkRemotingCore-pv5.9.so.1, but it still reverts to the build directory.

Maybe my CMakeListst.txt is incorrect?

if (${ParaView_VERSION} VERSION_GREATER_EQUAL 5.7)
    find_package(ParaView REQUIRED
        COMPONENTS PythonCatalyst
        OPTIONAL_COMPONENTS VTK::ParallelMPI)
else()
    find_package(ParaView REQUIRED
        COMPONENTS vtkPVPythonCatalyst
        OPTIONAL_COMPONENTS vtkParallelMPI)
endif()
8

Can you attach the complete output? The first find library that ends up find a library from the build dir would be the one to focus one. Once you load library from there, rpaths etc. would cause others from there to be loaded too.

9

Thanks @utkarsh.ayachit, I would really appreciate advice on this. Here’s my log.interIsoFoam (994.5 KB). If you search for sources or build you should see the first entry related to my build directory.

What is a bit strange is that find library is never invoked for vtkRemotingCore.so, but for vtkRemotingCore-pv5.9.so. The reference to vtkRemotingCore.so in my build dir appears after callinig init which makes me think there’s something wrong in my cmake invocation for OpenFOAM adaptor or for ParaView itself.

I normally compile ParaView with something like this:

cmake \
    -DCMAKE_BUILD_TYPE=RelWithDebInfo \
    ...
    -DCMAKE_INSTALL_PREFIX=/path/to/installation \
    ...
    && make \
    && make install