undefined symbol: PyExc_ValueError

In Situ Support
1

Hello,

I tried to build a program based on the Catalyst, the paraview and my program is ok to compile, but I got this error when I run the program:

./example/MandelbulbCatalystExample/MandelbulbDynamic: symbol lookup error: /home/zw241/cworkspace/build/build_paraview/lib64/libvtkWrappingPythonCore-pv5.8.so.1: undefined symbol: PyExc_ValueError

Then I tried to use the nm command to check the .so file:

$ nm /home/zw241/cworkspace/build/build_paraview/lib64/libvtkWrappingPythonCore-pv5.8.so.1 > nmresults.txt
$ cat nmresults.txt |grep PyExc_ValueError
                 U PyExc_ValueError
$ cat nmresults.txt |grep Py
                 U PyArg_ParseTuple
                 U PyArg_UnpackTuple
                 U PyBaseObject_Type
                 U PyBool_FromLong
                 U PyBool_Type
                 U PyBuffer_FillContiguousStrides
                 U PyBuffer_FillInfo
                 U PyBuffer_Release
                 U PyByteArray_Type
                 U PyBytes_AsString
                 U PyBytes_AsStringAndSize
                 U PyBytes_FromStringAndSize
                 U PyBytes_Size
                 U PyCFunction_NewEx
                 U PyCallable_Check
                 U PyDict_GetItem
                 U PyDict_New
...

It looks that PyExc_ValueError and some other necessary python type are not defined, and I am curious about this, I have enabled the paraview Catalyst when I built the paraview

$ cat CMakeCache.txt |grep Catalyst          
Catalyst_LIB_DEPENDS:STATIC=general;VTK::CommonCore;general;ParaView::RemotingApplication;general;VTK::FiltersGeneral;general;VTK::vtksys;general;VTK::ParallelMPI;
PythonCatalyst_LIB_DEPENDS:STATIC=general;ParaView::Catalyst;general;VTK::PythonInterpreter;general;ParaView::PythonInitializer;general;VTK::ParallelCore;
//Enable the ParaView::Catalyst module. 
VTK_MODULE_ENABLE_ParaView_Catalyst:STRING=YES
//Enable the ParaView::PythonCatalyst module. 
VTK_MODULE_ENABLE_ParaView_PythonCatalyst:STRING=YES
//Enable the ParaView::RemotingLive module. Catalyst LIVE
vtkPVCatalystPython_LIB_DEPENDS:STATIC=general;ParaView::Catalyst;general;VTK::WrappingPythonCore;
vtkPVPythonCatalystPython_LIB_DEPENDS:STATIC=general;ParaView::PythonCatalyst;general;VTK::WrappingPythonCore;
//ADVANCED property for variable: VTK_MODULE_ENABLE_ParaView_Catalyst
VTK_MODULE_ENABLE_ParaView_Catalyst-ADVANCED:INTERNAL=1
//MODIFIED property for variable: VTK_MODULE_ENABLE_ParaView_Catalyst
VTK_MODULE_ENABLE_ParaView_Catalyst-MODIFIED:INTERNAL=ON
//STRINGS property for variable: VTK_MODULE_ENABLE_ParaView_Catalyst
VTK_MODULE_ENABLE_ParaView_Catalyst-STRINGS:INTERNAL=YES;WANT;DONT_WANT;NO;DEFAULT
//ADVANCED property for variable: VTK_MODULE_ENABLE_ParaView_PythonCatalyst
VTK_MODULE_ENABLE_ParaView_PythonCatalyst-ADVANCED:INTERNAL=1
//MODIFIED property for variable: VTK_MODULE_ENABLE_ParaView_PythonCatalyst
VTK_MODULE_ENABLE_ParaView_PythonCatalyst-MODIFIED:INTERNAL=ON
//STRINGS property for variable: VTK_MODULE_ENABLE_ParaView_PythonCatalyst
VTK_MODULE_ENABLE_ParaView_PythonCatalyst-STRINGS:INTERNAL=YES;WANT;DONT_WANT;NO;DEFAULT

This is the Python3 info in CMakeCache

$ cat CMakeCache.txt |grep Python3
Python3_LIBRARY_DEBUG:FILEPATH=
Python3_LIBRARY_RELEASE:FILEPATH=/projects/community/python/3.8.5/gc563/lib/libpython3.8.a
_Python3_EXECUTABLE:FILEPATH=/projects/community/python/3.8.5/gc563/bin/python3.8
_Python3_INCLUDE_DIR:PATH=/projects/community/python/3.8.5/gc563/include/python3.8
_Python3_LIBRARY_RELEASE:FILEPATH=/projects/community/python/3.8.5/gc563/lib/libpython3.8.a
//Details about finding Python3
FIND_PACKAGE_MESSAGE_DETAILS_Python3:INTERNAL=[/projects/community/python/3.8.5/gc563/lib/libpython3.8.a][/projects/community/python/3.8.5/gc563/include/python3.8][cfound components:  Development ][v3.8(3.2)]
//ADVANCED property for variable: Python3_LIBRARY_DEBUG
Python3_LIBRARY_DEBUG-ADVANCED:INTERNAL=1
//MODIFIED property for variable: Python3_LIBRARY_DEBUG
Python3_LIBRARY_DEBUG-MODIFIED:INTERNAL=ON
//ADVANCED property for variable: Python3_LIBRARY_RELEASE
Python3_LIBRARY_RELEASE-ADVANCED:INTERNAL=1
_Python3_DEVELOPMENT_SIGNATURE:INTERNAL=e2eb5da72d168ff534301bea65f0cb00
//ADVANCED property for variable: _Python3_EXECUTABLE
_Python3_EXECUTABLE-ADVANCED:INTERNAL=1
//ADVANCED property for variable: _Python3_INCLUDE_DIR
_Python3_INCLUDE_DIR-ADVANCED:INTERNAL=1
_Python3_INTERPRETER_SIGNATURE:INTERNAL=4f8d5e5cc14b6010189102c56249c7e7
//ADVANCED property for variable: _Python3_LI

I also got the similar errors when I execute the pvbatch or pvpython:

$ ./pvbatch 
./pvbatch: symbol lookup error: /cache/home/zw241/cworkspace/build/build_paraview/install/bin/../lib64/libvtkWrappingPythonCore-pv5.8.so.1: undefined symbol: PyExc_ValueError
$ ./pvpython
./pvpython: symbol lookup error: /cache/home/zw241/cworkspace/build/build_paraview/install/bin/../lib64/libvtkWrappingPythonCore-pv5.8.so.1: undefined symbol: PyExc_ValueError

Thanks a lot for your help!

2

It looks that this problem is similar with this one (Building Paraview from source is missing pvbatch), but I’m still not sure the reason for this case since it seems that the python path looks good

3

You can rebuild ParaView with VTK_PYTHON_OPTIONAL_LINK=OFF to force libpython linking from libraries. We try to detect whether we need to do this, though it may have been fixed properly after 5.8. Are you using a 5.8.x release or master?

2 Likes
4

@ben.boeckel, what’s the reason to have VTK_PYTHON_OPTIONAL_LINK ON by default? Seems it me it’s only needed for wheels, in which case shouldn’t we only turn it ON when building wheels alone?

1 Like
5

Thanks l lot for your help! I use the version 5.8.0, does that because the python library is .a file? so I need to force it to be linked, if it is .so, we will not care about it?

I got some error like this when I set VTK_PYTHON_OPTIONAL_LINK as OFF:

/usr/bin/ld: /projects/community/python/3.8.5/gc563/lib/libpython3.8.a(abstract.o): relocation R_X86_64_32S against symbol `_Py_NotImplementedStruct' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /projects/community/python/3.8.5/gc563/lib/libpython3.8.a(boolobject.o): relocation R_X86_64_32S against symbol `_Py_FalseStruct' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /projects/community/python/3.8.5/gc563/lib/libpython3.8.a(bytearrayobject.o): relocation R_X86_64_32 against symbol `_PyByteArray_empty_string' can not be used when making a shared object; recompile with -fPIC
...

maybe it is the problem of the python3 on this cluster, do i need to rebuild a python3 with --enable-shared?

6

BTW, for the VTK/Wrapping/PythonCore in 5.8.0, it looks that there is typo about “PYTHON_VERSION_MAJOR” and PYTHON_MAJOR_VERSION" maybe this is fixed in the latest version.

7

Ah, you have a static Python. We should also force it off there.

As for your linking errors, your Python isn’t built with -fPIC, so you’ll need another Python to use any shared libraries with it.

And indeed that’s a typo. Thanks for pointing it out.

8

It’s also needed for Anaconda. I don’t know about other Python distributions.

9

Thanks a lot for your help!

10

Had similar issue, VTK_PYTHON_OPTIONAL_LINK=OFF flag solved the problem. Thank you.

11

Now hitting the same problem (with PV 5.9.1). I can solve it by adding in libcatalyst as a link dependency or forcibly loading libpython in my application. Neither of these seems quite right.

The new version has a ParaView::vtkcatalyst module, but that does not seem to do it either.

Quick tips?

12

Where are you getting the linker errors? In your application linking to ParaView libraries? If that’s the case, I would just use the VTK::Python target here for now. I think this has been fixed on master, at least for Catalyst2 though.