Hi,
I have a shared library (libscorep_substrate_Catalyst.so) which contains a Catalyst adapter (libPluginAdaptor.a). I can successfully load the library and use the adapter with simulation codes written in C++ or Fortran, but now I need to use it in a simulation code written in Python(3). The following minimal example is enough to reproduce the error I am getting:
#!/usr/bin/env python
import ctypes
import pathlib
if __name__ == "__main__":
# Load the shared library into ctypes
libname = pathlib.Path().absolute() / "libscorep_substrate_Catalyst.so"
c_lib = ctypes.CDLL(libname)
c_lib.cat_sco_initialize_()
for i in range(10):
print("time step = ", i)
c_lib.cat_sco_finalize_()
cat_sco_initialize_() will attempt to initialize the Catalyst adapter, but it crashes with the following backtrace:
[taurusi6604:1144 :0:1144] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x10)
==== backtrace (tid: 1144) ====
0 0x00000000001fd291 PySys_GetObject() /home/h9/alves/dlr/flowsim_builder/install/build/python/Python-3.7.4/./Python/sysmodule.c:69
1 0x000000000000e888 (anonymous namespace)::vtkPrependPythonPath() /lustre/ssd/ws/alves-ParaView/src/VTK/Utilities/PythonInterpreter/vtkPythonInterpreter.cxx:145
2 0x000000000000f011 vtkPythonInterpreter::PrependPythonPath() /lustre/ssd/ws/alves-ParaView/src/VTK/Utilities/PythonInterpreter/vtkPythonInterpreter.cxx:354
3 0x000000000000f2cb vtkPythonInterpreter::PrependPythonPath() /lustre/ssd/ws/alves-ParaView/src/VTK/Utilities/PythonInterpreter/vtkPythonInterpreter.cxx:376
4 0x000000000010dad1 vtkProcessModule::InitializePythonEnvironment() /lustre/ssd/ws/alves-ParaView/src/ParaViewCore/ClientServerCore/Core/vtkProcessModule.cxx:587
5 0x000000000010c328 vtkProcessModule::Initialize() /lustre/ssd/ws/alves-ParaView/src/ParaViewCore/ClientServerCore/Core/vtkProcessModule.cxx:277
6 0x000000000019e6f5 vtkInitializationHelper::Initialize() /lustre/ssd/ws/alves-ParaView/src/ParaViewCore/ServerManager/SMApplication/vtkInitializationHelper.cxx:194
7 0x0000000000015b86 vtkCPCxxHelper::New() /lustre/ssd/ws/alves-ParaView/src/CoProcessing/Catalyst/vtkCPCxxHelper.cxx:93
8 0x000000000001f2df vtkCPProcessor::NewInitializationHelper() /lustre/ssd/ws/alves-ParaView/src/CoProcessing/Catalyst/vtkCPProcessor.cxx:134
9 0x000000000001f319 vtkCPProcessor::Initialize() /lustre/ssd/ws/alves-ParaView/src/CoProcessing/Catalyst/vtkCPProcessor.cxx:142
10 0x000000000002754d plg_adp::initialize() ???:0
11 0x000000000001e7ac cat_sco_initialize_() ???:0
12 0x00000000000066d4 ffi_call_unix64() ???:0
13 0x0000000000005f53 ffi_call() ???:0
14 0x000000000000e9fd _call_function_pointer() /home/h9/alves/dlr/flowsim_builder/install/build/python/Python-3.7.4/Modules/_ctypes/callproc.c:827
15 0x000000000000e9fd _ctypes_callproc() /home/h9/alves/dlr/flowsim_builder/install/build/python/Python-3.7.4/Modules/_ctypes/callproc.c:1184
16 0x000000000000a835 PyCFuncPtr_call() /home/h9/alves/dlr/flowsim_builder/install/build/python/Python-3.7.4/Modules/_ctypes/_ctypes.c:3971
17 0x0000000000155537 _PyObject_FastCallKeywords() /home/h9/alves/dlr/flowsim_builder/install/build/python/Python-3.7.4/Objects/call.c:199
18 0x00000000001b107a call_function() /home/h9/alves/dlr/flowsim_builder/install/build/python/Python-3.7.4/Python/ceval.c:2694
19 0x00000000001b107a _PyEval_EvalFrameDefault() /home/h9/alves/dlr/flowsim_builder/install/build/python/Python-3.7.4/Python/ceval.c:3093
20 0x00000000001abeac _PyEval_EvalCodeWithName() /home/h9/alves/dlr/flowsim_builder/install/build/python/Python-3.7.4/Python/ceval.c:3930
21 0x00000000001abcd3 PyEval_EvalCodeEx() /home/h9/alves/dlr/flowsim_builder/install/build/python/Python-3.7.4/Python/ceval.c:3959
22 0x00000000001abc8b PyEval_EvalCode() /home/h9/alves/dlr/flowsim_builder/install/build/python/Python-3.7.4/Python/ceval.c:524
23 0x00000000001ab190 builtin_exec_impl() /home/h9/alves/dlr/flowsim_builder/install/build/python/Python-3.7.4/Python/bltinmodule.c:1079
24 0x00000000001ab190 builtin_exec() /home/h9/alves/dlr/flowsim_builder/install/build/python/Python-3.7.4/Python/clinic/bltinmodule.c.h:283
25 0x0000000000154a79 _PyMethodDef_RawFastCallKeywords() /home/h9/alves/dlr/flowsim_builder/install/build/python/Python-3.7.4/Objects/call.c:655
26 0x00000000001549e0 _PyCFunction_FastCallKeywords() /home/h9/alves/dlr/flowsim_builder/install/build/python/Python-3.7.4/Objects/call.c:734
27 0x00000000000e89df call_function() /home/h9/alves/dlr/flowsim_builder/install/build/python/Python-3.7.4/Python/ceval.c:4568
28 0x00000000000e89df _PyEval_EvalFrameDefault() /home/h9/alves/dlr/flowsim_builder/install/build/python/Python-3.7.4/Python/ceval.c:3124
29 0x00000000001abeac _PyEval_EvalCodeWithName() /home/h9/alves/dlr/flowsim_builder/install/build/python/Python-3.7.4/Python/ceval.c:3930
30 0x0000000000155c35 _PyFunction_FastCallKeywords() /home/h9/alves/dlr/flowsim_builder/install/build/python/Python-3.7.4/Objects/call.c:433
31 0x00000000001acfb5 call_function() /home/h9/alves/dlr/flowsim_builder/install/build/python/Python-3.7.4/Python/ceval.c:4616
32 0x00000000001acfb5 _PyEval_EvalFrameDefault() /home/h9/alves/dlr/flowsim_builder/install/build/python/Python-3.7.4/Python/ceval.c:3110
33 0x0000000000155df5 function_code_fastcall() /home/h9/alves/dlr/flowsim_builder/install/build/python/Python-3.7.4/Objects/call.c:283
34 0x00000000001acd52 call_function() /home/h9/alves/dlr/flowsim_builder/install/build/python/Python-3.7.4/Python/ceval.c:4616
35 0x00000000001acd52 _PyEval_EvalFrameDefault() /home/h9/alves/dlr/flowsim_builder/install/build/python/Python-3.7.4/Python/ceval.c:3124
36 0x00000000001abeac _PyEval_EvalCodeWithName() /home/h9/alves/dlr/flowsim_builder/install/build/python/Python-3.7.4/Python/ceval.c:3930
37 0x00000000001abcd3 PyEval_EvalCodeEx() /home/h9/alves/dlr/flowsim_builder/install/build/python/Python-3.7.4/Python/ceval.c:3959
38 0x00000000001abc8b PyEval_EvalCode() /home/h9/alves/dlr/flowsim_builder/install/build/python/Python-3.7.4/Python/ceval.c:524
39 0x00000000001ab190 builtin_exec_impl() /home/h9/alves/dlr/flowsim_builder/install/build/python/Python-3.7.4/Python/bltinmodule.c:1079
40 0x00000000001ab190 builtin_exec() /home/h9/alves/dlr/flowsim_builder/install/build/python/Python-3.7.4/Python/clinic/bltinmodule.c.h:283
41 0x0000000000154a79 _PyMethodDef_RawFastCallKeywords() /home/h9/alves/dlr/flowsim_builder/install/build/python/Python-3.7.4/Objects/call.c:655
42 0x00000000001549e0 _PyCFunction_FastCallKeywords() /home/h9/alves/dlr/flowsim_builder/install/build/python/Python-3.7.4/Objects/call.c:734
43 0x00000000001b0543 call_function() /home/h9/alves/dlr/flowsim_builder/install/build/python/Python-3.7.4/Python/ceval.c:4568
44 0x00000000001b0543 _PyEval_EvalFrameDefault() /home/h9/alves/dlr/flowsim_builder/install/build/python/Python-3.7.4/Python/ceval.c:3124
45 0x00000000001abeac _PyEval_EvalCodeWithName() /home/h9/alves/dlr/flowsim_builder/install/build/python/Python-3.7.4/Python/ceval.c:3930
46 0x0000000000155c35 _PyFunction_FastCallKeywords() /home/h9/alves/dlr/flowsim_builder/install/build/python/Python-3.7.4/Objects/call.c:433
47 0x00000000001acd52 call_function() /home/h9/alves/dlr/flowsim_builder/install/build/python/Python-3.7.4/Python/ceval.c:4616
48 0x00000000001acd52 _PyEval_EvalFrameDefault() /home/h9/alves/dlr/flowsim_builder/install/build/python/Python-3.7.4/Python/ceval.c:3124
49 0x00000000001abeac _PyEval_EvalCodeWithName() /home/h9/alves/dlr/flowsim_builder/install/build/python/Python-3.7.4/Python/ceval.c:3930
50 0x00000000001563af _PyFunction_FastCallDict() /home/h9/alves/dlr/flowsim_builder/install/build/python/Python-3.7.4/Objects/call.c:376
51 0x0000000000201ba1 pymain_run_module() /home/h9/alves/dlr/flowsim_builder/install/build/python/Python-3.7.4/Modules/main.c:320
52 0x00000000001ffa20 pymain_run_python() /home/h9/alves/dlr/flowsim_builder/install/build/python/Python-3.7.4/Modules/main.c:2864
53 0x00000000001ffa20 pymain_main() /home/h9/alves/dlr/flowsim_builder/install/build/python/Python-3.7.4/Modules/main.c:3028
54 0x00000000001ff80e _Py_UnixMain() /home/h9/alves/dlr/flowsim_builder/install/build/python/Python-3.7.4/Modules/main.c:3063
55 0x0000000000022555 __libc_start_main() ???:0
56 0x00000000004007ae _start() ???:0
=================================
[taurusi6604:01144] *** Process received signal ***
[taurusi6604:01144] Signal: Segmentation fault (11)
[taurusi6604:01144] Signal code: (-6)
[taurusi6604:01144] Failing at address: 0x205cc300000478
[taurusi6604:01144] [ 0] /lib64/libpthread.so.0(+0xf630)[0x2b5963bfe630]
[taurusi6604:01144] [ 1] /home/alves/dlr/flowsim_builder/install/python/python_3_7_4_gcc_system/lib/libpython3.7m.so.1.0(PySys_GetObject+0x11)[0x2b596386e291]
[taurusi6604:01144] [ 2] /lustre/ssd/ws/alves-ParaView/v5.7.0/install/lib64/libvtkPythonInterpreter-pv5.7.so.1(+0xe888)[0x2b5966401888]
[taurusi6604:01144] [ 3] /lustre/ssd/ws/alves-ParaView/v5.7.0/install/lib64/libvtkPythonInterpreter-pv5.7.so.1(_ZN20vtkPythonInterpreter17PrependPythonPathEPKc+0x83)[0x2b5966402011]
[taurusi6604:01144] [ 4] /lustre/ssd/ws/alves-ParaView/v5.7.0/install/lib64/libvtkPythonInterpreter-pv5.7.so.1(_ZN20vtkPythonInterpreter17PrependPythonPathEPKcS1_+0x267)[0x2b59664022cb]
[taurusi6604:01144] [ 5] /lustre/ssd/ws/alves-ParaView/v5.7.0/install/lib64/libvtkPVClientServerCoreCore-pv5.7.so.1(_ZN16vtkProcessModule27InitializePythonEnvironmentEv+0x45)[0x2b5968f25ad1]
[taurusi6604:01144] [ 6] /lustre/ssd/ws/alves-ParaView/v5.7.0/install/lib64/libvtkPVClientServerCoreCore-pv5.7.so.1(_ZN16vtkProcessModule10InitializeENS_12ProcessTypesERiRPPc+0x592)[0x2b5968f24328]
[taurusi6604:01144] [ 7] /lustre/ssd/ws/alves-ParaView/v5.7.0/install/lib64/libvtkPVServerManagerApplication-pv5.7.so.1(_ZN23vtkInitializationHelper10InitializeEiPPciP12vtkPVOptions+0x1af)[0x2b59677a66f5]
[taurusi6604:01144] [ 8] /lustre/ssd/ws/alves-ParaView/v5.7.0/install/lib64/libvtkPVCatalyst-pv5.7.so.1(_ZN14vtkCPCxxHelper3NewEv+0x168)[0x2b59663d9b86]
[taurusi6604:01144] [ 9] /lustre/ssd/ws/alves-ParaView/v5.7.0/install/lib64/libvtkPVCatalyst-pv5.7.so.1(_ZN14vtkCPProcessor23NewInitializationHelperEv+0x11)[0x2b59663e32df]
[taurusi6604:01144] [10] /lustre/ssd/ws/alves-ParaView/v5.7.0/install/lib64/libvtkPVCatalyst-pv5.7.so.1(_ZN14vtkCPProcessor10InitializeEPKc+0x37)[0x2b59663e3319]
[taurusi6604:01144] [11] /home/alves/phd/plugin/install/libscorep_substrate_Catalyst.so(_ZN7plg_adp10initializeEPKcmm+0x5f)[0x2b59663b154d]
[taurusi6604:01144] [12] /home/alves/phd/plugin/install/libscorep_substrate_Catalyst.so(cat_sco_initialize_+0x90f)[0x2b59663a87ac]
[taurusi6604:01144] [13] /sw/installed/libffi/3.2.1-GCCcore-8.2.0/lib64/libffi.so.6(ffi_call_unix64+0x4c)[0x2b59635486d4]
[taurusi6604:01144] [14] /sw/installed/libffi/3.2.1-GCCcore-8.2.0/lib64/libffi.so.6(ffi_call+0x153)[0x2b5963547f53]
[taurusi6604:01144] [15] /home/alves/dlr/flowsim_builder/install/python/python_3_7_4_gcc_system/lib/python3.7/lib-dynload/_ctypes.cpython-37m-x86_64-linux-gnu.so(_ctypes_callproc+0x29e)[0x2b597ec029fd]
[taurusi6604:01144] [16] /home/alves/dlr/flowsim_builder/install/python/python_3_7_4_gcc_system/lib/python3.7/lib-dynload/_ctypes.cpython-37m-x86_64-linux-gnu.so(+0xa835)[0x2b597ebfe835]
[taurusi6604:01144] [17] /home/alves/dlr/flowsim_builder/install/python/python_3_7_4_gcc_system/lib/libpython3.7m.so.1.0(_PyObject_FastCallKeywords+0x97)[0x2b59637c6537]
[taurusi6604:01144] [18] /home/alves/dlr/flowsim_builder/install/python/python_3_7_4_gcc_system/lib/libpython3.7m.so.1.0(_PyEval_EvalFrameDefault+0x484a)[0x2b596382207a]
[taurusi6604:01144] [19] /home/alves/dlr/flowsim_builder/install/python/python_3_7_4_gcc_system/lib/libpython3.7m.so.1.0(_PyEval_EvalCodeWithName+0x1cc)[0x2b596381ceac]
[taurusi6604:01144] [20] /home/alves/dlr/flowsim_builder/install/python/python_3_7_4_gcc_system/lib/libpython3.7m.so.1.0(PyEval_EvalCodeEx+0x43)[0x2b596381ccd3]
[taurusi6604:01144] [21] /home/alves/dlr/flowsim_builder/install/python/python_3_7_4_gcc_system/lib/libpython3.7m.so.1.0(PyEval_EvalCode+0x1b)[0x2b596381cc8b]
[taurusi6604:01144] [22] /home/alves/dlr/flowsim_builder/install/python/python_3_7_4_gcc_system/lib/libpython3.7m.so.1.0(+0x1ab190)[0x2b596381c190]
[taurusi6604:01144] [23] /home/alves/dlr/flowsim_builder/install/python/python_3_7_4_gcc_system/lib/libpython3.7m.so.1.0(_PyMethodDef_RawFastCallKeywords+0x79)[0x2b59637c5a79]
[taurusi6604:01144] [24] /home/alves/dlr/flowsim_builder/install/python/python_3_7_4_gcc_system/lib/libpython3.7m.so.1.0(_PyCFunction_FastCallKeywords+0x20)[0x2b59637c59e0]
[taurusi6604:01144] [25] /home/alves/dlr/flowsim_builder/install/python/python_3_7_4_gcc_system/lib/libpython3.7m.so.1.0(+0xe89df)[0x2b59637599df]
[taurusi6604:01144] [26] /home/alves/dlr/flowsim_builder/install/python/python_3_7_4_gcc_system/lib/libpython3.7m.so.1.0(_PyEval_EvalCodeWithName+0x1cc)[0x2b596381ceac]
[taurusi6604:01144] [27] /home/alves/dlr/flowsim_builder/install/python/python_3_7_4_gcc_system/lib/libpython3.7m.so.1.0(_PyFunction_FastCallKeywords+0x195)[0x2b59637c6c35]
[taurusi6604:01144] [28] /home/alves/dlr/flowsim_builder/install/python/python_3_7_4_gcc_system/lib/libpython3.7m.so.1.0(_PyEval_EvalFrameDefault+0x785)[0x2b596381dfb5]
[taurusi6604:01144] [29] /home/alves/dlr/flowsim_builder/install/python/python_3_7_4_gcc_system/lib/libpython3.7m.so.1.0(+0x155df5)[0x2b59637c6df5]
[taurusi6604:01144] *** End of error message ***
I think the crash happens when attempting to run the following lines in the code:
namespace plg_adp
{
void initialize(const char * python_script, const std::size_t nodeId, const std::size_t nNodes)
{
if (Processor == NULL)
{
Processor = vtkCPProcessor::New();
Processor -> Initialize();
}
else
{
Processor -> RemoveAllPipelines();
}
Processor is a static global variable inside libPluginAdaptor.a, initialized to NULL:
static vtkCPProcessor* Processor = NULL;
I am using ParaView 5.7.0. Would you have an idea about what is going on?
Thank you very much,