Cannot generate a safe runtime search path for target vtkRenderingOpenGL2

ParaView Support
1

Hey,

I have trouble to compile paraview using the nvidia proprietary driver instead of the files which belongs to mesa.

So the problem is that there is a libGL.so.1 in /usr/lib which belongs to mesa (18.2.8) and one in
/usr/lib/opengl/nvidia/lib (nvidia-driver).

I tried to use
-DOPENGL_gl_LIBRARY="/usr/lib/opengl/nvidia/lib/libGL.so"

and got this result:
'--------------------------------------------------------------------------------------------------------
– Configuring done
CMake Warning at VTK/CMake/vtkModuleMacros.cmake:575 (add_library):
Cannot generate a safe runtime search path for target
vtkPVServerManagerApplication because files in some directories may
conflict with libraries in implicit directories:

runtime library [libEGL.so.1] in /usr/lib may be hidden by files in:
  /usr/lib/opengl/nvidia/lib
runtime library [libGL.so.1] in /usr/lib may be hidden by files in:
  /usr/lib/opengl/nvidia/lib

Some of these libraries may not be found correctly.
Call Stack (most recent call first):
VTK/CMake/vtkModuleMacros.cmake:660 (vtk_add_library)
ParaViewCore/ServerManager/SMApplication/CMakeLists.txt:35 (vtk_module_library)

CMake Warning at VTK/CMake/vtkModuleMacros.cmake:575 (add_library):
Cannot generate a safe runtime search path for target pqCore because files
in some directories may conflict with libraries in implicit directories:

runtime library [libEGL.so.1] in /usr/lib may be hidden by files in:
  /usr/lib/opengl/nvidia/lib
runtime library [libGL.so.1] in /usr/lib may be hidden by files in:
  /usr/lib/opengl/nvidia/lib

Some of these libraries may not be found correctly.
…
…
– Generating done
CMake Warning:
Manually-specified variables were not used by the project:

EGL_opengl_LIBRARY

'------------------------------------------------------------------------------------------------------

Then I tried to use instead
OpenGL_GL_PREFERENCE=GLVND
but got this result (same like above but in the end another unused variable):
'------------------------------------------------------------------------------------------------------
…
…
…

Manually-specified variables were not used by the project:

OpenGL_GL_PREFERENCE"

'------------------------------------------------------------------------------------------------------

glxinfo | grep -i opengl:
'------------------------------------------------------------------------------------------------------
OpenGL vendor string: NVIDIA Corporation
OpenGL renderer string: Quadro P400/PCIe/SSE2
OpenGL core profile version string: 4.5.0 NVIDIA 415.25
OpenGL core profile shading language version string: 4.50 NVIDIA
OpenGL core profile context flags: (none)
OpenGL core profile profile mask: core profile
OpenGL core profile extensions:
OpenGL version string: 4.6.0 NVIDIA 415.25
OpenGL shading language version string: 4.60 NVIDIA
OpenGL context flags: (none)
OpenGL profile mask: (none)
OpenGL extensions:
OpenGL ES profile version string: OpenGL ES 3.2 NVIDIA 415.25
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.20
OpenGL ES profile extensions:
'------------------------------------------------------------------------------------------------------

So is /usr/lib/ hardcoded and always used first, even when the system is configured otherwise?

The program will compile anyway but it uses afterwards the wrong libGL.so.1 file because it uses the one from /usr/lib first, even when other programs don’t and work without any problem, because they use the right libGL.so.1 file from the profile.

OS: gentoo linux - amd64
Version: Paraview 5.6.0

So yes, the system has mesa for other stuff then paraview and the proprietary nvidia-driver for cuda for example.

Any help would be very appreciated.

Greetings
Galitus

2

The warning is showing up because there is also a libGL and libEGL in /usr/lib and there are other libraries in /usr/lib/opengl/nvidia/lib which exist in /usr/lib where the latter version is wanted (let’s call this libA). CMake is warning that it cannot create an unambiguous RPATH which loads the all libraries linked via the paths given.

If CMake creates an RPATH of /usr/lib:/usr/lib/opengl/nvidia/lib, /usr/lib/libGL.so is preferred over the NVIDIA copy. An RPATH of /usr/lib/opengl/nvidia/lib:/usr/lib means that /usr/lib/libA.so won’t be used since it also exists in the NVIDIA directory.

The solution is to use libA from the NVIDIA directory via the cache or use the /usr/lib/libGL.so versions.

3

After looking at an instance I’m seeing for this on some VTK dashboards, I don’t think that’s 100% right. I think CMake might be indicating that it doesn’t control the order the linker uses default paths. Something is trying to use /usr/lib/x86_64-linux-gnu/libGL.so explicitly. Searching in the cmake --trace-expand output should help find out what is trying to use it.

4

Hey Ben,

thanks for your response.
I’m not a cmake expert, so i’m not quite sure what I’m looking for or how to fix it.
So can I bother you again?

I made some uploads about the used GL cmake options :
https://paste.pound-python.org/show/YZhzmwtxRe740k3k9lfy/

and the cmake trace-expand with (grep -C5 libGL.so) :
https://paste.pound-python.org/show/ozfdVREFpXuT4JQQFC3Z/

Greetings

5

So, I looked into it and it’s actually a Qt bug. See https://bugreports.qt.io/browse/QTBUG-73207 for that. There’s not much VTK, ParaView, or CMake can do to remove the warning (since it is legitimate). You could hack your Qt5Gui CMake file to use the FindOpenGL.cmake cache variables instead.

6

Alright, so if I understand it right,
the cmake warnings are coming from the qtgui “submodule” which is used in vtk, etc. and can only be fixed upstream.

In my system the settings for opengl make sure, that the path for the files will be at the top of the
/etc/ld.so.conf file , so normally everything which will be compiled, should look into these folders at first or use the cache, or while not compiling, use these folders during runtime?

My ld cache is also showing the nvidia one first:
libGL.so.1 (libc6,x86-64) => /usr/lib64/opengl/nvidia/lib/libGL.so.1
libGL.so.1 (libc6,x86-64) => /usr/lib64/libGL.so.1

If I understand it right, this is a standard for nearly everything in linux? Compiling/linking, running, etc.?

What I don’t understand is, that normally nothing in paraview should be built statically or?
So normally it is linked against a filename and not against a path and then it should honor the path stuff from LD and use what is usable in the system and in which order the system specified the search path for libs.
But it seems that it has a path search order for libs built/written into the program code?

Which file will be compiled and use this bug?
Because if I ldd on all .so files from paraview, there is no file which is searching the libGL.so.1 file in /usr/lib .

I don’t really understand why it is using at runtime the wrong lib even when the system supplies other paths first.

Should I ask these questions in the QT bug report too?

BIG thanks for your help so far!

Greetings
Galitus

7

libGL is a bit weird since driver providers tend to override the distro-provided copies. The problem is that Qt doesn’t (currently) understand this, so it says “use the distro one”, but you find the vendor driver. CMake sees that this can be confusing and warns.

I don’t understand is, that normally nothing in paraview should be built statically or?

This warning only appears for builds against a shared libGL (read: pretty much always) when a vendor-provided libGL is not at the system location (read: proprietary drivers).

So normally it is linked against a filename and not against a path and then it should honor the path stuff from LD and use what is usable in the system and in which order the system specified the search path for libs.

Simplifying a little, ELF loading involves a set of SONAMEs (file names) and directories (RPATH, LD_LIBRARY_PATH, etc.). Each SONAME is looked up in each directory in order. CMake is warning that it cannot make an unambiguous set of directories for the given SONAME set that ensures the full paths given for the libraries are actually used. There is no “this library must be loaded from this full path” support in ELF.

I recommend reading man ld-linux for how ELF loading works if you want to know more.

But it seems that it has a path search order for libs built/written into the program code?

Qt is making assumptions and not using other paths to search.

Since libGL is a stable ABI on Linux, the warning is noise, but also cannot be suppressed/fixed without patching Qt’s CMake files (adding PATHS /usr/lib64/opengl/nvidia/lib to the find_library calls should work, but that’s a local fix). If the system ends up using the nvidia libGL at runtime, everything is OK. Otherwise, you may need to poke with LD_LIBRARY_PATH (please file an issue with VTK if this is the case so we can at least track it).

8

But isn’t it unimportant if it is unambiguous or not, because CMake just tries to make sure with finding the applications/libraries to meet the requirements?
So it checks, if it’s there and will continue with the procedure or stops the whole process or excludes a plugin or whatever is defined?

I did that partially and that’s why I don’t see the point why it still uses the wrong path during runtime.

Even for the runtime or only during the search of requirements? If the second is true then it should use the right directory during runtime because of the LD_LIBRARY_PATH anyway.

It is not using the nvidia libGL, because it will use the /usr/lib/libGL.so.1 when it is there.
If I rename it to /usr/lib/libGL.so.1.mesa, it cannot be used and then the /usr/lib/opengl/nvidia/lib/libGL.so.1 file will be used.
That’s what I meant with honoring the LD path order. That is not the case.
During runtime it starts the search for the lib in /usr/lib even when the LD path order says otherwise.

I just don’t get it, where the mistake is happening because if everything is dynamically linked then it should work in the end.

But thanks again for trying to explain. : )

9

But isn’t it unimportant if it is unambiguous or not, because CMake just tries to make sure with finding the applications/libraries to meet the requirements? So it checks, if it’s there and will continue with the procedure or stops the whole process or excludes a plugin or whatever is defined?

CMake is trying to make the runtime use the libraries you passed to the linker by default. It’s saying it can’t guarantee that, but whether it works or not is not something it passes judgement on (nor could it really), so it is just a warning rather than a failure.

If the second is true then it should use the right directory during runtime because of the LD_LIBRARY_PATH anyway.

As described in the Qt issue, the core goal of that code is to use the ANGLE support that Qt builds if it used it rather than the system one. Yes, it can’t force the runtime loader to use that exact library (LD_LIBRARY_PATH and LD_PRELOAD are always there to thwart such goals), but it can hint.

It is not using the nvidia libGL, because it will use the /usr/lib/libGL.so.1 when it is there.

If Qt finds the nvidia libGL.so, the generated RPATH could conclusively put it first in RPATH, but since it’s a cycle, CMake can’t order it and you get an undefined order. It just happens that it isn’t showing up first, so you’re losing out :confused: .

During runtime it starts the search for the lib in /usr/lib even when the LD path order says otherwise.

That’s the core reason why RPATH is deprecated in favor for RUNPATH. Unfortunately, RUNPATH is not inherited like RPATH is, so AFAIK, it usually ends up being used these days.

10

hey,

okay I think, now I got it. : )

Thanks again for your patience.

I now patched my local “/usr/lib/cmake/Qt5Gui/Qt5GuiConfigExtras.cmake” file and added
the PATHS string:
'------------------------------------------------------------------------------------------------------------------------
macro(qt5gui_find_extra_libs Name Libs LibDir IncDirs)
set(Qt5Gui${Name}LIBRARIES)
set(Qt5Gui${Name}_INCLUDE_DIRS ${IncDirs})
foreach(_lib ${Libs})
string(REGEX REPLACE [^_A-Za-z0-9] _ _cmake_lib_name ${lib})
if (NOT TARGET Qt5::Gui${_cmake_lib_name} AND NOT Qt5Gui${_cmake_lib_name}LIBRARY_DONE)
find_library(Qt5Gui${_cmake_lib_name}_LIBRARY ${_lib}
PATHS /usr/lib/opengl/nvidia/lib/ “${LibDir}”
NO_DEFAULT_PATH
)
'------------------------------------------------------------------------------------------------------------------------

And the CMake warning is gone and this is the output I added, so I guess it will be used:

– i am here:
– Qt5Gui_EGL_LIBRARIES

– _cmake_lib_name:EGL - everything: /usr/lib/opengl/nvidia/lib/libEGL.so

– i am here:
– Qt5Gui_OPENGL_LIBRARIES

– _cmake_lib_name:GL - everything: /usr/lib/opengl/nvidia/lib/libGL.so

BUUUT, during runtime it still uses the /usr/lib/libGL.so.1 file and not from the nvidia folder.

Do I have to do more?

As always thanks for your help and have a nice weekend.

11

What does the RPATH say for the binary you’re trying to run (readelf -d $binary)? An OpenGL-using library may have the relevant rpath as well. If you’re installing ParaView, it is likely that the rpaths are being stripped. Setting CMAKE_INSTALL_RPATH should help the install.

12

hmm, I checked the paraview binary and all libraries files:

readelf -d $(which paraview)
for i in $(equery f paraview |grep “.so."); do echo $i;readelf -ldews $i |grep -i "rpath”; done

but I only get runpath but not rpath stuff. (equery f - lists all files which are installed with the package)

so, I have no idea why it is loading the libGL.so from /usr/lib

here a strace -e trace=open,openat from paraview:
https://paste.pound-python.org/show/cVsBKIFbnMaFd1AZO0n8/

here a strace from everything:
https://paste.pound-python.org/show/v7krLIo1JhrB6aqHY2TM/

I will double check my paraview compilation tomorrow but any hints would be appreciated.

greetings and tanks again

13

LD_DEBUG=libs might be more useful than strace for figuring out what library is actually grabbing /usr/lib/libGL.so

14

here it is:
https://paste.pound-python.org/show/e0feTptozc9YNWgSzwIn/

line 316, it is just using the system search path, but I don’t know which lib is using the libGL.so

15

okay, I made another dump with LD_DEBUG=all

and got this:

 44296:     file=libGL.so.1 [0];  needed by /usr/lib64/libQt5Gui.so.5 [0]
 44296:     find library=libGL.so.1 [0]; searching
 44296:	 search path=/usr/lib64         (system search path)
 44296:	  trying file=/usr/lib64/libGL.so.1

so the system QT5 GUI lib is calling the wrong libGL.so.1

I guess it’s using its own Qt5GuiConfigExtras.cmake which will be in the source which will be used for compiling, so I guess, I have to modify this file before it’s getting used for compiling.

16

okay I compiled another program (testdisk with qphotorec), which uses QTgui for frontend and it uses the right libGL.so.

 10102:     file=libGL.so.1 [0];  needed by /usr/lib64/libQt5Gui.so.5 [0]
 10102:     find library=libGL.so.1 [0]; searching
 10102:	 search cache=/etc/ld.so.cache
 10102:	  trying file=/usr/lib64/opengl/nvidia/lib/libGL.so.1

but it doesn’t use cmake for compiling itself

I also patched the qtgui:

— a/src/gui/Qt5GuiConfigExtras.cmake.in 2019-01-29 17:01:11.058676548 +0100
+++ b/src/gui/Qt5GuiConfigExtras.cmake.in 2019-01-29 17:04:25.286675541 +0100
@@ -94,6 +94,8 @@
string(REGEX REPLACE “[^A-Za-z0-9]" "” _cmake_lib_name ${lib})
if (NOT TARGET Qt5::Gui${_cmake_lib_name} AND NOT Qt5Gui${_cmake_lib_name}LIBRARY_DONE)
find_library(Qt5Gui${_cmake_lib_name}_LIBRARY ${_lib}

  •           PATHS LD_LIBRARY_PATH \"${LibDir}\"

  •           NO_DEFAULT_PATH

!!IF !isEmpty(CROSS_COMPILE)
PATHS “${LibDir}”
!!IF !mac

but it didn’t help, the paraview binary (recompiled) still loads the wrong libGL.so