Failure to run ParaView 5.6.0-RC2 mesa build over remote X connection - Unable to find valid OpenGL 3.2 or later


(Joseph G. Hennessey) #21

I built ParaView with gcc 6.1.0 on a RHEL 7.4 system


(Utkarsh Ayachit) #22

I used your script (and directory structure) except on Ubuntu and still the binary works justs fine! As expected, --mesa option is not even required for this build since it automatically loads the mesa GL libs. I suspect you have LD_LIBRARY_PATH overrides or something like that that’s messing up the library lookup. What do you get if you do paraview --print. I get something like this:


 ~/K/T/D/p/5.6.0-RC2_mesa_mpich> ./bin/paraview --print
LD_LIBRARY_PATH=/home/utkarsh/Kitware/Temp/DAAC/pv-test/5.6.0-RC2_mesa_mpich/lib:/home/utkarsh/Kitware/Temp/DAAC/pv-test/5.6.0-RC2_mesa_mpich
/home/utkarsh/Kitware/Temp/DAAC/pv-test/5.6.0-RC2_mesa_mpich/lib/paraview

(Joseph G. Hennessey) #23

Utkarsh,

I have modified my LD_LIBRARY_PATH so it only has

…/pv-test/5.6.0-RC2_mesa_mpich/lib

in it, as your LD_LIBRARY_PATH has,

which I have confirmed by running paraview with the --print option

but I am still seeing the same errors. When I run with the --ldd option,

I am still seeing the libraries from /usr/lib64 being used. Is their a possibility
that /usr/lib64 is getting baked in at compile time? Since if they are truly
being loaded dynamically it is not clear to me how it is that libraries from
/usr/lib64 are being used when it is not in the LD_LIBRARY_PATH.

Thanks,

Joe


(Utkarsh Ayachit) #24

try running with LD_DEBUG=libs environment set. That will show how it’s looking for the libGL etc and how the one from /usr/lib64 is getting loaded. Maybe that will shed some light.


(Joseph G. Hennessey) #25

Utkarsh,

Running with the LD_DEBUG=libs environment set

I find that it is searching for libGLX.so.0 and libGLU.so.0 in the

…/pv-test/5.6.0-RC2_mesa_mpich/lib

directory not finding them (They are not there) and then trying to pull them from

/etc/ld.so.cache

which does have them in it in under /lib64 which is a symbolic link to /usr/lib64

It is doing the same with libOpenGL.so and libGLdispatch.so.0
finding them in /lib64 from the ld.so.cache

It does not seem to be looking for libGL.so or libGL.so.1 which actually exist in

…/pv-test/5.6.0-RC2_mesa_mpich/lib

also when it inits /lib64/libXcursor.so.1 it is searching for and finding libxcb-glx.so.0
again finding it in /lib64 from the ld.so.cache

finally when it inits …/pv-test/5.6.0-RC2_mesa_mpich/lib/plugins/platforminputcontexts/libcomposeplatforminputcontextplugin.so

it searches for libGLX_nvida.so.0, libnvidia-tls.so.390.46 and libnvidia-glcore.so.390.46
finding them all in /lib64 from the ld.so.cache

when it then attempts to init /lib64/libGLX_nvidia.so.0

I get the error

/lib64/libpthread.so.0: error: symbol lookup error: undefined symbol: pthread_setname_np, version GLIBC_2.2.5 (fatal)

Thanks,

Joe


(Utkarsh Ayachit) #26

What is triggering the request to load libGLX? Let’s find that out. In my version, nothing looks for libGLX.


(Joseph G. Hennessey) #27

Utkarsh,

paraview on startup is searching for /lib64/libc.so.6
which initializes paraview
and then transfers control to paraview

paraview then tries to find many libraries including for example

…/build/PV/Build_5.6.0-RC2_mesa_mpich/install/lib/libvtkpqCore-pv5.6.so.1

which for example is the fifth one that it tries to find

ldd on libvtkpqCore-pv5.6.so.1 shows that it is dependent on

libGLX.so.0, libGLU.so.1, libOpenGL.so.0, libGL.so.1 and libGLdispatch.so.0

So, it looks like these dependencies are getting pulled in while compiling and linking.

as these libraries are all searched for before the next init where it is trying to init /lib64/libpthread.so.0

Thanks,

Joe


(Joseph G. Hennessey) #28

Utkarsh,

This is all occurring on the Centennial HPCMP system by the way.

Thanks,

Joe


(Utkarsh Ayachit) #29

Joe, since I don’t have access to the machine, we’ll have to stick with this back and forth, I am afraid.

Let’s try this…

cd .../PV/Build_5.6.0-RC2_mesa_mpich/superbuild/paraview/build
# manually edit CMakeLists.txt to empty out OPENGL_glx_LIBRARY 
# and OPENGL_glu_LIBRARY
> make -j ...
> make install
> cd PV/Build_5.6.0-RC2_mesa_mpich
> ./install/bin/paraview

(Joseph G. Hennessey) #30

Utkarsh,

When I manually edit CMakeCache.txt (I think that is what you meant), to remove all of the calls to any of the /lib64/libGLU., /lib64/libGLX., /lib64/libEGL*, /lib64/libOpenGL*, or /lib64/libGLdispatch* libraries and then
do make -j … ; make install; etc, and run that version of paraview, then paraview works properly reporting that it finds GLEW_VERSION_3_3 and everything functions properly. So, it looks like something is wrong in the detection of the correct libraries to use for ParaView and the origin of the problem may lie in the interaction of cmake 3.12.2 and ParaView where something is causing it to add libraries that should not be used.

Thanks,

Joe


(Joseph G. Hennessey) #31

Utkarsh,

Any suggestions as to which component of ParaView’s CMake is pulling in the extraneous libraries?

Thanks,

Joe


(Utkarsh Ayachit) #32

@chuckatkins @ben.boeckel @Dave_DeMarle, any suggestions how to handle this? We could force set all these vars in the superbuild to empty if mesa is enabled.


(Joseph G. Hennessey) #33

Utkarsh,

I believe I have found the source of the problem. I have built ParaView 5.6.0-RC2 again with the same options from scratch with close attention to the CMake output and I found the warnings below where it says it is finding two versions of GL and using the legacy version. But when one looks at the CMakeCache.txt instead of just using the legacy version, it adds both sets of libraries to many of the paths. I used cmake 3.12.2, I know cmake 3.12.3 is out now, but it does not show any related bugs being fixed.

Thanks,

Joe

CMake Warning (dev) at …/build/PV/Build_5.6.0-RC2_mesa_mpich/install/share/cmake-3.12/Modules/FindOpenGL.cmake:270 (message):

Policy CMP0072 is not set: FindOpenGL prefers GLVND by default when
available. Run “cmake --help-policy CMP0072” for policy details. Use the
cmake_policy command to set the policy and suppress this warning.

FindOpenGL found both a legacy GL library:

OPENGL_gl_LIBRARY:.../build/PV/Build_5.6.0-RC2_mesa_mpich/install/lib/libGL.so

and GLVND libraries for OpenGL and GLX:

OPENGL_opengl_LIBRARY: /usr/lib64/libOpenGL.so
OPENGL_glx_LIBRARY: /usr/lib64/libGLX.so

OpenGL_GL_PREFERENCE has not been set to “GLVND” or “LEGACY”, so for
compatibility with CMake 3.10 and below the legacy GL library will be used.
Call Stack (most recent call first):

VTK/CMake/vtkOpenGL.cmake:127 (find_package)
Plugins/StreamLinesRepresentation/CMakeLists.txt:13 (include)

This warning is for project developers. Use -Wno-dev to suppress it.


(Joseph G. Hennessey) #34

Utkarsh,

So, I tried setting OpenGL_GL_PREFERENCE in ParaView’s main CMakeLists.txt file

set(OpenGL_GL_PREFERENCE “LEGACY”)

I no longer get the warning message, but sadly even after deleting the entire ParaView build
directory, the extra libraries still get added during the cmake configuration and generation.
They are just not warned about any more.

Thanks,

Joe


(Joseph G. Hennessey) #35

Utkarsh,

It would seem that the logic in FindOpenGL.cmake from cmake 3.12.2 is flawed.
FindOpenGL.cmake will always search for
libOpenGL.so and libGLX.so
and If if finds them (which it does in my case) it will always add them to the path as long as OPENGL_USE_EGL is not set. (which I am not setting as I want to use MESA)

as seen in lines 197-207

 # Search for the GLVND libraries.  We do this regardless of COMPONENTS; we'll
 # take into account the COMPONENTS logic later.
  find_library(OPENGL_opengl_LIBRARY
    NAMES OpenGL
    PATHS ${_OPENGL_LIB_PATH}
  )

  find_library(OPENGL_glx_LIBRARY
    NAMES GLX
    PATHS ${_OPENGL_LIB_PATH}
  )

and lines 290-303

  # now we append the libraries as appropriate.  The complicated logic
  # basically comes down to "use libOpenGL when we can, and add in specific
  # context mechanisms when requested, or we need them to preserve the previous
  # default where glx is always available."
  if((NOT OPENGL_USE_EGL AND
      NOT OPENGL_opengl_LIBRARY AND
          OPENGL_glx_LIBRARY AND
      NOT OPENGL_gl_LIBRARY) OR
     (NOT OPENGL_USE_EGL AND
      NOT OPENGL_glx_LIBRARY AND
      NOT OPENGL_gl_LIBRARY) OR
     (NOT OPENGL_USE_EGL AND
          OPENGL_opengl_LIBRARY AND
          OPENGL_glx_LIBRARY) OR
     (    OPENGL_USE_EGL))
    list(APPEND _OpenGL_REQUIRED_VARS OPENGL_opengl_LIBRARY)
  endif()

  # GLVND GLX library.  Preferred when available.
  if((NOT OPENGL_USE_OPENGL AND
      NOT OPENGL_USE_GLX AND
      NOT OPENGL_USE_EGL AND
      NOT OPENGL_glx_LIBRARY AND
      NOT OPENGL_gl_LIBRARY) OR
     (    OPENGL_USE_GLX AND
      NOT OPENGL_USE_EGL AND
      NOT OPENGL_glx_LIBRARY AND
      NOT OPENGL_gl_LIBRARY) OR
     (NOT OPENGL_USE_EGL AND
          OPENGL_opengl_LIBRARY AND
          OPENGL_glx_LIBRARY) OR
     (OPENGL_USE_GLX AND OPENGL_USE_EGL))
    list(APPEND _OpenGL_REQUIRED_VARS OPENGL_glx_LIBRARY)
  endif()

Thanks,

Joe


(Utkarsh Ayachit) #36

Joe, for now let’s stick with CMake older than 3.12. I am sure the VTK/ParaView code needs updating to use the new FindOpenGL properly. Since I am not sure I can bring my self to up to speed with the cmake changes and than be brave enough to update the VTK/ParaView code for the release, it’s worth tackling this one for the next. I’ve reported the issue here: https://gitlab.kitware.com/paraview/paraview/issues/18529


(Joseph G. Hennessey) #37

Utkarsh,

As an update, I have tried building the ParaView 5.6.0-RC2 superbuild for mesa
from scratch with cmake 3.11.4 (which works properly), The warning about finding 2 versions of GL
LEGACY and GLNVD still shows up, but only the LEGACY OpenGL libraries are added.

With cmake 3.12.3 (which does not work properly in the same manner as cmake 3.12.2)
the warning about finding 2 versions of GL show up and the libraries from both versions of OpenGL are added.

Fixing this is going to be difficult though, as a diff of the FindOpenGL.cmake between 3.12.3
and 3.11.4 show no difference. So while I do think the FindOpenGL.cmake is involved something
else must have also changed with regards to cmake and OpenGL between versions 3.11.4 and 3.12.3 of cmake to make 3.12.3 add both versions of the OpenGL libraries while 3.11.4 only adds the legacy versions.

Thanks,

Joe