Failure to run ParaView 5.6.0-RC2 mesa build over remote X connection - Unable to find valid OpenGL 3.2 or later


(Joseph G. Hennessey) #1

Hello,

I was trying to run the Mesa version of ParaView 5.6.0-RC2 over a remote X connection.
This was possible with the Mesa version of ParaView 5.5.1, but it seems to no longer be possible
in 5.6.0-RC2. The error I am getting is vtkGenerOpenGLRenderWindow is unable to find a valid OpenGL 3.2 or later implementation. Internally the file is checking for GLEW_VERSION_3_1 or GLEW_VERSION_3_2 but it seems remote X is limited to GLEW_VERSION_3.0 and so it fails to run.
Is there a way to get a Mesa version of ParaView 5.6.0-RC2 to work over a remote X connections, or is this just no longer going to be possible?

Thanks,

Joe


(Utkarsh Ayachit) #2

Can you try forcing the gl version? You can do that be setting the environment variable MESA_GL_VERSION_OVERRIDE=3.2.


(Joseph G. Hennessey) #3

Utkarsh,

I tried setting MESA_GL_VERSION_OVERRIDE to 3.2 and it did not work.
I searched the libGL.so produced by mesa and found that value in the file.

The error is coming from libvtkglew-pv5.6.so though that is where GLEW_VERSION_3_0
is being set to True, but higher versions such as GLEW_VERSION_3_1 and GLEW_VERSION_3_2 are
being set to false. The override does not seem to affect them and the check in
vtkOpenGLRenderWindow.cxx is still failing over remote X.

Thanks,

Joe


(Shawn Waldon) #4

I don’t see any changes between 5.5 and 5.6 in the version of glew that ships with VTK. I thought it might have been the update to a more recent version of glew, but that change was in 5.5.


(Joseph G. Hennessey) #5

Utkarsh,

I should say I found MESA_GL_VERSION_OVERRIDE in the libGL.so produced by mesa
not is value.

I think the problem is coming from remote X which does not support OpenGL 3.2 and
libvtkglew is detecting this instead of the value set by MESA_GL_VERSION_OVERRIDE,
when determining what the OpenGL version is. It looks like libvtkglew is parsing the
name of the string returned by glGetString but that somehow this is not being set properly
using the override by libGL with remote X.

Thanks,

Joe


(Joseph G. Hennessey) #6

Shawn,

While vtkglew has not changed, what has changed though is that vtkOpenGLRenderWindow.cxx
in OpenGLInitContext() it would in 5.5.1 call this->setContextSupportsOpenGL32(true) and continue when
the check for GLEW_VERSION_3_2 or GLEW_VERSION_3_1 failed.
While in 5.6.2 does not set anything equivalent perhaps these changes in vtkOpenGLRenderWindow.cxx
are causing the change in functionality. It worked in 5.5.1 but segfaults with version 5.6.0-RC2.

Thanks,

Joe


(Joseph G. Hennessey) #7

I have also tried setting PV_DEBUG_SKIP_OPENGL_VERSION_CHECK with this set I get a segault,
with it not set, I get vtkXOpenGLRenderWindow.cxx line 796 failed to create offscreen window and
I get vtkOpenGLRenderWindow.cxx line 736 vtkXOpenGLRenderWindow: GLEW could not be initialized.

Thanks,

Joe


(Joseph G. Hennessey) #8

It would seem that vtkOpenGLRenderWindow::CreateHardwareOffScreenWindow is failing.
Perhaps, I will need to create a debug build and trace this down more thoroughly to detect
the exact failure points and values at those points. I had hoped that someone would have
already hit this issue, but perhaps they have not.

Thanks,

Joe


(Utkarsh Ayachit) #9

Let me try to see if I can recreate this. So to reproduce your environment, is the following correct?

  1. a remote ® system with OpenGL < 3.2.
  2. a local system (L) with OpenGL >= 3.2
  3. on L, run ssh -X [remote hostname] paraview --mesa`

Is that correct?


(Joseph G. Hennessey) #10

Utkarsh,

  1. The remote system does not have hardware acceleration and is using mesa (i.e. the libGL.so built by mesa)
    where mesa should be reporting OpenGL > 3.2
  2. A local system (L) with hardware acceleration with OpenGL >= 3.2
  3. On L, run ssh -Y [remote hostname] and then once logged on running paraview --mesa
    3a) It is my understanding that the remote X protocol itself is probably the culprit, in that the
    higher versions of OpenGL are not supported over remote X, and that the remote X protocol
    reports that OpenGL is 3.0 instead of > 3.2 in that it is not capable of passing some of the newer
    calls. Under ParaView 5.5.1 I had to use PV_DEBUG_SKIP_OPENGL_VERSION_CHECK set to 1
    in order to get it to work, this no longer works for ParaView 5.6.0-RC2.

Thanks,

Joe


(Utkarsh Ayachit) #11

Joe, here’s what I tried.

From my desktop:

> ssh -Y vall 
> tar zxvf ParaView-5.6.0-RC2-Qt5-MPI-Linux-64bit.tar.gz
> cd ParaView-5.6.0-RC2-Qt5-MPI-Linux-64bit/
> ./bin/paraview  
         # ---- SEGFAULT! This is using hardware accelerated
         # OpenGL drivers installed on the machine
> ./bin/paraview --mesa
         # --- works!
         # I get a transparent opengl window -- but that's separate
         # issue that will be fixed before the final release.

Can you test with the ParaView binaries to confirm that appropriate mesa build getting used please? Thanks


(Joseph G. Hennessey) #12

Utkarsh,

Here is what I found, when I use the downloaded Linux binary.

./bin/paraview

Does not segfault, but does not produce the render window either. I am getting the same errors that I got with my superbuild compiled version, where it is complaining about about the error of not finding OpenGL 3.2 or later in vtkOpenGLRenderWindow.cxx line 749, the next error is that vtkXOpenGLRenderWindow.cxx line 796 failed to create an offscreen window, followed by an error in vtkOpenGLRenderWindow.cxx on line 736 GLEW could not be initialized.

This is basically the same behavior that I am seeing with the mesa build of ParaView-RC2 built using the superbuild script, even when using the --mesa flag on the superbuild built version.

When I run the download binary version with --mesa
./bin/paraview --mesa
It works mostly. The initial renderview is corrupted, but once I add a sphere and move the mouse it starts working correctly (the corruption of the window goes away) and I can see the sphere and it does not crash.
Perhaps, their is an issue with clearing the windows initially which is causing the initial corruption of the view.

It is unclear to me, why the downloaded binary is behaving differently then the superbuild compiled binary, but it is encouraging that it is functioning better. The mesa version built by the superbuild seems to be acting as if it were the hardware accelerated version even though I am sending it the --mesa flag to it.

Thanks,

Joe


(Utkarsh Ayachit) #13

yes, this is indeed the issue i referenced in my post. This is separate issue and will be fixed before the final release.

Let’s dive in further. Can you do a bin/paraview -ldd on your build of ParaView with and without --mesa and see what the dependencies it finds. Is it correctly referencing Mesa GL libs built by the superbuild?


(Joseph G. Hennessey) #14

Utkarsh,

I ran the check on the downloaded binaries and the superbuild built binaries.
with – ldd and with and without --mesa. Here is what I have found.

Downloaded binary version
libGL.so.1 => /lib64/libGL.so.1
libGLU.so.1 => /lib64/libGLU.so.1
libGLX.so.0 => /lib64/libGLX.so.0
libGLdispatch.so.0 => /lib64/libGLdispatch.so.0

Downloaded binary version with --mesa flag
libGL.so.1 => … /Paraview-5.6.0-RC2-Qt5-MPI-Linux-64bin/lib/mesa/libGL.so.1
libGLU.so.1 => /lib64/libGLU.so.1
no entries for libGLX.so.0 or libGLdispatch.so.0

Superbuilt compiled version
libGL.so.1 => … /Build_5_6_0-RC2_mesa_mpich/install/lib/libGL.so.1
libGLU.so.1 => /lib64/libGLU.so.1
libGLX.so.0 => /lib64/libGLX.so.0
libGLdispatch.so.0 => /lib64/libGLdispatch.so.0

Superbuilt compiled version with --mesa flag
libGL.so.1 => … /Build_5_6_0-RC2_mesa_mpich/install/lib/libGL.so.1
libGLU.so.1 => /lib64/libGLU.so.1
libGLX.so.0 => /lib64/libGLX.so.0
libGLdispatch.so.0 => /lib64/libGLdispatch.so.0

For the superbuild built version, the --mesa flag seems to have an effect on the libraries used for GL.

Thanks,

Joe


(Joseph G. Hennessey) #15

Utkarsh,

My superbuild compiled version, was compiled with
ENABLE_mesa=ON

Thanks,

Joe


(Utkarsh Ayachit) #16

roger that…let me do a clean build from scratch and see what’s going on. BTW, where do run the paraview package from? i.e. do you do make install or cpack or directly from the build directory?


(Joseph G. Hennessey) #17

Utkarsh,

I run the superbuild build script called build_script.sh in directory call build->PV
The build script builds cmake and then runs cmake on the superbuild source to configure ParaView in a subfolder called Build_5.6.0-RC2_mesa_mpich. I have the superbuild source itself in a subfolder called Source_5.6.0 and all of the downloads reside in a subfolder Downloads (I also have the cmake tar ball there as well). I am setting the install directory pv-test->5.6.0-RC2 and then running make install to actually build everything.

Here is the folder and file structure that I am using.

build->PV
build->PV->Downloads
build->PV->Source_5.6.0
build->build_script.sh
build->Build_5.6.0-RC2_mesa_mpich
build->pv-test
build->pv-test->5.6.0-RC2

Below is the csh script I am calling to build and install ParaView

Thanks,

Joe

#!/bin/csh

set paraview_version=5.6.0-RC2
set cmake_version=3.12.2
set build_extension=_mesa_mpich

if (! -d Build_${paraview_version}${build_extension}) mkdir Build_${paraview_version}${build_extension}
cd Build_${paraview_version}${build_extension}

if (! -d install) mkdir install
set INSTALLPATH=`pwd`/install

if (! -e ${INSTALLPATH}/bin/cmake) then
        echo "Need to build cmake"
        tar xzvf ../Downloads/cmake-${cmake_version}.tar.gz
        cd cmake-${cmake_version}
        ./configure --prefix=${INSTALLPATH}
        make -j 8 install
        cd ..
endif

set path=(${INSTALLPATH}/bin $path);rehash
setenv LD_LIBRARY_PATH ${INSTALLPATH}/install/lib:${LD_LIBRARY_PATH}
echo $LD_LIBRARY_PATH

cmake \
-DCMAKE_INSTALL_PREFIX:PATH=`pwd`/../../pv-test/${paraview_version}${build_extension} \
-DCMAKE_BUILD_TYPE=Release \
-DParaView_FROM_GIT=OFF \
-Dsuperbuild_download_location=`pwd`/../Downloads \
-DParaView_URL=`pwd`/../Downloads/ParaView-v${paraview_version}-source.tar.gz \
-DENABLE_boost=ON \
-DBoost_NO_BOOST_CMAKE:BOOL=ON \
-DENABLE_visitbridge:BOOL=ON \
-DENABLE_ffmpeg=ON \
-DENABLE_hdf5=ON \
-DENABLE_fontconfig=ON \
-DENABLE_freetype=ON \
-DENABLE_libxml2=ON \
-DENABLE_matplotlib=OFF \
-DENABLE_osmesa=OFF \
-DENABLE_mesa=ON \
-DENABLE_mpi=ON \
-DENABLE_numpy=ON \
-DENABLE_ospray=ON \
-DENABLE_cxx11=ON \
-DENABLE_paraview=ON \
-DENABLE_paraviewweb=ON \
-DENABLE_python=ON \
-DENABLE_qt5=ON \
-DENABLE_png=ON \
-DENABLE_qhull=ON \
-DENABLE_szip=ON \
-DPARAVIEW_USE_VISITBRIDGE=ON \
-DENABLE_cgns=OFF \
-DENABLE_silo=ON \
-DENABLE_tbb=ON \
-DENABLE_vtkm=ON \
-DUSE_SYSTEM_mpi=OFF \
-DUSE_SYSTEM_python=OFF \
-DUSE_SYSTEM_zlib=OFF \
-DSUPERBUILD_PROJECT_PARALLELISM=8 \
-Dparaview_SOURCE_SELECTION="5.6.0-RC2" \
-Dvtkm_SOURCE_SELECTION="stable" \
-DPARAVIEW_RENDERING_BACKEND:STRING="OpenGL2" \
-DVTK_RENDERING_BACKEND:STRING="OpenGL2" \
../Source_5.6.0

make install

(Utkarsh Ayachit) #18

So for my build, it nicely finds libGL.so from the mesa built by the superbuild when running from both build directory or installed location. let me try with your flags. can you confirm the SHA for your ParaView superbuild and that you’ve run git submodule update on the superbuild repo as well? My superbuild checkout is at “d725a65b12d5510b22311a48b36db7106bdc385a”.


(Joseph G. Hennessey) #19

Utkarsh,

I am using the tag for v5.6.0-RC2

v5.6.0-RC2 ParaView superbuild 5.6.0-RC2
d725a65b · Update the default version to 5.6.0-RC2 · 1 week ago

I ran inside my build->PV directory the following commands

git clone --recursive https://gitlab.kitware.com/paraview/paraview-superbuild.git Source_5.6.0
git fetch origin
git checkout v5.6.0-RC2
git submodule update

When I run git rev-parse HEAD in the directory build->Source_5.6.0 I get

d725a65b12d5510b22311a48b36db7106bdc385a

Which matches your value exactly.

Thanks,

Joe


(Joseph G. Hennessey) #20

I ran the git commands

git fetch origin
git checkout v5.6.0-RC2
git submodule update

git rev-parse HEAD

all in the directory build->Source_5.6.0

using git 2.19.0

Thanks,

Joe