Building Paraview "from scratch" in order to use a custom plugin

This error message is quite common. This most probably means your login or compute nodes don’t support windowed rendering. So it cannot render with OpenGL. You have to use a windowless offscreen renderer like OSMesa (for CPU rendering) or EGL (for GPU rendering) to build Paraview on the server side. You can’t build a working version on server side with Qt or OpenGL rendering.

There are nodes that support windowed rendering - that you have to reserve and connect to via RDP. I’ve tested the base v5.6.0 from the main paraview website via those nodes and works fine and handles example files etc. I don’t know anything about which mesa version those nodes use, but I need to type paraview --mesa for it to work so it definitely uses mesa.

Yes @npillai2 you need to have windowed rendering on nodes. However a much robust solution would be to use OsMesa (and not Mesa) as the renderer which will work on any node windowed or windowless.

@dutta-alankar

It seems like OSmesa does not already exist on the machine. I’m hoping that I don’t have to jump through a bunch of hoops to also install OSMesa.

@mwestphal

I tried an install with GNU complilers (4.8.5 I believe), and the install completed without any errors. However when I try running the executable (aliased to “runparaview”), I get this error despite trying to point to various places in the Qt installation

[22:18] npillai2@vclhpc2:~ $ setenv LD_LIBRARY_PATH $LD_LIBRARY_PATH\:/usr/local/usrapps/iabolotn/Qt5.9.0/5.9/gcc_64/lib
[22:21] npillai2@vclhpc2:~ $ runparaview --mesa                                                       This application failed to start because it could not find or load the Qt platform plugin "xcb"
in "".

Available platform plugins are: eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, xcb.

Reinstalling the application may fix this problem.
Abort (core dumped)
[22:21] npillai2@vclhpc2:~ $ runparaview
This application failed to start because it could not find or load the Qt platform plugin "xcb"
in "".

Available platform plugins are: eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, xcb.

Reinstalling the application may fix this problem.
Abort (core dumped)
[22:21] npillai2@vclhpc2:~ $ setenv LD_LIBRARY_PATH $LD_LIBRARY_PATH\:/usr/local/usrapps/iabolotn/Qt5.9.0/5.9/gcc_64/plugins/platforms
[22:24] npillai2@vclhpc2:~ $ runparaview                                                              This application failed to start because it could not find or load the Qt platform plugin "xcb"
in "".

Available platform plugins are: eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, xcb.

Reinstalling the application may fix this problem.
Abort (core dumped)
[22:24] npillai2@vclhpc2:~ $ setenv PATH $PATH\:/usr/local/usrapps/iabolotn/Qt5.9.0/5.9/gcc_64/plugins/platforms
[22:25] npillai2@vclhpc2:~ $ runparaview
This application failed to start because it could not find or load the Qt platform plugin "xcb"
in "".

Available platform plugins are: eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, xcb.

Reinstalling the application may fix this problem.
Abort (core dumped)
[22:25] npillai2@vclhpc2:~ $ setenv LD_LIBRARY_PATH $LD_LIBRARY_PATH\:/usr/local/usrapps/iabolotn/Qt5.9.0/5.9/gcc_64/plugins/xcbglintegrations/
[22:26] npillai2@vclhpc2:~ $ runparaview
This application failed to start because it could not find or load the Qt platform plugin "xcb"
in "".

Available platform plugins are: eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, xcb.

Reinstalling the application may fix this problem.
Abort (core dumped)
[22:26] npillai2@vclhpc2:~ $ runparaview --mesa
This application failed to start because it could not find or load the Qt platform plugin "xcb"
in "".

Available platform plugins are: eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, xcb.

Reinstalling the application may fix this problem.
Abort (core dumped)

If this is a random error, I don’t mind going through a reinstall, but I was wondering if you’d seen this before and if there was a simple solution to it.

I just tried using superbuild to get osmesa, and I can’t find my way around this error

[ 92%] Performing configure step for 'osmesa'
autoreconf: Entering directory `.'
autoreconf: configure.ac: not using Gettext
autoreconf: running: aclocal --force -I m4
autoreconf: configure.ac: tracing
autoreconf: running: libtoolize --copy --force
/usr/bin/libtoolize: line 1284: /usr/bin/touch: Permission denied
libtoolize: putting auxiliary files in AC_CONFIG_AUX_DIR, `bin'.
libtoolize: can not copy `/usr/share/libtool/config/ltmain.sh' to `bin/'
/usr/bin/libtoolize: line 1284: /usr/bin/touch: Permission denied
libtoolize: putting macros in AC_CONFIG_MACRO_DIR, `m4'.
libtoolize: can not copy `/usr/share/aclocal/libtool.m4' to `m4/'
/usr/bin/libtoolize: line 1284: /usr/bin/touch: Permission denied
libtoolize: can not copy `/usr/share/aclocal/ltoptions.m4' to `m4/'
/usr/bin/libtoolize: line 1284: /usr/bin/touch: Permission denied
libtoolize: can not copy `/usr/share/aclocal/ltsugar.m4' to `m4/'
/usr/bin/libtoolize: line 1284: /usr/bin/touch: Permission denied
libtoolize: can not copy `/usr/share/aclocal/ltversion.m4' to `m4/'
/usr/bin/libtoolize: line 1284: /usr/bin/touch: Permission denied
libtoolize: can not copy `/usr/share/aclocal/lt~obsolete.m4' to `m4/'
autoreconf: libtoolize failed with exit status: 1
CMake Error at /home/npillai2/custom_software/superbuild_paraview_5.8.0_gcc/build/superbuild/sb-osmesa-configure.cmake:47 (message):
  Failed with exit code 1

If I do

cmake -make \
 -DAC_CONFIG_AUX_DIR=/usr/local/usrapps/iabolotn/superbuild_paraview_5.8.0_gcc/build/local_bin \
 -DAC_CONFIG_MACRO_DIR=/usr/local/usrapps/iabolotn/superbuild_paraview_5.8.0_gcc/build/m4 \
 ../paraview-superbuild

The variables don’t get accepted.

CMake Warning:
  Manually-specified variables were not used by the project:

    AC_CONFIG_AUX_DIR
    AC_CONFIG_MACRO_DIR

Plus I think the issue is at

/usr/bin/libtoolize: line 1284: /usr/bin/touch: Permission denied

which is before the other variables come into play. Has anyone seen these errors before?

@npillai2
I remember getting an error similar to this back in January while trying to compile Paraview. But I can’t recall what exactly solved the issue.
Have you tried these?

set(ENABLE_osmesa OFF CACHE BOOL “”)
set(mesa_SWR_ARCH “avx,avx2” CACHE STRING “”)

I’m not sure why but I remember for OsMesa I was advised to try this.
You can see the thread of this post for reference

I had the same setting for mesa_SWR_ARCH, but I cannot imagine why setting osmesa to OFF would help me configure a build with osmesa (my dry build with all defaults worked - which had osmesa off), but the issue is I want to make a build with the GUI client, and this error also occurs when I try to compile with regular mesa.

Hi all,

I am the IT person in question.

If I can get help with what should work, maybe I can figure out why it doesn’t work.

Here is what I understand:

  • (We’re not adding firewall exceptions to allow pvserver.)
  • Naveen wants to interact with the GUI, not just submit batch visualization, so OSMesa is not sufficient.
  • On our system, Paraview will be able to open a window, but we don’t have GPUs.

I was trying 5.6.0 because that is the version we have installed as binaries and it works well. That is the only reason. If 5.8.0 has the most testing, I am happy to use that.

I thought I need ‘mesa’ option, but when I build with mesa, it doesn’t make a paraview executable. I can build with ‘qt5’, but then I get the OpenGL not supported error at runtime. When I said ‘qt not compatible with mesa’ in what Naveen is quoting, I just mean in the options in ccmake for superbuild are incompatible.

So if someone can just let me know what should theoretically work (am I supposed to get executable? am I supposed to use mesa? Will QT5 work if I link to some CPU GL libraries…???) If I know what should work, then I can sort out better if there are issues on our system, or whether I am just making incompetent cmake choices… I am forever grateful for the help.

I am using gcc compiler 4.8.5.

(We’re not adding firewall exceptions to allow pvserver.)

Which port can application use to communicate ? Is ssh communication ok ?

Naveen wants to interact with the GUI, not just submit batch visualization, so OSMesa is not sufficient.

pvserver with osmesa on a server and ParaView on a client would work.

On our system, Paraview will be able to open a window, but we don’t have GPUs.

pvserver + osmesa then

@npillai2 would then connect to the pvserver using any ParaView client.

Hi Mathieu, thanks for responding.

Our setup allows ssh from the campus network to a login node, then you can only access a compute node via the scheduler (LSF). ssh to login, then LSF to compute node. Applications (ParaView) may not be run on a login node. A user may reserve an interactive session on a compute node through LSF. (But then ParaView would need to be build with the GUI).

I was staring at your past threads and the GitHub issues where you are discussing documentation. I thought we need QT GUI with mesa library, but when I was reading superbuild README, I misunderstood and took ‘qt5, mesa, osmesa, egl are mutually exclusive’ to mean ‘you can’t specify more than one of those cmake options when configuring’.

So just now, I finished a successful install by choosing enable_qt5 and enable_mesa. I was just stuck on thinking it wasn’t possible to pick both. It worked fine.

Thanks again!
Lisa

Our setup allows ssh from the campus network to a login node, then you can only access a compute node via the scheduler (LSF). ssh to login, then LSF to compute node. Applications (ParaView) may not be run on a login node.

Then ssh tunneling is the solution for accessing pvserver from outside.

A user may reserve an interactive session on a compute node through LSF. (But then ParaView would need to be build with the GUI).

GUI + Mesa should work though.

I was staring at your past threads and the GitHub issues where you are discussing documentation. I thought we need QT GUI with mesa library, but when I was reading superbuild README, I misunderstood and took ‘qt5, mesa, osmesa, egl are mutually exclusive’ to mean ‘you can’t specify more than one of those cmake options when configuring’.

This documentation is confusing but we have not been able to improve it yet.
See here for reference : https://gitlab.kitware.com/paraview/paraview-superbuild/-/merge_requests/690

I finished a successful install by choosing enable_qt5 and enable_mesa. I was just stuck on thinking it wasn’t possible to pick both. It worked fine.

ok.

Hi Mathieu - does this mean that there’s a way to access pvserver without changes to the firewall? This is definitely the ideal scenario for both me and my group.

You can only access a compute node via LSF from a login node.

ssh tunnelling allows you to open a window on the local machine while you are using ssh. You cannot ssh into a compute node. You cannot run anything on a login node. I really don’t understand why anyone is saying that pvserver will work within those constraints.

I got ParaView to compile on our system, so I suspect that is the way to move forward on this.

Could also be worthwhile taking a look at what CINECA is doing
http://www.hpc.cineca.it/services/remote-visualisation

You mean LSF, once connected, do not provide an ip that can be connected to using either ssh or any other port ?

It does not.

Then, you are right, the pvserver solution won’t work. Qt + mesa seems indeed to be the way to go.

I just tried a superbuild with gcc compiler 4.8.5 enabling mesa, qt5, python2 & mpi.

With superbuild 5.6.0, I run into this error (seems mpi related).

[ 89%] Linking CXX shared library …/…/…/lib/libvtkPVServerManagerDefault-pv5.6.so
[ 89%] Building CXX object Wrapping/ClientServer/CMakeFiles/vtkPVServerManagerRenderingCS.dir/vtkSMRepresentationProxyClientServer.cxx.o
[ 89%] Built target vtkPVServerManagerDefault
Scanning dependencies of target vtkLegacyColorMapXMLToJSON
[ 89%] Building CXX object Utilities/LegacyColorMapXMLToJSON/CMakeFiles/vtkLegacyColorMapXMLToJSON.dir/vtkLegacyColorMapXMLToJSON.cxx.o
[ 89%] Building CXX object Wrapping/ClientServer/CMakeFiles/vtkPVServerManagerRenderingCS.dir/vtkSMRepresentationTypeDomainClientServer.cxx.o
[ 89%] Linking CXX executable …/…/bin/vtkLegacyColorMapXMLToJSON
…/…/lib/libIceTMPI.so: undefined reference to ompi_mpi_comm_null'** **../../lib/libIceTMPI.so: undefined reference to ompi_mpi_short’
…/…/lib/libIceTMPI.so: undefined reference to ompi_mpi_byte'** **../../lib/libIceTMPI.so: undefined reference to ompi_request_null’
…/…/lib/libIceTMPI.so: undefined reference to ompi_mpi_int'** **../../lib/libIceTMPI.so: undefined reference to ompi_mpi_double’
…/…/lib/libIceTMPI.so: undefined reference to `ompi_mpi_float’
collect2: error: ld returned 1 exit status
gmake[5]: *** [bin/vtkLegacyColorMapXMLToJSON] Error 1
gmake[4]: *** [Utilities/LegacyColorMapXMLToJSON/CMakeFiles/vtkLegacyColorMapXMLToJSON.dir/all] Error 2
gmake[4]: *** Waiting for unfinished jobs…
[ 89%] Building CXX object Wrapping/ClientServer/CMakeFiles/vtkPVServerManagerRenderingCS.dir/vtkSMRepresentedArrayListDomainClientServer.cxx.o
[ 89%] Building CXX object Wrapping/ClientServer/CMakeFiles/vtkPVServerManagerRenderingCS.dir/vtkSMSaveScreenshotProxyClientServer.cxx.o
[ 89%] Building CXX object Wrapping/ClientServer/CMakeFiles/vtkPVServerManagerRenderingCS.dir/vtkSMScalarBarWidgetRepresentationProxyClientServer.cxx.o
[ 89%] Building CXX object Wrapping/ClientServer/CMakeFiles/vtkPVServerManagerRenderingCS.dir/vtkSMSelectionHelperClientServer.cxx.o
[ 89%] Building CXX object Wrapping/ClientServer/CMakeFiles/vtkPVServerManagerRenderingCS.dir/vtkSMSelectionLinkClientServer.cxx.o
[ 89%] Building CXX object Wrapping/ClientServer/CMakeFiles/vtkPVServerManagerRenderingCS.dir/vtkSMSpreadSheetRepresentationProxyClientServer.cxx.o
[ 89%] Building CXX object Wrapping/ClientServer/CMakeFiles/vtkPVServerManagerRenderingCS.dir/vtkSMSpreadSheetViewProxyClientServer.cxx.o
[ 89%] Building CXX object Wrapping/ClientServer/CMakeFiles/vtkPVServerManagerRenderingCS.dir/vtkSMTextWidgetRepresentationProxyClientServer.cxx.o
[ 89%] Building CXX object Wrapping/ClientServer/CMakeFiles/vtkPVServerManagerRenderingCS.dir/vtkSMTransferFunctionManagerClientServer.cxx.o
[ 89%] Building CXX object Wrapping/ClientServer/CMakeFiles/vtkPVServerManagerRenderingCS.dir/vtkSMTransferFunctionPresetsClientServer.cxx.o
[ 89%] Building CXX object Wrapping/ClientServer/CMakeFiles/vtkPVServerManagerRenderingCS.dir/vtkSMTransferFunctionProxyClientServer.cxx.o
[ 89%] Building CXX object Wrapping/ClientServer/CMakeFiles/vtkPVServerManagerRenderingCS.dir/vtkSMUtilitiesClientServer.cxx.o
[ 89%] Building CXX object Wrapping/ClientServer/CMakeFiles/vtkPVServerManagerRenderingCS.dir/vtkSMViewLayoutProxyClientServer.cxx.o
[ 89%] Building CXX object Wrapping/ClientServer/CMakeFiles/vtkPVServerManagerRenderingCS.dir/vtkSMViewProxyClientServer.cxx.o
[ 89%] Building CXX object Wrapping/ClientServer/CMakeFiles/vtkPVServerManagerRenderingCS.dir/vtkSMViewProxyInteractorHelperClientServer.cxx.o
[ 89%] Building CXX object Wrapping/ClientServer/CMakeFiles/vtkPVServerManagerRenderingCS.dir/vtkSMViewResolutionDomainClientServer.cxx.o
[ 89%] Building CXX object Wrapping/ClientServer/CMakeFiles/vtkPVServerManagerRenderingCS.dir/vtkSMWidgetRepresentationProxyClientServer.cxx.o
[ 89%] Building CXX object Wrapping/ClientServer/CMakeFiles/vtkPVServerManagerRenderingCS.dir/vtkSMPythonViewProxyClientServer.cxx.o
[ 89%] Building CXX object Wrapping/ClientServer/CMakeFiles/vtkPVServerManagerRenderingCS.dir/vtkSMParaViewPipelineControllerWithRenderingClientServer.cxx.o
[ 89%] Building CXX object Wrapping/ClientServer/CMakeFiles/vtkPVServerManagerRenderingCS.dir/vtkPVServerManagerRenderingObjectFactoryClientServer.cxx.o
[ 89%] Building CXX object Wrapping/ClientServer/CMakeFiles/vtkPVServerManagerRenderingCS.dir/vtkPVServerManagerRenderingCSInit.cxx.o
[ 89%] Linking CXX static library …/…/lib/libvtkPVServerManagerRenderingCS-pv5.6.a
[ 89%] Built target vtkPVServerManagerRenderingCS
gmake[3]: *** [all] Error 2
CMake Error at /usr/local/usrapps/iabolotn/june_superbuild_5.6.0_gcc/build/superbuild/sb-paraview-build.cmake:47 (message):
Failed with exit code 2

**make[2]: *** [superbuild/paraview/stamp/paraview-build] Error 1
make[1]: *** [superbuild/CMakeFiles/paraview.dir/all] Error 2
make: *** [all] Error 2

With super build 5.8.0 the build compiles and installs without any obvious error messages, but then when I try to run the paraview, I get this message, which also looks mpi-related.

[3:18] npillai2@vclhpc26:~ $ /usr/local/usrapps/iabolotn/install/superbuild_gnu_5.8.0/bin/paraview --mesa
/usr/local/usrapps/iabolotn/install/superbuild_gnu_5.8.0/bin/paraview: symbol lookup error: /usr/local/usrapps/iabolotn/install/superbuild_gnu_5.8.0/lib/libvtkParallelMPI-pv5.8.so.1: undefined symbol: ompi_mpi_op_sum

I don’t quite know what it means or whether it has to do with the build or the environment it was run in - @mwestphal do you see anything I may have done wrong/ a step I missed? (or do you need more information?)

I defer to @ben.boeckel

I get the same error … Server build for GPU usage produces `undefined symbol: ompi_mpi_comm_null`