I got a response from IT saying that just having the port won’t work, so I guess I need to build Paraview with the GUI.
Paraview with GUI will only work on compute nodes if it is not windowless. You can try to compile Paraview with OsMesa and that will work on any CPU compute node windowless or not regardless. Meanwhile if you want to connect to the one you built on your login node you can try ssh npillai2@login.hpc.ncsu.edu -L 11111:login04:11111
Note: Sorry I initially have put -L argument before. I think this may just work. Can you please try it if possible?
Sorry I’m a little confused. Would you mind outlining the steps in order?
Normally, to connect to a pvserver I
- Run pvserver on the HPC
- Connect to the address via my paraview client
- Operate Paraview remotely.
At which point in this process do you want me to ssh npillai2@login.hpc.ncsu.edu -L 11111:login04:11111? [also maybe it should be ssh npillai2@login04.hpc.ncsu.edu -L 11111:login04:11111] as other wise I’m not guaranteed to land on the same login node.
You need to port forward to the same login node. So you maybe right about the address (or the login node is 04 from the latter part of the command). Please try both and see what works out.
On Paraview client side after you have done the port forwarding from the client side. You need to select Add Server from Edit Server Configuration window. Make sure host is set to localhost as you have already made the port forward. For Name you can choose anything you wish and now you should be good to go.
This is necessary as login nodes can only access your compute nodes (on which pvserver runs) in many cases and if you would like to use a client outside your login node, you can by issuing the command I mentioned port forward your 11111 port from login node to your client localhost (client ip) and then you can connect to the compute node as if it is directly accessible on a port on localhost. This is what you need to do in between steps 1. & 2. you mentioned in your previous post.
Thanks for the clarification. I was atleast able to connect to the server side Paraview (which I was able to verify via the existence of the plugin I built). However, I got the following errors, presumably revealing some of the reasons IT is so hesitant for me to use pvserver
ERROR: In /home/npillai2/custom_software/try_10_paraview/paraview/VTK/Rendering/OpenGL2/vtkOpenGLRenderWindow.cxx, line 749
vtkXOpenGLRenderWindow (0x2c821d0): Unable to find a valid OpenGL 3.2 or later implementation. Please update your video card driver to the latest version. If you are using Mesa please make sure you have version 11.2 or later and make sure your driver in Mesa supports OpenGL 3.2 such as llvmpipe or openswr. If you are on windows and using Microsoft remote desktop note that it only supports OpenGL 3.2 with nvidia quadro cards. You can use other remoting software such as nomachine to avoid this issue.
ERROR: In /home/npillai2/custom_software/try_10_paraview/paraview/VTK/Rendering/OpenGL2/vtkXOpenGLRenderWindow.cxx, line 796
vtkXOpenGLRenderWindow (0x2c821d0): failed to create offscreen window
ERROR: In /home/npillai2/custom_software/try_10_paraview/paraview/VTK/Rendering/OpenGL2/vtkOpenGLRenderWindow.cxx, line 736
vtkXOpenGLRenderWindow (0x2c821d0): GLEW could not be initialized.
I plan to look into this more later this week after I get concrete answers from the HPC IT (especially about why the normal pvserver method does not work).
This error message is quite common. This most probably means your login or compute nodes don’t support windowed rendering. So it cannot render with OpenGL. You have to use a windowless offscreen renderer like OSMesa (for CPU rendering) or EGL (for GPU rendering) to build Paraview on the server side. You can’t build a working version on server side with Qt or OpenGL rendering.
There are nodes that support windowed rendering - that you have to reserve and connect to via RDP. I’ve tested the base v5.6.0 from the main paraview website via those nodes and works fine and handles example files etc. I don’t know anything about which mesa version those nodes use, but I need to type paraview --mesa for it to work so it definitely uses mesa.
Yes @npillai2 you need to have windowed rendering on nodes. However a much robust solution would be to use OsMesa (and not Mesa) as the renderer which will work on any node windowed or windowless.
It seems like OSmesa does not already exist on the machine. I’m hoping that I don’t have to jump through a bunch of hoops to also install OSMesa.
I tried an install with GNU complilers (4.8.5 I believe), and the install completed without any errors. However when I try running the executable (aliased to “runparaview”), I get this error despite trying to point to various places in the Qt installation
[22:18] npillai2@vclhpc2:~ $ setenv LD_LIBRARY_PATH $LD_LIBRARY_PATH\:/usr/local/usrapps/iabolotn/Qt5.9.0/5.9/gcc_64/lib
[22:21] npillai2@vclhpc2:~ $ runparaview --mesa This application failed to start because it could not find or load the Qt platform plugin "xcb"
in "".
Available platform plugins are: eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, xcb.
Reinstalling the application may fix this problem.
Abort (core dumped)
[22:21] npillai2@vclhpc2:~ $ runparaview
This application failed to start because it could not find or load the Qt platform plugin "xcb"
in "".
Available platform plugins are: eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, xcb.
Reinstalling the application may fix this problem.
Abort (core dumped)
[22:21] npillai2@vclhpc2:~ $ setenv LD_LIBRARY_PATH $LD_LIBRARY_PATH\:/usr/local/usrapps/iabolotn/Qt5.9.0/5.9/gcc_64/plugins/platforms
[22:24] npillai2@vclhpc2:~ $ runparaview This application failed to start because it could not find or load the Qt platform plugin "xcb"
in "".
Available platform plugins are: eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, xcb.
Reinstalling the application may fix this problem.
Abort (core dumped)
[22:24] npillai2@vclhpc2:~ $ setenv PATH $PATH\:/usr/local/usrapps/iabolotn/Qt5.9.0/5.9/gcc_64/plugins/platforms
[22:25] npillai2@vclhpc2:~ $ runparaview
This application failed to start because it could not find or load the Qt platform plugin "xcb"
in "".
Available platform plugins are: eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, xcb.
Reinstalling the application may fix this problem.
Abort (core dumped)
[22:25] npillai2@vclhpc2:~ $ setenv LD_LIBRARY_PATH $LD_LIBRARY_PATH\:/usr/local/usrapps/iabolotn/Qt5.9.0/5.9/gcc_64/plugins/xcbglintegrations/
[22:26] npillai2@vclhpc2:~ $ runparaview
This application failed to start because it could not find or load the Qt platform plugin "xcb"
in "".
Available platform plugins are: eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, xcb.
Reinstalling the application may fix this problem.
Abort (core dumped)
[22:26] npillai2@vclhpc2:~ $ runparaview --mesa
This application failed to start because it could not find or load the Qt platform plugin "xcb"
in "".
Available platform plugins are: eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, xcb.
Reinstalling the application may fix this problem.
Abort (core dumped)
If this is a random error, I don’t mind going through a reinstall, but I was wondering if you’d seen this before and if there was a simple solution to it.
I just tried using superbuild to get osmesa, and I can’t find my way around this error
[ 92%] Performing configure step for 'osmesa'
autoreconf: Entering directory `.'
autoreconf: configure.ac: not using Gettext
autoreconf: running: aclocal --force -I m4
autoreconf: configure.ac: tracing
autoreconf: running: libtoolize --copy --force
/usr/bin/libtoolize: line 1284: /usr/bin/touch: Permission denied
libtoolize: putting auxiliary files in AC_CONFIG_AUX_DIR, `bin'.
libtoolize: can not copy `/usr/share/libtool/config/ltmain.sh' to `bin/'
/usr/bin/libtoolize: line 1284: /usr/bin/touch: Permission denied
libtoolize: putting macros in AC_CONFIG_MACRO_DIR, `m4'.
libtoolize: can not copy `/usr/share/aclocal/libtool.m4' to `m4/'
/usr/bin/libtoolize: line 1284: /usr/bin/touch: Permission denied
libtoolize: can not copy `/usr/share/aclocal/ltoptions.m4' to `m4/'
/usr/bin/libtoolize: line 1284: /usr/bin/touch: Permission denied
libtoolize: can not copy `/usr/share/aclocal/ltsugar.m4' to `m4/'
/usr/bin/libtoolize: line 1284: /usr/bin/touch: Permission denied
libtoolize: can not copy `/usr/share/aclocal/ltversion.m4' to `m4/'
/usr/bin/libtoolize: line 1284: /usr/bin/touch: Permission denied
libtoolize: can not copy `/usr/share/aclocal/lt~obsolete.m4' to `m4/'
autoreconf: libtoolize failed with exit status: 1
CMake Error at /home/npillai2/custom_software/superbuild_paraview_5.8.0_gcc/build/superbuild/sb-osmesa-configure.cmake:47 (message):
Failed with exit code 1
If I do
cmake -make \
-DAC_CONFIG_AUX_DIR=/usr/local/usrapps/iabolotn/superbuild_paraview_5.8.0_gcc/build/local_bin \
-DAC_CONFIG_MACRO_DIR=/usr/local/usrapps/iabolotn/superbuild_paraview_5.8.0_gcc/build/m4 \
../paraview-superbuild
The variables don’t get accepted.
CMake Warning:
Manually-specified variables were not used by the project:
AC_CONFIG_AUX_DIR
AC_CONFIG_MACRO_DIR
Plus I think the issue is at
/usr/bin/libtoolize: line 1284: /usr/bin/touch: Permission denied
which is before the other variables come into play. Has anyone seen these errors before?
@npillai2
I remember getting an error similar to this back in January while trying to compile Paraview. But I can’t recall what exactly solved the issue.
Have you tried these?
set(ENABLE_osmesa OFF CACHE BOOL “”)
set(mesa_SWR_ARCH “avx,avx2” CACHE STRING “”)
I’m not sure why but I remember for OsMesa I was advised to try this.
You can see the thread of this post for reference
I had the same setting for mesa_SWR_ARCH, but I cannot imagine why setting osmesa to OFF would help me configure a build with osmesa (my dry build with all defaults worked - which had osmesa off), but the issue is I want to make a build with the GUI client, and this error also occurs when I try to compile with regular mesa.
Hi all,
I am the IT person in question.
If I can get help with what should work, maybe I can figure out why it doesn’t work.
Here is what I understand:
- (We’re not adding firewall exceptions to allow pvserver.)
- Naveen wants to interact with the GUI, not just submit batch visualization, so OSMesa is not sufficient.
- On our system, Paraview will be able to open a window, but we don’t have GPUs.
I was trying 5.6.0 because that is the version we have installed as binaries and it works well. That is the only reason. If 5.8.0 has the most testing, I am happy to use that.
I thought I need ‘mesa’ option, but when I build with mesa, it doesn’t make a paraview executable. I can build with ‘qt5’, but then I get the OpenGL not supported error at runtime. When I said ‘qt not compatible with mesa’ in what Naveen is quoting, I just mean in the options in ccmake for superbuild are incompatible.
So if someone can just let me know what should theoretically work (am I supposed to get executable? am I supposed to use mesa? Will QT5 work if I link to some CPU GL libraries…???) If I know what should work, then I can sort out better if there are issues on our system, or whether I am just making incompetent cmake choices… I am forever grateful for the help.
I am using gcc compiler 4.8.5.
(We’re not adding firewall exceptions to allow pvserver.)
Which port can application use to communicate ? Is ssh communication ok ?
Naveen wants to interact with the GUI, not just submit batch visualization, so OSMesa is not sufficient.
pvserver with osmesa on a server and ParaView on a client would work.
On our system, Paraview will be able to open a window, but we don’t have GPUs.
pvserver + osmesa then
@npillai2 would then connect to the pvserver using any ParaView client.
Hi Mathieu, thanks for responding.
Our setup allows ssh from the campus network to a login node, then you can only access a compute node via the scheduler (LSF). ssh to login, then LSF to compute node. Applications (ParaView) may not be run on a login node. A user may reserve an interactive session on a compute node through LSF. (But then ParaView would need to be build with the GUI).
I was staring at your past threads and the GitHub issues where you are discussing documentation. I thought we need QT GUI with mesa library, but when I was reading superbuild README, I misunderstood and took ‘qt5, mesa, osmesa, egl are mutually exclusive’ to mean ‘you can’t specify more than one of those cmake options when configuring’.
So just now, I finished a successful install by choosing enable_qt5 and enable_mesa. I was just stuck on thinking it wasn’t possible to pick both. It worked fine.
Thanks again!
Lisa
Our setup allows ssh from the campus network to a login node, then you can only access a compute node via the scheduler (LSF). ssh to login, then LSF to compute node. Applications (ParaView) may not be run on a login node.
Then ssh tunneling is the solution for accessing pvserver from outside.
A user may reserve an interactive session on a compute node through LSF. (But then ParaView would need to be build with the GUI).
GUI + Mesa should work though.
I was staring at your past threads and the GitHub issues where you are discussing documentation. I thought we need QT GUI with mesa library, but when I was reading superbuild README, I misunderstood and took ‘qt5, mesa, osmesa, egl are mutually exclusive’ to mean ‘you can’t specify more than one of those cmake options when configuring’.
This documentation is confusing but we have not been able to improve it yet.
See here for reference : https://gitlab.kitware.com/paraview/paraview-superbuild/-/merge_requests/690
I finished a successful install by choosing enable_qt5 and enable_mesa. I was just stuck on thinking it wasn’t possible to pick both. It worked fine.
ok.
Hi Mathieu - does this mean that there’s a way to access pvserver without changes to the firewall? This is definitely the ideal scenario for both me and my group.
You can only access a compute node via LSF from a login node.
ssh tunnelling allows you to open a window on the local machine while you are using ssh. You cannot ssh into a compute node. You cannot run anything on a login node. I really don’t understand why anyone is saying that pvserver will work within those constraints.
I got ParaView to compile on our system, so I suspect that is the way to move forward on this.
Could also be worthwhile taking a look at what CINECA is doing
http://www.hpc.cineca.it/services/remote-visualisation
You mean LSF, once connected, do not provide an ip that can be connected to using either ssh or any other port ?