Complete compilation of Paraview (Readers+OSmesa+Python)

Dear Mathieu,

Thank you so much for your message. I dowloaded the super-build and I am trying to compile it with OSMESA (llvm and mesa are already compiled in my computer). I am using the following flags:

cmake \
-GNinja \
-DPARAVIEW_USE_PYTHON=ON \
-DPython3_EXECUTABLE:FILEPATH=/usr/bin/python3 \
-DPARAVIEW_USE_MPI=ON \
-DVTK_SMP_IMPLEMENTATION_TYPE=TBB \
-DPARAVIEW_ENABLE_VISITBRIDGE=ON \
-DOSMESA_INCLUDE_DIR=${InstallPath}/mesa-21.3.1/include \
-DOSMESA_LIBRARY=${InstallPath}/mesa/lib/x86_64-linux-gnu/libOSMesa.so \
-DCMAKE_BUILD_TYPE=Release ./paraview-superbuild

However it crashes because python 2 or 3 have not been enabled:

-- Building projects: cxx11, nlohmannjson, python, pybind11, paraview
CMake Error at superbuild/projects/python.cmake:2 (message):
  Python is enabled, but neither Python2 nor Python3 has been enabled.
Call Stack (most recent call first):
  superbuild/cmake/SuperbuildMacros.cmake:983 (include)
  superbuild/CMakeLists.txt:159 (superbuild_process_dependencies)

I also read the post below because some people were experiencing the same issue and you recommended using the flag " -D ENABLE_python3":

However for the current super-build version, that cmake flag is not recognized, I also tried:

-D ENABLE_python3 \
-DENABLE_python3 \
-DPARAVIEW_ENABLE_python3 \

But I cannot start the compilation.

Hi @Captain_Convergence

There is no reason to specify such cmake options, just enable what you need and the superbuild will download and build it for you. This is literally the point of the superbuild, you do not need to build the dependencies yourself.

So in your case, this should be enough:

cmake \
-GNinja \
-DENABLE_osmesa=ON
-DENABLE_python3=ON
-DENABLE_mpi=ON
-DENABLE_paraview=ON
-DCMAKE_BUILD_TYPE=Release
../paraview-superbuild

Also it looks like you are building an old version of the superbuild, is that intended ?

Best,

Dear Mathieu,

I tried with the flags that you mentioned and it seems to have worked. However I don’t see the “paraview” executable in the “install/bin” folder:

I can see that the rest of utilities have been succesfully compiled. How can I call the Paraview interface to directly work with the GUI?

You need to add ENABLE_qt5

I’m afraid ENABLE_osmesa and ENABLE_qt5 are incompatible. You can’t build ParaView Qt interface with osmesa at the same time.

@Captain_Convergence , why do you need osmesa rendering if you want to run ParaView also ?

The reason for this is that I need to use ParaView in parallel to compute some field in a reasonable time but I noticed that I have problems when using multiple CPUs, see my previous post here to now more about the issue:

However, I ran by chance in our HPC the ParaView version with osmesa using multiple-CPUs and I didn’t have the problem so now I believe that the solution to all my issues is to use a compiled version of ParaView using osmesa since I normally do CPU-based rendering.

I tried to compile the GUI using:

cmake \
-GNinja \
-DENABLE_qt5=ON \
-DENABLE_python3=ON \
-DENABLE_mpi=ON \
-DENABLE_paraview=ON \
-DCMAKE_BUILD_TYPE=Release ./paraview-superbuild

But I have the following compilation error linked with the LLVM installation:

It works for the -DENABLE_osmesa=ON compilation but it fails when I set -DENABLE_qt5=ON. Do I need to set any additional flag to correctly compile Paraview with the GUI?

I set -DENABLE_qt5=ON. Do I need to set any additional flag to correctly compile Paraview with the GUI?

Yes, we do not build Qt anymore, use the one from your system, but

If for some reason the binary release pvserver do not work for you, then use your osmesa build for the server and the binary release for the client.

So if I understand correctly, I can source the binary files from the latest Paraview Version (nightly/Master Version from https://www.paraview.org/download/) to use the GUI and then use pvserver to connect to the built version with OSMESA right? Something like this:

export PATH=/homeDir/ParaView-master-5.11.0/bin:$PATH
export PATH=/homeDir/ParaviewSuperbuild_OSMESA/install/bin:$PATH
alias svr='mpirun -np 6 pvserver --force-offscreen-rendering --server-port=11111'

I can connect to the SuperBuild server from the master version but then I have the problem regarding the readers since I haven’t compiled the version 5.11. Also I didn’t add -DPARAVIEW_ENABLE_VISITBRIDGE=ON \ during the superbuild compilation (since I thought that it would be included by default) so maybe that’s why the readers are not available in the pvserver. However adding the aforementioned flag doesn’t seem to work for the superbuild:

image

Why do you want to use master ? Juste use the last release, on both superbuild and binary.

I manage to build the 5.10.1 tag to completion on Ubuntu 20.04 (no discrete GPU)

However, when I try to do a ninja install I get the following errors

Copying /home/nyue/projects/ParaView/superbuild/qt-build/install/lib/catalyst/libcatalyst-paraview.so ==> lib/catalyst
CMake Error at cmake_install.cmake:1478 (message):
  Failed to install
  /home/nyue/projects/ParaView/superbuild/qt-build/install/lib/catalyst/libcatalyst-paraview.so:


  objdump:
  '/home/nyue/projects/ParaView/superbuild/qt-build/install/lib/catalyst/libcatalyst-paraview.so':
  No such file

  Traceback (most recent call last):

    File "/home/nyue/projects/ParaView/superbuild/paraview-superbuild_git/superbuild/cmake/scripts/fixup_bundle.unix.py", line 750, in <module>
      main(sys.argv[1:])
    File "/home/nyue/projects/ParaView/superbuild/paraview-superbuild_git/superbuild/cmake/scripts/fixup_bundle.unix.py", line 742, in main
      _install_binary(main_exe, is_excluded, bundle_dest, opts.libdir, installed, manifest, opts.source, dry_run=opts.dry_run, look_for_symlinks=opts.has_symlinks)
    File "/home/nyue/projects/ParaView/superbuild/paraview-superbuild_git/superbuild/cmake/scripts/fixup_bundle.unix.py", line 637, in _install_binary
      shutil.copy(binary.path, app_dest)
    File "/home/nyue/projects/ParaView/superbuild/qt-build/install/lib/python3.9/shutil.py", line 418, in copy
      copyfile(src, dst, follow_symlinks=follow_symlinks)
    File "/home/nyue/projects/ParaView/superbuild/qt-build/install/lib/python3.9/shutil.py", line 264, in copyfile
      with open(src, 'rb') as fsrc, open(dst, 'wb') as fdst:

  FileNotFoundError: [Errno 2] No such file or directory:
  '/home/nyue/projects/ParaView/superbuild/qt-build/install/lib/catalyst/libcatalyst-paraview.so'




make: *** [Makefile:120: install] Error 1
CMake Error at superbuild/cmake_install.cmake:66 (message):
  Failed to install the paraview/TGZ package.
Call Stack (most recent call first):
  cmake_install.cmake:47 (include)


FAILED: CMakeFiles/install.util 
cd /home/nyue/projects/ParaView/superbuild/qt-build && /home/nyue/systems/cmake/cmake-3.24.3-linux-x86_64/bin/cmake -P cmake_install.cmake
ninja: build stopped: subcommand failed.

Is catalyst mandatory ?

cmake \
    -D ENABLE_qt5=ON \
    -D ENABLE_python3=ON \
    -D ENABLE_mpi=ON \
    -D ENABLE_paraview=ON \
    -D CMAKE_INSTALL_PREFIX=$HOME/systems/ParaView/5.10.1 \
    -D CMAKE_BUILD_TYPE=Release \
    -G Ninja \
    ../paraview-superbuild_git

Hi

the superbuild do not use not support the install target.

Best,

The superbuild documentation about installing should be updated to reflect that install is not use nor supported.

What is the recommend step post-superbuild to deploy and package if install target is not the way to do it ?

See the packaging part: https://gitlab.kitware.com/paraview/paraview-superbuild#packaging

This installing parts looks like old doc that did not get removed, wdyt @ben.boeckel ?

False. It does. The documentation is up-to-date.

Well, this path is suspicious. Why is it looking for ParaView’s Catalyst implementation under Qt’s build tree?

I must remember it wrong then.

Hi @ben.boeckel

I just so happen to name my out of source build directory to be called qt-build, I am also planning to have an osmesa-build directory so that I don’t mix them up (qt and osmesa options cannot be mixed).

Or are you saying that libcatalyst should be a totally different piece of software and not something that should be built via superbuild ?

I am not familiar with catalyst, I don’t use with Paraview and hence have no experience with it.

Cheers

It’s an HPC thing normally, but it should be fine (it just runs the packaging script with the install destination as the place to put things instead of where CPack expects them.

Ah, ok, I see now. It just happened to look like qt/build.

If you don’t need catalyst, you can either disable it in the superbuild (ENABLE_catalyst=OFF) and possibly see where in projects/unix/paraview.bundle.unix.cmake it comes from (near line 260 or so).

2 Likes

@ben.boeckel

FYI, that option generated a warning

-- Generating done
CMake Warning:
  Manually-specified variables were not used by the project:

    ENABLE_catalyst

Oh, I see the ParaView 5.10.1 tag above now. Are you trying to build ParaView master with the 5.10.1 tag of the superbuild?