# Compiling headless ParaView with OSMesa

**URL:** https://discourse.paraview.org/t/compiling-headless-paraview-with-osmesa/8377
**Category:** ParaView Support
**Created:** [November 11, 2021, 4:06pm UTC](https://discourse.paraview.org/t/compiling-headless-paraview-with-osmesa/8377 "2021-11-11T16:06:26Z")
**Posts on this page:** 14
**Page:** 1

<div class="post-metadata">

### Author: ![robertsawko](https://discourse.paraview.org/user_avatar/discourse.paraview.org/robertsawko/32/6283_2.png) [@robertsawko](https://discourse.paraview.org/u/robertsawko)
#### Post date: [November 11, 2021, 4:06pm UTC](https://discourse.paraview.org/t/compiling-headless-paraview-with-osmesa/8377/1 "2021-11-11T16:06:26Z")

</div>

I am playing with [in-situ visualisation](https://discourse.paraview.org/t/catalyst-v2-for-openfoam/6903) and I have made some progress with EGL on GPU nodes, but would now like to enable it on every CPU-only nodes as well. I believe this should be possible with the use of OSMesa, but I am getting a little confused about dependencies. My main question is whether it’s possible to build a fully headless ParaView with OSMesa i.e. without any reference to X11 or other display server?

I have looked at @mwestphal [tutorial](https://discourse.paraview.org/t/building-paraview-with-gpu-accelerated-mesa/1317), but I see that `meson` has an option `-Dplatforms='x11'`. I have also looked at [this](https://kitware.github.io/paraview-docs/v5.9.1/cxx/Offscreen.html) page in the documentation and I infer that I will need `VTK_USE_X=OFF` and `VTK_OPENGL_HAS_OSMESA=ON`. I am assuming this to be possible as there is a pre-compiled headless server with OSMesa on PV download page but please advise.

I’ve also got two questions or notes about that doc page:

> `VTK_OPENGL_HAS_OSMESA`: When `ON`, implies that ParaView can use OSMesa to support headless modes of operation.

Is that meant to be _headless_ or just _offscreen_? Or is “Off-screen rendering in Mesa” actually _headless_ according to the terminology?

> All combinations of above options can be turned on or off independently except that presently `VTK_OPENGL_HAS_EGL` and `VTK_OPENGL_HAS_OSMESA` are mutually exclusive i.e. only one of the two can be ON at the same time. This is because the current version of Mesa (17.1.5) doesn’t support EGL for OpenGL, it’s only supported for OpenGL-ES.

Is this still true? I was looking at Mesa 20+ and I think it now supports EGL.

---

<div class="post-metadata">

### Author: ![mwestphal](https://discourse.paraview.org/user_avatar/discourse.paraview.org/mwestphal/32/17_2.png) [@mwestphal](https://discourse.paraview.org/u/mwestphal)
#### Post date: [November 11, 2021, 4:40pm UTC](https://discourse.paraview.org/t/compiling-headless-paraview-with-osmesa/8377/2 "2021-11-11T16:40:18Z")

</div>

Offscreen mesa require a XOrg server.  
Headless osmesa does not require a XOrg server.

Mesa 20 now supports EGL but VTK/ParaView does not yet support Mesa + EGL.

> I am assuming this to be possible as there is a pre-compiled headless server with OSMesa on PV download page but please advise.

Of course this is possible, just build paraview and osmesa using the paraview superbuild, this is the easiest way forward: [https://gitlab.kitware.com/paraview/paraview-superbuild](https://gitlab.kitware.com/paraview/paraview-superbuild)

---

<div class="post-metadata">

### Author: ![robertsawko](https://discourse.paraview.org/user_avatar/discourse.paraview.org/robertsawko/32/6283_2.png) [@robertsawko](https://discourse.paraview.org/u/robertsawko)
#### Post date: [November 18, 2021, 3:15pm UTC](https://discourse.paraview.org/t/compiling-headless-paraview-with-osmesa/8377/3 "2021-11-18T15:15:08Z")

</div>

Thanks. I started working with superbuild, but I am already getting a little confused. I have three questions

1. Do I understand correctly that options `ENABLE_something` mean that `something` is not to be built by superbuild. However if `something` is already built is it possible to point to an existing installation? For instance Python3 and MPI are already on this cluster, but I couldn’t succeed with the configuration until I switched it ON.
2. I don’t understand differences between editions, specifically `CATALYST` and `CATALYST_RENDERING`. What does it mean for Catalyst or Paraview not to have rendering capabilities?
3. Can you confirm that I use download-all correctly. I’ve done it like this:

```bash
ccmake \
  ...
  -DSUPERBUILD_OFFLINE_BUILD=ON \
  ../paraview-superbuild
make download-all
# submit a compilation job to compute

```

My current cmake invocation for future reference:

```bash
cmake \
    -DPARAVIEW_BUILD_EDITION=CATALYST_RENDERING \
    -DSUPERBUILD_OFFLINE_BUILD=ON \
    -DENABLE_mpi=ON \
    -DENABLE_python3=ON \
    -DENABLE_osmesa=ON \
    -DCMAKE_INSTALL_PREFIX=$INSTALL_DIR \
    ../paraview-superbuild

```

---

<div class="post-metadata">

### Author: ![mwestphal](https://discourse.paraview.org/user_avatar/discourse.paraview.org/mwestphal/32/17_2.png) [@mwestphal](https://discourse.paraview.org/u/mwestphal)
#### Post date: [November 18, 2021, 3:17pm UTC](https://discourse.paraview.org/t/compiling-headless-paraview-with-osmesa/8377/4 "2021-11-18T15:17:15Z")

</div>

> [@robertsawko](#):
>
> Do I understand correctly that options `ENABLE_something` mean that `something` is not to be built by superbuild. However if `something` is already built is it possible to point to an existing installation? For instance Python3 and MPI are already on this cluster, but I couldn’t succeed with the configuration until I switched it ON.

No ENABLE\_something means that pvsb will try to build and use it. ENABLE\_something + USE\_SYSTEM\_something, means that pvsb will only try to use it from the system and not building it.

You 3. seems correct.

---

<div class="post-metadata">

### Author: ![robertsawko](https://discourse.paraview.org/user_avatar/discourse.paraview.org/robertsawko/32/6283_2.png) [@robertsawko](https://discourse.paraview.org/u/robertsawko)
#### Post date: [November 19, 2021, 2:57pm UTC](https://discourse.paraview.org/t/compiling-headless-paraview-with-osmesa/8377/5 "2021-11-19T14:57:41Z")

</div>

Thanks, again. Mathieu. I think I am very close to getting it work. My non-system Python and non-MPI config builds fine, but now I am trying with system Python and MPI and it gets past a configuration stage but breaks with

```bash
[20%] Performing build step for 'pythonsetuptools'
cd /lustre/scafellpike/local/HT04121/aas01/shared/software/paraview-superbuild/build-alt/superbuild/pythonsetuptools/src && /lustre/scafellpike/local/apps/hierarchy/core/cmake/3.21.1/bin/cmake -P /lustre/scafellpike/local/HT04121/aas01/shared/software/paraview-superbuild/build-alt/superbuild/sb-pythonsetuptools-build.cmake
Traceback (most recent call last):
  File "setup.py", line 9, in <module>
    import setuptools
  File "/netfs/smain01/scafellpike/local/HT04121/aas01/shared/software/paraview-superbuild/build-alt/superbuild/pythonsetuptools/src/setuptools/ __init__.py", line 19, in <module>
    from setuptools.dist import Distribution
  File "/netfs/smain01/scafellpike/local/HT04121/aas01/shared/software/paraview-superbuild/build-alt/superbuild/pythonsetuptools/src/setuptools/dist.py", line 34, in <module>
    from setuptools import windows_support
  File "/netfs/smain01/scafellpike/local/HT04121/aas01/shared/software/paraview-superbuild/build-alt/superbuild/pythonsetuptools/src/setuptools/windows_support.py", line 2, in <module>
    import ctypes
  File "/lustre/scafellpike/local/HT04121/aas01/shared/apps/compiler/gcc/6.5.0/python/3.8.12/lib/python3.8/ctypes/ __init__.py", line 7, in <module>
    from _ctypes import Union, Structure, Array
ModuleNotFoundError: No module named '_ctypes'
CMake Error at /lustre/scafellpike/local/HT04121/aas01/shared/software/paraview-superbuild/build-alt/superbuild/sb-pythonsetuptools-build.cmake:47 (message):
  Failed with exit code 1

```

I am tempted to just give up on system Python and have a separate Python in ParaView. We can always disable loading of other Pythons through modules, but EGL PV built against the same Python so I just wanted to be consistent.

My current build call for reference:

```bash
cmake \
    -DPARAVIEW_BUILD_EDITION=CATALYST_RENDERING \
    -DSUPERBUILD_OFFLINE_BUILD=ON \
    -DENABLE_python3=ON \
    -DUSE_SYSTEM_python3=ON \
    -DENABLE_mpi=ON \
    -DUSE_SYSTEM_mpi=ON \
    -DENABLE_osmesa=ON \
    -DENABLE_llvm=ON \
    -DCMAKE_INSTALL_PREFIX=$INSTALL_DIR \
    ../paraview-superbuild \
    && make download-all \
   # Job submission with parallel make

```

---

<div class="post-metadata">

### Author: ![robertsawko](https://discourse.paraview.org/user_avatar/discourse.paraview.org/robertsawko/32/6283_2.png) [@robertsawko](https://discourse.paraview.org/u/robertsawko)
#### Post date: [November 19, 2021, 3:04pm UTC](https://discourse.paraview.org/t/compiling-headless-paraview-with-osmesa/8377/6 "2021-11-19T15:04:59Z")

</div>

And sorry, one more slightly unrelated question. Is there a way to see what options for each software were used for a given superbuild configuration? If I want to inspect step-by-step how software stack was built because, can I do that?

---

<div class="post-metadata">

### Author: ![mwestphal](https://discourse.paraview.org/user_avatar/discourse.paraview.org/mwestphal/32/17_2.png) [@mwestphal](https://discourse.paraview.org/u/mwestphal)
#### Post date: [November 19, 2021, 3:06pm UTC](https://discourse.paraview.org/t/compiling-headless-paraview-with-osmesa/8377/7 "2021-11-19T15:06:35Z")

</div>

> [@robertsawko](#):
>
> `ModuleNotFoundError: No module named '_ctypes'`

No idea, maybe @ben.boeckel can help.

> Is there a way to see what options for each software were used for a given superbuild configuration? If I want to inspect step-by-step how software stack was built because, can I do that?

You can read the CMakeCache.txt but it is very verbose, nothing more then that I’m afraid.

---

<div class="post-metadata">

### Author: ![robertsawko](https://discourse.paraview.org/user_avatar/discourse.paraview.org/robertsawko/32/6283_2.png) [@robertsawko](https://discourse.paraview.org/u/robertsawko)
#### Post date: [November 19, 2021, 3:17pm UTC](https://discourse.paraview.org/t/compiling-headless-paraview-with-osmesa/8377/8 "2021-11-19T15:17:41Z")

</div>

> [@robertsawko](#):
>
> `ModuleNotFoundError: No module named '_ctypes'`

Actually a quick search suggests it’s `libffi` and `libffi-devel` that have something to do with interfacing between functions of different languages. It may take me a while to get this installed on this computer, but I will see what I can do.

---

<div class="post-metadata">

### Author: ![ben.boeckel](https://discourse.paraview.org/letter_avatar_proxy/v4/letter/b/ea5d25/32.png) [@ben.boeckel](https://discourse.paraview.org/u/ben.boeckel)
#### Post date: [November 19, 2021, 11:42pm UTC](https://discourse.paraview.org/t/compiling-headless-paraview-with-osmesa/8377/9 "2021-11-19T23:42:42Z")

</div>

> [@robertsawko](#):
>
> If I want to inspect step-by-step how software stack was built because, can I do that?

You can look at the `superbuild/sb-*-configure.cmake` (there are also `-build.cmake` and `-install.cmake` files, but these are fairly boring usually) files in the build directory.

As for Python, yes, it appears that the Python you have is not enough to build some things (such as `setuptools` apparently?).

---

<div class="post-metadata">

### Author: ![robertsawko](https://discourse.paraview.org/user_avatar/discourse.paraview.org/robertsawko/32/6283_2.png) [@robertsawko](https://discourse.paraview.org/u/robertsawko)
#### Post date: [November 24, 2021, 10:34pm UTC](https://discourse.paraview.org/t/compiling-headless-paraview-with-osmesa/8377/10 "2021-11-24T22:34:33Z")

</div>

Thanks, a lot.

I have one final, I think, set of questions about superbuild. I noticed that I didn’t get the `include/`, `share/` and `*.cmake` file in my installation. Unfortunately, I need some of them as I am building my Catalyst adaptors so will need to compile whilst pointing to existing ParaView.

For comparison the EGL version that I installed manually has the follow structure

```auto
egl:
bin include lib64 share

osmesa:
bin lib

```

I am now rerunning the build with

```auto
DCMAKE_BUILD_TYPE_paraview=RelWithDebInfo

```

Is there anything else I need to be aware of?

---

<div class="post-metadata">

### Author: ![ben.boeckel](https://discourse.paraview.org/letter_avatar_proxy/v4/letter/b/ea5d25/32.png) [@ben.boeckel](https://discourse.paraview.org/u/ben.boeckel)
#### Post date: [November 25, 2021, 3:14pm UTC](https://discourse.paraview.org/t/compiling-headless-paraview-with-osmesa/8377/11 "2021-11-25T15:14:34Z")

</div>

> [@robertsawko](#):
>
> I noticed that I didn’t get the `include/`, `share/` and `*.cmake` file in my installation.

What do you mean by “installation”? The results of `make install` at the superbuild level? If you want that to generate headers and such, you need the `paraviewsdk` project enabled.

---

<div class="post-metadata">

### Author: ![robertsawko](https://discourse.paraview.org/user_avatar/discourse.paraview.org/robertsawko/32/6283_2.png) [@robertsawko](https://discourse.paraview.org/u/robertsawko)
#### Post date: [November 25, 2021, 3:50pm UTC](https://discourse.paraview.org/t/compiling-headless-paraview-with-osmesa/8377/12 "2021-11-25T15:50:40Z")

</div>

Thanks, Ben and Mathieu. That is exactly what I have meant.

I am recompiling and if I get it right this time, I will post my final cmake/make invocations.

If I ever get my project closer to where the proposal says it will be and if you happen to be at the same conference I should at the very minimum take you out for a drink!

---

<div class="post-metadata">

### Author: ![robertsawko](https://discourse.paraview.org/user_avatar/discourse.paraview.org/robertsawko/32/6283_2.png) [@robertsawko](https://discourse.paraview.org/u/robertsawko)
#### Post date: [November 26, 2021, 3:54pm UTC](https://discourse.paraview.org/t/compiling-headless-paraview-with-osmesa/8377/13 "2021-11-26T15:54:19Z")

</div>

Setting `ENABLE_paraviewsdk` worked for PV and VTK, but I am sorry, I fear that only generated more questions…

1. Is it possible get headers of all the packages to be also installed under current configuration? Currently a CMake of my Catalyst adaptopr is failing because it cannot find OSMesa headers which are not installed on this machine.
2. Do I need to set CMAKE\_PREFIX\_PATH or CMAKE\_MODULE\_PATH and I do need to cover each subdirectory?
3. What does `DSUPERBUILD_DEBUG_CONFIGURE_STEPS=ON` do?

Here’s my current invocation.

```bash
cmake \
    -DCMAKE_BUILD_TYPE_paraview=RelWithDebInfo \
    -DPARAVIEW_BUILD_EDITION=CANONICAL \
    -DSUPERBUILD_OFFLINE_BUILD=ON \
    -DENABLE_python3=ON \
    -DENABLE_mpi=ON \
    -DUSE_SYSTEM_mpi=ON \
    -DENABLE_osmesa=ON \
    -DENABLE_llvm=ON \
    -DENABLE_paraviewsdk=ON \
    -DSUPERBUILD_DEBUG_CONFIGURE_STEPS=ON \
    -DCMAKE_INSTALL_PREFIX=$INSTALL_DIR \
    ../paraview-superbuild \
    && make download-all \
    && make install

```

---

<div class="post-metadata">

### Author: ![ben.boeckel](https://discourse.paraview.org/letter_avatar_proxy/v4/letter/b/ea5d25/32.png) [@ben.boeckel](https://discourse.paraview.org/u/ben.boeckel)
#### Post date: [November 28, 2021, 1:12am UTC](https://discourse.paraview.org/t/compiling-headless-paraview-with-osmesa/8377/14 "2021-11-28T01:12:26Z")

</div>

If you’re building on a single machine, I would recommend building against the `$superbuild_build_dir/install` directly rather than the artifacts of any of the superbuild’s install mechanisms.