Issue detecting build configuration with legacy Catalyst with CMake versions 3.23 and above.

YvanFournier · December 22, 2022, 1:07am

Hello,

For legacy Catalyst support in EDF’s code_saturne tool, we generate a test project, using:
cmake -DCMAKE_MODULE_PATH=<paraview_prefix_path> <mini_project_dir>

Where the mini_project_dir contains a CMakeLists.txt and CoProcessingTest.cxx files, as per https://github.com/code-saturne/code_saturne/tree/master/build-aux/catalyst.

With CMake 3.22 and below, I get some warnings, but mostly working code, with the following output

/home/D43345/usr/bin/cmake -DCMAKE_PREFIX_PATH=/home/D43345/opt/paraview-5.11/scibian10_egl -DCMAKE_MODULE_PATH=/home/D43345/opt/paraview-5.11/scibian10_egl ~/test/cmake/catalyst
– The CXX compiler identification is GNU 8.3.0
– The C compiler identification is GNU 8.3.0
– Detecting CXX compiler ABI info
– Detecting CXX compiler ABI info - done
– Check for working CXX compiler: /usr/bin/c++ - skipped
– Detecting CXX compile features
– Detecting CXX compile features - done
– Detecting C compiler ABI info
– Detecting C compiler ABI info - done
– Check for working C compiler: /usr/bin/cc - skipped
– Detecting C compile features
– Detecting C compile features - done
– Found MPI_C: /usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi.so (found version “3.1”)
– Found MPI_CXX: /usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi_cxx.so (found version “3.1”)
– Found MPI: TRUE (found version “3.1”) found components: C CXX
– Looking for pthread.h
– Looking for pthread.h - found
– Performing Test CMAKE_HAVE_LIBC_PTHREAD
– Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
– Looking for pthread_create in pthreads
– Looking for pthread_create in pthreads - not found
– Looking for pthread_create in pthread
– Looking for pthread_create in pthread - found
– Found Threads: TRUE
– Could NOT find TBB (missing: TBB_DIR)
CMake Warning at /home/D43345/opt/paraview-5.11/scibian10_egl/lib/cmake/paraview-5.11/ParFlow-vtk-module-find-packages.cmake:115 (find_package):
By not providing “Findnlohmann_json.cmake” in CMAKE_MODULE_PATH this
project has asked CMake to find a package configuration file provided by
“nlohmann_json”, but CMake did not find one.

Could not find a package configuration file provided by “nlohmann_json”
with any of the following names:
nlohmann_jsonConfig.cmake
nlohmann_json-config.cmake
Add the installation prefix of “nlohmann_json” to CMAKE_PREFIX_PATH or set
“nlohmann_json_DIR” to a directory containing one of the above files. If
“nlohmann_json” provides a separate development package or SDK, be sure it
has been installed.
Call Stack (most recent call first):
/home/D43345/opt/paraview-5.11/scibian10_egl/lib/cmake/paraview-5.11/ParaViewPlugins-paraview_plugins-targets-depends.cmake:246 (include)
/home/D43345/opt/paraview-5.11/scibian10_egl/lib/cmake/paraview-5.11/paraview-config.cmake:92 (include)
CMakeLists.txt:6 (find_package)

– Could not find the ParFlow package due to a missing dependency: nlohmann_json
– Found MPI: TRUE (found version “3.1”) found components: C
– Found Python3: /usr/bin/python3.7 (found suitable version “3.7.3”, minimum required is “3.7”) found components: Interpreter Development.Module Development.Embed
– Found ZLIB: /usr/lib/x86_64-linux-gnu/libz.so (found version “1.2.11”)
– Could NOT find TBB (missing: TBB_DIR)
– Configuring done
– Generating done
– Build files have been written to: /home/D43345/test/cmake/build

I can then use the generated project to extract compile and link flags.

With CMake 3.23 and above, I get:

/home/D43345/opt/cmake-3.25/bin/cmake -DCMAKE_PREFIX_PATH=/home/D43345/opt/paraview-5.11/scibian10_egl -DCMAKE_MODULE_PATH=/home/D43345/opt/paraview-5.11/scibian10_egl ~/test/cmake/mini
– The CXX compiler identification is GNU 8.3.0
– The C compiler identification is GNU 8.3.0
– Detecting CXX compiler ABI info
– Detecting CXX compiler ABI info - done
– Check for working CXX compiler: /usr/bin/c++ - skipped
– Detecting CXX compile features
– Detecting CXX compile features - done
– Detecting C compiler ABI info
– Detecting C compiler ABI info - done
– Check for working C compiler: /usr/bin/cc - skipped
– Detecting C compile features
– Detecting C compile features - done
– Found MPI_C: /usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi.so (found version “3.1”)
– Found MPI_CXX: /usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi_cxx.so (found version “3.1”)
– Found MPI: TRUE (found version “3.1”) found components: C CXX
– Performing Test CMAKE_HAVE_LIBC_PTHREAD
– Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
– Looking for pthread_create in pthreads
– Looking for pthread_create in pthreads - not found
– Looking for pthread_create in pthread
– Looking for pthread_create in pthread - found
– Found Threads: TRUE
– Could NOT find TBB (missing: TBB_DIR)
CMake Warning at /home/D43345/opt/paraview-5.11/scibian10_egl/lib/cmake/paraview-5.11/ParFlow-vtk-module-find-packages.cmake:115 (find_package):
By not providing “Findnlohmann_json.cmake” in CMAKE_MODULE_PATH this
project has asked CMake to find a package configuration file provided by
“nlohmann_json”, but CMake did not find one.

Could not find a package configuration file provided by “nlohmann_json”
with any of the following names:
nlohmann_jsonConfig.cmake
nlohmann_json-config.cmake
Add the installation prefix of “nlohmann_json” to CMAKE_PREFIX_PATH or set
“nlohmann_json_DIR” to a directory containing one of the above files. If
“nlohmann_json” provides a separate development package or SDK, be sure it
has been installed.
Call Stack (most recent call first):
/home/D43345/opt/paraview-5.11/scibian10_egl/lib/cmake/paraview-5.11/ParaViewPlugins-paraview_plugins-targets-depends.cmake:246 (include)
/home/D43345/opt/paraview-5.11/scibian10_egl/lib/cmake/paraview-5.11/paraview-config.cmake:92 (include)
CMakeLists.txt:6 (find_package)

– Could not find the ParFlow package due to a missing dependency: nlohmann_json
CMake Error at /home/D43345/opt/paraview-5.11/scibian10_egl/lib/cmake/paraview-5.11/vtk/VTK-vtk-module-find-packages.cmake:162 (find_package):
By not providing “Findospray.cmake” in CMAKE_MODULE_PATH this project has
asked CMake to find a package configuration file provided by “ospray”, but
CMake did not find one.

Could not find a package configuration file provided by “ospray” (requested
version 2.1) with any of the following names:
osprayConfig.cmake
ospray-config.cmake
Add the installation prefix of “ospray” to CMAKE_PREFIX_PATH or set
“ospray_DIR” to a directory containing one of the above files. If “ospray”
provides a separate development package or SDK, be sure it has been
installed.
Call Stack (most recent call first):
/home/D43345/opt/paraview-5.11/scibian10_egl/lib/cmake/paraview-5.11/vtk/vtk-config.cmake:152 (include)
/home/D43345/opt/paraview-5.11/scibian10_egl/lib/cmake/paraview-5.11/paraview-config.cmake:170 (find_package)
CMakeLists.txt:6 (find_package)

– Configuring incomplete, errors occurred!
See also “/home/D43345/test/cmake/build/CMakeFiles/CMakeOutput.log”.
See also “/home/D43345/test/cmake/build/CMakeFiles/CMakeError.log”.

The mentioned logs are low level and actually contain similar information in both cases, so provide no additional information here.

Part of the issue seems to be related to building Ospray and Embree using ParaView Superbuild, which does not install the relevant cmake directories in lib even when installing the SDK (it only does so for ParaView itself). This was not an apparent issue for CMake 2.22 though (all the libraries are in a same directory, and the project does not need to access the matching low-level headers).

Copying missing cmake files, headers, and some library symbolic links from the superbuild’s install directory to the ParView install directory, I still get an issue for other packages:

/home/D43345/opt/cmake-3.25/bin/cmake -DCMAKE_PREFIX_PATH=/home/D43345/opt/paraview-5.11/scibian10_egl ~/test/cmake/catalyst/
– The CXX compiler identification is GNU 8.3.0
– The C compiler identification is GNU 8.3.0
– Detecting CXX compiler ABI info
– Detecting CXX compiler ABI info - done
– Check for working CXX compiler: /usr/bin/c++ - skipped
– Detecting CXX compile features
– Detecting CXX compile features - done
– Detecting C compiler ABI info
– Detecting C compiler ABI info - done
– Check for working C compiler: /usr/bin/cc - skipped
– Detecting C compile features
– Detecting C compile features - done
– Found MPI_C: /usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi.so (found version “3.1”)
– Found MPI_CXX: /usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi_cxx.so (found version “3.1”)
– Found MPI: TRUE (found version “3.1”) found components: C CXX
– Performing Test CMAKE_HAVE_LIBC_PTHREAD
– Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
– Looking for pthread_create in pthreads
– Looking for pthread_create in pthreads - not found
– Looking for pthread_create in pthread
– Looking for pthread_create in pthread - found
– Found Threads: TRUE
– Found nlohmann_json: /home/D43345/opt/paraview-5.11/scibian10_egl/lib/cmake/nlohmann_json/nlohmann_jsonConfig.cmake (found version “3.9.1”)
– Found MPI: TRUE (found version “3.1”) found components: C
– Found Python3: /usr/bin/python3.7 (found suitable version “3.7.3”, minimum required is “3.7”) found components: Interpreter Development.Module Development.Embed
– Found ZLIB: /usr/lib/x86_64-linux-gnu/libz.so (found version “1.2.11”)
– Configuring done
CMake Error at /home/D43345/opt/paraview-5.11/scibian10_egl/lib/cmake/paraview-5.11/vtk/VTK-targets.cmake:582 (set_target_properties):
The link interface of target “VTK::opengl” contains:
OpenGL::OpenGL
but the target was not found. Possible reasons include:
* There is a typo in the target name.
* A find_package call is missing for an IMPORTED target.
* An ALIAS target is missing.
Call Stack (most recent call first):
/home/D43345/opt/paraview-5.11/scibian10_egl/lib/cmake/paraview-5.11/vtk/vtk-config.cmake:138 (include)
/home/D43345/opt/paraview-5.11/scibian10_egl/lib/cmake/paraview-5.11/paraview-config.cmake:66 (find_package)
CMakeLists.txt:6 (find_package)

CMake Error at /home/D43345/opt/paraview-5.11/scibian10_egl/lib/cmake/paraview-5.11/vtk/VTK-targets.cmake:730 (set_target_properties):
The link interface of target “VTK::RenderingRayTracing” contains:
ospray::ospray
but the target was not found. Possible reasons include:
* There is a typo in the target name.
* A find_package call is missing for an IMPORTED target.
* An ALIAS target is missing.
Call Stack (most recent call first):
/home/D43345/opt/paraview-5.11/scibian10_egl/lib/cmake/paraview-5.11/vtk/vtk-config.cmake:138 (include)
/home/D43345/opt/paraview-5.11/scibian10_egl/lib/cmake/paraview-5.11/paraview-config.cmake:66 (find_package)
CMakeLists.txt:6 (find_package)

CMake Error at /home/D43345/opt/paraview-5.11/scibian10_egl/lib/cmake/paraview-5.11/vtk/VTK-targets.cmake:1316 (set_target_properties):
The link interface of target “VTK::IOCatalystConduit” contains:
catalyst::catalyst
but the target was not found. Possible reasons include:
* There is a typo in the target name.
* A find_package call is missing for an IMPORTED target.
* An ALIAS target is missing.
Call Stack (most recent call first):
/home/D43345/opt/paraview-5.11/scibian10_egl/lib/cmake/paraview-5.11/vtk/vtk-config.cmake:138 (include)
/home/D43345/opt/paraview-5.11/scibian10_egl/lib/cmake/paraview-5.11/paraview-config.cmake:66 (find_package)
CMakeLists.txt:6 (find_package)

– Generating done
CMake Generate step failed. Build files cannot be regenerated correctly.

So there are 2 issues here:

Superbuild only installs the ParaView SDK, not those of prerequisites, which is not enough
CMake does not simply provide warnings about system libraries which do not have associated .cmake files anymore, but fails when searching for such libraries.

The first issue I can work around using a 2 step install, using Superbuild only for the prerequisites, then installing ParaView (or inserting a copy between SuperBuild’s “make” and “make install”.

The second issue, which did not occur prior to CMake 3.23, I do not know how to handle. I did not see anything in the 3.23 release notes which seems relevant, so I guess I missed something.

I won’t have this issue with Catalyst 2 (using a simpler build), but I have only gotten started, and need to support our legacy adapter for some time. Sticking with an old CMake does not seem the right way to handle this…

YvanFournier · December 22, 2022, 1:52am

Addendum: by bisection, I can see that the line in CMakeLists leading to the cascade of errors is the one containing:
ParaView::PythonCatalyst

In the

target_link_libraries(CoProcessingTest
  PRIVATE

Section. Lower level VTK dependencies seem OK.

Also, I have the same issue using CMake with the Examples/Catalyst directory of the PAraView source tree itself. So I guess there may be some configuration or command-line option to solve this, but I can’t find which.

mwestphal · December 22, 2022, 7:36am

FYI @Charles_Gueunet @nicolas.vuaille @Francois_Mazen

Hi @YvanFournier ,

Which version of the superbuild and ParaView are you trying to build exactly ?

Best,

YvanFournier · December 22, 2022, 10:12am

Hello @Mathieu Westphal, @Charles_Gueunet @nicolas.vuaille @Francois_Mazen,
I’m trying to build ParaView 5.11 (using the 5.11.0 tag on the Superbuild).
But I also reproduce it with my ParaView 5.9 or intermediate version builds, or a ParaView 5.11 buiild from SALOME 9.10, as soon as as use CMake 3.23 or above to build the project based on ParaView.

I had initially encountered this type error earlier, using ParaView 5.9 on a different machine, but thought it might be a packaging issue. Seeing that the same issue occurs with CMake 3.24 (binary from SALOME) or 3.25 (my local build), this seems to be due to some change in CMake.

YvanFournier · December 22, 2022, 2:41pm

I ran a bisection on Cmake, and on the master branch, the issue appears with the following commit, from Dec 16 2021:

ea050286e7 CMP0028: Report the target whose link interface has an offending item

With the previous commit:

a97c92e46e CMP0028: Report backtrace to link item rather than target creation

Everything still works.

That helped me see that adding:
cmake_policy(SET CMP0028 OLD)

in the CMakeLists.txt can work around this issue. As this old policy is deprecated, this is still an issue. Any suggestions (maybe a syntax change) ? As mentioned above, this also affects the legacy Catalyst examples. If the old policy is deprecated but can still be used in 2 years, that should allow the switch to Catalyst 2 with some overlap. Otherwise we may have a regression sometime.

Any help in porting to the new behavior (or update of the Catalyst examples relative to this issue) would be welcome.

mwestphal · December 23, 2022, 8:26am

Maybe @ben.boeckel have an idea.

mirenradia · February 7, 2023, 5:03pm

I think this is the same as the ParaView issue I reported last April. Is there any update on resolving this, @mwestphal, @ben.boeckel, @nicolas.vuaille?