paraview superbuild failing for 5.10

Development
1

I’m getting a build issue with the PV 5.10 superbuild (using the V5.10.0 git tag) and gcc 7.3.0 and the following:

cmake -DBUILD_TESTING=OFF -DCMAKE_INSTALL_PREFIX=/app/DAAC/paraview/5.10.0_osmesa_intelmpi/ -DENABLE_python3=ON -DENABLE_osmesa=ON -DENABLE_openvdb=ON -DENABLE_mpi=ON -DENABLE_numpy=ON -DUSE_SYSTEM_mpi=ON -DSUPERBUILD_PROJECT_PARALLELISM=30 ../paraview-superbuild/

I’m hitting this on two separate systems. It looks to be failing in Cython with:

running build_ext

Error compiling Cython file:
------------------------------------------------------------
...
object_expr = TypedExprNode(py_object_type, may_be_none=True)
# Fake rhs to silence "unused variable" warning
fake_rhs_expr = TypedExprNode(unspecified_type)


class ControlBlock(object):
^
------------------------------------------------------------

Cython/Compiler/FlowControl.py:36:0: Compiler crash in AnalyseDeclarationsTransform

ModuleNode.body = StatListNode(FlowControl.py:1:0)
StatListNode.stats[14] = StatListNode(FlowControl.py:36:0)
StatListNode.stats[0] = CClassDefNode(FlowControl.py:36:0,
    class_name = 'ControlBlock',
    doc = "Control flow graph node. Sequence of assignments and name references.\n\n       children  set of children nodes\n       parents   set of parent nodes\n       positions set of position markers\n\n       stats     list of block statements\n       gen       dict of assignments generated by this block\n       bounded   set  of entries that are definitely bounded in this block\n\n       Example:\n\n        a = 1\n        b = a + c # 'c' is already bounded or exception here\n\n        stats = [Assignment(a), NameReference(a), NameReference(c),\n                     Assignment(b)]\n        gen = {Entry(a): Assignment(a), Entry(b): Assignment(b)}\n        bounded = set([Entry(a), Entry(c)])\n\n    ",
    visibility = 'private')

Compiler crash traceback from this point on:
  File "/p/home/apps/DAAC/paraview/5.10.0_osmesa_intelmpi_build/build/superbuild/pythoncython/src/Cython/Compiler/Visitor.py", line 180, in _visit
    return handler_method(obj)
  File "/p/home/apps/DAAC/paraview/5.10.0_osmesa_intelmpi_build/build/superbuild/pythoncython/src/Cython/Compiler/ParseTreeTransforms.py", line 1636, in visit_CClassDefNode
    self._inject_pickle_methods(node)
  File "/p/home/apps/DAAC/paraview/5.10.0_osmesa_intelmpi_build/build/superbuild/pythoncython/src/Cython/Compiler/ParseTreeTransforms.py", line 1704, in _inject_pickle_methods
    checksum = '0x%s' % hashlib.md5(' '.join(all_members_names).encode('utf-8')).hexdigest()[:7]
ValueError: [digital envelope routines: EVP_DigestInit_ex] disabled for fips
Compiling /p/home/apps/DAAC/paraview/5.10.0_osmesa_intelmpi_build/build/superbuild/pythoncython/src/Cython/Plex/Scanners.py because it changed.
Compiling /p/home/apps/DAAC/paraview/5.10.0_osmesa_intelmpi_build/build/superbuild/pythoncython/src/Cython/Plex/Actions.py because it changed.
Compiling /p/home/apps/DAAC/paraview/5.10.0_osmesa_intelmpi_build/build/superbuild/pythoncython/src/Cython/Compiler/Scanning.py because it changed.
Compiling /p/home/apps/DAAC/paraview/5.10.0_osmesa_intelmpi_build/build/superbuild/pythoncython/src/Cython/Compiler/Visitor.py because it changed.
Compiling /p/home/apps/DAAC/paraview/5.10.0_osmesa_intelmpi_build/build/superbuild/pythoncython/src/Cython/Compiler/FlowControl.py because it changed.
Compiling /p/home/apps/DAAC/paraview/5.10.0_osmesa_intelmpi_build/build/superbuild/pythoncython/src/Cython/Runtime/refnanny.pyx because it changed.
Compiling /p/home/apps/DAAC/paraview/5.10.0_osmesa_intelmpi_build/build/superbuild/pythoncython/src/Cython/Compiler/FusedNode.py because it changed.
Compiling /p/home/apps/DAAC/paraview/5.10.0_osmesa_intelmpi_build/build/superbuild/pythoncython/src/Cython/Tempita/_tempita.py because it changed.
[1/8] Cythonizing /p/home/apps/DAAC/paraview/5.10.0_osmesa_intelmpi_build/build/superbuild/pythoncython/src/Cython/Compiler/FlowControl.py
Traceback (most recent call last):
  File "/p/home/apps/DAAC/paraview/5.10.0_osmesa_intelmpi_build/build/superbuild/pythoncython/src/setup.py", line 228, in <module>
    setup(
  File "/app/DAAC/paraview/5.10.0_osmesa_intelmpi_build/build/install/lib/python3.9/site-packages/setuptools/__init__.py", line 153, in setup
    return distutils.core.setup(**attrs)
  File "/app/DAAC/paraview/5.10.0_osmesa_intelmpi_build/build/install/lib/python3.9/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/app/DAAC/paraview/5.10.0_osmesa_intelmpi_build/build/install/lib/python3.9/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/app/DAAC/paraview/5.10.0_osmesa_intelmpi_build/build/install/lib/python3.9/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/app/DAAC/paraview/5.10.0_osmesa_intelmpi_build/build/install/lib/python3.9/distutils/command/build.py", line 135, in run
    self.run_command(cmd_name)
  File "/app/DAAC/paraview/5.10.0_osmesa_intelmpi_build/build/install/lib/python3.9/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/app/DAAC/paraview/5.10.0_osmesa_intelmpi_build/build/install/lib/python3.9/distutils/dist.py", line 984, in run_command
    cmd_obj.ensure_finalized()
  File "/app/DAAC/paraview/5.10.0_osmesa_intelmpi_build/build/install/lib/python3.9/distutils/cmd.py", line 107, in ensure_finalized
    self.finalize_options()
  File "/p/home/apps/DAAC/paraview/5.10.0_osmesa_intelmpi_build/build/superbuild/pythoncython/src/Cython/Distutils/build_ext.py", line 20, in finalize_options
    self.distribution.ext_modules[:] = cythonize(
  File "/p/home/apps/DAAC/paraview/5.10.0_osmesa_intelmpi_build/build/superbuild/pythoncython/src/Cython/Build/Dependencies.py", line 1102, in cythonize
    cythonize_one(*args)
  File "/p/home/apps/DAAC/paraview/5.10.0_osmesa_intelmpi_build/build/superbuild/pythoncython/src/Cython/Build/Dependencies.py", line 1225, in cythonize_one
    raise CompileError(None, pyx_file)
Cython.Compiler.Errors.CompileError: /p/home/apps/DAAC/paraview/5.10.0_osmesa_intelmpi_build/build/superbuild/pythoncython/src/Cython/Compiler/FlowControl.py
CMake Error at /app/DAAC/paraview/5.10.0_osmesa_intelmpi_build/build/superbuild/sb-pythoncython-build.cmake:47 (message):
  Failed with exit code 1


make[2]: *** [superbuild/pythoncython/stamp/pythoncython-build] Error 1
make[1]: *** [superbuild/CMakeFiles/pythoncython.dir/all] Error 2
make: *** [all] Error 2

Any ideas?

2

It looks like the OpenSSL has some hashes disabled (md5?) and therefore cannot be used. Is there some FIPS-locked build in use here?

3

With some investigation I was told that our openssl disables some md5 hashes so I needed to build openssl in the superbuild. In order to do that I made the following change to the main CMakeLists.txt (not the correct alphabetical location but you get the idea):

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 1c21243..fcfccde 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -116,7 +116,8 @@ function (superbuild_find_projects var)
     xdmf3
     zfp
     zlib
-    zstd)
+    zstd
+    openssl)
 
   # These projects only support Intel processors.
   set(tbb_supported OFF)

Then I added -DENABLE_openssl=ON -DSUPERBUILD_ENABLE_OPENSSL:BOOL=ON -D_superbuild_enable_openssl:BOOL=ON to my CMake superbuild configuration. Not sure if all of this was necessary but it worked for me. Should these changes, in some form or other, go into the PV superbuild?

openssl.common.cmake states:

if (NOT _superbuild_enable_openssl)
  message(FATAL_ERROR
    "Due to complications around shipping OpenSSL in binaries, the superbuild "
    "requires explicit permission to deal with OpenSSL. Please see "
    "documentation for use of OpenSSL.")
endif ()

but I couldn’t find any documentation on use of OpenSSL. Did I just fail to find it or is it missing?

4

ParaView doesn’t have any OpenSSL bits, so it hasn’t been documented there. In the meantime, you can just pass -D_superbuild_enable_openssl=ON to bypass this.

5

It looks like at least some of the other parts I mentioned before seem to be necessary. When I just do -D_superbuild_enable_openssl=ON I get the following:

 Building projects: cxx11, mpi, nlohmannjson, bzip2, zlib, png, ffi, sqlite, python3, pythonsetuptools, pythongast, pythonbeniget, pythonply, pythonpythran, pythoncython, numpy, tbb, cxx14, zstd, blosc, boost,
 openvdb, llvm, expat, pythonmako, ninja, meson, osmesa, paraview
 CMake Warning (dev) at superbuild/cmake/SuperbuildMacros.cmake:213 (message):
   Optional dependencies for python3 not found: openssl
6

Yes, that is an additional thing that’s needed to what else you had tried, not sufficient on its own.

7

To close this out, the PV superbuild has been improved to allow building openssl. See the MR.