Need some help debugging build failure of 5.8.0

I’m attempting to build ParaView 5.8.0, specifically an MPI-enabled server-only version based on EGL that is to be used on our GPU nodes. I’m not building it directly myself, but using the EasyBuild framework. I previously sucesfully built 5.6.0 and now am trying to update the recipe to 5.8.0 but hit upon this error during the build (full logs linked below):

../../lib64/libvtkCommonCore-pv5.8.so.5.8: undefined reference to `vtkloguru::g_preamble_time'
../../lib64/libvtkCommonCore-pv5.8.so.5.8: undefined reference to `vtkloguru::get_thread_name(char*, unsigned long long, bool)'
../../lib64/libvtkCommonCore-pv5.8.so.5.8: undefined reference to `vtkloguru::add_callback(char const*, void (*)(void*, vtkloguru::Message const&), void*, int, void (*)(void*), void (*)(void*))'
../../lib64/libvtkCommonCore-pv5.8.so.5.8: undefined reference to `vtkloguru::log(int, char const*, unsigned int, char const*, ...)'
../../lib64/libvtkCommonCore-pv5.8.so.5.8: undefined reference to `vtkloguru::LogScopeRAII::~LogScopeRAII()'
../../lib64/libvtkCommonCore-pv5.8.so.5.8: undefined reference to `vtkloguru::remove_callback(char const*)'
../../lib64/libvtkCommonCore-pv5.8.so.5.8: undefined reference to `vtkloguru::vstrprintf[abi:cxx11](char const*, __va_list_tag*)'
../../lib64/libvtkCommonCore-pv5.8.so.5.8: undefined reference to `vtkloguru::set_thread_name(char const*)'
../../lib64/libvtkCommonCore-pv5.8.so.5.8: undefined reference to `vtkloguru::init(int&, char**, char const*)'
../../lib64/libvtkCommonCore-pv5.8.so.5.8: undefined reference to `vtkloguru::LogScopeRAII::LogScopeRAII(int, char const*, unsigned int, char const*, ...)'
../../lib64/libvtkCommonCore-pv5.8.so.5.8: undefined reference to `vtkloguru::g_stderr_verbosity'
../../lib64/libvtkCommonCore-pv5.8.so.5.8: undefined reference to `vtkloguru::add_file(char const*, vtkloguru::FileMode, int)'
../../lib64/libvtkCommonCore-pv5.8.so.5.8: undefined reference to `vtkloguru::current_verbosity_cutoff()'
../../lib64/libvtkCommonCore-pv5.8.so.5.8: undefined reference to `vtkloguru::g_preamble_date'

This is with Intel ICC 18.0.3.20180410. The EasyBuild recipe shown below gives some indication as to the CMake flags being set and dependency version used. I can’t immediately see what is going wrong, as the vtkloguru library seems to be correctly built. Also, iterating any changes is quite slow as I need to go through EasyBuild for each re-build. So any hints as to what might be going on are greatly appreciated.

EasyBuild recipe:

# EGL-based offscreen rendering server. This module only
# include pvserver and pvbatch, NOT the GUI binary (paraview). 
# In fact, it does not contain any X windows related functionality
# at all and doesn't need an X server for GPU-accelerated
# rendering.
# See https://blog.kitware.com/off-screen-rendering-through-the-native-platform-interface-egl/

easyblock = 'CMakeMake'

name = 'ParaView-server-egl'
version = '5.8.0'
versionsuffix = '-mpi-nvidia'

homepage = "http://www.paraview.org"
description = "ParaView is a scientific parallel visualizer. This module only contains the EGL-based offscreen rendering server"

toolchain = {'name': 'intel', 'version': '2018b'}
toolchainopts = {'pic': True, 'usempi': True}

download_suffix = 'download.php?submit=Download&version=v%(version_major_minor)s&type=source&os=all&downloadFile='
source_urls = ['http://www.paraview.org/paraview-downloads/%s' % download_suffix]
sources = ["ParaView-v%(version)s.tar.gz"]

dependencies = [
    ('libGLU-nvidia',   '9.0.0'),
    ('zlib',            '1.2.11'),
    ('HDF5',            '1.10.2'),
    ('Python',          '3.6.6'),
    ('FFmpeg',          '4.1'),
]

builddependencies = [('CMake', '3.12.1')]

# Need RHEL packages
osdependencies = [(
    # EGL headers and library
    'mesa-libEGL-devel',
    'mesa-libEGL',
)]

separate_build_dir = True

# Check to see if this is an actual node with NVidia hardware. 
# Note that it assumes that the libGL libraries are the actual NVidia versions.
# XXX PM 20190920 - Disabled as we currently don't seem to have a way to build on
# the GPU nodes directly. Looking at libGLU-nvidia it seems the system-wide
# nvidia libs *are* picked up correctly anyway
#preconfigopts = " ls -l /dev/nvidiactl && "

configopts = ''
configopts += '-DPARAVIEW_BUILD_SHARED_LIBS=ON '
# Without internet connection turn off testing (uncomment the following line)
configopts += '-DBUILD_TESTING=OFF '
configopts += '-DPARAVIEW_INSTALL_DEVELOPMENT_FILES=ON '
configopts += '-DPARAVIEW_ENABLE_FFMPEG=ON '
configopts += '-DPARAVIEW_USE_PYTHON=ON '
configopts += '-DPARAVIEW_USE_MPI=ON '
#configopts += '-DVTK_USE_SYSTEM_HDF5=ON '
# https://git.ia.surfsara.nl/easybuild-surfsara/easyconfigs-surfsara/issues/107#note_34665
configopts += '-DPARAVIEW_USE_VTKM=OFF '

# No GUI!
configopts += '-DPARAVIEW_USE_QT=OFF '

# Use EGL for OpenGL
configopts += '-DVTK_USE_X=OFF '
configopts += '-DVTK_OPENGL_HAS_EGL=ON '
configopts += '-DVTK_OPENGL_HAS_OSMESA=OFF '
configopts += '-DVTK_USE_OFFSCREEN_EGL=ON '
#EGL_INCLUDE_DIR
#EGL_LIBRARY
#EGL_opengl_LIBRARY
#configopts += '-DOPENGL_INCLUDE_DIR=$EBROOTMESA/include -DOPENGL_gl_LIBRARY=$EBROOTMESA/lib/libGL.%s ' % SHLIB_EXT
#configopts += '-DOSMESA_INCLUDE_DIR=$EBROOTMESA/include -DOSMESA_LIBRARY=$EBROOTMESA/lib/libOSMesa.%s ' % SHLIB_EXT
configopts += '-DOPENGL_glu_LIBRARY=$EBROOTLIBGLUMINNVIDIA/lib/libGLU.%s ' % SHLIB_EXT

# This EasyConfig suffers from this issue, which is probably specific to intel-2018b 
# https://git.ia.surfsara.nl/easybuild-surfsara/easyconfigs-surfsara/issues/104 
# see also https://git.ia.surfsara.nl/easybuild-surfsara/easyconfigs-surfsara/issues/107
# workaround: 
configopts += ' -DCMAKE_C_FLAGS="$CMAKE_C_FLAGS -no-ipo -ip" -DCMAKE_CXX_FLAGS="$CMAKE_CXX_FLAGS -no-ipo -ip"'

# Or consult https://gitlab.kitware.com/vtk/vtk/blob/master/Documentation/dev/git/data.md
# and download ExternalData to $EASYBUILD_SOURCEPATH and adjust -DExternalData_OBJECT_STORES accordingly
# Without internet connection, comment the following two lines (configopts and prebuildopts)
#configopts += '-DExternalData_OBJECT_STORES=%(builddir)s/ExternalData '

# PM (but actually inspired by JD ;-))
# Added RPATH entry pointing to ParaView's own libraries (under lib/paraview-X.Y), 
# otherwise the EasyBuild sanity check fails during build as it tries to verify that all 
# binaries can find their libraries through RPATH.
configopts += '-DCMAKE_EXE_LINKER_FLAGS=-Wl,-R\$ORIGIN/../lib/paraview-%(version_major_minor)s -DCMAKE_SHARED_LINKER_FLAGS=-Wl,-R\$ORIGIN '

# The ParaView server can be cranky, test downloads are quite often failing, especially in the case
# of parallel downloads. Using ; insted of && gives a second chance to download the test files, if the
# first serial attempt would fail.
#prebuildopts = 'make VTKData ;'

moduleclass = 'vis'

Build output, contains both EasyBuild as well as CMake output:

ParaView-server-egl-5.8.0-intel-2018b-mpi-nvidia.ebout.log.gz (141.9 KB)

@ben.boeckel

@paulmelis could you attach the CMakeCache.txt generated by cmake?

Here you go: CMakeCache.txt.bz2 (24.1 KB)

Cc: @utkarsh.ayachit for the loguru error. Do we have a mismatch where the code is expecting loguru, but the CMake isn’t building it somehow?

@ben.boeckel, based on that, no, vtkloguru is indeed being built. Maybe something is not correct on linking side where it ends up picking up a wrong version of the library?

@paulmelis, do you have other / older ParaView’s in your PATH, LD_LIBRARY_PATH or some such?

No, I don’t think so, but will check tomorrow. In the meantime I’ve tried the GCC-based build of the same version and that finishes correctly. So this appears to an Intel compiler related thing. I’ll see if I can spot some differences in the build logs.