ParaView (5.9.0) superbuild with mesa

ParaView Support
1

Hello,

I’m trying to build ParaView with Qt and Mesa, and am getting the following build error. Any suggestions? Thanks.

-Jeff

Making all in targets/libgl-xlib

CC xlib.lo

CXXLD libGL.la

clang-11: warning: argument unused during compilation: ‘-pthread’ [-Wunused-command-line-argument]

clang-11: warning: argument unused during compilation: ‘-pthread’ [-Wunused-command-line-argument]

/nobackupp16/swbuild/jcbecker/spack/opt/spack/linux-sles12-sandybridge/gcc-8.4.0/binutils-2.35.1-pwww3zdjhutfspu3aezhvbes5hh2v6kp/bin/ld: …/…/…/…/src/gallium/auxiliary/.libs/libgallium.a(u_debug_symbol.o):(.bss+0x28): multiple definition of `symbols_hash’; …/…/…/…/src/gallium/auxiliary/.libs/libgallium.a(u_debug_stack.o):(.bss+0x0): first defined here

clang-11: error: linker command failed with exit code 1 (use -v to see invocation)

Makefile:666: recipe for target ‘libGL.la’ failed

gmake[7]: *** [libGL.la] Error 1

2

This looks like a Mesa problem. @chuckatkins Thoughts?

3

Hi guys,

I’m having a similar problem building with a superbuild gallium osmesa with llvm-11 on debian. Did you ever figure out what was causing the symbols_hash problem? This is one of the top links to the problem on google, If you’ve got a solution it might increase the visibility to others.

4

This might be related to the default for -fcommon switching in the toolchain. You can try the -fno-common flag.

5

I’m also having an issue compiling the ParaView 5.9.0 superbuild with Qt and Mesa, but I am getting a different error:

gmake[7]: Leaving directory `/home/shelf1/motorsports/software/Visualization/build/paraview/5.9.0_super/superbuild/mesa/src/src/gallium/drivers/llvmpipe'
Making all in drivers/swr
gmake[7]: Entering directory `/home/shelf1/motorsports/software/Visualization/build/paraview/5.9.0_super/superbuild/mesa/src/src/gallium/drivers/swr'
  GEN      rasterizer/jitter/gen_state_llvm.h
  GEN      rasterizer/jitter/gen_builder.hpp
  GEN      rasterizer/jitter/gen_builder_meta.hpp
  GEN      rasterizer/jitter/gen_builder_intrin.hpp
Traceback (most recent call last):
  File "./rasterizer/codegen/gen_llvm_types.py", line 24, in <module>
Traceback (most recent call last):
  File "./rasterizer/codegen/gen_llvm_ir_macros.py", line 24, in <module>
Traceback (most recent call last):
Traceback (most recent call last):
  File "./rasterizer/codegen/gen_llvm_ir_macros.py", line 24, in <module>
  File "./rasterizer/codegen/gen_llvm_ir_macros.py", line 24, in <module>
    from gen_common import *
    from gen_common import *
  File "/home/shelf1/motorsports/software/Visualization/build/paraview/5.9.0_super/superbuild/mesa/src/src/gallium/drivers/swr/rasterizer/codegen/gen_common.py", line 31, in <module>
  File "/home/shelf1/motorsports/software/Visualization/build/paraview/5.9.0_super/superbuild/mesa/src/src/gallium/drivers/swr/rasterizer/codegen/gen_common.py", line 31, in <module>
    from gen_common import *
      File "/home/shelf1/motorsports/software/Visualization/build/paraview/5.9.0_super/superbuild/mesa/src/src/gallium/drivers/swr/rasterizer/codegen/gen_common.py", line 31, in <module>
from gen_common import *
  File "/home/shelf1/motorsports/software/Visualization/build/paraview/5.9.0_super/superbuild/mesa/src/src/gallium/drivers/swr/rasterizer/codegen/gen_common.py", line 31, in <module>
    from mako.template import Template
    from mako.template import Template
  File "/home/shelf1/motorsports/software/Visualization/build/paraview/5.9.0_super/install/lib/python3.8/site-packages/mako/template.py", line 10, in <module>
  File "/home/shelf1/motorsports/software/Visualization/build/paraview/5.9.0_super/install/lib/python3.8/site-packages/mako/template.py", line 10, in <module>
    from mako.template import Template
    from mako.template import Template
  File "/home/shelf1/motorsports/software/Visualization/build/paraview/5.9.0_super/install/lib/python3.8/site-packages/mako/template.py", line 10, in <module>
  File "/home/shelf1/motorsports/software/Visualization/build/paraview/5.9.0_super/install/lib/python3.8/site-packages/mako/template.py", line 10, in <module>
    from mako.lexer import Lexer
  File "/home/shelf1/motorsports/software/Visualization/build/paraview/5.9.0_super/install/lib/python3.8/site-packages/mako/lexer.py", line 11, in <module>
    from mako.lexer import Lexer
  File "/home/shelf1/motorsports/software/Visualization/build/paraview/5.9.0_super/install/lib/python3.8/site-packages/mako/lexer.py", line 11, in <module>
    from mako.lexer import Lexer
    from mako.lexer import Lexer
  File "/home/shelf1/motorsports/software/Visualization/build/paraview/5.9.0_super/install/lib/python3.8/site-packages/mako/lexer.py", line 11, in <module>
  File "/home/shelf1/motorsports/software/Visualization/build/paraview/5.9.0_super/install/lib/python3.8/site-packages/mako/lexer.py", line 11, in <module>
    from mako import parsetree, exceptions, compat
    from mako import parsetree, exceptions, compat
  File "/home/shelf1/motorsports/software/Visualization/build/paraview/5.9.0_super/install/lib/python3.8/site-packages/mako/parsetree.py", line 9, in <module>
  File "/home/shelf1/motorsports/software/Visualization/build/paraview/5.9.0_super/install/lib/python3.8/site-packages/mako/parsetree.py", line 9, in <module>
    from mako import parsetree, exceptions, compat
  File "/home/shelf1/motorsports/software/Visualization/build/paraview/5.9.0_super/install/lib/python3.8/site-packages/mako/parsetree.py", line 9, in <module>
        from mako import parsetree, exceptions, compat
from mako import exceptions, ast, util, filters, compat
  File "/home/shelf1/motorsports/software/Visualization/build/paraview/5.9.0_super/install/lib/python3.8/site-packages/mako/exceptions.py", line 273, in <module>
  File "/home/shelf1/motorsports/software/Visualization/build/paraview/5.9.0_super/install/lib/python3.8/site-packages/mako/parsetree.py", line 9, in <module>
        from mako import exceptions, ast, util, filters, compat
from mako import exceptions, ast, util, filters, compat
  File "/home/shelf1/motorsports/software/Visualization/build/paraview/5.9.0_super/install/lib/python3.8/site-packages/mako/exceptions.py", line 273, in <module>
  File "/home/shelf1/motorsports/software/Visualization/build/paraview/5.9.0_super/install/lib/python3.8/site-packages/mako/exceptions.py", line 273, in <module>
    _install_highlighting()
    from mako import exceptions, ast, util, filters, compat
  File "/home/shelf1/motorsports/software/Visualization/build/paraview/5.9.0_super/install/lib/python3.8/site-packages/mako/exceptions.py", line 270, in _install_highlighting
  File "/home/shelf1/motorsports/software/Visualization/build/paraview/5.9.0_super/install/lib/python3.8/site-packages/mako/exceptions.py", line 273, in <module>
    _install_highlighting()
  File "/home/shelf1/motorsports/software/Visualization/build/paraview/5.9.0_super/install/lib/python3.8/site-packages/mako/exceptions.py", line 270, in _install_highlighting
    _install_highlighting()
  File "/home/shelf1/motorsports/software/Visualization/build/paraview/5.9.0_super/install/lib/python3.8/site-packages/mako/exceptions.py", line 270, in _install_highlighting
    _install_highlighting()
  File "/home/shelf1/motorsports/software/Visualization/build/paraview/5.9.0_super/install/lib/python3.8/site-packages/mako/exceptions.py", line 270, in _install_highlighting
    _install_pygments()
  File "/home/shelf1/motorsports/software/Visualization/build/paraview/5.9.0_super/install/lib/python3.8/site-packages/mako/exceptions.py", line 255, in _install_pygments
    _install_pygments()
  File "/home/shelf1/motorsports/software/Visualization/build/paraview/5.9.0_super/install/lib/python3.8/site-packages/mako/exceptions.py", line 255, in _install_pygments
    _install_pygments()
  File "/home/shelf1/motorsports/software/Visualization/build/paraview/5.9.0_super/install/lib/python3.8/site-packages/mako/exceptions.py", line 255, in _install_pygments
            from mako.ext.pygmentplugin import syntax_highlight  # noqa
from mako.ext.pygmentplugin import syntax_highlight  # noqa
_install_pygments()
  File "/home/shelf1/motorsports/software/Visualization/build/paraview/5.9.0_super/install/lib/python3.8/site-packages/mako/ext/pygmentplugin.py", line 7, in <module>
      File "/home/shelf1/motorsports/software/Visualization/build/paraview/5.9.0_super/install/lib/python3.8/site-packages/mako/ext/pygmentplugin.py", line 7, in <module>
  File "/home/shelf1/motorsports/software/Visualization/build/paraview/5.9.0_super/install/lib/python3.8/site-packages/mako/exceptions.py", line 255, in _install_pygments
from mako.ext.pygmentplugin import syntax_highlight  # noqa
  File "/home/shelf1/motorsports/software/Visualization/build/paraview/5.9.0_super/install/lib/python3.8/site-packages/mako/ext/pygmentplugin.py", line 7, in <module>
    from pygments.lexers.web import \
  File "/usr/lib/python2.7/site-packages/pygments/lexers/__init__.py", line 18, in <module>
        from pygments.lexers.web import \
from pygments.lexers.web import \
  File "/usr/lib/python2.7/site-packages/pygments/lexers/__init__.py", line 18, in <module>
  File "/usr/lib/python2.7/site-packages/pygments/lexers/__init__.py", line 18, in <module>
    from mako.ext.pygmentplugin import syntax_highlight  # noqa
  File "/home/shelf1/motorsports/software/Visualization/build/paraview/5.9.0_super/install/lib/python3.8/site-packages/mako/ext/pygmentplugin.py", line 7, in <module>
        from pygments.plugin import find_plugin_lexers
from pygments.plugin import find_plugin_lexers
    from pygments.plugin import find_plugin_lexers
  File "/usr/lib/python2.7/site-packages/pygments/plugin.py", line 39, in <module>
  File "/usr/lib/python2.7/site-packages/pygments/plugin.py", line 39, in <module>
  File "/usr/lib/python2.7/site-packages/pygments/plugin.py", line 39, in <module>
    from pygments.lexers.web import \
  File "/usr/lib/python2.7/site-packages/pygments/lexers/__init__.py", line 18, in <module>
    from pygments.plugin import find_plugin_lexers
  File "/usr/lib/python2.7/site-packages/pygments/plugin.py", line 39, in <module>
    import pkg_resources
            import pkg_resources
import pkg_resources
import pkg_resources
  File "/home/shelf1/motorsports/software/Visualization/build/paraview/5.9.0_super/install/lib/python3.8/site-packages/pkg_resources/__init__.py", line 1380
    raise SyntaxError(e) from e
          File "  File " /home/shelf1/motorsports/software/Visualization/build/paraview/5.9.0_super/install/lib/python3.8/site-packages/pkg_resources/__init__.py/home/shelf1/motorsports/software/Visualization/build/paraview/5.9.0_super/install/lib/python3.8/site-packages/pkg_resources/__init__.py ", line ", line  1380 1380
 
      File "     raise SyntaxError(e) from e
/home/shelf1/motorsports/software/Visualization/build/paraview/5.9.0_super/install/lib/python3.8/site-packages/pkg_resources/__init__.py raise SyntaxError(e) from e
    ", line         1380   
          raise SyntaxError(e) from e
                                    ^
        SyntaxError   :          invalid syntax   
           ^
 ^
 SyntaxError SyntaxError:  :  invalid syntax ^
invalid syntax

SyntaxError: invalid syntax
gmake[7]: *** [rasterizer/jitter/gen_state_llvm.h] Error 1
gmake[7]: *** Waiting for unfinished jobs....
gmake[7]: *** [rasterizer/jitter/gen_builder.hpp] Error 1
gmake[7]: *** [rasterizer/jitter/gen_builder_intrin.hpp] Error 1
gmake[7]: *** [rasterizer/jitter/gen_builder_meta.hpp] Error 1
gmake[7]: Leaving directory `/home/shelf1/motorsports/software/Visualization/build/paraview/5.9.0_super/superbuild/mesa/src/src/gallium/drivers/swr'
gmake[6]: *** [all-recursive] Error 1
gmake[6]: Leaving directory `/home/shelf1/motorsports/software/Visualization/build/paraview/5.9.0_super/superbuild/mesa/src/src/gallium'
gmake[5]: Leaving directory `/home/shelf1/motorsports/software/Visualization/build/paraview/5.9.0_super/superbuild/mesa/src/src'
gmake[5]: *** [all-recursive] Error 1
gmake[4]: *** [all] Error 2
gmake[4]: Leaving directory `/home/shelf1/motorsports/software/Visualization/build/paraview/5.9.0_super/superbuild/mesa/src/src'
gmake[3]: *** [all-recursive] Error 1
gmake[3]: Leaving directory `/home/shelf1/motorsports/software/Visualization/build/paraview/5.9.0_super/superbuild/mesa/src'
CMake Error at /home/shelf1/motorsports/software/Visualization/build/paraview/5.9.0_super/superbuild/sb-mesa-build.cmake:47 (message):
  Failed with exit code 2


make[2]: *** [superbuild/mesa/stamp/mesa-build] Error 1
make[2]: Leaving directory `/home/shelf1/motorsports/software/Visualization/build/paraview/5.9.0_super'
make[1]: *** [superbuild/CMakeFiles/mesa.dir/all] Error 2
make[1]: Leaving directory `/home/shelf1/motorsports/software/Visualization/build/paraview/5.9.0_super'
make: *** [all] Error 2

The Python syntax is incorrect? The mesa package is based on Python/2.7 but because I’m also asking the ParaView Superbuild for the Python3 project, the mesa project is getting confused and finding the wrong version of Python?

6

You have a Python2 floating around. We require Python3 everywhere. It looks like mesa found a Python2 and is trying to use it. I’d recommend looking around the CMakeCache.txt and ensure that nothing points to Python2.

7

The easiest solution to my problem is to turn off the swr driver with -Dmesa_USE_SWR:STRING="OFF". That’s my personal resolution, but in an effort to help others I’ll continue to investigate.

Here is build/paraview/5.9.0_super_swr/CMakeCache.txt:
CMakeCache.txt (45.5 KB)
I see nothing in here related to python2

However, in build/paraview/5.9.0_super_swr/superbuild/mesa/src/src/gallium/drivers/swr/Makefile I see only references to python2 (line numbers left in for reference):

1156 PYTHON2 = python2.7
...
1579 PYTHON_GEN = $(AM_V_GEN)$(PYTHON2) $(PYTHON_FLAGS)
...
4254 rasterizer/jitter/gen_state_llvm.h: rasterizer/codegen/gen_llvm_types.py rasterizer/codegen/templates/gen_llvm.hpp rasterizer/core/state.h rasterizer/codegen/gen_common.py                                                              
4255         $(MKDIR_GEN)                                                                                                                                                                                                                     
4256         $(PYTHON_GEN) \                                                                                                                                                                                                                  
4257                 $(srcdir)/rasterizer/codegen/gen_llvm_types.py \                                                                                                                                                                         
4258                 --input $(srcdir)/rasterizer/core/state.h \                                                                                                                                                                              
4259                 --output rasterizer/jitter/gen_state_llvm.h                                                                                                                                                                              
4260         $(AM_V_GEN)touch $@                                                                                                                                                                                                              
4261                                                                                                                                                                                                                                          
4262 rasterizer/jitter/gen_builder.hpp: rasterizer/codegen/gen_llvm_ir_macros.py rasterizer/codegen/templates/gen_builder.hpp rasterizer/codegen/gen_common.py                                                                                
4263         $(MKDIR_GEN)                                                                                                                                                                                                                     
4264         $(PYTHON_GEN) \                                                                                                                                                                                                                  
4265                 $(srcdir)/rasterizer/codegen/gen_llvm_ir_macros.py \                                                                                                                                                                     
4266                 --input $(LLVM_INCLUDEDIR)/llvm/IR/IRBuilder.h \                                                                                                                                                                         
4267                 --output rasterizer/jitter \                                                                                                                                                                                             
4268                 --gen_h                                                                                                                                                                                                                  
4269         $(AM_V_GEN)touch $@                                                                                                                                                                                                              
4270                                                                                                                                                                                                                                          
4271 rasterizer/jitter/gen_builder_meta.hpp: rasterizer/codegen/gen_llvm_ir_macros.py rasterizer/codegen/templates/gen_builder.hpp rasterizer/codegen/gen_common.py                                                                           
4272         $(MKDIR_GEN)                                                                                                                                                                                                                     
4273         $(PYTHON_GEN) \                                                                                                                                                                                                                  
4274                 $(srcdir)/rasterizer/codegen/gen_llvm_ir_macros.py \                                                                                                                                                                     
4275                 --output rasterizer/jitter \                                                                                                                                                                                             
4276                 --gen_meta_h                                                                                                                                                                                                             
4277         $(AM_V_GEN)touch $@                                                                                                                                                                                                              
4278                                                                                                                                                                                                                                          
4279 rasterizer/jitter/gen_builder_intrin.hpp: rasterizer/codegen/gen_llvm_ir_macros.py rasterizer/codegen/templates/gen_builder.hpp rasterizer/codegen/gen_common.py                                                                         
4280         $(MKDIR_GEN)                                                                                                                                                                                                                     
4281         $(PYTHON_GEN) \                                                                                                                                                                                                                  
4282                 $(srcdir)/rasterizer/codegen/gen_llvm_ir_macros.py \                                                                                                                                                                     
4283                 --output rasterizer/jitter \                                                                                                                                                                                             
4284                 --gen_intrin_h                                                                                                                                                                                                           
4285         $(AM_V_GEN)touch $@

This looks like the build is requiring python2 to be used to automatically generate CPP rasterizer source code from python files.

build/paraview/5.9.0_super_swr/superbuild/mesa/src/src/gallium/drivers/swr/rasterizer/jitter/meson.build:

 22 gen_builder_hpp = custom_target(                                                                                                                                                                                                          
 23   'gen_builder.hpp',                                                                                                                                                                                                                      
 24   input : [                                                                                                                                                                                                                               
 25     swr_gen_llvm_ir_macros_py,                                                                                                                                                                                                            
 26     join_paths(                                                                                                                                                                                                                           
 27       dep_llvm.get_configtool_variable('includedir'), 'llvm', 'IR',                                                                                                                                                                       
 28       'IRBuilder.h'                                                                                                                                                                                                                       
 29     )                                                                                                                                                                                                                                     
 30   ],                                                                                                                                                                                                                                      
 31   output : 'gen_builder.hpp',                                                                                                                                                                                                             
 32   command : [                                                                                                                                                                                                                             
 33     prog_python2, '@INPUT0@', '--input', '@INPUT1@', '--output', '@OUTPUT@',                                                                                                                                                              
 34     '--gen_h', '--output-dir', '@OUTDIR@'                                                                                                                                                                                                 
 35   ],                                                                                                                                                                                                                                      
 36   depend_files : swr_gen_builder_depends,                                                                                                                                                                                                 
 37   build_by_default : true,                                                                                                                                                                                                                
 38 )                                                                                                                                                                                                                                         
 39                                                                                                                                                                                                                                           
 40 gen_builder_meta_hpp = custom_target(                                                                                                                                                                                                     
 41   'gen_builder_meta.hpp',                                                                                                                                                                                                                 
 42   input : '../codegen/gen_llvm_ir_macros.py',                                                                                                                                                                                             
 43   output : 'gen_builder_meta.hpp',                                                                                                                                                                                                        
 44   command : [                                                                                                                                                                                                                             
 45     prog_python2, '@INPUT0@', '--gen_meta_h', '--output', '@OUTPUT@',                                                                                                                                                                     
 46     '--output-dir', '@OUTDIR@'                                                                                                                                                                                                            
 47   ],                                                                                                                                                                                                                                      
 48   depend_files : swr_gen_builder_depends,                                                                                                                                                                                                 
 49 )                                                                                                                                                                                                                                         
 50                                                                                                                                                                                                                                           
 51 gen_builder_intrin_hpp = custom_target(                                                                                                                                                                                                   
 52   'gen_builder_intrin.hpp',                                                                                                                                                                                                               
 53   input : '../codegen/gen_llvm_ir_macros.py',                                                                                                                                                                                             
 54   output : 'gen_builder_intrin.hpp',                                                                                                                                                                                                      
 55   command : [                                                                                                                                                                                                                             
 56     prog_python2, '@INPUT0@', '--gen_intrin_h', '--output', '@OUTPUT@',                                                                                                                                                                   
 57     '--output-dir', '@OUTDIR@'                                                                                                                                                                                                            
 58   ],                                                                                                                                                                                                                                      
 59   depend_files : swr_gen_builder_depends,                                                                                                                                                                                                 
 60 )

Are you sure the build requires Python3 everywhere?

8

Well, the modern one does. If you’re on the 5.9.0 tag, I suppose some things could still want Python2. I don’t recall how much PYTHONPATH stuff we do that may mess things up (something certainly seems to confuse things at least by your logs).

I might recommend pulling the Docker image mentioned in .gitlab/os-linux.yml (kitware/paraview:ci-superbuild-centos7-20210113) or building it (.gitlab/ci/docker/centos7 if it has expired from DockerHub) and building using the instructions in the CI scripts.