Note: this discussion only applies to Linux, and not Windows or macOS
Anyone who has used ParaView in HPC environments should be very familiar with this MPI specific issue: if want to use the MPI implementation provided by your HPC system (which is most likely tuned for your HPC hardware) you must make sure your application executables are built against the same MPI implementation. For most ParaView HPC site administrators and users, this translates to the aphorism: for HPC, build your own ParaView from source; don’t use ParaView binaries from paraview.org.
The MPICH ABI compatibility initiative announced in SC13 and now supported by several mainstream MPICH implementations has the potential to make this obsolete. In other words, paraview.org binaries can potentially work with your HPC provided MPI implementation without the need to compile from source!
ParaView superbuild (as of ParaView 5.8) is already using a compatible version of MPICH (3.3). Thus, technically, the ParaView binaries already support this. The problem is that since the package includes
libmpi.so.12 under the standard library loading search path, the loader loads the libmpi we include in the package rather than the one provided by the platform (one can manually just delete it, but that’s hardly an elegant solution).
One possible solution to address this is to follow the pattern we already use for choosing between Mesa GL and system GL. For GL, by default, the ParaView executables use system GL. To use mesa instead, one runs the launcher
paraview-mesa that sets appropriate environment variables to load the mesa GL libraries instead of your system ones. We could follow the same pattern for MPI: let ParaView executables use system MPI by default, but then provide a new launcher,
paraview-mpi, that lets one use the MPI implementation packaged with ParaView instead. The advantage of having two launchers is that they can be combined as needed e.g. to launch paraview using mesa and packaged mpi, one can run
paraview-mesa --backend swr paraview-mpi paraview or
paraview-mpi paraview-mesa --backend swr paraview. The disadvantage is that the command line gets quite confusing.
Second option is we provide a single launcher
paraview-launcher that can setup paths needed for using packaged Mesa GL or MPI implementation.
However, both these approaches have a serious disadvantage when used for MPI implementation selection. While with GL, it was okay to expect most systems to have a GL implementation, it’s not reasonable to expect the same for MPI. If no compatible MPI implementation is available on the system, the paraview binaries will simply fail to start unless the launcher executable is used.
Third option is that we change the package so that by default the standard executables i.e.
pvpython etc. are launchers themselves. They setup paths such that the packaged MPI is used. By passing optional command line arguments e.g.
--system-mpi, they can be made to skip that path setup and use system MPI instead. Similarly it can handle Mesa GL vs System GL, with system GL being the default behavior.
Fourth option is that we simply provide separate downloads: one with mpi libraries included in the package, and another without those for those who want to use compatible system MPI implementation (similar to how we do it for Windows).
Thoughts? Any other suggestions?