Problem with MPI_Comm in ParaView 5.10.0 RC1 when using External MPI version

Development
1

There is a new issue in ParaView 5.10.0-RC1 when using External MPI version that was not present when doing the same with ParaView 5.9.1 or below.

When building ParaView 5.1.0-RC1 with the superbuild script and utilizing an external MPI version.

I get an error from the file

Build_directory/superbuild/paraview/src/VTK/ThirdParty/ioss/vtkioss/Ioss_CodeTypes.h

on line 68 where the definition of MPI_Comm

using MPI_Comm = int;

conflicts with the external version of MPI_Comm

using MPI_Comm = unsigned int;

from mpi.h in the external version.

This conflict was not present in version 5.9.1 or below.

A simple workaround is to set the value to be an unsigned int in the file Ioss_CodeTypes.h

This change will allow ParaView to build and function correctly but should not be necessary.

Thanks,

Joe Hennessey

2

Hi Joe!
I pointed the architect/ implementer of the IOSS library at this post. Ben from Kitware also.
Alan

3

Does the external mpi that you are using define MPI_COMM_WORLD ? The line you are hitting shuold only be hit if the mpi being used does not define MPI_COMM_WORLD… It is inside a #if block:

#ifndef MPI_COMM_WORLD
#define MPI_COMM_WORLD 0
using MPI_Comm       = int;
#endif
4

Here is the start of the mpi.h file that I am using. As far as I can tell this would define MPI_COMM_WORLD in the enum below.

#ifndef MPI_H_INCLUDED
#define MPI_H_INCLUDED

#if defined(__cplusplus)
extern "C" {
#endif

/******************/
/* MPI-1 bindings */
/******************/


#define MPI_BOTTOM              ((MPI_Aint)0)

typedef long                    MPI_Aint;
typedef unsigned int            MPI_Request;
typedef unsigned int            MPI_Group;
typedef unsigned int            MPI_Comm;
typedef unsigned int            MPI_Errhandler;
typedef int                     MPI_Fint;
typedef unsigned int            MPI_Op;
typedef unsigned int            MPI_Datatype;
typedef unsigned int            MPI_Win;
typedef long long               MPI_Count;

#define MPI_FILE_DEFINED
typedef struct ADIOI_FileD *MPI_File;

typedef struct {
        int MPI_SOURCE;
        int MPI_TAG;
        int MPI_ERROR;
        int reserved[1];
        long size;
} MPI_Status;

extern MPI_Status mpi_sgi_status_ignore;
#define MPI_STATUS_IGNORE (&mpi_sgi_status_ignore)
#define MPI_STATUSES_IGNORE (MPI_STATUS_IGNORE)

extern int mpi_sgi_inplace;
#define MPI_IN_PLACE ( (void*)&mpi_sgi_inplace)

typedef MPI_Status MPI_F08_status;

extern MPI_F08_status * MPI_F08_STATUS_IGNORE;
extern MPI_F08_status * MPI_F08_STATUSES_IGNORE;

enum {
        MPI_COMM_NULL           = 0,
        MPI_COMM_WORLD          = 1,
        MPI_COMM_SELF           = 2,
        _MPI_SGI_COMM_LAST
};
5

[Ignore this comment – I think the issue is that this is a parallel build, but SEACAS_HAVE_MPI is not defined]

It “defines” MPI_COMM_WORLD, but it doesn’t #define MPI_COMM_WORLD. I was under the impression that MPI_COMM_WORLD was required to be in a #define, but obviously that is not the case here…

It looks like some other codes have had issues with this assumption also… (Ambuiguous ADIOS constructors on HPE MPT MPI where MPI_COMM_WORLD is an integer · Issue #2893 · ornladios/ADIOS2 · GitHub) and it looks like Microsoft also uses an enum for MPI_COMM_WORLD and not a #define. I will have to figure out a better way to handle the case that this was meant to solve…

For now, I would suggest removing the entire #if block for MPI_COMM_WORLD and MPI_COMM_SELF

6

[Ignore this comment – I think the issue is that this is a parallel build, but SEACAS_HAVE_MPI is not defined]
I need to figure out why I needed this code in the first place since it is right after an include <mpi.h> which should provide what is needed…

7

Sorry for all the churn.

I think that maybe SEACAS_HAVE_MPI is not being defined which means that mpi.h is not being included as it should… This should maybe be a better name for this symbol, but needs to be defined if the IOSS library is being compiled using mpi…

8

AFAICT, SEACAS_HAVE_MPI gets into SEACASIoss_config.h. Is this header not being included somewhere it needs to be?

9

It should be included in Ioss_CodeTypes.h (the file we are looking at) a few lines above where the problem is showing up.

10

Is it defined in the config.h header on the machine? If not, then we need to look at why this code isn’t working as expected.

11

Hmm. Not in our import. Was this include added after where we are based upon?

Edit: Not seeing an include on master either.

12

Line 47

13

Indeed. but that is after the check for SEACAS_HAVE_MPI (I stopped looking once I got to the use), so that’s why it isn’t working.

Gah, too early in the morning. The BUILT_IN_SIERRA check is also setup, not use. Hrm.

14

@joeh Could you please provide the SEACASIoss_config.h header that is around? Should be in the build tree (or install tree).

15

Ben,

Here are the contents of the file
Build_directory/superbuild/paraview/build/VTK/ThirdParty/ioss/vtkioss/SEACASIoss_config.h

/* #undef SEACAS_HAVE_MPI */

/* #undef IOSS_THREADSAFE */

/* Define to indicate that the SEACAS project is being built with the KOKKOS package enabled*/
/* #undef SEACAS_HAVE_KOKKOS */

/* #undef SEACAS_HAVE_FAODEL */

/* #undef SEACAS_HAVE_DATAWARP */

#define SEACAS_HAVE_EXODUS

#define SEACAS_HAVE_CGNS

/* #undef SEACAS_HAVE_PAMGEN */

/* #undef HAVE_SEACASIOSS_ADIOS2 */
16

So SEACAS_HAVE_MPI is not set correctly (assuming this is a parallel build). In the CMakeLists.txt that creates the SEACASIoss_config.h, the setting is controlled by TPL_ENABLE_MPI which should be defined if you have a parallel build…

17

Gregory,

It is a parallel build.

Thanks,

Joe

18

Note that what SEACAS thinks is a parallel build may not match with what VTK thinks. Here, we have a non-parallel netcdf (and probably hdf5), so we cannot enable Ioss MPI support. Unfortunately, it seems that MPI is included elsewhere in an Ioss-using TU and the MPI compat stuff gets confused. I think that, instead of trying to define MPI_* types directly, it’d be better to use new type names.

#ifdef SEACAS_HAVE_MPI
typedef MPI_Comm Ioss_MPI_Comm;
#else
typedef … Ioss_MPI_Comm;
#endif
19

If you are using an mpi compiler, then you are doing a seacas parallel build. If you have a serial netCDF and HDF5, then IOSS will still enable parallel capabilities, but only using file-per-rank type of parallel execution and not parallel-IO (which requires parallel netCDF and HDF5).

I will grant that this mode is not tested / built very often, so there are perhaps issues that have snuck into the build, but it is possible to build IOSS and SEACAS parallel with serial netCDF and HDF5.

20

The symbol PARALLEL_AWARE_EXODUS is used when netCDF and HDF5 are compiled with parallel-IO capability.