# Problem with MPI\_Comm in ParaView 5.10.0 RC1 when using External MPI version

**URL:** https://discourse.paraview.org/t/problem-with-mpi-comm-in-paraview-5-10-0-rc1-when-using-external-mpi-version/8222
**Category:** Development
**Created:** [October 19, 2021, 7:59pm UTC](https://discourse.paraview.org/t/problem-with-mpi-comm-in-paraview-5-10-0-rc1-when-using-external-mpi-version/8222 "2021-10-19T19:59:20Z")
**Posts on this page:** 20
**Page:** 1

<div class="post-metadata">

### Author: ![joeh](https://discourse.paraview.org/letter_avatar_proxy/v4/letter/j/ce7236/32.png) [@joeh](https://discourse.paraview.org/u/joeh)
#### Post date: [October 19, 2021, 7:59pm UTC](https://discourse.paraview.org/t/problem-with-mpi-comm-in-paraview-5-10-0-rc1-when-using-external-mpi-version/8222/1 "2021-10-19T19:59:20Z")

</div>

There is a new issue in ParaView 5.10.0-RC1 when using External MPI version that was not present when doing the same with ParaView 5.9.1 or below.

When building ParaView 5.1.0-RC1 with the superbuild script and utilizing an external MPI version.

I get an error from the file

Build\_directory/superbuild/paraview/src/VTK/ThirdParty/ioss/vtkioss/Ioss\_CodeTypes.h

on line 68 where the definition of MPI\_Comm

using MPI\_Comm = int;

conflicts with the external version of MPI\_Comm

using MPI\_Comm = unsigned int;

from mpi.h in the external version.

This conflict was not present in version 5.9.1 or below.

A simple workaround is to set the value to be an unsigned int in the file Ioss\_CodeTypes.h

This change will allow ParaView to build and function correctly but should not be necessary.

Thanks,

Joe Hennessey

---

<div class="post-metadata">

### Author: ![wascott](https://discourse.paraview.org/letter_avatar_proxy/v4/letter/w/8e8cbc/32.png) [@wascott](https://discourse.paraview.org/u/wascott)
#### Post date: [October 19, 2021, 8:03pm UTC](https://discourse.paraview.org/t/problem-with-mpi-comm-in-paraview-5-10-0-rc1-when-using-external-mpi-version/8222/2 "2021-10-19T20:03:54Z")

</div>

Hi Joe!  
I pointed the architect/ implementer of the IOSS library at this post. Ben from Kitware also.  
Alan

---

<div class="post-metadata">

### Author: ![Gregory\_Sjaardema](https://discourse.paraview.org/user_avatar/discourse.paraview.org/gregory_sjaardema/32/7744_2.png) [@Gregory\_Sjaardema](https://discourse.paraview.org/u/Gregory_Sjaardema)
#### Post date: [October 19, 2021, 8:12pm UTC](https://discourse.paraview.org/t/problem-with-mpi-comm-in-paraview-5-10-0-rc1-when-using-external-mpi-version/8222/3 "2021-10-19T20:12:08Z")

</div>

Does the external mpi that you are using define `MPI_COMM_WORLD` ? The line you are hitting shuold only be hit if the mpi being used does not define `MPI_COMM_WORLD`… It is inside a #if block:

```auto
#ifndef MPI_COMM_WORLD
#define MPI_COMM_WORLD 0
using MPI_Comm = int;
#endif

```

---

<div class="post-metadata">

### Author: ![joeh](https://discourse.paraview.org/letter_avatar_proxy/v4/letter/j/ce7236/32.png) [@joeh](https://discourse.paraview.org/u/joeh)
#### Post date: [October 19, 2021, 8:16pm UTC](https://discourse.paraview.org/t/problem-with-mpi-comm-in-paraview-5-10-0-rc1-when-using-external-mpi-version/8222/4 "2021-10-19T20:16:58Z")

</div>

Here is the start of the mpi.h file that I am using. As far as I can tell this would define MPI\_COMM\_WORLD in the enum below.

```auto
#ifndef MPI_H_INCLUDED
#define MPI_H_INCLUDED

#if defined(__cplusplus)
extern "C" {
#endif

/ ****************** /
/* MPI-1 bindings */
/ ****************** /

#define MPI_BOTTOM ((MPI_Aint)0)

typedef long MPI_Aint;
typedef unsigned int MPI_Request;
typedef unsigned int MPI_Group;
typedef unsigned int MPI_Comm;
typedef unsigned int MPI_Errhandler;
typedef int MPI_Fint;
typedef unsigned int MPI_Op;
typedef unsigned int MPI_Datatype;
typedef unsigned int MPI_Win;
typedef long long MPI_Count;

#define MPI_FILE_DEFINED
typedef struct ADIOI_FileD *MPI_File;

typedef struct {
        int MPI_SOURCE;
        int MPI_TAG;
        int MPI_ERROR;
        int reserved[1];
        long size;
} MPI_Status;

extern MPI_Status mpi_sgi_status_ignore;
#define MPI_STATUS_IGNORE (&mpi_sgi_status_ignore)
#define MPI_STATUSES_IGNORE (MPI_STATUS_IGNORE)

extern int mpi_sgi_inplace;
#define MPI_IN_PLACE ( (void*)&mpi_sgi_inplace)

typedef MPI_Status MPI_F08_status;

extern MPI_F08_status * MPI_F08_STATUS_IGNORE;
extern MPI_F08_status * MPI_F08_STATUSES_IGNORE;

enum {
        MPI_COMM_NULL = 0,
        MPI_COMM_WORLD = 1,
        MPI_COMM_SELF = 2,
        _MPI_SGI_COMM_LAST
};

```

---

<div class="post-metadata">

### Author: ![Gregory\_Sjaardema](https://discourse.paraview.org/user_avatar/discourse.paraview.org/gregory_sjaardema/32/7744_2.png) [@Gregory\_Sjaardema](https://discourse.paraview.org/u/Gregory_Sjaardema)
#### Post date: [October 19, 2021, 8:34pm UTC](https://discourse.paraview.org/t/problem-with-mpi-comm-in-paraview-5-10-0-rc1-when-using-external-mpi-version/8222/5 "2021-10-19T20:34:14Z")

</div>

[Ignore this comment – I think the issue is that this is a parallel build, but `SEACAS_HAVE_MPI` is not defined]

It “defines” MPI\_COMM\_WORLD, but it doesn’t `#define` MPI\_COMM\_WORLD. I was under the impression that `MPI_COMM_WORLD` was required to be in a `#define`, but obviously that is not the case here…

It looks like some other codes have had issues with this assumption also… ([Ambuiguous ADIOS constructors on HPE MPT MPI where MPI\_COMM\_WORLD is an integer · Issue #2893 · ornladios/ADIOS2 · GitHub](https://github.com/ornladios/ADIOS2/issues/2893)) and it looks like Microsoft also uses an enum for MPI\_COMM\_WORLD and not a `#define`. I will have to figure out a better way to handle the case that this was meant to solve…

For now, I would suggest removing the entire #if block for `MPI_COMM_WORLD` and `MPI_COMM_SELF`

---

<div class="post-metadata">

### Author: ![Gregory\_Sjaardema](https://discourse.paraview.org/user_avatar/discourse.paraview.org/gregory_sjaardema/32/7744_2.png) [@Gregory\_Sjaardema](https://discourse.paraview.org/u/Gregory_Sjaardema)
#### Post date: [October 19, 2021, 8:36pm UTC](https://discourse.paraview.org/t/problem-with-mpi-comm-in-paraview-5-10-0-rc1-when-using-external-mpi-version/8222/6 "2021-10-19T20:36:03Z")

</div>

[Ignore this comment – I think the issue is that this is a parallel build, but `SEACAS_HAVE_MPI` is not defined]  
I need to figure out why I needed this code in the first place since it is right after an `include <mpi.h>` which should provide what is needed…

---

<div class="post-metadata">

### Author: ![Gregory\_Sjaardema](https://discourse.paraview.org/user_avatar/discourse.paraview.org/gregory_sjaardema/32/7744_2.png) [@Gregory\_Sjaardema](https://discourse.paraview.org/u/Gregory_Sjaardema)
#### Post date: [October 19, 2021, 8:46pm UTC](https://discourse.paraview.org/t/problem-with-mpi-comm-in-paraview-5-10-0-rc1-when-using-external-mpi-version/8222/7 "2021-10-19T20:46:12Z")

</div>

Sorry for all the churn.

I think that maybe `SEACAS_HAVE_MPI` is not being defined which means that mpi.h is not being included as it should… This should maybe be a better name for this symbol, but needs to be defined if the IOSS library is being compiled using mpi…

---

<div class="post-metadata">

### Author: ![ben.boeckel](https://discourse.paraview.org/letter_avatar_proxy/v4/letter/b/ea5d25/32.png) [@ben.boeckel](https://discourse.paraview.org/u/ben.boeckel)
#### Post date: [October 19, 2021, 10:23pm UTC](https://discourse.paraview.org/t/problem-with-mpi-comm-in-paraview-5-10-0-rc1-when-using-external-mpi-version/8222/8 "2021-10-19T22:23:27Z")

</div>

AFAICT, `SEACAS_HAVE_MPI` gets into `SEACASIoss_config.h`. Is this header not being included somewhere it needs to be?

---

<div class="post-metadata">

### Author: ![Gregory\_Sjaardema](https://discourse.paraview.org/user_avatar/discourse.paraview.org/gregory_sjaardema/32/7744_2.png) [@Gregory\_Sjaardema](https://discourse.paraview.org/u/Gregory_Sjaardema)
#### Post date: [October 19, 2021, 10:45pm UTC](https://discourse.paraview.org/t/problem-with-mpi-comm-in-paraview-5-10-0-rc1-when-using-external-mpi-version/8222/9 "2021-10-19T22:45:39Z")

</div>

It should be included in `Ioss_CodeTypes.h` (the file we are looking at) a few lines above where the problem is showing up.

---

<div class="post-metadata">

### Author: ![ben.boeckel](https://discourse.paraview.org/letter_avatar_proxy/v4/letter/b/ea5d25/32.png) [@ben.boeckel](https://discourse.paraview.org/u/ben.boeckel)
#### Post date: [October 19, 2021, 11:00pm UTC](https://discourse.paraview.org/t/problem-with-mpi-comm-in-paraview-5-10-0-rc1-when-using-external-mpi-version/8222/10 "2021-10-19T23:00:24Z")

</div>

Is it defined in the config.h header on the machine? If not, then we need to look at why [this code](https://gitlab.kitware.com/vtk/vtk/-/blob/master/ThirdParty/ioss/vtkioss/CMakeLists.txt#L7) isn’t working as expected.

---

<div class="post-metadata">

### Author: ![ben.boeckel](https://discourse.paraview.org/letter_avatar_proxy/v4/letter/b/ea5d25/32.png) [@ben.boeckel](https://discourse.paraview.org/u/ben.boeckel)
#### Post date: [October 19, 2021, 11:01pm UTC](https://discourse.paraview.org/t/problem-with-mpi-comm-in-paraview-5-10-0-rc1-when-using-external-mpi-version/8222/11 "2021-10-19T23:01:37Z")

</div>

> [@Gregory\_Sjaardema](#):
>
> It should be included in `Ioss_CodeTypes.h` (the file we are looking at) a few lines above where the problem is showing up.

Hmm. Not in [our import](https://gitlab.kitware.com/vtk/vtk/-/blob/master/ThirdParty/ioss/vtkioss/Ioss_CodeTypes.h). Was this include added after where we are based upon?

Edit: Not seeing an include on `master` either.

---

<div class="post-metadata">

### Author: ![Gregory\_Sjaardema](https://discourse.paraview.org/user_avatar/discourse.paraview.org/gregory_sjaardema/32/7744_2.png) [@Gregory\_Sjaardema](https://discourse.paraview.org/u/Gregory_Sjaardema)
#### Post date: [October 19, 2021, 11:24pm UTC](https://discourse.paraview.org/t/problem-with-mpi-comm-in-paraview-5-10-0-rc1-when-using-external-mpi-version/8222/12 "2021-10-19T23:24:34Z")

</div>

Line 47

---

<div class="post-metadata">

### Author: ![ben.boeckel](https://discourse.paraview.org/letter_avatar_proxy/v4/letter/b/ea5d25/32.png) [@ben.boeckel](https://discourse.paraview.org/u/ben.boeckel)
#### Post date: [October 20, 2021, 11:18am UTC](https://discourse.paraview.org/t/problem-with-mpi-comm-in-paraview-5-10-0-rc1-when-using-external-mpi-version/8222/13 "2021-10-20T11:18:42Z")

</div>

Indeed. but that is _after_ the check for `SEACAS_HAVE_MPI` (I stopped looking once I got to the use), so that’s why it isn’t working.

Gah, too early in the morning. The `BUILT_IN_SIERRA` check is also setup, not use. Hrm.

---

<div class="post-metadata">

### Author: ![ben.boeckel](https://discourse.paraview.org/letter_avatar_proxy/v4/letter/b/ea5d25/32.png) [@ben.boeckel](https://discourse.paraview.org/u/ben.boeckel)
#### Post date: [October 20, 2021, 9:42pm UTC](https://discourse.paraview.org/t/problem-with-mpi-comm-in-paraview-5-10-0-rc1-when-using-external-mpi-version/8222/14 "2021-10-20T21:42:33Z")

</div>

@joeh Could you please provide the `SEACASIoss_config.h` header that is around? Should be in the build tree (or install tree).

---

<div class="post-metadata">

### Author: ![joeh](https://discourse.paraview.org/letter_avatar_proxy/v4/letter/j/ce7236/32.png) [@joeh](https://discourse.paraview.org/u/joeh)
#### Post date: [October 21, 2021, 12:30am UTC](https://discourse.paraview.org/t/problem-with-mpi-comm-in-paraview-5-10-0-rc1-when-using-external-mpi-version/8222/15 "2021-10-21T00:30:24Z")

</div>

Ben,

Here are the contents of the file  
Build\_directory/superbuild/paraview/build/VTK/ThirdParty/ioss/vtkioss/SEACASIoss\_config.h

```auto
/* #undef SEACAS_HAVE_MPI */

/* #undef IOSS_THREADSAFE */

/* Define to indicate that the SEACAS project is being built with the KOKKOS package enabled*/
/* #undef SEACAS_HAVE_KOKKOS */

/* #undef SEACAS_HAVE_FAODEL */

/* #undef SEACAS_HAVE_DATAWARP */

#define SEACAS_HAVE_EXODUS

#define SEACAS_HAVE_CGNS

/* #undef SEACAS_HAVE_PAMGEN */

/* #undef HAVE_SEACASIOSS_ADIOS2 */

```

---

<div class="post-metadata">

### Author: ![Gregory\_Sjaardema](https://discourse.paraview.org/user_avatar/discourse.paraview.org/gregory_sjaardema/32/7744_2.png) [@Gregory\_Sjaardema](https://discourse.paraview.org/u/Gregory_Sjaardema)
#### Post date: [October 21, 2021, 12:34am UTC](https://discourse.paraview.org/t/problem-with-mpi-comm-in-paraview-5-10-0-rc1-when-using-external-mpi-version/8222/16 "2021-10-21T00:34:02Z")

</div>

So `SEACAS_HAVE_MPI` is not set correctly (assuming this is a parallel build). In the CMakeLists.txt that creates the SEACASIoss\_config.h, the setting is controlled by `TPL_ENABLE_MPI` which should be defined if you have a parallel build…

---

<div class="post-metadata">

### Author: ![joeh](https://discourse.paraview.org/letter_avatar_proxy/v4/letter/j/ce7236/32.png) [@joeh](https://discourse.paraview.org/u/joeh)
#### Post date: [October 21, 2021, 12:34am UTC](https://discourse.paraview.org/t/problem-with-mpi-comm-in-paraview-5-10-0-rc1-when-using-external-mpi-version/8222/17 "2021-10-21T00:34:54Z")

</div>

Gregory,

It is a parallel build.

Thanks,

Joe

---

<div class="post-metadata">

### Author: ![ben.boeckel](https://discourse.paraview.org/letter_avatar_proxy/v4/letter/b/ea5d25/32.png) [@ben.boeckel](https://discourse.paraview.org/u/ben.boeckel)
#### Post date: [October 21, 2021, 11:26am UTC](https://discourse.paraview.org/t/problem-with-mpi-comm-in-paraview-5-10-0-rc1-when-using-external-mpi-version/8222/18 "2021-10-21T11:26:27Z")

</div>

Note that what SEACAS thinks is a parallel build may not match with what VTK thinks. Here, we have a non-parallel netcdf (and probably hdf5), so we cannot enable Ioss MPI support. Unfortunately, it seems that MPI is included elsewhere in an Ioss-using TU and the MPI compat stuff gets confused. I think that, instead of trying to define `MPI_*` types directly, it’d be better to use new type names.

```auto
#ifdef SEACAS_HAVE_MPI
typedef MPI_Comm Ioss_MPI_Comm;
#else
typedef … Ioss_MPI_Comm;
#endif

```

---

<div class="post-metadata">

### Author: ![Gregory\_Sjaardema](https://discourse.paraview.org/user_avatar/discourse.paraview.org/gregory_sjaardema/32/7744_2.png) [@Gregory\_Sjaardema](https://discourse.paraview.org/u/Gregory_Sjaardema)
#### Post date: [October 21, 2021, 1:54pm UTC](https://discourse.paraview.org/t/problem-with-mpi-comm-in-paraview-5-10-0-rc1-when-using-external-mpi-version/8222/19 "2021-10-21T13:54:41Z")

</div>

If you are using an mpi compiler, then you are doing a seacas parallel build. If you have a serial netCDF and HDF5, then IOSS will still enable parallel capabilities, but only using file-per-rank type of parallel execution and not parallel-IO (which requires parallel netCDF and HDF5).

I will grant that this mode is not tested / built very often, so there are perhaps issues that have snuck into the build, but it is possible to build IOSS and SEACAS parallel with serial netCDF and HDF5.

---

<div class="post-metadata">

### Author: ![Gregory\_Sjaardema](https://discourse.paraview.org/user_avatar/discourse.paraview.org/gregory_sjaardema/32/7744_2.png) [@Gregory\_Sjaardema](https://discourse.paraview.org/u/Gregory_Sjaardema)
#### Post date: [October 21, 2021, 1:57pm UTC](https://discourse.paraview.org/t/problem-with-mpi-comm-in-paraview-5-10-0-rc1-when-using-external-mpi-version/8222/20 "2021-10-21T13:57:20Z")

</div>

The symbol `PARALLEL_AWARE_EXODUS` is used when netCDF and HDF5 are compiled with parallel-IO capability.

[Next page](https://discourse.paraview.org/t/problem-with-mpi-comm-in-paraview-5-10-0-rc1-when-using-external-mpi-version/8222.md?page=2)
