Overview
I’m working on some CFD code based on the AMReX library. It provides utilities to map a uniform Catersian AMR mesh to a conduit::Node object based on the nestsets blueprint. However, when the data is processed by paraview catalyst, this->GetRefinementRatio in vtkOverlappingAMRMetaData::CalculateParentChildRelationShip sometimes returns invalid refinement ratios like -2026166424 or 506225616 on some MPI ranks, leading to hanging and crash of the program.
Reproduce
To reproduce the issue, I build a minimal test case including a 64x16x16 grid. There are 12 block domains on 2 levels of refinement (4 on level 0 and 8 on level 1) distributed over 4 MPI procs. The AMR generated blueprint meshes are written to 4 yaml files for debugging (see attachments). I checked these with files with my eyes but found no problem.
Strangely, if the total grid dimension is increased, the problem is gone. Run the executable with only one process has the same effect.
bp_mesh_4.0.yaml (76.5 KB)
bp_mesh_4.1.yaml (81.2 KB)
bp_mesh_4.2.yaml (117.7 KB)
bp_mesh_4.3.yaml (110.5 KB)
Build Information
I use the Spack package manager to organize all my dependencies. The spec is
11 ~ │ - amrex@73b6cea70705ab07a4dc8e26ee8449cb12b371b9=26.05 +mpi~ascent+conduit+hdf5~fft~linear_solvers~sycl~shared+plotfile_tools+tiny_profil
│ e build_type=Release
12 │ - conduit@ca56731fbfe71c199ae7a5e5db71868eef15d47d=0.9.6 ~fortran+mpi+hdf5+python
13 │ - libcatalyst@2.1.0 +mpi+python
14 │ - paraview@=6.1.0 build_edition=catalyst_rendering +mpi+tbb~qt~x+python+libcatalyst+hdf5+osmesa_fallback
I managed to build them mannually but the problem still exists. It may or maynot be relavant to paraview. Any help is appreciated.