Declaration of ghost cells for rectilinear grid


(Bastian) #1

I looked at just about any Catalyst example I could find that demonstrates the handling of ghost cells for (though I couldn’t find any for rectilinear grids), and still I can’t get it to function.

Could anyone help me out here, I just can’t see what I am doing wrong.

I have a Fortran code where call createcpdata(gis,gie,gjs,gje,gks,gke,nxb,nyb,nzb,xgrid,ygrid,zgrid) inside FEFortranAdaptor.F90 (1.8 KB)
has the grid defined.

createcpdata is defined in FECxx.cxx (4.3 KB).
This function contains a section where ghost cells are marked:

    // Mark ghost cells
    vtkNew<vtkUnsignedCharArray> ghostCells;
    ghostCells->SetNumberOfTuples(grid->GetNumberOfCells());
    ghostCells->SetName(vtkDataSetAttributes::GhostArrayName());
    ghostCells->Fill(0);
    grid->GetCellData()->AddArray(ghostCells);
    
    vtkIdType cellId = 0;
    for (int k = *ks; k < *ke+1; k++)
    {
      bool zGhosts = ((k == *ks) || (k == *ke)) && (k > 0) && (k < *nz-1);
      for (int j = *js; j < *je+1; j++)
      {
        bool yGhosts = ((j == *js) || (j == *je)) && (j > 0) && (j < *ny-1);
        for (int i = *is; i < *ie+1; i++)
        {
          bool xGhosts = ((i == *is) || (i == *ie)) && (i > 0) && (i < *nx-1);
          if (xGhosts || yGhosts || zGhosts)
          {
            cout << "ghost: (" << i << ", " << j << ", " << k << ")\n";
            ghostCells->SetValue(cellId, ghostCells->GetValue(cellId) | vtkDataSetAttributes::DUPLICATECELL); 
          }
          cellId++;
        } 
      }   
    }

When running in serial, there are no ghost cells and the line ghostCells->SetValue(...) is never reached.
I can do live visualization and have the coprocessor script write .pvtr/.vtr files which I can load in Paraview without any issue.

When running in parallel cells are marked as ghostcells in the above section. I verified that the right cells are marked. (Subdomains overlap by two layers of cells due to one layer of ghost cells.)
When doing live visualization Paraview crashes due to a Segmentation fault.
I can still have a set of .pvtr/.vtr files written and load the .pvtr file in Paraview. However, when slicing the field data I am confronted with a gap shown in the screenshot on the left. One x-normal layer of data is not displayed.
When loading the two .vtr files individually, this gap is not present (right).

Another mystery:
Ghost cells are no longer declared as such in any succeeding time step, i.e. where the .pvtr file read

    <PCellData>
      <PDataArray type="UInt8" Name="vtkGhostType"/>
      <PDataArray type="Float64" Name="pressure"/>
    </PCellData>

the very first time it was produced, it then reads

    <PCellData>
      <PDataArray type="Float64" Name="pressure"/>
    </PCellData>

in all the succeeding time steps.


Strange values when adding cell data to grid
(Bastian) #2

If it is not possible to have rectilinear grids with ghost cells in Catalyst please let me know.