I looked at just about any Catalyst example I could find that demonstrates the handling of ghost cells for (though I couldn’t find any for rectilinear grids), and still I can’t get it to function.
Could anyone help me out here, I just can’t see what I am doing wrong.
I have a Fortran code where call createcpdata(gis,gie,gjs,gje,gks,gke,nxb,nyb,nzb,xgrid,ygrid,zgrid) inside FEFortranAdaptor.F90 (1.8 KB)
has the grid defined.
createcpdata is defined in FECxx.cxx (4.3 KB).
This function contains a section where ghost cells are marked:
// Mark ghost cells
vtkNew<vtkUnsignedCharArray> ghostCells;
ghostCells->SetNumberOfTuples(grid->GetNumberOfCells());
ghostCells->SetName(vtkDataSetAttributes::GhostArrayName());
ghostCells->Fill(0);
grid->GetCellData()->AddArray(ghostCells);
vtkIdType cellId = 0;
for (int k = *ks; k < *ke+1; k++)
{
bool zGhosts = ((k == *ks) || (k == *ke)) && (k > 0) && (k < *nz-1);
for (int j = *js; j < *je+1; j++)
{
bool yGhosts = ((j == *js) || (j == *je)) && (j > 0) && (j < *ny-1);
for (int i = *is; i < *ie+1; i++)
{
bool xGhosts = ((i == *is) || (i == *ie)) && (i > 0) && (i < *nx-1);
if (xGhosts || yGhosts || zGhosts)
{
cout << "ghost: (" << i << ", " << j << ", " << k << ")\n";
ghostCells->SetValue(cellId, ghostCells->GetValue(cellId) | vtkDataSetAttributes::DUPLICATECELL);
}
cellId++;
}
}
}
When running in serial, there are no ghost cells and the line ghostCells->SetValue(...) is never reached.
I can do live visualization and have the coprocessor script write .pvtr/.vtr files which I can load in Paraview without any issue.
When running in parallel cells are marked as ghostcells in the above section. I verified that the right cells are marked. (Subdomains overlap by two layers of cells due to one layer of ghost cells.)
When doing live visualization Paraview crashes due to a Segmentation fault.
I can still have a set of .pvtr/.vtr files written and load the .pvtr file in Paraview. However, when slicing the field data I am confronted with a gap shown in the screenshot on the left. One x-normal layer of data is not displayed.
When loading the two .vtr files individually, this gap is not present (right).
Another mystery:
Ghost cells are no longer declared as such in any succeeding time step, i.e. where the .pvtr file read
<PCellData>
<PDataArray type="UInt8" Name="vtkGhostType"/>
<PDataArray type="Float64" Name="pressure"/>
</PCellData>
the very first time it was produced, it then reads
<PCellData>
<PDataArray type="Float64" Name="pressure"/>
</PCellData>
in all the succeeding time steps.
