# BUG RectilinearGrid pvtr data with ghost cells not displayed correctly

**URL:** https://discourse.paraview.org/t/bug-rectilineargrid-pvtr-data-with-ghost-cells-not-displayed-correctly/1566
**Category:** ParaView Support
**Created:** [March 30, 2019, 8:35am UTC](https://discourse.paraview.org/t/bug-rectilineargrid-pvtr-data-with-ghost-cells-not-displayed-correctly/1566 "2019-03-30T08:35:59Z")
**Posts on this page:** 11
**Page:** 1

<div class="post-metadata">

### Author: ![bastian](https://discourse.paraview.org/user_avatar/discourse.paraview.org/bastian/32/7513_2.png) [@bastian](https://discourse.paraview.org/u/bastian)
#### Post date: [March 30, 2019, 8:35am UTC](https://discourse.paraview.org/t/bug-rectilineargrid-pvtr-data-with-ghost-cells-not-displayed-correctly/1566/1 "2019-03-30T08:35:59Z")

</div>

I am passing data on a rectilinear grid to Catalyst for in-situ visualization.  
I set up the grid in the function `createcpdata_` in the attached file [FECxx.cxx](https://discourse.paraview.org/uploads/default/original/2X/5/55519ed0821eec9de585e8fbe453b17a2ee8236a.cxx) (4.2 KB)

I have the python-coprocessor script currently output the data using the `XMLPRectilinearGridWriter` (see the attached file [coproc.py](https://discourse.paraview.org/uploads/default/original/2X/a/abe824e3bf0249dc520b47dc6ce19268ba9b4572.py) (2.8 KB)).  
This generates a main `.pvtr` file not holding any data but referring to the multiple `.vtr` files in a subfolder which are generated by the different MPI ranks.

For debugging purpose I am processing a small dataset using two processes, and 7×6×5 cells in total. The first process gets 5×6×5 cells, the second one gets 4×6×5 cells. These overlap by two layers in the x-direction, since the cells of each process contain one layer of ghost cells on all six sides.

When loading the main `.pvtr` file I am confronted with missing cells as can by seen in the following slice:  
 ![Screenshot%20from%202019-03-29%2019-55-26](https://discourse.paraview.org/uploads/default/original/2X/b/bbe30c13f0bfe494b7d78fbf1b14cca7227820f9.png)  
The blueish cells belong to rank0. I expect to see 3×3 cells, that’s fine. For some reason, the cells written by rank1 are not entirely visible. Here the first layer of cells normal to the x-axis is lacking.

When loading the two corresponding `.vtr` files individually into Paraview, then I get a seamless representation with only the ghost cells being hidden. In the following visualization I sliced the data of the two `.vtr` files at slightly different levels of y:  
 ![Screenshot%20from%202019-04-02%2009-49-00](https://discourse.paraview.org/uploads/default/original/2X/2/247dc6466b012bbad7d0bf77c07c9c2714631d6c.png)

Here is the data (`.pvtr` and `.vtr` files): [data.zip](https://discourse.paraview.org/uploads/default/original/2X/7/702e9d4848bb3c6909f9a8ccb11ab65175e3fad0.zip) (1.7 KB)

**Should I open an issue somewhere else?**  
**Is there a workaround?**

I can reproduce this behavior in Paraview 5.5.2 and 5.6.0

---

<div class="post-metadata">

### Author: ![bastian](https://discourse.paraview.org/user_avatar/discourse.paraview.org/bastian/32/7513_2.png) [@bastian](https://discourse.paraview.org/u/bastian)
#### Post date: [April 2, 2019, 11:12am UTC](https://discourse.paraview.org/t/bug-rectilineargrid-pvtr-data-with-ghost-cells-not-displayed-correctly/1566/3 "2019-04-02T11:12:14Z")

</div>

I noticed that no information about ghost cells is written to any output `.pvtr`/`.vtr` file but the very first one that my coprocessor creates. Maybe this is gives a clue to anyone.

I also wondered if maybe my declaration of ghostcells at the six sides of the global simulation domain is incompatible with Paraviews’ concept of ghost cells, since there are no overlapping subdomains in these regions. So I excluded these cells from being marked as ghost cells, but the issue remains.

`.pvtr` file on the left, the two `.vtr` files on the right.

 ![Screenshot%20from%202019-04-02%2015-59-41](https://discourse.paraview.org/uploads/default/original/2X/2/2ab077c52abce84611061c89c24c03ff2f9d9d59.png) ![Screenshot%20from%202019-04-02%2015-59-58](https://discourse.paraview.org/uploads/default/original/2X/f/fdf2eb07b361ab48f6943723fcddb6c97a742ad5.png)

[data\_2.zip](https://discourse.paraview.org/uploads/default/original/2X/6/6c77b713ae3ebbbac8896df9df7910c399ff4662.zip) (1.6 KB)

---

<div class="post-metadata">

### Author: ![bastian](https://discourse.paraview.org/user_avatar/discourse.paraview.org/bastian/32/7513_2.png) [@bastian](https://discourse.paraview.org/u/bastian)
#### Post date: [April 3, 2019, 8:07am UTC](https://discourse.paraview.org/t/bug-rectilineargrid-pvtr-data-with-ghost-cells-not-displayed-correctly/1566/4 "2019-04-03T08:07:43Z")

</div>

No idea? I guess I’ll open an issue over at gitlab then?

---

<div class="post-metadata">

### Author: ![bastian](https://discourse.paraview.org/user_avatar/discourse.paraview.org/bastian/32/7513_2.png) [@bastian](https://discourse.paraview.org/u/bastian)
#### Post date: [April 3, 2019, 9:21am UTC](https://discourse.paraview.org/t/bug-rectilineargrid-pvtr-data-with-ghost-cells-not-displayed-correctly/1566/5 "2019-04-03T09:21:56Z")

</div>

I’m having another look at my `pvtr`-file:

```auto
<VTKFile type="PRectilinearGrid" version="1.0" byte_order="LittleEndian" header_type="UInt64">
  <PRectilinearGrid WholeExtent="0 7 0 6 0 5" GhostLevel="0">
    <PCellData>
      <PDataArray type="UInt8" Name="vtkGhostType"/>
      <PDataArray type="Float64" Name="pressure"/>
    </PCellData>
    <PCoordinates>
      <PDataArray type="Float64"/>
      <PDataArray type="Float64"/>
      <PDataArray type="Float64"/>
    </PCoordinates>
    <Piece Extent="0 5 0 6 0 5" Source="pressure_21/pressure_21_0.vtr"/>
    <Piece Extent="3 7 0 6 0 5" Source="pressure_21/pressure_21_1.vtr"/>
  </PRectilinearGrid>
</VTKFile>

```

Noticing `GhostLevel="0"` … shouldn’t the ghost level be 1?

 ![190403_sketch_ghost_cell_topology](https://discourse.paraview.org/uploads/default/original/2X/9/96c06798e3e207d29e7bb01e294162073533008b.png)

Slide 26 of [this presentation](http://calcul.math.cnrs.fr/IMG/pdf/jeanfavre_parallelvisualization.pdf) had me test a few modifications:

**1. Changed `GhostLevel="0"` to `GhostLevel="1"` or `GhostLevel="2"`**  
No impact:  
 ![Screenshot%20from%202019-04-03%2011-00-22](https://discourse.paraview.org/uploads/default/original/2X/e/e4d643b28b76111145dc4779b308a4feb6eab00b.png)

**2. Changed the extent of the first piece from `Extent="0 5 0 6 0 5"` to `Extent="0 4 0 6 0 5"`**  
This fixes the representation regardless of what value `GhostLevel` is set to.  
 ![Screenshot%20from%202019-04-03%2011-06-56](https://discourse.paraview.org/uploads/default/original/2X/b/b188b0b7aca36f046c59b14c3e954917c045c32d.png)

**But why is that so?**  
In this official [Kitware presentation](https://www.osti.gov/servlets/purl/1116421) I am taught on slide 66 that an extent denotes the indices of the points limiting the cells. Changing the maximum extent of the first rank’s part of the grid from 5 to 4 contradicts this concept. _(Do also not get confused that on slide 67 of the presentation three columns should be painted blue instead of two)_

**This can’t possibly be the solution, …**  
because if I reduce the extent this way where the grid is defined in the catalyst adaptor function, I won’t have the right number of cells.

---

<div class="post-metadata">

### Author: ![danlipsa](https://discourse.paraview.org/user_avatar/discourse.paraview.org/danlipsa/32/1563_2.png) [@danlipsa](https://discourse.paraview.org/u/danlipsa)
#### Post date: [June 6, 2019, 5:38pm UTC](https://discourse.paraview.org/t/bug-rectilineargrid-pvtr-data-with-ghost-cells-not-displayed-correctly/1566/6 "2019-06-06T17:38:30Z")

</div>

Bastian,  
The Kitware presentation you referred to is old, here is a link to a description of how ghost cells are stored currently.

> **[Ghost and Blanking (Visibility) Changes - Kitware Blog](https://blog.kitware.com/ghost-and-blanking-visibility-changes/)**
>
> Motivation Ghost cells are layers of cells at boundaries of pieces of a dataset. These cells are used by data parallel algorithms, to insure the ... Read More

  
Your file already uses that format.

It seems that the reason why you are missing cells is that those cells are marked as ghosts:

 ![ghost](https://discourse.paraview.org/uploads/default/original/2X/e/e0a16b4e67f6c2086470abcce0accd3bbb8210ef.png)

This picture shows a slice through your data twice. The first time I used the data you send and got the same slice. The second time, I modified your data by renaming vtkGhostType to vtkGhostTypes so that it is not interpreted by ParaView, and then extracted only the cells that have vtkGhostTypes 1.  
Those are red. So cells are missing because are marked as ghosts.

In the attachment I included the two datasets and a ParaView state file that produces this image.

[ghost.zip](https://discourse.paraview.org/uploads/default/original/2X/e/edde15192869fe0378eab21b31f4f032aaa87c61.zip) (40.3 KB)

---

<div class="post-metadata">

### Author: ![bastian](https://discourse.paraview.org/user_avatar/discourse.paraview.org/bastian/32/7513_2.png) [@bastian](https://discourse.paraview.org/u/bastian)
#### Post date: [June 21, 2019, 3:56pm UTC](https://discourse.paraview.org/t/bug-rectilineargrid-pvtr-data-with-ghost-cells-not-displayed-correctly/1566/7 "2019-06-21T15:56:34Z")

</div>

Dear Dan,

> [@danlipsa](#):
>
> So cells are missing because are marked as ghosts.

I still fail to understand why that should be so. The missing cells are marked as ghosts in `pressure_20_1.vtr`, true, but they are not in `pressure_20_0.vtr`. When viewing the combination of the two, there should not be a gap.

Here are screenshots where I visualize the “GhostTypes” fields you made available in the ghost.zip-file:

`pressure_20_0.vtr` on the left and `pressure_20_1.vtr` to the right:

 ![pressure_20_0_GhostCellType](https://discourse.paraview.org/uploads/default/original/2X/d/dba12029a657b764fbb7c95100cd4b024865b223.png) ![pressure_20_1_GhostCellType](https://discourse.paraview.org/uploads/default/original/2X/d/d625cc643e4ec78028a51c1db4bde4858c83409c.png)

And here is the combination of the two:

 ![pressure_20_both_GhostCellType](https://discourse.paraview.org/uploads/default/original/2X/e/ee24ba50d1d31b6e81216d4d3a730a3fda2b529e.png)

As you can see, there is no gap.  
In the position where a gap is present when viewing `pressure_20.pvtr` I would expect data from `pressure_20_1.vtr` (right) to be displayed.

We may have diverging ideas of the concept of ghost cells though.

---

<div class="post-metadata">

### Author: ![danlipsa](https://discourse.paraview.org/user_avatar/discourse.paraview.org/danlipsa/32/1563_2.png) [@danlipsa](https://discourse.paraview.org/u/danlipsa)
#### Post date: [May 14, 2020, 4:58pm UTC](https://discourse.paraview.org/t/bug-rectilineargrid-pvtr-data-with-ghost-cells-not-displayed-correctly/1566/8 "2020-05-14T16:58:25Z")

</div>

Bastian,  
You are right. This seems to be a bug in how pvtr reads two pieces on one node. I also tried to read this data on two nodes but that did not seem to work correctly either.  
Here is a picture for reading the data on one node (left) as well as reading the individual pieces (right top and right bottom)

 ![ghost](https://discourse.paraview.org/uploads/default/original/2X/6/656dcc484dd69c79938f4f5a1ce1400b32c6e369.png)

Here is the picture for reading the data on two nodes.

 ![ghost-parallel](https://discourse.paraview.org/uploads/default/original/2X/5/50d3459b49c46d66c54a4ed0488f69a27292a3c9.png)

[ghost.zip](https://discourse.paraview.org/uploads/short-url/apEgB1TuMoWibjVY1vqRo0Xmxc7.zip) (3.9 MB)

---

<div class="post-metadata">

### Author: ![danlipsa](https://discourse.paraview.org/user_avatar/discourse.paraview.org/danlipsa/32/1563_2.png) [@danlipsa](https://discourse.paraview.org/u/danlipsa)
#### Post date: [May 14, 2020, 5:34pm UTC](https://discourse.paraview.org/t/bug-rectilineargrid-pvtr-data-with-ghost-cells-not-displayed-correctly/1566/9 "2020-05-14T17:34:45Z")

</div>

Here is the bug report I filed for this  
[https://gitlab.kitware.com/paraview/paraview/-/issues/19927](https://gitlab.kitware.com/paraview/paraview/-/issues/19927)

---

<div class="post-metadata">

### Author: ![bastian](https://discourse.paraview.org/user_avatar/discourse.paraview.org/bastian/32/7513_2.png) [@bastian](https://discourse.paraview.org/u/bastian)
#### Post date: [May 14, 2020, 7:18pm UTC](https://discourse.paraview.org/t/bug-rectilineargrid-pvtr-data-with-ghost-cells-not-displayed-correctly/1566/10 "2020-05-14T19:18:17Z")

</div>

Thank you Dan!

---

<div class="post-metadata">

### Author: ![mwestphal](https://discourse.paraview.org/user_avatar/discourse.paraview.org/mwestphal/32/17_2.png) [@mwestphal](https://discourse.paraview.org/u/mwestphal)
#### Post date: [May 15, 2020, 1:41am UTC](https://discourse.paraview.org/t/bug-rectilineargrid-pvtr-data-with-ghost-cells-not-displayed-correctly/1566/11 "2020-05-15T01:41:29Z")

</div>

@bastian : didn’t we had the same discussion one year ago ? [Declaration of ghost cells for rectilinear grid](https://discourse.paraview.org/t/declaration-of-ghost-cells-for-rectilinear-grid/1585)

---

<div class="post-metadata">

### Author: ![bastian](https://discourse.paraview.org/user_avatar/discourse.paraview.org/bastian/32/7513_2.png) [@bastian](https://discourse.paraview.org/u/bastian)
#### Post date: [May 16, 2020, 12:04pm UTC](https://discourse.paraview.org/t/bug-rectilineargrid-pvtr-data-with-ghost-cells-not-displayed-correctly/1566/12 "2020-05-16T12:04:13Z")

</div>

@mwestphal : you’re right; I don’t remember why I ended up opening a new discussion; it is indeed confusing, sorry for that
