Which information from ghosting cells/points is actually used? Do we need to faithfully reproduce all of the characteristics of the ghost cell, or can we cut corners (heavily) and provide a simplified replacement cell?
With a fully hex mesh hex that is split into two domains, the cells of each of side of the processor boundary are obviously also hexes. For defining that processor boundaries are internal to the mesh, I could also add some ghost cells. But suppose we want to be really lazy (efficient) and not send about all these hex cells and their extra points, but something much more compact. For example, the cell-centre value and perhaps some characteristic length. On the receiving side it should then be possible to insert a new ghost cell with an additional single point as its apex. For the hex mesh, these “popup” ghost cells would be pyramids.
I am guessing that this approach might be enough to provide a simple means of tagging that the processor boundary is actually internal. But is any of the ghost cell geometry used for gradients etc?
Of course, the overhead with transmitting several hexes across processors is not likely to be too horrible, but the bookkeeping of the point numbering does become rather a large headache. When I start getting rather complex polyhedra that I need to send around, it gets even worse.
I would appreciate any thoughts (and warnings) about handling ghosting.
This can get complex quite fast, especially when considering trying to handle multiple levels of ghost cells. Also, many filters use ghost information quite differently so a simplification for one filter wouldn’t work for another and this information would have to be passe through the pipeline (e.g. ghost level one requires simplification A while ghost level two requires simplification B, which probably just means that you could only do simplification B on the second ghost level).
I thought about going down the route of just saying what types of ghost adjancies would be requested at a level (e.g. face ghosts which are only needed for determining the exterior surface vs. edge ghosts which I can’t think of a good example off the top of my head vs. point ghosts which are needed for the point data to cell data filter) but even that’s quite a bit of work.
I’d be happy to give feedback but I fear that the conclusion will be that this is going to be too much of a change and will also break compatibility with VisIt.
Thanks for the quick response. To clarify things, I’m not wishing to change anything within VTK handling. Just looking for a really simple way to fake halo cells (changed the title accordingly) with minimal bookkeeping, data transmission and conversion. So if I had a hex, but represented it as in the halo as a pyramid, the base face connectivity would be correct, and I’d have a cell value associated with it. Of course, this cell value would probably be associated with the wrong volume etc, but that’s what I’m wonder about. How far can I manage with such a cheap solution?
I can’t think of any individual filters right now that would fail in this scenario for values in the non-ghost cells. Assuming that you’d provide at most one level of ghost cells, once a user adds a second filter that requires a level of ghost cells in order to provide partition independent results it would automatically make the output somewhat suspect regardless of your ghost cell approximation. So given this I think this wouldn’t make anything worse off but I would document the heck out of it.