Catalyst2: Artifacts on Processes Boundaries

[cartagena]

I am trying to visualize a particle simulation with in-situ Catalyst2 support, and while it is correct on single core (produced image is consistent with what ParaView creates), in multi-core setting I am getting artifacts on processes boundaries:

1-core Catalyst visualization:

2-core Catalyst visualization:

3-core Catalyst visualization:

2-core vs 1-core images difference:

4-core vs 1-core images difference:

Some details about simulation:

1. Data is list of particles, each particle has 3 properties - scalar, 3-dimensional vector, and 3x3 tensor (unrolled into 9 separate scalar properties in the Conduit node field setter). In the images, glyphs are oriented w.r.t. vector, colored w.r.t tensor[1][1], scaled w.r.t scalar.
2. Problem persists in the absence of ghost cells. I have verified that if 1-core gets 64K particles, then 2-core get 2x32K, 4-core get 4x16K.
3. I use the following Conduit Mesh Blueprint:
   coords/type = explicit
   mesh/type = unstructured
   mesh/elements/shape = point
   mesh/elements/connectivity = <index array from 0 to num_particles_local)
4. I have verified that in all 3 cases I pass correct coordinates and property values to Conduit nodes.

[utkarsh.ayachit]

Are you showing glyphs for all points in the dataset ? Default is to pick a random set limited by a max count, IIRC. That can yield different set of glyphs as number of ranks changes, but will still be representing the data correctly.

[cartagena]

Could you please elaborate on random set, how exactly is this tuned? What I did is the following:

1. Generate VTP for representative dataset with single core
2. Load VTP into ParaView, apply coloring and glyph filter, disable showing points, create PNG extractor
3. Save Catalyst state as Python script; change source to TrivialProducer(registration_name="particles")

So I guess I should be showing glyphs for all points

[Alexandre_Minot]

Utkarsh is pointing out that in your step 2, when you add the Glyph filter, under “masking” in the property pane, make sure you use the glyph mode “all points” and not “uniform spatial distribution”.

[cartagena]

Thanks, seems that random sampling was the problem, images are now consistent between single- and multi-core with “all points” mode. I guess there isn’t a mechanism to “synchronize” random sampling across processes in Catalyst (simulating working with .pvtp in ParaView)?