# Adding Properties for vtkPartitionedDataSetCollection

**URL:** https://discourse.paraview.org/t/adding-properties-for-vtkpartitioneddatasetcollection/6061
**Category:** Development
**Tags:** proposal
**Created:** [December 15, 2020, 1:14am UTC](https://discourse.paraview.org/t/adding-properties-for-vtkpartitioneddatasetcollection/6061 "2020-12-15T01:14:24Z")
**Posts on this page:** 20
**Page:** 1

<div class="post-metadata">

### Author: ![vbolea](https://discourse.paraview.org/user_avatar/discourse.paraview.org/vbolea/32/4121_2.png) [@vbolea](https://discourse.paraview.org/u/vbolea)
#### Post date: [December 15, 2020, 1:14am UTC](https://discourse.paraview.org/t/adding-properties-for-vtkpartitioneddatasetcollection/6061/1 "2020-12-15T01:14:24Z")

</div>

As discussed in [this topic](https://discourse.paraview.org/t/deprecating-multiblock-datasets/4950/3), we will finally departure from `multiblock` datasets by introducing the `vtkPartitionedDataSetCollection` and friends.

An important feature needed for those hierarchical datasets are properties, such as `visibility` or `opacity`, which makes uses of the its hierarchy —similar to `vtkCompositeDataDisplayAttributes.h` for multiblock datasets.

To enable this feature we added the missing `vtkPartitionedDataSetCollectionMapper` which takes into consideration those new introduced properties and made several changes to the `vtkDataAssembly` class.

## Example usage

Here is an example of API usage the new proposed feature.

```auto
  // We want to hide the elements under //left/r2 in this example
  vtkNew<vtkDataAssembly> assembly;
  const auto base = assembly->AddNodes({ "left", "right" });
  const auto right = assembly->AddNodes({ "r1", "r2" }, base[1]);

  assembly->AddDataSetIndices(base[0], { 0, 1 });
  assembly->AddDataSetIndices(right[0], { 2 });
  assembly->AddDataSetIndices(right[1], { 3 });

  // Every dataset contained in the //left/r2 subtree would inherit
  // those properties when rendering. 
  assembly->SetProperty(right[1], Visibility, "false");
 
  auto ret = assembly->GetProperty(right[1], Visibility);

  assembly->UnSetProperty(right[1], Visibility);

  for (auto prop : assembly->GetProperties(right[1]))
  {
    std::cout << auto ret = assembly->GetProperty(right[1], prop) << std::endl;
  }

  // Similar methods have been introduced for the class vtkDataAssemblyVisitor.

```

## Discussion

This work came with few design decisions, some of which I wish to bring to debate in this thread.

1. In this work we set properties in the hierarchy using the class `vtkDataAssembly`. This effectively groups all the storing logic into `vtkDataAssembly` at the expense of adding more responsibilities to the class.  
Alternatively, this API could be moved to the new introduced mapper, making thismapper in charge of storing the properties and also taking decisions based on them – such as disabling or changing the opacity of an entire subtree.

2. Following the design decision of implementing the properties features in the `vtkDataAssembly`. We currently store the properties into the vtkDataAssembly XML DOM, using `pugiXML` library. I wonder if there are any scenarios in which this can be an inconvenience or a potential bottleneck.

Please feel free to add any comment or suggestion as this is still work in progress.

## Related links

- MR: [https://gitlab.kitware.com/vtk/vtk/-/merge\_requests/7426/diffs#c9372a45daa2147a5771e6c91641f57a8d0f93ac](https://gitlab.kitware.com/vtk/vtk/-/merge_requests/7426/diffs#c9372a45daa2147a5771e6c91641f57a8d0f93ac)

---

<div class="post-metadata">

### Author: ![vbolea](https://discourse.paraview.org/user_avatar/discourse.paraview.org/vbolea/32/4121_2.png) [@vbolea](https://discourse.paraview.org/u/vbolea)
#### Post date: [December 15, 2020, 1:14am UTC](https://discourse.paraview.org/t/adding-properties-for-vtkpartitioneddatasetcollection/6061/2 "2020-12-15T01:14:46Z")

</div>

@utkarsh.ayachit

---

<div class="post-metadata">

### Author: ![utkarsh.ayachit](https://discourse.paraview.org/user_avatar/discourse.paraview.org/utkarsh.ayachit/32/39_2.png) [@utkarsh.ayachit](https://discourse.paraview.org/u/utkarsh.ayachit)
#### Post date: [December 15, 2020, 2:17pm UTC](https://discourse.paraview.org/t/adding-properties-for-vtkpartitioneddatasetcollection/6061/3 "2020-12-15T14:17:59Z")

</div>

A few comments:

- This approach of assigning properties directly on the vtkDataAssembly is better than assigning those on to the mapper (as was done previously with CompositePolyDataMapper[2]). This will avoid the need for client applications to maintain any separate datastructures to preserve them if the assembly changes.
- We need to provide convenience API on vtkDataAssembly (or some utility class) to set properties using path expressions.
- We probably need more VTK-esque way of storing properties. One option is to use the `vtkInformation` and `vtkInformationKey...` mechanism. This makes it possible for client-code to define arbitrary new properties and storing them in a type-safe way. The questions with this approacg: is can that be made efficient performance and memory-usage wise, and can that be serialized/serialized easily.

---

<div class="post-metadata">

### Author: ![dcthomp](https://discourse.paraview.org/user_avatar/discourse.paraview.org/dcthomp/32/20_2.png) [@dcthomp](https://discourse.paraview.org/u/dcthomp)
#### Post date: [January 1, 2021, 8:24pm UTC](https://discourse.paraview.org/t/adding-properties-for-vtkpartitioneddatasetcollection/6061/4 "2021-01-01T20:24:16Z")

</div>

1. Storing visual style on the vtkDataAssembly means it will be the same across multiple views. At least some of our customers do not want that.
2. We have another application ([aeva](https://aeva.readthedocs.io/)) for which customers want not only control over opacity and block color, but also the representation style (surface, surface-with-edges, wireframe, etc.). We are also looking at migrating to `vtkPartitionedDataSetCollection` and `vtkDataAssembly`.
3. Have you considered something more like CSS that separates the style from the data?

---

<div class="post-metadata">

### Author: ![utkarsh.ayachit](https://discourse.paraview.org/user_avatar/discourse.paraview.org/utkarsh.ayachit/32/39_2.png) [@utkarsh.ayachit](https://discourse.paraview.org/u/utkarsh.ayachit)
#### Post date: [January 2, 2021, 1:55pm UTC](https://discourse.paraview.org/t/adding-properties-for-vtkpartitioneddatasetcollection/6061/5 "2021-01-02T13:55:13Z")

</div>

> [@dcthomp](#):
>
> Have you considered something more like CSS that separates the style from the data?

Interesting thought. Can you elaborate on how that might look?

---

<div class="post-metadata">

### Author: ![dcthomp](https://discourse.paraview.org/user_avatar/discourse.paraview.org/dcthomp/32/20_2.png) [@dcthomp](https://discourse.paraview.org/u/dcthomp)
#### Post date: [January 2, 2021, 4:58pm UTC](https://discourse.paraview.org/t/adding-properties-for-vtkpartitioneddatasetcollection/6061/6 "2021-01-02T16:58:06Z")

</div>

Just have vtkDataObject and vtkDataSetAttributes (or vtkFieldData?) hold a set of classes. The representation for a view would hold style strings that determine visual style based on the set classes, just the way [CSS rulesets](https://developer.mozilla.org/en-US/docs/Web/CSS/Syntax) map groups of HTML elements to property values in declaration blocks.

The equivalent of HTML elements in [CSS selectors](https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Selectors) would be VTK class names. So some markup might look like

```css
/* Declare some default style for vtkPolyData: */
polydata { line-thickness: 1px; opacity: 0.5; }
/* ... and override polydata with "selected" in its list of classes: */
polydata.selected { line-thickness: 2px; opacity: 1.0; }
/* Allow point-data markups to specify scalar array coloring: */
polydata pointdata { map-scalars: true, colormap: cool-to-warm; }
/* Allow representations to be chosen by selector */
image: { representation: outline; }
image.active { representation: surface; }

```

The vtkDataAssembly [path queries](https://vtk.org/doc/nightly/html/classvtkDataAssembly.html#DataAssemblyPathQueries) are intentionally XPath-like, but there are [mappings](https://github.com/svenheden/xpath-to-css) [between](https://css2xpath.github.io/) XPath and CSS selectors.

You can also imagine some [pseudo-classes](https://developer.mozilla.org/en-US/docs/Web/CSS/Pseudo-classes) would be useful for things like changing appearance upon mouse hovering:

```css
polydata::hover {
  line-thickness: 2px;
  line-color: #774c77;
  poly-color: #c37dc3;
}

```

Whenever a representation’s CSS style string changes, the representation would parse it to discover the set of unique combinations of visual styles possible.

Whenever the input data changes, the representation would traverse the vtkDataAssembly, evaluate what style applies to each leaf node, and develop a rendering plan that decomposes rendering into a sequence (or sequences if rendering in parallel) of simple operations.

---

<div class="post-metadata">

### Author: ![dcthomp](https://discourse.paraview.org/user_avatar/discourse.paraview.org/dcthomp/32/20_2.png) [@dcthomp](https://discourse.paraview.org/u/dcthomp)
#### Post date: [January 2, 2021, 5:41pm UTC](https://discourse.paraview.org/t/adding-properties-for-vtkpartitioneddatasetcollection/6061/7 "2021-01-02T17:41:56Z")

</div>

To map @vbolea’s example into CSS would look something like this:

```auto
vtkNew<vtkDataAssembly> assembly; // populated as above.
vtkNew<vtkDataAssemblyRepresentation> representation;
representation->SetInputDataObject(assembly);
representation->SetStyle(R"(
  node[name='right'] node[name='r2'] { visibility: false; }
)");
vtkNew<vtkView> view;
view->AddRepresentation(representation);

```

although some less wordy version of the selector group might make sense:

```auto
representation->SetStyle(R"(
  node-xpath[/right/r2] { visibility: false; }
)");

```

---

<div class="post-metadata">

### Author: ![utkarsh.ayachit](https://discourse.paraview.org/user_avatar/discourse.paraview.org/utkarsh.ayachit/32/39_2.png) [@utkarsh.ayachit](https://discourse.paraview.org/u/utkarsh.ayachit)
#### Post date: [January 4, 2021, 4:43pm UTC](https://discourse.paraview.org/t/adding-properties-for-vtkpartitioneddatasetcollection/6061/8 "2021-01-04T16:43:30Z")

</div>

Let me start by saying, I really like this approach!

Doing it all the way as you’re suggesting with pseudo-classes etc is very cool, but will probably be too much to chew right off the bat.

Does anyone have any experience with CSS + C++? I’ve been googling for css parser libraries etc. Don’t think I can spot what the canonical way in my quick searches.

---

<div class="post-metadata">

### Author: ![utkarsh.ayachit](https://discourse.paraview.org/user_avatar/discourse.paraview.org/utkarsh.ayachit/32/39_2.png) [@utkarsh.ayachit](https://discourse.paraview.org/u/utkarsh.ayachit)
#### Post date: [January 4, 2021, 5:48pm UTC](https://discourse.paraview.org/t/adding-properties-for-vtkpartitioneddatasetcollection/6061/9 "2021-01-04T17:48:17Z")

</div>

Here’s a revised proposal, which includes some changes to vtkDataAssembly itself. Just to clarify, primarily talking about VTK-level here. So no notion of representations + active state etc.

1. Make `vtkDataAssembly` a standard DOM. This will add some rules to the supported types of node-names e.g. they cannot contain spaces and will need to be a valid XML element name. This is slightly different than how it’s currently implemented. That was done to support node names with spaces. In hindsight, that’s unnecessary and if needed, we can use the now supported “attributes” to add “label” which can include spaces etc.
2. Add support for attributes to the nodes in the assembly. Nothing special here, standard with DOMs. Allows for things like “label”, or “class” in future. [Exodus Assemblies](https://github.com/gsjaardema/seacas/wiki/Assemblies#motivation) also support attributes per assembly node and this provides a mechanism to add those to `vtkDataAssembly`.
3. Create a new mapper, say `vtkDataAssemblyPolyDataMapper`, which subclasses [`vtkMapper`](https://vtk.org/doc/nightly/html/classvtkMapper.html). It’ll support the standard API to get/set root-level properties for appearance. These values together with the values setup on the `vtkProperty` and `vtkActor` define the visual properties that get used with the mapper starts rendering.
4. `vtkDataAssemblyPolyDataMapper` takes a CSS string that can be used to override the visual properties. The implementation will scan through the assembly, determining computed visual properties combing the defaults with the overrides in the CSS and then render accordingly.
5. CSS selectors can support attributes specified on the DataAssembly, besides the node names.
6. The set of properties supported by the CSS can grow overtime, we start with a simple set.
7. In ParaView GUI, when we use the **Multiblock Inspector** to setup properties, we’ll be generating a CSS.
8. This concept can then be extended to other mappers, e.g. `vtkDataAssemblyGlyphMapper`. `vtkDataAssemblyMoleculeMapper` etc.

---

<div class="post-metadata">

### Author: ![dcthomp](https://discourse.paraview.org/user_avatar/discourse.paraview.org/dcthomp/32/20_2.png) [@dcthomp](https://discourse.paraview.org/u/dcthomp)
#### Post date: [January 5, 2021, 4:58pm UTC](https://discourse.paraview.org/t/adding-properties-for-vtkpartitioneddatasetcollection/6061/10 "2021-01-05T16:58:01Z")

</div>

@utkarsh.ayachit That sounds great!

> [@utkarsh.ayachit](#):
>
> 1. Create a new mapper, say `vtkDataAssemblyPolyDataMapper`

I like it, especially if we can _eventually_ have something like a `vtkDispatchingMapper` that chooses an underlying mapping approach based on CSS-like markup (e.g., vtkPolyData marked as glyphs, so use the glyph mapper).

---

<div class="post-metadata">

### Author: ![dcthomp](https://discourse.paraview.org/user_avatar/discourse.paraview.org/dcthomp/32/20_2.png) [@dcthomp](https://discourse.paraview.org/u/dcthomp)
#### Post date: [January 5, 2021, 5:00pm UTC](https://discourse.paraview.org/t/adding-properties-for-vtkpartitioneddatasetcollection/6061/11 "2021-01-05T17:00:31Z")

</div>

> [@utkarsh.ayachit](#):
>
> Does anyone have any experience with CSS + C++? I’ve been googling for css parser libraries etc. Don’t think I can spot what the canonical way in my quick searches.

I don’t know of C++ CSS parser, but PEGTL has been really good to use and is already [in VTK as a TPL](https://gitlab.kitware.com/vtk/vtk/-/tree/master/ThirdParty/pegtl).

I agree that the whole deal is too much to do at once.

---

<div class="post-metadata">

### Author: ![dcthomp](https://discourse.paraview.org/user_avatar/discourse.paraview.org/dcthomp/32/20_2.png) [@dcthomp](https://discourse.paraview.org/u/dcthomp)
#### Post date: [January 5, 2021, 5:04pm UTC](https://discourse.paraview.org/t/adding-properties-for-vtkpartitioneddatasetcollection/6061/12 "2021-01-05T17:04:22Z")

</div>

@utkarsh.ayachit It is not an implementation, but [here](https://github.com/taocpp/PEGTL/discussions/184) is a PEGTL issue discussing CSS parsing.

---

<div class="post-metadata">

### Author: ![utkarsh.ayachit](https://discourse.paraview.org/user_avatar/discourse.paraview.org/utkarsh.ayachit/32/39_2.png) [@utkarsh.ayachit](https://discourse.paraview.org/u/utkarsh.ayachit)
#### Post date: [January 5, 2021, 6:43pm UTC](https://discourse.paraview.org/t/adding-properties-for-vtkpartitioneddatasetcollection/6061/13 "2021-01-05T18:43:35Z")

</div>

sigh! searching the universe for a reasonable CSS parsing library has been quite futile. I don’t want to write my own parser using PEGTL or whatever, but seems like the only path forward :/…disappointing.

---

<div class="post-metadata">

### Author: ![utkarsh.ayachit](https://discourse.paraview.org/user_avatar/discourse.paraview.org/utkarsh.ayachit/32/39_2.png) [@utkarsh.ayachit](https://discourse.paraview.org/u/utkarsh.ayachit)
#### Post date: [January 5, 2021, 7:16pm UTC](https://discourse.paraview.org/t/adding-properties-for-vtkpartitioneddatasetcollection/6061/14 "2021-01-05T19:16:25Z")

</div>

@ben.boeckel to the rescue; [libcss](https://source.netsurf-browser.org/libcss.git/) seems like a decent place to start. parsing the docs now; 🤞

---

<div class="post-metadata">

### Author: ![dcthomp](https://discourse.paraview.org/user_avatar/discourse.paraview.org/dcthomp/32/20_2.png) [@dcthomp](https://discourse.paraview.org/u/dcthomp)
#### Post date: [January 26, 2021, 6:00pm UTC](https://discourse.paraview.org/t/adding-properties-for-vtkpartitioneddatasetcollection/6061/15 "2021-01-26T18:00:02Z")

</div>

@utkarsh.ayachit Any idea on when `vtkDataAssemblyPolyDataMapper` will come to life? And will your first pass include transforms for partitions inside the collection? We are starting to move SMTK to use vtkPartitionedDataSetCollection and that is one of the features we like about the proposed mapper!

---

<div class="post-metadata">

### Author: ![utkarsh.ayachit](https://discourse.paraview.org/user_avatar/discourse.paraview.org/utkarsh.ayachit/32/39_2.png) [@utkarsh.ayachit](https://discourse.paraview.org/u/utkarsh.ayachit)
#### Post date: [January 26, 2021, 6:16pm UTC](https://discourse.paraview.org/t/adding-properties-for-vtkpartitioneddatasetcollection/6061/16 "2021-01-26T18:16:02Z")

</div>

@vbolea is working on it…I’ll defer to him on the timeline. Priority-wise, it no longer is a requirement to get done for 5.10 since I think we can support a temporary workaround in ParaView. PartitionedDataSetCOllection support can still be implemented with a small hack to the old multiblock mapper hence the change in priorities. But if there’s scope for joining efforts, let’s talk!

---

<div class="post-metadata">

### Author: ![dcthomp](https://discourse.paraview.org/user_avatar/discourse.paraview.org/dcthomp/32/20_2.png) [@dcthomp](https://discourse.paraview.org/u/dcthomp)
#### Post date: [January 26, 2021, 9:17pm UTC](https://discourse.paraview.org/t/adding-properties-for-vtkpartitioneddatasetcollection/6061/17 "2021-01-26T21:17:51Z")

</div>

Sure! @vbolea, I’m happy to help plan, sketch out tests/examples, review, or implement some things — especially support for transforms.

---

<div class="post-metadata">

### Author: ![vbolea](https://discourse.paraview.org/user_avatar/discourse.paraview.org/vbolea/32/4121_2.png) [@vbolea](https://discourse.paraview.org/u/vbolea)
#### Post date: [January 27, 2021, 1:37pm UTC](https://discourse.paraview.org/t/adding-properties-for-vtkpartitioneddatasetcollection/6061/18 "2021-01-27T13:37:48Z")

</div>

@dcthomp @utkarsh.ayachit

I know I am late to the discussion but I wish to add some ideas.

I really like these ideas of using CSS to query the DOM, this gives us a lot of flexibility without reinventing the wheel.

With regards to `libcss`, a deeper research shows that it has strong dependencies of other components of the project NetSurf (LibParserUtils, LibWapcaplel ). This means that for ParaView to add this dependencies we will also need to add another dependencies and its corresponding compatible versioning combinations which will add additional complexity to our building system.

Another CSS parser which seems popular in GitHub is : [https://github.com/hackers-painters/katana-parser](https://github.com/hackers-painters/katana-parser). It does not have external dependencies and it seems to do the work, unfortunately it has not been active for a few years.

In any case these concern are just a matter of finding the appropiate library

There is some work done in the latest push in which I introduce the `vtkDataAssemblyRepresentation` where we can set properties using XPath ids, unfortunately I have not given it as much time as I wanted since in the last month there has been a mix of vacations and other urgent tasks.

Before I move forward I would like to take into consideration the selection of the CSS library previously stated.

---

<div class="post-metadata">

### Author: ![utkarsh.ayachit](https://discourse.paraview.org/user_avatar/discourse.paraview.org/utkarsh.ayachit/32/39_2.png) [@utkarsh.ayachit](https://discourse.paraview.org/u/utkarsh.ayachit)
#### Post date: [January 27, 2021, 3:00pm UTC](https://discourse.paraview.org/t/adding-properties-for-vtkpartitioneddatasetcollection/6061/19 "2021-01-27T15:00:41Z")

</div>

Agreed, `libcss` is a non-starter esp with all the dependencies it needs. Katana sounds reasonable enough. Will need to be cmakified and added as TPL, but the code seems compact enough that I doubt it’ll be too difficult. Let go ahead with Katana.

---

<div class="post-metadata">

### Author: ![vbolea](https://discourse.paraview.org/user_avatar/discourse.paraview.org/vbolea/32/4121_2.png) [@vbolea](https://discourse.paraview.org/u/vbolea)
#### Post date: [January 27, 2021, 3:19pm UTC](https://discourse.paraview.org/t/adding-properties-for-vtkpartitioneddatasetcollection/6061/20 "2021-01-27T15:19:35Z")

</div>

@utkarsh.ayachit I like the idea, I will fork it in our gitlab to “cmakefy” it and add it in the paraview codebase as a thirdparty subtree

[Next page](https://discourse.paraview.org/t/adding-properties-for-vtkpartitioneddatasetcollection/6061.md?page=2)