Adding Properties for vtkPartitionedDataSetCollection

Here’s a revised proposal, which includes some changes to vtkDataAssembly itself. Just to clarify, primarily talking about VTK-level here. So no notion of representations + active state etc.

  1. Make vtkDataAssembly a standard DOM. This will add some rules to the supported types of node-names e.g. they cannot contain spaces and will need to be a valid XML element name. This is slightly different than how it’s currently implemented. That was done to support node names with spaces. In hindsight, that’s unnecessary and if needed, we can use the now supported “attributes” to add “label” which can include spaces etc.
  2. Add support for attributes to the nodes in the assembly. Nothing special here, standard with DOMs. Allows for things like “label”, or “class” in future. Exodus Assemblies also support attributes per assembly node and this provides a mechanism to add those to vtkDataAssembly.
  3. Create a new mapper, say vtkDataAssemblyPolyDataMapper, which subclasses vtkMapper. It’ll support the standard API to get/set root-level properties for appearance. These values together with the values setup on the vtkProperty and vtkActor define the visual properties that get used with the mapper starts rendering.
  4. vtkDataAssemblyPolyDataMapper takes a CSS string that can be used to override the visual properties. The implementation will scan through the assembly, determining computed visual properties combing the defaults with the overrides in the CSS and then render accordingly.
  5. CSS selectors can support attributes specified on the DataAssembly, besides the node names.
  6. The set of properties supported by the CSS can grow overtime, we start with a simple set.
  7. In ParaView GUI, when we use the Multiblock Inspector to setup properties, we’ll be generating a CSS.
  8. This concept can then be extended to other mappers, e.g. vtkDataAssemblyGlyphMapper. vtkDataAssemblyMoleculeMapper etc.
2 Likes

@utkarsh.ayachit That sounds great!

I like it, especially if we can eventually have something like a vtkDispatchingMapper that chooses an underlying mapping approach based on CSS-like markup (e.g., vtkPolyData marked as glyphs, so use the glyph mapper).

I don’t know of C++ CSS parser, but PEGTL has been really good to use and is already in VTK as a TPL.

I agree that the whole deal is too much to do at once.

@utkarsh.ayachit It is not an implementation, but here is a PEGTL issue discussing CSS parsing.

sigh! searching the universe for a reasonable CSS parsing library has been quite futile. I don’t want to write my own parser using PEGTL or whatever, but seems like the only path forward :/…disappointing.

@ben.boeckel to the rescue; libcss seems like a decent place to start. parsing the docs now; :crossed_fingers:

@utkarsh.ayachit Any idea on when vtkDataAssemblyPolyDataMapper will come to life? And will your first pass include transforms for partitions inside the collection? We are starting to move SMTK to use vtkPartitionedDataSetCollection and that is one of the features we like about the proposed mapper!

@vbolea is working on it…I’ll defer to him on the timeline. Priority-wise, it no longer is a requirement to get done for 5.10 since I think we can support a temporary workaround in ParaView. PartitionedDataSetCOllection support can still be implemented with a small hack to the old multiblock mapper hence the change in priorities. But if there’s scope for joining efforts, let’s talk!

Sure! @vbolea, I’m happy to help plan, sketch out tests/examples, review, or implement some things — especially support for transforms.

@dcthomp @utkarsh.ayachit

I know I am late to the discussion but I wish to add some ideas.

I really like these ideas of using CSS to query the DOM, this gives us a lot of flexibility without reinventing the wheel.

With regards to libcss, a deeper research shows that it has strong dependencies of other components of the project NetSurf (LibParserUtils, LibWapcaplel ). This means that for ParaView to add this dependencies we will also need to add another dependencies and its corresponding compatible versioning combinations which will add additional complexity to our building system.

Another CSS parser which seems popular in GitHub is : https://github.com/hackers-painters/katana-parser. It does not have external dependencies and it seems to do the work, unfortunately it has not been active for a few years.

In any case these concern are just a matter of finding the appropiate library

There is some work done in the latest push in which I introduce the vtkDataAssemblyRepresentation where we can set properties using XPath ids, unfortunately I have not given it as much time as I wanted since in the last month there has been a mix of vacations and other urgent tasks.

Before I move forward I would like to take into consideration the selection of the CSS library previously stated.

Agreed, libcss is a non-starter esp with all the dependencies it needs. Katana sounds reasonable enough. Will need to be cmakified and added as TPL, but the code seems compact enough that I doubt it’ll be too difficult. Let go ahead with Katana.

@utkarsh.ayachit I like the idea, I will fork it in our gitlab to “cmakefy” it and add it in the paraview codebase as a thirdparty subtree

That can be done as the last step (and it’ll need to be added to VTK). I probably is totally fine to continue development on the mapper simply using an external build of the Katana. That’s probably better since it will help us identify issues with the library sooner – in case, it turns out not suitable for our use.

1 Like

Hi all, any progress on vtkDataAssemblyPolyDataMapper and CSS? It is becoming a priority for aeva/cmb/smtk.

No, we don’t have it on our active task list right now since we have higher priority tasks.

2 years later… this is back on our priority with some changes. I’ll write that up separately. All the points discussed here are useful and will potentially make it into the new mapper.

1 Like

The customer wants to map different scalar arrays per partition in the partitioned dataset. This is definitely possible because all partitions in a partitioned dataset are expected to have the same field arrays. It’s only a matter of selecting the ‘active’ scalar array when mapper builds the VAO or texture buffers. Citing DataAssembly doc:

All non-null partitions have similar field and attribute arrays. For example, if a vtkPartitionedDataSet comprises of vtkDataSet subclasses, all will have exactly the same number of point data/cell data arrays, with same names, same number of components, and same data types.

Before addressing that, I want to bring to light that there is confusion in VTK for mappers related to the composite dataset, there are two of them!

  1. vtkCompositePolyDataMapper: is a part of RenderingCore, it is meant to render vtkCompositeDataset instances using a single actor. Internally, it creates a unique vtkPolyDataMapper for each dataset and invokes mapper->Render(ren, act). The datasets are uploaded one by one. This mapper does NOT use vtkCompositeDataDisplayAttributes. This mapper relies on SetScalarArray and uses that array to map colors for all blocks without any hesitation.
  2. vtkCompositePolyDataMapper2: is a part of RenderingOpenGL2. It has a LOT of code. This mapper can set per-block visual properties. It also uploads all datasets in one shot instead of using one mapper per block. This is currently used by ParaView. This mapper relies on SetScalarArray and uses it for all blocks without any hesitation.

I want to ask if the vtkCompositePolyDataMapper sees any use at all. Or is that technical debt? We can refactor these two mappers so that there is just one vtkCompositePolyDataMapper that supports per-dataset display attributes, color mapping with different scalar arrays, and uploads all datasets in one go. The concrete vtkCompositePolyDataMapper in RenderingOpenGL will simply implement the actual upload and render functionality.

The vtkPartitionedDataSetCollectionMapper(renamed from vtkDataAssemblyPolyDataMapper) could sit in RenderingCore, reusing vtkCompositePolyDataMapper internally along with CSS styled selectors for display attributes.

FYI, @cory.quammen

AFAIK, vtkCompositePolyDataMapper can be thought of as technical debt. I can’t think of any use-cases where one can’t replace it with vtkCompositePolyDataMapper2 instead.

2 Likes

CPDM2 will be deprecated because it’s perf and functionality is now available from CPDM. Related post on VTK discourse - Deprecating vtkCompositePolyDataMapper2 in favor of vtkCompositePolyDataMapper - Development - VTK

1 Like