Deprecating MultiBlock datasets

The parts of ParaView that work with backwards compatibility that could potentially be affected are:

  1. Catalyst adaptors
  2. State files
  3. Python scripts
  4. Programmable filters (and later editions)

These are things that users work with regularly that may no longer be backwards compatible that I’m concerned about, especially since people have invested time into making these things work.

I do like the idea of improving on the multiblock dataset as there are quite a bit of issues with using it. Starting over may be what’s needed.

Indeed. I suspect it will be mix of automatic conversion and migration docs to handle backwards compatibility. By making sure this effort pays off dividends in added features and simplicity, we can hopefully ameliorate the pain.

I worry about backward compatibility for custom plugins as well, as I am extensively employing the vtkMultiBlockDataSetAlgorithm and the MultiBlock inspector in my own custom ParaView plugins (C++). It would be very convenient, after MultiBlocks get deprecated, to have clear instructions on how to switch to the new classes (the mentioned vtkPartitionedDataSet, vtkPartitionedDataSetCollection or vtkDataAssembly). If this could be provided then the transition might not be so painful then, otherwise, those who heavily use the MultiBlockDataSets, including myself, might remain “stuck” at the older ParaView versions for some time.

Note, there are filters that ParaView will use internally to convert MutliblockDataSet to the new classes for at least another version or so. So in first pass, your plugin should be not affected at all.

Indeed, an extensive guide for transitioning is a requirement for this work.

1 Like

This looks very good, great job.

Is there a reason why a vtkPartitionedDataSetCollection cannot have more than one vtkDataAssembly attached to it?

I’m thinking it would be very useful if users are allowed to have multiple representations of the same data without the need to make multiple vtkPartitionedDataSetCollection instances. It would also give users a quick way to know how many assemblies a particular vtkPartitionedDataSetCollection has and quick way to access them.

To compliment this, we can allow users to add annotations on the assemblies in order to provide some kind of description or purpose for an assembly and so that searching for a particular assembly is easy. Also, since we will have multiple assemblies we should be able to set one of them to be the default assembly. Doing that will allow the vtkPartitionedDataSetCollection to behave as if it has only one assembly in use cases where users don’t care about the other assemblies that the data has. Essentially it will behave like it it would currently.

In summary the changes would look something like this:

vtkDataAssembly

// provide a way to add anotations to assemblies 
void SetAnnotation	(const char * key, const char * value)	

vtkPartitionedDataSetCollection

// allow users to add more than one data assembly
void AddDataAssembly(vtkDataAssembly *assembly) 

// setDataAssembly would become setDefaultDataAssembly
void SetDefaultDataAssembly(vtkDataAssembly *assembly)

// GetDataAssembly would become GetDefaultDataAssembly
vtkDataAssembly* GetDefaultDataAssembly ()

// Provide a way to grab certain assemblies using a map of annotation key value pairs
// passing in an empty map can indicate that the user wants all the assemblies
std::set<vtkDataAssembly*> GetDataAssemblies(const std::map<std::string, std::string>& annotations, bool match_all)

Is there a reason why a vtkPartitionedDataSetCollection cannot have more than one vtkDataAssembly attached to it?

That had indeed crossed my mind. No particular reason, honestly. Figured we could add support for it when we have a compelling use-case. It does add a bit of complexity to the UI.

Another question is whether the ‘default-assembly’ is a parameter on the vtkPartitionedDataSetCollection (as you have done in the API) or on a filter – similar to how vtkAlgorithm::SetInputArrayToProcess lets user choose which array to operate on, we could have SetInputDataAssemblyToProcess to choose which assembly to operate on. Or it could be a combination of both: i.e. choose default on the PDC and then let filters override which they operate on.

I don’t know how compelling my use-case is but I do see this addition as a way for the VTK Session in SMTK to take greater advantage of the Graph Resource. Conceptually the Graph Resource is very similar to vtkDataAssembly, the only difference is that it has the option to hold data local to it as well as reference data from an external dataset.

Model Resource currently acts like the SMTK wrapper for vtkMultiBlockDataSet (in the VTK session atleast) therefore the data and assembly of the data are coupled. However, when vtkMultiBlockDataSet gets deprecated we can have Model Resource (or some new type of Resource) act as the SMTK wrapper for vtkPartitionedDataSetCollection while Graph Resource acts as the SMTK wrapper for vtkDataAssembly.

Allowing multiple assemblies will then become very useful because in SMTK we are dealing with CAE workflows. A lot of workflows require users to take a single set of input data and move it through workflow phases that change its assembly in different ways. For example, we build Assembly 1 and present it to the user as their new representation, then collect some more settings and build Assembly 2. Sometimes the user just wants to view the input data using a different Assembly structure without necessarily moving it through any workflow phase. An example of that that would be building Assembly 1 and Assembly 2 at the same time.

In any case, I think allowing all the different assemblies to be connected to each other (through the vtkPartitionedDataSetCollection that they assemble) would simplify a lot of complexity on the SMTK side of things. It would reduce the need to copy input data and make getting from one assembly to another very easy.

Maybe in the future we can even rectify the differences between Graph Resource and vtkDataAssembly by allowing vtkDataAssemblies to ‘override’ or provide a local copy of certain vtkPartitionDataSets in the vtkPartitionedDataSetCollection that it is set on. In addition, the vtkDataAssembly can provide vtkPartitionDataSets that are meant to add on to or augment the linked vtkParitionedDataSetCollection when the particular vtkDataAssembly is used.

I like this approach a lot. It introduces a lot of flexibility.

1 Like