We dropped support for Catalyst editions with the CMake modularization changes introduced in ParaView 5.7 with intention that we’ll add support for a more robust, maintainable, and easy-to-use mechanism with similar goals.
Here’s an initial proposal to get the discussion started.
Classifying CMake options
Looking at all PARAVIEW_* options currently available, we can broadly classify them as follows:
Features: These are additive components that can be added to the ParaView build e.g. PARAVIEW_ENABLE_GDAL, PARAVIEW_ENABLE_FFMPEG, etc.
Plugins: These are options that enable/disable plugins e.g. PARAVIEW_PLUGIN_ENABLE_GMVReader, etc.
Capabilities: These are options that broadly affect the build potentially impacting all the enabled features, plugins, etc. These include options such as PARAVIEW_USE_MPI, PARAVIEW_ENABLE_PYTHON etc.
Builds options: These are options that broadly impact the build e.g. PARAVIEW_BUILD_SHARED_LIBS.
Let’s first standardize option-naming scheme based on the category they belong to.
Category
Option prefix
Examples
Features
PARAVIEW_ENABLE_
PARAVIEW_ENABLE_FFMPEGPARAVIEW_ENABLE_ADIOS2
Plugins
PARAVIEW_PLUGIN_ENABLE_
PARAVIEW_PLUGIN_ENABLE_GMVReader
Capabilities
PARAVIEW_USE_
PARAVIEW_USE_MPI, PARAVIEW_USE_PYTHON
Build options
PARAVIEW_BUILD_
PARAVIEW_BUILD_SHARED_LIBS
If we adopt this convention, then some of the long-used options will need to be changed. For example, PARAVIEW_ENABLE_PYTHON becomes PARAVIEW_USE_PYTHON. We will generate deprecation warning messages for such variable name changes to avoid confusion as users migrate to using these new variables.
Essentials (formerly called Catalyst Editions)
To support building more compact versions of ParaView (or Catalyst libraries) , we will provide a build option that can be supplied on the command line before the first CMake run called PARAVIEW_BUILD_ESSENTIALS_ONLY (open to suggestions on this one). If provided and set to ON or TRUE, this will have the following effect:
All plugin and feature options will default to OFF. This ensures that the build does not have additional features unless explicitly requested.
Several new features may become available e.g. PARAVIEW_ESSENTIALS_ENABLE_RENDERING, PARAVIEW_ESSENTIALS_ENABLE_IO. These options help further qualify what gets included in the essential set. Think of these as features that are not optional unless building Essentials. These options are hidden and enabled by default if PARAVIEW_BUILD_ESSENTIALS_ONLY is not provided or OFF.
Here, instead of multiple Catalyst editions, users build the essentials needed for ParaView/Catalyst together with explicitly chosen features.
This is simply a proposal to get the discussion going. Comments, other suggestions, criticisms are welcome. The plan is to have the agreed-upon proposal implemented within the next week or so for inclusion in the ParaView 5.8 release planned for early next year.
It’s great to hear that you’re pushing to get the Catalyst editions back into v5.8!
I know not too many users will be interested in what was previously called “configuring their own Catalyst edition” but every once in a while that was incredibly useful. One edition example for v5.6 that I needed is at https://github.com/acbauer/ParaViewParticleTrackingCatalystEdition to see what was needed for one simulation code I’m working with.
I would think removing all of the readers but having an option to keep the XML readers for restarts would be good. Removing all of the time dependent filters would also be good since at this point the only filter that can really do any in situ computations is the vtkInSituPParticlePathFilter filter, and even that I think takes special work to take advantage of. If the Intel Optane/temporal in situ work becomes mature then this will likely change though.
I’m sure I’ll have more thoughts on this as things proceed but overall this looks good to me so far.
It is conceivable that we provide ParticleTracking as a standard feature that can be enabled/disabled through an option e.g. PARAVIEW_ENABLE_PARTICLE_TRACKING. This feature could then enable VTK modules necessary to support the same.
Indeed. I suspect PARAVIEW_ESSENTIALS_ENABLE_IO will have dependent options that will help further qualify the readers/writers etc.
I have an essentials-only build of ParaView without rendering or optional-io, and with Python enabled and built static. The binary ends up being ~62M. Compare this to full ParaView (~250M) and Catayst-Base-Python edition from ParaView 5.6 (~33M). Note the Catalyst-base edition didn’t have any filters at all.
Implementation for these changes is available here. Here are some changes made to the original proposal as I implementing this.
Essentials (formerly called Catalyst Editions)
The ParaView build system (which is based on VTK’s) comprises of modules. Modules like VTK::CommonCore, VTK::CommonDataModel, ParaView::RemotingServerManager etc. are core modules that are necessary for ParaView. While modules like VTK::IOTRUCHAS, ParaView::RemotingViews are optional modules and can be turned off if one is not interested in the features they provide. Strictly speaking one can build variants of ParaView by manually turning on (or off) modules that are needed (or not needed) for a particular build. However, there are 100s of modules and toggling them individually can be quite tedious and confusing. To make this more manageable, we have a new CMake setting PARAVIEW_BUILD_ESSENTIALS.
PARAVIEW_BUILD_ESSENTIALS is a new CMake settings that can be used to control which modules are enabled by default. Currently supported values are CORE, RENDERING and CANONICAL. Based on the value set, the build will enable appropriate by default as needed. One can then enable additional modules to customize the build. For example. if you wanted to a build with only the core functionality and the exodus IO, you’d set PARAVIEW_BUILD_ESSENTIALS to CORE and then set VTK_MODULE_ENABLE_ParaView_VTKExtensionsIOExodus to YES or WANT.
Setting PARAVIEW_BUILD_ESSENTIALS to CORE only enables the modules needed for core ParaView, while RENDERING adds views and representations to the CORE set. CANONICAL is the default ParaView build.
Note that all the proxy definition XMLs have been split so that a proxy definition gets added only if the appropriate module is enabled. Thus, in a build variant you can query proxy definition manager to figure out whether the necessary capability is supported by the build.
All pending MRs, please make sure you rebase your topics on latest master to avoid any merge issues and make it easier on the dashboards before triggering Do: test.
As a consequence of this change, module names and targets in ParaView have changed. I’ll be updating the docs next. Essentially, if you get errors about missing ParaView::.. targets, it’s because it has moved/renamed. Best approach is to remove that target from your cmake file and then recompile. You should then get errors for missing headers. Locating which module those headers are in and then adding that module as the dependency is the cleanest approach and should help you spot unnecessary dependencies.
The source code has the following broad grouping:
Clients/: contains modules for ParaView clients/applications including the Qt GUI, Catalyst, command line executables, etc.
VTKExtensions/: Modules that contain vtkObject subclasses as well as XMLs (optional) for proxy definitions. These often build on modules already in VTK.
Remoting/: Modules that have the client-server logic including the servermanager (Remoting/ServerManager). This also includes modules that define vtkSMProxy subclasses for specific functionality e.g. Remoting/Animation, Remoting/Views, Remoting/Live etc.
I feel these still another iteration that needs to happen over this directory structure to make it easier to follow, but this at least removes the unnecessary ServerManager/ServerImplementation split and makes it easier to build a functional CORE and RENDERING variant with ease.
As with any major changes I’d expect some instability with dashboards etc. in the coming days. Please feel free to report any issues you encounter. Don’t forget to do a clean build first. Thanks in advance for your patience!
Note @Andy_Bauer - All old and wonderful things come back around. Catalyst Editions bacame Catalyst Essentials, which then has reverted back to our beloved Catalyst Editions. Documentation will be updated as we get to it.