New module system landing soon

Hi all,

VTK’s side of the new module system is landing in the next week or so. ParaView will follow shortly after.

The VTK announcement is here: https://public.kitware.com/pipermail/vtk-developers/2019-January/036648.html
Its merge request is here: https://gitlab.kitware.com/vtk/vtk/merge_requests/5020
The ParaView MR will be up once VTK merges, but the branch as-is can be seen here (updated sporadically): https://gitlab.kitware.com/ben.boeckel/paraview/tree/new-cmake-module

CMake changes:

  • PARAVIEW_BUILD_PLUGIN_XXXPARAVIEW_PLUGIN_ENABLE_XXX
  • CMake APIs for plugins, clients, and server manager XML files have been updated and made much more uniform. As with VTK, there is inline documentation with the code which will eventually be extracted and added to ParaView’s Doxygen documentation.
  • ClientServer wrapping uses the same patterns as Python and Java now (plugins still get it “for free”).
  • ServerManager XML files can now be attached to other modules (though this is not done right now). For example, ParaView can add a proxy to the VTK::IONetCDF module and it will be used properly. This is in contrast to the way it is done currently where the XML file is associated with ParaView::ServerManagerApplication and it just hopes that VTK::IONetCDF is available in some way for the ClientServer wrappings.
  • ServerManager, ClientServer, and plugins are built in such a way to provide targets for proper initialization (e.g., ParaView makes a paraview_server_manager target for all of its XML outputs). See the documentation for usage details.

The details for VTK are largely the same as for ParaView except:

  • Only a handful of plugins have been updated at the moment; the rest will be updated after the initial merge.
  • Examples are not updated yet.
  • The catalyze.py and the edition manifests are not yet updated.
  • VisItBridge needs updated for the new system (will likely be done in concert with https://gitlab.kitware.com/paraview/paraview/issues/18681).

Some other tasks that need to be done once it lands:

  • The ParaViewCore/Testing directory needs integrated. As far as I can tell, it isn’t used on master either.
  • The XML tests should be sorted a bit better.
  • ServerManager XML files should be split to be associated with the modules providing the classes behind the proxies (the new CMake API for this makes it much easier).
  • ParaView also has lots of unused .sha512 data files laying around. Whether to remove them from the repo or add tests to use them should be determined.
  • Some modules still have optional public dependencies. These should be resolved.

The merge request is here: https://gitlab.kitware.com/paraview/paraview/merge_requests/2971

A superbuild followup MR will happen once this has landed.

Nice work ben ! Building it right now.

Let us know when we can investigate and open related issues.

@ben.boeckel: could you share a basic ParaView plugin example ?

The only ported ones are those in ParaView itself so far. External plugins night need some more CMake code (e.g., find_package(ParaView) and that might still have issues), but I don’t think they’ll need too much.

Nonetheless, I’m unable to add a plugin in ParaView either.
I’m even unable to add another dummy filter to ArrowGlyph plugin.

I must be missing something, the client server wrapping does not take place.

Did you look at those in ParaView? In order to be wrapped in ClientServer, a module must be created. The vtk_module_scan and vtk_module_build boilerplate in the plugin’s CMakeLists.txt will likely be refactored out and made simpler with a new function the plugin CMake API provides, but for now it’s still necessary.

Of course, I used them as a base.

Here is a simple patch that tries to add a MyElevationFilter to the ArrowGlyph plugin, without success.
The CS wrapping is not done.

0001-jpouderoux-test.patch (7.5 KB)

Mathieu:
Fix by moving Filters::Core in DEPENDS instead of PRIVATE_DEPENDS

Indeed, thanks !

This is one is kinda hard to spot.

@ben.boeckel A first error when trying to build an external plugin:
in paraview-config.cmake.in: at lines 42 and 43, it should be

include("${CMAKE_CURRENT_LIST_DIR}/vtk/vtkModuleWrapPython.cmake")
include("${CMAKE_CURRENT_LIST_DIR}/vtk/vtk-python.cmake")

instead of


include("${CMAKE_CURRENT_LIST_DIR}/vtkModuleWrapPython.cmake")
include("${CMAKE_CURRENT_LIST_DIR}/vtk-python.cmake")

Yeah, paraview-config.cmake needs some other love as well. Those files should actually just be handled with the find_package(VTK) that should be there. Once I have the buildbots happier, I’ll focus on that. ParaView has been a rockier merge since my machines test much less of ParaView comparatively and I hadn’t tested plugins on other platforms…

@ben.boeckel Do you have an agenda for this? When can users expect something stable on this side?

The big issues right now is plugin loading on Windows (debugging now) and Python on Linux not working on the buildbot (next). I’m working on the build failures while I’m getting builds to debug for those. Hopefully it won’t be more than a couple more days…

Great. I guess you also noticed that default plugins are no more listed in the Plugin Manager, which is boring.

@ben.boeckel : much better now. Everything works when adding a plugin in pv except for the UI_RESSOURCE with a .qrc file. Is that suposed to work ?

I don’t think I’ve seen that not working. What isn’t working about it?

When loading a plugin where we added:

paraview_add_plugin( ...
UI_RESOURCES BagPlotViewsAndFilters.qrc
)

Plugin load fails and prints:

ERROR: In /.../paraview/ParaViewCore/ClientServerCore/Core/vtkPVPluginLoader.cxx, line 418
vtkPVPluginLoader (0x55f98a2f3e20): /.../paraview-build/lib/paraview-5.6/plugins/BagPlotViewsAndFilters/BagPlotViewsAndFilters.so: undefined symbol: _ZTV28BagPlotViewsAndFiltersPlugin

That’s the vtable for the plugin class. Is the generated source missing a dtor or another function which is declared in the header, but not in the source?

1 Like

Yes but when removing UI_RESOURCES, plugin works as expected - see https://gitlab.kitware.com/paraview/paraview/merge_requests/2974#note_508209.