Dear forum,
My name is Raffi Enficiaud, and I am developing a plugin in Paraview for brain data visualization. I am relatively new to Paraview/VTK and plugin developments, so please accept my apologies if my questions seem naive.
My question concern general design of a python programmable filter, and how to improve the user “experience” by making things faster and easier to interact with. Python obviously is not the fastest programming language, but this apart, I would like to have some better insights on how to shape the overall architecture of the plugin such that it scales better to computations and interactions.
Right now the pipeline looks like this:
- some mesh
- python programmable filter
The python filter depends on:
- the mesh upstream
- an external file (numpy array) that is transformed depending on the mesh
- some parameters that affect the processing performed by the plugin
- some parameters that affect the rendering of the plugin
the step 2 is the most time consuming, and ideally I want to avoid any recomputation that is unnecessary. Especially, I want to avoid reprocessing everything is the user change some visualization parameter from 3/ (for instance, he/she selects a specific trajectory or primitive that should be shown or hidden).
The processing is performed in numpy, so there are vtk<->numpy transformations I would like to keep as minimal as possible.
Here are some ideas, but I do not know how to achieve those. Any hint would be more than welcome:
-
divide the plugin into 2 or 3: a processing plugin and a graphic primitive plugin. The questions are then:
- is it possible a plugin to output a numpy array, and this array being passed to downstream plugins?
- if so, how do I name the array that is being passed downstream? how do I distinguish between the different sources of information?
- is the user required to manually add those 2 plugins in the pipeline? Is there a way to hide this, in the sense that the end user does not need to know there are 2 plugins? For instance, the user selects my plugin, and automatically two elements are added to the pipeline without the user doing any manual configuration. I am asking this because the targeted users are not techy at all.
- if the processing plugin does not need any input (just a file that is outside of the pipeline), then it would make sense to make it a programmable source. Yet, I would like to be able to change some of the parameters of the source. I am currently using examples for generating the XML out of some python description from there: https://blog.kitware.com/easy-customization-of-the-paraview-python-programmable-filter-property-panel/ and that I adapted to my needs. Is a programmable source working the same?
-
The other option would be caching some information that does not need to be recomputed.
- is there a way to store information in a global memory, such that it can be reused the next time the pipeline executes the plugin? Is there a best disk location to cache information on disk?
- is there a way to have an information about the upstream pipeline? If the pipeline changes somehow, some computation need to be executed again with the new incoming data. Is there a flag or a hash that might be used to indicate that the incoming data has changed since last processing?
-
Finally, to accelerate the processing, some python parts might be compiled as python extensions. But there I am facing some problems.
Paraview on OSX and Linux are both compiled against the system python, which is a good news for ABI compatibility. The version of numpy might however not match the one that comes from the system or any virtualenv. So best is that, during the compilation of the python extension, we use the same environment as the one provided by Paraview. My python extension uses cython. Was there any successful attempt to compile a python extension into Paraview with cython?
Thank you for your time, and thanks in advance for any hint,
Best,
Raffi Enficiaud