Performance issue with LoadPlugin() (TTK module)

hi there,

I am the lead developer of ttk (https://topology-tool-kit.github.io/), an open-source library for topological data analysis, which massively relies on the vtk/paraview ecosystem.

ttk ships with a plugin for paraview.
we use this plugin to provide python support to our users.

currently, we recommend users to use pvpython, as described in this example:


on my system, this example takes 2 seconds to execute.

however, several users expressed the need to use a vanilla python interpreter instead of pvpython (in particular, to integrate ttk into an existing python project).

to run with a default python interpreter (as described there: https://kitware.github.io/paraview-docs/latest/python/quick-start.html), I setup the PYTHONPATH and LD_LIBRARY_PATH as follows:
$ export PYTHONPATH="/usr/local/lib/python3.6/site-packages/"
$ export LD_LIBRARY_PATH="/usr/local/lib/:/usr/local/lib/plugins"
(paraview is installed in /usr/local and the ttk plugin in /usr/local/lib/plugins, I use python 3.6)

next, in my python code (https://github.com/topology-tool-kit/ttk/blob/master/examples/python/ttkExample-python.py), I add, line 34, the following instruction:
LoadPlugin(“libTopologyToolKit.so”, ns=globals())

the corresponding diff is shown here:

and the rest of the script executes properly, … but it takes 40 seconds! :frowning:
38 of these are spent in the call LoadPlugin()

so here are my questions:

  • is there anything that I did wrong here?
  • is there a serious performance issue with this function?

my understanding of the situation is that pvpython (https://github.com/Kitware/ParaView/blob/master/CommandLineExecutables/pvpython.h) is loading the plugin natively in C++ while LoadPlugin() does some additional parsing in python.
could that be it?

thanks a lot for your help!
cheers,

2 Likes

I also noticed this issue some time ago, and work around this by loading plugins from the PV_PLUGIN_PATH or PV_PLUGIN_CONFIG_FILE environment variables, which is much faster than LoadPlugin().

Does export PV_PLUGIN_PATH=/path/to/ttk/plugins work for you?
On my system, all plugins in the specified directory are loaded by from paraview.simple import *.

1 Like

awesome lutz!
moving /path/to/ttk/plugins from LD_LIBRARY_PATH to PV_PLUGIN_PATH actually made the trick! it’s running in 2 seconds again now :slight_smile:

I realize we discussed that at some point a few days ago but I thought the LD_LIBRARY_PATH was the official way to go instead (see https://kitware.github.io/paraview-docs/latest/python/quick-start.html)

thanks a lot for your help!!!
cheers,

@utkarsh.ayachit Utkarsh, should the documentation be updated to suggest (and perhaps prefer) this other method?

An issue should also be logged imo, these timings are not normal.

Agreed. And if for some reason we cannot improve the plugin loading speed, documentation changes are in order.

1 Like

I’m able to reproduce this issue, albeit not as drastically.
With a build master and LagrangianParticleTracker plugin.

PV_PLUGIN_PATH : 1.8s
LoadPlugin : 3s

I’ve opened an issue here :
https://gitlab.kitware.com/paraview/paraview/issues/19681

@Kenneth_Moreland : I think this should be fixed instead.

@julien-tierny : A fix has been merged, please use LoadPlugins instead. See here for more info :
https://gitlab.kitware.com/paraview/paraview/-/merge_requests/4092

If you could test, that would be great ?

Also, do you have a gitlab account, I tried to tag you there but couldn’t find you.

hi mathieu,
thanks for your message. I’ll try to have a look at that (I need to dig to find these scripts). cheers,