Is it possible to create data "on the fly" with the python API?

Suppose that I have a piece of code (in python) that generates some data. For the sake of the argument, let’s assume is something like a 3D version of the Mandelbrot set. Let’s also assume that I can compute batches of that data fairly quickly.

Is it possible to use Paraview’s Python API to generate the data on-the-fly as needed (i.e. at the requested resolution, zoom level, and bounding-box), rather than having to generate a dedicated file beforehand assuming the highest resolution for the highest zoom level for the largest bounding box?

I could not find anything about this in the documentation or in the forums, so it’s probably impossible? Or did I miss anything? It’d be very convenient for some cases, the Mandelbrot set is an extreme example, in which the possible zoom level is effectively infinite, or at least precision level in IEEE arithmetic, and the necessary file would be of effectively infinite, or at least too big to be easily managed

Hi @Davide and welcome to ParaView Discourse !

One way is to create a filter that generates the data you are interested at will every time is updated. You can then offer controls to the user to modify the resolution /bounds or other attributes of the data source. For example the wavelet source in paraview gives you ability to change its bounds and you could also have one to specify the number of cells. The formula for generating the data is fixed (but parametrized) however the data are generated based on user input “on the fly”.

To start look at the PythonAlgorithm documentation here and especially the PythonSuperquadricSource example.

Thank you for the warm welcome and for the answer!

If I understand correctly that does indeed what I am looking for, but I am really confused about how to use it. I was expecting something of the like request data giving me some bounding box in x-y-z (and corresponding resolution) and me having to fill a data structure (maybe numpy array, but probably more likely some C-based vtk data structures) with values corresponding to what the data to visualize is in that bounding box at that resolution. Ideally Paraview will not call my function twice requesting the same data again, but will cache it instead.

That is what I was expecting and maybe that is what the output.ShallowCopy(self._realAlgorithm.GetOutput()) does but there is no argument passed, i.e. where’s the bounding box? Maybe in this example it doesn’t matter, but the whole point for this (as my example with the Mandelbrot set should make clear) is for me not to have create the whole dataset at full resolution if the visualization is going to happen only in a certain (small) zone.

, but I am really confused about how to use it

I totally understand ! It me a while to figure out how to use them and I am still learning about it. The RequestData API is more general that just passing extends it gives you a vkInformationVectors which can hold a number of different types of requests including timestep update, extends update etc.

I think the RequestSubset algorithm here is a good match for what you are trying to achieve. I would recommend to go through the whole article first to get a gasp of how information flows through a VTK pipeline. If you need more details I suggest to take a look at the “VTK pipeline primer” series which can be found here.

I was expecting something of the like request data giving me some bounding box in x-y-z (and corresponding resolution) and me having to fill a data structure (maybe numpy array, but probably more likely some C-based vtk data structures) with values corresponding to what the data to visualize is in that bounding box at that resolution.

The above example covers all of these.

Ideally Paraview will not call my function twice requesting the same data again, but will cache it instead.

You could keep a cache in the filter in a form of a map between extends used in the past and actual data. Maybe there is something already at the VTK level for that but I am not sure about this.

1 Like

you can also use ProgrammableSource, which may be simpler to use as a starting point.

https://docs.paraview.org/en/latest/ReferenceManual/pythonProgrammableFilter.html