I have a fairly large dataset and a fairly complicated pipeline and I would like to avoid redundant data loads. Specifically, I perform several clips of the domain, compute several different integrals in each clip and then I want to see temporal statistics for some integrals and plot over time for others.
My question is: does Plot Data Over Tim and Temporal Statistics filters execute independently? It would make sense in this instance to load a new time, update all downstream filters before loading another time step. Is that what happens? Please let me know.
PlotDataOverTime and TemporalStatitics are both CONTINUE_EXECUTABLE filters which means they will trigger each a full pass on the timesteps, which can be very heavy as the data will be loaded for every time steps, for each filter.
A way to avoid that is to add a TemporalCache filter after your data reader and set the cache size to the number of timesteps in your data.
This will of course increase memory usage by a lot but you will load each timesteps only once.
Thanks, @mwestphal. I am going to read into it presently. I didn’t quite appreciate what Temporal Cache filter cache is. Can I just ask if it makes sense to cache the entire data source or the clips or is that the trade off I need to check myself?
Sorry, I have unmarked it as a solution as I am not so sure Temporal Caching works in a case I described. Temporal Statistics and Plot Over Time from my different clips still seem to execute independently.
I feel like a simple solution would be to implement a stepper through all time steps and reading values of all my integrals and noting them down somewhere. I could then compute averages and plots over time.
Or maybe what I should do is to group all my integrals into a single data sets and then put Temporal Statistics or Plot Data Over Time on top of it?
Of course they do. The temporal cache only avoid reading the timesteps multiples times.
I feel like a simple solution would be to implement a stepper through all time steps and reading values of all my integrals and noting them down somewhere. I could then compute averages and plots over time.
Not supported by VTK. Interesting idea but definitely not trivial.