ParaView Time Refactoring

Development
1

This thread is intended to continue the discussion started here Paraview Animation Proposal, about the time support in ParaView.

TLDR;
Using time can be very confusing for some user, specially when using animation. For instance, it is hard to control frame-per-second settings when playing or saving an animation.

Current state of ParaView

Paraview currently interacts with three different time values:

  • The current Pipeline Time.

  • The Animation Time, usually the same as Pipeline Time unless modified in the timekeeper.

  • The User Time (in seconds) perceived by the user.

The AnimationScene object is responsible for the time consistency within the application.

It is updated when a new dataset is loaded and it exposes available timesteps to the user. Those timesteps are read from files and ParaView does not know about their units (so it can be seconds, hours, months ā€¦).

The Timekeeper is used to map Animation Time and Pipeline Time. By default their are equal. (but they still donā€™t have a unit)

From the doc: TimeKeeper can be thought of as an application wide clock.

Time in the GUI

Mainly, the user has access to the Animation Time.

Toolbars

  • VCR Controls: play / stop / ā€¦ buttons allow to navigate through the timesteps.

When using the Play button, frames are generated as fast as the pipeline can process the new timestep.

  • Current Time Control: a combobox and a spinbox to read / write the Animation Time. These are the values of the Pipeline Time mapped by the TimeKeeper.

Views (aka Dock Widgets)

  • Time Inspector

A timeline with temporal sources, showing a tick for each timestep.

This includes the previous toolbars as control widgets.

  • Animation View

A timeline with timekeeper and manually added properties. One track represents one property over Animation Time. Specifically, the TimeKeeper represents Pipeline Time over Animation Time. If the Timekeeper is disabled here, the pipeline is not anymore updated with new time values (but nothing is shown in the toolbars / time inspector)

Time in Animation

There are three animation modes:

  • Sequence: define a number of frames, evenly distributed, to be played between a start and an end Animation Time.

Effective User Time depends on the time needed to compute the pipeline and render each frame. Pipeline Time is set as the nearest existing timestep. If no timestep exists, pipeline is not recomputed.

  • Real Time: define the user-perceived duration for the animation.

Nb frame is adjusted at runtime depending on the time needed to compute pipeline and render scene. Pipeline Time is set as the nearest existing timestep. If no timestep exists, pipeline is not recomputed.

Duration is in User Time, but the number of frames can vary wildly depending on the time to render a single frame.

  • Snap To Timestep: render successively one frame per data timestep.

Effective User Time depends on the time needed to compute pipeline and render each frame.

Note that all those modes (including snap to timesteps) do not take into account the data time value. It only goes timestep to timestep.

The Pipeline Time entry can be modified using the variable time in the timekeeper. It lets the user modify it relatively to the Animation Time. It is only useful in the case of temporal data, since a non temporal data does not rely on the Pipeline Time, but only on Animation Time.

These settings are very flexible, however they are confusing and can have unpredictable behaviours for the user, especially in the case of non temporal data.

Proposition

We want to easily setup the User Time of the animation while keeping a control on the framerate for performance / quality purpose.

It is an intermediate between the current ā€˜do as much as the you canā€™ in term of frame (Real time mode) or in term of user-perceived time (Sequence mode and Snap to Timestep mode).

The main idea is that the Animation Time and the User Time are now one and the same.

Animation Time now has a unit, seconds (s).

The Real Time, the Sequence and the Snap to Timesteps modes are removed and replaced by:

  • a Frame per Second control (in Animation User Time).

** !! ** In this mode, Pipeline Time will be set to the previous existing timestep.

  • an option to use only exactly frame per existing TimeSteps

If the FPS and the duration are not compatible due to compute time, we can drop frames and warn the user.

If the data time is not at a human scale (i.e. not in sec / minutes), This can be detected and corrected automatically by the timekeeper while informing the user.

We could also support a unit converter in the timekeeper (is Pipeline Time in sec ? or in a month ?) to avoid human conversion.

Also note that the timekeeper is not only a factor between times but an actual mapping, that can use several interpolated points.

We also need to configure the animation writers to respect the FPS and the targeted User Time.

2 Likes
2

co-authors: @Guillaume_Gindre @mwestphal

1 Like
3

This is a step in the right direction, but I still think it is too focused on playback time and not focused enough on presenting the sequence of data available. This is not the same use case of streaming contenting from Netflix. There is no unified expectance of playback time nor any enforceable expectation of how fast frames can be provided. Rather, time in ParaView is more like a flip book of captured time instances. I think dropping frames is a bad idea, even if you warn the user that you are doing it.

Rather than have a frame per second control, I suggest having a number of frames control. The second option would basically be the same: a checkbox to use the frames from files (really, any pipeline time, but I expect users will understand time from files better). When the checkbox is on (the default), then the number of frames control is grayed out (since that is specified from the pipeline). When the pipeline has no explicit time, the checkbox is grayed out and the number of frames is available.

Iā€™m not enamored with the idea of trying to control the time of the animation by trying to adjust the frame rate. That sounds extremely frustrating. Having the simulation time match the desired playback is the exception rather than the rule. Even if the simulation is on a ā€œhuman scaleā€ and the time is actually captured, it is still very common to need to adjust playback. A car crash will probably need to be slowed down. A cloud formation will probably need to be sped up. The flip book metaphor is better for (implicitly) setting the playback time since the number of timesteps is probably indicative of how long the animation should play.

Note that these controls donā€™t apply to animation writers. Movie playback follows more closely to the ā€œNetflixā€ analogy, and specifying animation length and/or frame rates is much more appropriate.

4

I am not a huge fan of this. Can you explain the motivating use-case please? What is the primary use-case that is driving this change?

5

I donā€™ see what Animation Time represents if not User Time. And as the interface mainly speak in term of Animation Time, it is not clear.

Example

timeline_annotated
timeline_annotated1326Ɨ165 18.9 KB

  • red: the animation times
  • blue: the pipeline times
  • green: current pipeline time
  • dark: current animation time
  • orange: In sequence mode, some data timesteps are not used
  • Animation Time goes from 4 to 8.2
  • Pipeline Time goes from 0.4 to 0.82

If I press play, (in sequence or snap to timestep), it goes as fast as possible.
So what is the meaning of animation time here ?? How can I specify the duration of my animation ? Here is the old real time mode. But then, no more control on how many frames will be generated ā€¦
Going back in sequence mode, I would like to slow down some part (with timekeeper). The global duration increases but not the number of frame, leading to a less fluid rendering.

Also, video makers speak in term of duration and framerate, not total number of frame in the video.

Indeed, with our solution there is problem when playing animation (in PV) if the frame rate cannot be respected, that can be lead to either drop frames or slow down rendering.

6

Seems to me that all this is stemming from the fact that you want to link world-time with time in ParaView. As @Kenneth_Moreland mentioned, ParaVIew is not a video application. Itā€™s not meant for editing videos. Thus, an expectation that time in the application is same as what the user-perceives in real world is not reasonable. In fact, Iā€™d argue that we get rid of real-time mode entirely. It was only added as a toy since it was technically possible. In hindsight, it should have never been there since that is probably genesis of this expectation that the time in ParaView has anything to do with world-time.

ParaView is intended for data processing. Processing involves adding filters. Thus each timestep, typically, lots of processing may happen. Any expectation that this processing is at interactive framerates is only true for tiny datasets and simple analysis pipelines. That being the case, hitting play is never expected to be like hitting play in a video player. Hitting play is simply expected to advance through time in the application with no relation to world-time (perceived by the user).

7

Sorry for being slow to the party. Took me a while to get through these threads.

Ken and Utkarsh are completely correct. Users do NOT want ParaView time to generally match real world time. In fact, in 14 years of working with users, I believe I have seen them request this once or twice.

Here is a very typical use case for CTH. CTH is a shock physics code. A bullet will be fired into a cement wall. The bullet is moving at kilometers per second. The simulation starts putting out data when the bullet is 1 cm from the wall. In 0.1 mm increments, move the bullet towards the wall internally in CTH, BUT write out data every 1mm (every 10th timestep). When the bullet first touches the cement wall, start dumping all timesteps for the next 100 timesteps. Then, go back to dumping every 10th timestep for the next 1000 timesteps. Users want to see how the cement fractures, and becomes fragments. Then, they want to measure the mass, velocity and energy of the fragments. Users donā€™t want to see this in realtime, they donā€™t want to guess seconds into the run, and they donā€™t want to have to calculate units change every time they run ParaView. They donā€™t want to mess with a GUI. They just want to see their data - every single frame that they wrote to disk. Almost always.

Ken and Utkarsh are correct.

8

Thinking about this further, I believe I have almost never seen a simulation where the simulation output is in real time. Jet engines, car crashes, supernova, cans getting crushed, pulse power ā€œshotsā€, galaxies moving, car engine pistons running, etc. always want to see from the start of some event (piston top dead center) to the end of some event (piston top dead center, second time it gets here). Further, if an analyst writes out data (which for large runs is VERY, VERY EXPENSIVE), they want to see what they wrote out.

9

I donā€™t have any real input as to your proposed change, other than a change would be welcome! I use a lot of audio and video tools and every time I go to generate a video in Paraview it takes a bit of trial and error to get the result I desire. Things come out too slow, too fast - now I just suck frames into Premiere to make it all easier to deal with.

Camera trials. It would also be beneficial to allow for multiple cameras in the video pipeline - with only one active of course. To compare different camera setups one has to save the state, changing the camera, then restoring the state if the new idea is not good :slight_smile:

10

@xxrich: You input about CameraTrials would be valuable in the other more GUI centred improvements thread : Paraview Animation Proposal

11

I find for day-to-day work scientists seem to always want to see flipbook animations. If they are unhappy with the playback speed, itā€™s usually because they really want slow-mo, and there are various ways to provide that. For conference and public outreach videos where playback speed can vary, I find it easier just to work with exported frames in a video editor.

12

The objective is to simplfy the UI and the fonctionnalities so that any user willing to use animation in ParaView is able to do it easily. Using user-perceived time was an idea to achieve that but it looks like it was not the right one.

Let say we drop the proposition from @nicolas.vuaille post and try something new, letā€™s try to list what is easy are hard to do with the current system :

Easy:

  • See a flipbook animation of a temporal dataset
  • Export a flipbook animation of a temporal dataset
  • Make a single parameter of a filter change over time

Hard:

  • Understand the different time mode and which one should be used
  • Make a temporal dataset and animation view time work togetther in an expected way
  • Work with multiple temporal datasets

Animation is always a complex topic that user do not want to try out as it seems overly complicated. The objective here is just to fix this.

13

Ensight has had the flipbook concept since forever. It is EXTREMELY helpful! flipbook++ :slight_smile: (and if we can run it within a CAVE like theirs did :-))
-rich

14

Iā€™d like to throw in my $0.02 in for the ā€œHardā€ things to listed by @mwestphal.

Based on the discussion, I think that this is improved if not outright fixed by dropping the Real Time mode. I was a big advocate for this mode when we first made the animation panel. But after seeing users struggle to understand and use it, I think it makes sense to drop this mode. That is left with a simple choice of following the time steps in data or defining a number of frames to evenly space from start to finish. Weā€™ve already talked about ways to make this selection more natural using a simple checkbox.

Iā€™m a bit confused about why this is ā€œhardā€ in usual circumstances. Under usual circumstances, the pipeline time and the animation time are locked together. So when you specify you want to animate something from t0 to t1, those times are the same as those in the temporal dataset. I donā€™t know how to make things more clear than that.

The only time this gets complicated is when you want the temporal dataset time to change nonlinearly with the animation progress. For example, itā€™s tricky if you want to pause the dataset time to move the camera. But so far I havenā€™t seen anything suggested to make that easier. I think this kind of interaction is inherently hard, and itā€™s OK if it is not as trivial as setting up other animation features. Iā€™m all for making this easier, but we need to make sure we donā€™t accidentally make other common animation controls harder.

I am again confused on why this is considered ā€œhardā€. I find that ParaView excels at dealing with multiple temporal datasets at the same time. As long as both datasets are properly marked with simulation time on the timesteps, ParaView synchronizes both and everything works beautifully.

Things get more tricky if one or both datasets are not properly marked with simulation time. In that case, simply using the Temporal Shift Scale filter solves the problem. I agree that there is room for improvement here as it is not an obvious solution and the controls can be tricky. But once we point a user to this solution, it usually solves the problem.

I believe problems mainly come in when temporal data does not have the right timesteps. But I think the solutions for those are outside of the animation panels. For example, we should expand the support for time stamps in datasets. Legacy and XML VTK files should definitely be able to specify the simulation time. We could also better support existing time in files. We could understand time units better and better handle units like ā€œdays since 1970-01-01ā€.

In summary, I think the hardest things in the animation controls (if Real Time goes away) deal with changing the playback rate of the pipeline (start/stop or speedup/slowdown) during the animation, and things get really tricky if you want to modify pipeline objects separately (which goes against the design of the VTK pipeline).

15

The one use I have seen for ā€œReal Timeā€ is if users are trying to make a movie, and they want the movie to play in say 30 seconds. This is done though the Animation View, set time to Real Time, then mess with the time in Save Animation/ Advanced properties.

16

That is not correct. When saving animation to movie file, the sequence (or snap to timesteps) is used and frame rate should be used to control the rate at which the video is played back.

17

I think what @wascott meant was that if you are setting up an animation it is convenient if you can get it to play in ParaView the same or close to what will be written to the file. This is the same as wanting to set the view size to match that of a screenshot and animation. Sure, ParaView will resize the image to whatever you want when you create a screenshot, but it can be frustrating if the resizing does not come out the way you thought it would.

Likewise for the time. I think Alan is saying that if the animation does not play in ParaView at the same speed as you want for the movie file, then you might be unpleasantly surprised by the effect of the changed movie playback.

The question is whether this is a big enough use case to make the animation view more complicated for everyone. Maybe the Real Time option can just be hidden. Maybe hide some ā€œPlayback Previewā€ somewhere in the menu?

18

I am not sure how real-time is currently useful even for this case. For anything but trivial datasets, the real-time mode will most likely skip over timesteps. The one case maybe that the data has fewer timesteps and you want to play through them slowly ā€“ I can see real-time mode helping there.

But still very confusing :). My vote is still to get rid of it :).

19

Really good guess, Ken, but not right. If I am making a large movie, that is taking minutes per frame to make, yet each frame represents a fraction of a second, I have two options. One is to create a flipbook, and then post process to make a movie. In the post processing tool, it will ask me how fast the movie that is being created plays (with default being as fast as possible, one frame after the other). The alternative is to have ParaView create the movie (an .avi), and I want to tell ParaView how fast to play the movie. For instance, if I want to create a fire simulation, and the simulation is dumping timesteps representing every 200 miliseconds, I want 5 frames per second. I donā€™t care if ParaView can make the movie play at 10 frames per second, or 2 frames per second. Too slow or fast, of course, is defined by the analyst.

Another example could be where I want to use the Temporal Interpolator. Now, frames are ā€œmade upā€. One way to make movies is by using Sequence, and lots more frames. Notice I have a smoother simulation, but am not making a movie that will play in a set, known period of time. Alternatively, I use RealTime, and tell ParaView how many seconds to make the movie. Hmmm ā€¦ I just tried this, and onscreen looks correct, but the .avi does not properly use the output of the temporal interpolator.

I donā€™t think this functionatity is used much, but I can think of usecases for it. Want me to ask my users if they ever use it?

20

It sounds like your only use case is that you want to specify the length of animation when saving. That seems like a straightforward thing to add to the Save Animation Options. That would make it a lot easier than switching to Real Time mode to set the animation length.