Issue Running Animations in VR

Aditya_Agashe · July 28, 2021, 1:26am

I am trying to run visualizations in VR through Paraview and have been having trouble playing animations. The screen would jump around chaotically after clicking play, rather than displaying a smooth animation. After some digging, I have concluded that the issue is likely because of the data size.

To create the animation, I had taken a volume data set of a variable lambda with multiple timesteps. Then, I made an isosurface at a certain lambda value for each time step using the contour filter. I would then save the geometry of the isosurfaces, and run the animation on just the geometry. The animation would not work when trying to play it in VR because the screen would bounce around and not play a smooth visualization. So, I tried changing how I was making the isosurface. At higher lambda values, the isosurface is smaller, and thus contains less data. I made an isosurface using these higher lambda values, and again saved the geometries. This time, the animation was able to play in VR without issues.

Because the animation works for higher lambda values where there is less data, and doesn’t work for lower lambda values with more data, I am thinking data size is the issue. I have attached a folder with the data if that would be helpful. The folder has 2 folders inside it, one for a working case of lambda value 2, and a non-working case of lambda value 1. Inside each folder I’ve uploaded the isosurfaces for the corresponding lambda values, as well as a recording depicting what happens in VR. The lambda = 2 geometries are about 5 Mb each, while the lambda = 1 geometries are about 7 Mb each. Both have 15 frames each. These are just two sample lambda values. The problem gets even worse at values smaller than 1 and better at values bigger than 2.

Would using a machine with more memory help at all? Or is there something else I could be doing to alleviate this? I would appreciate any info on if there is a way to solve this problem for bigger data sizes.

Folder: Paraview VR Issue - Google Drive

To run the cases, go into either the non-working or working folder, where you will see a folder titled “lambda1” or “lambda2”. Download these two folders, then start Paraview. Click File->Open, and then navigate inside either the lambda1 or lambda2 directory. All 15 vtm files will show as a “group”, which you can open as one file by clicking on it. After making them visible, you can animate them in Paraview by hitting the triangular “play” icon. You can animate it in VR by going to Tools → Manage Plugins, and then loading the OpenVR plugin. Finally, you can simply click “send to openVR” to see the animation in VR.

martink · July 30, 2021, 8:32pm

The issue (most likely) is that ParaView caches animation data in host memory, not on the GPU. Your dataset happens to work for me but it is right on the edge of failing on my system. The issue being that ParaView rebuilds the GPU data every frame and uploads it to the GPU. So when rendering at 90hz it doesn’t take too much data to make that process too slow.

We have wanted to add animation caching on the GPU for a while but right now we don’t have it.

You definitely can do what you are looking to do in VTK if you do not need ParaView.

The basic approach is for each timestep create an actor + mapper and add them all to the renderer. Then turn their visibility all off. When rendering you setup a start callback on the renderer and turn one actor visible. Next frame turn a different one visible.

If you want to do it in ParaView you can but it is a bit complex. Here is a state file (for master PV) that shows how to do it. The basic gist is that it lumps all the time steps together into one piece of data that sits on the GPU and then uses shader replacements to rotate through what time step gets shown.

animatedIso.pvsm (335.4 KB)

Shader replacements are tricky as some PV settings can break them etc but they can be very powerful in some cases like this.

Aditya_Agashe · July 31, 2021, 1:51am

Thank you for the feedback. I will look into using VTK for my project. In regards to the state file you attached, I’m having trouble opening it. Paraview starts not responding, which it doesn’t seem to come out of. When opening other state files of my own, Paraview is able to open them within a few seconds, but for this one after about 10 minutes it was still not responding. Is there a particular way in which I’m supposed to open it? Or am I supposed to wait longer? I’ve attached the error message it gives me below if that helps.

From the error message displayed, I’m thinking it might not be working because it’s looking for a file address that doesn’t exist on my laptop

martink · July 31, 2021, 2:35pm

Someone else may understand file IO issues, it is not something I work in on ParaView. Having said that just load the state file into an editor and replace C:\Users\ken.martin with whatever your paths are and that should get past that issue.

Here is a link to the PV I am using if that helps as well. https://data.kitware.com/#item/61055f552fa25629b98a94c2

Aditya_Agashe · July 31, 2021, 3:34pm

Thank you. Editing the file paths worked. However, I’m having some issues with the Group Time Steps filter that you have.

How did you get that filter to appear? For me, when I load in the 15 lambda time steps into Paraview, every filter is grayed out, including Group Time Steps. I’ve attached a picture of this below

grayed_out

The animation runs very quickly and smoothly. However, when I try rotating the isosurface during the animation there is considerable lag, with it loading GeometryRepresentationWithFaces for a few seconds.

Also, the Group Time Steps filter doesn’t seem able to be colored by VorticityX, as when I select that option no color appears, and the isosurface simply fades in and out upon clicking play.

In regards to VTK, I don’t have much experience with that, which is why I was using Paraview. I am currently trying to figure out how VTK works and how to use it for my project. In VTK, can I load in my individual .vtm timesteps and then run an animation, or would I have to recreate the isosurfaces from scratch?

martink · July 31, 2021, 6:37pm

You have to hit apply (the green button in your picture) first. Then the available filters will show up. Using the state file I provided you will not be able to color it without a better understanding of the shaders. Direct VTK is probably the way to go, you would be able to load your individual timesteps.

Aditya_Agashe · July 31, 2021, 7:15pm

Thank you for your help. I am trying to go through a tutorial of VTK online so that I can do my project that way. However, I still do have one question about the Group Time Steps filter. When I do it myself, all the time steps seem to get combined into one isosurface and one timestep, as shown in the picture below. When I click animate, the isosurface is static. Is there certain settings I need to set in properties to make it the way yours was?

Aditya_Agashe · August 12, 2021, 2:50am

Hi Ken,

Thank you for your tips. I was able to get VTK working, and was able to animate my .vtp files. Now, I want to be able to view my animation in VR. Do you know a way to do that or a resource that I could use?

martink · August 14, 2021, 1:51am

You need to use OPenVR versions of Camera, Renderer, RenderWindow and RenderWIndowInteractor. something vaguely like

#include <vtkNew.h>

#include <vtkOpenVRCamera.h>

#include <vtkOpenVRInteractorStyle.h>

#include <vtkOpenVRRenderer.h>

#include <vtkOpenVRRenderWindow.h>

#include <vtkOpenVRRenderWindowInteractor.h>

#include <vtkPolyDataMapper.h>



int main(int argc, char* argv[])

{

  // Setup the OpenVR rendering classes

  vtkNew<vtkActor> actor;

  vtkNew<vtkOpenVRRenderer> renderer;

  vtkNew<vtkOpenVRRenderWindow> renderWindow;

  vtkNew<vtkOpenVRRenderWindowInteractor> iren;

  vtkNew<vtkOpenVRInteractorStyle> style;

  vtkNew<vtkOpenVRCamera> cam;



  renderWindow->AddRenderer( renderer.Get() );

  renderer->AddActor( actor.Get() );

  iren->SetRenderWindow( renderWindow.Get() );

  iren->SetInteractorStyle( style.Get() );

  renderer->SetActiveCamera( cam.Get() );

Aditya_Agashe · August 14, 2021, 2:28am

I had tried simply replacing the import statements before (and changing instances in the code), but that didn’t work. I would get an error like this: “Cannot open include file: ‘vtkOpenVRCamera.h’: No such file or directory.” I had downloaded the OpenVR SDK as specified on vtk.org/flavors/ from the following link: GitHub - ValveSoftware/openvr: OpenVR SDK, but am still getting that error.

I thought maybe there was something I had to do with the OpenVR SDK download first. Thus, I tried configuring & generating it into a different folder using CMake, which created the solution file. However, when trying to open the solution file and click build all, I got errors like: “Cannot open include file: ‘vrcore/pathtools_public.h’: No such file or directory”.

Was there a step that I’m missing or something I did incorrectly? Please advise where I went wrong.

martink · August 16, 2021, 2:24pm

Did you turn on OpenVR in your VTK build?

Aditya_Agashe · August 16, 2021, 4:01pm

No, I don’t think so. By VTK build are you referencing when I used CMake to configure and generate the VTK download from the website? If so, was there a checkbox for OpenVR that I was supposed to click during the process? And would I now have to re-do the process of using CMake and Build All in Visual Studio to create the VTK folders from scratch? And if I do it from scratch, would my current VTK programs stop working?

Aditya_Agashe · August 16, 2021, 9:42pm

Hi Ken,

I think I figured out how you are supposed to turn on a module in VTK. This is what I did:

I saw that I don’t have to do it all from scratch again, so I am merely re-Cmaking into the existing build folder. However, when I tried to click configure, I got this error:

It’s telling me it can’t find OpenVR, which I don’t know how to fix. Please advise on what to do.

martink · August 16, 2021, 11:06pm

git clone GitHub - ValveSoftware/openvr: OpenVR SDK and then point the cmake vars to the right spots in that tree

Aditya_Agashe · August 17, 2021, 1:08am

Ok thank you. I have downloaded the folder from that github link. However, I am not sure how to “point” cmake vars to a desired location. I’ve tried looking online for some answers but still am unsure of what variables it is that I’m supposed to change in CMake. There’s a lot of info on CMake online and it is hard for me to determine which of that is useful to my task. I also tried looking online and through the file directory myself, but could not find any clues that could help me find what places in the OpenVR SDK folder are the “right spots”, and how to point CMake to them.

Aditya_Agashe · August 17, 2021, 10:29pm

Hi, so what I ended up doing was going into the initial VTK folder I downloaded, and navigating to Rendering->OpenVR. There, I found a lot of header files that I copied into my VTK folder in Program Files (x86). Now, I once again tried to Cmake and build my VTK animation, with the relevant header files replaced with OpenVR ones. This did make my initial error that said “Cannot open include file: ‘vtkOpenVRCamera.h’ go away. However, it replaced it with a new error, as shown below:

So it was able to find vtkOpenVRCamera.h, but in that header file it refers to ‘vtkRenderingOpenVRModule.h’ which I don’t have. I searched “This PC” for it and no results came up. Do you know how I can get this header file?

Also, I had taken all the header files from my initial VTK download folder itself, I never used the Github folder. This is because I had found a lot of the OpenVR header files in the VTK download folder, but could not find them in the Github folder. What am I supposed to be using the Github folder for?

Aditya_Agashe · August 18, 2021, 2:22am

I think I finally figured out what you meant by pointing cmake vars. I’ve attached a screenshot below of what I did:

With this, I was able to successfully build and generate using CMake. However, when I went to try and build my solution file in Visual Studio, I once again ran into trouble. It first told me that it could not find “openvr.h”, even though it was present in openvr-master/headers. I do not know why that happened, but I tried to circumvent it by just copy pasting it into the VTK folder where all the other header files were. However, upon doing so, it gave me the error below.

How can I fix this?

martink · August 18, 2021, 2:16pm

Make sure your application’s CmakeLists file depends on VTK::RenderingOpenVR something like

target_link_libraries(myapp PRIVATE
    VTK::CommonComputationalGeometry
    VTK::CommonSystem
    VTK::IOCore
etc ...
    VTK::InteractionWidgets
    VTK::RenderingOpenVR
)

martink · August 18, 2021, 2:19pm

Also if you don’t have much experience using CMake, VTK can be a bit much at first. There are some good books and introductions to CMake out there such as An Introduction to Modern CMake · Modern CMake

Aditya_Agashe · August 18, 2021, 3:42pm

I’ve looked through my CmakeLists for my animation, but the target_link_libraries section doesn’t look like the excerpt you gave. Rather, it just says “Private $(VTK_Libraries)$” for both add_executable and target_link_libraries. However, I still added VTK::RenderingOpenVR to the end of both. Then, when I went to configure/generate with CMake, it gave the following error:

I’ve attached my CMakeLists.txt file if that helps.

Also, could the problem be because I still haven’t added the OpenVR module to VTK yet? As I said in my earlier post, I managed to point cmake vars to the right spot when configuring the VTK download, so CMake was able to work. But, I still haven’t been able to build the solution file in Visual Studio because it can’t open source file “openvr.h”.
CMakeLists.txt (1.2 KB)