Updates to Virtual Reality support in ParaView

There have been some updates to the VR capabilities in ParaView that I wanted to share. They are still in early development but I have put up an executable here if you would like to try them out. They include displaying parts of the PV GUI directly in VR, the ability to add points to a source using the controller in VR, and collaboration. I created a quick example video here to show what some of this looks like. It is not a fancy video, more of a live screen grab :slight_smile: Just wanted to get the idea across.

The changes to support this are merged into VTK (including putting Qt widgets into renderers) but not yet merged into PV. Over the next week or so I’ll try to get the PV changes merged in as well. There are rough edges etc so if you run into problems please feel free to let me know.

Thanks!
Ken

11 Likes

Awesome stuff, Ken!

Wow, this is really great!
I like the principle of having the Qt GUI in the renderer opens a lot of opportunities.

Updated with a new executable from here https://data.kitware.com/#item/5d0922c4676794a06e4cf623

The original executable would crash if you pressed some of the buttons before bringing up the PV VR gui.

Looks great @martink. How is the GUI / QT rendered inside the rendered view?

Using the new vtkQWidgetWidget class in VTK (event handling). Which in turn uses a vtkQWidgetRepresentation which uses a vtkQWidgetTexture (the actual drawing). The basic gist is we share a texture between Qt and VTK. Qt paints the texture, VTK then renders it as part of the VR world.

This MR has the key pieces https://gitlab.kitware.com/vtk/vtk/merge_requests/5553/diffs

Nice thanks @martink it seems to work well. One caveat is that the QT UI might be too dense for the VR so perhaps we may need a custom interface for interactions in the VR environment.

Those are awesome new features @martink !

Is it possible to trigger an animation from the VR-GUI?

Animation performance (and then exposing the GUI in VR) are definitely on the todo list for this summer/early fall.

Nice work, Ken.

Sounds great.

Will the VR-feature support clustering/CAVE in future again?

Works just perfectly like in your video! The only thing I noticed that some UI elements (text, buttons) are displayed too large in VR (in an HTC Vive) because I have a 24 inch 4K monitor where I had to scale up the user interface in the Windows display settings.

I guess this will not land in ParaView 5.7?

I believe the OpenVR support (and dialog, menu) will be in 5.7 but without the collaboration support as the superbuild does not build those parts right now.

@martink This looks really great. We now have the hardware and would like to try to use PV with VR. What is the current status? All info I can find is older than this thread. Is the plugin still limited to Linux/MacOS?

It works under Windows with PV-5.8.0

1 Like

I am having trouble using Windows with PV-5.8.0, all I have done is downloaded PV-5.8.0, loaded the OpenVR plugin, rendered an object, and clicked “Send to OpenVR” while my headset was connected to SteamVR. Is there something I am missing? In VR there is a page that says “Paraview waiting…” and then my computer shuts down. Thanks!

Hello @martink,
many thanks for this amazing video! I wonder about the hardware setup you need for the collaboration mode. Could you please give some details? I guess you need two VR sets (obviosly :wink:) connected to two separate GPUs plugged into one computer? Or are the two headsets connected to a single GPU? Also, will it work equally well for Oculus Rift S and HTC Vive? Or are there some limitations?
Thank you very much for sharing your experience!

A post was split to a new topic: Help with vtkQTWitdgetWidget