# Paraview doesn't perform ray tracing (OSPRay) correctly with MPI
**URL:** https://discourse.paraview.org/t/paraview-doesnt-perform-ray-tracing-ospray-correctly-with-mpi/11529
**Category:** ParaView Support
**Tags:** mpi
**Created:** [February 28, 2023, 9:23am UTC](https://discourse.paraview.org/t/paraview-doesnt-perform-ray-tracing-ospray-correctly-with-mpi/11529 "2023-02-28T09:23:09Z")
**Posts on this page:** 5
**Page:** 1
### Author:  [@eve-le-guillou](https://discourse.paraview.org/u/eve-le-guillou)
#### Post date: [February 28, 2023, 9:23am UTC](https://discourse.paraview.org/t/paraview-doesnt-perform-ray-tracing-ospray-correctly-with-mpi/11529/1 "2023-02-28T09:23:09Z")
Hi all.
I can’t seem to install Paraview with MPI and OSPRay properly, though my pipeline executes, my output is incorrect.
I use Paraview 5.10.1.
I have installed OSPRay v2.9.0 (with Embree 3.13.3, Oidn 1.4.3, oneDNN 2.6, RKCommon 1.9.0, OpenVKL 1.2.0 and Snappy latest)
To install OPSRay with MPI, I simply put the compilation variable OSPRAY\_MODULE\_MPI of OSPRay to ON. This is the only depedency for which I set any MPI related compilation variable.
For Paraview, I set PARAVIEW\_ENABLE\_RAYTRACING to ON and VTK\_ENABLE\_OSPRAY to ON, as well as PARAVIEW\_USE\_MPI to ON.
I use OpenMPI 4.0.3.
I don’t want to use MPI for Offloading computation of OSPRay, I want to use OSPRay while computing an example in MPI.
Here is the pipeline I use:
[test\_ospray.py](https://discourse.paraview.org/uploads/short-url/uJ926coufF0vTrjZl2U4PXIGdx8.py) (4.8 KB)
The data file can be downloaded here: [ttk-data/pegasus.vtu at dev · topology-tool-kit/ttk-data · GitHub](https://github.com/topology-tool-kit/ttk-data/blob/dev/pegasus.vtu)
It is quite small (25MB).
I left the rendering options, as this is a rendering problem, but the pipeline is actually very simple:
```
Pegasus > RedistributeDataSet
```
I execute the pipeline using:
```
mpirun -n 8 pvbatch pipeline.py
```
I also tried with the pvserver:
```
mpirun -n 8 pvserver
paraview
```
And connected it to the server with the Paraview client.
Everything computes without error messages, however I obtain the following image:
As can be seen, the Ray tracing is only performed on the domain of each process, independently of each other. For example, the wing of the pegasus should put a shadow on his leg, but that is only done on the part of the leg that is contained by the process containing the wing.
The Raytracing of one process to another is not performed here.
However, when I click on the Pegasus and move it around, the shading is indeed performed correctly as shown here.
When I release the mouse, the error of Raytracing comes back again.
I tried setting the following variables:
```
export OSPRAY_DEFAULT_DEVICE="mpi"
export OSPRAY_LOAD_MODULES="mpi"
```
But that won’t change anything.
I tried using the variable OSPRAY\_DEVICE, setting it to mpiOffload or mpiDistributed, but that causes Paraview to crash, so I am thinking that this might not be the way and it also seems to be for distributing OSPRay computation and Paraview computation.
I don’t know what else to try. I haven’t been able to find a step by step documentation for this specific case for Paraview.
Thanks for any help.
---
### Author:  [@mwestphal](https://discourse.paraview.org/u/mwestphal)
#### Post date: [February 28, 2023, 9:27am UTC](https://discourse.paraview.org/t/paraview-doesnt-perform-ray-tracing-ospray-correctly-with-mpi/11529/2 "2023-02-28T09:27:48Z")
Can you try with the binary release of ParaView ?
[https://www.paraview.org/download/](https://www.paraview.org/download/)
---
### Author:  [@eve-le-guillou](https://discourse.paraview.org/u/eve-le-guillou)
#### Post date: [February 28, 2023, 9:45am UTC](https://discourse.paraview.org/t/paraview-doesnt-perform-ray-tracing-ospray-correctly-with-mpi/11529/3 "2023-02-28T09:45:02Z")
I just did with Paraview 5.11.0 and it works perfectly. Paraview 5.10.1 conflicts with my personal installation.
---
### Author:  [@Dave\_DeMarle](https://discourse.paraview.org/u/Dave_DeMarle)
#### Post date: [February 28, 2023, 8:06pm UTC](https://discourse.paraview.org/t/paraview-doesnt-perform-ray-tracing-ospray-correctly-with-mpi/11529/4 "2023-02-28T20:06:43Z")
I’m very surprised that it works with 5.11.
Shadow, reflection, refraction and ambient occlusion effects need non-local information to be calculated correctly. Ie a camera ray hits the leg on one rank, then a secondary ray is sent toward the light on that same rank. It should hit the wing and cause a shadow, but the wing is on a different rank so it misses and the leg it gets full luminance.
There are research projects out there, the most recent is TACC’s GraviT ( [Gravit - Texas Advanced Computing Center (utexas.edu)](https://www.tacc.utexas.edu/research-development/tacc-software/gravit)) that send either rays or data around the network to solve the problem, but none of them have yet been integrated into ParaView for either OSPRay or OptiX.
Until that happens, my general recommendation is to keep all of these effects off when using ParaView at scale. ParaView with Sandia’s IceT ( [IceT – Center for Computing Research (CCR) (sandia.gov)](https://www.sandia.gov/ccr/software/icet/)) can (or at least should - we are looking at a couple of reported bugs with them) depth composite primary rays the same way it does for rasterized images.
---
### Author:  [@eve-le-guillou](https://discourse.paraview.org/u/eve-le-guillou)
#### Post date: [March 6, 2023, 1:02pm UTC](https://discourse.paraview.org/t/paraview-doesnt-perform-ray-tracing-ospray-correctly-with-mpi/11529/5 "2023-03-06T13:02:17Z")
Understood, thank you for answering me.