# Issue with Programmable Filter on OSMesa compiled version for Unix, but not with Windows installation.

**URL:** https://discourse.paraview.org/t/issue-with-programmable-filter-on-osmesa-compiled-version-for-unix-but-not-with-windows-installation/2067
**Category:** ParaView Support
**Tags:** python
**Created:** [June 18, 2019, 1:12pm UTC](https://discourse.paraview.org/t/issue-with-programmable-filter-on-osmesa-compiled-version-for-unix-but-not-with-windows-installation/2067 "2019-06-18T13:12:41Z")
**Posts on this page:** 17
**Page:** 1

<div class="post-metadata">

### Author: ![Joeri\_van\_Engelen](https://discourse.paraview.org/letter_avatar_proxy/v4/letter/j/13edae/32.png) [@Joeri\_van\_Engelen](https://discourse.paraview.org/u/Joeri_van_Engelen)
#### Post date: [June 18, 2019, 1:12pm UTC](https://discourse.paraview.org/t/issue-with-programmable-filter-on-osmesa-compiled-version-for-unix-but-not-with-windows-installation/2067/1 "2019-06-18T13:12:42Z")

</div>

**The problem**  
I use a script to convert PointData to CellData in the programmable filter (the netCDFreader returns PointData). Cory once wrote it and it works fine on my Windows Laptop with the normal Paraview Installation (v5.4.1). However, on my OSMesa compiled version on Linux (Paraview 5.4.1, Mesa 17.2.8), it seizes to work. For some reason my Programmable Filter does not seem to do anything, even though no error is thrown, but no data is passed. When I remove the Programmable Filter, to just plot PointData, everything renders fine again.

I’m interested in your thoughts as to the cause of this problem.

**The things I tried**  
I first thought it was a python path error only in the programmable filter. But it does not throw any import errors and when I print my sys.path in the filter script, it is identical to sys.path in my pvbatch script, which has no trouble with processing data.

Furthermore, when I define the following function to print the variable names in a dataset (and to distuingish cell data or point data).

```auto
 def get_arraynames(Property):
     names = {}
     
     cd = servermanager.Fetch(Property).GetCellData()
     names["cd"] = [cd.GetArrayName(i) for i in range(cd.GetNumberOfArrays())]
     
     pd = servermanager.Fetch(Property).GetPointData()
     names["pd"] = [pd.GetArrayName(i) for i in range(pd.GetNumberOfArrays())]
     return(names)

```

All Properties in the pipeline after my programmable filter on the Windows version get a list under “cd”, however this list is empty on the Linux version I compiled. (“pd” is also empty for the filter in that version. Before the programmable filter, “pd” contains the names both on Linux as Windows, as expected)

The filter script is hardcoded in my pvbatch script as a 1 line string with “\n” (as traced by Paraview on Windows), so I think it should not be a carriage return Windows-Linux conversion thing.

**The filter script**

```auto
dims = inputs[0].GetDimensions()
ext = inputs[0].GetExtent()
output.SetDimensions(dims[0]+1, dims[1]+1, dims[2]+1)
output.SetExtent(ext[0], ext[1]+1, ext[2], ext[3]+1, ext[4], ext[5]+1)
inputPd = inputs[0].PointData
outputCd = output.CellData
for array in inputPd:
   print(type(array))
   outputCd.append(array, array.GetName())

```

**bash script with which I compiled Paraview:**

```auto
 #!/bin/bash

module load cmake

mkdir paraview.bld
cd paraview.bld

MESA_INSTALL_PREFIX=$HOME/visualization/mesa
LLVM_INSTALL_PREFIX=$HOME/visualization/llvm

export PATH=$PATH:$HOME/visualization/llvm/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:${MESA_INSTALL_PREFIX}/lib
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:${LLVM_INSTALL_PREFIX}/lib

cmake \
  -DPARAVIEW_ENABLE_PYTHON=ON \
  -DCMAKE_INSTALL_PREFIX=$HOME/visualization/paraview \
  -DPYTHON_EXECUTABLE=$HOME/anaconda3/envs/py27/bin/python2.7 \
  -DPYTHON_INCLUDE_DIR=$HOME/anaconda3/envs/py27/include/python2.7 \
  -DPYTHON_LIBRARY=$HOME/anaconda3/envs/py27/lib/libpython2.7.so.1.0 \
  -DPARAVIEW_BUILD_QT_GUI=OFF \
  -DVTK_USE_X=OFF \
  -DOPENGL_INCLUDE_DIR=IGNORE \
  -DOPENGL_xmesa_INCLUDE_DIR=IGNORE \
  -DOPENGL_gl_LIBRARY=IGNORE \
  -DOSMESA_INCLUDE_DIR=${MESA_INSTALL_PREFIX}/include \
  -DOSMESA_LIBRARY=${MESA_INSTALL_PREFIX}/lib/libOSMesa.so \
  -DVTK_OPENGL_HAS_OSMESA=ON \
  -DVTK_USE_OFFSCREEN=OFF \
  ../ParaView-v5.4.1
 
make -j32
make -j32 install

```

---

<div class="post-metadata">

### Author: ![Joeri\_van\_Engelen](https://discourse.paraview.org/letter_avatar_proxy/v4/letter/j/13edae/32.png) [@Joeri\_van\_Engelen](https://discourse.paraview.org/u/Joeri_van_Engelen)
#### Post date: [June 25, 2019, 1:58pm UTC](https://discourse.paraview.org/t/issue-with-programmable-filter-on-osmesa-compiled-version-for-unix-but-not-with-windows-installation/2067/2 "2019-06-25T13:58:48Z")

</div>

Are there any suggestions as to what I should look for to debug this problem further? Or how I can improve this problem description to increase the ease of responding (in other words: how I can help you in helping me)?

---

<div class="post-metadata">

### Author: ![utkarsh.ayachit](https://discourse.paraview.org/user_avatar/discourse.paraview.org/utkarsh.ayachit/32/39_2.png) [@utkarsh.ayachit](https://discourse.paraview.org/u/utkarsh.ayachit)
#### Post date: [June 25, 2019, 2:38pm UTC](https://discourse.paraview.org/t/issue-with-programmable-filter-on-osmesa-compiled-version-for-unix-but-not-with-windows-installation/2067/3 "2019-06-25T14:38:33Z")

</div>

Since this is image data, is your programmable filter also providing the `RequestInformation` script? You will need to provide that too to correctly report the data extents being generated since they are different from the input.

---

<div class="post-metadata">

### Author: ![Joeri\_van\_Engelen](https://discourse.paraview.org/letter_avatar_proxy/v4/letter/j/13edae/32.png) [@Joeri\_van\_Engelen](https://discourse.paraview.org/u/Joeri_van_Engelen)
#### Post date: [June 26, 2019, 8:09am UTC](https://discourse.paraview.org/t/issue-with-programmable-filter-on-osmesa-compiled-version-for-unix-but-not-with-windows-installation/2067/4 "2019-06-26T08:09:36Z")

</div>

No, I have not, thanks. The windows version strangely enough did not require this. Are there any example scripts available somewhere?

---

<div class="post-metadata">

### Author: ![cory.quammen](https://discourse.paraview.org/user_avatar/discourse.paraview.org/cory.quammen/32/11193_2.png) [@cory.quammen](https://discourse.paraview.org/u/cory.quammen)
#### Post date: [July 18, 2019, 1:58am UTC](https://discourse.paraview.org/t/issue-with-programmable-filter-on-osmesa-compiled-version-for-unix-but-not-with-windows-installation/2067/5 "2019-07-18T01:58:36Z")

</div>

Yes, please see this example: [https://www.paraview.org/Wiki/Python\_Programmable\_Filter#Changing\_Data\_Type](https://www.paraview.org/Wiki/Python_Programmable_Filter#Changing_Data_Type)

---

<div class="post-metadata">

### Author: ![Joeri\_van\_Engelen](https://discourse.paraview.org/letter_avatar_proxy/v4/letter/j/13edae/32.png) [@Joeri\_van\_Engelen](https://discourse.paraview.org/u/Joeri_van_Engelen)
#### Post date: [July 24, 2019, 11:37am UTC](https://discourse.paraview.org/t/issue-with-programmable-filter-on-osmesa-compiled-version-for-unix-but-not-with-windows-installation/2067/6 "2019-07-24T11:37:41Z")

</div>

Thank you, I have added the following script to `RequestInformation` Script:

```auto
from paraview import util

ext = inputs[0].GetExtent()

util.SetOutputWholeExtent(self, [ext[0], ext[1], ext[2], ext[3], ext[4], ext[5]])

```

Again it works fine on Windows (interestingly enough I had to remove the +1 behind `ext[1]`,`ext[3]`, and `ext[5]`), but I’m getting the same (Python) errors I was getting before on Unix:

```auto
Traceback (most recent call last):
  File "<string>", line 22, in <module>
  File "<string>", line 4, in RequestInformation
TypeError: 'NoneType' object has no attribute ' __getitem__'
Traceback (most recent call last):
  File "<string>", line 22, in <module>
  File "<string>", line 5, in RequestData
TypeError: 'NoneType' object has no attribute ' __getitem__'

```

I also see there is an RequestUpdateExtent script, should I add something there as well?

Furthermore, from the above error description it seems that the filter already gets a None before it even can do something with the RequestInformation scripts, despite that point data is going into the ProgrammableFilter.

Summary of printouts in this pseudo-code:

```auto
print(get_arraynames(netCDFReader1))
#Prints
#{'pd': ['conc', 'dcdt', 'fhead', 'head', 'vx', 'vy', 'vz'], 'cd': []}
programmableFilter1 = ProgrammableFilter(Input=netCDFReader1)
#...
#Here all the scripts etc. are assigned to programmableFilter1
#...
print(get_arraynames(programmableFilter1))
#Prints
#{'pd': [], 'cd': []}

```

My programmable filter script now depends on `inputs` which is `None` in the Unix version I run and a `vtk.numpy_interface.dataset_adapter.DataSet` object in the Windows version.

Update: So apparently it sets `hasnumpy` to `False` on the Unix version.

---

<div class="post-metadata">

### Author: ![cory.quammen](https://discourse.paraview.org/user_avatar/discourse.paraview.org/cory.quammen/32/11193_2.png) [@cory.quammen](https://discourse.paraview.org/u/cory.quammen)
#### Post date: [July 24, 2019, 1:40pm UTC](https://discourse.paraview.org/t/issue-with-programmable-filter-on-osmesa-compiled-version-for-unix-but-not-with-windows-installation/2067/7 "2019-07-24T13:40:41Z")

</div>

You build the linux version yourself, correct? If you built against an installed version of Python, you’ll need to install numpy with either your package manager or pip so that it is available in your Python installation.

---

<div class="post-metadata">

### Author: ![Joeri\_van\_Engelen](https://discourse.paraview.org/letter_avatar_proxy/v4/letter/j/13edae/32.png) [@Joeri\_van\_Engelen](https://discourse.paraview.org/u/Joeri_van_Engelen)
#### Post date: [July 24, 2019, 2:03pm UTC](https://discourse.paraview.org/t/issue-with-programmable-filter-on-osmesa-compiled-version-for-unix-but-not-with-windows-installation/2067/8 "2019-07-24T14:03:33Z")

</div>

Yes correct,

The Anaconda environment I am using does have numpy installed

I think I’m getting Python environments mixed as I include a anaconda environment but the installed build also has its own python environment.

Should I use `-DPARAVIEW_ENABLE_NUMPY=ON` when building?

Update: Nope that did not fix it.

---

<div class="post-metadata">

### Author: ![cory.quammen](https://discourse.paraview.org/user_avatar/discourse.paraview.org/cory.quammen/32/11193_2.png) [@cory.quammen](https://discourse.paraview.org/u/cory.quammen)
#### Post date: [July 24, 2019, 2:14pm UTC](https://discourse.paraview.org/t/issue-with-programmable-filter-on-osmesa-compiled-version-for-unix-but-not-with-windows-installation/2067/9 "2019-07-24T14:14:45Z")

</div>

Backing up a bit, what are you using to build ParaView? Are you using the superbuild?

> [@Joeri\_van\_Engelen](#):
>
> I think I’m getting Python environments mixed as I include a anaconda environment but the installed build also has its own python environment.

ParaView’s Python environment is taken from whichever one you build against. Are you building against the Anaconda environment?

> [@Joeri\_van\_Engelen](#):
>
> Should I use `-DPARAVIEW_ENABLE_NUMPY=ON` when building?

That’s not a defined option in ParaView or ParaView’s superbuild AFAIK.

---

<div class="post-metadata">

### Author: ![Joeri\_van\_Engelen](https://discourse.paraview.org/letter_avatar_proxy/v4/letter/j/13edae/32.png) [@Joeri\_van\_Engelen](https://discourse.paraview.org/u/Joeri_van_Engelen)
#### Post date: [July 24, 2019, 2:57pm UTC](https://discourse.paraview.org/t/issue-with-programmable-filter-on-osmesa-compiled-version-for-unix-but-not-with-windows-installation/2067/10 "2019-07-24T14:57:57Z")

</div>

I am building ParaView from a ParaView-v5.4.1.tar.gz file I found on a portal (forgot the link, since I downloaded it a month ago).  
I followed these instructions [https://www.paraview.org/Wiki/ParaView/ParaView\_And\_Mesa\_3D](https://www.paraview.org/Wiki/ParaView/ParaView_And_Mesa_3D)

I am not sure what “building against” means here, I installed a conda python 2.7 environment with numpy and I used these options:

-DPYTHON\_EXECUTABLE=$HOME/anaconda3/envs/py27/bin/python2.7   
-DPYTHON\_INCLUDE\_DIR=$HOME/anaconda3/envs/py27/include/python2.7   
-DPYTHON\_LIBRARY=$HOME/anaconda3/envs/py27/lib/libpython2.7.so.1.0 \

Before building, I ensured that this environment is added to $PYTHONPATH.

However, if I add this environment to $PYTHONPATH before running `pvbatch`, I get segmentation faults.

---

<div class="post-metadata">

### Author: ![cory.quammen](https://discourse.paraview.org/user_avatar/discourse.paraview.org/cory.quammen/32/11193_2.png) [@cory.quammen](https://discourse.paraview.org/u/cory.quammen)
#### Post date: [July 24, 2019, 3:01pm UTC](https://discourse.paraview.org/t/issue-with-programmable-filter-on-osmesa-compiled-version-for-unix-but-not-with-windows-installation/2067/11 "2019-07-24T15:01:14Z")

</div>

Maybe a dumb question, but if you run `$HOME/anaconda3/envs/py27/bin/python2.7`, does `import numpy` work?

---

<div class="post-metadata">

### Author: ![Joeri\_van\_Engelen](https://discourse.paraview.org/letter_avatar_proxy/v4/letter/j/13edae/32.png) [@Joeri\_van\_Engelen](https://discourse.paraview.org/u/Joeri_van_Engelen)
#### Post date: [July 25, 2019, 7:23am UTC](https://discourse.paraview.org/t/issue-with-programmable-filter-on-osmesa-compiled-version-for-unix-but-not-with-windows-installation/2067/12 "2019-07-25T07:23:14Z")

</div>

Good that you also force me to check the basics (as this is probably a silly mistake I made), but unfortunately it is not this one:

```auto
$ $HOME/anaconda3/envs/py27/bin/python2.7
Python 2.7.15 | packaged by conda-forge | (default, Feb 28 2019, 04:00:11)
[GCC 7.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>> numpy.zeros((10))
array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])

```

---

<div class="post-metadata">

### Author: ![Joeri\_van\_Engelen](https://discourse.paraview.org/letter_avatar_proxy/v4/letter/j/13edae/32.png) [@Joeri\_van\_Engelen](https://discourse.paraview.org/u/Joeri_van_Engelen)
#### Post date: [July 25, 2019, 8:12am UTC](https://discourse.paraview.org/t/issue-with-programmable-filter-on-osmesa-compiled-version-for-unix-but-not-with-windows-installation/2067/13 "2019-07-25T08:12:26Z")

</div>

OK, I got a bit further with bug tracking. When I do `print(sys.path)` in the Filter script, it outputs:  
`['/nfs/home1/jengelen/synthdelta/delta_aquifer/post/paraview_macros', '/home/jengelen/visualization/paraview/lib/paraview-5.4/site-packages/vtk', '/home/jengelen/visualization/paraview/lib/paraview-5.4/site-packages', '/home/jengelen/visualization/paraview/lib/paraview-5.4', '/nfs/home1/jengelen/synthdelta/jobs', '/home/jengelen/visualization/paraview/lib/paraview-5.4/site-packages', '/usr/lib64/python27.zip', '/usr/lib64/python2.7', '/usr/lib64/python2.7/plat-linux2', '/usr/lib64/python2.7/lib-tk', '/usr/lib64/python2.7/lib-old', '/usr/lib64/python2.7/lib-dynload', '/usr/lib64/python2.7/site-packages', '/usr/lib64/python2.7/site-packages/gst-0.10', '/usr/lib64/python2.7/site-packages/gtk-2.0', '/usr/lib/python2.7/site-packages']`

So apparently it built against a python2.7 installation in the /usr/ folder on the cluster instead of the folders I specified (I guess the Red Hat Enterprise server comes with this by default).

---

<div class="post-metadata">

### Author: ![Joeri\_van\_Engelen](https://discourse.paraview.org/letter_avatar_proxy/v4/letter/j/13edae/32.png) [@Joeri\_van\_Engelen](https://discourse.paraview.org/u/Joeri_van_Engelen)
#### Post date: [July 25, 2019, 9:55am UTC](https://discourse.paraview.org/t/issue-with-programmable-filter-on-osmesa-compiled-version-for-unix-but-not-with-windows-installation/2067/14 "2019-07-25T09:55:46Z")

</div>

OK, I tried rebuilding, ensuring that the anaconda environment was activated and in my PYTHONPATH, but it did not work. Even though during building it prints:  
`-- Found PythonInterp: /home/jengelen/anaconda3/envs/py27/bin/python2.7 (found suitable version "2.7.15", minimum required is "2")`

The `/usr/lib/python2.7` keeps on popping up in the filter script.

It is probably easier to make the filter script independent of numpy. So by not using `inputs`, but use `self.GetIntput()` or something. Is that possible?

---

<div class="post-metadata">

### Author: ![Joeri\_van\_Engelen](https://discourse.paraview.org/letter_avatar_proxy/v4/letter/j/13edae/32.png) [@Joeri\_van\_Engelen](https://discourse.paraview.org/u/Joeri_van_Engelen)
#### Post date: [July 25, 2019, 11:53am UTC](https://discourse.paraview.org/t/issue-with-programmable-filter-on-osmesa-compiled-version-for-unix-but-not-with-windows-installation/2067/15 "2019-07-25T11:53:12Z")

</div>

OK, I have fixed my problem by getting rid of the numpy dependency, as it was unnecessary.

To avoid having to use `inputs`, I changed my programmable filter script to:

```auto
inp = self.GetInput()
dims = inp.GetDimensions()
ext = inp.GetExtent()

oup = self.GetOutput()
oup.SetDimensions(dims[0]+1, dims[1]+1, dims[2]+1)
oup.SetExtent(ext[0], ext[1]+1, ext[2], ext[3]+1, ext[4], ext[5]+1)

N=inp.GetPointData().GetNumberOfArrays()

for i in range(N):
    data = inp.GetPointData().GetAbstractArray(i)
    oup.GetCellData().AddArray(data)

```

Thanks Cory and Utkarsh for the support, your questions guided me in the right direction!

---

<div class="post-metadata">

### Author: ![cory.quammen](https://discourse.paraview.org/user_avatar/discourse.paraview.org/cory.quammen/32/11193_2.png) [@cory.quammen](https://discourse.paraview.org/u/cory.quammen)
#### Post date: [July 25, 2019, 1:54pm UTC](https://discourse.paraview.org/t/issue-with-programmable-filter-on-osmesa-compiled-version-for-unix-but-not-with-windows-installation/2067/16 "2019-07-25T13:54:06Z")

</div>

Yep, that will work if you don’t mind giving up the `numpy` support.

@ben.boeckel Any ideas for getting ParaView to build with Joeri’s Anaconda installation instead of the Python in `/usr/lib`?

---

<div class="post-metadata">

### Author: ![ben.boeckel](https://discourse.paraview.org/letter_avatar_proxy/v4/letter/b/ea5d25/32.png) [@ben.boeckel](https://discourse.paraview.org/u/ben.boeckel)
#### Post date: [July 26, 2019, 11:35am UTC](https://discourse.paraview.org/t/issue-with-programmable-filter-on-osmesa-compiled-version-for-unix-but-not-with-windows-installation/2067/17 "2019-07-26T11:35:39Z")

</div>

I’d just go through the `CMakeCache.txt` and find any mention of Python from `/usr/lib` and change it to the one in the Anaconda env.