Mismatch of points/tuples between vtkmCleanGrid and its VTK-numpy-wrapper

Hello, I have a vtkUnstructuredGrid dataset that I filter with a vtkmCleanGrid VTK object in a programmable filter, whose objective is to perform the divergence operator on the VTK-numpy wrapped output of the vtkmCleanGrid filter. The programmable filter is the following:

import vtk
from vtk.numpy_interface import algorithms as algs
from vtk.numpy_interface import dataset_adapter as dsa

input0 = inputs[0]
input0_vtk = input0.VTKObject
print(f"Input0 nb of points: {input0_vtk.GetNumberOfPoints()}")

vtkClean = vtk.vtkmCleanGrid()
vtkClean.SetInputData(0, input0_vtk)
vtkClean.Update()
print(f"vtkClean output nb of points: {vtkClean.GetOutput().GetNumberOfPoints()}")

print()

print(f"Wrapped Input0 nb of tuples: {input0.PointData['AERO_Velocity_Vector'].shape}")

vtkClean_wrapped = dsa.WrapDataObject(vtkClean.GetOutput())
print(f"Wrapped vtkClean output nb of tuples: {vtkClean_wrapped.PointData['AERO_Velocity_Vector'].shape}")

algs.divergence(vtkClean_wrapped.PointData["AERO_Velocity_Vector"])

However, there seems to be a mismatch between the number of points in the vtkmCleanGrid VTK output and the number of tuples of arrays in the vtkmCleanGrid wrapped output. More precisely, it seems that the wrapped output contains as many tuples as the VTK input of vtkmCleanGrid (input0: 56912), but not as many as the vtkmCleanGrid output (56855). This raises the following RuntimeError when trying to perform algorithms.divergence on the wrapped vtkmCleanGrid output:

Traceback (most recent call last):
  File "<string>", line 22, in <module>
  File "<string>", line 22, in RequestData
  File "C:\ParaView\bin\Lib\site-packages\vtkmodules\numpy_interface\algorithms.py", line 125, in new_dsfunc
    return dsfunc(array, ds)
  File "C:\ParaView\bin\Lib\site-packages\vtkmodules\numpy_interface\internal_algorithms.py", line 257, in divergence
    g = gradient(narray, dataset)
  File "C:\ParaView\bin\Lib\site-packages\vtkmodules\numpy_interface\internal_algorithms.py", line 320, in gradient
    res = _cell_derivatives(narray, dataset, attribute_type, cd)
  File "C:\ParaView\bin\Lib\site-packages\vtkmodules\numpy_interface\internal_algorithms.py", line 44, in _cell_derivatives
    raise RuntimeError('The number of points does not match the number of tuples in the array')
RuntimeError: The number of points does not match the number of tuples in the array
Input0 nb of points: 56912
vtkClean output nb of points: 56855

Wrapped Input0 nb of tuples: (56912, 3)
Wrapped vtkClean output nb of tuples: (56912, 3)

I am using Paraview 5.10.1 (VTK 9.0.20210922). Please find data to reproduce the issue :

data.vtu (5.8 MB)

Is this problem a known issue ? Has it been solved in more recent versions of Paraview/VTK ?

Thanks a lot, Have a nice day.

you may want to also share this here for more answers: https://discourse.vtk.org/

Thank you @mwestphal ! The post in the VTK discourse is Mismatch of points/tuples between vtkmCleanGrid and its VTK-numpy-wrapper - Development - VTK

Can you try this with the latest version of ParaView? This might be a bug that has already been fixed.

Hello @Kenneth_Moreland. Thanks for replying !

I tried with Paraview 6.1.1 (VTK 9.6.2), and the issue is still here and is the same. Here is the output error message:

Traceback (most recent call last):
  File "<string>", line 22, in <module>
  File "<string>", line 22, in RequestData
  File "C:\ParaView 6.1.1\bin\lib\site-packages\vtkmodules\numpy_interface\numpy_algorithms.py", line 105, in new_dsfunc
    return dsfunc(array, ds)
           ^^^^^^^^^^^^^^^^^
  File "C:\ParaView 6.1.1\bin\lib\site-packages\vtkmodules\numpy_interface\internal_algorithms.py", line 257, in divergence
    g = gradient(narray, dataset)
        ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ParaView 6.1.1\bin\lib\site-packages\vtkmodules\numpy_interface\internal_algorithms.py", line 320, in gradient
    res = _cell_derivatives(narray, dataset, attribute_type, cd)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ParaView 6.1.1\bin\lib\site-packages\vtkmodules\numpy_interface\internal_algorithms.py", line 44, in _cell_derivatives
    raise RuntimeError('The number of points does not match the number of tuples in the array')
RuntimeError: The number of points does not match the number of tuples in the array
Input0 nb of points: 56912
vtkClean output nb of points: 56855

Wrapped Input0 nb of tuples: (56912, 3)
Wrapped vtkClean output nb of tuples: (56912, 3)

Using Paraview 6.2.0 preview (VTK 9.7.0) gives the same error. The only change is the vtkClean output nb of points which now equals 56857.

Thanks !

@Bodacious, thanks for exploring some more. I was able to replicate your problem and can verify that this is a bug with the vtkmCleanGrid filter. I pushed a changed that should fix this and other issues here: https://gitlab.kitware.com/vtk/vtk/-/merge_requests/13631. I expect this change to hit ParaView 6.3.

In the meantime, try adding vtkClean.CompactPointsOn() to your script before the vtkClean.Update() call. I think that will work around the problem for now.

@Kenneth_Moreland Adding CompactPointsOn() does solve the number of points issue in the vtkmCleanGrid output:

Input0 nb of points: 56912
vtkClean output nb of points: 56855


Wrapped Input0 nb of tuples: (56912, 3)
Wrapped vtkClean output nb of tuples: (56855, 3)

Thanks for pushing the change !

Once Applying the CompactPointsOn(), the algs.divergence operator raises the following wranings/errors:

Generic Warning: In C:\glr\builds\paraview\paraview-ci\build\superbuild\paraview\src\VTK\Common\Core\vtkMath.cxx, line 1334

Unable to factor linear system



ERROR: In C:\glr\builds\paraview\paraview-ci\build\superbuild\paraview\src\VTK\Common\DataModel\vtkTetra.cxx, line 952

vtkTetra (0000024729B64780): Jacobian inverse not found



ERROR: In C:\glr\builds\paraview\paraview-ci\build\superbuild\paraview\src\VTK\Common\DataModel\vtkTetra.cxx, line 955

vtkTetra (0000024729B64780): Matrix:(-0.000124249,-0.000116736,-0.00172846 7.4649e-06,-1.26474e-05,2.27988e-06 0,0,0)

In the programmable filter, if I change
algs.divergence(vtkClean_wrapped.PointData["AERO_Velocity_Vector"]) by

a = algs.divergence(vtkClean_wrapped.PointData["AERO_Velocity_Vector"])
vtkClean_wrapped.PointData.append(a, "Velocity_div")

output.ShallowCopy(vtkClean_wrapped.VTKObject)

the previous warnings/error disappear but the resulting Velocity_div contains some NaN values (see image below) (maybe because some cells are degenerate ??). Can the vtkmCleanGrid filter create some degenerate cells that would cause algs.divergence to fail at some points/cells ?

Thanks a lot !

Hello again, I’d like to report a vtkmCleanGrid bug that you might have solved with the push you did. It is also solved with the CompactPointsOn() method:

I am using a(-nother) programmable filter in Paraview 5.10.1 to clip an input dataset at a given x-axis position. The clip is done usinga vtkTableBasedClipDataSet. In order to prevent the points duplication due to this object, I clean this object with a vtkmCleanGrid. However, the array at of the output is disrupted with respect to the input array (see pictures).

The programmable filter is the following :

import vtk
input0_vtk = inputs[0].VTKObject

n_sect, k_sect = 6,4
xmin,xmax,_,_,_,_ = input0_vtk.GetBounds()

vtkPlane = vtk.vtkPlane()
vtkPlane.SetNormal(1,0,0)
vtkPlane.SetOrigin(xmin + (k_sect+1)*(xmax-xmin)/n_sect,0,0)

vtkClip_section = vtk.vtkTableBasedClipDataSet()
vtkClip_section.SetClipFunction(vtkPlane)
vtkClip_section.SetInputData(0,input0_vtk)
vtkClip_section.Update()
if vtkClip_section.GetOutput().GetBounds()[0] > xmin:
    vtkClip_section.SetInsideOut(not vtkClip_section.GetInsideOut())
vtkClip_section.Update()

vtkClean_section = vtk.vtkmCleanGrid()
#vtkClean_section.CompactPointsOn()
#vtkClean_section = vtk.vtkCleanUnstructuredGrid()
vtkClean_section.SetInputConnection(0,vtkClip_section.GetOutputPort())
vtkClean_section.Update()

output.ShallowCopy(vtkClean_section.GetOutput())

The problem still appears in Paraview 6.1.1 (VTK 9.6.2) and 6.2.0 (VTK 9.7.0).

The disruption do not exist when the CompactPointsOn() method is applied to the vtkmCleanGrid.

I tried to clean the vtkTableBasedClipDataSet with a vtkCleanUnstructuredGrid, and the array is not disrupted. However vtkCleanUnstructuredGrid is not available in Paraview 5.10.1 (VTK 9.0.20210922).

Data for reproduction :

data_Aphi.7z (8.7 MB)

Thanks, have a nice day.

Yes. I verified the same change fixes this issue as well. Both are caused by the same underlying problem: the point ids are being changed but the point field arrays are just copied from input to output.