Memory error Clip by Surface

ParaView Support
1

I am trying to use the surface clip as described here: Slice/Clip with a surface - Tips and Tricks - ParaView.

The operation works well for a small dataset, but it fails on larger data, even with very large memory systems. It fails much more quickly when using distributed parallelism (2 nodes).

I load the data from exodus files, about 500M cells and points, for a total of 85 GB. I use MergeBlocks to convert it to UnstructuredGrid, on the theory that most filters work well with that.

I load the surface (PolyData, saved as .pvd, 282 MB without any fields), run AppendReduce (18GB), then Extract Surface (because AppendReduce returns an UnstructuredGrid). The result has 475M cells, 17.6 GB.

I’m running this on two large-memory nodes (~2TB each).

I would appreciate any advice on how to get this to work. It fails with out-of-memory errors.

image
image1917×1034 120 KB

Pipeline:

2

Hi @woodscn

A proper implementation that doesn’t rely on a programmable filter seems necessary to me.

3

Thanks for sharing the screenshot with memory inspector. It looks like you’ve got plenty of RAM available before the Programmable Filter. I wouldn’t suspect the Programmable Filter running Python itself is the cause of the OOM error. Does ParaView’s Clip filter with a Plane or other implicit function complete on this same volume dataset? If not, then the problem is with clipping in general. The output of clipping is tetrahedra, so if you start with hexes, for instance, and those get converted to tetrahedra, your memory use will go way up for the additional cell connectivity data.

If Clip works otherwise, then my suspicion is that the additional cell locator objects allocated in vtkImplicitPolyData might be spiking memory usage.

Try replacing

function = vtk.vtkImplicitPolyDataDistance()
function.SetInput(inp)

with

function = vtk.vtkPlane()
function.SetOrigin(0,0,0)
function.SetNormal(0,1,0)

and see if that completes. If so, then we’ve got the problem narrowed down to excessive memory usage in vtkImplicitPolyDataDistance.

1 Like
4

Standard Clip with a Plane, no Crinkle Clip, works fine and is performant.

I tried with this Programmable Filter:

import paraview.modules.vtkPVVTKExtensionsFiltersGeneral as ext

inpMain = self.GetInputDataObject( 0, 0 )
out = self.GetOutput()

clipper = ext.vtkPVMetaClipDataSet()
clipper.SetInputDataObject(inpMain)
clipper.SetInsideOut(True)
function = vtk.vtkPlane()
function.SetOrigin(0,1,0)
function.SetNormal(1,1,1)
clipper.SetDataSetClipFunction(function)
clipper.Update()

outData = clipper.GetOutputDataObject(0)
out.DeepCopy(outData)

This also works fine and quickly.