It looks like the array dispatch for SOA arrays with number of components>1 isn’t working. I’ve built ParaView with VTK_DISPATCH_SOA_ARRAYS enabled (I also tried to build with VTK_DISPATCH_TYPED_ARRAYS enabled but had build issues with this option, but this seems like a legacy option that shouldn’t be needed anyway). I’m using the Catalyst V2 CxxImageDataExample (Examples/Catalyst2/CxxImageDataExample/) to test along with this Catalyst script which just tries to write out a Slice data extract: catalyst_pipeline.py (2.3 KB)
To make sure it’s not just my build I also tried with the PV 5.11 RC1 candidate and today’s nightly master install. I get the same incorrect results of essentially uninitialized values on the slice.
So is this a CMake config issue that’s not set properly for all instances, a bug in the array dispatch or something else?
Note that I’m running serial to simplify things and so I won’t have to worry about MPI mismatch issues.
An update on this issue – it looks like it’s the Slice filter that’s having issues. The following VTK commit is the one that introduced the bug:
commit cb236cc661fb8323eceea488eb39273cdb9023a3
Author: Spiros Tsalikis <spiros.tsalikis@kitware.com>
Date: Wed May 25 07:51:44 2022 -0400
vtkCutter now also delegates to vtkPlaneCutter whenever possible
vtkPlaneCutter now also delegates to vtk3DLinearGridPlaneCutter
In the case of vtkUniformGrid, vtkCutter used to use the generic
Dataset cutter, while it could use the StructuredPoints cutter.
Due to this change the expected cells in TestCompositeCutter needed to change.
Filters/Core/Testing/Python/TestCompositeCutter.py | 10 +-
Filters/Core/vtkCutter.cxx | 149 +++++++--------------
Filters/Core/vtkCutter.h | 19 ++-
Filters/Core/vtkPlaneCutter.cxx | 40 +++++-
Filters/Core/vtkPlaneCutter.h | 25 ++--
5 files changed, 109 insertions(+), 134 deletions(-)
I’m using Examples/Catalyst2/CxxImageDataExample but that kind of raises a valid point also. Is there a good source for creating SOA arrays for testing? Maybe vtkConduitSource but that’s probably another topic.
i wrote the following code to create a dataset with soa:
vtkNew<vtkSphereSource> sphere;
sphere->SetRadius(1);
sphere->SetPhiResolution(1000);
sphere->SetThetaResolution(1000);
sphere->Update();
auto output = sphere->GetOutput();
vtkNew<vtkSOADataArrayTemplate<double>> randomArray;
randomArray->SetName("Random");
randomArray->SetNumberOfComponents(1);
randomArray->SetNumberOfTuples(sphere->GetOutput()->GetNumberOfPoints());
for (int i = 0; i < sphere->GetOutput()->GetNumberOfPoints(); ++i)
{
randomArray->SetValue(i, output->GetPoint(i)[0]);
}
sphere->GetOutput()->GetPointData()->AddArray(randomArray);
// write to file
vtkNew<vtkXMLPolyDataWriter> writer;
writer->SetFileName("test.vtp");
writer->SetInputData(output);
writer->Write();
Slice seems to be working just fine. Did your partitioned dataset had image data instead of polydata?
if yes, i might know where this issue comes from.
I encountered issues when the input image data has a scalar array (which you can get using GetScalars()) with more than 1 component.
Basically, vtkFlyingEdgesPlaneCutter expects that the scalar array will have 1 component instead of >1. Is that your case?
Sorry, yeah the array needs to have more than a single component. I didn’t notice that before. The single array component is the trivial case where SOA is essentially the same as AOS.
Andy, I should mention here that SOA array support in VTK/ParaView is very limited. Buyer beware!
You’ll run into a lot of situations where filters just do not work because they are modifying a copy of the data buffer arranged in AOS order, which is done to support the GetVoidPointer() function. This may be one of those situations.
This is very disconcerting then because it means that ParaView Catalyst support is potentially very limited as well. Essentially ParaView Catalyst users are forced to use AOS style arrays which may force them into doing deep copies of data. I suppose the GetVoidPointer() will do the deep copy anyway but only when called, not all of the time.
I suppose I would go as far as saying that array dispatch just needs to work. I’m not sure what the point of having it not working is. Even worse, from my experience discovering this bug is I see that when array dispatch fails the following is the warning:
( 1.854s) [pvbatch ]vtkSOADataArrayTemplate:338 WARN| 23vtkSOADataArrayTemplateIdE (0x24bf270): GetVoidPointer called. This is very expensive for non-array-of-structs subclasses, as the scalar array must be generated for each call. Using the vtkGenericDataArray API with vtkArrayDispatch are preferred. Define the environment variable VTK_SILENCE_GET_VOID_POINTER_WARNINGS to silence this warning.
This should be an error letting the user know that something is wrong.
Another point of concern is that for Catalyst V2 API users they may not know they’re using AOS arrays.
The dispatch works just fine. The problem is that not all filters use array dispatch, but instead use the vtkTemplateMacro which relies on GetVoidPointer(). The work that needs to be done is to eliminate the need for GetVoidPointer(), but that will require modification of something like 600 VTK filters. In other words, it will be a massive undertaking to fully and efficiently support SOA arrays.
I agree it should all just work, but at the moment I wouldn’t suggest that anyone use SOA arrays. I have been burned by trying to do the same a couple times now.
My understanding is probably incomplete here but I thought that when GetVoidPointer() was called for an SOA data array that this was being “dispatched” in some way to be properly handled. If this is something else then this is what needs to have a working implementation because this operation that currently isn’t working and for hundreds of filters to not be working is a big concern for in situ. Even if this operation to make GetVoidPointer() function properly for SOA arrays is slow and memory intensive, at least it works instead of failing and even worse failing without proper error feedback.
Additionally, should GetVoidPointer() be deprecated? This way at least people will stop using the method when they develop internally in VTK.
Ideally, yes. However, there’d have to be suppression in VTK itself for it without actually getting VTK itself off of it. And without that, there’s no actual sunset date for the method (usually “after the next release”).