# Array dispatch issues

**URL:** https://discourse.paraview.org/t/array-dispatch-issues/10328
**Category:** Development
**Tags:** vtk
**Created:** [September 15, 2022, 2:58pm UTC](https://discourse.paraview.org/t/array-dispatch-issues/10328 "2022-09-15T14:58:34Z")
**Posts on this page:** 18
**Page:** 1

<div class="post-metadata">

### Author: ![Andy\_Bauer](https://discourse.paraview.org/user_avatar/discourse.paraview.org/andy_bauer/32/5442_2.png) [@Andy\_Bauer](https://discourse.paraview.org/u/Andy_Bauer)
#### Post date: [September 15, 2022, 2:58pm UTC](https://discourse.paraview.org/t/array-dispatch-issues/10328/1 "2022-09-15T14:58:34Z")

</div>

It looks like the array dispatch for SOA arrays with number of components\>1 isn’t working. I’ve built ParaView with `VTK_DISPATCH_SOA_ARRAYS` enabled (I also tried to build with `VTK_DISPATCH_TYPED_ARRAYS` enabled but had build issues with this option, but this seems like a legacy option that shouldn’t be needed anyway). I’m using the Catalyst V2 CxxImageDataExample (Examples/Catalyst2/CxxImageDataExample/) to test along with this Catalyst script which just tries to write out a Slice data extract:  
[catalyst\_pipeline.py](https://discourse.paraview.org/uploads/short-url/8Z8hWNAWoI0V5N0YLylAOWaqDPY.py) (2.3 KB)

To make sure it’s not just my build I also tried with the PV 5.11 RC1 candidate and today’s nightly master install. I get the same incorrect results of essentially uninitialized values on the slice.

So is this a CMake config issue that’s not set properly for all instances, a bug in the array dispatch or something else?

Note that I’m running serial to simplify things and so I won’t have to worry about MPI mismatch issues.

Thanks,  
Andy

---

<div class="post-metadata">

### Author: ![Andy\_Bauer](https://discourse.paraview.org/user_avatar/discourse.paraview.org/andy_bauer/32/5442_2.png) [@Andy\_Bauer](https://discourse.paraview.org/u/Andy_Bauer)
#### Post date: [September 16, 2022, 8:38pm UTC](https://discourse.paraview.org/t/array-dispatch-issues/10328/2 "2022-09-16T20:38:40Z")

</div>

An update on this issue – it looks like it’s the `Slice` filter that’s having issues. The following VTK commit is the one that introduced the bug:

```auto
commit cb236cc661fb8323eceea488eb39273cdb9023a3
Author: Spiros Tsalikis <spiros.tsalikis@kitware.com>
Date: Wed May 25 07:51:44 2022 -0400

    vtkCutter now also delegates to vtkPlaneCutter whenever possible
    
    vtkPlaneCutter now also delegates to vtk3DLinearGridPlaneCutter
    
    In the case of vtkUniformGrid, vtkCutter used to use the generic
    Dataset cutter, while it could use the StructuredPoints cutter.
    Due to this change the expected cells in TestCompositeCutter needed to change.

 Filters/Core/Testing/Python/TestCompositeCutter.py | 10 +-
 Filters/Core/vtkCutter.cxx | 149 +++++++--------------
 Filters/Core/vtkCutter.h | 19 ++-
 Filters/Core/vtkPlaneCutter.cxx | 40 +++++-
 Filters/Core/vtkPlaneCutter.h | 25 ++--
 5 files changed, 109 insertions(+), 134 deletions(-)

```

---

<div class="post-metadata">

### Author: ![cory.quammen](https://discourse.paraview.org/user_avatar/discourse.paraview.org/cory.quammen/32/11193_2.png) [@cory.quammen](https://discourse.paraview.org/u/cory.quammen)
#### Post date: [September 19, 2022, 5:14pm UTC](https://discourse.paraview.org/t/array-dispatch-issues/10328/3 "2022-09-19T17:14:37Z")

</div>

@spyridon97 could you please investigate?

---

<div class="post-metadata">

### Author: ![spyridon97](https://discourse.paraview.org/user_avatar/discourse.paraview.org/spyridon97/32/11569_2.png) [@spyridon97](https://discourse.paraview.org/u/spyridon97)
#### Post date: [September 19, 2022, 5:32pm UTC](https://discourse.paraview.org/t/array-dispatch-issues/10328/4 "2022-09-19T17:32:06Z")

</div>

On it.

---

<div class="post-metadata">

### Author: ![spyridon97](https://discourse.paraview.org/user_avatar/discourse.paraview.org/spyridon97/32/11569_2.png) [@spyridon97](https://discourse.paraview.org/u/spyridon97)
#### Post date: [September 19, 2022, 5:35pm UTC](https://discourse.paraview.org/t/array-dispatch-issues/10328/5 "2022-09-19T17:35:58Z")

</div>

@Andy_Bauer Could i have the dataset that you used to evaluate that the SOA\_Dispatch does not work?

---

<div class="post-metadata">

### Author: ![Andy\_Bauer](https://discourse.paraview.org/user_avatar/discourse.paraview.org/andy_bauer/32/5442_2.png) [@Andy\_Bauer](https://discourse.paraview.org/u/Andy_Bauer)
#### Post date: [September 19, 2022, 5:44pm UTC](https://discourse.paraview.org/t/array-dispatch-issues/10328/6 "2022-09-19T17:44:28Z")

</div>

I’m using `Examples/Catalyst2/CxxImageDataExample` but that kind of raises a valid point also. Is there a good source for creating SOA arrays for testing? Maybe `vtkConduitSource` but that’s probably another topic.

---

<div class="post-metadata">

### Author: ![spyridon97](https://discourse.paraview.org/user_avatar/discourse.paraview.org/spyridon97/32/11569_2.png) [@spyridon97](https://discourse.paraview.org/u/spyridon97)
#### Post date: [September 19, 2022, 5:56pm UTC](https://discourse.paraview.org/t/array-dispatch-issues/10328/7 "2022-09-19T17:56:57Z")

</div>

i wrote the following code to create a dataset with soa:

```nohighlight
    vtkNew<vtkSphereSource> sphere;
    sphere->SetRadius(1);
    sphere->SetPhiResolution(1000);
    sphere->SetThetaResolution(1000);
    sphere->Update();
    auto output = sphere->GetOutput();

    vtkNew<vtkSOADataArrayTemplate<double>> randomArray;
    randomArray->SetName("Random");
    randomArray->SetNumberOfComponents(1);
    randomArray->SetNumberOfTuples(sphere->GetOutput()->GetNumberOfPoints());
    for (int i = 0; i < sphere->GetOutput()->GetNumberOfPoints(); ++i)
    {
        randomArray->SetValue(i, output->GetPoint(i)[0]);
    }
    sphere->GetOutput()->GetPointData()->AddArray(randomArray);

    // write to file
    vtkNew<vtkXMLPolyDataWriter> writer;
    writer->SetFileName("test.vtp");
    writer->SetInputData(output);
    writer->Write();

```

Slice seems to be working just fine. Did your partitioned dataset had image data instead of polydata?  
if yes, i might know where this issue comes from.

---

<div class="post-metadata">

### Author: ![spyridon97](https://discourse.paraview.org/user_avatar/discourse.paraview.org/spyridon97/32/11569_2.png) [@spyridon97](https://discourse.paraview.org/u/spyridon97)
#### Post date: [September 19, 2022, 6:08pm UTC](https://discourse.paraview.org/t/array-dispatch-issues/10328/8 "2022-09-19T18:08:12Z")

</div>

I also tried to create a wavelet with sos as follows:

```nohighlight
    vtkNew<vtkRTAnalyticSource> wavelet;
    wavelet->SetWholeExtent(-10, 10, -10, 10, -10, 10);
    wavelet->SetCenter(0, 0, 0);
    wavelet->Update();
    auto output = wavelet->GetOutput();

    vtkNew<vtkSOADataArrayTemplate<double>> randomArray;
    randomArray->SetName("Random");
    randomArray->SetNumberOfComponents(1);
    randomArray->SetNumberOfTuples(output->GetNumberOfPoints());
    for (int i = 0; i < output->GetNumberOfPoints(); ++i)
    {
        randomArray->SetValue(i, std::sin(output->GetPoint(i)[0]));
    }
    output->GetPointData()->SetScalars(randomArray);

    vtkNew<vtkXMLImageDataWriter> writer;
    writer->SetFileName("test.vti");
    writer->SetInputData(output);
    writer->Write();

```

Still the slice filter seems to work just fine.

---

<div class="post-metadata">

### Author: ![Andy\_Bauer](https://discourse.paraview.org/user_avatar/discourse.paraview.org/andy_bauer/32/5442_2.png) [@Andy\_Bauer](https://discourse.paraview.org/u/Andy_Bauer)
#### Post date: [September 19, 2022, 6:26pm UTC](https://discourse.paraview.org/t/array-dispatch-issues/10328/9 "2022-09-19T18:26:50Z")

</div>

Yes, the partitioned dataset has image data.

---

<div class="post-metadata">

### Author: ![Andy\_Bauer](https://discourse.paraview.org/user_avatar/discourse.paraview.org/andy_bauer/32/5442_2.png) [@Andy\_Bauer](https://discourse.paraview.org/u/Andy_Bauer)
#### Post date: [September 19, 2022, 6:30pm UTC](https://discourse.paraview.org/t/array-dispatch-issues/10328/10 "2022-09-19T18:30:54Z")

</div>

Can you share your `test.vti` here?

---

<div class="post-metadata">

### Author: ![Andy\_Bauer](https://discourse.paraview.org/user_avatar/discourse.paraview.org/andy_bauer/32/5442_2.png) [@Andy\_Bauer](https://discourse.paraview.org/u/Andy_Bauer)
#### Post date: [September 19, 2022, 6:52pm UTC](https://discourse.paraview.org/t/array-dispatch-issues/10328/11 "2022-09-19T18:52:35Z")

</div>

I’ve been running with the  
[catalyst\_pipeline.py](https://discourse.paraview.org/uploads/short-url/8Z8hWNAWoI0V5N0YLylAOWaqDPY.py) (2.3 KB)  
Catalyst script with `Examples/Catalyst2/CxxImageDataExample`.

---

<div class="post-metadata">

### Author: ![spyridon97](https://discourse.paraview.org/user_avatar/discourse.paraview.org/spyridon97/32/11569_2.png) [@spyridon97](https://discourse.paraview.org/u/spyridon97)
#### Post date: [September 19, 2022, 6:54pm UTC](https://discourse.paraview.org/t/array-dispatch-issues/10328/12 "2022-09-19T18:54:25Z")

</div>

I encountered issues when the input image data has a scalar array (which you can get using GetScalars()) with more than 1 component.  
Basically, vtkFlyingEdgesPlaneCutter expects that the scalar array will have 1 component instead of \>1. Is that your case?

---

<div class="post-metadata">

### Author: ![Andy\_Bauer](https://discourse.paraview.org/user_avatar/discourse.paraview.org/andy_bauer/32/5442_2.png) [@Andy\_Bauer](https://discourse.paraview.org/u/Andy_Bauer)
#### Post date: [September 19, 2022, 6:57pm UTC](https://discourse.paraview.org/t/array-dispatch-issues/10328/13 "2022-09-19T18:57:57Z")

</div>

Sorry, yeah the array needs to have more than a single component. I didn’t notice that before. The single array component is the trivial case where SOA is essentially the same as AOS.

---

<div class="post-metadata">

### Author: ![cory.quammen](https://discourse.paraview.org/user_avatar/discourse.paraview.org/cory.quammen/32/11193_2.png) [@cory.quammen](https://discourse.paraview.org/u/cory.quammen)
#### Post date: [September 20, 2022, 10:31am UTC](https://discourse.paraview.org/t/array-dispatch-issues/10328/14 "2022-09-20T10:31:55Z")

</div>

Andy, I should mention here that SOA array support in VTK/ParaView is very limited. Buyer beware!

You’ll run into a lot of situations where filters just do not work because they are modifying a copy of the data buffer arranged in AOS order, which is done to support the `GetVoidPointer()` function. This _may_ be one of those situations.

---

<div class="post-metadata">

### Author: ![Andy\_Bauer](https://discourse.paraview.org/user_avatar/discourse.paraview.org/andy_bauer/32/5442_2.png) [@Andy\_Bauer](https://discourse.paraview.org/u/Andy_Bauer)
#### Post date: [September 20, 2022, 12:46pm UTC](https://discourse.paraview.org/t/array-dispatch-issues/10328/15 "2022-09-20T12:46:52Z")

</div>

This is very disconcerting then because it means that ParaView Catalyst support is potentially very limited as well. Essentially ParaView Catalyst users are forced to use AOS style arrays which may force them into doing deep copies of data. I suppose the `GetVoidPointer()` will do the deep copy anyway but only when called, not all of the time.

I suppose I would go as far as saying that array dispatch **just needs to work**. I’m not sure what the point of having it not working is. Even worse, from my experience discovering this bug is I see that when array dispatch fails the following is the **warning** :

```auto
( 1.854s) [pvbatch]vtkSOADataArrayTemplate:338 WARN| 23vtkSOADataArrayTemplateIdE (0x24bf270): GetVoidPointer called. This is very expensive for non-array-of-structs subclasses, as the scalar array must be generated for each call. Using the vtkGenericDataArray API with vtkArrayDispatch are preferred. Define the environment variable VTK_SILENCE_GET_VOID_POINTER_WARNINGS to silence this warning.

```

This should be an **error** letting the user know that something is wrong.

Another point of concern is that for Catalyst V2 API users they may not know they’re using AOS arrays.

---

<div class="post-metadata">

### Author: ![cory.quammen](https://discourse.paraview.org/user_avatar/discourse.paraview.org/cory.quammen/32/11193_2.png) [@cory.quammen](https://discourse.paraview.org/u/cory.quammen)
#### Post date: [October 2, 2022, 6:57pm UTC](https://discourse.paraview.org/t/array-dispatch-issues/10328/16 "2022-10-02T18:57:47Z")

</div>

> [@Andy\_Bauer](#):
>
> I suppose I would go as far as saying that array dispatch **just needs to work**.

The dispatch works just fine. The problem is that not all filters use array dispatch, but instead use the vtkTemplateMacro which relies on `GetVoidPointer()`. The work that needs to be done is to eliminate the need for `GetVoidPointer()`, but that will require modification of something like 600 VTK filters. In other words, it will be a massive undertaking to fully and efficiently support SOA arrays.

I agree it should all just work, but at the moment I wouldn’t suggest that anyone use SOA arrays. I have been burned by trying to do the same a couple times now.

---

<div class="post-metadata">

### Author: ![Andy\_Bauer](https://discourse.paraview.org/user_avatar/discourse.paraview.org/andy_bauer/32/5442_2.png) [@Andy\_Bauer](https://discourse.paraview.org/u/Andy_Bauer)
#### Post date: [October 3, 2022, 7:10pm UTC](https://discourse.paraview.org/t/array-dispatch-issues/10328/17 "2022-10-03T19:10:49Z")

</div>

My understanding is probably incomplete here but I thought that when `GetVoidPointer()` was called for an SOA data array that this was being “dispatched” in some way to be properly handled. If this is something else then this is what needs to have a working implementation because this operation that currently isn’t working and for hundreds of filters to not be working is a big concern for in situ. Even if this operation to make `GetVoidPointer()` function properly for SOA arrays is slow and memory intensive, at least it works instead of failing and even worse failing without proper error feedback.

Additionally, should `GetVoidPointer()` be deprecated? This way at least people will stop using the method when they develop internally in VTK.

---

<div class="post-metadata">

### Author: ![ben.boeckel](https://discourse.paraview.org/letter_avatar_proxy/v4/letter/b/ea5d25/32.png) [@ben.boeckel](https://discourse.paraview.org/u/ben.boeckel)
#### Post date: [October 30, 2022, 3:46pm UTC](https://discourse.paraview.org/t/array-dispatch-issues/10328/18 "2022-10-30T15:46:09Z")

</div>

> [@Andy\_Bauer](#):
>
> Additionally, should `GetVoidPointer()` be deprecated? This way at least people will stop using the method when they develop internally in VTK.

Ideally, yes. However, there’d have to be suppression in VTK itself for it without actually getting VTK itself off of it. And without that, there’s no actual sunset date for the method (usually “after the next release”).
