Finding differences between datasets


(Paul) #1

Hello,

I am having difficulties comparing a dataset from a finite volume simulation on an unstructured grid with the analytical solution that I computed in Python and wrote to csv-data format for comparison in ParaView. In principle I could do the whole thing in Python but the interpolation to get both datasets on the same grid is so slow there that it is not feasible (and actually very fast in ParaView).

Now I fully understand that I have to bring both the simulation- and the analytical data on the same mesh. I tried the Point Volume Interpolator to get the simulation data on a 3-D rectangular mesh, outlined by a box from -2.5…2.5 in all three dimensions and a resolution of 100 in all three directions. I did the same with the analytical result I computed with Python also from within ParaView. I grouped the two datasets together and see all the arrays I want to compare with one another (some scalar and vector fields, each vector component separately) available at that point in the pipeline. When I try (either with the Calculator or Python-Calculator) to compute something as simple as
abs(field1-field2)/abs(field2)
I get a bunch of vtk error-messages (see below) of which I cannot get a clue how to solve the problem. Actually, I get these errors already when, starting in the pipeline from the combined data set, in the calculator I simply try to pass an array, say field1 to a new result array name, so I am not even sure whether the problem is tied to the combination of the two datasets.

Thank you very much for your time!

ERROR: In /builddir/build/BUILD/ParaView-v5.4.1/VTK/Common/Misc/vtkFunctionParser.cxx, line 1244
vtkFunctionParser (0x5615972626a0): Syntax error: expecting a variable name;  see position 0

ERROR: In /builddir/build/BUILD/ParaView-v5.4.1/VTK/Common/Misc/vtkFunctionParser.cxx, line 1244
vtkFunctionParser (0x5615972626a0): Syntax error: expecting a variable name;  see position 0

Warning: In /builddir/build/BUILD/ParaView-v5.4.1/VTK/Filters/Core/vtkArrayCalculator.cxx, line 405
vtkPVArrayCalculator (0x561597244260): An error occurred when parsing the calculator's function.  See previous errors.
...

(Shawn Waldon) #2

Hi Paul,

How did you group the datasets together? The group datasets filter? If so you have a multiblock dataset with two datasets inside it. Each dataset has the arrays from one of your input datasets but neither has both. So each array in your calculator expression is only valid for half of your data. Even though they have the same structure they are still separate grids and you need both sets of arrays on the same grid.

You probably want something like Resample With Dataset or Resample To Image to combine your datasets into one grid with the data arrays from both.

HTH,
Shawn


(Cory Quammen (Kitware)) #3

@Paul If you have both data sets resampled onto the same mesh, I recommend using the Append Attributes filter to merge the different fields into the same data set.


(Paul) #4

Thank you Shawn and Cory very much for your fast replies and suggestions!

The option Append Attributes just as Resample to Image are grayed out when I select the two datasets on the same mesh simultaneously in the pipeline.

Also, when trying Shawn’s suggestion to resample e.g. the second dataset with the first one I get error messages (listed below).

My first dataset is a Multi-Block Dataset (because our plasma code divides the domain which I cannot avoid) which I grouped together (maybe appending already at this stage is the way to go?) and after computing some auxiliary quantities like vector components in curvilinear coordinates and applying the final Point Volume Interpolator, I get a Multi-Block Dataset with two Images (Uniform Rectangular Grid).

For the second dataset (analytical data) I also used the Point Volume Interpolator (it’s of course not multi-block) but a single Image (Uniform Rectilinear Grid) with the same X,Y,Z extents and bounds but only half the Number of Cells and half the Number of Points (which is maybe due to the multi-block structure of the first dataset, so probably counted twice here…)

However, as I said appending or resampling to image is not available when these two instances are selected in the pipeline together. The only available option is the filter “group datasets” that does not bring all arrays onto the same mesh as I need for calculating relative errors of our numerical method.

Thank you,
Paul

Error messages from Resample with Dataset:
ERROR: In /builddir/build/BUILD/ParaView-v5.4.1/VTK/Common/Misc/vtkFunctionParser.cxx, line 1244
vtkFunctionParser (0x561597d32880): Syntax error: expecting a variable name; see position 7

ERROR: In /builddir/build/BUILD/ParaView-v5.4.1/VTK/Common/Misc/vtkFunctionParser.cxx, line 1244
vtkFunctionParser (0x561597d32880): Syntax error: expecting a variable name; see position 7

Warning: In /builddir/build/BUILD/ParaView-v5.4.1/VTK/Filters/Core/vtkArrayCalculator.cxx, line 405
vtkPVArrayCalculator (0x561562f0fc00): An error occurred when parsing the calculator’s function. See previous errors.

ERROR: In /builddir/build/BUILD/ParaView-v5.4.1/VTK/Common/Misc/vtkFunctionParser.cxx, line 1244
vtkFunctionParser (0x561597d32880): Syntax error: operator expected; see position 3

ERROR: In /builddir/build/BUILD/ParaView-v5.4.1/VTK/Common/Misc/vtkFunctionParser.cxx, line 1244
vtkFunctionParser (0x561597d32880): Syntax error: operator expected; see position 3

Warning: In /builddir/build/BUILD/ParaView-v5.4.1/VTK/Filters/Core/vtkArrayCalculator.cxx, line 405
vtkPVArrayCalculator (0x561562f0fc00): An error occurred when parsing the calculator’s function. See previous errors.

ERROR: In /builddir/build/BUILD/ParaView-v5.4.1/VTK/Common/Misc/vtkFunctionParser.cxx, line 1244
vtkFunctionParser (0x561597d32880): Syntax error: operator expected; see position 3

ERROR: In /builddir/build/BUILD/ParaView-v5.4.1/VTK/Common/Misc/vtkFunctionParser.cxx, line 1244
vtkFunctionParser (0x561597d32880): Syntax error: operator expected; see position 3

Warning: In /builddir/build/BUILD/ParaView-v5.4.1/VTK/Filters/Core/vtkArrayCalculator.cxx, line 405
vtkPVArrayCalculator (0x561562f0fc00): An error occurred when parsing the calculator’s function. See previous errors.

ERROR: In /builddir/build/BUILD/ParaView-v5.4.1/VTK/Common/ExecutionModel/vtkExecutive.cxx, line 784
vtkPVCompositeDataPipeline (0x561597e95dd0): Algorithm vtkResampleWithDataSet(0x7f6c60033500) returned failure for request: vtkInformation (0x561595382f50)
Debug: Off
Modified Time: 86506784
Reference Count: 1
Registered Events: (none)
Request: REQUEST_DATA_OBJECT
FROM_OUTPUT_PORT: 0
ALGORITHM_AFTER_FORWARD: 1
FORWARD_DIRECTION: 0

ERROR: In /builddir/build/BUILD/ParaView-v5.4.1/VTK/Common/ExecutionModel/vtkExecutive.cxx, line 784
vtkPVCompositeDataPipeline (0x561597e95dd0): Algorithm vtkResampleWithDataSet(0x7f6c60033500) returned failure for request: vtkInformation (0x561595382f50)
Debug: Off
Modified Time: 86506784
Reference Count: 1
Registered Events: (none)
Request: REQUEST_DATA_OBJECT
FROM_OUTPUT_PORT: 0
ALGORITHM_AFTER_FORWARD: 1
FORWARD_DIRECTION: 0

Error messages from resample to image:
ERROR: In /builddir/build/BUILD/ParaView-v5.4.1/VTK/Common/Misc/vtkFunctionParser.cxx, line 1244
vtkFunctionParser (0x561597d32880): Syntax error: expecting a variable name; see position 7

ERROR: In /builddir/build/BUILD/ParaView-v5.4.1/VTK/Common/Misc/vtkFunctionParser.cxx, line 1244
vtkFunctionParser (0x561597d32880): Syntax error: expecting a variable name; see position 7

Warning: In /builddir/build/BUILD/ParaView-v5.4.1/VTK/Filters/Core/vtkArrayCalculator.cxx, line 405
vtkPVArrayCalculator (0x561562f0fc00): An error occurred when parsing the calculator’s function. See previous errors.

ERROR: In /builddir/build/BUILD/ParaView-v5.4.1/VTK/Common/Misc/vtkFunctionParser.cxx, line 1244
vtkFunctionParser (0x561597d32880): Syntax error: operator expected; see position 3

ERROR: In /builddir/build/BUILD/ParaView-v5.4.1/VTK/Common/Misc/vtkFunctionParser.cxx, line 1244
vtkFunctionParser (0x561597d32880): Syntax error: operator expected; see position 3

Warning: In /builddir/build/BUILD/ParaView-v5.4.1/VTK/Filters/Core/vtkArrayCalculator.cxx, line 405
vtkPVArrayCalculator (0x561562f0fc00): An error occurred when parsing the calculator’s function. See previous errors.

ERROR: In /builddir/build/BUILD/ParaView-v5.4.1/VTK/Common/Misc/vtkFunctionParser.cxx, line 1244
vtkFunctionParser (0x561597d32880): Syntax error: operator expected; see position 3

ERROR: In /builddir/build/BUILD/ParaView-v5.4.1/VTK/Common/Misc/vtkFunctionParser.cxx, line 1244
vtkFunctionParser (0x561597d32880): Syntax error: operator expected; see position 3

Warning: In /builddir/build/BUILD/ParaView-v5.4.1/VTK/Filters/Core/vtkArrayCalculator.cxx, line 405
vtkPVArrayCalculator (0x561562f0fc00): An error occurred when parsing the calculator’s function. See previous errors.

ERROR: In /builddir/build/BUILD/ParaView-v5.4.1/VTK/Common/ExecutionModel/vtkExecutive.cxx, line 784
vtkPVCompositeDataPipeline (0x561597e95dd0): Algorithm vtkResampleWithDataSet(0x7f6c60033500) returned failure for request: vtkInformation (0x561595382f50)
Debug: Off
Modified Time: 86506784
Reference Count: 1
Registered Events: (none)
Request: REQUEST_DATA_OBJECT
FROM_OUTPUT_PORT: 0
ALGORITHM_AFTER_FORWARD: 1
FORWARD_DIRECTION: 0

ERROR: In /builddir/build/BUILD/ParaView-v5.4.1/VTK/Common/ExecutionModel/vtkExecutive.cxx, line 784
vtkPVCompositeDataPipeline (0x561597e95dd0): Algorithm vtkResampleWithDataSet(0x7f6c60033500) returned failure for request: vtkInformation (0x561595382f50)
Debug: Off
Modified Time: 86506784
Reference Count: 1
Registered Events: (none)
Request: REQUEST_DATA_OBJECT
FROM_OUTPUT_PORT: 0
ALGORITHM_AFTER_FORWARD: 1
FORWARD_DIRECTION: 0

ERROR: In /builddir/build/BUILD/ParaView-v5.4.1/VTK/Common/ExecutionModel/vtkExecutive.cxx, line 784
vtkPVCompositeDataPipeline (0x561597e95dd0): Algorithm vtkResampleWithDataSet(0x7f6c60033500) returned failure for request: vtkInformation (0x561595382f50)
Debug: Off
Modified Time: 86506784
Reference Count: 1
Registered Events: (none)
Request: REQUEST_DATA_OBJECT
FROM_OUTPUT_PORT: 0
ALGORITHM_AFTER_FORWARD: 1
FORWARD_DIRECTION: 0

ERROR: In /builddir/build/BUILD/ParaView-v5.4.1/VTK/Common/ExecutionModel/vtkExecutive.cxx, line 784
vtkPVCompositeDataPipeline (0x561597e95dd0): Algorithm vtkResampleWithDataSet(0x7f6c60033500) returned failure for request: vtkInformation (0x561595382f50)
Debug: Off
Modified Time: 86506784
Reference Count: 1
Registered Events: (none)
Request: REQUEST_DATA_OBJECT
FROM_OUTPUT_PORT: 0
ALGORITHM_AFTER_FORWARD: 1
FORWARD_DIRECTION: 0

ERROR: In /builddir/build/BUILD/ParaView-v5.4.1/VTK/Common/ExecutionModel/vtkExecutive.cxx, line 784
vtkPVCompositeDataPipeline (0x561597e95dd0): Algorithm vtkResampleWithDataSet(0x7f6c60033500) returned failure for request: vtkInformation (0x561595382f50)
Debug: Off
Modified Time: 86506784
Reference Count: 1
Registered Events: (none)
Request: REQUEST_DATA_OBJECT
FROM_OUTPUT_PORT: 0
ALGORITHM_AFTER_FORWARD: 1
FORWARD_DIRECTION: 0


(Paul) #5

Hi again, Shawn

Thank you for your help! I finally got it working after using Append Datasets to the MultiBlock DataSet.

Best,
Paul


(Manaf Muhammed) #6

please tell me how to append datasets or append attributes


(Manaf Muhammed) #7

hi
i think for openFoam files you nedd to do mergeblocks seperately for calculator 1 and calculator 2
then you will get append attributes