Last year, VTK added a new image comparison mechanism: Enhancing VTK's Image Testing Framework: A Transition to SSIM and Lab Channels - Development - VTK
This mechanism was recently integrated in ParaView and can be used for better image comparison when testing in ParaView plugins and ParaView based applications.
Please note using this new mechanism is optional, you can keep using the “Legacy” comparison mechanism, which is not planned for deprecation for now.
1. CMake
The first step is to position a dedicated CMake variable in your ParaView plugin:
set(DEFAULT_USE_SSIM_IMAGE_COMP ON)
2. XML testing
In your testing XMLs, if you use pqcompareview instruction, make sure to remove or update the threshold attribute, as the range has changed:
<pqcompareview object="pqClientMainWindow/centralwidget/MultiViewWidget/CoreWidget/qt_tabwidget_stackedwidget/MultiViewWidget1/Container/Frame.0/CentralWidgetFrame/Viewport" baseline="$PARAVIEW_DATA_ROOT/Baseline/MyTest.png" threshold="5" width="1000" height="800" />
Could become:
<pqcompareview object="pqClientMainWindow/centralwidget/MultiViewWidget/CoreWidget/qt_tabwidget_stackedwidget/MultiViewWidget1/Container/Frame.0/CentralWidgetFrame/Viewport" baseline="$PARAVIEW_DATA_ROOT/Baseline/MyTest.png" threshold="0.005" width="1000" height="800" />
3. Python testing
Python testing has been reworked to use the same stack as XML testing, so the logic is the same.
The threshold attribute should be updated, eg:
Testing.compareImage(renderView1.GetRenderWindow(), baseline_file, threshold=50)
Could become:
Testing.compareImage(renderView1.GetRenderWindow(), baseline_file, threshold=0.005)
4. Wait, my test is failing now!
Indeed, the legacy comparison mechanism was not really good at detecting certain type of visual changes, which means that you may not have detected some changes in the rendering of your test.
Carefully check each failing baseline, some may be actual issue in your code that require fixes, in that case, it should be investigated and fixed.
Some may be false positive caused by changes in ParaView, eg: the recent change of default colormap and default background color:
- https://www.kitware.com/new-default-colormap-and-background-in-next-paraview-release/
- Replacement default color map and background palette
In that case, you may want to update the baselines.
5. Supporting multiple versions of ParaView
If your plugin or application is supporting multiple version of ParaView, you may have supplemental challenges:
pqcompareviewthresholdattribute: You need to remove it and rely on the default threshold- Python testing
thresholdattribute: You need to remove it and rely on the default threshold - Difference of rendering between versions: You can use the alternative baseline mechanism described here: https://www.paraview.org/paraview-docs/latest/cxx/md__builds_gitlab-kitware-sciviz-ci_Testing_README.html, or you can modify your test to not rely on defaults behavior of ParaView but instead forces these to specific values.