Error reading large VTK files in the Windows version

ParaView Support
1

I have large (3.5 Gb+) VTK files generated by our CFD software running on Ubuntu (WSL2 in Windows 11 in fact).

Paraview 5.10.0-RC1 running on Ubuntu WSL2 opens the file correctly, albeit the Nvidia virtualised driver tends to crash so I cannot actually use this method. Previously using previous versions of Paraview I successfully worked with the results files in native Ubuntu. The point is that our results files are not corrupted or faulty.

Paraview 5.10.0-RC1 running on Windows 11 while opening the same files reports the errors previously reported by others in other threads:

ERROR: In C:\glr\builds\paraview\paraview-ci\build\superbuild\paraview\src\VTK\IO\XMLParser\vtkXMLDataParser.cxx, line 655
vtkXMLDataParser (0000021A1A785910): Error reading uncompressed binary data header.  Read 0 of 4 bytes.

ERROR: In C:\glr\builds\paraview\paraview-ci\build\superbuild\paraview\src\VTK\IO\XML\vtkXMLStructuredDataReader.cxx, line 345
vtkXMLImageDataReader (0000021A067EEAB0): Error reading extent 1 597 1 309 1 285 from piece 0

ERROR: In C:\glr\builds\paraview\paraview-ci\build\superbuild\paraview\src\VTK\IO\XML\vtkXMLDataReader.cxx, line 410
vtkXMLImageDataReader (0000021A067EEAB0): Cannot read point data array "vorticityNorm" from PointData in piece 0.  The data array in the element may be too short.

ERROR: In C:\glr\builds\paraview\paraview-ci\build\superbuild\paraview\src\VTK\IO\XMLParser\vtkXMLDataParser.cxx, line 655
vtkXMLDataParser (0000021A1A785910): Error reading uncompressed binary data header.  Read 0 of 4 bytes.

ERROR: In C:\glr\builds\paraview\paraview-ci\build\superbuild\paraview\src\VTK\IO\XML\vtkXMLStructuredDataReader.cxx, line 345
vtkXMLImageDataReader (0000021A067EEAB0): Error reading extent 1 597 1 309 1 285 from piece 0

ERROR: In C:\glr\builds\paraview\paraview-ci\build\superbuild\paraview\src\VTK\IO\XML\vtkXMLDataReader.cxx, line 410
vtkXMLImageDataReader (0000021A067EEAB0): Cannot read point data array "vorticityNorm" from PointData in piece 0.  The data array in the element may be too short.

ERROR: In C:\glr\builds\paraview\paraview-ci\build\superbuild\paraview\src\VTK\IO\XMLParser\vtkXMLDataParser.cxx, line 655
vtkXMLDataParser (0000021A1A785910): Error reading uncompressed binary data header.  Read 0 of 4 bytes.

ERROR: In C:\glr\builds\paraview\paraview-ci\build\superbuild\paraview\src\VTK\IO\XML\vtkXMLStructuredDataReader.cxx, line 345
vtkXMLImageDataReader (0000021A067EEAB0): Error reading extent 1 597 1 309 1 285 from piece 0

ERROR: In C:\glr\builds\paraview\paraview-ci\build\superbuild\paraview\src\VTK\IO\XML\vtkXMLDataReader.cxx, line 410
vtkXMLImageDataReader (0000021A067EEAB0): Cannot read point data array "vorticityNorm" from PointData in piece 0.  The data array in the element may be too short.

ERROR: In C:\glr\builds\paraview\paraview-ci\build\superbuild\paraview\src\VTK\IO\XMLParser\vtkXMLDataParser.cxx, line 655
vtkXMLDataParser (0000021A1A785910): Error reading uncompressed binary data header.  Read 0 of 4 bytes.

ERROR: In C:\glr\builds\paraview\paraview-ci\build\superbuild\paraview\src\VTK\IO\XML\vtkXMLStructuredDataReader.cxx, line 345
vtkXMLImageDataReader (0000021A067EEAB0): Error reading extent 1 597 1 309 1 285 from piece 0

ERROR: In C:\glr\builds\paraview\paraview-ci\build\superbuild\paraview\src\VTK\IO\XML\vtkXMLDataReader.cxx, line 410
vtkXMLImageDataReader (0000021A067EEAB0): Cannot read point data array "vorticityNorm" from PointData in piece 0.  The data array in the element may be too short.

ERROR: In C:\glr\builds\paraview\paraview-ci\build\superbuild\paraview\src\VTK\IO\XMLParser\vtkXMLDataParser.cxx, line 655
vtkXMLDataParser (0000021A1A785910): Error reading uncompressed binary data header.  Read 0 of 4 bytes.

ERROR: In C:\glr\builds\paraview\paraview-ci\build\superbuild\paraview\src\VTK\IO\XML\vtkXMLStructuredDataReader.cxx, line 345
vtkXMLImageDataReader (0000021A067EEAB0): Error reading extent 1 597 1 309 1 285 from piece 0

ERROR: In C:\glr\builds\paraview\paraview-ci\build\superbuild\paraview\src\VTK\IO\XML\vtkXMLDataReader.cxx, line 410
vtkXMLImageDataReader (0000021A067EEAB0): Cannot read point data array "vorticityNorm" from PointData in piece 0.  The data array in the element may be too short.

Thus, this is a Windows build specific problem. Please advise if this is known, recognised and if there is a solution or a workaround on the way.

2

Some more information.

Windows version is able to open a 162 Mb VTK file without errors, but as mentioned above it fails with the 3.5 Gb VTK file.

Thus it seems to have some hard coded file size limit where it apparently truncates it to some size that it can handle. This of course does not work.

Linux version running in WSL2 Ubuntu 20.04 on the same workstation has no problems with opening the same files.

3

Which compilers were used?

4

I do not know. Both versions were downloaded precompiled. Do you think building the Windows version from source would fix this? Which compiler do you suggest?

5

Which format are your partitions ?

6

ext4, but even when I copy the domain file to different ntfs partition the error remains.

7

I would put data and ParaView on ntfs partition and check again.

8

OK to explain.

  1. Linux ParaView is on ext4.
  2. Our native Linux CFD solver is on ext4 and the results directory is on ext4.
  3. Linux software and partitions are running and mounted under WSL2 on Windows 11.
  4. Windows ParaView is on an ntfs partition on the same Windows 11 machine that is running WSL2

Linux ParaView can read and visualize the results data.

Windows ParaView throws up errors with the large domain file (the 3.5 Gb one) when I try to read it from the ext4, or when I copy this file to an ntfs partition. Both files show the same error reading the file and the result cannot be visualized.

Windows ParaView can read the smaller 162 Mb file both from ext4 and the ntfs partitions.

9

My initial thought, which might be completely erroneous, was that long in Linux is 8 bytes whereas it’s 4 bytes in Windows. Maybe that range is being exceeded.

I was going to suggest compilling with clang for both systems to rule out a compiler difference. But it’s probably faster to scan the code for a long type.

10

Thank you for this. Unfortunately this is beyond my skill level. I am hoping that someone from Kitware would flag this issue and propose a fix.

11

I suggest you raise an issue here https://gitlab.kitware.com/paraview/paraview/-/issues with a link to the (compressed) 3.5Gb VTK data file which causes the problem.

12

Done, thank you for the link. I looked for it on GitHub, not realising they are hosting the code with issues tracking on GitLab.

13

Is it not possible to zip that file? I certainly won’t be pulling down 3.5Gb.

14

:slight_smile: Let me try

15

It is now 2.48Gb: https://velocite-public.s3.fr-par.scw.cloud/domain_00010000.zip?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=SCW4RPBSRG82A0JQQSTB%2F20211019%2Ffr-par%2Fs3%2Faws4_request&X-Amz-Date=20211019T054859Z&X-Amz-Expires=3596&X-Amz-Signature=f9ed586080046ca85380cfe70218dc5c1df575efba945601c56462b31814dd4d&X-Amz-SignedHeaders=host

16

What’s the difference between the 162Mb file and this one? Apart from size.

17

Both files are created by the same CFD solver so there should be no differences besides the size.

18

So basically just a larger physical domain rather than more time steps?

19

Yes, it is the identical domain. The smaller file represents the results on the surface of the object, while the domain contains the results for the volume outside of the object.

1 Like
20

@vmajor This link has expired. Please repost it (preferably 7zip compressed) and I will have a look at it over xmas.