Error reading large VTK files in the Windows version

Which compilers were used?

I do not know. Both versions were downloaded precompiled. Do you think building the Windows version from source would fix this? Which compiler do you suggest?

Which format are your partitions ?

ext4, but even when I copy the domain file to different ntfs partition the error remains.

I would put data and ParaView on ntfs partition and check again.

OK to explain.

  1. Linux ParaView is on ext4.
  2. Our native Linux CFD solver is on ext4 and the results directory is on ext4.
  3. Linux software and partitions are running and mounted under WSL2 on Windows 11.
  4. Windows ParaView is on an ntfs partition on the same Windows 11 machine that is running WSL2

Linux ParaView can read and visualize the results data.

Windows ParaView throws up errors with the large domain file (the 3.5 Gb one) when I try to read it from the ext4, or when I copy this file to an ntfs partition. Both files show the same error reading the file and the result cannot be visualized.

Windows ParaView can read the smaller 162 Mb file both from ext4 and the ntfs partitions.

My initial thought, which might be completely erroneous, was that long in Linux is 8 bytes whereas it’s 4 bytes in Windows. Maybe that range is being exceeded.

I was going to suggest compilling with clang for both systems to rule out a compiler difference. But it’s probably faster to scan the code for a long type.

Thank you for this. Unfortunately this is beyond my skill level. I am hoping that someone from Kitware would flag this issue and propose a fix.

I suggest you raise an issue here https://gitlab.kitware.com/paraview/paraview/-/issues with a link to the (compressed) 3.5Gb VTK data file which causes the problem.

Done, thank you for the link. I looked for it on GitHub, not realising they are hosting the code with issues tracking on GitLab.

Is it not possible to zip that file? I certainly won’t be pulling down 3.5Gb.

:slight_smile: Let me try

It is now 2.48Gb: https://velocite-public.s3.fr-par.scw.cloud/domain_00010000.zip?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=SCW4RPBSRG82A0JQQSTB%2F20211019%2Ffr-par%2Fs3%2Faws4_request&X-Amz-Date=20211019T054859Z&X-Amz-Expires=3596&X-Amz-Signature=f9ed586080046ca85380cfe70218dc5c1df575efba945601c56462b31814dd4d&X-Amz-SignedHeaders=host

What’s the difference between the 162Mb file and this one? Apart from size.

Both files are created by the same CFD solver so there should be no differences besides the size.

So basically just a larger physical domain rather than more time steps?

Yes, it is the identical domain. The smaller file represents the results on the surface of the object, while the domain contains the results for the volume outside of the object.

1 Like

@vmajor This link has expired. Please repost it (preferably 7zip compressed) and I will have a look at it over xmas.

Thank you Todd!

Here is a fresh 6 day link: https://velocite-public.s3.fr-par.scw.cloud/domain_00010000.zip?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=SCW7BRXPRJM3D3G7DZKG%2F20211221%2Ffr-par%2Fs3%2Faws4_request&X-Amz-Date=20211221T112852Z&X-Amz-Expires=521990&X-Amz-Signature=713f4ebeaf7627ac9c3287dbba20a4287fc046d84846e358e3ee353ef7bef248&X-Amz-SignedHeaders=host

Hi @vmajor

A type conversion error is happening in the XML parser when the byte count passes 2^31 on Windows which results in a negative byte index. Here’s a quick and dirty fix for you to use, assuming you can do a local build, while a more rigorous solution is worked out. It won’t get you past the 4Gb file size limit on Windows but it will load past the 2Gb point. I hope that helps.

vtkTypeInt64 vtkXMLParser::GetXMLByteIndex()
{
  XML_Parser parser = static_cast<XML_Parser>(this->Parser);
#ifdef _WIN32
  unsigned long result = XML_GetCurrentByteIndex(parser);
  return result;
#else
  return XML_GetCurrentByteIndex(parser);
#endif 
}
1 Like