My first time trying to open a rather large ensight gold case file (~400 million hexa8 elements, 100 time steps, 1 scalar, 1 3D vector) did not go so well.
This is the first time I am having this issue, smaller cases (let’s say around 200 million elements, total file size was larger than the case I am having trouble with, due to higher number of time steps) worked like a charm.
What I did: open PV, open case file, click “Apply”.
PV 5.6 crashes with the following output to shell:
terminate called after throwing an instance of ‘std::bad_array_new_length’
what(): std::bad_array_new_length
Aborted (core dumped)
So I downloaded the latest version 5.9 and tried again, with -l=pv_debug.txt,TRACE
It still crashes, here is the output I get: pv_59_shell.txt (14.1 KB) pv_59_debug.txt (320.1 KB)
Did I do something wrong, or have I run into a software limitation?
I dug a little deeper. I am reasonably certain that my ensight geo file is fine according to the documentation.
I wrote a small routine that reads it, and does some plausibility checks.
However, running ens_checker tool on that file, it claims that some of the node indices are negative. Which is not the case according to my check routine. Weirdly enough, ens_checker reports the first error as:
For hexa8 element number 67108864:
connectivity element 6, is given as -144802628
This is beyond the number of nodes (373714618) in this part
67108864 elements x (8 nodes per element) x (4 byte per node) = 2147483648 byte = 2^31 byte.
In the documentation for ensight gold file format, I found no mention of such a limit. And I would expect that ens_checker would warn about the amount of elements being too high, if elements x nodes exceeds such a limit.
Any ideas would be welcome.
my export routine works correctly, at least according to the ensight gold file format documentation
the documentation mentions no element or node count limits with unformatted files. There are file size limits, but I am way below them
ens_checker fails as soon as one field is larger than 2^31 Byte. In my case the field with 8 node indices for each hexa8 element
Paraview is a bit more lenient, but fails as soon as (number of elements x number of nodes per element) exceeds 2^31. Even if the actual number of nodes is lower, due to many elements sharing the same nodes
I wasted a lot of time with ensight gold file format
The .geo file is about 14GiB. I don’t think I can upload that anywhere.
I might have some idea why ens_checker fails. The way I have to open the files in fortran for writing differs depending on where I want to use them. Found via trial&error:
open(output_unit,file=trim(adjustl(out_file)), status='replace', action='write', form='unformatted', access='stream') ! for paraview
open(output_unit,file=trim(adjustl(out_file)), status='replace', action='write', form='unformatted') ! for ens_checker
I guess the latter is record-based?, and the record length might be limited by a 32bit integer. Just a guess, not that I knew a whole lot about that. And I can read it just fine with my own program. But it’s besides the point anyway, I need Paraview to read it, and don’t care too much about ens_checker at this point.
hi @flotus1 , I am also facing the same issue with larges ensight gold files.
In my case however the size of elements are different.
I have two poly-hexcore meshes, one with ~70 million elements (nodes little less than 100 million) and the other with 140 million elements (nodes around 250 million).
I am using Linux and Paraview 5.4.1 version. The same python script works for 70M elements but crashes with bad_array_new_length error for 140M elements.
I can see total number of nodes is less than 2^31 but it still crashes.
Did you find a way to fix this or is there any other workaround for this ?
Thank you in advance
hi @bmk : I can kind of see why your limits might be different with polyhedral cells. The relevant limit for paraview seems to be the amount of “unmerged” vertices. I.e. PV treats the vertices of each element separately during import, resulting in a much larger amount of vertices. With hexa8 elements, I have 8 “unmerged” vertices per element. You probably have more due to polyhedral cells.
Unfortunately, I don’t know of any workaround for this bug that doesn’t involve switching to a different file format entirely.
Yeah, CGNS would be the obvious choice. Or VTK if that still ticks all your boxes.
I didn’t get around to changing my implementation to CGNS yet, but that’s my plan for the future.
Hello, for information the overflow issue in the Ensight Gold reader has been fixed in VTK and is now available in ParaView nightly builds (or next release)