jfavre
(Jean M. Favre)
June 5, 2025, 7:29am
1
Dear Support
I have converted a time-series of data with 10641 timesteps, for which I have already created an animation, so I know that I have valid data. I saved the data with the command
SaveData(‘/capstor/scratch/cscs/jfavre/xxx/xxx.vtkhdf’, proxy=reader, WriteAllTimeSteps=1)
there was no error shown while saving the data.
When reading the data back using the new format, all timesteps are good up until timestep 6740, and then, without any error messages, all timesteps show data filled with 0. I have tried reading it with v5.13 and 6.0-RC1
The data is rather large 114G, so not easy to share. The structure is:
/ Group
/VTKHDF Group
/VTKHDF/Connectivity Dataset {815616/Inf}
/VTKHDF/NumberOfCells Dataset {10641/Inf}
/VTKHDF/NumberOfConnectivityIds Dataset {10641/Inf}
/VTKHDF/NumberOfPoints Dataset {10641/Inf}
/VTKHDF/Offsets Dataset {203905/Inf}
/VTKHDF/PointData Group
/VTKHDF/PointData/acceleration Dataset {3390222600/Inf, 3}
/VTKHDF/PointData/displacement Dataset {3390222600/Inf, 3}
/VTKHDF/PointData/velocity Dataset {3390222600/Inf, 3}
/VTKHDF/Points Dataset {318600/Inf, 3}
/VTKHDF/Steps Group
/VTKHDF/Steps/CellOffsets Dataset {10641/Inf}
/VTKHDF/Steps/ConnectivityIdOffsets Dataset {10641/Inf}
/VTKHDF/Steps/PartOffsets Dataset {10641/Inf}
/VTKHDF/Steps/PointDataOffsets Group
/VTKHDF/Steps/PointDataOffsets/acceleration Dataset {10641/Inf}
/VTKHDF/Steps/PointDataOffsets/displacement Dataset {10641/Inf}
/VTKHDF/Steps/PointDataOffsets/velocity Dataset {10641/Inf}
/VTKHDF/Steps/PointOffsets Dataset {10641/Inf}
/VTKHDF/Steps/Values Dataset {10641}
/VTKHDF/Types Dataset {203904/Inf}
is there some known limitation, with number of timesteps, or size, or whatever?
Jean M. Favre:
6740
Hello Jean, curiously 6740/10641 is nearly equal to 2147483647 (int32 limit)/3390222600… that sounds like we use an int32 for indexing somewhere we should be using an int64
jfavre
(Jean M. Favre)
June 6, 2025, 6:53am
3
Louis Gombert:
3390222600
Good math. Can it be fixed before the 6.0 release?
Could you please first make sure that:
VTK_USE_64BIT_IDS
is turned on for your ParaView build
All of your offset arrays, including those in the Steps
group, are in fact int64
jfavre
(Jean M. Favre)
June 6, 2025, 7:11am
5
grep VTK_USE_64BIT_IDS CMakeCache.txt
VTK_USE_64BIT_IDS:BOOL=ON
//ADVANCED property for variable: VTK_USE_64BIT_IDS
VTK_USE_64BIT_IDS-ADVANCED:INTERNAL=1
What is the output of h5dump -H
on your file?
jfavre
(Jean M. Favre)
June 6, 2025, 2:27pm
7
HDF5 “/capstor/scratch/cscs/jfavre/Salvus/salvus.vtkhdf” {
GROUP “/” {
GROUP “VTKHDF” {
ATTRIBUTE “Type” {
DATATYPE H5T_STRING {
STRSIZE 16;
STRPAD H5T_STR_NULLTERM;
CSET H5T_CSET_ASCII;
CTYPE H5T_C_S1;
}
DATASPACE SCALAR
}
ATTRIBUTE “Version” {
DATATYPE H5T_STD_I64LE
DATASPACE SIMPLE { ( 2 ) / ( 2 ) }
}
DATASET “Connectivity” {
DATATYPE H5T_STD_I64LE
DATASPACE SIMPLE { ( 815616 ) / ( H5S_UNLIMITED ) }
}
DATASET “NumberOfCells” {
DATATYPE H5T_STD_I64LE
DATASPACE SIMPLE { ( 10641 ) / ( H5S_UNLIMITED ) }
}
DATASET “NumberOfConnectivityIds” {
DATATYPE H5T_STD_I64LE
DATASPACE SIMPLE { ( 10641 ) / ( H5S_UNLIMITED ) }
}
DATASET “NumberOfPoints” {
DATATYPE H5T_STD_I64LE
DATASPACE SIMPLE { ( 10641 ) / ( H5S_UNLIMITED ) }
}
DATASET “Offsets” {
DATATYPE H5T_STD_I64LE
DATASPACE SIMPLE { ( 203905 ) / ( H5S_UNLIMITED ) }
}
GROUP “PointData” {
DATASET “acceleration” {
DATATYPE H5T_IEEE_F32LE
DATASPACE SIMPLE { ( 3390222600, 3 ) / ( H5S_UNLIMITED, 3 ) }
}
DATASET “displacement” {
DATATYPE H5T_IEEE_F32LE
DATASPACE SIMPLE { ( 3390222600, 3 ) / ( H5S_UNLIMITED, 3 ) }
}
DATASET “velocity” {
DATATYPE H5T_IEEE_F32LE
DATASPACE SIMPLE { ( 3390222600, 3 ) / ( H5S_UNLIMITED, 3 ) }
}
}
DATASET “Points” {
DATATYPE H5T_IEEE_F32LE
DATASPACE SIMPLE { ( 318600, 3 ) / ( H5S_UNLIMITED, 3 ) }
}
GROUP “Steps” {
ATTRIBUTE “NSteps” {
DATATYPE H5T_STD_I64LE
DATASPACE SCALAR
}
DATASET “CellOffsets” {
DATATYPE H5T_STD_I64LE
DATASPACE SIMPLE { ( 10641 ) / ( H5S_UNLIMITED ) }
}
DATASET “ConnectivityIdOffsets” {
DATATYPE H5T_STD_I64LE
DATASPACE SIMPLE { ( 10641 ) / ( H5S_UNLIMITED ) }
}
DATASET “PartOffsets” {
DATATYPE H5T_STD_I64LE
DATASPACE SIMPLE { ( 10641 ) / ( H5S_UNLIMITED ) }
}
GROUP “PointDataOffsets” {
DATASET “acceleration” {
DATATYPE H5T_STD_I64LE
DATASPACE SIMPLE { ( 10641 ) / ( H5S_UNLIMITED ) }
}
DATASET “displacement” {
DATATYPE H5T_STD_I64LE
DATASPACE SIMPLE { ( 10641 ) / ( H5S_UNLIMITED ) }
}
DATASET “velocity” {
DATATYPE H5T_STD_I64LE
DATASPACE SIMPLE { ( 10641 ) / ( H5S_UNLIMITED ) }
}
}
DATASET “PointOffsets” {
DATATYPE H5T_STD_I64LE
DATASPACE SIMPLE { ( 10641 ) / ( H5S_UNLIMITED ) }
}
DATASET “Values” {
DATATYPE H5T_IEEE_F32LE
DATASPACE SIMPLE { ( 10641 ) / ( 10641 ) }
}
}
DATASET “Types” {
DATATYPE H5T_STD_U8LE
DATASPACE SIMPLE { ( 203904 ) / ( H5S_UNLIMITED ) }
}
}
}
}