Error loading VTKHDF unstructured grid

ParaView Support
1

I am tryinv to use VTKHDF format and I have create a sample file to test it, but paraview is giving errors when trying to read it:

( 113.892s) [paraview        ]       vtkExecutive.cxx:729    ERR| vtkPVCompositeDataPipeline (0x7073a80): Algorithm vtkFileSeriesReader (0x71770e0) returned failure for request: vtkInformation (0x7073dc0)                                                                                                                
  Debug: Off                                                                                                                                                                                                                                                                                                                
  Modified Time: 513016                                                                                                                                                                                                                                                                                                     
  Reference Count: 1                                                                                                                                                                                                                                                                                                        
  Registered Events: (none)                                                                                                                                                                                                                                                                                                 
  Request: REQUEST_DATA_OBJECT                                                                                                                                                                                                                                                                                              
  ALGORITHM_AFTER_FORWARD: 1                                                                                                                                                                                                                                                                                                
  FORWARD_DIRECTION: 0                                                                                                                                                                                                                                                                                                      
                                                                                                                                                                                                                                                                                                                            
                                                                                                                                                                                                                                                                                                                            
( 113.945s) [paraview        ]vtkHDFReaderImplementat:323    ERR| vtkHDFReader (0x69eb5c0): Cannot get number of steps if the group is not open                                                                                                                                                                             
( 113.945s) [paraview        ]       vtkHDFReader.cxx:526    ERR| vtkHDFReader (0x69eb5c0): HDF dataset type unknown: -1                                                                                                                                                                                                    
( 113.945s) [paraview        ]       vtkExecutive.cxx:729    ERR| vtkPVCompositeDataPipeline (0x7073a80): Algorithm vtkFileSeriesReader (0x71770e0) returned failure for request: vtkInformation (0x7073dc0)                                                                                                                
  Debug: Off                                                                                                                                                                                                                                                                                                                
  Modified Time: 513016                                                                                                                                                                                                                                                                                                     
  Reference Count: 1                                                                                                                                                                                                                                                                                                        
  Registered Events: (none)                                                                                                                                                                                                                                                                                                 
  Request: REQUEST_DATA_OBJECT                                                                                                                                                                                                                                                                                              
  ALGORITHM_AFTER_FORWARD: 1                                                                                                                                                                                                                                                                                                
  FORWARD_DIRECTION: 0                                                                                                                                                                                                                                                                                                      
                                                                                                                                                                                                                                                                                                                            
                                                                                                                                                                                                                                                                                                                            
( 113.949s) [paraview        ]vtkHDFReaderImplementat:323    ERR| vtkHDFReader (0x69eb5c0): Cannot get number of steps if the group is not open
( 113.949s) [paraview        ]       vtkHDFReader.cxx:526    ERR| vtkHDFReader (0x69eb5c0): HDF dataset type unknown: -1
( 113.949s) [paraview        ]       vtkExecutive.cxx:729    ERR| vtkPVCompositeDataPipeline (0x7073a80): Algorithm vtkFileSeriesReader (0x71770e0) returned failure for request: vtkInformation (0x7073dc0)
  Debug: Off
  Modified Time: 513016
  Reference Count: 1
  Registered Events: (none)
  Request: REQUEST_DATA_OBJECT
  ALGORITHM_AFTER_FORWARD: 1
  FORWARD_DIRECTION: 0

...

Is there something wrong in my file?

This is the file I am trying to load
geodesic.0.hdf (97.2 KB)

2

I confirm that your file is not correctly written. Please read https://www.kitware.com/how-to-write-time-dependent-data-in-vtkhdf-files/ and VTK: VTK HDF File Format

FYI @Lucas_Givord

3

Could you point me where the file is wrong. I am unable to find it. I have read the documentation before writing my code, and I have read it again, but I can’t find the issue.

I have seen that the Points dataset must be an 1D dataset instead of 2D, but even with that changes, its still giving the same errors

4

I have checked again and I found that I am missing the VTKHDF initial group. I thought that the initial group was directly the file, but it looks like we need to write all the data below /VTKHDF, and not directly below /. Probably that could be make more clear in the documentation.

In any case, I still get the same errors…

5

I have finally managed to open my file. So there were several issues. I will write the issues and some thoughts as a user/developer:

First of all, the Type attribute in VTKHDF must be an ASCII. This is in the documentation, but I clearly didn’t read it. However, I find it a bit unconvenient. I think I would be better to have an enum, the same way there is the vtkCellType enum. 0: Image Data, 1: Unstructured grid…

Second, the data type for the datasets is not clear. I was writing them as unsigned int, just to get some errors while reading the file with the VTKHDF reader in ParaView. I haven’t found anywhere where the data type expected for each dataset is documented.

6

I have finally managed to open my file. So there were several issues. I will write the issues and some thoughts as a user/developer:

Great!

First of all, the Type attribute in VTKHDF must be an ASCII. This is in the documentation, but I clearly didn’t read it. However, I find it a bit unconvenient. I think I would be better to have an enum, the same way there is the vtkCellType enum. 0: Image Data, 1: Unstructured grid…

We matched the other VTK formats and also it is self documented. But numbers would have been fine as well.

Second, the data type for the datasets is not clear. I was writing them as unsigned int, just to get some errors while reading the file with the VTKHDF reader in ParaView. I haven’t found anywhere where the data type expected for each dataset is documented.

Is the third paragraph from the following link answering your question?

https://examples.vtk.org/site/VTKFileFormats/#hdf-file-formats

7

Not really. In that paragraph it says, at least as I understand, that datasets will be converted to VTK arrays matching the underlying data type.

However, Types array must be unsigned char, according to paraview error if you use another data type. Connectivity and Offset arrays must be signed integers, I have tested unsigned int and int and only the int was working. Not sure if it also works with long int. AFAIK, this is not documented for the VTKHDF format.

8

I see, yes you are right. Thanks for the report. I’ll update the documentation.

You can also check the examples at the end of the document
https://examples.vtk.org/site/VTKFileFormats/#examples
and the tests in VTK for actual data that works with the readers.

9

Yes, I actually had to look into those examples to find that my offsets/connectivity ids should be signed integers instead of unsigned. Not sure if long or even short signed integers will work. Would be nice if the documentation specifies this.

I would consider this topic closed, as the issues are solved. However, I would like to ask also if there is any estimation of when VTKHDF will support hierarchical datasets (vtkMultiBlockDataset for example).

10

This is the design:
https://discourse.vtk.org/t/composite-data-sets-for-the-vtkhdf-format/12190

@Mathieu Westphal Any plans to continue this work?

11

If the development is stalled, I guess I could contribute. I recently made a HDF reader for my own format, similar to what VTKHDF provides, where the groups hierarchy was converted into a vtkMultiBlockDataSet.

12

Hi @Juan_Jose_Casafranca @danlipsa,

I am precisely working on it, I would tell you when the MR will be ready if you want to review it :slightly_smiling_face:

Also, thanks for all of theses feedback if you can ping me directly on each issue that you will open, it would be great

13

In which repo should I posted the bugs I find? VTK or ParaView? I think I have found a problem regarding the Type string

14

as vtkHDF is a file format specific to vtk, I think the best place for these issues should be in vtk https://gitlab.kitware.com/vtk/vtk/-/issues

Thanks!