I have ADIOS2 files written using the ADIOS2 BP5 engine.
However, Paraview 5.12.0-RC1 can’t read them. It gives this error:
ERROR: In vtkADIOS2CoreImageReader.cxx, line 397
vtkADIOS2CoreImageReader (0000024CCA07C450): failed to open and read meta data: e[1;36m[Sun Dec 17 12:51:35 2023]e[1;34m [ADIOS2 EXCEPTION]e[0m <Core> <IO> <Operator> : ERROR: this version didn't compile with BP5 library, can't use BP5 engine
e[0m
ERROR: In vtkADIOS2CoreImageReader.cxx, line 431
vtkADIOS2CoreImageReader (0000024CCA07C450): unable to open file and data
ERROR: In vtkExecutive.cxx, line 730
vtkPVCompositeDataPipeline (0000024CCA2D4840): Algorithm vtkADIOS2CoreImageReader (0000024CCA07C450) returned failure for request: vtkInformation (0000024CD2D53E10)
Debug: Off
Modified Time: 444176
Reference Count: 1
Registered Events: (none)
Request: REQUEST_INFORMATION
FORWARD_DIRECTION: 0
ALGORITHM_AFTER_FORWARD: 1
I assume this is a bug. Is there a version of Paraview that was compiled with BP5 support?
Hmm that’s weird. So I just tested out the 5.12-RC1 binary for Mac and was able to open a BP5 file with the ADIOS2CoreImageReader without problem, so the ADIOS used to build the binaries do have BP5 support turned on.
What platform are you on? What version of ADIOS2 did you use to write the files?
I’ve definitely seen misleading error messages from ADIOS at times (like in the issue Lucas linked), so it seems that something else is probably happening but it’s not giving an error message at the real failure. If you can share the file, I can investigate it with a debugger.
Thanks! I was able to open it on my Mac, so it is just a Windows issue. I looked in a build log and it seems that for some reason on Windows BP5 support isn’t being built, though I’m not sure why yet. As far as I can tell we’re not explicitly disabling it in our configuration. I know it took the ADIOS team a bit of time to get BP5 supported on Windows, but we’re using ADIOS2.9.2 when building ParaView, so we should be able to build it with that support. I’m guessing that our Windows machines must be missing a dependency needed to enable BP5 support so the CMake configuration is automatically silently turning it off since we don’t explicitly turn on that option. I’ll try to get it fixed in time to go into RC2.
I tried opening the same dataset on my Mac. Each individual file is opened properly, but when I open it as a timeseries, the time slider shows spurious times and timesteps, and does not allow me to examine anything except the first timestep/ADIOS2 iteration.
I get this error message:
ERROR: In vtkADIOS2CoreImageReader.cxx, line 505
vtkADIOS2CoreImageReader (0x291e1c490): The requested time step 1 is not available!
ERROR: In vtkExecutive.cxx, line 730
vtkPVCompositeDataPipeline (0x600001a0c600): Algorithm vtkADIOS2CoreImageReader (0x291e1c490) returned failure for request: vtkInformation (0x60000c2ad2c0)
Debug: Off
Modified Time: 762121
Reference Count: 1
Registered Events: (none)
Request: REQUEST_DATA
FROM_OUTPUT_PORT: 0
ALGORITHM_AFTER_FORWARD: 1
FORWARD_DIRECTION: 0
I’ve attached the full timeseries (only 2 timesteps/iterations) that I used to test this here: adios2_bp5_files.tar.gz (2.7 MB)
So as far as I can tell by digging around in ADIOS CMake files, BP5 isn’t available for Windows in 2.9.2. Did you write the file on a non-Windows system? I originally assumed it was written on Windows, but I want to double check.
So I think a 2.10 release is supposed to happen soon. The person who does the ADIOS2 releases is out today, but I’ll check with him tomorrow on when that is supposed to be. Anyway once that happens and we update ParaView to build with that version, you could download a nightly on Windows.
As for the time error, I did recently find some bugs in this reader. Those fixes will be in the next RC. I’ve got a release build going right now so I can test it with your file. If it still doesn’t work, I’ll try to get a fix in for it soon.
Oh wait, I glossed over the part about each time step being in its own file. I don’t think we support that in any of the ADIOS readers (there’s 2 others besides the one you’re using). ADIOS has an open issue for a feature that would handle this (see this issue), but it’s quite old so I’m not sure if it’s still planned. We haven’t planned to support this directly in the ADIOS readers in ParaView, but I can double check on that.
Is there a reason you’re writing each step in its own file? Usually we recommend to write all time steps to a single file, since the file doesn’t get corrupted if your application crashes, but I’m sure there are other reasons for doing this.
This is mostly for backward compatibility, since our previous file format wrote to separate files for each timestep. It also makes it easier to decimate in time resolution for long-term storage or transfer over the internet.
Does the openPMD reader support separate files per timestep? We used openPMD’s ADIOS2 backend to write these files and selected the “file-based iteration encoding” (Concepts — openPMD-api 0.15.2 documentation). I’d hoped this would be supported by all of the tools that could read it.
I just tried opening the files with the openPMD reader on Mac, and it seems to work perfectly. I was thrown off by the fact that it didn’t work on Windows, but that was probably due to the fact that it couldn’t read the BP5 files themselves.
Okay good! Glad that works with the OpenPMD reader. As for the ADIOS readers, I’ll bring up the file series support with the ADIOS team, though it sounds like in your case you’ll just stick with the OpenPMD reader and not use the ADIOS Image reader.
I asked about the ADIOS 2.10 release and it’s going to start soon (in the next week). If that gets finished before the ParaView 5.12 release process is finished, we plan to switch that to use 2.10, in which case the OpenPMD reader and other ADIOS readers would be able to read BP5 files on Windows.
Unfortunately we won’t be able to support reading BP5 on Windows with any ADIOS readers (including OpenPMD) for 5.12.0. BP5 isn’t able to be used on Windows until ADIOS 2.10. There’s been a release candidate for 2.10, but the final release isn’t expected for another month or so, so we won’t be able to update ADIOS in ParaView until the next release.