I have sets of time ordered *.vtp files containing the shape of a flow free surface.
Usually around ~200 files, total size beetween 20 - 75 GB, the files start small but as the domain gets filled up they get to around 200-600 MB
When I export from flow3d’s native format I get the sequence of *.vtp files and a *.pvd file thath has a list of the *.vtp files.
When the geometry gets cached into RAM (by pressing play with the “cache geometry” option) the animation is somewhat smooth and exporting different angels to a video file is reasonably fast.
The question is:
Is there a way to cache the geometry in parallel? Paraview seems to be doing it sequentially as my cpu/disk/ram hardly seem strained when doing it and it takes a long time.
I figure there must be a way to read and cache those files faster, or some other way to create animations from different angles using parallel reads and processes from that set of files.
I’m afraid no, ParaView distribute data in space, not in time. In any case unless you are working with very special kind of filesystem, parallel IO is usually not a good idea.
You need to read these files to get the data and your disk cannot read multiple location at the same time.
Once it is cached, it is fast, right ?
Hi Matthew thanks for looking into this!
I wouldn’t call it fast once cached but it is certainly better.
It’s maybe ok when the dataset is 20GB but when the dataset is 75GB it still chugs put I can get the animations
In any case it’s way better that opening the flow3d native file in flow3d post (a fork of paraview)
This is why I just wrote a script that opens flow3d post and dumps the surface into the vtp files
I think I might need to figure out what exactly is being exported into the vtp files, one thing that doesn’t make sense is that the vtps are supposed to have only the surface results yet they are just as large as the full volume results in flow3d format
specs are 56 core Xeon w9, nvidia a4000, 256 Gb ram, ssd
Is your RAM full in that case ?
to have only the surface results yet they are just as large as the full volume results in flow3d format
That is definitely an issue
RAM is not completely full in that case, but it’s close, which is also a bit weird since 75 GB << 256 GB
I agree that the files being so large just for the surface is wrong, is there any way to use the flow3d post *.flsgrf reader in vanilla paraview? that might be the key to make sure im extracting only the surface data (for smallest file size possible)
To be honest I thought that was what I was doing in flow 3d post, but now im doubting it’s doing it right.
The main reason I’m looking into this a bit more is because with this workstation I can definitely run a larger simulation than what I can get an animation for reliably/fast so post-processing has kind of become my bottle neck.