Hello,
I have a very large decomposed OpenFOAM case with (72 processors folders). The size of the case is more than 200GB. I used to use the reconstructPar utility of OpenFOAM to construct only a few timesteps that I can load on Paraview (This is much faster). However, for some reason ( bug investigation: Work in progress ) there are some issues with this utility that prevent proper reconstrcution of the case no matter how I try (So it’s not an option)
When I load the decomposed case in Paraview 5.8.1 it takes about 8-9 hours to open. I know, after that I can use the “Extract time steps” filter to save data for some specific (range) timesteps. However, I wonder why Paraview doesn’t provide an option at the startup (after selecting the .foam file) to load only a very specific time range of data. For example, if my simulation time is from 0 to 3500s, the user could load only data from t=3200 to 3400 (maybe also with a custom frequency: e.g each 5 steps).
Any ideas why this is not implemented? or maybe is there a similar feature I am not aware of?
Can’t see an immediate reason for the (initial?) slow loading except that the dispatch in the POpenFOAMReader dispatches each processor block (around line 212-214 of vtkPOpenFOAMReader.cxx) and this will scan the times for each processor block. Ie, a readdir() is invoked 72 times. Even if you happen to use the “list by controlDict” option, it appears to require a filestat each time.
On top of this, the checks for topology and point motion are checking and trying to read <time>/polyMesh/{faces,points,faces.gz,points.gz} for each time! This could at least include a short-circuit on the <time>/polyMesh/ existence. These too will be hit nProc*nTimes*(2 or 4) times.
It looks to me that we should have a mechanism to find the times and then reuse them at each stage, as well as add short-circuiting.
Would you be able to contribute any type of financing to help with tackling this issue or others? It really does consume a lot of time to fix and test these things.
I used to use the reconstructPar utility of OpenFOAM to construct only a few timesteps that I can load on Paraview (This is much faster).
If you only want visualization like that, then IMO it would be faster to use foamToVTK -parallel instead of first reconstructing OpenFOAM. Has time-ranges, patch selection (or exclusion) etc.
Hi Mark.
Thank you very much for your reply.
Unfortunately I am not that expert in programming. I am an average user, I may only be able to report bugs.
I will try to open an issue on Paraview’s repo on gitlab.
Thank you.
EDIT:
BTW, I’ve already reported an issue regarding reconstructPar on OpenFOAM’s repository.
re-inherit time names and time values from the first processor.
The topology scanning is part of we need to do to detect if mesh motion or a topology change has occurred. We still need this information eventually, but in the old code it did a conservative check for reading the FoamFile header for <time>/polyMesh/{faces,points} with fallback for gz files too. So for 72 procs x 6000 steps = 864000 attempted file reads.
I now have short-circuited on the existence/non-existence of the <time>/polyMesh/ directory, and replaced the file reads with a filestat for a file-check. This should help.
Next, I’ve remove the initial scanning with a lazy cached evaluation. On startup, on the first time directory (not all 6000) will be visited. On subsequent changes of time, check to see if that index has already been visited, or populate missing values up to that point. I think we still have to do this for each processor directory separately, unless we actually start sharing the cache as well.
The final bit is to avoid the repetitive listing and parsing of the time-directories. Now the processor0 gathers the times and their names (as before), but now uses those same names/values when the readers are created for processor1 … processorN. Haven’t properly tested when running in parallel with MPI, but that could be something that you could verify.
After these changes, I would like to see if we managed to move the loading performance compared to the stock paraview 5.9.0
Update: scanning of OpenFOAM time directories has been merged into the master branch and should be now be faster since the number of filestat operations has been reduced. Unfortunately the changes won’t be appearing in the upcoming paraview-5.9.1 since they are more than just bug fixes.
Thanks for starting this thread and adapting the code. However, I often load OpenFOAM cases over network and even with 5.9.1 loading a case with many timesteps is very slow. My trick is to hide the time folders I do not need into a subfolder, which leads to a speedup more of less equal to the ratio of the number of timesteps before and after hiding the unwanted timesteps (so think 100 timesteps can mean 1 minute instead of two hours just to load the case, no rendering of any timestep other than the first one). I wonder if anyone has an easier (or less invasive) workaround? I don’t understand why Paraview has to go through all the time folders before I actually ask it to change step: is there a good reason for this? I thought each time folder was totally independent from the other, because an OpenFOAM restart is always possible no matter what time folder is taken, but maybe I’m missing some fundamental concept here…
Thanks. I tried 5.10.1 for a quick check with a small decomposed case of 90,000 cells over 10 processors with about 10 volume fields per output timestep and 1290 such timesteps.
I’m loading it over sshfs with a ~ 60Mbps bandwidth but paraview just slowly transfers data with the host at about 50 kbps in both upload and download. Should I expect a near instant load or be more patient?
Hi Mathieu,
thanks for the tip. Yes I use local rendering because I find it simpler to use (no need to start a server connection or install libraries on the remote machine and faster mouse response once the data is loaded), but yes, remote rendering is much faster at loading!
bug investigation: Work in progress