The .geo (Houdini) file writer is available in parallel, but it moves all of the data to rank 0. Would it be possible to have the .geo writer truly work and write in parallel? Is this a file format issue, or a ParaView implementation issue, or just the effort to implement this writer in parallel issue?
This is an issue on huge data - say hundreds of millions of cells.
Hello,
To write such files, ParaView uses the VTK class vtkHoudiniPolyDataWriter which produces files according the following specification: http://www.sidefx.com/docs/houdini15.0/io/formats/geo
Clearly, this file format is not a parallel file format and so the writer. Then it’s a default behavior of ParaView to reduce data to rank 0 before writing for non-parallel file formats.
That said, it could be possible quite easily to use a Programmable Filter that would pass the dataset to an instance of the Houdini writer with a formatted filename with processor rank.
Hope this helps,
Best,
Joachim
Thanks Joachim. Good answer.
My user actually came up with another good idea, which I will write up. The ability to have the .geo writer only write intermitent timesteps. Then, he could run multiple writes in parallel that way. Good idea, actually.