I’m using the gridwriter.py sample script to write data out from Catalyst. The issue is as the number of ranks increases, the number of output files also goes up. Is there a way to group data from several ranks to output fewer files, and thus reduce writing out time?
You can use the
AggregateDataset filter for this.
aggregateDataset1 = AggregateDataset(Input=grid)
aggregateDataset1.NumberOfTargetProcesses = 8
NumberOfTargetProcesses is the number of ranks that will write out files.
Do you just put that in the gridWriter script?
No, I took that from another Catalyst 1.0 script. You’ll have to adapt it to your script.