Is there an upper limit to the number of files that can be read in paraview

ParaView Support
1

Is there an upper limit to the number of files that can be read in paraview, I am trying to open over 500 vtk files in bulk in paraview using python script, but without success, I would like to know if this is possible.
Note: The program should be fine, I can successfully add 200 vtk files using the program

2

Please share your error.

3

@mwestphal I have experienced something similar using Paraview. I have 400 files in a folder which I’m trying to open as a group. The file numbering starts from *_0 to *_400, but only 392 files show up instead of 400 (401 if I don’t skip *_0). The files were generated from ‘foamToVTK’ tool in OpenFOAM 7.0.

(I did notice that opening the foam files directly (decomposed case) didn’t have this issue. It showed all the 400 files. However I think the mechanism of opening is different since I just opened an empty ‘dummy.foam’ file in the openfoam case directory, instead of group opening files.)

Screenshot from 2023-04-25 17-38-22

I’ve checked the missing files by opening them individually and they open/plot fine. The first few lines of each .vtk file are:

# vtk DataFile Version 2.0
datafilename
BINARY
DATASET UNSTRUCTURED_GRID
POINTS 2001831 float

I did a few sanity checks to see if I was running out of ram (usage was at 24%), on different machines (same issue encountered on ubuntu 22, ubuntu 20.04, centos 7), if the files not loading were any different (there seems to be no difference between the files, other than the field data values), tried loading in through client and pvserver (still only showed up 392 files), tried on different versions (5.11.1, 5.8.1, 5.6.3). I’m not sure what the problem is!

4

Could you share your files ?

5

The files I used are too large to be shared online. I’ve tried to reproduce it by copying a single file to multiple files with names in series. However, I wasn’t able to reproduce it.

6

This discussion was helpful for me:
https://www.paraview.org/pipermail/paraview/2010-May/017475.html

Hence, on Windows the solution was to add the following lines to the script:

import win32file
win32file._setmaxstdio(8192)

This increases the maximum number of open files from 512 to 8192 (as described here).
In my case, the EnSightReader was used with a transient data set. It crashed when reaching the 507th time step.

Maybe this is something that can be fixed from within ParaView by properly opening and closing files when cycling through the time steps. But not sure…

1 Like