You can also filter out the NaNs with a selection operation.
The following is with ParaView 5.11.0. Open the Find Data panel, set the Data Producer to the CSV file reader, pick one of the coordinate columns and set the find criteria to “is NaN”. Click the Invert Selection and you’ll have selected all rows without NaN coordinates. Click the Extract button to extract a table with rows that don’t have the NaN points.
