# Custom CSV Format with Two Different Field Lengths for Import

**URL:** https://discourse.paraview.org/t/custom-csv-format-with-two-different-field-lengths-for-import/7516
**Category:** ParaView Support
**Created:** [June 30, 2021, 2:24am UTC](https://discourse.paraview.org/t/custom-csv-format-with-two-different-field-lengths-for-import/7516 "2021-06-30T02:24:37Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![Joe\_Gorse](https://discourse.paraview.org/user_avatar/discourse.paraview.org/joe_gorse/32/7023_2.png) [@Joe\_Gorse](https://discourse.paraview.org/u/Joe_Gorse)
#### Post date: [June 30, 2021, 2:24am UTC](https://discourse.paraview.org/t/custom-csv-format-with-two-different-field-lengths-for-import/7516/1 "2021-06-30T02:24:37Z")

</div>

Greetings,

I have CSV files with a 7 row pattern which constitutes one point in time. The first row is 6 fields. The second to the seventh row are 10 fields. This can go one for 10’s of megabytes.

How can I teach Paraview this format for ingesting the data?

```auto
2021-06-09 00:00:00.442,-1,-1,40AD,43,48
2021-06-09 00:00:00.442,1,0,1010,0,42,65535,37,14709,21142
2021-06-09 00:00:00.442,1,1,237,1948,44,65535,36,14913,21142
2021-06-09 00:00:00.442,2,0,915,0,42,65535,36,14719,21139
2021-06-09 00:00:00.442,2,1,242,1978,43,65535,38,14910,21139
2021-06-09 00:00:00.442,3,0,422,0,45,65535,37,14699,13519
2021-06-09 00:00:00.442,3,1,294,1960,46,65535,36,14898,13519
2021-06-09 00:00:10.632,-1,-1,40AD,43,49
2021-06-09 00:00:10.632,1,0,550,1952,42,65535,37,14802,21142
2021-06-09 00:00:10.632,1,1,279,0,44,65535,36,14815,21142
2021-06-09 00:00:10.632,2,0,657,1970,42,65535,36,14807,21139
2021-06-09 00:00:10.632,2,1,262,0,43,65535,38,14817,21139
2021-06-09 00:00:10.632,3,0,829,0,45,65535,37,14693,13519
2021-06-09 00:00:10.632,3,1,234,1960,46,65535,36,14905,13519
2021-06-09 00:00:20.187,-1,-1,40AD,43,49
2021-06-09 00:00:20.187,1,0,297,1952,42,65535,37,14811,21142
2021-06-09 00:00:20.187,1,1,489,0,44,65535,36,14808,21142
2021-06-09 00:00:20.187,2,0,308,1970,42,65535,36,14821,21139
2021-06-09 00:00:20.187,2,1,456,0,43,65535,38,14807,21139
2021-06-09 00:00:20.187,3,0,798,1963,45,65535,37,14787,13519
2021-06-09 00:00:20.187,3,1,242,0,46,65535,36,14815,13519

```

Currently, Paraview only reads up to Field 5 (sixth field) and then ignores the remaining columns.

Cheers,  
Joe

---

<div class="post-metadata">

### Author: ![mwestphal](https://discourse.paraview.org/user_avatar/discourse.paraview.org/mwestphal/32/17_2.png) [@mwestphal](https://discourse.paraview.org/u/mwestphal)
#### Post date: [June 30, 2021, 7:12am UTC](https://discourse.paraview.org/t/custom-csv-format-with-two-different-field-lengths-for-import/7516/2 "2021-06-30T07:12:51Z")

</div>

No you can’t unless you manually (using sed or another tools) add zeroes or Nans in the missing collumns.

Modifying the csv reader to support that should be doable, as well as writing a simple ProgrammableSource to read the data yourself.

---

<div class="post-metadata">

### Author: ![Joe\_Gorse](https://discourse.paraview.org/user_avatar/discourse.paraview.org/joe_gorse/32/7023_2.png) [@Joe\_Gorse](https://discourse.paraview.org/u/Joe_Gorse)
#### Post date: [July 2, 2021, 10:14am UTC](https://discourse.paraview.org/t/custom-csv-format-with-two-different-field-lengths-for-import/7516/3 "2021-07-02T10:14:30Z")

</div>

Mathieu,

Thank you for writing.

I found the CSV Reader. There appears to be a C++ reference and Python plugin:  
[https://gitlab.kitware.com/paraview/paraview/-/blob/master/VTKExtensions/IOGeneral/vtkEnsembleDataReader.cxx#L211](https://gitlab.kitware.com/paraview/paraview/-/blob/master/VTKExtensions/IOGeneral/vtkEnsembleDataReader.cxx#L211)

[https://gitlab.kitware.com/paraview/paraview/-/blob/master/Examples/Plugins/PythonAlgorithm/PythonAlgorithmExamples.py#L132](https://gitlab.kitware.com/paraview/paraview/-/blob/master/Examples/Plugins/PythonAlgorithm/PythonAlgorithmExamples.py#L132)

And the ProgrammableSource is a python script output:  
[https://kitware.github.io/paraview-docs/latest/python/paraview.simple.ProgrammableSource.html](https://kitware.github.io/paraview-docs/latest/python/paraview.simple.ProgrammableSource.html)

Choices.

---

<div class="post-metadata">

### Author: ![Francois\_Mazen](https://discourse.paraview.org/user_avatar/discourse.paraview.org/francois_mazen/32/6483_2.png) [@Francois\_Mazen](https://discourse.paraview.org/u/Francois_Mazen)
#### Post date: [July 2, 2021, 1:56pm UTC](https://discourse.paraview.org/t/custom-csv-format-with-two-different-field-lengths-for-import/7516/4 "2021-07-02T13:56:37Z")

</div>

I’m not sure that your file fits CSV file format. In your case, one point in time is defined by several line with variable length. CSV expects one line per entry with fixed row count.

To stick with the CSV file format I would recommend to merge the 7 rows pattern to one row in order to get the 1 line = 1 entry contract.

Your first point defined by this:

```nohighlight
A1,B1,C1
D1,E1
F1,G1
A2,B2,C2
D2,E2
F2,G2

```

should become:

```nohighlight
A1,B1,C1,D1,E1,F1,G1
A2,B2,C2,D2,E2,F2,G2

```

Best,
