Hi,
I’m using python to create my VTK files in an XML format and I’ve managed to get everything working for ascii files, binary files and appended files with raw encoding. However I am unable to get the appended data to work and always seem to get the following error in paraview
Error reading uncompressed binary data header. Read 7 of 8 bytes.
ERROR: In ./VTK/IO/XMLParser/vtkXMLDataParser.cxx, line 653
vtkXMLDataParser (0x55f909190850): Error reading uncompressed binary data header. Read 7 of 8 bytes.
For the following examples I’m just generating random data (with a seed) in python:
np.random.seed(77)
npoints = 20
x = np.random.rand(npoints)
y = np.random.rand(npoints)
z = np.random.rand(npoints)
ID = np.arange(npoints) + 1
pressure = np.random.rand(npoints)
temp = np.random.rand(npoints)
force = np.random.rand(npoints, 3)
pos = np.vstack([x, y, z]).T
The ascii file I generate is this:
<?xml version="1.0"?>
<VTKFile type="PolyData" version="1.0" byte_order="LittleEndian" header_type="UInt64">
<PolyData>
<Piece NumberOfPoints="20" NumberOfVerts="20" NumberOfLines="0" NumberOfStrips="0" NumberOfPolys="0">
<PointData Scalars="1_temp" Vectors="3_force">
<DataArray Name="1_temp" NumberOfComponents="1" type="Float64" format="ascii">
0.4204270035661765 0.9490145030363609 0.7864711816566549 0.4815468959885630 0.0119215535827562 0.0597207807092479
0.8257177483439162 0.4046369872739349 0.0531531123171690 0.2596390482071086 0.2811506610562717 0.9433030410624577
0.8936293596271191 0.2022892694899581 0.1003608087345019 0.5730430655359530 0.7453773741488717 0.6007604785160442
0.9092395428890321 0.9385777624120453
</DataArray>
<DataArray Name="2_pressure" NumberOfComponents="1" type="Float64" format="ascii">
0.5890720970505128 0.3268986086578718 0.1448144482177102 0.2576384903723663 0.8574458883976873 0.6839737520406283
0.9986326035811655 0.3194556656082242 0.4534082266242594 0.8676221777748950 0.6385221808378060 0.3318923019829476
0.5419776327479333 0.1123238088406830 0.8165234436255825 0.2209316463478144 0.5149742681090496 0.5260877679247680
0.2770886884732112 0.9044206316605501
</DataArray>
<DataArray Name="3_force" NumberOfComponents="3" type="Float64" format="ascii">
0.5615661006777212 0.8908316333576722 0.3124782869770458 0.7710535361311034 0.2136507101264129 0.2883443963127228
0.2734427951922838 0.2485007122242413 0.0144745384478043 0.6091129763543156 0.8618458103278591 0.2591405994003485
0.8668647914278148 0.4265298541907321 0.1603909008190777 0.0565316315409430 0.4228071934122273 0.7359960547933326
0.4769537916279307 0.7006992708984805 0.9261806156102312 0.4526842850674920 0.3917947827194505 0.3592419246788816
0.9207452697946850 0.0278180708490150 0.2349904981777623 0.0795443510228309 0.5657883468686661 0.2948885661324933
0.7193769267541885 0.0988020129816635 0.0268196366997491 0.7018310907399513 0.7914683737300843 0.3782688291338249
0.2473226108228560 0.2627823218786545 0.4984137186921088 0.3041873590603215 0.7186392108467865 0.6106608253770643
0.0646729072148587 0.1480143523868742 0.9702434112898988 0.4715222037958284 0.7331722522876482 0.2106973939969884
0.6426786141941642 0.9115921673564267 0.0345287813667512 0.1773567248506672 0.2962514934470256 0.3126333475071587
0.3249249442272334 0.5780774130897147 0.9983199556809254 0.3547831120917448 0.0207988496596299 0.0996248310145578
</DataArray>
</PointData>
<CellData>
</CellData>
<Points>
<DataArray Name="Points" NumberOfComponents="3" type="Float64" format="ascii">
0.9191090317991818 0.4523130130592968 0.4455559100158843 0.6421955999238554 0.1757747380313244 0.9770526614339072
0.7537122297291767 0.0493769993144870 0.7293940090533850 0.1393145681027350 0.2924753386949548 0.3122350583311531
0.0873195479896001 0.0667991310914613 0.8947552392940534 0.7880020580364918 0.7511564859911245 0.7832735951745574
0.3261509370344888 0.0637715240806562 0.2620003413345412 0.5410678214759677 0.4319083194117714 0.3094831861549427
0.2402351758680060 0.3641724134430167 0.1294506297724709 0.5454229255759049 0.1519715256925609 0.4221713562335928
0.4005545036798892 0.5467103397159158 0.9397650349673574 0.7151918872797569 0.4432930376023336 0.3670428731605497
0.8366799410811948 0.0360613100029110 0.4347749726828846 0.5884811425466829 0.8228931912295053 0.9170935543593193
0.2961545640333596 0.2732926831299841 0.9472939210005341 0.2810176906539642 0.1689852192728417 0.2547729480887090
0.7055972446574292 0.6443697547109174 0.7097035779964328 0.4225964301897521 0.1075410788025926 0.0499764048396870
0.0573159852472062 0.3532450981171027 0.1347439306209732 0.7470273067071216 0.3857036581156064 0.5065059431067037
</DataArray>
</Points>
<Verts>
<DataArray Name="connectivity" NumberOfComponents="1" type="Int32" format="ascii">
0 1 2 3 4 5
6 7 8 9 10 11
12 13 14 15 16 17
18 19
</DataArray>
<DataArray Name="offsets" NumberOfComponents="1" type="Int32" format="ascii">
1 2 3 4 5 6
7 8 9 10 11 12
13 14 15 16 17 18
19 20
</DataArray>
</Verts>
<Lines>
<DataArray Name="connectivity" NumberOfComponents="1" type="Float64" format="ascii">
</DataArray>
<DataArray Name="offsets" NumberOfComponents="1" type="Float64" format="ascii">
</DataArray>
</Lines>
<Strips>
<DataArray Name="connectivity" NumberOfComponents="1" type="Float64" format="ascii">
</DataArray>
<DataArray Name="offsets" NumberOfComponents="1" type="Float64" format="ascii">
</DataArray>
</Strips>
<Polys>
<DataArray Name="connectivity" NumberOfComponents="1" type="Float64" format="ascii">
</DataArray>
<DataArray Name="offsets" NumberOfComponents="1" type="Float64" format="ascii">
</DataArray>
</Polys>
</Piece>
</PolyData>
</VTKFile>
And the binary file generated is this:
<?xml version="1.0"?>
<VTKFile type="PolyData" version="1.0" byte_order="LittleEndian" header_type="UInt64">
<PolyData>
<Piece NumberOfPoints="20" NumberOfVerts="20" NumberOfLines="0" NumberOfStrips="0" NumberOfPolys="0">
<PointData Scalars="1_temp" Vectors="3_force">
<DataArray Name="1_temp" NumberOfComponents="1" type="Float64" format="binary">
oAAAAAAAAAAaAqupRujaP08Rv6lTXu4/IceOnMUq6T9et3ASqtHePwApC9ZTaog/gFsT4LiTrj/kpJugR2zqPzb8xYiS5dk/EGwqfuI2qz8yGjQZ7Z3QP+gRn1df/tE/SJTy24kv7j85AUuZnJjsP8hCZWKd5Mk/iJZQ9z6xuT/ppTVpXlbiP3O6pKYh2uc/Q5X+CW454z86B56GfRjtP8IASjvUCO4/
</DataArray>
<DataArray Name="2_pressure" NumberOfComponents="1" type="Float64" format="binary">
oAAAAAAAAADMLfq5rdniPxTIUiTo69Q/PKuKo0eJwj+YxJUmJn3QP1k/GFwycOs/+8cK7Bzj5T/KNqNczPTvP4LKEy32cdQ/0KpF8KME3T+netqVj8PrP9cBjxHGbuQ/DOazNbk91T+gG/p54VfhP3BriM1Awbw/70DZxfUg6j/4jtv5fEfMP7vq+VCreuA/IgDCA7bV4D/WX8Ux0rvRP0TpWYkD8ew/
</DataArray>
<DataArray Name="3_force" NumberOfComponents="3" type="Float64" format="binary">
4AEAAAAAAABvf554WfjhP2m+JHd4rOg/MF+rNRaA0T8rWiB/2n3jP/uPJztbves/oNRQybbxrD+4sC8yaYbePxLL8YHH+Nw/NOC2yL527T+QfznCBF20P9D/vMIiBec/Q2nBeWZ15j8E04VuRKjPP/xSwUHOd9M/yNBsVWeOsD8ABSl3ay3eP2Guub3SkOQ/5Ml6CqCzxj/uE0f+kcvUP3zb5jnEtNY/Tm5wV7GB7D+YS2EO6FjLP+CVBBDfzs8/U7Axqj2U6z/0E6HfQ0zbPyAFDudFD9s/buGa4CBs5j92gKFsKhPZP4AnIyJXfJw/S0jIKfAa4j+oA4m2Fku5Pwimn3u1U+k/mOOb8WzR0D/uuYaoF//mPyCNa2Ei8sI/nx+9pyV25z9UvUJWwyvtP6bA79LI9dI/99n4M5x/4j8Ag1QsS0yVP2qy0e2k/9M/9HgJDjx00j/AaRCq06SNP6Jh33PCldA/hAXMZLCHxD8AUSqZR43nP9yFx4dFo+0/3EN219H91j+MqUUsKxTOP5I34Ep039I/gGgQWpt2mz+edo52jjXYPzJDaqcC5t8/5Ss+koiK4z92yRTpOwzvPxzlSNgh+Mo/QFt7pcGtoT/6qctMLwLUP/P/ErE88u8/sLkTTwOBuT8=
</DataArray>
</PointData>
<CellData>
</CellData>
<Points>
<DataArray Name="Points" NumberOfComponents="3" type="Float64" format="binary">
4AEAAAAAAACJJCFYV2ntPw5GqUey8tw/DLKD7/yD3D+f22nJ3YzkP1hcp1/Jf8Y/MIJq8QNE7z+UBikcaR7oP6DudSrxR6k/MgvZGjJX5z84xu1MD9XBP+g0pXvqt9I/6JoMwaj70z+wterqklq2PyBlcHO/GbE/5ClWvdWh7D9RUY4XUDfpPz9TsFN5Ceg/ywpjyZMQ6T9Y3QcuqN/UP8CtV6JUU7A/+qZkFJ3E0D8sCMV2bVDhP7ybr8pipNs/iKHMkJLO0z+s19m3BsDOPyD2dc+ZTtc/AOGoltaRwD/2yXrHGnThPzjqYo7Nc8M/JjMVAtsE2z9MhWRbr6LZP4fgrq6mfuE/uH1jH44S7j/2ZMcY2uLmPzT6wsLpXtw/gBUdZKF91z/LNgUDFcbqP0ASOsagdqI/wrQyaFrT2z96n7Fn1tTiP03RDRokVeo/xA/rlNRY7T8YZ8VFMvTSPz7hEZigfdE/a7NMVztQ7j+6Mr2fMfzRPxxiJMNOocU/opP8MTNO0D+ecT6sQJTmP1N64FGtnuQ/5SkrR+S15j+2kMTl0QvbP5AGb+jPh7s/cFq34IGWqT9Qe1lUhVitPzio+FORm9Y/WACtA0o/wT9k1XDPpefnP0h2Y2Ver9g/Ce+b80s14D8=
</DataArray>
</Points>
<Verts>
<DataArray Name="connectivity" NumberOfComponents="1" type="Int32" format="binary">
UAAAAAAAAAAAAAAAAQAAAAIAAAADAAAABAAAAAUAAAAGAAAABwAAAAgAAAAJAAAACgAAAAsAAAAMAAAADQAAAA4AAAAPAAAAEAAAABEAAAASAAAAEwAAAA==
</DataArray>
<DataArray Name="offsets" NumberOfComponents="1" type="Int32" format="binary">
UAAAAAAAAAABAAAAAgAAAAMAAAAEAAAABQAAAAYAAAAHAAAACAAAAAkAAAAKAAAACwAAAAwAAAANAAAADgAAAA8AAAAQAAAAEQAAABIAAAATAAAAFAAAAA==
</DataArray>
</Verts>
<Lines>
<DataArray Name="connectivity" NumberOfComponents="1" type="Float64" format="binary">
</DataArray>
<DataArray Name="offsets" NumberOfComponents="1" type="Float64" format="binary">
</DataArray>
</Lines>
<Strips>
<DataArray Name="connectivity" NumberOfComponents="1" type="Float64" format="binary">
</DataArray>
<DataArray Name="offsets" NumberOfComponents="1" type="Float64" format="binary">
</DataArray>
</Strips>
<Polys>
<DataArray Name="connectivity" NumberOfComponents="1" type="Float64" format="binary">
</DataArray>
<DataArray Name="offsets" NumberOfComponents="1" type="Float64" format="binary">
</DataArray>
</Polys>
</Piece>
</PolyData>
</VTKFile>
Both of these work fine in Paraview with no errors.
I also successfully create the appended data using the raw formatting but when I create the append file with base64 encoding paraview fails to read this file:
<?xml version="1.0"?>
<VTKFile type="PolyData" version="1.0" byte_order="LittleEndian" header_type="UInt64">
<PolyData>
<Piece NumberOfPoints="20" NumberOfVerts="20" NumberOfLines="0" NumberOfStrips="0" NumberOfPolys="0">
<PointData Scalars="1_temp" Vectors="3_force">
<DataArray Name="1_temp" type="Float64" format="appended" NumberOfComponents="1" offset="0"/>
<DataArray Name="2_pressure" type="Float64" format="appended" NumberOfComponents="1" offset="168"/>
<DataArray Name="3_force" type="Float64" format="appended" NumberOfComponents="3" offset="336"/>
</PointData>
<CellData>
</CellData>
<Points>
<DataArray Name="Points" type="Float64" format="appended" NumberOfComponents="3" offset="824"/>
</Points>
<Verts>
<DataArray Name="connectivity" type="Int32" format="appended" NumberOfComponents="1" offset="1312"/>
<DataArray Name="offsets" type="Int32" format="appended" NumberOfComponents="1" offset="1400"/>
</Verts>
<Lines>
<DataArray Name="connectivity" type="Float64" format="appended" NumberOfComponents="1" offset="1488"/>
<DataArray Name="offsets" type="Float64" format="appended" NumberOfComponents="1" offset="1496"/>
</Lines>
<Strips>
<DataArray Name="connectivity" type="Float64" format="appended" NumberOfComponents="1" offset="1504"/>
<DataArray Name="offsets" type="Float64" format="appended" NumberOfComponents="1" offset="1512"/>
</Strips>
<Polys>
<DataArray Name="connectivity" type="Float64" format="appended" NumberOfComponents="1" offset="1520"/>
<DataArray Name="offsets" type="Float64" format="appended" NumberOfComponents="1" offset="1528"/>
</Polys>
</Piece>
</PolyData>
<AppendedData encoding="base64">
_oAAAAAAAAA==GgKrqUbo2j9PEb+pU17uPyHHjpzFKuk/XrdwEqrR3j8AKQvWU2qIP4BbE+C4k64/5KSboEds6j82/MWIkuXZPxBsKn7iNqs/Mho0Ge2d0D/oEZ9XX/7RP0iU8tuJL+4/OQFLmZyY7D/IQmVineTJP4iWUPc+sbk/6aU1aV5W4j9zuqSmIdrnP0OV/gluOeM/Ogeehn0Y7T/CAEo71AjuPw==oAAAAAAAAA==zC36ua3Z4j8UyFIk6OvUPzyriqNHicI/mMSVJiZ90D9ZPxhcMnDrP/vHCuwc4+U/yjajXMz07z+CyhMt9nHUP9CqRfCjBN0/p3ralY/D6z/XAY8Rxm7kPwzmszW5PdU/oBv6eeFX4T9wa4jNQMG8P+9A2cX1IOo/+I7b+XxHzD+76vlQq3rgPyIAwgO21eA/1l/FMdK70T9E6VmJA/HsPw==4AEAAAAAAA==b3+eeFn44T9pviR3eKzoPzBfqzUWgNE/K1ogf9p94z/7jyc7W73rP6DUUMm28aw/uLAvMmmG3j8Sy/GBx/jcPzTgtsi+du0/kH85wgRdtD/Q/7zCIgXnP0NpwXlmdeY/BNOFbkSozz/8UsFBznfTP8jQbFVnjrA/AAUpd2st3j9hrrm90pDkP+TJegqgs8Y/7hNH/pHL1D982+Y5xLTWP05ucFexgew/mEthDuhYyz/glQQQ387PP1OwMao9lOs/9BOh30NM2z8gBQ7nRQ/bP27hmuAgbOY/doChbCoT2T+AJyMiV3ycP0tIyCnwGuI/qAOJthZLuT8Ipp97tVPpP5jjm/Fs0dA/7rmGqBf/5j8gjWthIvLCP58fvaclduc/VL1CVsMr7T+mwO/SyPXSP/fZ+DOcf+I/AINULEtMlT9qstHtpP/TP/R4CQ48dNI/wGkQqtOkjT+iYd9zwpXQP4QFzGSwh8Q/AFEqmUeN5z/chceHRaPtP9xDdtfR/dY/jKlFLCsUzj+SN+BKdN/SP4BoEFqbdps/nnaOdo412D8yQ2qnAubfP+UrPpKIiuM/dskU6TsM7z8c5UjYIfjKP0Bbe6XBraE/+qnLTC8C1D/z/xKxPPLvP7C5E08Dgbk/4AEAAAAAAA==iSQhWFdp7T8ORqlHsvLcPwyyg+/8g9w/n9tpyd2M5D9YXKdfyX/GPzCCavEDRO8/lAYpHGke6D+g7nUq8UepPzIL2RoyV+c/OMbtTA/VwT/oNKV76rfSP+iaDMGo+9M/sLXq6pJatj8gZXBzvxmxP+QpVr3Voew/UVGOF1A36T8/U7BTeQnoP8sKY8mTEOk/WN0HLqjf1D/ArVeiVFOwP/qmZBSdxNA/LAjFdm1Q4T+8m6/KYqTbP4ihzJCSztM/rNfZtwbAzj8g9nXPmU7XPwDhqJbWkcA/9sl6xxp04T846mKOzXPDPyYzFQLbBNs/TIVkW6+i2T+H4K6upn7hP7h9Yx+OEu4/9mTHGNri5j80+sLC6V7cP4AVHWShfdc/yzYFAxXG6j9AEjrGoHaiP8K0Mmha09s/ep+xZ9bU4j9N0Q0aJFXqP8QP65TUWO0/GGfFRTL00j8+4RGYoH3RP2uzTFc7UO4/ujK9nzH80T8cYiTDTqHFP6KT/DEzTtA/nnE+rECU5j9TeuBRrZ7kP+UpK0fkteY/tpDE5dEL2z+QBm/oz4e7P3Bat+CBlqk/UHtZVIVYrT84qPhTkZvWP1gArQNKP8E/ZNVwz6Xn5z9IdmNlXq/YPwnvm/NLNeA/UAAAAAAAAA==AAAAAAEAAAACAAAAAwAAAAQAAAAFAAAABgAAAAcAAAAIAAAACQAAAAoAAAALAAAADAAAAA0AAAAOAAAADwAAABAAAAARAAAAEgAAABMAAAA=UAAAAAAAAA==AQAAAAIAAAADAAAABAAAAAUAAAAGAAAABwAAAAgAAAAJAAAACgAAAAsAAAAMAAAADQAAAA4AAAAPAAAAEAAAABEAAAASAAAAEwAAABQAAAA=AAAAAAAAAA==AAAAAAAAAA==AAAAAAAAAA==AAAAAAAAAA==AAAAAAAAAA==AAAAAAAAAA==
</AppendedData>
</VTKFile>
There is almost zero information on this in the VTK documentation, just that you can use it.
The appended data section begins with the first character after the underscore inside the AppendedData element. The underscore is not part of the data, but is always present. Data in this section is always in binary form, but can be compressed and/or base64 encoded. The byte-order of the data matches that specified by the byte_order attribute of the VTKFile element. Each DataArray’s data are stored contiguously and appended immediately after the previous DataArray’s data without a seperator. The DataArray’s offset attribute indicates the file position offset from the first character after the underscore to the beginning its data.
The only information I could find that suggests how it should be written is at this post - the key being that the block size needs to written separately from the data, but this seems to be the opposite of the inline binary encoding.
Does anyone know why the base64 appended data doesn’t work?