Can .vtp files hold multiple data fields or just one?

WarinSpace_Game · November 13, 2023, 6:18pm

The following Python code converts a .vtk file to a .vtp file for viewing using e.g. Glance:

# Read VTK
reader = vtkStructuredGridReader()
reader.SetFileName(filename + ".vtk")

# Extract surface
surface_filter = vtkDataSetSurfaceFilter()
surface_filter.SetInputConnection(reader.GetOutputPort())

clean_filter = vtkCleanPolyData()
clean_filter.SetInputConnection(surface_filter.GetOutputPort())

# Save output to a VTP file
writer = vtkXMLPolyDataWriter()
writer.SetFileName(filename + ".vtp")
writer.SetInputConnection(clean_filter.GetOutputPort())
writer.Write()

This works fine, but when I have multiple fields, such as Directivity_dBi and Directivity_linear:

# vtk DataFile Version 3.0
3D Farfield
ASCII
DATASET STRUCTURED_GRID
DIMENSIONS 1 91 181
POINTS 16471 double
-1.000000e+00 1.000000e+00 4.006663e+01
4.257683e-01 1.000000e+00 4.003194e+01
1.847900e+00 1.000000e+00 3.993014e+01
...
-1.000000e+00 1.000000e+00 -1.754531e+01


POINT_DATA 16471
SCALARS Directivity_dBi double 1
LOOKUP_TABLE default
7.775559e+00
7.765758e+00
7.738521e+00
...
-1.634019e+01
-1.635408e+01
-1.633920e+01

SCALARS Directivity_linear double 1
LOOKUP_TABLE default
5.991781e+00
5.978273e+00
5.940897e+00
...
2.322637e-02
2.315217e-02
2.323166e-02

…it only seems to write Directivity_dBi to the .vtp file, ignoring Directivity_linear completely. I am able to view pattern_1_2400000000.0.vtk and both Directivity entries are there:

Attached both files for reference (input .vtk and output .vtp):

VTK: pattern_1_2400000000.0.vtk (1.1 MB)
VTP: pattern_1_2400000000.0.vtp (687.8 KB)

Is there anything I could modify to ensure both data fields are passed?

mwestphal · November 14, 2023, 8:55am

Can .vtp files hold multiple data fields or just one?

Of course they can (generated with ParaView directly):
pattern_1_2400000000.0_kw.vtp (1.0 MB)

I don’t see anything wrong in your VTK code though. Inspect the outputs of each of your filter to see why the array is disapearing.

WarinSpace_Game · November 14, 2023, 12:28pm

Printing surface_filter doesn’t show much (or at least I’m not sure if there’s anything interesting to extract):

vtkDataSetSurfaceFilter (0x118645190)
  Debug: Off
  Modified Time: 136
  Reference Count: 2
  Registered Events: (none)
  Executive: 0x600002168500
  ErrorCode: Undefined error: 0
  Information: 0x60000393c5a0
  AbortExecute: Off
  Progress: 0
  Progress Text: (None)
  PieceInvariant: 0
  PassThroughCellIds: Off
  PassThroughPointIds: Off
  OriginalCellIdsName: vtkOriginalCellIds
  OriginalPointIdsName: vtkOriginalPointIds
  NonlinearSubdivisionLevel: 1
  FastMode: 0
  Delegation: 0

And removing clean_filter doesn’t fix the problem, so that filter is definitely not the issue. Are you able to reproduce this when running the script using the vtk module?

Perhaps another question I should ask in advance is this: if I also wish to assign different coordinate points, i.e. this part:

DATASET STRUCTURED_GRID
DIMENSIONS 1 91 181
POINTS 16471 double
-1.000000e+00 1.000000e+00 4.006663e+01
4.257683e-01 1.000000e+00 4.003194e+01
1.847900e+00 1.000000e+00 3.993014e+01
...

to the second set of the scalar data, can VTP support this? That’s my ultimate goal, so that the coordinates also change when the scalar values are different.

Another alternative would be to produce 2 .vtk files (which I can handle easily), but I’m not sure how we could merge them into a single .vtp.

mwestphal · November 14, 2023, 2:17pm

Print the output of the filter instead

WarinSpace_Game · November 14, 2023, 3:12pm

surface_filter.GetOutput() returns:

vtkPolyData (0x13868c3e0)
  Debug: Off
  Modified Time: 357
  Reference Count: 2
  Registered Events: (none)
  Information: 0x60000133b300
  Data Released: False
  Global Release Data: Off
  UpdateTime: 0
  Field Data:
    Debug: Off
    Modified Time: 354
    Reference Count: 1
    Registered Events: (none)
    Number Of Arrays: 0
    Number Of Components: 0
    Number Of Tuples: 0
  Number Of Points: 0
  Number Of Cells: 0
  Cell Data:
    Debug: Off
    Modified Time: 357
    Reference Count: 1
    Registered Events:
      Registered Observers:
        vtkObserver (0x60000397f990)
          Event: 33
          EventName: ModifiedEvent
          Command: 0x6000014703c0
          Priority: 0
          Tag: 1
    Number Of Arrays: 0
    Number Of Components: 0
    Number Of Tuples: 0
    Copy Tuple Flags: ( 1 1 1 1 1 0 1 1 1 1 1 )
    Interpolate Flags: ( 1 1 1 1 1 0 0 1 1 1 1 )
    Pass Through Flags: ( 1 1 1 1 1 1 1 1 1 1 1 )
    Scalars: (none)
    Vectors: (none)
    Normals: (none)
    TCoords: (none)
    Tensors: (none)
    GlobalIds: (none)
    PedigreeIds: (none)
    EdgeFlag: (none)
    Tangents: (none)
    RationalWeights: (none)
    HigherOrderDegrees: (none)
  Point Data:
    Debug: Off
    Modified Time: 356
    Reference Count: 1
    Registered Events:
      Registered Observers:
        vtkObserver (0x60000397f960)
          Event: 33
          EventName: ModifiedEvent
          Command: 0x6000014703c0
          Priority: 0
          Tag: 1
    Number Of Arrays: 0
    Number Of Components: 0
    Number Of Tuples: 0
    Copy Tuple Flags: ( 1 1 1 1 1 0 1 1 1 1 1 )
    Interpolate Flags: ( 1 1 1 1 1 0 0 1 1 1 1 )
    Pass Through Flags: ( 1 1 1 1 1 1 1 1 1 1 1 )
    Scalars: (none)
    Vectors: (none)
    Normals: (none)
    TCoords: (none)
    Tensors: (none)
    GlobalIds: (none)
    PedigreeIds: (none)
    EdgeFlag: (none)
    Tangents: (none)
    RationalWeights: (none)
    HigherOrderDegrees: (none)
  Bounds:
    Xmin,Xmax: (1, -1)
    Ymin,Ymax: (1, -1)
    Zmin,Zmax: (1, -1)
  Compute Time: 0
  Editable: false
  Number Of Points: 0
  Point Coordinates: 0x0
  PointLocator: 0x0
  CellLocator: 0x0
  Number Of Vertices: 0
  Number Of Lines: 0
  Number Of Polygons: 0
  Number Of Triangle Strips: 0
  Number Of Pieces: 1
  Piece: -1
  Ghost Level: 0
  CellsBounds:
    Xmin,Xmax: (1, -1)
    Ymin,Ymax: (1, -1)
    Zmin,Zmax: (1, -1)
  CellsBounds Time: 0

mwestphal · November 14, 2023, 4:18pm

After updating the filter.

WarinSpace_Game · November 14, 2023, 11:01pm

After surface_filter.Update():

vtkPolyData (0x1116476f0)
  Debug: Off
  Modified Time: 560
  Reference Count: 2
  Registered Events: (none)
  Information: 0x600000d4fe40
  Data Released: False
  Global Release Data: Off
  UpdateTime: 561
  Field Data:
    Debug: Off
    Modified Time: 513
    Reference Count: 1
    Registered Events: (none)
    Number Of Arrays: 0
    Number Of Components: 0
    Number Of Tuples: 0
  Number Of Points: 16471
  Number Of Cells: 16200
  Cell Data:
    Debug: Off
    Modified Time: 560
    Reference Count: 1
    Registered Events:
      Registered Observers:
        vtkObserver (0x60000270f900)
          Event: 33
          EventName: ModifiedEvent
          Command: 0x600000a086e0
          Priority: 0
          Tag: 1
    Number Of Arrays: 0
    Number Of Components: 0
    Number Of Tuples: 0
    Copy Tuple Flags: ( 1 1 1 1 1 1 1 1 1 1 1 )
    Interpolate Flags: ( 1 1 1 1 1 1 0 1 1 1 1 )
    Pass Through Flags: ( 1 1 1 1 1 1 1 1 1 1 1 )
    Scalars: (none)
    Vectors: (none)
    Normals: (none)
    TCoords: (none)
    Tensors: (none)
    GlobalIds: (none)
    PedigreeIds: (none)
    EdgeFlag: (none)
    Tangents: (none)
    RationalWeights: (none)
    HigherOrderDegrees: (none)
  Point Data:
    Debug: Off
    Modified Time: 558
    Reference Count: 1
    Registered Events:
      Registered Observers:
        vtkObserver (0x60000270f8d0)
          Event: 33
          EventName: ModifiedEvent
          Command: 0x600000a086e0
          Priority: 0
          Tag: 1
    Number Of Arrays: 1
    Array 0 name = Directivity_dBi
    Number Of Components: 1
    Number Of Tuples: 16471
    Copy Tuple Flags: ( 1 1 1 1 1 1 1 1 1 1 1 )
    Interpolate Flags: ( 1 1 1 1 1 1 0 1 1 1 1 )
    Pass Through Flags: ( 1 1 1 1 1 1 1 1 1 1 1 )
    Scalars:
      Debug: Off
      Modified Time: 555
      Reference Count: 1
      Registered Events: (none)
      Name: Directivity_dBi
      Data type: double
      Size: 16471
      MaxId: 16470
      NumberOfComponents: 1
      Information: 0x0
      Name: Directivity_dBi
      Number Of Components: 1
      Number Of Tuples: 16471
      Size: 16471
      MaxId: 16470
      LookupTable: (none)
    Vectors: (none)
    Normals: (none)
    TCoords: (none)
    Tensors: (none)
    GlobalIds: (none)
    PedigreeIds: (none)
    EdgeFlag: (none)
    Tangents: (none)
    RationalWeights: (none)
    HigherOrderDegrees: (none)
  Bounds:
    Xmin,Xmax: (-31.173, 29.0485)
    Ymin,Ymax: (-29.3445, 31.424)
    Zmin,Zmax: (-19.2903, 40.0666)
  Compute Time: 574
  Editable: false
  Number Of Points: 16471
  Point Coordinates: 0x600001b0b600
  PointLocator: 0x0
  CellLocator: 0x0
  Number Of Vertices: 0
  Number Of Lines: 0
  Number Of Polygons: 16200
  Number Of Triangle Strips: 0
  Number Of Pieces: 1
  Piece: 0
  Ghost Level: 0
  CellsBounds:
    Xmin,Xmax: (-31.173, 29.0485)
    Ymin,Ymax: (-29.3445, 31.424)
    Zmin,Zmax: (-19.2903, 40.0666)
  CellsBounds Time: 575

Looks like only Directivity_dBi is written and Directivity_linear is ignored.

mwestphal · November 15, 2023, 8:19am

Then check the output of the reader itself.

WarinSpace_Game · November 15, 2023, 8:28am

reader.Update() and then reader.GetOutput() returns:

vtkStructuredGrid (0x110fa1bc0)
  Debug: Off
  Modified Time: 380
  Reference Count: 2
  Registered Events: (none)
  Information: 0x6000003707e0
  Data Released: False
  Global Release Data: Off
  UpdateTime: 385
  Field Data:
    Debug: Off
    Modified Time: 350
    Reference Count: 1
    Registered Events: (none)
    Number Of Arrays: 0
    Number Of Components: 0
    Number Of Tuples: 0
  Number Of Points: 16471
  Number Of Cells: 16200
  Cell Data:
    Debug: Off
    Modified Time: 358
    Reference Count: 1
    Registered Events:
      Registered Observers:
        vtkObserver (0x600002930fc0)
          Event: 33
          EventName: ModifiedEvent
          Command: 0x600000424690
          Priority: 0
          Tag: 1
    Number Of Arrays: 0
    Number Of Components: 0
    Number Of Tuples: 0
    Copy Tuple Flags: ( 1 1 1 1 1 0 1 1 1 1 1 )
    Interpolate Flags: ( 1 1 1 1 1 0 0 1 1 1 1 )
    Pass Through Flags: ( 1 1 1 1 1 1 1 1 1 1 1 )
    Scalars: (none)
    Vectors: (none)
    Normals: (none)
    TCoords: (none)
    Tensors: (none)
    GlobalIds: (none)
    PedigreeIds: (none)
    EdgeFlag: (none)
    Tangents: (none)
    RationalWeights: (none)
    HigherOrderDegrees: (none)
  Point Data:
    Debug: Off
    Modified Time: 380
    Reference Count: 1
    Registered Events:
      Registered Observers:
        vtkObserver (0x600002930f90)
          Event: 33
          EventName: ModifiedEvent
          Command: 0x600000424690
          Priority: 0
          Tag: 1
    Number Of Arrays: 1
    Array 0 name = Directivity_dBi
    Number Of Components: 1
    Number Of Tuples: 16471
    Copy Tuple Flags: ( 1 1 1 1 1 0 1 1 1 1 1 )
    Interpolate Flags: ( 1 1 1 1 1 0 0 1 1 1 1 )
    Pass Through Flags: ( 1 1 1 1 1 1 1 1 1 1 1 )
    Scalars:
      Debug: Off
      Modified Time: 377
      Reference Count: 1
      Registered Events: (none)
      Name: Directivity_dBi
      Data type: double
      Size: 16472
      MaxId: 16470
      NumberOfComponents: 1
      Information: 0x0
      Name: Directivity_dBi
      Number Of Components: 1
      Number Of Tuples: 16471
      Size: 16472
      MaxId: 16470
      LookupTable: (none)
    Vectors: (none)
    Normals: (none)
    TCoords: (none)
    Tensors: (none)
    GlobalIds: (none)
    PedigreeIds: (none)
    EdgeFlag: (none)
    Tangents: (none)
    RationalWeights: (none)
    HigherOrderDegrees: (none)
  Bounds:
    Xmin,Xmax: (-31.173, 29.0485)
    Ymin,Ymax: (-29.3445, 31.424)
    Zmin,Zmax: (-19.2903, 40.0666)
  Compute Time: 399
  Editable: false
  Number Of Points: 16471
  Point Coordinates: 0x600001528510
  PointLocator: 0x0
  CellLocator: 0x0
  Dimensions: (1, 91, 181)
  Extent: 0, 0, 0, 90, 0, 180
)

So it seems that the problem is in the reading itself rather than the filters.

mwestphal · November 15, 2023, 8:31am

Looks like the reader is providing only a single array. Try calling ReadAllScalarsOn() and ReadAllSVectorsOn() before updating.

WarinSpace_Game · November 15, 2023, 8:34am

Yes! That finally fixed it, thanks a lot!

That helps. Do we also know whether different points can be used for the second array?

WarinSpace_Game:

Perhaps another question I should ask in advance is this: if I also wish to assign different coordinate points, i.e. this part:
DATASET STRUCTURED_GRID
DIMENSIONS 1 91 181
POINTS 16471 double
-1.000000e+00 1.000000e+00 4.006663e+01
4.257683e-01 1.000000e+00 4.003194e+01
1.847900e+00 1.000000e+00 3.993014e+01
...
to the second set of the scalar data, can VTP support this? That’s my ultimate goal, so that the coordinates also change when the scalar values are different.

Another alternative would be to produce 2 .vtk files (which I can handle easily), but I’m not sure how we could merge them into a single .vtp.

mwestphal · November 15, 2023, 8:38am

That helps. Do we also know whether different points can be used for the second array?

You should either use another .vtp enterely or store the secondary points as a 3-component point data and switch the points using a dedicated filter (can be done with a calculator).

WarinSpace_Game · November 15, 2023, 9:27am

Is there any way this could be done in Glance? My goal is to easily toggle between the two by simply changing the selection from the dropdown:

which obviously changes the color, but not the coordinates of the points.

I could obviously export 2 separate .vtp files, but I’d prefer to toggle between the two using a dropdown instead of displaying it as if it is a completely different set of data.

mwestphal · November 15, 2023, 9:31am

I’m quite sure this cannot be done in glance out of the box.

jourdain · November 16, 2023, 3:13am

If you use time as a fake dimension, you can export your data and view it in glance so you can switch between 1 mesh and another but the format will have to be the “web export (.vtkjs)” from ParaView.

WarinSpace_Game · November 27, 2023, 6:14am

Thank you folks - I ended up using separate VTPs and everything works fine.

Quick question: what’s the proper way to close off the VTK reader? I seem to be getting some 'ascii' codec can't decode byte 0xe2 in position 9403: ordinal not in range(128) errors which only come up after my VTK operations, so it seems that some file readers might be interfering and need to be properly closed off:

# This works
with open("some_file.txt", "r") as file:
    file_content = file.read()

# Read VTK
reader = vtkStructuredGridReader()
reader.SetFileName(filename + ".vtk")

# Extract surface
surface_filter = vtkDataSetSurfaceFilter()
surface_filter.SetInputConnection(reader.GetOutputPort())

clean_filter = vtkCleanPolyData()
clean_filter.SetInputConnection(surface_filter.GetOutputPort())

# Save output to a VTP file
writer = vtkXMLPolyDataWriter()
writer.SetFileName(filename + ".vtp")
writer.SetInputConnection(clean_filter.GetOutputPort())
writer.Write()

# This fails
with open("some_file.txt", "r") as file:
    file_content = file.read()

mwestphal · November 27, 2023, 8:25am

You can try deleting the writer and the reader, but that looks like a bug to me.

WarinSpace_Game · November 27, 2023, 10:23am

I tried del reader, writer but the issue persists.

mwestphal · November 27, 2023, 10:31am

Call the python garbage collector ?

WarinSpace_Game · November 27, 2023, 12:06pm

Same thing happens after calling gc.collect() after the del.