I am new to this community. I am one of the members of https://github.com/ladybug-tools
and we building tools for building performance simulation and visualization.
We are looking for a 3d web viewer that can load large geometry data sets. We write the 3d geometry and metadata into json format so that we can use it in different applications.
Which Paraview product should I look into to read and load json files? I have a sample json attached.
If you are looking to visualize geometry in the web, vtk.js can do just that. With your json file, this becomes a matter of file formats and I/O. Are the models in your JSON file defined according to some standard? Or is it a format specific to your suite of tools?
The JSONReader provided by vtk.js simply reads JSON into a JS object, so it’s not 100% applicable.
Chances are, you will need to write a custom reader to read your JSON file format into something vtk.js understands.
We’re not following any standard per see. Our json file describes geometry and the metadata both and we expected that we will need to write a connection.
What we can really appreciate some help on is the product that we should look into. We deal with a large models with several buildings in a neighborhood. We’re looking for a Paraview web product that can handle complexity at that scale.
How many triangles do you expect per scene? That’s one aspect that affects performance.
I would first suggest looking at vtk.js for your visualizations, as that is the easiest thing to evaluate. The quickest way to do that is to convert/export your scene in STL format and load it into ParaView Glance (a general-purpose viewer built on top of vtk.js). In addition, if Glance works for you and is to your liking, there’s the option to build your viewer off of Glance, or you can start from scratch with vtk.js if you need a different UX.
If your models and scenes are too complex for performant client-side rendering, then you might want to look into ParaViewWeb, which provides server-side rendering for a web-based frontend. (Here is some sample code for ParaViewWeb).
I was able to use Paraview Glance to load an .STL file I exported. But this file format only carries goemetry information. Am I correct? Metadata is very important to us.
Yes, that format only carries geometry information as far as I know. My suggestion to convert to STL was only to let you evaluate the performance of your large models and scenes in vtk.js to see if it fits your requirements.
In order to render your JSON files as models and keep your metadata in vtk.js, you will need to write a custom vtk.js reader that takes as input your JSON, and outputs vtk.js polydata, which can be rendered. I can give you pointers on that if you need it. (I would link to resources now but I’m on my phone.) Your reader can also return any metadata associated with your models. At that point, it’s up to you and your app as to how to manage your metadata. (An example would be clicking on a model in 3D and bringing up associated data with it in a side panel.)
I would add that if your data is “big” I would not go with JSON as it has a lot of overhead. One thing that we do inside vtk.js is to bundle into a zip some json files for the text based metadata and binary arrays for the actual geometry (xyz, connectivity, fields…). That allow us to still keep a decent size while using the best format for each piece we need.
Really appreciate your support. I spent a day to educate myself about different offerings of kitware. I watched this video that was helpful.
I really like what you mentioned and I started looking for a file format that I can use to write a box in vtk.js. It would really help if you can point me at some resource that shows me the structure (schema) of a vtk.js file and how to convert it into a binary vtk.js that I can load on Paraview Glance.
I was digging into this and I found this. This example is great in terms of showing the file format. Please correct me if I am wrong but you don’t recommend VTK format anymore. (Since you are calling it legacy)
Because you want to have your metadata, you still probably want your own format but in general you can see how our geometry data structure looks like and how we use it here and here. Where you can download and the data here and here (you will have to also fetch the binary arrays from the original metadata). Knowing those could all be bundled inside a zip…
This should give you plenty to think about how you want to encode your data.
Thank you for the resouces @jourdain but I had visited thos docs earlier. I feel even more lost now. I will try to reframe my question.
I found this resource quite usefule and it shows how a VTK file should be structured and the what should be the datatypes.
Is this not usefule at all if I need to use vtk.js? I somehow feel that there is a connection between VTK format and vtk.js that I am not able to figure out. Is there a converter already for vtk to vtk.js?
What I’m trying to say, is that you don’t need to use an existing format, because you have additional metadata that are not part of the vtk data model but you want to still make the link with your metadata and the mesh to some degree.
The vtkHttpDataSetReader should be a good start as it is mostly a simple JSON description pointing to binary arrays (float*) that get injected into a vtkPolyData for the xyz coordinates, triangles/polys connectivity and maybe some fields. If you want a single file, you can still bundle all the files inside a zip and your are still good to go.
What really matter is that you need to understand how a vtkPolyData is constructed. From your json the xyz array that I’m talking about is the same thing as a flat Float32Array of your “vertices”. Then the polys are basically your faces but like a Uint16Array = [face.length, idx0, idx…, face.length, idx0, idx…]
Here is a more concrete example of constructing geometry from scratch. It constructs a polydata that represents a 1.5x1.5 square.
import vtkPolyData from 'vtk.js/Sources/Common/DataModel/PolyData';
const polydata = vtkPolyData.newInstance();
// create our points that will be used to construct our polygons
const numberOfPoints = 4;
const points = new Float32Array(numberOfPoints * 3);
// add 4 points to construct our square
// each line is one 3D point.
points[0] = 0; point[1] = 0; point[2] = 0;
points[3] = 0; point[4] = 1.5; point[5] = 0;
points[6] = 1.5; point[7] = 1.5; point[8] = 0;
points[9] = 1.5; point[10] = 0; point[11] = 0;
// construct our cells. You can think of one polygon as one cell.
const polygons = new Uint32Array(1 + 4);
polygons[0] = 4; // next 4 values are indexes that refer to a point from our points array.
polygons[1] = 0; // first point (0, 0, 0)
polygons[2] = 1; // second point (0, 1.5, 0)
polygons[3] = 2; // third point (1.5, 1.5, 0)
polygons[4] = 3; // fourth point (1.5, 0, 0)
// the 3 means each point has 3 components (x,y,z)
polydata.getPoints().setData(points, 3);
polydata.getPolys().setData(polygons, 1);
Note the above code isn’t tested, but should be mostly correct.
You can also look at the sources files for Source generators such as ConeSource to see how geometry is generated. For you, you’d want to basically read in your JSON files and construct these polydata objects.
I want to thank you both for helping me earlier. Your help was instrumental in us realeasing honeybee-vtk. Documentation here.
This is our first release and now we are working on generating .glance files so that a user does not have to change the colors everytime they generate and load a new file.
Just looked at your to_vtk.py and ideally you want to create a single vtkPolyData without using vtkAppendPolyData().
It should be straight forward to do something along those lines
polydata = vtkPolyData()
points = vtkPoints()
polys = vtkCellArray()
polydata.SetPoints(points)
polydata.SetPolys(polys)
for cell in cells:
size = len(cell)
polys.InsertNextCell(size)
for point in cell:
pid = points.InsertNextPoint(tulpe(point))
polys.InsertCellPoint(pid)
.glance is essentially a .json file (named index.json). You can rename the file to .zip and unzip to see what a .glance file really is. You can customize this file to apply colors to your datasets and apply some other customization. We do it here
You need to have a .vtk/.vtp file for each of your dataset and have these files in one directory. Next, you also place the index.json in the same directory.
Once that is done, use this module to create a .vtkjs file. We do it here. This .vtkjs will have multiple datasets and you can load this .vtkjs in Paraview Glance.
Once, you assign scalars, you can follow the procedure I mentioned above and you will have a .vtkjs file that has a dataset with scalars. The coloring will appear when you load this .vtkjs in Paraview Glance.
Do you know if it is possible to also load a glance file into Paraview Glance and color with a range scalar?
I don’t think that’s how it works. glance essentially captures the state of your file so when you load the glance file again, you have the same .vtkjs file. So if you loaded a .vtkjs on Paraview Glance and let’s say you changed the color scheme of a scalar. Once you download a .glance file, it will have all the changes that you made in the .vtkjs that you loaded.
The glance file let you pick which field you want to look at (no need to do setScalars), provide a given lookup table (even a custom one), adjust the color range so it does not necessary match the range of the array you are looking at.
