Hi. I have developed a code which writes a VTKHDF file. I have visualized the file when it is generated using 20/30 cores. However, with 420 cores, paraview takes a lot of time and even then I am unable to visualize my file. Also, I have noticed unexpected quitting of paraview console any warning.
Any help or tips will be greatly appreciated. Is there a problem with VTKHDF for large number of processors? Does the visualization slow down in such cases?
Regards
Suman
It looks like you are using ParaView 5.11.1.
Please update to ParaView 5.13
Thanks Mathieu. Is version the issue?
Regards
Suman
Lots of improvements have been made in ParaView VTKHDF since 5.11, so I think updating would potentially fix the issue.
I downloaded Paraview 5.13 and tried opening my file. The problem still persists.
Regards
Suman
Hello @Suman_Vajjala
Is there a problem with VTKHDF for large number of processors?
Not AFAIK.
Does the visualization slow down in such cases?
indeed the visualization could have an impact, you could hack it by close the render view and open a spreadsheet view
Also, I have noticed unexpected quitting of paraview console any warning.
seems definitely be a bug, is it possible for you to share your data? btw what is his Type? (Unstructured Grid, …)
it would be useful to be sure that your data is a valid VTKHDF file
Hi, Lucas. My data is in the unstructured grid format. I have 17 datasets in the file. I can share the file but it is ~10 GB. How do I do it?
By the way, when I just select one dataset to load then I am able to visualize the data. So, I am guessing that the process is slow.
Regards
Suman
You can use google drive to share big files.
Any alternative platform? My google drive is almost at full capacity.
Regards
Suman
for 10GB, I don’t know, you can use any other platform that you trust
Dear @Lucas_Givord @mwestphal I’m still trying to figure out a way to share my dataset.
When I am loading my VTK HDF file, I get the following error message:
Loguru caught a signal: SIGSEGV
Stack trace:
76 0x40810a /home/vksuman/ParaView-5.11.1-MPI-Linux-Python3.9-x86_64/bin/paraview-real() [0x40810a]
75 0x7f52dad37495 __libc_start_main + 245
74 0x407d9a /home/vksuman/ParaView-5.11.1-MPI-Linux-Python3.9-x86_64/bin/paraview-real() [0x407d9a]
73 0x7f52d7f86ba3 QCoreApplication::exec() + 131
72 0x7f52d7f7da8a QEventLoop::exec(QFlagsQEventLoop::ProcessEventsFlag) + 298
71 0x7f528a762063 /home/vksuman/ParaView-5.11.1-MPI-Linux-Python3.9-x86_64/plugins/platforms/…/…/lib/libQt5XcbQpa.so.5(+0x58063) [0x7f528a762063]
70 0x7f52d8659f5b QWindowSystemInterface::sendWindowSystemEvents(QFlagsQEventLoop::ProcessEventsFlag) + 171
69 0x7f52d8680205 QGuiApplicationPrivate::processWindowSystemEvent(QWindowSystemInterfacePrivate::WindowSystemEvent*) + 261
68 0x7f52d867ed3b QGuiApplicationPrivate::processMouseEvent(QWindowSystemInterfacePrivate::MouseEvent*) + 1051
67 0x7f52d7f7f0c3 QCoreApplication::notifyInternal2(QObject*, QEvent*) + 259
66 0x7f52da0c052f QApplication::notify(QObject*, QEvent*) + 543
65 0x7f52da0b9bec QApplicationPrivate::notify_helper(QObject*, QEvent*) + 156
64 0x7f52da114a6b /home/vksuman/ParaView-5.11.1-MPI-Linux-Python3.9-x86_64/bin/…/lib/libQt5Widgets.so.5(+0x1b9a6b) [0x7f52da114a6b]
63 0x7f52da111cda /home/vksuman/ParaView-5.11.1-MPI-Linux-Python3.9-x86_64/bin/…/lib/libQt5Widgets.so.5(+0x1b6cda) [0x7f52da111cda]
62 0x7f52da0bfb3c QApplicationPrivate::sendMouseEvent(QWidget*, QMouseEvent*, QWidget*, QWidget*, QWidget**, QPointer&, bool, bool) + 460
61 0x7f52d7f7f0c3 QCoreApplication::notifyInternal2(QObject*, QEvent*) + 259
60 0x7f52da0c13a7 QApplication::notify(QObject*, QEvent*) + 4247
59 0x7f52da0b9bec QApplicationPrivate::notify_helper(QObject*, QEvent*) + 156
58 0x7f52da0f7d88 QWidget::event(QEvent*) + 424
57 0x7f52da1a15b5 QAbstractButton::mouseReleaseEvent(QMouseEvent*) + 213
56 0x7f52da1a13ea /home/vksuman/ParaView-5.11.1-MPI-Linux-Python3.9-x86_64/bin/…/lib/libQt5Widgets.so.5(+0x2463ea) [0x7f52da1a13ea]
55 0x7f52da1a004a /home/vksuman/ParaView-5.11.1-MPI-Linux-Python3.9-x86_64/bin/…/lib/libQt5Widgets.so.5(+0x24504a) [0x7f52da1a004a]
54 0x7f52da19fe62 QAbstractButton::clicked(bool) + 50
53 0x7f52d7fba423 /home/vksuman/ParaView-5.11.1-MPI-Linux-Python3.9-x86_64/bin/…/lib/libQt5Core.so.5(+0x2e2423) [0x7f52d7fba423]
52 0x7f52d9977cdf pqPropertiesPanel::apply() + 671
51 0x7f52d9817322 pqPropertiesPanel::applied(pqProxy*) + 50
50 0x7f52d7fba634 /home/vksuman/ParaView-5.11.1-MPI-Linux-Python3.9-x86_64/bin/…/lib/libQt5Core.so.5(+0x2e2634) [0x7f52d7fba634]
49 0x7f52da89e87e pqApplyBehavior::applied(pqPropertiesPanel*, pqProxy*) + 206
48 0x7f52da89fbde pqApplyBehavior::showData(pqPipelineSource*, pqView*) + 414
47 0x7f52c961c40c vtkSMParaViewPipelineControllerWithRendering::ShowInPreferredView(vtkSMSourceProxy*, int, vtkSMViewProxy*) + 172
46 0x7f52c9619d04 vtkSMParaViewPipelineControllerWithRendering::UpdatePipelineBeforeDisplay(vtkSMSourceProxy*, int, vtkSMViewProxy*) + 212
45 0x7f52d70eb9fc vtkSMSourceProxy::UpdatePipeline(double) + 140
44 0x7f52d70571e8 vtkSMOutputPort::UpdatePipelineInternal(double, bool) + 184
43 0x7f52d6f91d35 vtkPVSessionBase::ExecuteStream(unsigned int, vtkClientServerStream const&, bool) + 53
42 0x7f52d6f9301b vtkPVSessionCore::ExecuteStream(unsigned int, vtkClientServerStream const&, bool) + 59
41 0x7f52d6f931e2 vtkPVSessionCore::ExecuteStreamInternal(vtkClientServerStream const&, bool) + 242
40 0x7f52d5b64b0d vtkClientServerInterpreter::ProcessStream(vtkClientServerStream const&) + 29
39 0x7f52d5b6486e vtkClientServerInterpreter::ProcessOneMessage(vtkClientServerStream const&, int) + 1294
38 0x7f52d5b6413d vtkClientServerInterpreter::ProcessCommandInvoke(vtkClientServerStream const&, int) + 1229
37 0x7f52d5b63973 vtkClientServerInterpreter::CallCommandFunction(char const*, vtkObjectBase*, char const*, vtkClientServerStream const&, vtkClientServerStream&) + 355
36 0x7f52ca543e6e vtkSIMetaReaderProxyCommand(vtkClientServerInterpreter*, vtkObjectBase*, char const*, vtkClientServerStream const&, vtkClientServerStream&, void*) + 446
35 0x7f52d5b63973 vtkClientServerInterpreter::CallCommandFunction(char const*, vtkObjectBase*, char const*, vtkClientServerStream const&, vtkClientServerStream&) + 355
34 0x7f52ca54913b vtkSISourceProxyCommand(vtkClientServerInterpreter*, vtkObjectBase*, char const*, vtkClientServerStream const&, vtkClientServerStream&, void*) + 971
33 0x7f52d6fd0154 vtkSISourceProxy::UpdatePipeline(int, double, bool) + 468
32 0x7f52d0142d7a vtkStreamingDemandDrivenPipeline::Update(int, vtkInformationVector*) + 282
31 0x7f52d00d163a vtkDemandDrivenPipeline::UpdateData(int) + 138
30 0x7f52d01414f9 vtkStreamingDemandDrivenPipeline::ProcessRequest(vtkInformation*, vtkInformationVector**, vtkInformationVector*) + 953
29 0x7f52d00d301e vtkDemandDrivenPipeline::ProcessRequest(vtkInformation*, vtkInformationVector**, vtkInformationVector*) + 222
28 0x7f52d00cb651 vtkCompositeDataPipeline::ForwardUpstream(vtkInformation*) + 337
27 0x7f52d01414f9 vtkStreamingDemandDrivenPipeline::ProcessRequest(vtkInformation*, vtkInformationVector**, vtkInformationVector*) + 953
26 0x7f52d00d307a vtkDemandDrivenPipeline::ProcessRequest(vtkInformation*, vtkInformationVector**, vtkInformationVector*) + 314
25 0x7f52d00cc779 vtkCompositeDataPipeline::ExecuteData(vtkInformation*, vtkInformationVector**, vtkInformationVector*) + 121
24 0x7f52d00cfc57 vtkDemandDrivenPipeline::ExecuteData(vtkInformation*, vtkInformationVector**, vtkInformationVector*) + 55
23 0x7f52d00d76d0 vtkExecutive::CallAlgorithm(vtkInformation*, int, vtkInformationVector**, vtkInformationVector*) + 64
22 0x7f52c99a90f5 vtkFileSeriesReader::ProcessRequest(vtkInformation*, vtkInformationVector**, vtkInformationVector*) + 309
21 0x7f52c99a8ea5 vtkFileSeriesReader::RequestData(vtkInformation*, vtkInformationVector**, vtkInformationVector*) + 181
20 0x7f52a88bae57 vtkHDFReader::RequestData(vtkInformation*, vtkInformationVector**, vtkInformationVector*) + 279
19 0x7f52a88babdd vtkHDFReader::Read(vtkInformation*, vtkUnstructuredGrid*) + 285
18 0x7f52d0142d7a vtkStreamingDemandDrivenPipeline::Update(int, vtkInformationVector*) + 282
17 0x7f52d00d163a vtkDemandDrivenPipeline::UpdateData(int) + 138
16 0x7f52d01414f9 vtkStreamingDemandDrivenPipeline::ProcessRequest(vtkInformation*, vtkInformationVector**, vtkInformationVector*) + 953
15 0x7f52d00d307a vtkDemandDrivenPipeline::ProcessRequest(vtkInformation*, vtkInformationVector**, vtkInformationVector*) + 314
14 0x7f52d00cc779 vtkCompositeDataPipeline::ExecuteData(vtkInformation*, vtkInformationVector**, vtkInformationVector*) + 121
13 0x7f52d00cfc57 vtkDemandDrivenPipeline::ExecuteData(vtkInformation*, vtkInformationVector**, vtkInformationVector*) + 55
12 0x7f52d00d76d0 vtkExecutive::CallAlgorithm(vtkInformation*, int, vtkInformationVector**, vtkInformationVector*) + 64
11 0x7f52d0523d61 vtkAppendDataSets::ProcessRequest(vtkInformation*, vtkInformationVector**, vtkInformationVector*) + 241
10 0x7f52d0524458 vtkAppendDataSets::RequestData(vtkInformation*, vtkInformationVector**, vtkInformationVector*) + 648
9 0x7f52d0142d7a vtkStreamingDemandDrivenPipeline::Update(int, vtkInformationVector*) + 282
8 0x7f52d00d163a vtkDemandDrivenPipeline::UpdateData(int) + 138
7 0x7f52d01414f9 vtkStreamingDemandDrivenPipeline::ProcessRequest(vtkInformation*, vtkInformationVector**, vtkInformationVector*) + 953
6 0x7f52d00d307a vtkDemandDrivenPipeline::ProcessRequest(vtkInformation*, vtkInformationVector**, vtkInformationVector*) + 314
5 0x7f52d00cc779 vtkCompositeDataPipeline::ExecuteData(vtkInformation*, vtkInformationVector**, vtkInformationVector*) + 121
4 0x7f52d00cfc57 vtkDemandDrivenPipeline::ExecuteData(vtkInformation*, vtkInformationVector**, vtkInformationVector*) + 55
3 0x7f52d00d76d0 vtkExecutive::CallAlgorithm(vtkInformation*, int, vtkInformationVector**, vtkInformationVector*) + 64
2 0x7f52d0526742 vtkAppendFilter::RequestData(vtkInformation*, vtkInformationVector**, vtkInformationVector*) + 2242
1 0x7f52cf501d00 vtkUnstructuredGrid::GetCellPoints(long long, vtkIdList*) + 192
0 0x7f52dad4b340 /lib64/libc.so.6(+0x36340) [0x7f52dad4b340]
( 65.809s) [paraview ] :0 FATL| Signal: SIGSEGV
This is when I used 1200 series. With 50 processors, my dataset loads just fine on Paraview. Do I have any errors in creating the HDF file?
Regards
Suman
Hello @Suman_Vajjala,
it’s hard to say without the data however based on the backtrace (crash in CellPoints of the unstructured grid) you should check how you treat the Connectivity, maybe you have some index issue in this array?
I’m still trying to figure out a way to share my dataset.
If the issue is related to the number of processors, is it possible for you to test and try to reproduce the bug with a smaller vtkhdf dataset?
@Lucas_Givord @mwestphal I am sharing my VTK HDF file. Please let me know why I am having errors visualizing it.
