Hi,
I’m trying to understand how VTK/Catalyst handle communications. I see a class hierarchy going down from a vtkCommunicator class. I was assuming vtkCPProcessor::Initialize would be taking a pointer to such an abstract class, but actually it takes a pointer to a vtkMPICommunicatorOpaqueComm. Why is that? Is vtkCPProcessor capable of working only with an MPI-based communicator?
Thanks.
Yes, right now Catalyst is assuming MPI based communications since the pipelines it uses is going to be using MPI for parallel processing. The best example for this right now is the MPISubCommunicatorExample example.
I was hopping I could write child classes of vtkMultiProcessController and vtkCommunicator. Looking at how Catalyst depends on MPI, I can only see such a strong dependency in vtkCPProcessor. There is no reference to MPI in vtkCPPipeline…
I guess the real question is: if I call vtkMultiProcessController::SetGlobalControler with a pointer to an object inheriting from vtkMultiProcessController that is not an MPI controller, this global controller will then be picked up by vtkCPProcessor in vtkCPProcessor.cxx line 147. If I do that, should I be expecting Catalyst to blow up later on?
Yeah, what you’re trying to do should work. As long as your derived class for vtkMultiProcessController works properly within VTK and ParaView it should work properly within Catalyst. vtkCommunicator and its derived classed may not be needed since that is usually just a helper class for vtkMultiProcessController so depending on your implementation you may be able to skip doing anything there.
The only place where something may blow up is where a filter tries to get at the real MPI communicator. In VTK and ParaView this has traditionally been done in the parallel readers that use MPI IO so that’s not much of a worry in Catalyst. Now with using DIY2 in some of the filters though this could cause problems there too. I’m not familiar enough with how DIY2 is used inside of VTK to answer that question though. As far as I know, the VTKm filters don’t touch the vtkMultiProcessController so I think you’ll be fine there.
Thanks!
Actually, thinking a bit more about this I just grepped for places in ParaView and VTK that cast to the vtkMPIController and got:
[acbauer@oa ParaView]$ grep -r "vtkMPIController::SafeDownCast" *
ParaViewCore/ClientServerCore/Core/vtkProcessModule.cxx: vtkMPIController::SafeDownCast(vtkMultiProcessController::GetGlobalController()))
ParaViewCore/VTKExtensions/Default/vtkAMRDualGridHelper.cxx: vtkMPIController* controller = vtkMPIController::SafeDownCast(this->Controller);
ParaViewCore/VTKExtensions/Default/vtkAMRDualGridHelper.cxx: vtkMPIController* controller = vtkMPIController::SafeDownCast(this->Controller);
ParaViewCore/VTKExtensions/Default/vtkAMRDualGridHelper.cxx: vtkMPIController* controller = vtkMPIController::SafeDownCast(this->Controller);
ParaViewCore/VTKExtensions/Default/vtkAMRDualGridHelper.cxx: vtkMPIController* controller = vtkMPIController::SafeDownCast(this->Controller);
ParaViewCore/VTKExtensions/Default/vtkAMRConnectivity.cxx: vtkMPIController* mpiController = vtkMPIController::SafeDownCast(controller);
ParaViewCore/VTKExtensions/Default/vtkUnstructuredPOPReader.cxx: vtkMPIController::SafeDownCast(vtkMultiProcessController::GetGlobalController());
VTK/Domains/ParallelChemistry/vtkPSimpleBondPerceiver.cxx: vtkMPIController::SafeDownCast(vtkMultiProcessController::GetGlobalController());
VTK/Filters/ParallelDIY2/vtkDIYAggregateDataSetFilter.cxx: diy::mpi::communicator comm = GetDiyCommunicator(vtkMPIController::SafeDownCast(controller));
VTK/Filters/ParallelDIY2/vtkDIYAggregateDataSetFilter.cxx: vtkMPIController::SafeDownCast(vtkMultiProcessController::GetGlobalController());
VTK/Filters/ParallelFlowPaths/vtkPLagrangianParticleTracker.cxx: : Controller(vtkMPIController::SafeDownCast(vtkMultiProcessController::GetGlobalController()))
VTK/Filters/ParallelFlowPaths/vtkPStreamTracer.cxx: this->Controller = vtkMPIController::SafeDownCast(vtkMultiProcessController::GetGlobalController());
VTK/Filters/ParallelFlowPaths/vtkPStreamTracer.cxx: if (!vtkMPIController::SafeDownCast(this->Controller) || this->Controller->GetNumberOfProcesses() == 1)
VTK/Filters/ParallelGeometry/Testing/Cxx/UnstructuredGhostZonesCommon.cxx: vtkMPIController::SafeDownCast(
VTK/Filters/ParallelGeometry/vtkPConnectivityFilter.cxx: vtkMPIController::SafeDownCast(globalController)->PartitionController(hasCells, 0));
VTK/Filters/ParallelGeometry/vtkPDistributedDataFilter.cxx: if (vtkMPIController::SafeDownCast(this->Controller))
VTK/Filters/ParallelGeometry/vtkPDistributedDataFilter.cxx: vtkMPIController *mpiContr = vtkMPIController::SafeDownCast(this->Controller);
VTK/Filters/ParallelGeometry/vtkPDistributedDataFilter.cxx: vtkMPIController *mpiContr = vtkMPIController::SafeDownCast(this->Controller);
VTK/Filters/ParallelGeometry/vtkPDistributedDataFilter.cxx: vtkMPIController *mpiContr = vtkMPIController::SafeDownCast(this->Controller);
VTK/Filters/ParallelGeometry/vtkPDistributedDataFilter.cxx: vtkMPIController *mpiContr = vtkMPIController::SafeDownCast(this->Controller);
VTK/Filters/ParallelGeometry/vtkPDistributedDataFilter.cxx: vtkMPIController *mpiContr = vtkMPIController::SafeDownCast(this->Controller);
VTK/Filters/ParallelGeometry/vtkPDistributedDataFilter.cxx: vtkMPIController *mpiContr = vtkMPIController::SafeDownCast(this->Controller);
VTK/Filters/ParallelGeometry/vtkPDistributedDataFilter.cxx: vtkMPIController *mpiContr = vtkMPIController::SafeDownCast(this->Controller);
VTK/Filters/ParallelGeometry/vtkPStructuredGridConnectivity.cxx: vtkMPIController::SafeDownCast(this->Controller);
VTK/Filters/ParallelGeometry/vtkPStructuredGridConnectivity.cxx: vtkMPIController::SafeDownCast(this->Controller);
VTK/Filters/ParallelGeometry/vtkPStructuredGridConnectivity.cxx: vtkMPIController::SafeDownCast(this->Controller);
VTK/Filters/ParallelGeometry/vtkPUnstructuredGridGhostCellsGenerator.cxx: vtkMPIController::SafeDownCast(this->Controller)->PartitionController(hasCells, 0));
VTK/Filters/ParallelMPI/Testing/Cxx/TestImplicitConnectivity.cxx: vtkMPIUtilities::Printf(vtkMPIController::SafeDownCast(Controller),
VTK/Filters/ParallelMPI/Testing/Cxx/TestImplicitConnectivity.cxx: vtkMPIUtilities::Printf(vtkMPIController::SafeDownCast(Controller),
VTK/Filters/ParallelMPI/Testing/Cxx/TestImplicitConnectivity.cxx: vtkMPIController::SafeDownCast(Controller),"Grid has no gaps!\n");
VTK/Filters/ParallelMPI/Testing/Cxx/TestImplicitConnectivity.cxx: vtkMPIUtilities::Printf(vtkMPIController::SafeDownCast(Controller),
VTK/Filters/ParallelMPI/Testing/Cxx/TestImplicitConnectivity.cxx: vtkMPIController::SafeDownCast(Controller),"%s\n",oss.str().c_str());
VTK/Filters/ParallelMPI/Testing/Cxx/TestImplicitConnectivity.cxx: vtkMPIUtilities::Printf(vtkMPIController::SafeDownCast(Controller),
VTK/Filters/ParallelMPI/Testing/Cxx/TestImplicitConnectivity.cxx: vtkMPIController::SafeDownCast(Controller),"%s\n",oss.str().c_str());
VTK/Filters/ParallelMPI/Testing/Cxx/TestImplicitConnectivity.cxx: vtkMPIUtilities::Printf(vtkMPIController::SafeDownCast(Controller),
VTK/Filters/ParallelMPI/Testing/Cxx/TestImplicitConnectivity.cxx: vtkMPIController::SafeDownCast(Controller),"%s\n",oss.str().c_str());
VTK/Filters/ParallelMPI/Testing/Cxx/TestImplicitConnectivity.cxx: vtkMPIUtilities::Printf(vtkMPIController::SafeDownCast(Controller),
VTK/Filters/ParallelMPI/Testing/Cxx/TestImplicitConnectivity.cxx: vtkMPIController::SafeDownCast(Controller),"%s\n",oss.str().c_str());
VTK/Filters/ParallelMPI/Testing/Cxx/TestImplicitConnectivity.cxx: vtkMPIUtilities::Printf(vtkMPIController::SafeDownCast(Controller),
VTK/Filters/ParallelMPI/Testing/Cxx/TestImplicitConnectivity.cxx: vtkMPIController::SafeDownCast(Controller),"%s\n",oss.str().c_str());
VTK/Filters/ParallelMPI/Testing/Cxx/TestImplicitConnectivity.cxx: vtkMPIUtilities::Printf(vtkMPIController::SafeDownCast(Controller),
VTK/Filters/ParallelMPI/Testing/Cxx/TestImplicitConnectivity.cxx: vtkMPIUtilities::Printf(vtkMPIController::SafeDownCast(Controller),
VTK/Filters/ParallelMPI/vtkDistributedPointCloudFilter.cxx: vtkMPIController* controller = vtkMPIController::SafeDownCast(this->Controller);
VTK/Filters/ParallelMPI/vtkDistributedPointCloudFilter.cxx: kdTreeRounds.push_back(vtkMPIController::SafeDownCast(this->Controller));
VTK/Filters/ParallelMPI/vtkPExtractGrid.cxx: this->Controller = vtkMPIController::SafeDownCast(
VTK/Filters/ParallelMPI/vtkPExtractRectilinearGrid.cxx: this->Controller = vtkMPIController::SafeDownCast(
VTK/Filters/ParallelMPI/vtkPExtractVOI.cxx: this->Controller = vtkMPIController::SafeDownCast(
VTK/Filters/ParallelMPI/vtkStructuredImplicitConnectivity.cxx: this->Controller = vtkMPIController::SafeDownCast(
VTK/IO/ADIOS2/vtkADIOS2CoreImageReader.cxx: vtkMPIController* mpiController = vtkMPIController::SafeDownCast(controller);
VTK/IO/MPIImage/vtkMPIImageReader.cxx: = vtkMPIController::SafeDownCast(this->Controller);
VTK/IO/MPIImage/vtkPNrrdReader.cxx: = vtkMPIController::SafeDownCast(this->Controller);
VTK/IO/MPIParallel/vtkMPIMultiBlockPLOT3DReader.cxx: vtkMPIController::SafeDownCast(this->Controller) != nullptr);
VTK/IO/ParallelNetCDF/vtkPNetCDFPOPReader.cxx: this->SetController(vtkMPIController::SafeDownCast(
VTK/Rendering/ParallelLIC/vtkPPainterCommunicator.cxx: vtkMPIController *mpiController = vtkMPIController::SafeDownCast(controller);
So the testing stuff you’re safe with, the readers should be fine except for very rare instances of in situ use cases, anything using DIY2 is probably not going to work and a couple of filters will probably cause issues. Any questions on this stuff though you’ll just have to go through and work with people that know those parts of the code…