If you added support for non-blocking communication to the generic vtkCommunicator superclass, then I think it would make a lot of sense for IceT to just use that within ParaView/Catalyst. So, yes, that would be a useful contribution.
As an aside, yes, vtkMPICommunicator supports asynchronous communication, but it’s superclass does not. Since that only works with MPI anyway, I did not feel building a custom IceT communicator to use vtkMPICommunicator was any more useful than just using the MPI communicator that comes with IceT.