Reverse connection breaks at the server side for MPI_ERR_COUNT: invalid count argument

An active reverse client-server connection breaks with the following notification at the server side:


Client connected.
( 129.953s) [pvserver.1 ] vtkMPICommunicator.cxx:71 WARN| MPI had an error
MPI_ERR_COUNT: invalid count argument
MPI_ABORT was invoked on rank 1 in communicator MPI_COMM_WORLD
with errorcode 2.
NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
You may or may not see output from other processes, depending on exactly when Open MPI kills them.
Loguru caught a signal: SIGTERM

The stack trace following this is available at https://paste.ubuntu.com/p/sZCRjN74zn/

The client then signals that the server side has disconnected, the application may be in an unrecoverable state, and so on and so forth

Would you have any suggestion to pinpoint the reason of the error above?

Difference of version between client and server

  • Nope: pvserver --version and paraview --version is 5.7.0
  • I also probed whether either side could have timed out the reverse connection. To that end I created the file ./ssh/ssh_config (with 600 permission as required by man ssh_config) and set ServerAliveInterval 30; I did this at both sides for good measure. To no avail.
  • In the end, most probably it was a malformed state file (pvsm) trying to upload data from the server side. The error arose within two minutes of having loaded the data. Now it has been going on seamlessly for more than 15 minutes.
  • I then also imagine that it was no limit in the client’s working memory.