# Luigi ssh tests
**URL:** https://discourse.paraview.org/t/luigi-ssh-tests/2675
**Category:** Dashboards
**Created:** [October 3, 2019, 3:28am UTC](https://discourse.paraview.org/t/luigi-ssh-tests/2675 "2019-10-03T03:28:52Z")
**Posts on this page:** 9
**Page:** 1
### Author:  [@mwestphal](https://discourse.paraview.org/u/mwestphal)
#### Post date: [October 3, 2019, 3:28am UTC](https://discourse.paraview.org/t/luigi-ssh-tests/2675/1 "2019-10-03T03:28:52Z")
SSH tests have been acting up lately.
I’m keeping an eye on it and see if that can be improved.
---
### Author:  [@cory.quammen](https://discourse.paraview.org/u/cory.quammen)
#### Post date: [October 3, 2019, 2:12pm UTC](https://discourse.paraview.org/t/luigi-ssh-tests/2675/2 "2019-10-03T14:12:12Z")
👍
---
### Author:  [@mwestphal](https://discourse.paraview.org/u/mwestphal)
#### Post date: [October 7, 2019, 6:30am UTC](https://discourse.paraview.org/t/luigi-ssh-tests/2675/3 "2019-10-07T06:30:24Z")
Here is what’s happening. One of the SSH test rarely fail for a yet to be identified reason, killing the paraview executable. However, the server that have been run by the test is not killed, since it is not managed by smTestDriver but actually run with .sh script, like a real life pvserver.
Once a remaining pvserver is running, not a single SSH test will be able to pass, as they all uses the same port.
I do hope to fix the yet to be identified reason, but this shows a (small) design problem with these test, of which I’m sole responsible 🙂.
Indeed, a failing test should just be rerun and pass, it should not fail and break all the following build.
I see a few mitigations possible :
1. Use random port in the SSH tests : no way to do that currently, and in any case, each failing test will let a pvserver process on the buildbot.
2. Kill all pvserver after each build : Overkill but may be a temporary solution
3. Use smTestDriver to run the pvserver : I’m not sure that it is possible, but if smTestDriver could be carefully told how to configure it’s pvserver, that may work.
Let me know what you think @utkarsh.ayachit @cory.quammen
---
### Author:  [@cory.quammen](https://discourse.paraview.org/u/cory.quammen)
#### Post date: [October 7, 2019, 6:30pm UTC](https://discourse.paraview.org/t/luigi-ssh-tests/2675/4 "2019-10-07T18:30:17Z")
Nice detective work! For a quick stopgap, option 2 seems reasonable. In the long run, making smTestDriver work is probably the way to go.
---
### Author:  [@mwestphal](https://discourse.paraview.org/u/mwestphal)
#### Post date: [October 8, 2019, 4:18am UTC](https://discourse.paraview.org/t/luigi-ssh-tests/2675/5 "2019-10-08T04:18:59Z")
@ben.boeckel : Is option 2 doable ? If yes, could you do it ?
---
### Author:  [@utkarsh.ayachit](https://discourse.paraview.org/u/utkarsh.ayachit)
#### Post date: [October 8, 2019, 5:58pm UTC](https://discourse.paraview.org/t/luigi-ssh-tests/2675/6 "2019-10-08T17:58:23Z")
are these tests run serially? If so, can the server launch script that you have do a `killall -9 pvserver` before starting the pvserver process? That’ll avoid requiring any changes on the buildbot/test runner side.
---
### Author:  [@ben.boeckel](https://discourse.paraview.org/u/ben.boeckel)
#### Post date: [October 8, 2019, 7:27pm UTC](https://discourse.paraview.org/t/luigi-ssh-tests/2675/7 "2019-10-08T19:27:13Z")
`smTestDriver` can certainly run `pvserver`. See the argument slinging done in `CMake/ParaViewTesting.cmake` to get arguments to the server and such to that program.
---
### Author:  [@mwestphal](https://discourse.paraview.org/u/mwestphal)
#### Post date: [October 9, 2019, 2:50am UTC](https://discourse.paraview.org/t/luigi-ssh-tests/2675/8 "2019-10-09T02:50:18Z")
Thanks for your suggestions !
---
### Author:  [@mwestphal](https://discourse.paraview.org/u/mwestphal)
#### Post date: [October 23, 2019, 11:43am UTC](https://discourse.paraview.org/t/luigi-ssh-tests/2675/9 "2019-10-23T11:43:53Z")
Turns out it is much more complex than just killing pvserver.
One of the tests using the “Terminal” mechanism that enables to spawn a terminal to run the server in with a reverse tunelling ends up leaving a the ssh tunnel alive if pvserver is killed with a SIGINT or a SIGTERM.
I’ve implemented a quick and dirty fix to make sure the dashboard stays clean. This should enable me to see if the test still fails occasionally and investigate more deeply.
[https://gitlab.kitware.com/paraview/paraview/merge\_requests/3626](https://gitlab.kitware.com/paraview/paraview/merge_requests/3626)