You're right, it didn't hurt
However, that doesn't change anything.
I should have mentioned, that this is not just when using ssh. I have a slimserver set up streaming music and that dies too when I access it from outside my FW. I believe slimserver "streams" in the way that it sends a chunk of music and waits for you to request the next track. So, in between tracks I get disconnected.