If your workplace is any similar to mine, you have to open an SSH connection within another SSH connection to remotely access your workstation: Matryosshka!
lucas@laptop ><((("> ssh email@example.com firstname.lastname@example.org ><((("> ssh lb@workstation lb@workstation ><(((">
There are various reasons for such a setup, which we will let admins discuss; we devs just have to accept it as a fact of life.
I'm mostly working on my notebook (I don't like fixed desks too much) and the Wi-Fi happens to be on a different network than the workstations. This makes (at least) two things more complicated than they'd otherwise be:
The latter is especially awesome, since it'd allow me to use the notebook on my notebook (ok, I'll call the latter laptop from now on) while all code runs on the more powerful workstation.
SSH and netcat, a match made in heaven
nc) is a nifty tool which,
at its core, creates a connection and lets you send anything to it.
A bit like a lower-level
telnet on steroids. It can be used for many things,
the simplest being exploring a network protocol:
><((("> nc lucasb.eyer.be 80 GET / HTTP/1.0 Host: lucasb.eyer.be [OH CRAP SO MUCH HTML]
After having pressed
enter twice (because the HTTP protocol requires that),
you should get my home page sent back to you.
Ad-hoc file transfer
Another use-case, just as useful, is ad-hoc file transfer. On the receiving machine, run
><((("> nc -l -p 1337 >out.file
This command instructs netcat to listen on port
1337 and the
>out.file tells your shell not to print stdout to the screen but rather
write it to the file
out.file. (Note that there are multiple versions of
netcat in the wild, some of which need you to specify the port with
and some of which forbid it. Read your
man nc.) Now that netcat is
listening, on the sending machine, run:
><((("> nc receiver 1337 <in.file
This opens a connection to
receiver (you might want to put an IP address
here) on port
1337 and writes the content of
in.file to that connection.
There are 3 caveats here:
- The receiver needs to be able to listen on said port, i.e. firewalls, routers and ISPs can cause problems.
- The receiving end doesn't know when transfer is over.
- There is no progress indicator.
It is easy to work around the latter two by using your brain or google.
Back to SSH
Now that we understood netcat, let's go back to our SSH. There are two features of SSH we'll use. The first one is rather trivial: if called with superfluous arguments, SSH will run these as a command on the remote host and then disconnect. For example,
><((("> ssh email@example.com ls -l firstname.lastname@example.org's password: total 12345 drwxr-xr-x 4 user group 4096 date secret_folder -rw-r--r-- 1 user group 7811485 date trade_document.pdf [...] ><(((">
The second one is slightly more involved: the
ProxyCommand option. From the
Specifies the command to use to connect to the server. The command string extends to the end of the line, and is executed with the user's shell. In the command string, any occurrence of '%h' will be substituted by the host name to connect, '%p' by the port, and '%r' by the remote user name. The command can be basically anything, and should read from its standard input and write to its standard output. [...]
This directive is useful in conjunction with nc(1) and its proxy support. For example, the following directive would connect via an HTTP proxy at 192.0.2.0:
ProxyCommand /usr/bin/nc -X connect -x 192.0.2.0:8080 %h %p
This gives us the needed hint. Basically, what it means is that SSH will first
run whatever command you're passing (on
laptop) and "communicate" to that
command through stdin/stdout. Sounds like a fit made in heaven for netcat.
The final command
><((("> ssh -o "ProxyCommand ssh email@example.com nc %h %p" lb@workstation
That's the magic command.
ProxyCommand, it will first open an SSH connection to
in which it will start netcat to connect to
SSH wants to send will use the "inner" SSH to travel between
company.com and then netcat to travel between
If you understood that, it should be obvious that this use of netcat is no
Making it more convenient
After using this for some time, you'll be annoyed by having to enter such a long command and your password twice, every time. In this case, use one or two SSH keypairs. Or maybe you're only allowed to use SSH with keypairs. In any case, there are plenty of good tutorials online about just that.
Once you've got the keypairs ready, you can add the following to your
~/.ssh/config to ease the pain:
host company HostName company.com User beyer IdentityFile ~/.ssh/company host workstation HostName workstation User lb IdentityFile ~/.ssh/workstation ProxyCommand ssh company nc %h %p
You can now simply use
workstation in any
scp) commands on
><((("> scp results.csv workstation:/home/lb/results.csv ><((("> ssh workstation lb@workstation$
Killed by signal 1
Whenever you leave the double-tunnel (using
Ctrl+d of course!), you will be
greeted by a weird "Killed by signal 1" message:
><((("> ssh workstation lb@workstation$ [Ctrl+d] Connection to workstation closed. Killed by signal 1. ><(((">
This doesn't mean anything bad, but it can become annoying. POSIX signal 1 is
SIGHUP, which stands for "hang up" and dates back to the modem days and means
that the controlling terminal has been closed. You might have learned the trick
of starting a program with
nohup to let it outlive your terminal window;
nohup just eats up the
SIGHUP message that's being sent to the program when
you close the window.
In our case, the message comes from the
ssh executed in the
option, which tells you it exits because its controlling terminal, the actual
SSH command, has been closed. Of course, that's what we did. You can get rid of
this message by telling
ssh to be quiet (
-q) except for errors. So the
fixed line would be:
ProxyCommand ssh -q company nc %h %p
Jupyter (IPython) notebook
Finally, thanks to having set-up your
~/.ssh/config file as above, you can
follow any of the tutorials online on how to run the Jupyter (ex-IPython)
notebook server on
workstation and access it in your browser on
Here's a quick reference anyways.
You can either connect to
workstation and start the notebook server there,
ideally in a tmux session so you can reattach to it later:
lucas@notebook ><((("> ssh workstation lb@workstation ><((("> tmux lb@workstation ><((("> ipython notebook --no-browser --port=1337
Now type in
:det and then hit the return key. You can now
disconnect from that SSH session (
Ctrl+d) and later reattach to it by typing
tmux ls to see the active sessions and then
tmux at -t SESSION_ID.
If you just want a quick session, which will die as soon as your laptop disconnects, you can also just run
lucas@notebook ><((("> ssh workstation ipython notebook --no-browser --port=1337
But I'd rather go with the first way so that IPython survives me walking around with the laptop. Now that the server is running, create a local redirect:
lucas@notebook ><((("> ssh -N -f -L localhost:8080:localhost:1337 workstation
This creates a forwarding from
localhost:1337 (and here comes the non-obvious part) on
You can optionally drop the first
localhost since it's implied, i.e. just
The mnemonic I use to remember that command is the national football league, where the football itself is small compared to our (European) football, hence lowercase f.
PS: Tasty copy-pasta for self:
ssh -N -f -L :8080:localhost:1337 grolsch