"When SCP does not suffice"
Transferring a file with SSH

You all know scp as part of the OpenSSH suite, which copies files from and/or to a remote host. But when it comes to reading and/or writing to non-regular files/objects, scp stops working. Here is a short introduction how you can get around this unimplemented feature.

Back in the past, scp was just a wrapper script for ssh but later became its own application. Still, ssh itself has not lost the features it possessed those days. ssh is not only an interactive program that gives you a shell on a remote host, but also a non-interactive mode (which is e.g. very useful in cronjobs). Most of you probably do not know what's happening when you type

$ ssh user@host ls -l

but exactly that is a non-interactive session. In both modes, interactive and non-ia, the local STDIN/STDOUT is blindly connected to the remote STDIN/STDOUT -- otherwise you would not get any output from ls -l of course. Some might also have discovered by now that this works:

$ ssh user@host "ls -l" >lsoutput.txt

I.e. a redirection of ssh's output. SSH continues to be flexible -- you could use a filter program that resides on the remote host:

$ ssh -C user@host "tr A-Z a-z" <input.txt >output.txt

This will give input.txt. (-C will request compression.) The technicians might now wonder how this reliably works, because the remote side usually allocates a pty (pseudo terminal). Adding the -t option makes the trick visible: Pseudo-terminal will not be allocated because stdin is not a terminal. (This is very important because a text file just is not a terminal.) I still suggest using the -T option to explicitly deactivate allocating a pty.

As with text files, you can transfer anything else, even block devices. Before I will hand out the final command, there is one thing to pay attention. Try this: ssh user@localhost and then hit the tilde, then the period key. Now think of what happened if that sequence "~." was in your input file. (Luckily, this escape sequence, as documented in ssh(1) is disabled when no pty is allocated.) Still, it's best to supply -e none. The full command to do a network backup thus is:

# tar -cvj / | ssh -T -e none "cat >/backup.tar.bz2"

Or with a little shell squash and different encryption:

# tar -cvj / | ssh -Tcblowfish -enone "cat >/backup.tar.bz2"

Details to this command: tar packs all directories and files of the local host (starting at the root directory /) into an archive using bzip2 compression (-j) and writes this archive to STDOUT. (Typically for tar (with or without compression): the archive is output on-the-fly, so the RAM usage is very low.) ssh reads that from STDIN and passes the data to the remote host. cat's STDIN is the network socket, i.e. cat gets the archive on STDIN, and displays it to its STDOUT. (The nature of cat.) The invisble shell (!) then redirects it to the file backup.tbz2.

Additional keywords: on the fly pipe transmission