How to use the Linux ‘scp’ command without a password to make remote backups

Summary: This article is about how to create a public and private key pair so you can use the Linux ssh and scp commands to connect to a remote server without using a password. In the last example I show, this lets me automate a remote server backup process.

Introduction

Over the last few years I’ve ended up creating a large collection of websites and web applications on a variety of Linux servers that are hosted with different companies like GoDaddy, A2 Hosting, and Linode. I recently embarked on a mission to automate the backup processes for all these sites, and as a result of this effort, I thought I’d share what I’ve learned.

As a result, after reading this tutorial, you will have learned how to:

• Create a public and private key pair.
• Install your public key on your remote Unix and Linux servers.
• Use ssh to login to your remote servers without using a password.
• Use ssh to run commands (such as backup scripts) on your remote servers without using a password.
• Use scp to copy files to and from your remote servers without a password.

If you’re ready, let’s begin.

Step 1: Generate a public and private key pair

The following Unix/Linux commands (and resulting system output) demonstrate how to create a public and private key pair on your local Unix computer system.

When I run this first command on my Mac laptop — which is a MacOS system (which is itself a version of Unix) — it creates two files in a directory named .ssh, which is located in my home directory.

So that directory is named /Users/al/.ssh

The two files it creates are named id_rsa and id_rsa.pub. The first file (id_rsa) contains my private key, and the second file (id_rsa.pub) contains my public key.

It's important to note that if you just go with the defaults, as I'm about to show, and you already have a file named id_rsa, your system should warn you, like this:

$ ssh-keygen -t rsa

Generating public/private rsa key pair.
Enter file in which to save the key (/Users/al/.ssh/id_rsa): 
/Users/al/.ssh/id_rsa already exists.
Overwrite (y/n)? _

If you’re very comfortable with this process and you want to overwrite your id_rsa file, do what you're comfortable with.

However, if this is the first time you’ve ever done this, and you get this warning message, for some reason this file has already been created, so be very careful about how you proceed. In fact, I recommend that you don't proceed until you understand more about what's happening here.

Note 1: You can also have the system write this output to a different file, but since I’m essentially writing this for people new to generating public/private key pairs, the point of this paragraph is that your current id_rsa is there for a reason, so be careful.

Note 2: If you want to write the output to a different file, see this note about How to use ssh when your private key file is not named id_rsa.

Given that warning, here’s how I just created a public and private key pair on my current MacBook. What I typed is shown in bold, and what the system generated as output is shown in a normal font. Note that in this step you’ll want to use a passphrase, so make sure you are ready for that as well:

$ ssh-keygen -t rsa

Generating public/private rsa key pair.
Enter file in which to save the key (/Users/al/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase):                   <-- i use a passphrase here
Enter same passphrase again:                                  <-- and here
Your identification has been saved in /Users/al/.ssh/id_rsa.
Your public key has been saved in /Users/al/.ssh/id_rsa.pub.
The key fingerprint is:
6f:16:29:90:46:b6:88:34:3d:81:07:fc:bd:1a:fc:db al@Al-Alexanders-MacBook.local
The key's randomart image is:
+--[ RSA 2048]----+
| .++..o          |
| .oo++ o         |
|  .o.o=          |
|    ....   .     |
|   .   .S o      |
|    o .  o .     |
|     +    +      |
|    . .. o       |
|      ..E        |
+-----------------+

As you can see from the output of this command:

• Your private key is in a file named ~/.ssh/id_rsa
• Your public key is in a file named ~/.ssh/id_rsa.pub

Feel free to use vi or cat to look at both files if you like, but don't change them. As you’ll see, they are both plain text files.

You should never give your private key to anyone else, so for all intents and purposes, the id_rsa file will just stay right where it is.

As for your public key (id_rsa.pub), you're going to copy that to your remote servers in the following step.

Step 2: Copy your public key to your remote servers

The next step is to copy the id_rsa.pub file to the remote server you want to be able to access with ssh and/or scp without using a password. For the purposes of this article, I'll refer to that remote system pluto.example.com. (Pluto seems pretty remote.)

First, scp that file to pluto as you normally would, supplying a password during the scp process:

$ scp id_rsa.pub al@pluto.example.com:./

If you're not familiar with the scp command, it lets you securely copy files from one computer system to another, essentially working on top of the ssh command. (The ./ at the end of that command puts the file in my home directory on pluto.)

Next, ssh into pluto, again supplying your password when prompted:

$ ssh al@pluto.example.com

On pluto, if the .ssh directory does not already exist in your home directory, create it. Assuming you are in your home directory, just create it like any other directory:

$ mkdir .ssh

Now copy the id_rsa.pub file to a new file named authorized_keys in that .ssh directory, like this:

$ cp id_rsa.pub .ssh/authorized_keys

Assuming those steps worked without error, if you now cd into your .ssh directory:

$ cd .ssh

and then use the ls command, you’ll see your file in this directory with the proper name:

$ ls -al

total 12
drwxr-xr-x 2 al al 4096 Jul 21 17:45 .
drwx---r-x 9 al al 4096 Jul 10 10:46 ..
-rw-r--r-- 1 al al  419 Jul 21 17:46 authorized_keys

Again assuming that all these steps worked, you’re now ready to test your remote login without using a password.

Step 3: Test your ssh login

To test that you can log in to your remote server using ssh without a password, either (a) logout of your remote server, or (b) open a new ssh window on your local computer.

In your ssh/terminal window, all you have to do to test the system is to attempt to login to your remote server as you did before:

$ ssh al@pluto.example.com

If everything is setup properly, you will be logged into your remote server without having to enter a password.

Note: The first time you do this, you may have to enter the passphrase you used with the ssh-keygen command. After that, you won’t need to use that passphrase again to log in.

If everything worked, congratulations, your computer systems are now ready to use your public and private key pair to let you use ssh and scp without having to enter a password.

And if you’re interested in making automated scp backups as I am, you’re now ready to take those steps.

Step 4: Install your backup scripts on the remote servers

Where you go from here depends on your backup process, but I'll describe my approach, and let you take it from there.

In short, on all of my remote servers I have the exact same setup, which consists of a 'backup' directory in my home directory of each server. I put two scripts in that directory, one to backup my MySQL data:

/home/al/backup/mysql-bu.sh

and a second script that I use to backup my web/HTML directory:

/home/al/backup/html-bu.sh

Both of these scripts have the execute bits set (file permission is "-rwxr-xr-x", or 755), and both scripts write their output to files in this same directory. The mysql backup file will be named pluto.mysql.gz, and the HTML/www backup file will be named pluto.html.tgz.

Step 5: Run your backup scripts with ssh

Given this setup, I can now use ssh to remotely run my backup scripts. I run the remote MySQL backup script like this:

ssh al@pluto.example.com "/home/al/backup/mysql-bu.sh"

and I run the HTML/www backup script (which is basically just a wrapper around the tar command) like this:

ssh al@pluto.example.com "/home/al/backup/html-bu.sh"

As mentioned, this creates two files in the current directory with names like these:

pluto.mysql.gz
pluto.html.tgz

Now I'm ready for my scp command.

Step 6: Use scp to copy your backup files back home

With these backup files created, I can now scp them back to my local computer without using a password. Because of the naming convention used, I can just use one scp command to copy the files back to my local system:

scp al@pluto.example.com:/home/al/backup/*gz .

When this command finishes running, those two backup files have been copied to my local computer, and my work is done.

ssh, scp, backups, and public/private key pairs

I simplified a few things in this article so we could just focus on the essentials, but I hope you can see how this system lets us create a remote server backup system that we can automate. As you can imagine, we can now create a shell script with a for loop like this that can be used to automate the backup and download process for all our planetary systems:

# skipping a few planets ...
for planet in mercury venus earth mars pluto
do
  ssh al@pluto.example.com "/home/al/backup/mysql-bu.sh"
  ssh al@pluto.example.com "/home/al/backup/html-bu.sh"
  scp al@pluto.example.com:/home/al/backup/*gz .
  echo "$planet has been backed up"
done

Of course you can automate the running of the backup scripts using cron/crontab on each system, but again my purpose for this article is to demonstrate the ssh/scp backup approach.

My backup scripts

Before I go, I'll also share my backup scripts. My MySQL backup script has actually been out here for quite some time, so I'll just link to it:

• My MySQL backup shell script

And as mentioned, my HTML directory backup script is basically a wrapper around the tar command. Here's a simplified version of what I actually use:

#!/bin/sh

cd /home/al/html
tar czvf /home/al/backup/pluto-html.tgz .

My actual backup script is more complicated than this, because my remote servers are with a variety of hosting companies (GoDaddy, A2 Hosting, etc.) with different directory structures, but in the end, that's all that HTML backup script does.

Summary: Using scp to make remote backups without a password

I hope this tutorial on how to use scp to make automated remote backups without using a password has been helpful. As you saw, what I demonstrated in this article is how to:

• Create a public and private key pair
• Install your public key on remote Unix and Linux servers
• ssh into your remote servers without a password
• Use ssh to run commands on your remote servers without using a password
• Use scp to copy files to and from your remote servers without a password

If you'd like to use the cron/crontab command to further automate this approach, my website has many Linux crontab examples, and you can easily combine what's in those tutorials with the information in this tutorial.

For more information on public and private keys, see this Wikipedia article on public key cryptography. Also, kudos to this Linux journal article for first demonstrating much of this approach. If you happen to be a Java programmer, the Java programming language offers a similar approach, which I describe in my Java keytool and keystore tutorials.