Summary: This article is about how to create a public and private key pair so you can use
scp without using a password, which in my example lets me automate a remote server backup process.
Over the last two years I’ve ended up creating a large collection of websites and web applications on a variety of Linux servers that are hosted with different companies like GoDaddy and A2 Hosting. I recently embarked on a mission to automate the backup processes for all these sites, and as a result of this effort, I thought I’d share what I’ve learned.
As a result, after reading this tutorial, you will have learned how to:
- Create a public and private key pair.
- Install your public key on your remote Unix and Linux servers.
- Use ssh to login to your remote servers without using a password.
- Use ssh to run commands (such as backup scripts) on your remote servers without using a password.
- Use scp to copy files to and from your remote servers without a password.
If you’re ready, let’s begin.
Step 1: Generate a public and private key pair
The following Unix/Linux commands (and resulting system output) demonstrate how to create a public and private key pair on your local Unix computer system.
When I run this command on my Mac laptop, which is a MacOS system (which is itself a version of Unix), it creates two files in a directory named .ssh, which is located in my home directory. Those two files are named id_rsa and id_rsa.pub. The first file (id_rsa) contains my private key, and the second file (id_rsa.pub) contains my public key.
It's important to note that if you just go with the defaults, as I'm about to show, and you already have a file named id_rsa, your system should warn you, like this:
$ ssh-keygen -t rsa Generating public/private rsa key pair. Enter file in which to save the key (/Users/al/.ssh/id_rsa): /Users/al/.ssh/id_rsa already exists. Overwrite (y/n)? _
If you’re very comfortable with this process and you want to overwrite your id_rsa file, do what you're comfortable with.
However, if this is the first time you’ve ever done this, and you get this warning message, for some reason this file has already been created, so be very careful about how you proceed. In fact, I recommend that you don't proceed until you understand more about what's happening here.
Note 1: You can also have the system write this output to a different file, but since I’m essentially writing this for people new to generating public/private key pairs, the point of this paragraph is that your current id_rsa is there for a reason, so be careful.
Note 2: If you want to write the output to a different file, see this note about How to use ssh when your private key file is not named id_rsa.
Given that warning, here’s how I just created a public and private key pair on my current MacBook. What I typed is shown in bold, and what the system generated as output is shown in a normal font:
$ ssh-keygen -t rsa Generating public/private rsa key pair. Enter file in which to save the key (/Users/al/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /Users/al/.ssh/id_rsa. Your public key has been saved in /Users/al/.ssh/id_rsa.pub. The key fingerprint is: 6f:16:29:90:46:b6:88:34:3d:81:07:fc:bd:1a:fc:db al@Al-Alexanders-MacBook.local The key's randomart image is: +--[ RSA 2048]----+ | .++..o | | .oo++ o | | .o.o= | | .... . | | . .S o | | o . o . | | + + | | . .. o | | ..E | +-----------------+
As you can see from the output of this command:
- Your private key is in a file named ~/.ssh/id_rsa
- Your public key is in a file named ~/.ssh/id_rsa.pub
Feel free to use vi or cat to look at both files if you like, but don't change them. As you’ll see, they are both plain text files.
You should never give your private key to anyone else, so for all intents and purposes, the id_rsa file will just stay right where it is.
As for your public key (id_rsa.pub), you're going to copy that to your remote servers in the following step.
Step 2: Copy your public key to your remote servers
The next step is to copy the id_rsa.pub file to the remote server you want to be able to access with
scp without using a password. For the purposes of this article, I'll refer to that remote system pluto.example.com. (Pluto seems pretty remote.)
scp that file to pluto as you normally would, supplying a password during the
$ scp id_rsa.pub firstname.lastname@example.org:./
If you're not familiar with the
scp command, it lets you securely copy files from one computer system to another, essentially working on top of the
ssh command. (The ./ at the end of that command puts the file in my home directory on pluto.)
ssh into pluto, again supplying your password when prompted:
$ ssh email@example.com
On pluto, if the .ssh directory does not already exist in your home directory, create it. Assuming you are in your home directory, just create it like any other directory:
$ mkdir .ssh
Now copy the id_rsa.pub file to a new file named authorized_keys in that .ssh directory, like this:
$ cp id_rsa.pub .ssh/authorized_keys
Assuming those steps worked without error, if you now
cd into your .ssh directory:
$ cd .ssh
and then use the
ls command, you’ll see your file in this directory with the proper name:
$ ls -al total 12 drwxr-xr-x 2 al al 4096 Jul 21 17:45 . drwx---r-x 9 al al 4096 Jul 10 10:46 .. -rw-r--r-- 1 al al 419 Jul 21 17:46 authorized_keys
Again assuming that all these steps worked, you’re now ready to test your remote login without using a password.
Step 3: Test your ssh login
To test that you can log in to your remote server using ssh without a password, either (a) logout of your remote server, or (b) open a new ssh window on your local computer.
In your ssh/terminal window, all you have to do to test the system is to attempt to login to your remote server as you did before:
$ ssh firstname.lastname@example.org
If everything is setup properly, you will be logged into your remote server without having to enter a password. If so, congratulations, your computer systems are now ready to use your public and private key pair to let you use ssh and scp without having to enter a password.
And of course what that means is that simple, automated scp backups are now within your reach.
Step 4: Install your backup scripts on the remote servers
Where you go from here depends on your backup process, but I'll describe my approach, and let you take it from there.
In short, on all of my remote servers I have the exact same setup, which consists of a 'backup' directory in my home directory of each server. I put two scripts in that directory, one to backup my MySQL data:
and a second script that I use to backup my web/HTML directory:
Both of these scripts have the execute bits set (file permission is "-rwxr-xr-x", or 755), and both scripts write their output to files in this same directory. The mysql backup file will be named pluto.mysql.gz, and the HTML/www backup file will be named pluto.html.tgz.
Step 5: Run your backup scripts with ssh
Given this setup, I can now use ssh to remotely run my backup scripts. I run the remote MySQL backup script like this:
ssh email@example.com "/home/al/backup/mysql-bu.sh"
and I run the HTML/www backup script (which is basically just a wrapper around the tar command) like this:
ssh firstname.lastname@example.org "/home/al/backup/html-bu.sh"
As mentioned, this creates two files in the current directory with names like these:
Now I'm ready for my scp command.
Step 6: Use scp to copy your backup files back home
With these backup files created, I can now scp them back to my local computer without using a password. Because of the naming convention used, I can just use one scp command to copy the files back to my local system:
scp email@example.com:/home/al/backup/*gz .
When this command finishes running, those two backup files have been copied to my local computer, and my work is done.
ssh, scp, backups, and public/private key pairs
I simplified a few things in this article so we could just focus on the essentials, but I hope you can see how this system lets us create a remote server backup system that we can automate. As you can imagine, we can now create a shell script with a for loop like this that can be used to automate the backup and download process for all our planetary systems:
# skipping a few planets ... for planet in mercury venus earth mars pluto do ssh firstname.lastname@example.org "/home/al/backup/mysql-bu.sh" ssh email@example.com "/home/al/backup/html-bu.sh" scp firstname.lastname@example.org:/home/al/backup/*gz . echo "$planet has been backed up" done
Of course you can automate the running of the backup scripts using cron/crontab on each system, but again my purpose for this article is to demonstrate the ssh/scp backup approach.
My backup scripts
Before I go, I'll also share my backup scripts. My MySQL backup script has actually been out here for quite some time, so I'll just link to it:
And as mentioned, my HTML directory backup script is basically a wrapper around the tar command. Here's a simplified version of what I actually use:
#!/bin/sh cd /home/al/html tar czvf /home/al/backup/pluto-html.tgz .
My actual backup script is more complicated than this, because my remote servers are with a variety of hosting companies (GoDaddy, A2 Hosting, etc.) with different directory structures, but in the end, that's all that HTML backup script does.
Summary: Using scp to make remote backups without a password
I hope this tutorial on how to use scp to make automated remote backups without using a password has been helpful. As you saw, what I demonstrated in this article is how to:
- Create a public and private key pair
- Install your public key on remote Unix and Linux servers
- ssh into your remote servers without a password
sshto run commands on your remote servers without using a password
scpto copy files to and from your remote servers without a password
If you'd like to use the cron/crontab command to further automate this approach, my website has many Linux crontab examples, and you can easily combine what's in those tutorials with the information in this tutorial.
For more information on public and private keys, see this Wikipedia article on public key cryptography. Also, kudos to this Linux journal article for first demonstrating much of this approach. If you happen to be a Java programmer, the Java programming language offers a similar approach, which I describe in my Java keytool and keystore tutorials.