A Linux wget command shell script

Linux wget command FAQ: Can you share an example of a wget command used in a Linux shell script?

Here's a Unix/Linux shell script that I created to download a specific URL on the internet every day using the wget command. Note that I also use the date command to create a dynamic filename, which I'll describe shortly.

Linux “wget” script

Here's the source code for my Linux shell script which runs the desired wget command. This script is run from my Linux crontab file to download the file from the URL shown.

#!/bin/sh

# alvinalexander.com
# a shell script used to download a specific url.
# this is executed from a crontab entry every day.

DIR=/cygdrive/c/Al/Reports

# wget output file
FILE=dailyinfo.`date +"%Y%m%d"`

# wget log file
LOGFILE=wget.log

# wget download url
URL=http://foo.com/myurl.html

cd $DIR
wget $URL -O $FILE -o $LOGFILE

You don't need to write a shell script with all those variables, but in this case I did. (Also, that's not the real URL that I use.) I've found that my scripts tend to grow over time, and I might as well define variables from the beginning.

Discussion

There are a couple of fun parts about this wget script. The first thing is that I create the output filename dynamically using the Unix/Linux date command. The portion of the code that looks like this:

FILE=dailyinfo.`date +"%Y%m%d"`

actually puts the string "dailyinfo.20070201" into the FILE variable. I like this because a) it gives me a unique filename every day, and b) it makes a large group of filenames easy to search and sort. The backticks surrounding the date command mean "run this command first, and put the output of the command right here". I use that syntax a lot in this situation, and also when I'm creating do/while loops in shell scripts.

Hopefully everything else is pretty straightforward.