wget command FAQ: Can you share an example of a
wget command used in a Linux shell script?
Here's a Unix/Linux shell script that I created to download a specific URL on the internet every day using the
wget command. Note that I also use the
date command to create a dynamic filename, which I'll describe shortly.
Linux “wget” script
Here's the source code for my Linux shell script which runs the desired
wget command. This script is run from my Linux crontab file to download the file from the URL shown.
#!/bin/sh # devdaily.com # a shell script used to download a specific url. # this is executed from a crontab entry every day. DIR=/cygdrive/c/Al/Reports # wget output file FILE=dailyinfo.`date +"%Y%m%d"` # wget log file LOGFILE=wget.log # wget download url URL=http://foo.com/myurl.html cd $DIR wget $URL -O $FILE -o $LOGFILE
You don't need to write a shell script with all those variables, but in this case I did. (Also, that's not the real URL that I use.) I've found that my scripts tend to grow over time, and I might as well define variables from the beginning.
wget script - discussion
There are a couple of fun parts about this
wget script. The first thing is that I create the output filename dynamically using the Unix/Linux
date command. The portion of the code that looks like this:
actually puts the string "dailyinfo.20070201" into the
FILE variable. I like this because a) it gives me a unique filename every day, and b) it makes a large group of filenames easy to search and sort. The backticks surrounding the date command mean "run this command first, and put the output of the command right here". I use that syntax a lot in this situation, and also when I'm creating do/while loops in shell scripts.
Hopefully everything else is pretty straightforward.
Was looking for a simple script to automate checking my various domains - and this was it. I've added another parameter to the wget (-i list-of-urls) and piped the log file to mail... and the job's done.