Linux wget command FAQ: Can you share an example of a wget command used in a Linux shell script?
Here's a Unix/Linux shell script that I created to download a specific URL on the internet every day using the
wget command. Note that I also use the
date command to create a dynamic filename, which I'll describe shortly.
Here's the source code for my Linux shell script which runs the desired wget command. This script is run from my Linux crontab file to download the file from the URL shown.
#!/bin/sh # devdaily.com # a shell script used to download a specific url. # this is executed from a crontab entry every day. DIR=/cygdrive/c/Al/Reports # wget output file FILE=dailyinfo.`date +"%Y%m%d"` # wget log file LOGFILE=wget.log # wget download url URL=http://foo.com/myurl.html cd $DIR wget $URL -O $FILE -o $LOGFILE
You don't need to write a shell script with all those variables, but in this case I did. (Also, that's not the real URL that I use.) I've found that my scripts tend to grow over time, and I might as well define variables from the beginning.
There are a couple of fun parts about this Linux wget script. The first thing is that I create the output filename dynamically using the Unix/Linux
date command. The portion of the code that looks like this:
actually puts the string "dailyinfo.20070201" into the
FILE variable. I like this because a) it gives me a unique filename every day, and b) it makes a large group of filenames easy to search and sort. The backticks surrounding the date command mean "run this command first, and put the output of the command right here". I use that syntax a lot in this situation, and also when I'm creating do/while loops in shell scripts.
Hopefully everything else is pretty straightforward. The weird thing about wget is that you have to specify the output file using an option, and can't just redirect the output. There may be an option that lets you write the output to STDOUT, but I didn't look for that, and just used the
-O option when I found it.