Linux shell script FAQ: How can I deal with spaces (blank spaces) in my input data when I'm writing a shell script for loop or while loop?
I was just working on a Linux shell script, and ran into the ages-old problem of handling data that has spaces (space characters) in it. I run into this any time I try to read a data file with blank spaces in it, or when I run into files and directories with spaces in their names. Whenever I try to work this data like this in a shell script for loop, the spaces always ruin what I'm trying to accomplish.
Fortunately I learned how to handle spaces in shell script for loops a few years ago. Here’s how.
Handling spaces in shell script data and for loops
For most problems, all you have to do is change the input field separator (IFS) in your shell script, before running your
for loop. In this way the shell won’t try to break your data apart by spaces, and will specifically only treat the newline character as the IFS.
For instance, I was just trying to read a pipe-delimited data file that looks like this:
foo|Foo|http://foo.com bar|Bar|http://bar.com foo-bar|Foo Bar|http://foobar.com
That space in the third line was blowing up my shell script algorithm, as the
for loop kept treating “Foo” and “Bar” as two separate items, even though they were intended to be one field, delimited by the pipe characters.
To read this pipe-delimited data file properly, I wrote my for loop as normal, but added the
IFS definition before the
for loop, like this:
# set the input field separator IFS=$'\n' # the normal shell script for loop for line in `cat projects.cfg` do # get variables from project config file dirName="`echo $line | cut -f1 -d '|'`" projectName="`echo $line | cut -f2 -d '|'`" projectUrl="`echo $line | cut -f3 -d '|'`" # do the work down here ... done
After changing the
projectName variable contained “Foo Bar” when it encountered that third line with the space character in it, which is what I wanted. (Until setting the
projectName contained only “Foo,” and then the script went haywire.)
I hope this tip on handling spaces with the input field separator in your Linux shell scripts and
for loops has been helpful.