Handling spaces in Linux shell script input (and for loops)

Linux shell script FAQ: How can I deal with spaces (blank spaces) in my input data when I'm writing a shell script for loop or while loop?

I was just working on a Linux shell script, and ran into the ages-old problem of handling data that has spaces (space characters) in it. I run into this any time I try to read a data file with blank spaces in it, or when I run into files and directories with spaces in their names. Whenever I try to work this data like this in a shell script for loop, the spaces always ruin what I'm trying to accomplish.

Fortunately I learned how to handle spaces in shell script for loops a few years ago. Here’s how.

Handling spaces in shell script data and for loops

For most problems, all you have to do is change the input field separator (IFS) in your shell script, before running your for loop. In this way the shell won’t try to break your data apart by spaces, and will specifically only treat the newline character as the IFS.

For instance, I was just trying to read a pipe-delimited data file that looks like this:

foo|Foo|http://foo.com
bar|Bar|http://bar.com
foo-bar|Foo Bar|http://foobar.com

That space in the third line was blowing up my shell script algorithm, as the for loop kept treating “Foo” and “Bar” as two separate items, even though they were intended to be one field, delimited by the pipe characters.

To read this pipe-delimited data file properly, I wrote my for loop as normal, but added the IFS definition before the for loop, like this:

# set the input field separator
IFS=$'\n'

# the normal shell script for loop
for line in `cat projects.cfg`
do

  # get variables from project config file
  dirName="`echo $line | cut -f1 -d '|'`"
  projectName="`echo $line | cut -f2 -d '|'`"
  projectUrl="`echo $line | cut -f3 -d '|'`"

  # do the work down here ...

done

After changing the IFS, the projectName variable contained “Foo Bar” when it encountered that third line with the space character in it, which is what I wanted. (Until setting the IFS, the projectName contained only “Foo,” and then the script went haywire.)

I hope this tip on handling spaces with the input field separator in your Linux shell scripts and for loops has been helpful.