pipeline

How to use ‘awk’ to print columns from a text file (in any order)

One of my favorite ways to use the Unix awk command is to print columns of information from text files, including printing columns in a different order than they are in in the text file. Here are some examples of how awk works in this use case.

This is a page from my book, Functional Programming, Simplified

Functional Programming is Like Unix Pipelines

“Pipes facilitated function composition on the command line. You could take an input, perform some transformation on it, and then pipe the output into another program. This provided a very powerful way of quickly creating new functionality with simple composition of programs. People started thinking how to solve problems along these lines.”

Alfred Aho, one of the creators of the AWK programming language, in the book, Masterminds of Programming

Hints for writing Unix tools

Marius Eriksen has a good article titled Hints for writing Unix tools. Some key points: a) consume input from stdin, produce output to stdout; b) output should be free from headers or other decoration; c) output should be simple to parse and compose. There’s much more to it than that, and it’s a good read (or reminder).

How to build a pipeline of external commands in Scala

This is an excerpt from the Scala Cookbook (partially modified for the internet). This is Recipe 12.14, “How to build a pipeline of external commands in Scala.”

Problem

You want to execute a series of external commands, redirecting the output from one command to the input of another command, i.e., you want to pipe the commands together.

A guide rope for functional programming (Mary Rose Cook)

Mary Rose Cook has written a great, simple introduction to functional programming. The image shows the introduction to her article. I find it interesting to think of certain aspects of functional programming as writing pipelines, as in Unix/Linux pipeline commands. If your methods/functions have no side effects, you can write pipelines in your functional code.

Alaska - The Dalton Highway, the road to Deadhorse and Prudhoe Bay

(Click the image to see a much larger version.)

A photo showing a very small part of the Dalton Highway, which is the one and only road to Deadhorse and Prudhoe Bay. The Alaska Pipeline is shown in the left side of the photo. I took this photo while driving north in late August, 2007.

How to execute (exec) external system commands in Scala

Scala exec FAQ: How do I execute external system commands in Scala?

When it comes to executing external system commands, Scala is a dramatic improvement over Java. The operators Scala makes available are much more like Perl or Ruby, and the operators themselves are consistent with traditional shell commands, and are therefore easy to remember. Let's take a look at a few examples.

Linux ‘alias’ command examples

Unix and Linux aliases are a really nice way of customizing the command line to work the way you want it to work. With alias commands, you're essentially doing a little programming, and creating new Unix and Linux commands custom-tailored to the way you work. You can create aliases to use instead of existing commands, and you can also create aliases as Linux command pipelines.

An `egrep` example with multiple regular expressions

Summary: How to use the Linux egrep command with multiple regular expressions (regex patterns).

As a quick note here today, I just used the Linux egrep command to perform a case-insensitive search on multiple regular expressions (regex patterns). Really, what I did was a little more complicated:

locate -i calendar | grep Users | egrep -vi 'twiki|gif|shtml|drupal-7|java|PNG'

As you can see from that command, I did this: