unix

Hints for writing Unix tools

Marius Eriksen has a good article titled Hints for writing Unix tools. Some key points: a) consume input from stdin, produce output to stdout; b) output should be free from headers or other decoration; c) output should be simple to parse and compose. There’s much more to it than that, and it’s a good read (or reminder).

How to replace newline character with sed on Mac OS X (macOS)

I don’t have much time to explain this today, but ... if you want to see how to use the sed command on a Mac OS X (macOS) system to search for newline characters in the input pattern and replace them with something else in the replacement pattern, this example might point you in the right direction.

How to fire Mac OS X notifications with AppleScript (and Scala)

Summary: This tutorial demonstrates how to fire MacOS system notifications with AppleScript (and Scala or Java).

In this article I assume that you already know at least a little bit about how to use AppleScript, and just want to know how to trigger a MacOS notification. At the end of the tutorial I show how to invoke the AppleScript code using Scala and Java.

Unix: How to find files with multiple filename extensions

As I mentioned in my How to find multiple filenames with Linux find tutorial, you can use find command syntax like this to find files with multiple filename extensions:

find iTunes \( -name "*.mp3" -o -name "*.m4a" \)

As that command shows, I ran this find command to find all of my music files under my iTunes directory, including .mp3 and .m4a filename extensions.

While I’m in the neighborhood, this is the full find command I use to backup all of my iTunes files that have changed or been added in the last 180 days:

find iTunes \( -name "*.mp3" -o -name "*.m4a" \) -type f -mtime -180 -print0 | xargs -0 tar rvf NewMusic.tar

There’s probably an easier way to do this, but that backup command works for me.

Two more Textmate commands (capitalize, CSV to list)

As a “note to self,” I wrote two more Textmate commands yesterday, one to capitalize each word in a selection of words, and another to convert a CSV list of words to a simple list. Here’s the source code for the Capitalize command:

#!/bin/sh

perl -ne 'print ucfirst $_'

The $_ portion of that Perl command isn’t required, but I include it as a reminder to myself about how Textmate commands and snippets work.

Here’s the source code for my Textmate command that uses the Unix tr command to convert a CSV list of words (such as a paragraph of comma-separated words) into a simple list of words:

#!/bin/sh

tr , "\n"

As you can see, those commands are fairly simple. If you know Unix/Linux and then know a little about how to write Textmate commands, you can usually get it to do what you want. I like that you can use any Mac/Unix programming language or tool to solve the problem at hand.

Nginx configuration: How to drop the query string on a rewrite

As a quick note, if you need to drop the query string when configuring an Nginx rewrite request, this syntax works:

rewrite ^/foo/bar.*$  /bar?  permanent;

The key is to use the ? character at the end of the URL/URI you are redirecting users to. That drops the query string, so the user will be redirected to the exact /bar URI.

For more information, see the Nginx rewrite module page.

What is the Unix/Linux “bit bucket”?

Unix/Linux FAQ: What is the “bit bucket”?

The bit bucket is a way of referring /dev/null. Sending output to the /dev/null device file is like sending output directly to the trash. That’s why you see code like this a lot of times:

aCommand 2> /dev/null

That’s a way of saying, “Run the command aCommand and send it’s error output to the bit bucket.” In use like this, “error output” refers to STDERR, and redirecting STDERR to the bit bucket is the same as throwing it into the trash (or throwing it into a black hole, if you prefer).

Another way you can demonstrate this is by sending STDOUT to the bit bucket. In this next example, I send the output from the ls command to the bit bucket:

ls -l > /dev/null

If you run that command, you won’t see any output because the standard output — STDOUT — is redirected to /dev/null. There’s no practical reason for doing this in the real world; I just wanted to demonstrate that you can redirect both STDOUT and STDERR to the bit bucket, if you ever need to.

For more information, here’s a short tutorial on How to redirect Unix STDOUT and STDERR to the same location.

How to use the Linux sed command to delete a range of lines

In a previous blog post I demonstrated how to use sed to insert text before or after a line in many files, and in this example I'd like to demonstrate how to delete a range of lines using sed.

sed delete - How to delete a range of lines using sed

The problem I had today was that I just re-generated 99 HTML files for my Introduction to Unix/Linux tutorial using Latex2HTML, and it generates a bunch of "junk" in my HTML files that looks like this:

Sorting ‘ls’ command output by filesize

I just noticed that some of the MySQL files on this website had grown very large, so I wanted to be able to list all of the files in the MySQL data directory and sort them by filesize, with the largest files shown at the end of the listing. This ls command did the trick, resulting in the output shown in the image:

ls -Slhr

The -S option is the key, telling the ls command to sort the file listing by size. The -h option tells ls to make the output human readable, and -r tells it to reverse the output, so in this case the largest files are shown at the end of the output.

How to use the Linux ‘scp’ command without a password to make remote backups

Summary: How to create a public and private key pair to use ssh and scp without using a password, which lets you automate a remote server backup process.

Over the last two years I've ended up creating a large collection of websites and web applications on a variety of Linux servers that are hosted with different companies like GoDaddy and A2 Hosting. I recently embarked on a mission to automate the backup processes for all these sites, and as a result of this effort, I thought I'd share what I've learned here.