Posts in the “linux-unix” category

Teleport: The Unix/Linux ‘cd’ command, improved

[toc]

Summary: By keeping a history of the directories you've visited, the Teleport command is an improvement on the Unix/Linux cd command. By having a memory, Teleport lets you jump from one directory to any previously visited directory, easily.

January, 2015 Update: The Teleport command now supports Bash completion. For more details on this, see the Github INSTALL.md file.

Ubuntu ‘apt-get’ list of commands (list, update, upgrade, cheatsheet)

I have a couple of Ubuntu Linux systems, including Raspberry Pi systems, test servers, and production servers. It seems like every time I have to use an apt-get or other apt command, I always have to search for the command I need. To put an end to that, I’m creating this “apt-get reference page.” It’s very terse, as I’ve just written it for myself, but I hope it’s also helpful for others.

How to control/configure vim colors

[toc]

vim colors FAQ: Can you provide details on how to control/configure colors in the vim editor (i.e., vim color settings)?

Sure. When using vim syntax highlighting, a common complaint is that the default color scheme is a little too bold. In this article I'll try to demonstrate how you can change the colors in vim to be a little more pleasing, or at least be more in your control.

How to use the Linux ‘scp’ command without a password to make remote backups

Summary: How to create a public and private key pair to use ssh and scp without using a password, which lets you automate a remote server backup process.

Over the last two years I've ended up creating a large collection of websites and web applications on a variety of Linux servers that are hosted with different companies like GoDaddy and A2 Hosting. I recently embarked on a mission to automate the backup processes for all these sites, and as a result of this effort, I thought I'd share what I've learned here.

Linux: Recursive file searching with `grep -r` (like grep + find)

[toc]

Unix/Linux grep FAQ: How can I perform a recursive search with the grep command in Linux?

Two solutions are shown next, followed by some additional details which may be useful.

Solution 1: Combine 'find' and 'grep'

For years I always used variations of the following Linux find and grep commands to recursively search subdirectories for files that match a grep pattern:

find . -type f -exec grep -l 'alvin' {} \;

This command can be read as, “Search all files in all subdirectories of the current directory for the string ‘alvin’, and print the filenames that contain this pattern.” It’s an extremely powerful approach for recursively searching files in all subdirectories that match the pattern I specify.

Solution 2: 'grep -r'

However, I was just reminded that a much easier way to perform the same recursive search is with the -r flag of the grep command:

grep -rl alvin .

As you can see, this is a much shorter command, and it performs the same recursive search as the longer command, specifically:

  • The -r option says “do a recursive search”
  • The -l option (lowercase letter L) says “list only filenames”
  • As you’ll see below, you can also add -i for case-insensitive searches

If you haven’t used commands like these before, to demonstrate the results of this search, in a PHP project directory I’m working in right now, this command returns a list of files like this:

./index.tpl
./js/jquery-1.6.2.min.js
./webservice/ws_get_table_names.php

More: Search multiple subdirectories

Your recursive grep searches don’t have to be limited to just the current directory. This next example shows how to recursively search two unrelated directories for the case-insensitive string "alvin":

grep -ril alvin /home/cato /htdocs/zenf

In this example, the search is made case-insensitive by adding the -i argument to the grep command.

Using egrep recursively

You can also perform recursive searches with the egrep command, which lets you search for multiple patterns at one time. Since I tend to mark comments in my code with my initials ("aja") or my name ("alvin"), this recursive egrep command shows how to search for those two patterns, again in a case-insensitive manner:

egrep -ril 'aja|alvin' .

Note that in this case, quotes are required around my search pattern.

Summary: `grep -r` notes

A few notes about the grep -r command:

  • This particular use of the grep command doesn’t make much sense unless you use it with the -l (lowercase "L") argument as well. This flag tells grep to print the matching filenames.
  • Don’t forget to list one or more directories at the end of your grep command. If you forget to add any directories, grep will attempt to read from standard input (as usual).
  • As shown, you can use other normal grep flags as well, including -i to ignore case, -v to reverse the meaning of the search, etc.

Here’s the section of the Linux grep man page that discusses the -r flag:

-R, -r, --recursive
Read all files under each directory, recursively; this is
equivalent to the -d recurse option.

  --include=PATTERN
  Recurse in directories only searching file matching PATTERN.

  --exclude=PATTERN
  Recurse in directories skip file matching PATTERN.

As you’ve seen, the grep -r command makes it easy to recursively search directories for all files that match the search pattern you specify, and the syntax is much shorter than the equivalent find/grep command.

For more information on the find command, see my Linux find command examples, and for more information on the grep command, see my Linux grep command examples.

Linux crontab “every X minutes or hours” examples

[toc]

Linux crontab FAQ: How do I schedule Unix/Linux crontab jobs to run at time intervals, like “Every five minutes,” “Every ten minutes,” “Every half hour,” and so on?

Solution: I’ve posted other Unix/Linux crontab tutorials here before — How to edit your Linux crontab file, Example Linux crontab file format) — but I’ve never included a tutorial that covers the crontab “every” options, so here are some examples to demonstrate this crontab syntax.

Linux process memory usage: How to sort ‘ps’ command output

Linux ps sort FAQ: Can you share some examples of how to sort the ps command?

Sure. In this article we'll take a look at how to sort the Linux ps command output -- without using the Linux sort command.

Before we get started, the important thing to know is that the Linux ps command supports a --sort argument, and that argument takes a number of key values, and those keys indicate how you want to support the ps output.

Here's a quick look at the --sort information from the ps command man page:

Dozens of Unix/Linux 'find' command examples

[toc]

Linux/Unix FAQ: Can you share some Linux find command examples?

Sure. The Linux find command is very powerful. It can search the entire filesystem to find files and directories according to the search criteria you specify. Besides using the find command to locate files, you can also use it to execute other Linux commands (grep, mv, rm, etc.) on the files and directories that are found, which makes find even more powerful.

Dozens of Unix/Linux 'grep' command examples

[toc]

Linux grep FAQ: Can you share some Linux/Unix grep command examples?

Sure. The name grep means “general regular expression parser,” but you can think of the grep command as a search or find command for Unix and Linux systems: It’s used to search for text strings and regular expressions within one or more files.

I think it’s easiest to learn how to use the grep command by showing examples, so let’s dive right in.

.

.

.

.

.

(this section left blank for the long Table Of Contents over on the right)

.

.

.

.

.

Unix/Linux: Changing many files with the same multiline string pattern

Unix/Linux tip: How to change/edit the content on thousands of files using awk and a Unix shell script.

As a note to self, today I had to delete the same lines (or line patterns) from over 10,000 files on this Unix/Linux server. Basically I was looking in each file for a specific pattern, and if that pattern was found, a ceertain number of lines should be deleted, starting at that point.

How to use the Linux sed command to delete a range of lines

In a previous blog post I demonstrated how to use sed to insert text before or after a line in many files, and in this example I'd like to demonstrate how to delete a range of lines using sed.

sed delete - How to delete a range of lines using sed

The problem I had today was that I just re-generated 99 HTML files for my Introduction to Unix/Linux tutorial using Latex2HTML, and it generates a bunch of "junk" in my HTML files that looks like this:

MacOS: How to batch-resize images with the ImageMagick mogrify command

Mac batch image resizing FAQ: Is there a built-in macOS command I can use to batch resize images and photos on my macOS computer?

This article shows a “Mac batch image resize” approach you can use from the Mac Terminal command line, and in the link I share below I also show to how to batch resize images using a Mac GUI tool.

How to use ‘awk’ to print columns from a text file (in any order)

Unix/Linux FAQ: How to print columns of data from text files on Unix systems.

Background

One of my favorite ways to use the Unix awk command is to print columns of information from text files, including printing columns in a different order than they are in in the text file. Here are some examples of how awk works in this use case.

Linux/Unix: How to edit your crontab file with “crontab -e”

Linux crontab FAQ: How do I edit my Unix/Linux crontab file?

I was working with an experienced Linux sysadmin a few days ago, and when we needed to make a change to the root user crontab file, I was really surprised to watch him cd to the root user’s cron folder, make changes to the file, then do a kill -HUP on the crontab process.

Thinking he knew something I didn’t know, I asked him why he did all of that work instead of just entering this:

Wanting a Unix alias that takes a command-line argument (shell script, actually)

Today I wanted to create a Unix alias that took an argument (command-line argument), but from what I saw, that wasn’t going to be easy, so I created this little shell script to do what I want. It fails gracefully if you don’t supply a command-line argument, and runs the desired command if you do supply it:

#!/bin/sh

# NAME:    scw
# VERSION: 0.1
# PURPOSE: a script that works like a Unix alias
#          that requires a command-line argument

filename=""
if [ $1 ]
then
    filename="$1"
else
    echo "PURPOSE: Run 'scala-cli <filename> --watch'"
    echo "USAGE:   scw <filename>"
    exit 1
fi

scala-cli $filename --watch

I keep a bin directory in my home directory, and it’s in my PATH, so I just drop this shell script in there and then I can use it like a Unix alias.

You can also use it as an example for any similar script you want to run, i.e., a simple shell script that handles a command-line argument.

Linux shell script: while loop and sleep example

Linux shell script FAQ: Can you share a Linux shell script while loop example? While you're at it, can you show how to use the sleep command in the shell script while loop?

Sure. As a little background, I've written a program I call an Email Agent that periodically scans my email inbox, and does a lot of things to the inbox, including deleting the over 6,000 spams that I receive in a typical day. A recent problem with the Agent is that it runs too fast, apparently overwhelming the sendmail process on the machine that it runs on.

How to redirect Unix/Linux STDOUT and STDERR to the same file/location

Unix/Linux redirection FAQ: How do I redirect Unix STDOUT and STDERR to the same file with one command?

To redirect both STDOUT and STDERR to the same file with one Unix/Linux command, use this syntax:

my-shell-script.sh > /dev/null 2>&1

As you can see in this command, I'm redirecting STDOUT to /dev/null as normal, and then the special 2>&1 syntax tells your Bash shell to redirect STDERR to STDOUT (which is already pointing to /dev/null).