Posts in the “linux-unix” category

Linux crontab examples (every X minutes or hours)

Linux crontab FAQ: How do I schedule Unix or Linux crontab jobs to run at intervals, like “Every five minutes,” “Every ten minutes,” “Every half hour,” and so on?

Solution: I’ve posted other Unix/Linux crontab tutorials here before (How to edit your Linux crontab file, Example Linux crontab file format), but I’ve never included a tutorial that covers the “every” options, so here are some examples to demonstrate this crontab syntax.

An example Linux crontab file

Linux crontab format FAQ: Do you have an example of a Unix/Linux crontab file format?

I have a hard time remembering the crontab file format, so I thought I’d share an example crontab file here today. The following file is the root crontab file from a CentOS Linux server I use in a test environment.

How to edit your crontab file with “crontab -e”

Linux crontab FAQ: How do I edit my Unix/Linux crontab file?

I was working with an experienced Linux sysadmin a few days ago, and when we needed to make a change to the root user crontab file, I was really surprised to watch him cd to the root user’s cron folder, make changes to the file, then do a kill -HUP on the crontab process.

Thinking he knew something I didn’t know, I asked him why he did all of that work instead of just entering this:

Unix/Linux: Find all files that contain multiple strings/patterns

When using Unix or Linux, if you ever need to find all files that contain multiple strings/patterns, — such as finding all Scala files that contain 'try', 'catch', and 'finally' — this find/awk command seems to do the trick:

find . -type f -name *scala -exec awk 'BEGIN {RS=""; FS="\n"} /try/ && /catch/ && /finally/ {print FILENAME}' {} \;

As shown in the image below, all of the matching filenames are printed out. As Monk says, you’ll thank me later. :)

(I should mention that I got part of the solution from this gnu.org page.)

Update: My File Find utility

For a potentially better solution, see my File Find utility, which lets you search for multiple regex patterns in files.

A BIG collection of Unix/Linux 'find' command examples

Linux/Unix FAQ: Can you share some Linux find command examples?

Sure. The Unix/Linux find command is very powerful. It can search the entire filesystem to find files and directories according to the search criteria you specify. Besides using the find command to locate files, you can also execute other Linux commands (grep, mv, rm, etc.) on the files and directories you find, which makes find extremely powerful.

How to show hidden/invisible characters in vi/vim

I just learned that you can show hidden characters like tabs and end-of-line/newline characters in vi/vim with its set list command. Just go into last-line mode with the : character, then use that command:

:set list

When I do that in my current file, the $ is used to show newline characters, and TAB characters show up as ^I:

#!/bin/sh$
exec scala "$0" "$@"$
!#$
$
println("Hello, world")$
$
^I// tab test$
$
$

If you ever need to show hidden/invisible characters in vi/vim, I hope this is helpful.

An awk script to extract source code blocks from Markdown files

I just wrote this awk script to extract all of the Scala source code examples out of a Markdown file. It can easily be converted to extract all of the source code examples out of an Asciidoc file, which is something else I will do with it eventually.

Here’s the awk script:

BEGIN {
    # awk doesn’t have true/false variables, so
    # create our own, and initialize our variable.
    true = 1
    false = 0
    printLine = false
}

{
    # look for ```scala to start a block and ``` to stop a block.
    # `[:space:]*` that is used below means “zero or more spaces”.
    if ($0 ~ /^```scala/) {
        printLine = true
        print ""
    } else if ($0 ~ /^```[:space:]*$/) {
        # if printLine was true, we were in a ```scala block,
        # so print the end matter, then make printLine false
        # so printing will stop
        if (printLine == true) {
            print "```"
        }
        printLine = false
    }
 
    if (printLine) print $0
}

How to use the Linux ‘scp’ command without a password to make remote backups

Summary: How to create a public and private key pair to use ssh and scp without using a password, which lets you automate a remote server backup process.

Over the last two years I've ended up creating a large collection of websites and web applications on a variety of Linux servers that are hosted with different companies like GoDaddy and A2 Hosting. I recently embarked on a mission to automate the backup processes for all these sites, and as a result of this effort, I thought I'd share what I've learned here.

A BIG collection of Unix/Linux ‘grep’ command examples

[toc]

Linux grep FAQ: Can you share some Linux/Unix grep command examples?

Sure. The name grep means "general regular expression parser", but you can think of the grep command as a “search” command for Unix and Linux systems: It’s used to search for text strings and regular expressions within one or more files.

I think it’s easiest to learn how to use the grep command by showing examples, so let’s dive right in.

Unix/Linux shell script reference page

Linux shell script test syntax

All of the shell script tests that follow should be performed between the bracket characters [ and ], like this:

if [ true ]
then
  # do something here
fi

Very important: Make sure you leave spaces around the bracket characters.

I'll show more detailed tests as we go along.

Linux shell file-related tests

To perform tests on files use the following comparison operators:

[toc hidden:1]

MacOS: How to batch-resize images with the ImageMagick mogrify command

Mac batch image resizing FAQ: Is there a built-in Mac OS X command I can use to batch resize images and photos on my Mac OS X  computer?

This article shows a “Mac batch image resize” approach you can use from the Mac Terminal command line, and in the link I share below I also show to how to batch resize images using a Mac GUI tool.

How do I sort a Unix directory listing by file size?

To sort a Unix / Linux directory listing by file size, you just need to add one or more options to the base ls. On Mac OS X (which runs a form of Unix) this command works for me:

ls -alS

That lists the files in order, from largest to smallest. To reverse the listing so it shows smallest to largest, just add the 'r' option to that command:

ls -alSr

For another article related to finding large files, see my article, How to find the largest files under a directory on MacOS.

Teleport: The Unix/Linux ‘cd’ command, improved

[toc]

Summary: By keeping a history of the directories you've visited, the Teleport command is an improvement on the Unix/Linux cd command. By having a memory, Teleport lets you jump from one directory to any previously visited directory, easily.

January, 2015 Update: The Teleport command now supports Bash completion. For more details on this, see the Github INSTALL.md file.

Unix: How to find files with multiple filename extensions

As I mentioned in my How to find multiple filenames with Linux find tutorial, you can use find command syntax like this to find files with multiple filename extensions:

find iTunes \( -name "*.mp3" -o -name "*.m4a" \)

As that command shows, I ran this find command to find all of my music files under my iTunes directory, including .mp3 and .m4a filename extensions.

While I’m in the neighborhood, this is the full find command I use to backup all of my iTunes files that have changed or been added in the last 180 days:

find iTunes \( -name "*.mp3" -o -name "*.m4a" \) -type f -mtime -180 -print0 | xargs -0 tar rvf NewMusic.tar

There’s probably an easier way to do this, but that backup command works for me.

Unix/Linux: How to use sudo and curl to download and install a command

If you ever need a Linux/Unix shell command to download and install another command using curl and sudo privileges, I just used this command to install a command-line tool named Ammonite:

sudo sh -c '(echo "#!/usr/bin/env sh" && \
curl -L https://github.com/lihaoyi/Ammonite/releases/download/2.0.4/2.13-2.0.4) \
> /usr/local/bin/amm && chmod +x /usr/local/bin/amm' && amm

I show the command on multiple lines, but you can also make it a one-line command, as it is on the Ammonite page. As shown, this runs the sudo command, which requires your admin password, and then it runs the curl command inside a Unix/Linux shell environment, writing its output to /usr/local/bin/amm.

How to use one version number variable in many Linux shell scripts

As a brief note to self, I currently have a situation where I have about ten shell scripts, and each script uses the same jar file, but I keep changing the version number of that jar file each time I have a new release of my Static Drupal application. For example, my current jar file has this name:

Static-Drupal-assembly-1.2.jar

So to keep all of those shell scripts using the correct, current version of my jar file, I created a file named VERSION that has this version variable:

version=1.2

Then in my other shell script files, the first thing I do is “source” that file to get the version variable into those other shell scripts. One of the first lines in my ten shell scripts looks like this:

. ./VERSION

By doing this, I now only have to change the version number in that first file, and it’s automatically picked up in all of my other shell scripts.

A Linux shell script that shows find and tar with multiple image filenames

As a brief “note to self,” this is the Bourne shell script I use to copy images from my Drupal 8 sites directory to the same directory on my new “Static Drupal” website:

tarFile=newImagesFromDrupalSitesDir.tar
drupalHtmlDir=/drupal8/html
staticHtmlDir=/staticDrupal/html

cd $drupalHtmlDir
rm $tarFile 2> /dev/null

# create a tar file containing all new images
find sites -type f \( -name "*.jpg" -o -name "*.jpeg"  -o -name "*.png" -o -name "*.gif"  \) -mtime -2 -print0 | xargs -0 tar rvf $tarFile

# TODO make sure the tar file exists
if [ -e $tarFile ]
then
    echo "tar file exists, moving it to $staticHtmlDir"
    mv $tarFile $staticHtmlDir
    cd $staticHtmlDir
    tar xvf $tarFile
    rm $tarFile
else
    echo "POSSIBLE ERROR: the tar file DOES NOT exist"
fi

I changed the three initial variable names, but the rest of the script shows one possible way to copy all of the images in the original sites directory into the new Static Drupal directory. If you wanted to see things such as how to use multiple filenames with the Linux find command, or how to use the find command to create a tar file, I hope this example is helpful.

An Apache name based virtual host (NameVirtualHost) tutorial

Apache virtual hosting FAQ: How do I configure Apache to run multiple virtual hosts (name-based virtual hosts)?

An Apache name-based virtual host (NameVirtualHost) tutorial: As a quick reminder to myself for something I just did using MAMP on my Mac OS X system, here's how I configured the MAMP Apache server to use name based virtual hosting for two websites that I'm developing at the same time.

Unix/Linux ‘cut’ command examples

Linux cut command FAQ: Can you share some Linux cut command examples?

The Linux cut command is a really great command filter to know. You can use it to do all sorts of cool things when processing text files and text in command pipelines.

Using the cut command with /etc/passwd

For a first cut command example, I'll use the /etc/passwd file on my Unix system. I use this file because fields in the file are separated by the ":" character, which make it very easy to work with.