linux-unix

recent posts related to linux and unix

Linux backups: Using find, xargs, and tar to create a huge archive

I did something wrong in a previous blog entry that led me to use the pax command to create a large backup/archive. There’s nothing wrong with using pax — other than the fact that it’s not available for Cygwin — and I really needed to created a huge archive.

What wasn’t working

In my earlier blog post I stated that something like this did not work for me when trying to create a large backup using find, xargs, and tar:

find . -type f -name "*.java" | xargs tar cvf myfile.tar

What was happening was that as xargs was managing the input to the tar command, tar kept re-writing the archive. That is, each time xargs passed a new block of input files to tar, tar perceived it as a new command, and went on to re-create the file named myfile.tar. So, instead of the huge myfile.tar that I expected, I ended up with only a few files in the archive.

A big collection of Unix/Linux 'find' command examples

Linux/Unix FAQ: Can you share some Linux find command examples?

Sure. The Unix/Linux find command is very powerful. It can search the entire filesystem to find files and directories according to the search criteria you specify. Besides using the find command to locate files, you can also execute other Linux commands (grep, mv, rm, etc.) on the files and directories you find, which makes find extremely powerful.

Sorting Unix 'ls' command output by filesize

I just noticed that some of the MySQL files on this website had grown very large, so I wanted to be able to list all of the files in the MySQL data directory and sort them by filesize, with the largest files shown at the end of the listing. This ls command did the trick, resulting in the output shown in the image:

ls -Slhr

The -S option is the key, telling the ls command to sort the file listing by size. The -h option tells ls to make the output human readable, and -r tells it to reverse the output, so in this case the largest files are shown at the end of the output.

Unix/Linux: Find all files that contain multiple strings/patterns

When using Unix or Linux, if you ever need to find all files that contain multiple strings/patterns, — such as finding all Scala files that contain 'try', 'catch', and 'finally' — this find/awk command seems to do the trick:

find . -type f -name *scala -exec awk 'BEGIN {RS=""; FS="\n"} /try/ && /catch/ && /finally/ {print FILENAME}' {} \;

As shown in the image below, all of the matching filenames are printed out. As Monk says, you’ll thank me later. :)

(I should mention that I got part of the solution from this gnu.org page.)

Linux: Recursive file searching with grep -r (like grep + find) alvin July 23, 2019 - 7:46pm

Linux grep FAQ: How can I perform a recursive search with the grep command in Linux?

Solution: find + grep

For years I always used variations of the following Linux find and grep commands to recursively search subdirectories for files that match a grep pattern:

find . -type f -exec grep -l 'alvin' {} \;

This command can be read as, “Search all files in all subdirectories of the current directory for the string ‘alvin’, and print the filenames that contain this pattern.” It’s an extremely powerful approach for recursively searching files in all subdirectories that match the pattern I specify.

How to use the Linux ‘scp’ command without a password to make remote backups

Summary: How to create a public and private key pair to use ssh and scp without using a password, which lets you automate a remote server backup process.

Over the last two years I've ended up creating a large collection of websites and web applications on a variety of Linux servers that are hosted with different companies like GoDaddy and A2 Hosting. I recently embarked on a mission to automate the backup processes for all these sites, and as a result of this effort, I thought I'd share what I've learned here.

How to control/configure vim colors

vim colors FAQ: Can you provide details on how to control/configure colors in the vim editor (vim color settings)?

When using vim syntax highlighting, a common complaint is that the default color scheme is a little too bold. In this article I'll try to demonstrate how you can change the colors in vim to be a little more pleasing, or at least be more in your control.

vi and vim `set` command examples

If you work with the vi editor a lot, you'll find yourself tinkering with various vi configuration settings from time to time. Many times you'll want to modify the configuration of your current vi session, and to do that you'll use one of many available vi set commands. In this vi (and vim) tutorial, I'll share the vi set commands I use most often.

Linux crontab examples (every X minutes or hours)

Table of Contents1 - Linux crontab: How to run a command every minute2 - Descriptions of the crontab date/time fields3 - Run a crontab command every hour4 - Run a crontab entry every day5 - Run a crontab entry every 5 minutes6 - Unix and Linux “crontab every” summary7 - Unix and Linux crontab reference information

Linux crontab FAQ: How do I schedule Unix or Linux crontab jobs to run at intervals, like “Every five minutes,” “Every ten minutes,” “Every half hour,” and so on?

Solution: I’ve posted other Unix/Linux crontab tutorials here before (How to edit your Linux crontab file, Example Linux crontab file format), but I’ve never included a tutorial that covers the “every” options, so here are some examples to demonstrate this crontab syntax.