Scala code to find (and move or remove) duplicate files

My MacBook recently told me I was running out of disk space. I knew that the way I was backing up my iPhone was resulting in me having multiple copies of photos and videos, so I finally decided to fix that problem by getting rid of all of the duplicate copies of those files.

So I wrote a little Scala program to find all the duplicates and move them to another location, where I could check them before deleting them. The short story is that I started with over 28,000 photos and videos, and the code shown below helped me find nearly 5,000 duplicate photos and videos under my ~/Pictures directory that were taking up over 18GB of storage space. (Put another way, deleting those files saved me 18GB of storage.)

Linux: How to get the basename from the full filename

As a quick note today, if you’re ever writing a Linux shell script and need to get the filename from a complete (canonical) directory/file path, you can use the Linux basename command like this:

$ basename /foo/bar/baz/foo.txt

The Android “adb shell list files permission denied” error

As a brief note, today I tried to list the files in my Android application, which was running on a physical Android device — a Nexus 9 — with this adb shell command:

adb shell com.alvinalexander.mybrowser ls /data/data/com.alvinalexander.mybrowser

When I did that, I got an Android/ADB “permission denied” error.

The short story is that a solution to this problem is to run the same command, but with the run-as argument, like this:

A Linux shell script to rename files with a counter and copy them

As a brief note today, I was recently looking for all Messages/iMessage files that are stored on my Mac, and I used this shell script to copy all of those files — many of which have the same name — into a directory named tmpdir, giving them all new names during the copy process:

for i in `cat myfiles`
    fname=`basename $i`
    cp $i tmpdir/${count}-${fname}
    count=`expr $count + 1`

Unix: How to find files with multiple filename extensions

As I mentioned in my How to find multiple filenames with Linux find tutorial, you can use find command syntax like this to find files with multiple filename extensions:

find iTunes \( -name "*.mp3" -o -name "*.m4a" \)

As that command shows, I ran this find command to find all of my music files under my iTunes directory, including .mp3 and .m4a filename extensions.

While I’m in the neighborhood, this is the full find command I use to backup all of my iTunes files that have changed or been added in the last 180 days:

find iTunes \( -name "*.mp3" -o -name "*.m4a" \) -type f -mtime -180 -print0 | xargs -0 tar rvf NewMusic.tar

There’s probably an easier way to do this, but that backup command works for me.

What are the Drupal 8 Twig template front page file-naming conventions?

Drupal FAQ: What are the Twig template file naming conventions for Drupal 8? Specifically, how do I need to name a template file so it will be used for the front page of my website?

In short, the Drupal 8 Twig “HTML” template file has this name:


and if you want to create a custom HTML front page template, use this name:


The same goes for “page” and “node” files, where the default page template file has this name:

How to use the Linux sed command to edit many files in place (and make a backup copy)

Warning: The following sed commands are very powerful, so you can modify a lot of files successfully — or really screw things up — all in one command. :)

Yesterday I ran into a situation where I had to edit over 250,000 files, and with that I also thought, "I need to remember how to use the Unix/Linux sed command." I knew what editing commands I wanted to run -- a series of simple find/replace commands -- but my bigger problem was how to edit that many files in place.

Linux: How to find multiple filenames with the ‘find’ command

Linux find command FAQ: How can I write one find command to find multiple filenames (or filename patterns)? For example, I want to find all the files beneath the current directory that end with the file extensions ".class" and ".sh".

You can use the Linux find command to find multiple filename patterns at one time, but for most of us the syntax isn't very common. In short, the solution is to use the find command's "or" option, with a little shell escape magic. Let's take a look at several examples.

Handling spaces in Linux shell script input (and for loops)

Linux shell script FAQ: How can I deal with spaces (blank spaces) in my input data when I'm writing a shell script for loop or while loop?

I was just working on a Linux shell script, and ran into the ages-old problem of handling data that has spaces (space characters) in it. I run into this any time I try to read a data file with blank spaces in it, or when I run into files and directories with spaces in their names. Whenever I try to work this data like this in a shell script for loop, the spaces always ruin what I'm trying to accomplish.