file

My Scala Sed project: More features, returning strings

Table of Contents1 - Basic use2 - Using a Map3 - Match expressions4 - Sed limitations5 - My Sed project6 - Bonus: Factories and HOFs

My Scala Sed project is still a work in progress, but I made some progress on a new version this week. My initial need this week was to have Sed return a String rather than printing directly to STDOUT. This change gave me more ability to post-process a file. After that I realized it would really be useful if the custom function I pass to Sed had two more pieces of information available to it:

  • The line number of the string Sed passed to it
  • A Map of key/value pairs the helper function could use while processing the file

Note: In this article “Sed” refers to my project, and “sed” refers to the Unix command-line utility.

Back to top

Basic use

In a “basic use” scenario, this is how I use the new version of Sed in a Scala shell script to change the “layout:” lines in 55 Markdown files whose names are in the files-to-process.txt file:

A Scala method to write a list of strings to a file alvin July 6, 2019 - 7:50pm

As a brief note today, here’s a Scala method that writes the strings in a list — more accurately, a Seq[String] — to a file:

def writeFile(filename: String, lines: Seq[String]): Unit = {
    val file = new File(filename)
    val bw = new BufferedWriter(new FileWriter(file))
    for (line <- lines) {
        bw.write(line)
    }
    bw.close()
}

Unix/Linux: Find all files that contain multiple strings/patterns

When using Unix or Linux, if you ever need to find all files that contain multiple strings/patterns, — such as finding all Scala files that contain 'try', 'catch', and 'finally' — this find/awk command seems to do the trick:

find . -type f -name *scala -exec awk 'BEGIN {RS=""; FS="\n"} /try/ && /catch/ && /finally/ {print FILENAME}' {} \;

As shown in the image, all of the matching filenames are printed out. As Monk says, you’ll thank me later. :)

(I should mention that I got part of the solution from this gnu.org page.)

Some Java file utilities

As a bit of warning, this is some old Java code, but if you want to create your own Java file utilities (utility methods), this code might help you get started:

A large collection of Unix/Linux ‘grep’ command examples

Linux grep commands FAQ: Can you share some Linux/Unix grep command examples?

Sure. The name grep means "general regular expression parser", but you can think of the grep command as a "search" command for Unix and Linux systems: it's used to search for text strings and more-complicated "regular expressions" within one or more files.

I think it's easiest to learn how to use the grep command by showing examples, so let's dive right in.

Scala code to find (and move or remove) duplicate files

My MacBook recently told me I was running out of disk space. I knew that the way I was backing up my iPhone was resulting in me having multiple copies of photos and videos, so I finally decided to fix that problem by getting rid of all of the duplicate copies of those files.

So I wrote a little Scala program to find all the duplicates and move them to another location, where I could check them before deleting them. The short story is that I started with over 28,000 photos and videos, and the code shown below helped me find nearly 5,000 duplicate photos and videos under my ~/Pictures directory that were taking up over 18GB of storage space. (Put another way, deleting those files saved me 18GB of storage.)

The Kotlin forEach println syntax

It’s a little hard to move back and forth between Scala and Kotlin because of some of the differences between the languages. Skipping the long story, here’s an example of how to print every line in a list of strings in Kotlin using forEach and println. First the setup:

import java.io.File
fun readFile(filename: String): List<String> = File(filename).readLines()
val lines = readFile("/etc/passwd")

Then here are two different ways to use forEach with println: