Found commands

Data wrangling

`head`

Displays the first N lines of it’s input, by default 10.

`tail`

Displays the last N lines of it’s input, by default 10.

`sort`

Outputs its input in sorted order.

`uniq`

Outputs only unique adjacent lines from its input.

Typically used with sort: ... | sort | uniq

uniq -c gives a count for how many times each line is repeated.

A real example that came up for me today

$ for f in /usr/bin/*; do \
    if man -w "$(basename $f)" > /dev/null 2>&1; then \
      echo "man page"; \
    else \
      echo "no man page"; \
    fi; \
  done | sort | uniq -c
 764 man page
 151 no man page

`cut`

Outputs specific bytes, characters, or fields from lines of its input.

`join`

Join two files based on a common key.

$ cat foo.txt
a       10
c       30
b       20
$ cat bar.txt
c       Barney
a       Fred
b       Wilma
$ join <(sort foo.txt) <(sort bar.txt)
a 10 Fred
b 20 Wilma
c 30 Barney

`curl`

Downloads html or files from the internet via url.

$ curl https://gigamonkeys.com/ | head
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  2924  100  2924    0     0  11425      0 --:--:-- --:--:-- --:--:-- 11421
<!doctype html>
<html lang='en'>
  <head>
    <meta charset="UTF-8">
    <title>gigamonkeys</title>
    <link rel="stylesheet" type="text/css" href="index.css">
    <link rel="icon" type="image/png" sizes="16x16" href="/favicon-16x16.png">
  </head>
  <body>

System monitoring

`df`

Prints a list of the different file systems on your computer and how much space is available and how much is used

`du`

Show how big a file is.

du -sh summary in human readable form.

$ du -sh * | sort -hr | head -6
293M        node_modules
263M        eleventy
 93M        ec2
 91M        public
 18M        original-images
 17M        static

Shows the six biggest files or directories in the current directory.

`free`

Prints the current memory usage stats

$ free
         total     used    free  shared  buff/cache  available
Mem:   8119864  1721948  258692   64424     6139224    6011080
Swap:        0        0       0

$ free -h
       total   used   free  shared  buff/cache  available
Mem:   7.7Gi  1.7Gi  170Mi    62Mi       5.9Gi      5.7Gi
Swap:     0B     0B     0B

`ps`

Show running processes.

By default shows your own processes. With appropriate flags shows more.

$ ps | head
  PID TTY           TIME CMD
 1435 ttys000    0:00.04 -bash
 4166 ttys000    0:00.01 tmux
20243 ttys001    0:00.00 head
78011 ttys001    0:00.75 /opt/homebrew/bin/bash --noediting --login -i
75187 ttys002    0:00.62 -bash
20228 ttys003    0:00.41 /Users/peter/.nvm/versions/node/v22.12.0/bin/node index.js
75337 ttys003    0:00.02 -bash
75785 ttys003    0:00.01 make serve
75786 ttys003    0:00.26 npm exec nodemon --delay 2 --watch . -e js,mjs,json,njk,html,sql index.js

`kill`

Sends a signal to a running process.

kill -9 is the big hammer.

E.g. kill 75785 or kill -9 75785.

“If I kill random processes will I break something? :)”.

Yup.

`neofetch`

Prints a cool graphic with system and info and ACSII art the OS’s logo

Bash builtins

`$_`

A variable that stores the last arguments of the last command used.

E.g.

mkdir foo
cd $_

`$(command)`

Get the output of running a command as a string value.

$ x="$(ls -1 | wc -l)"; echo "$x files"
   21 files

`$((expression))`

Evaluates a mathematical expression in bash: $((1+1))

`while`

A looping construct.

Common form: while read -r p; do ...

“Why does read p change with every iteration?”

`history`

Prints a list of the commands used in the terminal.

Test expressions

These are expressions we can use in [] or [[]].

`-le`

Is first argument numerically less than or equal to the second:

if [[ $x -lt $y ]]; then
  echo "yup";
else
  echo "nope";
fi

`-z`

Is its argument an empty string:

if [[ -z "$somevar" ]]; then ...

`$-`

$- is a variable that stores a list of letters representing the currently active options in the shell.

This one is a bit obscure.

Power tools

`find`

Recursively searches for files by all kinds of criteria.

# find empty files
find . -empty

# Find all .java files, similar to ls **/*.java
find . -name '*.java'

# Find all .java files over 10,000 characters
find . -name '*.java' -a -size +10000c # finh ja

`sed`

Stream editor. Reads input and applies changes to certain lines.

Basically a mini programmig language.

`awk`

“Pattern-directed scanning and processing language”

Good for slicing and dicing columnar data.

Another mini programming language.

`perl`

If sed and awk had a very weird baby that was later bitten by a radioactive spider and developed super powers.

`vi`

One of the two old-school editors that continues to thrive in the modern world.

These days the vi on your system is almost certainly actually a program called vim.

Good to know because almost always there on any Unix system.

And a viable choice for your main editor if you want to go that way.

Uncategorized

`jot`

Generates sequences of numbers. See also seq.

`locate`

A tool for searching your files.

Requires a process running that builds an index of your files in the background.

Not used much any more in my experience.

`factor`

Calculates prime factors of a number of its argument or the numbers in its input.

`factor` examples

$ factor 1009 1010
1009: 1009
1010: 2 5 101

$ echo "1009 1010" | factor
1009: 1009
1010: 2 5 101

$ jot 100 1000 | factor | egrep '^([0-9]+): \1$' | cut -d : -f 1 | head -5
1009
1013
1019
1021
1031

`chown`

The chown command changes the owner of a file or directory. This can be really useful if you want to adjust the permissions of certain files.

`cmp`

Compares two files to the byte to find out if they are identical or not.

Compare to diff which actually shows you the difference.

`openssl`

“Make a random 12 character password. This could be useful for making passwords quickly.”

Well, there’s more to it than that. openssl is a whole suite of tools for dealing with encryption.

`compgen`

This is pretty obscure. I'm not even sure how you would really use it.

It's not daily tool.

`figlet`

 $ echo "foobar" | figlet
  __             _
 / _| ___   ___ | |__   __ _ _ __
| |_ / _ \ / _ \| '_ \ / _` | '__|
|  _| (_) | (_) | |_) | (_| | |
|_|  \___/ \___/|_.__/ \__,_|_|

Evil

`:() { :|: & };:`

“Returns you to the directory you were in when the bash thingy was started.”

Uh. No. Not that at all.

This is another good way to (temporarily) turn your computer into a doorstop.

It's called a “fork bomb”.

Found commands

Data wrangling

head

tail

sort

uniq

A real example that came up for me today

cut

join

curl