unix

Table of Contents

1. Common usage

1.1. Redirection

  • > is the redirection operator
  • & is an indicator that the following word is a fd, not a filename.
    • Eg. >1: redirecting to a file named 1, >&1: redirecting to fd:1

1.1.1. stdout and stderr

  • POSIX standard
    • redirect stdout
    • redirect stderr to stdout
    • command >/dev/null 2>&1
  • Bash shorthand (redirect both stdout and stderr)
    • command &> /dev/null

1.2. Heredoc

  • This is bash only, fish does not support heredocs(<<)
  • <<< is same but limited by one line
# redirected to file
cat <<POOP>somefile.txt
something
something else
POOP
# redirected to stdout
cat <<POOP
something
something else
POOP

1.3. Bash

  • bash runs all the commands in the script in the same process group

1.4. xargs

# file creation
cat list.txt #a b c d
xargs -t touch < list.txt # a b c d list.txt
echo "x y z" | xargs touch # a b c d list.txt x y z

# replace string
seq 10 | xargs -I X echo xXx
# x1x
# x2x
# x3x

seq 9 | xargs -L 3
# 1 2 3
# 4 5 6
# 7 8 9

2. one-liner dumps

# from local to server
rsync -avzhP -e "ssh -i path_to_key" /from/dir/ username@host:/to/dir/

# grep and replace
$ grep -rl assets | xargs sed -i 's/assets/\/img\//g'

# list duplicates using jq
cat myson.json | jq '[.data[].url]| group_by(.) | map(select(length>1))'

# remove a field from each object in an json array
cat data.json | jq 'map(del(.property_name))'

# Poor man's stress test by @bwplokta
# Replaces every newline with x then `head` for 4GB(4*10^9bytes)
# then feed it to `grep`, grep will slowly consume 4GB of memory.
yes | tr \\n x | head -c $BYTES | grep showmedamoney

3. More than a command

3.1. Flash drives

3.1.1. Formatting drive

dd if=/dev/zero of=/dev/sdX status=progress # clear out the disk
fdisk /dev/sdb
mkfs.vfat /dev/sdb1 # or mkfs -t vfat /dev/sdb1
  • Putting a FS on a partition == "Making a filesystem" (FS is what gets mounted)

3.1.3. Other notes

  • Sometimes usb devices may not show up in lsblk after kernel upgrade even if dmesg is alright and lsusb detects things. This is normal. Just reboot.

Created: 2024-07-16 Tue 16:44

Validate