Keep in mind: many things you want to do in grep you can do in awk. Or, maybe there is just a better way. I learned grep early on and have been forcing it to do things it wasn’t really designed to do, so here I will actually be listing solutions using other tools – tools that may be better designed for the task at hand.
awk:
awk '/textstring/ {print FNR}' textfile
sed:
sed -n '/pattern/=' filename
(taken from SO)
$ cat animals
dog
cat
dolphin
cat
$ sed "/cat/ { N; s/cat\n/giraffe\n&/ }" animals
dog
giraffe
cat
dolphin
cat
sed explained:
/cat/N)& represents the matched stringcat << EOF >> /tmp/myfile.txt
Contents that are added to the file at the end
You can indent files by spacing them appropriately
EOF
EOF (called the LimitString) is actually the very first item on the line. Do not indent, do not do anything other than make EOF begin at position #1. If you wish to indent your LimitString, you can use <<- to disable leading tabs.LimitString: cat << 'EOF' > /tmp/myfile.txtsudo: cat << 'EOF' | sudo tee /etc/http.d/my_ssl_configI was working on some cryptography homework and remembered that I wanted to plug my iPhone into my iMac to charge it up. I have a USB dongle that gives me room for 4 more ports. I have noticed that the dongle is rather pathetic and can’t handle much throughput or utilization. It doesn’t have enough for all of my peripherals so I try to use it for light-weight things like a wireless receiver for my Microsoft mouse and other stuff. Well, this time I had a USB drive plugged in in addition to my iPhone, wireless sensor, and something else. I opened Finder and checked my iPhone and noticed that it hadn’t been backed up for over two months. I started an encrypted backup of my phone. A few moments later, my external hard drive unmounted on its own. I think the dongle basically stopped communicating with it because of the communication requirements of the iPhone backup. I decided to move that USB drive to a port that is on my iMac directly. When I plugged it in, after roughly two or three minutes, I recieved the error that the disk could not be mounted.
I tried to mount it by hand from the command line and got “Resource busy”. Interesting. What’s using it? I’m not.
What disks are attached?
$ diskutil list
/dev/disk0 (internal, physical):
#: TYPE NAME SIZE IDENTIFIER
0: GUID_partition_scheme *3.0 TB disk0
1: EFI EFI 209.7 MB disk0s1
2: Apple_APFS Container disk2 3.0 TB disk0s2
/dev/disk1 (internal, physical):
#: TYPE NAME SIZE IDENTIFIER
0: GUID_partition_scheme *121.3 GB disk1
1: EFI EFI 209.7 MB disk1s1
2: Apple_APFS Container disk2 121.1 GB disk1s2
/dev/disk2 (synthesized):
#: TYPE NAME SIZE IDENTIFIER
0: APFS Container Scheme - +3.1 TB disk2
Physical Stores disk1s2, disk0s2
1: APFS Volume Macintosh HD - Data 728.4 GB disk2s1
2: APFS Volume Preboot 81.8 MB disk2s2
3: APFS Volume Recovery 526.6 MB disk2s3
4: APFS Volume VM 9.7 GB disk2s4
5: APFS Volume Macintosh HD 11.0 GB disk2s5
/dev/disk3 (external, physical):
#: TYPE NAME SIZE IDENTIFIER
0: GUID_partition_scheme *2.0 TB disk3
1: EFI EFI 209.7 MB disk3s1
2: Microsoft Basic Data PJADATA 2.0 TB disk3s2
Let’s do a generalized search for “disk” because a) I know /dev/disk3s2 isn’t mounted; and b) if something is using that device, it is using the device itself, not a disk as a mount:
$ sudo lsof | grep disk UserEvent 155 root txt REG 1,7 29360 1152921500312401745 /System/Library/UserEventPlugins/com.apple.diskarbitration.plugin/Contents/MacOS/com.apple.diskarbitration diskarbit 184 root cwd DIR 1,7 704 2 / diskarbit 184 root txt REG 1,7 161760 1152921500312399232 /usr/libexec/diskarbitrationd diskarbit 184 root txt REG 1,7 28056 12987595370 /Library/Preferences/Logging/.plist-cache.8jY7AXDN diskarbit 184 root txt REG 1,7 28504528 1152921500312401357 /usr/share/icu/icudt64l.dat diskarbit 184 root txt REG 1,7 1558736 1152921500312400397 /usr/lib/dyld diskarbit 184 root 0r CHR 3,2 0t0 319 /dev/null diskarbit 184 root 1u CHR 3,2 0t0 319 /dev/null diskarbit 184 root 2u CHR 3,2 0t846 319 /dev/null diskmanag 388 root cwd DIR 1,7 704 2 / diskmanag 388 root txt REG 1,7 2190864 1152921500312399722 /usr/libexec/diskmanagementd diskmanag 388 root txt REG 1,7 28056 12987595370 /Library/Preferences/Logging/.plist-cache.8jY7AXDN diskmanag 388 root txt REG 1,7 1558736 1152921500312400397 /usr/lib/dyld diskmanag 388 root 0r CHR 3,2 0t0 319 /dev/null diskmanag 388 root 1u CHR 3,2 0t0 319 /dev/null diskmanag 388 root 2u CHR 3,2 0t0 319 /dev/null UserEvent 511 pja txt REG 1,7 29360 1152921500312401745 /System/Library/UserEventPlugins/com.apple.diskarbitration.plugin/Contents/MacOS/com.apple.diskarbitration fsck_exfa 50042 root 3u CHR 1,13 0t0 2223 /dev/rdisk3s2
Ah! fsck is running! The disk filesystem is being checked. Now, let’s narrow our search just out of curiosity of what else fsck is doing:
$ sudo lsof | grep fsck fsck_exfa 50042 root cwd DIR 1,7 704 2 / fsck_exfa 50042 root txt REG 1,7 90400 1152921500312404229 /System/Library/Filesystems/exfat.fs/Contents/Resources/fsck_exfat fsck_exfa 50042 root txt REG 1,7 1558736 1152921500312400397 /usr/lib/dyld fsck_exfa 50042 root 0u CHR 3,2 0t180 319 /dev/null fsck_exfa 50042 root 1u CHR 3,2 0t180 319 /dev/null fsck_exfa 50042 root 2u CHR 3,2 0t180 319 /dev/null fsck_exfa 50042 root 3u CHR 1,13 0t0 2223 /dev/rdisk3s2
When it was finished, my disk mounted all by itself.
$ mount |grep disk3 /dev/disk3s2 on /Volumes/PJADATA (exfat, local, nodev, nosuid, noowners)
basename "$FILE" filename="$(basename $FILE)" path="$(dirname $FILE)"
Bash doesn’t have built-in true and false boolean values that can be used to test for validity. It’s often just as clear to use 0 or 1 for false and true, respectively.
“For interactive use, like one-liners, make sure to leave a space after !, or it will do history expansion. ((! foo)) works, so does ! ((foo)). ((foo || bar)) works as expected”[1]. Also, remember that bash considers an exit code of 0 success while an exit code of 1 is failure. If you are working with true being equal to 1 and false being equal to 0 your may have to adjust some bash-y expectations.
#!/bin/bash flag=0 if (( flag )); then echo "Condition is true"; fi false=0 true=1 ((false)) && echo false ((true)) && echo true ((!false)) && echo not false ((!true)) && echo not true
(Source: stackoverflow.com)
if...then...elif...else...fi
if TEST-COMMAND1; then STATEMENTS1 elif TEST-COMMAND2; then STATEMENTS2 elif [ $test_var -ge 3 -a $test_var -lt 11 ]; then STATEMENTS3 else STATEMENTS4 fi
Find files that have been changed in the last 24 hours.
find /var/lib/ -mtime -1 -ls
-1 means anything changed one day or less ago. +1 means anything that has changed at least one day ago. Having only 1 means exactly one day ago.
(Source: stackoverflow.com)
Loop through a list of files.
while read patch; do
patch -p1 -f < ${patch} &>> $PATCH_LOG
done < <(ls -l ${PATCH_SUBDIR}/*.patch)
For even more reference, see the holy grail of bash PEs.
Replace a character in a string with another character.
$ VER=4.19.87
$ NEWVER="${VER//./_}"
$ echo "$NEWVER"
4_19_87
When you experience odd behavior with finding or not finding an installed or uninstalled program, you may need to refresh your remembered locations.
hash -r
See the following references: man hash | pip3 is looking for a wrong path | what does hash -r command do? | what is the purpose of the hash command
printf an exclamation mark.Option 1: use single-quoted strings.
printf 'Unable to find config file. Exiting!'
Option 2: add to the end of the string, outside of the quotes
printf "Unable to find config file. Exiting" !
Option 3: print ASCII representation of ! character
printf "\041"
Option 4: Mix double quotes with single quotes (extension to Option 1)
printf "Unable to find config file. Exiting"'!'" (that prints an exclamation mark just fine.\n"
Option 5: use a format string
printf "This (%s) is an exclamation mark%s\n" ! !
printf Two Variablesprintf "%s %s" % "${MYSTRING1}" "${MYSTRING2}"
for i in $(seq 1 $END); do printf "$i\n"; done
Unless specifically required to not be, the second line of all Bash scripts ought to be: set -euf -o pipefail
set -x ls $mydir set +x
NOTE: In Jenkins, set -x is the default. This echoes all commands. Override by putting the following at the top of the “Execute Shell” build step:
#!/bin/bash +x
(I do not use “downcase” – it’s “lower case“ – that word needs to be exonerated from its duty of describing a letter case.)
Entire Word
I’m not even going to try to out-do this answer.
Though I will drop it here for quick reference:
$ myvar="True" $ echo "$myvar" | tr '[:upper:]' '[:lower:]' true
First Letter
foo="$(tr '[:lower:]' '[:upper:]' <<< ${foo:0:1})${foo:1}"
Again, great post on StackOverflow. Dropping it here for quick reference.
#!/bin/bash
firstString="I love Suzi and Marry"
secondString="Sara"
echo "${firstString/Suzi/$secondString}"
# prints 'I love Sara and Marry'
$ shfmt -w -i 4 -sr <file(s) or path...>
man page
TLDP BASH Programming How-To
TLDP Advanced Bash-Scripting Guide
Google’s Shell Style Guide
-f Check if a file exists
FILE=/tmp/myfile.txt if [[ -f "$FILE" ]]; then echo "$FILE exists" fi
-z Check if a variable is empty
if [ -z "$var" ]; then printf '$var is empty\n'; else printf '$var is not empty\n'; fi
NOTE: Do not use this if you do not want others to see your script or bash history, which includes your user’s sudo password.
Use case: SSH to a Clonezilla Server that is already running and already has a specific user created on it. This user can SSH but the files on the Clonezilla live system are read-only so you can not edit /etc/sudoers file to specify NOPASSWD for your user.
$ echo <password> | sudo -S <command>
Trickster: to avoid exposing the password on the command history, start the command with a SPACE character.
if [ $(id -u) -eq 0 ]; then printf "You are root\n" else printf "You are not root\n" fi
For easier cleanup, use trap
MyVar="${DEPLOY_ENV:-default_value}"