User Tools

Site Tools


shell

Shell

Echo:

-n do not output the trailing newline

-e enable interpretation of backslash escapes (example:
prints \)


nicky $: ps fxo pid,cmd
PID CMD
 7997 emacs --daemon
 8117  \_ /usr/bin/python /usr/bin/ipython -c ?import sys, site?site.addsitedir('.')?import anaconda_mode?anaconda_mode.main(sys.argv[1:])? ---

to show up an obviousness between an empty variable and an unset variable:

$ set -u
echo $test
zsh: test: parameter not set

All variables will be automatically exported to child processes

$ set -a

Do not executed script commands but show them

$ set -n

Raise an exception if using of non set variable

#! /bin/sh
set -u (set -o nounset)
 
if [ -z "$string" ]; then
    echo "empty string"
fi

Follow the program

#! /bin/sh
set -x (set -o xtrace)

tee

plumbig tool. Put that tool into the pipeline to get an outing while sending an extra copy in any given files. Ideal to debug.

$ ps aux | tee /dev/tty | grep postgres > postgres_process.log

bc - An arbitrary precision calculator language

$ bc -l
25 * a(1)
quit
#! /bin/sh
total=$(echo "45.6 * 2.3" | bc -l)
echo $total

Internal variables

The $? parameter give us the return code of last tool executed:

$ mkdir /folder
$ $?

Then you can compare if it is equals to 0 …

Another solution would be to use {}

{ psql -h localhost -U user1 db -c "insert ...." 2 >> sql_trace.log; } || { echo >&2 "Can not insert to the database"; exit 2; }

Create a directory and go inside directly:

$ mkdir my_folder && $_

Process indentifier of current shell (main shell)

$ echo $$

Process indentifier of current father shell: PPID ( Parent PID)

$ echo $PPID

Process indentifier of the last command executed in background

$ echo $!

User identifier (used by the kernel to check every authorization needed)

$ echo $UID

List all arguments given to a script:

$ echo "$@"

IFS (Internal Field Separator) : Define the default operator used to cut words got on a line By default: space, tab and carrier return

Note: That variable is never imported to child shells https://www.admin-linux.fr/ifs-le-separateur-standard-du-shell/

extraction of sub string

$ variable="totofjweofjwefjw"
$ echo ${variable:5:3}
fjw
 
$ echo ${variable:1}
otofjweofjwefjw

**Operator # and ##**

${var#Pattern}
${var##Pattern}

${var#Pattern} Remove from $var the shortest part of $Pattern that matches the front end of $var.

${var##Pattern} Remove from $var the longest part of $Pattern that matches the front end of $var.

Examples for #:

$ querty="QWERTYUIOPASDFGHJKLMNBVCXZ"
echo ${querty#QWER}
TYUIOPASDFGHJKLMNBVCXZ
$ echo ${querty#*}
QWERTYUIOPASDFGHJKLMNBVCXZ
echo ${querty#*[URI]} # removing from beginning until the first letter found
TYUIOPASDFGHJKLMNBVCXZ
* -> any string (may be empty)

? -> any character

\* -> *

\\ -> \

\? -> ?

[a-z][A-Z][0-9]

^ or ! -> The matching excludes any character that belongs to the expression.

Parameter substitution (lentgh, substitution …):

http://www.tldp.org/LDP/abs/html/parameter-substitution.html#PSUB2

Get the file extension:

$ schema_file=schema.sql
$ echo "${filename##*.}"
sql

Get the length of variable:

$ schema_file="schema_blablalala"
$ echo "${#schema_file}"
17

Exceptions:

For an array, use

${#array[*]}
${#array[@]} 

give the number of elements in the array.

Input/Output with cat

$ cat << EOF
 
- Choose a function to start
 
1 - Create
2 - List
 
0 - Quit
 
EOF
 
while [ $"user_input" != 0 ]; do
 
   echo -n "Please choose: "
   read
   case "$user_input" in
       1) echo "kkkk";;
       2) ...
       0) ...
       *) echo "Not a valid choice"
    # [yY]*) echo "lala"; 
   esac
 
done

Comparison

Tool Description
-d is a directory
-f is a file
-p is a pipe
-S is a socket
-h or -L is a symbolink link
-n string not null
-z string null
-w can write to the file
-x executable
-s File size > 0
-t TTY
file1 -nt file2 last modification of file1 is more recent
file1 -ot file2 last modification of file1 is older

getopts

Signal

Signal Description
SIGUSR1 Signal use by personal application
SIGUSR2 Signal use by personal application
SIGHUP Reload configuration (if not handle by the application, can kill the application: see nohup)
SIGTERM KILL (nice)
SIGINT KILL
SIGQUIT KILL
SIGTERM KILL (nasty)

SIGTERM, SIGINT, SIGQUIT, SIGKILL ⇒ from nice to nasty

Use the command kill to send a signal
kill -TERM $PID
kill -HUP $PID

Note: PID -1 means all processus

Capturing signals
function cleanup {
  # clean resources
  rm -r tmp_file
}
 
trap cleanup EXIT SIGHUP SIGINT SIGTERM

Tools

Tool Description
flock Lock file
mkfifo Pipe with file https://linux.die.net/man/3/mkfifo
file file is a standard Unix program for recognizing the type of data contained in a computer file. (wikipedia)
fmt The fmt command in Unix is used to format natural language text for humans to read (wikipedia)
seq generate a seq [0-10]

https://en.wikipedia.org/wiki/List_of_Unix_commands

Various

Copy some folder somewhere else:
$ cp /home/nicky/{work,document} backups/
select-do
select variable in words
do
   case $answer in
       Word1 )
          ****
          *;;
       word2 )
          ;;
   esac
done
Source command
$ source local_file.sh

Note: Do not need to be executable and can update the current shell environment

Transmit arguments received from parent process to the child process
$0 "$@" & 

This is usefull when you want to parallelize a huge migration. See also $! and wait (wait without arguments waits until all child processes are done)

compound statements

When you use the () to execute a command, it will be executed in a child process and will not update any stuff to the father process.

Input/output process

Redirect stderr to stdout
$ grep lala myfile > file 2>&1

Note: 2>&1 means we copy the configuration from stdout to stderr (can use &>file but not common).

$ echo "lalal" >&2

Note: >&2 means we copy the configuration from stderr to stdout

Buffer
Buffer Description
stderr No buffer
stdout (TTY) line-buffered
stdout (FILE) raw-bloc-buffered

Buffer: Stderr does not have a buffer. So when you redirect stderr to stdout, the synchronization is not perfect.

Log rotate

Dailyfile

This small tool copies the content of its standard input into daily switching files.

https://github.com/cpb-/dailyfile

Regular expression (Grep, Sed, AWK)

Grep
$ grep -F "toto" file

or

$ cat file | grep -F "toto"
Option Description
-F Not a regular expression but a simple string
-E Extented Regular expression (by default simple)
-i case insensitive
-v Print lines that do not contain the regular expression

Symbols:

Meta-character Description
. Any character
* 0, 1 or many occurences of the previous element
? 0 or 1 of the previous element
+ 1 or many occurences of the previous element
vertical bar alternative
[a-z] range
circumflex accent Beginning of the string
$ End of the string
\ Print a special character
{n,m} At least n occurences to m of the previous element
{n,} At least n occurences of the previous element
{0, m} Max m occurences but can be 0, 1 .. of the previous element
{n} Exact occurences of the previous element
( ) capturing group
\n Back reference to n th element
[] character class
SED
Option Description
-s substitution
-d delete
-p print
First line
$ sed -ne '1p' < file
Last line
$ sed -ne '/^$/p' < file
$ sed -ne '/^$/!p' < file
$ sed -ne '/pattern_to_find/p' < file
Delete lines that match a pattern
$ sed -e '/pattern_to_find/d' < file

opposite:

$ sed -e '/pattern_to_find/!d' < file
Substitution
Meta-character Description
g replace every pattern found
p print the line that was updated
w send the result to a file
i substitute the pattern between n,m
$ echo "windows linux linux " | sed -e 's/windows/linux/g'
$ echo "windows linux linux " | sed -e 's/windows/linux/1'

Note: only replace the first patter found

&:

$ echo "tata" | sed -e "s/tata/<title>&<\/title>/g"
AWK
shell.txt · Last modified: 2018/12/05 16:59 by nicky