The Bourne Again Shell

Table of Contents

last revision
16 August 2015, 12:36am
book quality
average
:

quote: "bash scripts are dyslexic poetry" (anon)

The Bourne Again Shell, or 'Bash' shell is one of the ways that a user may interact with the Linux Operating System. The Bash shell is extremely flexible and powerful since it has a full range of programming constructs, such as loops and tests as well as the ability to join commands in command chains using the pipe "|" symbol.

The name 'Bourne Again' comes from the fact that the 'bash' shell was a rewrite of an earlier unix shell called the 'Bourne Shell'.

Getting Help ‹↑›

view help for the built-in bash commands

 help

view help for the 'pwd' print working directory command

 help pwd

show all software about optical character recognition or speech

 apropos speech ocr | less

[= image/eg-apropos-ocr.png]

Optical character recognition means software which can 'recognise' written characters (letters) which form part of an image and can turn those characters into text.

display a short help message for all programs in the system

 whatis -r '.*' | less

show some installed programs having something to do with 'java'

 whatis -r java | less
 apropos -r java | less

Apropos is better than 'whatis' because it actually searches the man pages for mentions of the given term, where as 'whatis' just looks at the short description of the program

show the version number of the bash shell which you are using

 echo $BASH_VERSION

Hold down the escape key for a few seconds to see all commands

 ESC

Web Links ‹↑›

www: http://tldp.org/LDP/abs/html/
a tutorial
www: www.commandlinefu.com
lots of recipes
www: www.shelldorado.com
more bash tips
www: http://www.bash-hackers.org/wiki/doku.php
stuff about the bash shell
www: http://bashcurescancer.com/10-linux-commands-youve-never-used.html
about little used but useful commands
www: www.stackoverflow.com
This is a forum about any programming language, but it generally contains intelligent answers to bash questions.
www: http://www.faqs.org/faqs/unix-faq/shell/bash/
a bash faq

Traps And Gotchas ‹↑›

This section details some unpleasant surprises awaiting the unwary bash novice.

be careful about white space in bash statements

 d=2 NOT d = 2
 if [ $a -le 5 ] NOT if [ $a -le 5]

there must be a space after a { bracket

 for j in 1 2 3 4; {echo "j=$j"; }   NO!! doesnt work

No!!! you need spaces after "$a" and before "$b", for some reason

 a='ss'; [ "$a"=="$b" ] && echo yes

scripts with MS Windows newlines \r\n will not run

 #!/bin/bash\r\n   doesnt work, use the 'dos2unix' tool

the so called "back-ticks" (`) need to be escaped in shell scripts

 \`\`    ##(the ` character is similar to the '$(...\)' construct)

echo doesnt show newlines. It removes the newlines from string variables

 s="first "$'\n'"2nd "$'\n'"3rd"; echo $s

This prints "first 2nd 3rd" all on one line, BUT the variable "s" actually spans 3 lines. use 'echo "$s"' to actually show the real contents of the variable 's' with newline characters

cant use bash -v to get the version, use the following instead

 echo $BASH_VERSION

brace expansion cant have spaces

you cant background a job on the same line with another command

 ls &; clear   No!! this doesnt work
 ls &;         No!! also doesnt work

just omit the ';' semi-colon, but how does this work?

 ls & clear

Jargon ‹↑›

This section attempts to explain some of the technical terms which are often used when describing the bash shell. Many of these terms have a long (more than 30 years) tradition of use in Unix-type operating systems.

www: terminal
An rs-232 serial device consisting of a keyboard and screen which communicates with a computer, or else software which emulates this.
www: console
www: shell
This is a way of interacting with the operating system by typing a series of very cryptic and precise commands. Most users have an instinctive dislike and dread of this type of activity.
www: built-in
command a command which is part of bash (does not need any external program to execute). You can view some help for these commands by typing 'help command' in a console.
www: variable
A variable is an entity which may contain a value such as '2' or "The tree". The variable can change its value as often as is required (this is why its called a 'variable')
www: untyped
and typed variables variables can contain various values and those values can be classified into 'types' such as text (also called 'strings') whole numbers (also called 'integers') or decimal numbers (also called 'floating point numbers') Bash normally uses 'untyped' variables because it does not really care what type of value the variable holds (whether the variable is text, an number or something else). The bash shell attempts to 'guess' what type of value a variable holds, and then tries to do something 'sensible' with that value.
www: variable
interpolation This term refers to the process of inserting or substituting string (text) variables which are contained within other text variables. So if we have s=4; t="1 2 3 $s" then the value of the variable 't' will be '1 2 3 4' because the variable 's' was 'interpolated' within the variable 't'. This is a very handy feature of many modern scripting languages such as Perl, Ruby, Python etc
www: string
In the world of programming a string refers to a chunk of text. This is because 'strings' were first thought of as a string of characters.
www: BSD
Berkeley Standard Distribution of Unix. This is, more or less the version of Unix used with MacOSX and many of the standard programs are slightly different to their Gnu-Linux equivalents. For this reason, when writing bash scripts that need to run on both Mac OSX and Linux, the programmer must be sure to only use posix switches
www: FreeBSD
A free version of the BSD Unix operating system. This is not a popular as Linux, but has the reputation for being more secure and better for older machines. FreeBSD may be capable of running on old 486 machines
www: Posix
After the development of BSD Unix, there began to appear different versions of Unix which had small but significant differences. The Posix standard was an attempt to 'reunify' different Unix versions by specifying standards that all Unix type operating systems (including Linux) should implement. By conforming to posix standards, one is able to write bash shell scripts which should run correctly on a wide variety of different Unix-type systems, such as Mac OSX Linux and FreeBSD
www: globbing
The bash shell expands certain special characters (such as '*') into a list of file names in the current folder.
www: brace
expansion expands an expression containing braces '{}' into a series of numbers or files names.
www: script
A script is a set of instructions contained within a text file in a format which is readable and understandable by a person (after some training) which is also readable and executable by a computer program in order to carry out some particular task.
www: backtics
The 'backtic' (`) along with the more modern $() construct is used to insert the output of a command within another command or piece of text. It is an extemely powerful technique. The more modern $() may be nested but the older `` may not
www: option,
switch These 2 terms which are equivalent are used to alter the behavior of the application or script. Options or switches are usually written with a hyphen before a letter (for example '-a' or '-b') or with 2 hyphen before a word or words (eg: '--list' or '--version')
www: short
option The short option to a command is one hyphen followed by one letter (eg: '-a' or '-b')
www: long
option an option to a command written with words rather than a single letter (eg --debug)
www: parameter
Parameters are values which are passed to scripts and functions in order to provide information to the script or function. For example if a script is executed with './test.sh green white' then the first parameter has the value 'green' and the 2nd parameter has the value 'white' The parameters can be used within the script by accessing the variables $1, $2, ... etc.
www: bash
function a function is a way of coding a task that is opten repeated. A function is slightly more flexible than an alias, and slightly less flexible than a script.
www: alias
This is the simplest way to create new commands for tasks which you often carry out using the bash shell. Aliases cannot have parameters.
www: shebang
line The 'shebang' line is the line at the top of a unix script which begins with '#!' and then a path. This line tells the unix or linux operating system what program to use to execute the script
www: executable,
application, program All these terms are more or less equivalent and mean a file or files containing instructions which a computer can carry out in order to perform a particular task (such as surf the internet or write a document)
www: binary
file
www: text
file A text file represents a series of characters in some human writing system. Each character in the writing system (such as the chinese ideograph for 'rain' or the roman alphabet letter 'z' or the greek 'alpha' character) is given a unique number and is then encoded into a binary code which may be stored on a computer system.
www: character
www: text
encoding This is the way that each character in a text file is transformed into a unique number (in binary code). Ascii was an old text encoding which was used for encoding the roman alphabet. Utf8 is a more modern encoding which is capable of encoding any character in any known human writing system.
www: Unicode
Unicode is a very useful standard developed over a number of years by a committee to provide a way to categorise and enumerate all the characters in all the writing systems which are currently known in the world. This standard makes the job of writing text documents which can be used and read over the Internet in different countries, much easier.
www: locale
Different human cultures use different writing systems to write documents, display numbers in different ways and use different languages. For example, in Colombia the number 2.200,33 means '2 thousand, 2 hundred and 1/3 (approximately) but in an Anglo-Saxon country, the number would be written 2,200.33. Linux attempts to cater for these different cultures using 'locales'.
www: interface
The interface is the way that a human being interacts with a computer program. This is usually by typing text into small boxes, clicking 'buttons', choosing elements from lists and so on. An interface could also be based apon voice recognition and sound. In old unix programs the interface consists purely of text messages.
www: curses
curses is a type of user interface which runs within a terminal. It allows the user to use the arrow keys to move around the screen. It is essentially a more 'primitive' version of graphical windows. @@

Other Shells ‹↑›

While bash is considered the most popular and widely used shell for linux and other unixes, other shells exist.

find out the name of the running shell or script

 echo $0

Processes And Jobs ‹↑›

Any unix-style operating system (and most modern operating systems) can do more than one thing at once. Each of these tasks is called a 'job' or a 'process'. There is a subtle difference but we wont worry about it right now.

run the command apropos shell > shell-commands as a background job

 apropos shell > shell-commands &

use export to make a variable available to subprocesses

 export name=value

pause a currently executing job

 [control] z

show what jobs the shell is executing

 jobs

stop the second job in the job list

 kill %2  a job number is different from a process number

stop the process with id '1423'

 kill -9 1423

bring a job which is executing in the background to the foreground

 fg
if there is more than 1 background job, add a job-number

put into the background a job running in the foreground

 bg

show all executing processes using 'unix' options

 ps -e

show the process id and initiating command of all processes

 ps -eo pid,comm

run a command which will not be interrupted when the user logs out.

 nohup command

run a command with a priority less than normal

 nice command

display a list box of running processes

 echo "$(ps -eo pid,comm | zenity --list --column= --height 800)"

display a list- box of running processes and kill the process chosen

   p="$(ps -eo pid,comm | zenity --list --column= --height=800)"; kill -9 $(echo $p | sed 's/ .*$//');

Text Files ‹↑›

the right way to read a text file

   cat test.txt | while read l; do
     echo "$l"
   done

create a new file 'list' and type text into it

 cat > list -   press [control] + c to stop this

read 3 characters from an input stream

 echo abcdef > junk.txt; read -n 3 a <junk.txt; echo $a;   prints "abc"

loop through the words of a text file

 for i in $(cat example.txt); do echo $i; done
 for i in $(<example.txt); do echo $i; done  the same

loop through the lines of a text file

 while read -r a ;do echo $a; done < file.txt

loop through the characters of a text file

 while read -n 1 a ;do echo $a; done < file.txt

insert "yes" at the 4th character in a text file

 exec 3<> file.txt ; read -n 4 <&3 ; echo -n yes >&3 ; exec 3>&- ;

a very simple text translation script

    exec 3< file.txt
    while read a <&3
    do 
      echo -ne "--\n$a\n>>"; read b; echo $b >> file.tr.txt
    done; exec 3<&- 

a very simple text translation script with line numbers

    exec 3< test.txt
    n=1
    while read a <&3
    do 
      echo -ne "$n--\n$a\n>>"; read b; echo $b >> eg.tr.txt
      ((n++))
    done; exec 3<&- 

create a small amount of random text for filling up documents

 cat /usr/share/dict/words | shuf | head -100 | tr  '\n' ' '
 shuf -n 100 /usr/share/dict/words | tr  '\n' ' '  same, better

substitute the word 'blah' in 'f.txt' with lots of random text

 sed "s/blah/$(shuf -n 100 /usr/share/dict/words | tr  '\n' ' ')/" f.txt

the same, but really do the substitution in the file & make a backup

 sed -i.bak "s/blah/$(shuf -n 100 /usr/share/dict/words | tr  '\n' ' ')/" f.txt

append contents of the file 'file.txt' to itself

 cat file.txt | tee >> file.txt

Merging Text Files ‹↑›

print lines between 2 markers

 awk '/abc/{flag=1;next}/mno/{flag=0}flag' file

replace text between 2 markers sed or awk ----- SED: View Raw Code

sed -i -e "/^##START\$/,/^##STOP\$/c ##START\n${REPLACEMENT}\n##STOP" test

AWK: View Raw Code

,,,

awk "/##START/{p=1;print\"###START\n${REPLACEMENT}\n###STOP\";next}/##STOP/{p=0;next}!p" test.tmp > test

To insert before the ###marker### line :

// for each <line> of second_file.txt : // if <line> matches regexp ###marker###, outputs first_file.txt. // **without any condition :** print <line> awk '/###marker###/ { system ( "cat first_file.txt" ) } \ { print; } \' second_file.txt

To insert after the ###marker###line :

// for each <line> of second_file.txt : // **without any condition :** print <line> // if <line> matches regexp ###marker###, outputs first_file.txt. awk '{ print; } \ /###marker###/ { system ( "cat first_file.txt" ) } \' second_file.txt

To replace the ###marker### line :

// for each <line> of second_file.txt : // if <line> matches regexp ###marker###, outputs first_file.txt. // **else**, print <line> awk '/###marker###/ { system ( "cat first_file.txt" ) } \ !/###marker###/ { print; }' second_file.txt

If you want to do in-place replacement, use a temp file for being sure the pipe doesn't start before awk has read the entire file; add :

> second_file.txt.new mv second_file.txt{.new,} // (like "mv second_file.txt.new second_file.txt", but shorter to type !)

Editing Text Files ‹↑›

The $EDITOR variable determines the default text editor.

Select and Edit a File in the Current Directory

 PS3="Enter a number: "; select f in *.txt;do $EDITOR $f; break; done

But the command above breaks if any of the file names contain spaces You may be able to use the IFS variable to solve this.

choose a file to edit, even if the name contains a space

 IFS=$'\n'; PS3="Choose:"; select f in $(ls);do vim $f; break; done

Searching Text ‹↑›

search for comment lines in a file

 grep '^#' <file

search for some text in all files and subdirectories

 grep -ril text *

make searches within the 'less' pager case-insensitive

 alias less='less -i'

A function to search through files for the given words

   function has {
    help=" - searches for files containing a given text"
    if [ -z "$1" ]; then
      echo -e "usage: $FUNCNAME searchterm \n $help" 
      return 3
    fi
    find . | xargs grep -l "$*" | less
    }

view all the files containing text 'screen saver'

 has screen saver

Remote Development ‹↑›

using a remote shell

 putty (from MS Windows), ssh, telnet

Streams ‹↑›

The idea of a 'stream' is very important in unix and bash, since most work is carried out in this paradigm. Streams can be redirected and piped etc.

Append stdout and stderr to a file, and print stderr to the screen

 somecommand 2>&1 >> logfile | tee -a logfile

add some text to the begging and end of standard input

 echo hello | (echo yes; cat -; echo no)

Unix Threads ‹↑›

Unix provides several simple mechanisms for allowing 'threads' (also called processes, or running programs) to communicate between each other. This is probably the most powerful single feature of the unix operating system.

see also: message queues, signals

Redirection ‹↑›

redirect stdout and stderr at the same time

 $>             only the bash shell
 command 2>&1   older style

redirect both standard output error messages to the file "output"

 command > output.txt 2>&1

Pipe stdout and stderr, etc., to separate commands

 some_command >>(/bin/cmd_for_stdout) 2>>(/bin/cmd_for_stderr)

redirect error messages to "stdout" and output to the file "save.txt"

 fond "dd" 2>&1 >save.txt

swap the stdout and stderr stream for the 'boggle' command

 boggle 3>&1 1>&2 2>&3

accept input from a file

 grep text <file

redirect output from multiple programs into another program

 diff -u <(ls -c1 dir1) <(ls -c1 dir2)

save the output of "sort" in "sorted" and also pipe to "uniq"

 sort list.txt | tee sorted.txt |  uniq -c | less

send a list of users to the printer and to a file "users.txt"

 awk -F: '{ print $1}' /etc/passwd | sort | tee users.txt | lp

Tee ‹↑›

show the date and also append it to 2 files

 date | tee -a file1 file2

Record output of any command using 'tee' at backend

 ssh user@server | tee logfilename

Pipes ‹↑›

cat large file to clipboard with speed-o-meter

 pv large.xml | xclip

write text or append to a file

 cat <<.>> somefilename

Complex And Named Pipes ‹↑›

00- Named pipes are like files, they need to be explicitly deleted when they are no longer needed. - A named pipe can remove the need to save data to a temporary file -

mknod and mkfifo are equivalent?

a simple pipeline and its equivalent named pipe version

    # the anonymous pipe version
    echo "test" | wc  
    # the named pipe version which is equivalent
    mknod apipe p; wc apipe; echo "test" > apipe

 sudo mkfifo mypipe; wc -l < mypipe >&1 ; ls > mypipe; rm mypipe;

create a named pipe which compresses things sent to it, send something to it to get compressed and then delete it.

    mkfifo mypipe
    gzip -9 -c < mypipe > out.gz
    cat file > mypipe
    rm mypipe

create a pipe which compresses a file and loads the data into mysql

    mkfifo /tmp/namedPipe
    chmod 0666 /tmp/namedPipe
    gzip -d file.gz > /tmp/namedPipe
    LOAD DATA INFILE '/tmp/namedPipe' INTO TABLE tableName;

make some named pipes and redirect to them

 mkfifo pipe1 pipe2; ls | tee pipe1 | less  delete pipe1 later

pipe the output of 2 commands to another command

 (cd /user/home && ls) | less

Who needs pipes?

 B <<< $(A)

Multiple Commands ‹↑›

only execute "ls" if the change directory "cd" command was successful

 cd /user/home && ls

Using File Handles ‹↑›

open a for reading and give it the 'file handle' 3

 exec 3< file.txt

open a file for reading and writing, write the text "yes", then close it

 exec 3<> file.txt ; echo yes >&3 ; exec 3>&- ;

Loops ‹↑›

:

Bash has both 'while' and 'for' loops but sometimes it is possible to use 'xargs' or 'parallel' instead, which may be simpler and neater.

an interpreting loop which breaks when the user enters 'q'

 while true; do read a; echo $a; [ "$a" = "q" ] && break; done

skip a broken piece of a loop but not exit the loop entirely

 ctrl + \

display files with 4 files on each line

 ls | xargs -L4

use xargs interactively prompting at each command

 ls | xargs -p -I{} cp "{}" "{}.bak"

execute multiple commands for every 'wav' file (play/sleep)

 locate '*.wav' | shuf | xargs -I{} echo "play {};sleep 2" | bash

Interpreting Loops ‹↑›

Interpreting loops can just receive an input from the user and do some action.

The following script allows a user to create a transcription from an audio file (containing people speaking).

A long loop doing audio stuff

    #while read -sn 1 -p "?" com ; do 
    f=jenny.txt
    audio=jenny.wav

    at=0
    dur=0
    echo "An audio transcription script..."
    echo "audio file '$audio', transcription file '$f'"
    while read -sn 1 com ; do 
      echo $com
      if [ "$com" == "p" ]; then
        at=$[at + dur]
        start=$(date +'%s')
        echo "play from $at seconds"
        play -q $audio trim $at &
        pid=$!
        # echo "pid=$pid"
      fi
      if [ "$com" == "a" ]; then
        start=$(date +'%s')
        echo "play from ${at}s for $dur s"
        play -q $audio trim $at $dur &
      fi
      if [ "$com" == "s" ]; then
        #pkill play 
        kill -9 $pid 2>/dev/null
        end=$(date +'%s') 
        dur=$[end - start]
        echo "duration $dur"
      fi
      if [ "$com" == "b" ]; then
        read -p "Start time back (seconds):" new
        at=$[at - new]
        dur=0
      fi
      if [ "$com" == "f" ]; then
        read -p "Start time forward (seconds):" new
        at=$[at + new]
        dur=0
      fi
      if [ "$com" == "t" ]; then
        read -p "Enter new start time (seconds):" new
        at=$new
        dur=0
      fi
      if [ "$com" == "m" ]; then
        read -p "Enter new start time (minutes):" new
        at=$[new * 60]
        dur=0
      fi
      if [ "$com" == "w" ]; then
        echo "seconds: $at" >> $f 
        echo " " >> $f
        # start vim at last line in insert mode with easy exit (X/Z)
        vim +  -c 'nmap X :wq<cr>' -c 'nmap Z :wq<cr>' -c start $f
      fi
      if [ "$com" == "i" ]; then
        echo "transcription file: $f"
        echo "audio file: $audio"
        echo "current section duration (in seconds): $dur"
        echo "current play start point (in seconds): $at"
      fi
      if [ "$com" == "h" ]; then
        echo "p play"
        echo "a play same section again"
        echo "s stop playing"
        echo "b set start time back n seconds"
        echo "f set start time forward n seconds"
        echo "t enter a new start point in audio (in seconds)"
        echo "m enter a new start point in audio (in minutes)"
        echo "w transcribe"
        echo "i show info"
        echo "h show help"
        echo "x exit"
      fi
      if [ "$com" == "x" ]; then
        pkill play
        echo "goodbye!"
        exit
      fi
    done

  ,,,,



FOR LOOPS ....

  * show the bash help for the 'for' loop
  >> help for | less

  * a simple loop
  >> for j in 1 2 3 4; do echo "j=$j"; done

  * a for loop with variable and computed limit values
  >> a=4; for n in $(seq $a $((a+4))); do echo $n; done
  
  * an arithmetic style for loop
  >> for ((i=0; i<$MAX; i++)) do echo $i; done

  * in recent versions of bash we can use curly braces too.
  >> for j in 1 2 3 4;{ echo "j=$j"; }

  * print the numbers 1 to 20
  >> for j in $(seq 1 20); do echo "j=$j"; done
  >> for j in {1..20}; do echo "j=$j"; done   ##(bash 3.0 and after)

  * a for loop with leading zero in bash version 3
  >> for i in {0..1}{0..9}; do echo $i; done

  * for loop with leading zero in bash 3
  >> seq -s " " -w 3 20

  * print the numbers 1 to 20 with a step of 2
  >> for j in $(seq 1 2 20); do echo "j=$j"; done

  * loop through the files in a directory 
  >> for file in dir/*; do echo $file; done 

  * loop through the files in the current folder and subdirectories 
  >> for file in $(ls *); do echo $file; done 

  * find all file with names ending in '.html' or '.php' containing the word 'big'
  >> find / \(-name \*.html -o -name \*.php\) | xargs grep -i "big" 
  ##(this uses the tool 'xargs' instead of a 'for' loop)

  * a for loop with filling 0 format, with seq
  >> for i in `seq -f %03g 5 50 111`; do echo $i ; done

  * using 'jot' in a for loop 
  >> for f in `jot - 0 50 5` ; do ping -c 1 -m 50 10.0.2.$f ; done

  * nested brace expansion in a for loop
   for host in {frontend{1..5},backend{1..3}}.mycompany.com
   do
    ssh $host "echo -n $host; uptime"
   done

loop over input line by line, not word by word

   IFS=$'\n'
   for f in $(ls -ls); do
     echo "loop=$f"
   done

another way to read input line by line, not word by word

   ls -thor | while read f; do
     echo "loop=$f"
   done

While Loop ‹↑›

show the bash help for 'while'

 help while

loop while the counter is less than 5

 c=0; while [ $c -lt 5 ]; do echo c is $c; let c=c+1; done

a loop which reads a line of text from the user infinitely

 while read s; do echo =$s; done

a loop through files and delete each if user enters 'd'

different shells
csh - a shell with a c like syntax
ksh - the korn shell
dash - a 'sh' implementation
sh - an old shell
fish - a simple non-compatible shell

Infinite Loops ‹↑›

make an infinite loop with while (stop it with <control c>)

 while true ; do echo yes; done
 while test 1 ; do echo yes; done  the same
 while [ 1 ] ; do echo yes; done   the same

loop until the user types 'q' using ">" as a prompt

 while [ "$r" != "q" ] ;do read -p'>' r; done; unset r

Break And Continue ‹↑›

Break and continue are used within loops to skip certain iterations, for example to stop processing once a certain item it found within a list.

use break to break out of the loop

 for j in $(seq 1 20); do break; done   does nothing

continue goes immediately to the next iteration

 for j in $(seq 1 20); do continue; done   does nothing

If Statement ‹↑›

show the bash help for the 'if' command

 help if | less

a basic if statement

 for f in *; do echo $f; read x; [ "$x" == "d" ] && rm $f; done

a short-hand way of writing an 'if' statement

 [ this -eq this ] && echo 'true'   prints 'true'

a short hand way of writing an if else statement

 [ this -eq that ] && echo 'true' || echo 'false'

print a message and exit if no 'c' files are found in the folder

 ls *.c 2>/dev/null || { echo "No C files found"; exit; }

If Else ‹↑›

construct an if else alternative

   if [ "1" == "" ]; then
     echo 'true'
   else
     echo 'false'
   fi

if with 'elif' (else if) statement

 if [ ${f} == "xx" ]; then echo $f; fi

Case Statement ‹↑›

an example of a case statement

   echo "choose"; read $o
   case "$o" in
     1) station="mms://server:port/path" ;;
     2) station="mms://otherserver:port/otherpath" ;;
     [3-9]) echo "wrong number"
     [[:lower:]]) echo "lower case"
     [qQ]) exit ;; 
     *) echo "invalid"
   esac

Tests ‹↑›

In the bash shell tests return either 0 (true) or 1 (false)

test if one number is less than another

 c=1; if [ $c -lt 5 ]; then echo $c; fi

test if the variable is a directory

 test -d "$HOME"; echo $?          prints either 0 or 1

combine two tests with the shell "and" operator

  if [ -n "$var"] && [ -e "$var"]; then
     echo "\$var is not null and a file named $var exists!"
  fi

combine two test with the "and" -a operator (a less common way)

 [ $c -lt 4 -a $c -gt 1 ]

combine two test with the "or" operator

 [ $c -lt 4 -o $c -gt 1 ]

Negating Tests ‹↑›

The negation operator is the exclamation mark "!"

A space between "!" and the next argument is necessary. A space between '[' and '!' is also necessary

negate a test with a "!" symbol at the front of the test

 [ ! -e ~/plans.txt ] && echo "no plans"

This prints "no plans" if the file "plans.txt" does not exist

one can also use 'test' instead of the '[]' brackets

 test ! -e ~/plans.txt && echo "no plans"

echo 'does not exist' if 'plans.txt' doesnt exist

   if test ! -e ~/plans.txt 
   then 
    echo "no plans"
   fi 

test if a string variable is not empty

 i="  x"; test ! -z $i && echo "not empty"   prints "not empty"

File And Directory Names ‹↑›

 if [ "${f} == "xx" ]; then echo $f; elif [ "${f} == "yy" ]; then echo $f; fi

list only subdirectories in the current

 ls -d */
 find -type d -maxdepth 1   similar

print file information in a readable format

 ls -thor    this is more compact than 'ls -la'

Find all dot files and directories

 echo .*

a list of all files on one line separated by commas

 echo * | tr ' ' ','

an alias to go back to the previous directory

 alias b='cd -'

strip out the path from a file name

 f=$(pwd); f=$(basename $f); echo $f

display the fifth file in the current folder

 a=($(ls)); echo ${a[5]}

This prints the current folder without the path.

a bash shell alternative to 'basename'

 echo ${file##*/}

rename files replacing one text with another

     for f in $(find . -name '*replaceme.jpg') ;
     do mv $f ${f/replaceme/withme}; done

rename files ending in '.JPG' with '.jpg' even if file has a space in it

 locate '*.JPG' | xargs -I{} rename 's/\.JPG$/.jpg/' "{}"

The '-n' option to "rename" is handy because it allows you to see what would happen without actually doing it.

rename files replacing the extension ".html" with ".php"

 for f in *.html; do mv $f ${f/.html/.php}; done

display only directory names in a listing

 ls -l | grep '^d'

jump to the folder used in the last command

 ls /usr/share/doc/; cd !$

just the directory name of the last path entered

 !$:h

just the file name of the last path entered

 !$:t

File Name Extensions ‹↑›

strip the suffix off a file name $f

 echo ${f%.*}

The above works even when the file name has dots in it.

rename files inserting '400w' just before file name suffix ---------- for f in sf/img/*; do exten=${f##*.} echo exten=$exten newname=${f%.*}.400w.$exten echo newname=$newname convert -monitor -resize 400 $f $newname break; done ,,,

Finding Files ‹↑›

The 2 command line tools are 'find' and 'locate'. Locate is fast and simple, but cant find very recent files. Find is slow and complicated but can do everything.

find files with a '.mp3' extension

 locate -r '.mp3$'

File Tests ‹↑›

moving around folders
cd - - go to the previous folder
cd - go to the users home folder
ls /usr/; cd !$ - jump to the last folder mentioned

compare file ages

 -nt newer than, -ot older than

test if a file exists

 if [ -f file.ext ]; then echo ${file.ext}; done

Scripts ‹↑›

A script is a set of bash commands placed into a text file and made executable. By convention bash script files often have a filename extension of '.sh' although this is not necessary.

If your script is very small consider making it a function instead and place the function in the '.bashrc' configuration file

Some file tests
-d is the file a directory
-f is the file a normal file (not a directory)
-e does the file or folder exist
-r is the file readable
-w is the file writeable
-s is the file non-empty
-S is the file a socket
-z is the file or variable empty

make a script (text file) executable

 chmod +x list.sh

make a script (text file) executable for all users

 chmod a+x list.sh

a simple example script which lists a file

    #!/bin/bash
    [ -z $1 ] && echo usage: $0 filename && exit
    cat $1

Executing Bash Scripts ‹↑›

Bash scripts are often given a file name extension of '.sh' such as 'smallscript.sh' but this is not necessary.

place a reference to the bash executable at the top of the script

 #!/bin/bash

This is called the 'shebang' line - from hash-bang

execute a bash script called 'showpeople' in the current folder

 ./showpeople

execute a script which is in the 'tree' sub-folder

 tree/showpeople

The methods above are the most usually way to execute to a script.

another way to execute a script

 eval $(cat script)

execute a (non-executable) bash script

 source script ??

find out where the 'bash' executable is on the computer

 which bash

execute the bash commands contained in the file 'script'

 cat script | bash

Using the technique above, the file 'script' does not have to be executable.

run the bash script 'script.sh' in debugging mode

 bash -x ./script.sh

run a bash script in debug mode, show output and save it

 bash -x script.sh 2> log

execute a shell script in the background and dont interrupt it

 nohup /bin/sh myscript.sh 1>&2 &>/dev/null 1>&2 &>/dev/null&

Read commands from string and assign any arguments to positional params

 bash -c 'set w x y z; IFS=":-;"; echo "$*"'

Script Parameters ‹↑›

# This section deals with parameters which are passed from the # command line to a Bash script.

get the absolute path to your bash-script

 scriptpath=$(cd $(dirname $0);pwd)

display a message and exit if no parameters were set

 [ -z "$1" ] && echo "no parameters were given" && exit 1

exit if exactly 3 parameters were not given

   if [ $# -ne 3 ]; then
     echo "usage $0 <file> <to> <from>"
     exit 1
   fi

the string $@ contains an array of all the parameters

 a="$@"

all the parameters as a string not an array

 s="$*"

get the 2nd and rest of the parameters

 echo ${*:2}
 echo ${@:2}

get the 2nd and 3rd parameters

    echo ${@:2:3}
    if the script is invoked with
      ./script one 2 three 4
    then this will print
      "2 three"

do something for each parameter given

 files="$@"; for f in files; do echo $f; done

show the number of script parameters

 echo $#

test if a particular parameter exists

 if test -z "$1" ;then exit; fi   exits if no first parameter
 if [ -z "$1" ] ;then exit; fi    the same

assign a value to a variable depending if a parameter exists or not

 [ -z "$1" ] && x=”$1” || x="/bin/date"  (works ???)

if 'foo' has no value, assign it the value 'one'

 foo=""; echo ${foo:=one}  prints 'one'

maybe the quickest way to get the current program name minus the path

 programname="${0##*/}"

Putting A Script In The Path ‹↑›

to execute a script without the './' set the path to a user 'bin' folder

 mkdir ~/bin; [in ~/.bashrc add] export PATH=$PATH:$HOME/bin

execute the 'showpeople' bash script which is located 'on the path'

 showpeople

Another technique to make a bash script executable by everybody is to put a symbolic link to it in the '/usr/local/bin' folder.

put a symbolic link in the /usr/local/bin

 ln -s /usr/local/bin/go ~/scripts/super.sh (wrong ??)

search for a file in the system path

 type <filename>

Script Configuration ‹↑›

check the bash version number in a script

   if [[ $BASH_VERSION > "2.05a" ]]; then
     ...
   fi

check if the script has been run as root

    if [ $(id -u) != 0 ]; then
      echo "This script must be run as root" 1>&2
      exit 1
    fi

User Interaction ‹↑›

the steps to run a bash script
find out where you bash is with 'which bash'
put #!/bin/bash at the top of the script (see previous step)
write your bash script
make your script executable with 'chmod +x script'

Traditionally Unix and Linux was the domain of command line hackers who enjoyed hours of typing cryptic commands into a black screen with green text in a strange language devoid of vowels and meaning. But the fact remains that the average human being has a gut-instinct aversion to being forced to using the command line to do things. It is possible to use various Linux tools to present the user with a slightly more enjoyable interface.

User Input ‹↑›

The easiest way to interact with the user is just to ask the user to enter some information at a shell prompt. This can be down with the 'read' command.

display a prompt and get a response from the user

 echo enter name; read s; echo you typed $s;

get more than one value from the user

 read d m; echo day $d month $m

get a single key press from the user

 echo press key; key=$(dd bs=1 count=1 2> /dev/null); echo $key;

see http://tldp.org/LDP/abs/html/commandsub.html for more information (the terminal mode needs to be changed first)

press any key to continue

 read -sn 1 -p "press any key to continue..."; echo

function that outputs dots every second until command completes

 sleeper(){ while `ps -p $1 &>/dev/null`; do echo -n "${2:-.}"; sleep

display todays date on the desktop

 date | osd_cat

display a webpage as text and update the view it every 10 seconds

 watch --interval=10 lynx -dump http://dslrouter/stats.html

a loop through files and delete each if user enters 'd'

tools
read - allow the user to input some text
select - presents the user with a list to select from
whiptail - displays 'dialog boxes' in a shell window
dialog - like whiptail
osd_cat - display text on the desktop (not is the console)
zenity - display nice windows from shell scripts (uses gtk)
kdialog - like zenity but for the kde linux desktop
wish - run a tk window script
complete - automatically completes commands
perl tk - display windows from within a perl script
gtk - a library for writing window applications

Dialog And Whiptail ‹↑›

Whiptail and dialog are basically one step up from 'select', and one step down from 'yad' and 'zenity'. Dialog and whiptail are basically equivalent, but are both quircky and less useful than they should be.

Whiptail installed by default on most debian systems

a whiptail menu with a return value

 r=$(whiptail --title "files" --menu "choose" 20 60 15 a app b ball c cat 3>&1 1>&2 2>&3); echo $r

with brace expansion

 r=$(whiptail --title files --menu "choose" 20 60 15 {A..z} 3>&1 1>&2 2>&3); echo $r

a whiptail file listing

 r=$(whiptail --title files --menu "choose" 20 60 13 $(ls | sed "s/$/ ./" | tr '\n' ' ';) 3>&1 1>&2 2>&3); echo $r

Show a 'curses' based menu selector

 whiptail --checklist "Simple checkbox menu" 11 35 5 tag item status repeat tags 1

a trick to get a return value from whiptail

 foobar=$(whiptail --inputbox "Enter some text" 10 30 3>&1 1>&2 2>&3)

return values from dialog are more easily got

 result=$(dialog --output-fd 1 --inputbox "Enter some text" 10 30)

Present A Choice To The User ‹↑›

The 'select' tool is a simple way to present the user with a choice and perform and action based on that choice.

choose a file in the current folder to edit

 PS3="Enter a number: "; select f in *; do vim $f; break; done

[= image/eg-select-edit.png]

choose an image for a list and open it

 IFS=$'\n'; select f in $(locate -i '/home/*.jpg'); { xdg-open $f; break; }

Notice that select can use curly braces (like 'for') is modern versions of the bash shell.

set the 'IFS' variable if file names may contain spaces

 IFS=$'\n'; PS3="Choose dir:"; select f in $(find . -type d);{ cd $f;break;}

a function to list folders named with some text and jump to selected

 ju() { PS3="dir:"; select f in $(find ~ -type d -iname "*$1*"); do cd $f; break; done; }; ju doc;

choose from 2 phrases to print

 select f in "1 ball" "2 balls"; do echo $f; break; done

choose a big file to delete

 select f in $(find . -size +2M); { rm $f; echo $f; break; }

present the user with a choice of options

   if [ "1" == "" ]; then
     echo "
      [1]   Radio 1
      [2]   Radio 2
      Enter your choice: "
     read choice
   else
     choice="1"
   fi
   case "$choice" in
     1) station="mms://server:port/path" ;;
     2) station="mms://otherserver:port/otherpath" ;;
     *) echo "Wrong choice!" exit
   esac

use select to jump to a line in a text file containing the text 'determ'

    IFS=$'\n'; PS3="Choose:"; n=bash-book.txt; 
    select f in $(grep -n determ $n)
    do
      l=$(echo "$f" | cut -d' ' -f2)
      vim +$l $n; break;
    done
  ,,,,

  Notice the judicious use of the 'IFS' variable above. IFS determines how
  input will be split into elements. By default it will split all input on
  all whitespace (that is, into words) but here we want it to split the
  input into lines so we have to set IFS to only the new-line character. 

  The problem is that if the output contains more lines than the screen,
  the user will never see the beginning of the output.

  * use select to jump to a line in a text file containing the text 'determ' 
    IFS=$'\n'; PS3=""; n=test.txt; select f in $(grep -n determ $n); do l=$(echo "$f" | cut -d' ' -f2); vim +$l $n; break; done
  ,,,,

  * search for a word in 'words' then jump to the one selected in a window
  >> f=/usr/share/dict/words; vim +/$(grep -i jump $f | zenity --list --column= --height=800)/ $f

  * try to kill a process chosen by the user
  >> IFS=$'\n'; PS3="which?:"; select f in $(ps -ef|tr -s ' ');{ echo $f;break;}

YAD ....

  Yad is a replacement for zenity, and is much better.

ZENITY .... 

  zenity is a way of easily displaying windows with user
  interface elements which allow the user to carry out
  actions or answer questions etc. Or use tcl/tk

  * view the man page for zenity
  >> man zenity 

  * show a calender in a window 200 by 800 pixels
  >> zenity --calendar --title 'select a file' --height=800 --width=200

  * show a list box 800 pixel high and 800 pixels wide
  >> ls -thor | zenity --list --column=  --height=800 --width=800

FILE SELECTION ........

  * allow the user to select a file and echo the name of the chosen file
  >> zenity --file-selection --title 'select a file'
  
  * select a file in the /home/bob/os/ folder 
  >> zenity --file-selection --filename="/home/bob/os/" 

  * a function which searches for files and then opens the selected one
    function op {
      [ -z "$1" ] && echo "usage: $FUNCNAME <file>" && return 1;
      xdg-open "$(find ~ -iname '*$1*' | zenity --list --column= --width=700 --height=700)"
    }

display a list of all pngs on the computer and display selected one

 locate '*.png' | zenity --list --column=Images | xargs -I{} xdg-open "{}"

The 'xargs -I{} xdg-open "{}"' is important for the case where a file name contains a space.

locate all jpgs in user home folders

 xdg-open "$(locate -i '/home/*.jpg' | zenity --list --column=)"

The recipe above also handles filenames with spaces in them, owing to the judicious use of double quotes

 xdg-open "$(locate -i '/home/*.jpg' | zenity --list --column=)"

Text Entry ‹↑›

display a dialog which allows the user to enter some text

 zenity  --title="New File" --entry

display a window titled 'Name' with a text box and text prompt

 zenity  --title "Name"  --entry  --text "Enter your name"

List Boxes ‹↑›

make a window with a list-box with the files from current folder

 ls | zenity --list --column="test" --title='choose a file'

[= image/eg-zenity-list.png]

display a list of folders in the current folder with no window title

 ls -d */ | zenity --list --column="test"

display a list of files in a window with no title or column-title

 ls | zenity --list --column=

show a list of files and then display the file-name which was chosen

 echo "you chose $(ls | zenity --list --column=)"

jump to any folder in the current folder

 cd $(ls -d */ | zenity --list --column= --title="change folder")

jump to any folder in the users home directory (could be slow)

  cd $(find ~ -type d | zenity --list --column= )

an alias to display lists with the zenity tool

 alias zenls='zenity --list --column='

We can then use the alias above in a command pipe-line such as

 ls | zenls

make an editable text box with a file listing saving changes to 'j.txt'

 ls | zenity --text-info --editable > j.txt

Multicolumn List Boxes ‹↑›

create a list with 2 columns, one all 'false' and the other files

 zenity --list --column=buy --column=item $(echo * | sed 's/ / FALSE /g;s/^/FALSE /')
ls | tr '\n' ' '

create a list with 2 columns, the first check-boxes and the other files

 zenity --checklist --list --column=buy --column=item $(echo * | sed 's/ / FALSE /g;s/^/FALSE /')

a zenity mulicolumn example checkbox example

 zenity --list --checklist --column=Buy --column=Item TRUE "Red Apples" TRUE Oranges FALSE 'Green Pears' FALSE Toothpaste

 for f in *; do echo $f; read x; [ "$x" == "d" ] && rm $f; done

Zenity Notifications ‹↑›

show an icon in the notification area

 zenity --notification --window-icon=up.png --text "System update necessary!"

Kdialog ‹↑›

Tk And Tcl ‹↑›

It should be possible to use the 'wish' interpreter and the tk language to interact with the user in a 'graphical' fashion from a bash script, but this is a large topic.

www: http://pages.cpsc.ucalgary.ca/~saul/personal/archives/Tcl-Tk_stuff/tcl_examples/
some tk examples
a simple tk label example
    label .l1 -text "ground" -foreground blue 
    label .l2 -text "big" -font {-family times -size 24}
    grid .l1 -row 0
    grid .l2 -row 1

buttons --------- #!/usr/bin/wish button .my_button -text "Hello World" -command exit pack .my_button ,,,

a button which exits when clicked

 echo 'button .b -text "Hello World" -command exit; pack .b' | wish

[=0.2 image/eg-tk-button-exit.png]

a tk list-box with text items and file names

 (echo -n 'listbox .lb -selectmode multiple -height 14; .lb insert 0 sample stuff colors red yellow green '; echo -n * ; echo '; pack .lb') | wish

[=0.2 image/eg-tk-listbox-files.png]

another way to execute some tk or tcl commands

 cat script | wish

Perl Tk ‹↑›

Using perl with 'tk' is another way to interact with the user graphically.

install perl/tk with apt on a debian linux type system

 apt-get install perl-tk

another way to try to install the perl 'Tk' module

 perl -MCPAN -e 'install "Tk"' didnt work!!

view the very good help for the perl tk module

 man perl-tk

run the demonstration program showing all perl tk possibilities

 widget

[= image/eg-perltk-widget.png]

a perl-tk one liner to create a button

 echo "use Tk; new MainWindow->Button(-text=>'click!')->pack; MainLoop;" | perl

tk example

    #!/usr/bin/perl -w
    use Tk; use strict;
    my $main = new MainWindow;
    my $button = $main->Button();
    $button -> configure(-text => 'Press me!');
    $button -> pack;
    MainLoop; 

display a directory tree.

  use Tk;
  use Tk::DirTree;
  my $top = MainWindow->new;
  my $dl  = $top->Scrolled('DirTree')->pack(-expand => 1, -fill => 'both');
  MainLoop;

Program Output ‹↑›

suppress program output messages

 prog &> /dev/null

suppress error messages

 prog 2> /dev/null

print the return value of a program

 prog; echo $?

return a value to the shell

 exit 31      exits the script and returns 31

Randomness ‹↑›

play a random 'wav' sound file, but it fails if names have spaces

 a=($(ls *.wav)); play "${a[$RANDOM%${#a}]}"

play a random wav file in the current folder

 f=$(ls *.wav | sed -n "$[$RANDOM%$(ls | wc -l)]p" ); play "$f"

play a random wav, but this gives an error if no wavs available

 play "$(ls *.wav | shuf | head -1 )"

only print a message sometimes (randomly)

 (($RANDOM%6)) || echo 'hello world!'

display a random file in the current folder

 a=($(ls)); echo ${a[$RANDOM%${#a}]}

randomly shuffle lines of a file (but needs gnu sort, not bsd)

 sort --random-sort

another random file lines sort

 cat test.txt | while read f ; do printf "$RANDOM\t%s\n" "$f"; done | sort -n | cut -f2-

shuffle lines in a file on a system without 'shuf'

 cat /usr/share/dict/words | perl -MList::Util -e 'print List::Util::shuffle <>' | less
 perl -MList::Util=shuffle -e'print shuffle<>'

Special Variables ‹↑›

The script name is contained in $0

 echo usage: $0 filename

$? contains 0 or 1 the result of the last program (or test)

the process id of the script

 $$

Special Characters ‹↑›

add 2 newlines to a string variable

 s="tree"$'\n'$'\n'; echo "$s"  try "echo $s" to see an echo "gotcha"

Bash Expansion ‹↑›

Command Completion ‹↑›

execute the last command

 !!

execute the last command entered with root privileges

 sudo !!

enter the last command typed on the command line

 alt + .

inserts the last used arguments

 <esc> .

Inserts the results of an auto-completion in the command line

 ESC *

Configuring Command Completion ‹↑›

some useful settings in ~/.inputrc:


   set completion-ignore-case on
   set show-all-if-ambiguous on
   set show-all-if-unmodified on

Enable ** to expand files recursively (>=bash-4.0)

 shopt -s globstar

Enable automatic typo correction for directory names

 shopt -s cdspell

Bash autocomplete case insensitive search

 shopt -s nocaseglob

bash glob dot-files (such as '.bashrc')

 shopt -s dotglob

Lists In Bash ‹↑›

Bash has several methods of generating lists, including "globbing", and "brace expansion".

Globbing ‹↑›

rename a file using special brace expansion

 cp /long/path/file.{txt,doc}

assign a list of folder names to a variable

 d=$(echo */); echo $d

Be careful here: "d=*/; echo $d" appears to have the desired effect but in reality, the variable 'd' does not contain a list, just the characters '*/'

get a list of folder names and replace all '/' with ':'

 d=$(echo ~/sf/htdocs/books/*/); echo ${d//\//:}

get a list of folder names but only print the last element of the path

 d=$(echo ~/sf/htdocs/*/); d="basename "${d// /;basename }; eval $d

The recipe above is fairly nifty. It used globbing, string substitution and 'eval' to execute the bash commands, contained in the variable "d". In these capabilities it gets close to the power of a functional or "list-based" language such as lisp.

get a list of folder names and double up each folder name

 d=$(echo ~/sf/htdocs/*/); d="basename "${d// /;basename }; d=$(eval $d); echo $d | sed 's/\w\+/& &/g'
This prints some like 'docs docs html html img img ...' etc

Arithmetic Brace Lists ‹↑›

print the numbers from 1 to seven

 echo {1..7}      modern bash versions

enumerate with the numbers padded with zeroes

 echo {001..5}

a for loop with leading zero (bash version 3)

 for i in {0..1}{0..9}; do echo $i; done

use a brace arithmetic series with a 'step' value (bash version 4)

 echo {1..20..2}

Brace Expansion ‹↑›

Brace expansion is another way of generating lists of strings in the bash shell. In some ways this is reminiscent of list based programming languages, such as 'lisp' or 'scheme'. "Globbing" is another technique for generating lists of file names.

use a numeric sequence 'glob' (expansion) to create speech

 echo {1..3}" o'clock" ROCK | espeak

print an alphabetic expansion

 echo {a..z}"x "          prints 'ax bx cx ... zx'

multiply by nothing, zero string, create a file list

 echo *.html{"",.gz}

use an explicit list for with brace expansion

 echo n:{pin,nail,ball}
This will print 'n:pin n:nail n:ball'

an html style brace expansion

 echo "<li>"{a..z}"</li>"

dynamic globbing can be done like this

 e=4; eval "echo {1..$e}"; echo {1..4}

print a comma delimited file list

 a=$(echo *); echo ${a//" "/, }

print a colon delimited list of folders in the current folder

 a=$(echo */); echo ${a//" "/:}

do a 'dynamic' brace expansion on a list of file names

 echo "echo "{$(echo * | tr ' ' ',')}"xx" | bash

another way

 a=$(echo *); echo 'echo file:{'${a//" "/,}} | bash

make the listing one per line with the -e switch

 a=$(echo *); echo 'echo -e "\nfile:{'${a//" "/,}} | bash

make an html directory listing out of the current folder

 echo "echo \<li\>"{$(echo * | tr ' ' ',')}"\</li\>" | bash

The technique used above could equally be done using a 'for' loop such as 'for f in *; do ... ; done' but I find this brace expansion concept interesting.

make 2 backups of a file using brace expansion

 cp list{,bak,bak1}

using nested string brace expansion (bash version >= 3.0)

 echo {frontend{1..5},backend{1..3}}.server.net

list all the directories in the current folder

 echo */
 ls -d */
 ls -F | grep '/$'

list all hidden files (starting with '.') in the current folder

 echo .*

Executing Bash Commands ‹↑›

find out where the 'wget' program is

 type -a wget
 which wget    similar

execute bash commands from standard input

 echo "cd /; ls" | bash

execute commands in a string

 bash -c 'ls'

remove lots of files, but first check which ones will be removed

 find dir -name \*~ | xargs echo rm
 find dir -name \*~ | xargs echo rm | bash

use eval to execute the command 'ls'

 eval 'l''s'

Command Substitution ‹↑›

delete a list of files named in a text file

 rm `cat file`
 rm $(cat file)      this is the same

execute a command which is contained in a variable

 cmd=ls; $cmd

Process Substitution ‹↑›

compare the packages installed on 2 servers

  diff <(ssh server1 'rpm -qa | sort') <(ssh server2 'rpm -qa | sort')

Variable Assignment ‹↑›

use export to make a variable available to subprocesses

 export name=value

set a variable to the contents of a file

 v=$(<file)
 v=$(cat file);     the same

Variable Type ‹↑›

Variables in bash are normally 'untyped' but can be declared as a certain 'type' using the 'declare' keyword

check if a variable is not a number

 v='blah';if [ -z $(echo $v | grep [0-9]) ]; then echo "NON NUMERIC"; fi

check if a variable is not a number

 v='1';[ -z $(echo $v | grep [0-9]) ] && echo "not a number" || echo 'number'

declare variable as integer

 declare -i aa ; aa=3*8 ; echo $aa
 declare -i aa ; aa=3*8 ;aa='tree'; echo $aa

If a string is assigned to the variable then the variable will be set to '0'.

Arithmetic ‹↑›

Even And Odd ‹↑›

check if a number is even

 [ $((a%2)) -eq 0 ] && echo "even"

check if no of files in dir is even or odd

 n=$(ls | wc -l); [ $((n%2)) -ne 0 ] && echo "odd"

define a quick calculator function '?'

 ? () { echo "$*" | bc -l; }

The function above can be called with '? 4*5' for example, which will print '20'.

add two numbers using bash operators

 echo $((4+5))
 echo $[4+5]

perform addition with the 'bc' tool

 echo 1+1|bc

c style variable incrementation,

 a=8; (( a++ )); echo $a;       prints 9

integer arithmetic and assignment

 a=3; b=$(( a + 5 )); echo $b;  prints 8

add up all the numbers from 1 to 1000

 echo {1..1000} | sed 's/ \+/+/g' | bc
 seq 1 1000 | (sed 's/^/x+=/'; echo x) | bc   another way

sum all the numbers in the file 'list.txt' with each number on a line

 (sed 's/^/x+=/' list.txt ; echo x) | bc

display the product of 512 and 7

 expr 512 \* 7

creating a list of numbers for calculation

 echo $( du -sm /var/log/* | cut -f1 ) | sed 's/ /+/g'

a short counter

 yes '' | cat -n

Floating Point Arithmetic ‹↑›

Unfortunately many linuxes dont come with 'bc' installed by default, which means you may not be able to rely on these techniques.

divide 22 by 23 and print result to 4 decimal places

 echo "scale=4; 22/23" | bc

subtract one decimal number from another

  echo "4.54-6.332" | bc

a floating point operation with no precision specified

 echo 7/2 | bc -l               prints 3.50000000

another way but not working!!??

 bc -l <<< s(3/5)

Integer Variables ‹↑›

declare variable as integer

 declare -i aa ; aa=3*8 ; echo $aa

print 'yes' if the variable 'x' is a number

 i=3; if [ "$i" -eq "$i" 2>/dev/null ]; then echo yes; fi

increment a variable

 c=4; let c=c+1;    c is now 5

increment the counter by 1

 c=4; let c+=1;     c is now 5

decrement the counter by 1

 c=4; let c-=1;     c is now 3

some zenity options
--text "hello" - display hello at the top of the window
--column "file" - display 'file' at the top of the column

String Variables ‹↑›

echo the text 'linux' in reverse

 echo linux | rev

reverse the order of the characters in a string

 s='tree'; s=$(echo $s | rev); echo $s

check if a the the word 'madam' is a palindrome

comparing numbers, the tests are
-eq - equal to
-ne - not equal to
-lt - less than <
-le - less than or equal <=
-gt - greater than >
-ge - greater than or equal >=

Analysing Strings ‹↑›

test if a variable consists only of white space or nothing

 i="  "; if [ -z $i ]; then echo "empty"; fi
 i="  "; [ -z $i ] && echo "empty"            the same

test for an empty string variable

 i="  x"; test ! -z $i && echo "not empty"   prints "not empty"

Both of these scripts above will print 'empty'

test if the string variable $s contains the text 'tree'

 s='madam'; [ "$s" == "$(echo $s | rev)" ] && echo palindrome! || echo nope

print 'has a dot' if the string contains a full stop

 s='this tree'; if [[ $s == *tree* ]]; then echo "yes"; fi
 s='one.two'; if [[ $s == *.* ]]; then echo "has a dot"; fi

print 'no dot' if the variable 's' contains no full stop

 s='onetwo'; [[ $s != *.* ]] && echo "no dot"

display the number of characters in a string

 aa=bbbb; echo ${#aa}   prints 4

String Case ‹↑›

Convert the word 'tree' to uppercase

 echo tree | tr '[:lower:]' '[:upper:]'

convert the string variable 's' to upper case

 s="the grass"; s=$(echo $s | tr '[:lower:]' '[:upper:]');echo $s

Comparing Strings ‹↑›

test if 2 strings are not the same

 if test "$a" != "$b" ; then echo 'not the same'; fi
 if [ "$a" != "$b" ]; then echo 'not the same'; fi  another way

test if 2 strings are the same or different

 if test "$a" = "$b" ; then echo 'the same'; fi

Splitting Strings ‹↑›

split the string 'aa bb cc' on whitespace (prints 'bb')

 ARRAY=(aa bb cc);echo ${ARRAY[1]}

split the string "aa bb cc" into 3 variables

 read v1 v2 v3 < <(echo aa bb cc); echo $v2

split a string into an array using commas as delimiter

 s='a,b,c'; a=(${s//,/ }); echo ${a[1]}

 a=xyzxyz; a=${a//x/q}; echo $a;     prints qyzqyz
 ARRAY=(aa bb cc);echo ${ARRAY[1]}

Whitespace ‹↑›

show whitespace in a visible manner

 echo -e "\t\n" | cat -vte

Replacing And Substituting Strings ‹↑›

replace the first "x" in the string with a "q"

 a=xyzxyz; a=${a/x/q}; echo $a;     prints qyzxyz

replace all occurrences of 'x' in the string 'a'

 a=xyzxyz; a=${a//x/q}; echo $a;     prints qyzqyz

replace all occurrences of 'a' in a string with the word 'the'

 s="a book. a tree"; s=${s//a/the}; echo $s;

replace all occurrences of '.' in a string with a '/' slash character

 s="a.b.c"; s=${s//./\/}; echo $s;

Substrings Of Strings ‹↑›

extract a substring from a string

 s=abcde; echo ${s:1}   prints bcde, the string "s" is not changed

 s=abcdefghi; echo ${s:3:3}   prints def

delete an exact match of one variable within another

 f=tree; r=re; f=${f/$r/}; echo $f   <: prints "te" :>
 f=tree; r=re; f=${f##$r}; echo $f   <: the same :>

print only the first word of a sentence

 a="one two thre"; echo ${a%% *}

print only the last word in a sentence

 foo="simple forth machine"; echo ${foo##* }
This prints 'machine'

strip all but the last word from a sentence

 s="simple forth machine"; s=${s##* }; echo $s
The variable 's' now has the text "machine"

delete only the first match of a substring in a string

 foo="this is a test"; echo ${foo#t}   prints 'his is a test'

delete the shortest match starting from the left

 foo="this is a test"; echo ${foo#t*is}   prints 'is a test'

 s="12: this is a test"; echo ${s#t*t}

delete the longest match from the left

 foo="this is a test"; echo ${foo##t*is}   prints 'a test'

deletes the last match of a substring in a string

 foo="this is a test"; echo ${foo%t}    prints 'this is a tes'

delete the last 2 characters of a string

 foo="this is a test"; echo ${foo%??}    prints 'this is a te'

deletes the shortest match from the end

 foo="this is a test"; echo ${foo%t*st}    prints 'this is a'

deletes the longest match from the end of the string variable

 foo="this is a test"; echo ${foo%%t*st}    prints nothing

remove the file name extension from a filename

 f=index.txt; f=${f%.txt}; echo $f  prints 'index'
 f=index.txt; mv f ${f%.txt}        actually renames the file, careful

See http://tldp.org/LDP/abs/html/refcards.html for more patterns

Concatenating String Variables ‹↑›

Joining 2 or more string variables together into one big string is known as "concatenating".

append a string onto the end of a variable

 m=aa; m+=bb; m+=cc; echo $m   prints 'aabbcc'

The above is probably a recent bash extension.

add the string "tree" to the end of a variable

 f=green; f=$f"tree"; echo $f    prints "greentree"
 f=green; f=${f}tree; echo $f    the same
 f=green; f="${f}tree"; echo $f  the same again

join a string to the end of another string.

 f="this is";g=" a tree"; f=$f$g; echo $f  prints "this is a tree"
 f="this is";g=" a tree"; f="$f$g"; echo $f  the same

add 2 new-line characters to a string

 s=tree; s=$s$'\n'$'\n'; echo "$s"  the quotes on the final $s ARE needed

Writing Strings To A File ‹↑›

use echo -n "$data" instead of echo $data which 'squeezes' newlines

 echo -n "$data" > /home/username/data.txt

Here Documents ‹↑›

The practice of creating a long string variable between a start and an end label is called a 'here document'. This syntax exists in a number of programming languages (for example: PHP and Perl) Here documents are useful for printing or manipulating long multi-line strings.

use a 'here document', for long strings

   cat << endx 
     This is a string with several lines.
     The end label can be anything, and can
     appear in the text as long as it doesnt start
     the line (like this 'endhere')
endx                  
the end label 'endx' must be at the start of the line

put commands and variables in a 'here document' (use $(command))

    cat << ends
     The script name is $(basename $0)
ends

make a 'here document' which does not expand variables or commands

    cat <<'enddoc'
     commands such as $(pwd) will not be interpreted
     nor with variables such as $s
enddoc
the trick is to use "<<'LABEL'" instead of "<<LABEL"

www: http://forums.whirlpool.net.au/forum-replies-archive.cfm/759361.html
A good comment about newline characters and bash
assign a here document to a bash variable
   s="";while read -r line; do s=$s$'\n'$line; done <<zzz
   making a
   multi-line
   bash variable
zzz

# in the script above, the "read" statement reads one # line at a time from the here document and adds it # onto the 's' variable with a newline character

put string substitutions in a "here" document

    f=text.html
    cat << eos
     The script is ${f/.html/}
eos
this prints "the script is text"

pipe the output of a here document through another program


    cat << eos | fmt
     The script name is $(basename $0)
eos

run a tcl script as a here document

    cat << eos | tclsh 
      puts "Seconds: [clock seconds]" 
eos

loop through the lines a "here document"

  while read -r s; do echo $s; done <<zzz
  first line
  second line
  3rd line
zzz

Interpolation In Strings ‹↑›

Interpolation means substituting variables within a string value with their values.

insert the value (text) of the variable 'v' in a string

 v=green; echo "The grass is $v"
this prints "The grass is green"

Bash will generally 'interpolate' variables which are inserted with double-quoted (") strings. Interpolation is as idea which also occurs in perl, and other scripting languages.

stop a variable from interpolating with \$

 v=5; echo "\$v is $v"

string interpolation works with array values as well

 area[5]=bb; echo "The contents of area[5] is ${area[5]}."

a complex example of interpolation using eval

 a=b; eval $(echo "echo \"this is xx\"" | sed 's/xx/$a/')
 a=b; eval $(echo "echo this is xx" | sed 's/xx/$a/')    also works
 export a=b; echo "echo this is xx" | sed 's/xx/$a/' | bash  also

All 3 examples above do the same thing. But the last requires the 'export' statement to make the 'a' variable available to all subprocesses. This is because the 'bash' command begins a new process.

a simple 'templating' technique using interpolation

 echo -e 'todays \ndate \nis <date>' | (echo 'cat << EE';sed 's/<date>/$(date)/g'; echo 'EE') | bash

The text '<date>' in the input stream gets replaced by todays date. This is useful when the input-stream comes from a file.

use the technique above to substitute the date into the template

 cat template | (echo 'cat << EE';sed 's/<date>/$(date)/g'; echo 'EE') | bash

Date Variables ‹↑›

check if a date is valid, even a date within a leap year- not BSD ??

 date -d2009-05-18 > /dev/null 2>&1 ; echo $?

another date validation but not BSD unix

 date -d2009-02-33 > /dev/null 2>&1 ; [ $? ] && echo valid || echo invalid

display a millisecond time difference

 s=$(echo "scale=3; $(date +%s.%N)-$(date +%s.%N)" | bc);  echo ${s%??????}

display seconds and milliseconds since 1970 (gnu date only)

 s=$(date +%s.%N);  echo ${s%??????}

Arrays ‹↑›

Bash arrays are 0-based, that is the first element of an array 'list' is 'list[0]' not 'list[1]' (as in Ms Visual Basic)

www: http://tldp.org/LDP/abs/html/arrays.html
some information about bash arrays.
display the fifth file in the current folder
 a=($(ls)); echo ${a[5]}

display the file which has been most recently modified in the folder

 a=($(ls -t)); echo ${a[0]}

display a random file in the current folder

 a=($(ls)); echo ${a[$RANDOM%${#a}]}

echo each element of a comma delimited array.

 echo one,two,three | xargs -d, -I{} echo "{}"

Declaring Arrays ‹↑›

In the bash shell, it is normally not necessary to declare variables (that is, state what type a variable is- whether a string, an integer, or an array). But it may be advantageous in some cases.

declare an array variable 'list' explicitly

 declare -a list

explicitly un-declare an array 'aa'

 unset -v aa

Initialising Arrays ‹↑›

initialize an array variable called 'list'

 list=(ant dog cat)

the element list[0] is 'ant' the element list[1] is 'dog' ...

assign values to the 4th and 7th elements of the array 'area'

 area=([4]=hobart [7]=brunie)

make a new array 'files' from the current folder and display the 3rd

 files=( *.txt ); echo ${files[2]}

assign the lines of a text file to an array (1 line per element)

 ls -thor>t.txt; IFS=$'\n' aa=($(<t.txt)); echo "3rd=${aa[3]}"

the same as above but IFS may have to be reset later

 ls -thor>t.txt; IFS=$'\n'; aa=($(<t.txt)); echo "3rd=${aa[3]}"

assign the words of a text file to an array (1 word per element)

 cat /usr/share/dict/words>t.txt; aa=($(<t.txt));echo "100th=${aa[100]}"

create a new array and display all the elements

 list=( *.txt ); echo ${list[@]}
 list=(*.txt); echo ${list[*]}      also works

define the elements of an array over multiple lines

    list=("a dog"
     cat "a tree")
     echo ${list[0]}

define the elements of an array over multiple lines

    list=(
     "a dog"
     cat "a tree"
    )
    echo ${list[0]}

Big Arrays ‹↑›

load a very large array from the dictionary file

 aa=($(cat /usr/share/dict/words)); echo ${aa[1000]}

Since the standard unix dictionary may contain more than 200000 words the line above gives you an idea of how many elements a bash array may contain and how fast it takes to load that many elements

see how long it takes to load a big array

   f=/usr/share/dict/words; 
   time aa=($(cat $f));
   echo The array has ${#aa[@]} elements

Initializing Arrays From Files ‹↑›

assign the lines of a text file to an array (1 line per element)

 ls -thor > t.txt; IFS=$'\n'; aa=($(<t.txt)); echo "3rd=${aa[3]}"

Note the judicious use of IFS, since otherwise the text of the file 't.txt' would be split into words rather than lines.

assign the lines of a text file to an array (1 line per element)

    ls -thor > t.txt    
    IFS=$'\n'
    aa=($(cat t.txt))
    echo "3th element=${aa[3]}"

mapfile is also supposed to work, but not for me

 echo -e "a b\nc d\ne f" | mapfile -n 0 aa; echo ${aa[*]}
 echo -e "a b\nc d\ne f" | mapfile aa; echo ${aa[*]}
 echo -e "a b\nc d\ne f" | mapfile ; echo ${mapfile[*]}

another way to read the lines of 'test.txt' into an array

  i=1
  while read line
  do
      array[$i]="$line"
      echo "${array[$i]}"
      ((i++))
  done < test.txt

Initializing With Command Output ‹↑›

assign the lines of a text file to an array (1 line per element)

 IFS=$'\n' aa=($(ls -thor)); echo "4th element=${aa[3]}"

The command above is the simplest way to set the elements of an array to be the lines of the output of some command. But without the IFS variable being set, the elements of the array would be initialised to the 'words' (space separated) of the command output, not the lines. Notice how there is no semi-colon after the setting the IFS variable, this means that we only change the value of the variable for this one command and not subsequent commands.

we could also do this as follows but would have to reset IFS

 IFS=$'\n'; aa=($(ls -thor)); echo "4th element=${aa[3]}"

another technique to read lines into an array

  i=1
  ls -thor | while read line
  do
    array[$i]="$line";
    echo ${array[$i]}
    ((i++))
  done

read disk usage figures into an array, line by line

    IFS=$'\n' aa=($(du -sh *))
    echo "3th element=${aa[3]}"

another more complicated way to do the same thing

  du -sh * | while read line
  do
    aa[${#aa[@]}]="$line"
    echo "${aa[$((${#aa[*]}-1))]}"
  done

Copying Arrays ‹↑›

make a copy of array 'aa' in 'bb'

 aa=(1 2 3); bb=( "${aa[@]}" ); echo "${bb[*]}"

This technique may not work of the array 'aa' is 'sparse', which means that only some values have been set.

make a copy of an array.

 array2="${array1[@]}"

Adding an element to an array.

 aa= "${aa[@]}" "new element" )
 cc=( 1 2 ); cc[${#cc[*]}]="three"; echo ${cc[*]}

These fail for sparse arrays

Assigning Values ‹↑›

set the first element of the array 'list' to the text 'horse'

 list[0]=horse

set the 2nd element of the array 'list' to the text 'fast horse'

 list[1]="fast horse"   use quotes to allow value white-space

assign a list of files to an array

 list=( *.txt );  list is an array and not a string 

Modifying Arrays ‹↑›

create an array and replace all 's' characters with 'S' in all elements

 list=(*.txt); echo ${list[@]//s/S}

replace all 's' characters with 'S' in an array

 list=(*.txt); list=(${list[@]//s/S}); echo ${list[@]}

replace the first character of all array elements with an '.'

 list=(*.txt); list=(${list[@]/?/.}); echo ${list[@]}

Accessing Array Values ‹↑›

display the first element of an array variable 'list'

 echo ${list[0]}   not '$list[0]' which prints 'one[0]' 
 echo $list        this also displays only the 1st element 

display the 4th element of a bash array variable 'list'

 echo ${list[3]}

use string 'interpolation' to insert the value of an array in text

 echo "The Contents of area[5] is ${area[5]}."

display all the values of the array 'list'

 echo ${list[*]}
 echo ${list[@]}   there is some subtle difference, but what?

Array Length ‹↑›

display the number of elements in an array

 list=(one 2 three four);  echo ${#list[@]}

display the last element in an array

 a=(one 2 three four);  echo ${a[$((${#a[@]}-1))]}

display the oldest file in the current folder

 a=($(ls -t));  echo ${a[$((${#a[@]}-1))]}

Subsets Of Arrays ‹↑›

print the 2nd and remaining characters of the 1st element

 echo ${list:1} if list[0]=big this prints 'ig'

display the 2nd, 3rd and 4th elements of the array 'list'

 echo ${list[@]:1:3}

display the 2nd and all remaining elements of the array 'list'

 echo ${list[@]:1}

display the 3rd, 4th and 5th characters of element 1 of 'list'

 echo ${list[0]:2:4}   if list[0]=quick then prints 'ick' 
 echo ${list:2:4}      the same, [0] is assumed 

display the 2rd and 3rd characters of element 4 of 'list'

 echo ${list[3]:1:2}   if list[3]=water then prints 'at' 

bookmark folder for quick changing

 cdargs

Looping Through Arrays ‹↑›

loop through an array properly

   names=( "a dog" cat "a tree")
   for (( i = 0; i < ${#names[@]}; i++ ))
   do
     echo "element $i=${names[$i]}"
   done

note: theres no 'seq' on apple osx ! but you could use bash arithmetic expansion {1..20} etc

loop through an array properly, one element at a time, not by words

   aa=( "a dog" cat "a tree")
   for (( i = 0; i < ${#aa[@]}; i++ ))
   do
     echo "element $i=${aa[$i]}"
   done

getting a dynamic array element list, tricky but works

 aa=("a b" c "d e"); eval "echo {1..${#aa[@]}}";

a more tricky loop using bash globbing,

   aa=( "a dog" cat "a tree")
   for i in $(eval "echo {0..$((${#aa[@]}-1))}")
   do
     echo "element $i=${aa[$i]}"
   done

a more compact loop using braces instead of do/done

   aa=( "a dog" cat "a tree")
   for i in $(eval "echo {0..$((${#aa[@]}-1))}")
     { echo "$i=${aa[$i]}"; }

bash arithmetic, which prints 11

 a=12; a=$((a-1)); echo $a

loop through an array properly with seq, but this is untested

   aa=( "a dog" cat "a tree")
   for i in $(seq 0 $((${#aa[@]}-1))) 
   do
     echo "element $i=${names[$i]}"
   done

Looping Through Words Of Arrays ‹↑›

loop through all the words in the array elements

    list=( "a dog" cat "a tree")
    for f in ${list[@]}; do echo $f; done
    echo "list has ${#list[*]} elements"

loop through all the words of elements of an array

 aa=(*.txt); for f in ${aa[@]}; do echo $f; done

If the elements have spaces this may not be what you want

make an array from the disk usage and show the second element

 list=($(du -sh */)); echo ${list[1]}

More Arrays ‹↑›

load a file into an array 'aa' (but this is word by word, not by line)

 aa=(`cat "$filename"`)

define an array with 3 strings, some with spaces in them

 list=("a dog" cat "a tree"); echo ${list[@]}

use 'string interpolation' to show the 2nd element of and array

 list=("a dog" cat "a tree"); echo "2nd element=${list[1]}"

add an element to an array using 'eval' and a string

 aa=(one 2 three); eval "aa[3]='a frog'"; echo ${aa[3]}

assign an array element with spaces

 aa[2]="a frog"; echo ${aa[2]}

another 2 level evaluation technique for defining an array element

 aa=(one 2 three); eval $(echo "aa[3]='a frog'"); echo ${aa[3]}

what does this do

 aa=(*.txt); s="{${aa[@]}}"; s=${s// /,}"aa"; echo $s

Reading Line By Line ‹↑›

www: http://www.somebits.com/weblog/tech/bash-read-command.html
good hints about reading with read
www: http://codesnippets.joyent.com/posts/show/1899
another technique for reading lines into an array a knowledgeable post.
use 'mapfile' as another way to read into arrays
 ls -thor | mapfile aa; echo ${#aa[@]}

Using Jot For Example Data ‹↑›

make some random data, good ....

 jot

prints 21 decimal numbers increasing evenly from -1 to 1.

 jot 21 -1 1.00

generate the ascii character set, one character per line

 jot -c 128 0 | less

generate the strings 'xaa' through 'xaz'

 jot -w xa%c 26 a | less

generate 20 random 8-letter strings

 jot -r -c 160 a z | rs -g 0 8

generate an infinite number of yes's each on a new line

 jot -b yes 0

generate thirty 'ed' substitution commands applying to lines 2, 7, 12, etc

 jot -w %ds/old/new/ 30 2 - 5

The stuttering sequence 9, 9, 8, 8, 7, etc. can be produced

 jot - 9 0 -.5

create a file containing exactly 1024 bytes

 jot -b x 512 > block

set tabs four spaces apart starting at column 10, ending in column 132

 expand -`jot -s, - 10 132 4`

print all lines 80 characters or longer,

 grep $(jot -s "" -b . 80)

Paste ‹↑›

List the files in the current directory in three columns:

 ls | paste - - -     tab delimited
 ls | xargs -L3       space delimited

paste corresponding lines of 'f1' & 'f2' into 2 columns separated by ','

 jot -w tree 10 > f1; jot -w leaf 10 > f2; paste -d, f1 f2

show the sub-folders in a 3 column output separated by commas ','

 ls -d */ | paste -d',' - - -

Combine pairs of lines from a file into single lines:

 jot -w tree 20 > f; paste -s -d '\t\n' f

Combine pairs of lines from a file into comma delimited lines

 jot -w tree 20 > f; paste -s -d ',\n' f

The jot is just used to create some example data

Number the lines in a file, similar to 'nl'

 sed = myfile | paste -s -d '\t\n' - -

Create a colon-separated list of folders named bin, suitable for use in the PATH environment variable:

 find / -name bin -type d | paste -s -d : -

 s='one.two'; [[ $s == *.* ]] && echo "has a dot"

End Paste ‹↑›

Extended Globbing ‹↑›

'Globbing' is the process of expanding certain special characters (often called 'wildcards') into a list of files, or into a pattern to match against another string.

activate special wildcard 'globbing' in scripts

 shopt -s extglob

remove all files except ones with names ending in .jpg

 shopt -s extglob; rm !(*.jpg)

remove all files except *.jpg and *.gif and *.png

 shopt -s extglob; rm !(*.jpg|*.gif|*.png)

trim leading and trailing whitespace from a variable (needs extglob)

 x=${x##+([[:space:]])}; x=${x%%+([[:space:]])}

show visible the characters used to split strings in globbing

 echo "$IFS" | cat -vte

other text data tools
lam - join file lines (how is this different from 'paste'?)
join - a text relational database tool

delete the second field in a ':' delimited variable

 shopt -s extglob; a=one:two:3; a=${a/:+([^:])}; echo $a

set the ifs variable to only new lines

 IFS=$'\n'

The line about seems mysterious and cryptic, but has an enormous use in many situations. If you ever want to loop through the lines of some output, using 'for' or 'select' then you need to set the IFS variable first. This is because IFS determines how input will be split into 'tokens' when iterating through some text will 'for' etc. By default 'for' and 'select' will use IFS to split all output on all whitespace (not just new-lines)

 IFS=$'\n'; PS3="Enter a number: "; select f in $(ls -la);do vim $f; break; done
 PS3="Enter a number: "; select f in $(ls -la);do vim $f; break; done

String Substitution ‹↑›

see the 'extglob' shell option for better bash patterns

 If $sub matches at front end of $s, substitute $replacement for $sub
 ${s/#sub/replacement}

If $substring matches back end of $string, substitute $replacement for $substring.

 ${string/%substring/replacement}

Prefix Deletion And Substitution ‹↑›

The ${x#..} operator is equivalent to a regular expression such as /^.../

In bash patterns '?' is equivalent to /./ in a regular expression and '*' is equivalent to /.*/ With ${x#} '*' is not greedy but with ${x##} '*' is greedy

delete up to and including the 1st colon in a string variable

 s="one:two:3:four"; s=${s#?*:}; echo $s      prints 'two:3:four'
 s="one:two:3:four"; s=${s#[^:]*:}; echo $s   the same effect

delete up to and including the last colon in a string variable

 s="one:two:3:four"; s=${s##?*:}; echo $s      prints 'four'

 s="one:two:3:four"; s=${s#*:}; echo $s      prints 'four'

Suffix Deletion And Substitution ‹↑›

The ${x%...} operator is equivalent to a /...$/ regular expression pattern. In this context the '*' operator is not greedy

delete from the last ':' colon to the end of a string variable

 s="one:two:3:four"; s=${s%:?*}; echo $s     prints 'one:two:3'
 s="one:two:3:four"; s=${s%:[^:]*}; echo $s  the same effect

String Deletion And Substitution ‹↑›

Modern versions of Bash allow for some very capable string substitution. In some cases these techniques may replace what was formerly done with a 'sed' command.

Note the ${x/..} is a 'greedy' matching operator when used with the '*' wildcard character. It will match as much text as it can.

delete everything after the 1st colon character (not including ':')

 a=name:david; a=${a/:*/:}; echo $a       this prints 'name:'

delete everything after the 1st colon character (including the ':')

 a=name:ben; a=${a/:*}; echo $a      prints 'name'
 a=name:ben; a=${a/:*/}; echo $a     the same, but unnecessary

delete all fields except the 1st and last in a ':' colon delimited variable

 a=name:alex:john:fox; a=${a/:*:/:}; echo $a    prints 'name:fox'

 a=6:vim:42:emacs; a=${a/:[^:]*/:}; echo $a     prints 'name:fox'

 a=6:b:j:fox; a=${a/:[^:]:/:}; echo $a    prints 'name:fox'

delete everything except the 1st word in a bash variable

 a="why is this so"; a=${a/[ ]*/}; echo $a    prints 'why'
 a="why is this so"; a=${a/ *}; echo $a      the same, prints 'why'

delete all words in a bash variable except the 1st and the last

 a="why is this so"; a=${a/[ ]*[ ]/ }; echo $a    prints 'why so'
 a="why is this so"; a=${a/ * / }; echo $a        the same

delete everything after a '.' or a ':' (which ever comes first)

 a=google.net:8080; a=${a/[:.]*}; echo $a      prints 'google'
 a=google.net:8080; echo $a | sed 's/[.:].*//'  the same, with sed

delete everything after the first lowercase roman letter

 a=12A:boat; a=${a/[a-z]*}; echo $a          prints '12A:'

delete all text after the 1st roman letter in a bash variable

 a=12A:boat; a=${a/[a-zA-Z]*}; echo $a    prints '12'

www: http://www.hypexr.org/bash_tutorial.php#colors
some interesting notes about bash tools
suppress error messages
 ls 2>/dev/null

pipe the output of 2 commands (on separate lines) to 'less'

    (ls;
      pwd) | less

another multi-line grouping that works, note the final ';' semicolon

   { ls; 
     pwd; } | less

a function to search some book files for recipes or descriptions


  function sr {
    if [ -z "$1" ]; then
      echo "usage: $FUNCNAME <search-term> [<search-term>] [<search-term>]"
      echo " - searches the book files for recipes with the search term(s)"
      echo " - the search term(s) can be in the recipe or the recipe description"
      echo "   or multiple search terms "
      return 1;
    fi
    ( find ~/sf/htdocs/books -name '*-book.txt' | xargs grep -hA2 "^ *\*.*$1.*$2.*$3" ;
  find ~/sf/htdocs/books -name '*-book.txt' | xargs grep -hB2 "^ *>>.*$1.*$2.*$3") | less
  }

If Else ‹↑›

an if elif statement

    if something ; then
      do_stuff
    elif something_else ; then
      do_other_stuff
    elif [... ] ; then
      echo "ss" 
    else
      :
    fi

use a ':' to do nothing in an if else clause

   if [ -z "$x" ]; then
     :
   else
     einfo "Not"
   fi

A Big List Of File Tests ‹↑›

-a file Exists (use -e instead) -b file Exists and is a block special file -c file Exists and is a character special file -d file Exists and is a directory -e file Exists -f file Exists and is a regular file -g file Exists and is set-group-id -h file Exists and is a symbolic link -k file Exists and its sticky bit is set -p file Exists and is a named pipe (FIFO) -r file Exists and is readable -s file Exists and has a size greater than zero -t fd Descriptor fd is open and refers to a terminal -u file Exists and its set-user-id bit is set -w file Exists and is writable -x file Exists and is executable -O file Exists and is owned by the effective user id -G file Exists and is owned by the effective group id -L file Exists and is a symbolic link -S file Exists and is a socket -N file Exists and has been modified since it was last read

Substrings ‹↑›

remove the shortest match from the front of all array elements

    echo ${list[@]#f*r} 
    if $list was 'forr five afur' before then afterwards
    'r five a'

remove the longest match from front of all elements of 'list'

 echo ${list[@]##t*e}
 'list' before: trepef four atrapee
 'list' after:  f four a

remoe the shortest match from back of all elements

 echo ${list[@]%h*e}

remove the longest match from back of all elements of the array

 echo ${list[@]%%t*e}

Replacing Substrings In Array Elements ‹↑›

replace the first occurrence of 'fiv' with 'XYZ' in all elements

 echo ${list[@]/fiv/XYZ}

Replace all occurrences of 'iv' with 'YY' in all elements

 echo ${list[@]//iv/YY}

Delete all occurrences of 'fi' in all elements of the array

 echo ${list[@]//fi/}

replace occurrences of 'e' with the output of a function 'rep'

    rep() {
      echo -n "!!!"
    }
    #echo ${list[@]/%e/$(rep)}
    echo ${list[@]/e/$(rep)}

Calculating With Arrays ‹↑›

add the 2 values of an array and assign to another

 area[5]=`expr ${area[11]} + ${area[13]}`
 area[5]=$(expr ${area[11]} + ${area[13]})  the same?

Array Length ‹↑›

display the length of the array (or the index of last value)

 echo ${#list}
 echo ${#list[*]}    the same, but possibly very different

print the length of the string of the 1st element of array 'list'

 echo ${#list[0]}  if list[0]=tree this prints '4'

print the length of the string of the 3st element of array 'list'

 echo ${#list[2]}  if list[2]=big this prints '3'

Joining Arrays ‹↑›

join the elements of an array with ',' commas

 a=(a b c); s=$IFS; IFS=","; new="${a[*]}"; IFS=$s; echo $new

another way to join an array

    #/!bin/bash
    foo=('foo bar' 'foo baz' 'bar baz')
    bar=$(printf ",%s" "${foo[@]}"); bar=${bar:1}
    echo $bar

Functions ‹↑›

Function definitions can be placed in the ".bashrc" file in order create new commands for the bash shell. This is more flexible than using 'aliases' because functions may have parameters. Functions may also be used within scripts to make those scripts simpler or more readable.

Functions are simpler than creating a new command with a script placed in the path.

www: http://www.cyberciti.biz/faq/bash-shell-script-function-examples/
some good information
a function to add a function you've defined to .bashrc
  addfunction () { 
    [ -z "$1" ] && echo "usage: $FUNCNAME <func-name>" && return 1
    declare -f $1 >> ~/.bashrc ;
  }

show a usage mesage for the function if no parameters are given

  function test { 
   if [ -z "$1" ]; then
     echo "usage: $FUNCNAME <parameter>"
     echo " - performs a useful task"
   return 1;
   fi
   ... 
  }
  ,,,,

  * make a function called 'boo' using your last command
  >> boo () { !!; }

  * a function which prints its own name
  >> test () { echo "$FUNCNAME"; }
  
  * a function which prints its own name
  >> test () { [ -z "$1" ] && echo "$FUNCNAME: parameter needed" && return 1; }
  
  * show what each user defined function does
   function fn {
    IFS=$'\n';for f in $(declare -F); { declare -f ${f##* }; } | \
        sed -n '/^_/d;/() *$/s///p;/^ *echo "/{s//  /;s/";//p}' | grep -v 'usage' | less
 }

This recipe assumes that user bash functions are documented with 'echo "blah"' statements. The recipe above uses several tricks. Such as sed '/n/s///' which reused the matched pattern in a sed expresions. In other words

 sed '/x/s///';
is the same as
 sed '/x/s/x//';
Functions starting with '_' are usually defined by the programs, not the user.

Traps And Gotchas ‹↑›

dont use 'exit' to return from a function, use 'return' instead. dont use '$0' for the function name use '$FUNCNAME' instead

If an alias is reused as a function name, you may get strange incomprehensible errors.

Function Definition ‹↑›

show the definition for the bash function 'af'

 declare -f af

show help for the bash built-in command 'declare'

 help declare | less

show all defined functions which dont begin with a '_'

 declare -F | grep -v " -f _" | less

define a function with a parameter

 function s { echo $1; }
 s(){ echo $1; }           this is the same

a function may span 2 or more lines

   function x {
     echo 'hello'; }

a function with one parameter

    function say {
      echo $1
    }
    say hello  ##(this just prints 'hello' )

create a read-onlyfunction

 test(){ echo "Foo"; }; readonly -f test

a recursive function (dont run this, because it is infinite)

 foo(){ foo; }

Example Functions ‹↑›

a function to change into a folder and list its contents

 function cdls { cd $1; ls; }

Redefine the cd command's behavior using 'builtin'

 cd() { builtin cd "${@:-$HOME}" && ls; }

a function which prints a message and returns if no parameter

  test () { 
    [ -z "$1" ] && echo "usage: $0 file" && return 1
    cat $1
  }

a function which calculates the arguments (eg: mm 5*7)

 mm () { echo "$*" | bc -l; }

a function to concatenate input

 calc(){ awk "BEGIN{ print $* }" ;}

change directory up a number of levels

 function ..(){ for ((j=${1:-1},i=0;i<j;i++));do builtin cd ..;done;}

make a folder if it doesnt exist and change into it

 function mcd() { [ -n "$1" ] && mkdir -p "$@" && cd "$1"; }

make a function to search the software repositories

 function af { apt-cache search $@ | less; }

Search for the file and open in vi editor.

 vifind() { vi `find . -name "$1"` }

function to output an ascii character given its decimal equivalent

 chr () { printf \\$(($1/64*100+$1%64/8*10+$1%8)); }

Function Variables ‹↑›

make function variables local

 add (){ local x=$1; local y=$2; echo $(( $x + $y )); }

Doing Nothing At All ‹↑›

Despite appearances, doing nothing can be quite useful in fact even vital in certain circumstances.

do nothing for 5 seconds

 sleep 5
 sleep 5s   the same
 sleep 2 3  the same
 sleep 1 4  the same again

do nothing for half a second

 sleep 0.5  gnu sleep

do nothing for one hundredth of a second

 sleep 0.01

do nothing for 5 minutes

 sleep 5m

do nothing but be successful

 true

print 'y' continuously and successfully

 yes   this is not really doing nothing, but anyway

Environment Variables ‹↑›

When the user logs in to a bash shell, certain variables are automatically set.

extglob extended patterns
?(pattern-list) - Matches zero or one occurrence of the given patterns.
*(pattern-list) - Matches zero or more occurrences of the given patterns.
+(pattern-list) - Matches one or more occurrences of the given patterns.
@(pattern-list) - Matches one of the given patterns.
!(pattern-list) - Matches anything except one of the given patterns.

www: http://tldp.org/LDP/abs/html/internalvariables.html
a good guide to the bash internal variables
important bash environment variables
$HOME - the users home folder
$PATH - where executable programs are located
$PWD - the current working folder
$USER - the name of the user logged in
$HOST - the name of the local computer

show the values of the environment variables

 env
 printenv

It is possible to change the value of an environment variable for only the following command and not for any subsequent commands. This is done as follows

show what time and date it is in new york

 TZ=America/New_York date

Configuring The Bash Shell ‹↑›

Put configuration for the bash shell in '.bashrc' or '.bashprofile' New commands for bash can be created either with aliases, functions or scripts.

List bash functions defined in .bash_profile or .bashrc

 declare -F | sed 's/^declare -f //'

show the path split into lines

 echo $PATH | tr : \\n

change default shell for all users (freebsd ??)

 cd /usr/home && for i in *;do chsh -s bash $i;done

enable change directory using variable names

 shopt -s cdable_vars

The Prompt ‹↑›

Display screen window number in the bash prompt

 [[ "$WINDOW" ]] && PS1="\u@\h:\w[$WINDOW]\$ "

display a prompt with colours

 export PS1='\[\033[0;35m\]\h\[\033[0;33m\] \w\[\033[00m\]: '

Aliases ‹↑›

Aliases are useful for abbreviating common commands . To save an alias put it in the ~/.bashrc file so that it will always be available. The limitation of aliases are that they cannot use command line parameters. To overcome this use 'functions' instead.

make all searches withing the 'less' pager case-insensitive

 alias less='less -i'

make an alias: a new command 'dir' which calls the "ls" command

 alias dir='ls -la | less'  aliases cant use parameters, such as $1

up - aliaes for moving up the directory tree

 alias up="cd .."; alias upp="cd ../.."; alias uppp="cd ../../..";

make some convenient aliases to edit and reload .bashrc and aliases

 alias aliases='vim ~/.bash_aliases; source ~/.bash_aliases'
 alias bashrc='vim ~/.bashrc; source ~/.bashrc'

Define an alias with a correct automatic completion

 old='apt-get'; new="su-${old}"; command="sudo ${old}"; alias "${new}=${command}"; $( complete | sed -n "s/${old}$/${new}/p" ); alias ${new}; complete -p ${new}

escape any command aliases

 alias v='less'; \v   the alias is not executed

Miscellaneous ‹↑›

pretty print a java code 'Co.java' 2 columns, landscape, gaudy header

 enscript -G2rE -b"Assignment 2" -p new.ps Co.java

switch virtual consoles

 alt+Fn, eg: alt+F1 to go to the first virtual terminal

add a folder to the executable path (put in ~/.bash_profile file to save it)

 export PATH=/path/to/folder:"${PATH}"
programs in this folder can then be executed with 'programname'

The Command Line ‹↑›

Command Line History ‹↑›

www: http://www.deer-run.com/~hal/UnixCommandLineKungFu.pdf
good tips for the command line
In Unix and Linux commands previously executed at the console are stored in a command history and are accessible and manipulable in a variety of ways.

Copy history from one terminal to another

 history -w <switch to another terminal> history -r

Getting Help For The History ‹↑›

view documentation about the history file

 man bash  then type '/history' to search the documentation

view the bash help for the 'history' command

 help history

Viewing The Command History ‹↑›

view the command line history

 history  this show the current sessions command history

view the last 10 commands of the command history

 history 10
 fc -nl -10  ?
 history | tail -10   the same

view recent commands without numbers

 fc -nl

list the last 100 commands in reverse order with no numbers

 fc -rnl -100

Searching The Command History ‹↑›

search the command history

 [control] + r
 /   the same if the command line is in 'vi' mode

Insert a comment on command line as a searchable reminder

 ls -alh #mycomment

display the top 10 commands from the command history

 history | awk '{print $2}' | awk 'BEGIN {FS="|"}{print $1}' | sort | uniq -c | sort -n | tail | sort -nr

Executing From The History ‹↑›

execute the 47th command in the command history

 !47

substitute 'l' with 's' in the last command and execute it.

 ^l^s

remove 'ls' from the last command

 ^ls

build up command pipelines with !!

 indent | !!     the previous command replaces '!!'

apply a 'less' pager command to a long text file path

 cat /long/path/name/document.txt
 !! | less

run the previous command with root privileges

 vim /etc/vim/vimrc
 sudo !!     this executes 'sudo vim /etc/vim/vimrc'

use !$ to get the argument from the last command

 cat file.txt
 vi !$           executes 'vi file.txt'

!$ - The last argument to the previous command

 svn status app/models/foo.rb; svn commit -m "Changed file" !$

get the command argument from the second last command

 vi !-2$

get all arguments from the previous command

 cat a.txt b.txt c.txt
 vi !*        executes 'vi a.txt b.txt c.txt'

print the last command which begins with the letter 'h'

 !h:p       ':p' stops the command from being executed

print the last command which contains the string 'hi'

 !?hi?:p

print the 507th command with any filename extension removed

 !507:r:p

print the 512nd command with a 'c' replaced by 'C'

 !512:s/c/C/:p
 !512:s/c/C/    actually execute the command
 !512:s/c/C     the same
 !512:s,c,C     the same
 !512:gs/c/C/   replace all occurrences of 'c' with 'C'

edit the last 7 commands in a text editor (determined by $FCEDIT)

 fc -6 0    watch out, the commands are executed after editing!

after fc, to not execute the commands, clear the editor buffer

 :bd    in vim, assuming that $FCEDIT is set to 'vim' or 'vi'

Runs previous bash command replacing 'tree' with 'leaf'

 ^tree^leaf

Do a command but skip recording it in the bash command history

 _cd ~/nsfw; mplayer midget_donkey.mpeg

Configuring The Command History ‹↑›

make sure that new commands are appended to the history file

 shopt -s histappend

see in which file your command history is stored

 echo $HISTFILE  this normally show '~/.bash_history'

view your (or the current users) history file

 less ~/.bash_history
 less $HISTFILE    technically better

see how many commands will be saved to your history file

 echo $HISTSIZE

view the maximum number of commands saveable to your history file

 echo $HISTFILESIZE

see how many commands are currently in your history file

 wc -l $HISTFILE

get rid of duplicate entries in the command history file

 sort -u $HISTFILE -o $HISTFILE

make the history file 500000 lines and write 100000 commands each time

 export HISTFILESIZE=500000
 export HISTSIZE=100000

put these values in '~/.bashrc' so they will be permanent

include timestamps for each command in the history file

 HISTTIMEFORMAT=yes

erase duplicates for the history file

 export HISTCONTROL="erasedups:ignoreboth"

dont put the 'exit' command in the history file

 export HISTIGNORE="&:[ ]*:exit"

Deleting The Command History ‹↑›

clear the command history for the current session

 history -c

delete the command history item 100

 history -d 100

Vim And Bash ‹↑›

It is possible to execute bash commands from within the vim text editor using a variety of techniques. It is also possible to 'filter' parts of a document using bash commands or send (pipe) part or all of a text document from vim to a bash command.

execute the bash 'date' command from within vim

 :!date

execute the current line as a bash command (doesnt insert results)

 :.w !bash

remove the comment character and execute current line as a bash command

 :.w !sed 's/^\#//' | bash

a command to execute a fragment between '---' and ',,,' with bash

 command! Bmr ?^ *---?+1,/^ *,,,/-1w !sed 's/^ *\#//' | bash | less

But the command above exits immediately even if the bash commands should read input from the user.

a vim command 'Bashr' to execute the current line in bash

 command! Bashr .w !sed 's/^ *\#//' | bash | less

a vim mapping to execute a bash command line in a document

 map ,bm :.w !sed 's/^ *\#//' \| bash

a vim command to make and show a screen shot of a bash command

 command! Bci .w !sed 's/^ *\#//' | bash; scrot -ud1 test.png; feh test.png

If we are editing a vim document which has a bash command such as "tree | head -20" on a line by itself, then we can execute and create a screenshot of the executed command using the vim command which we have just made above. We can do this by executing the command ":Bci" within vim

a vim command to make and show a screen shot saving to a given file name ??

 command! -nargs=1 Bci .w !sed 's/^ *\#//' | bash; scrot -o<args> -ud1 test.png; feh test.png

take a delayed screen-shot of the command on the current line

 :.w !sed 's/^/scrot -ubd1 test.png \& clear; /' | bash; sleep 1.5; feh test.png

If the line which the cursor is on in the current document contains the text 'top', then executing the above line will run that command in a bash shell and take a screen-shot of its output. The screen-shot will then be displayed

a command 'Scr' to take a delayed screen-shot and display it

 command! Scr .w !sed 's/^ *\#?/scrot -ubd2 test.png \& clear; /' | bash; sleep 2.5; feh test.png

If the cursor is positioned on a line with 'apropos sound | less' then we can take a screen-shot of the output of that command with ':Scr'

See the vim 'system' function for more advanced solutions

view the help for the vim built function 'system()'

 :h system

assign to the variable 'a' the output of the bash command 'ls'

 :let a=system("ls")

Screen Shots Of Bash ‹↑›

Sometime it is nice to make a picture of a bash terminal in order to try to impress somebody or for other reasons

create a screen shot of the 'top' command

 scrot -ud1 test.png & clear; top

make and display a screen shot of a zenity list box in 'sh.png'

 scrot -ubd2 sh.png & ls | zenity --list --column="test"; feh sh.png

more advanced bash variables
$IFS - determines how strings are split

Editing The Command Line ‹↑›

Enable programmable bash completion in debian linux

 aptitude install bash-completion ; source /etc/bash_completion

display all commandline editing keystrokes that use [ctrl]

 bind -p | grep -F "\C"

show what 'short-cut' keys are available on the command line

 bind -P

Go to end of current command line

 <control e>

Go to begin of current command line

 CTRL + a

set the command line editing mode to 'vi'

 set -o vi      the command line now seems like the 'vi' text editor

set the command line editing mode to 'emacs'

 set -o emacs   this is the default mode

view the shell edit mode variables

 set -o

once in 'vi' edit mode, to edit the command line in $EDITOR text editor

 [esc] + v    if $EDITOR is 'vim' then vim will open

enable vi mode line reading in a shell script and store line in 'a'

 set -o vi; read -e a;   the -e switch uses the 'readline' function

change the default text editor for the system

 export EDITOR=vim    put in .bash_profile

make the vi editing mode the default

 set editing-mode vi    put this in the '.inputrc' file
type 'man readline' for more information

Bash command line editing is determined by the 'readline' program Other programs, like 'gnu-plot' also use the same client. Put 'set editing-mode vi' in .inputrc to automatically use the 'vi' editing mode for all readline clients./

Encantations ‹↑›

insert the file name of each html file at the top of the file with no extension

 for f in *.html; do i=${f/.html/}; sed -i.bak "1s/^/$f/" $f; done

Perl Versus Bash ‹↑›

Perl may be much faster than bash if bash is doing looping constructs, otherwise not.

loop through input with perl

  my $state = 0;
  while(<>) {
    exit if /TIMESTAMP2/;
    print $_ if $state == 1;
    $state = 1 if /TIMESTAMP1/;
  }

Fu Bash Tips ‹↑›

Emulate perl 'print "#" x 20, "\n"'

 printf '%*s\n' 20 | tr ' ' '#'

Getting the last argument from the previous command

 cd !$

Diff on two variables

 diff <(echo "$a") <(echo "$b")

Get the last string of previous command with !$

 $mkdir mydir -> mv !$ yourdir -> $cd !$

Perform a branching conditional

 true && { echo success;} || { echo failed; }

Runs previous command replacing foo by bar every time that foo occurs

 !!:gs/foo/bar

Check command history, but avoid running it

 !whatever:p

Clear mistyped passwords from password prompt

 ^u

invoke an editor to write a long, complex, or tricky command line

 control-x e

Salvage a borked terminal

 reset

Logging Out ‹↑›

execute a command on logout

 trap cmd 0

close shell keeping all subprocesses running

 disown -a && exit

Colours ‹↑›

make folder listings always colourized

 alias ls='ls --color=always'

Colours on the screen can be obtained using Ansi escape sequences and the 'echo' command

display a red 'R' and a yellow 'Y' on the screen

 echo -e "\033[1;31mR\033[1;33mY"

display hi in all the colours

 echo -e "\x1B["{0..1}";3"{0..7}"m hi"

The line above uses bash exterpolation to generate a set of strings.

display hello in red (uses a hex code and no bold)

 RED="\x1B[31m"; echo -e "$RED hello"

show redish text

 echo -e "\033[47m\033[1;31m  Light Red on a grey background"
 echo -e "\033[1;31m  Light Red "

Remove color codes (special characters) with sed

 sed -r "s/\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[m|K]//g"

make a set of colour variables to use with 'echo -e'

    BLACK="\x1B[0;30m";
    GREY="\x1B[1;30m";
    RED="\x1B[0;31m";
    BRED="\x1B[1;31m"
    GREEN="\x1B[0;32m"
    BGREEN="\x1B[1;32m"
    YELLOW="\x1B[0;33m"
    BYELLOW="\x1B[1;33m"
    BLUE="\x1B[0;34m"
    BBLUE="\x1B[1;34m"
    MAGENTA="\x1B[0;35m"
    BMAGENTA="\x1B[1;35m"
    CYAN="\x1B[0;36m"
    BCYAN="\x1B[1;36m"
    WHITE="\x1B[0;37m"
    BWHITE="\x1B[1;37m"
    NORMAL="\x1B[0m"

display some colours

  echo -e "
     $BLACK BLACK $NORMAL x1B[0;30m 
     $GREY GREY $NORMAL x1B[1;30m 
     $RED RED $NORMAL x1B[0;31m 
     $BRED BOLD RED $NORMAL x1B[1;31m 
     $GREEN GREEN $NORMAL x1B[0;32m
     $BGREEN BOLD GREEN $NORMAL x1B[1;32m
     YELLOW=x1B[0;33m
     BYELLOW=x1B[1;33m
     BLUE=x1B[0;34m
     BBLUE=x1B[1;34m
     MAGENTA=x1B[0;35m
     BMAGENTA=x1B[1;35m
     CYAN=x1B[0;36m
     BCYAN=x1B[1;36m
     WHITE=x1B[0;37m
     BWHITE=x1B[1;37m
     NORMAL=x1B[0m
  "

scrot options
-u - take a screen shot of the focussed window
-d2 - delay for 2 seconds before capturing the image
-b - when capturing a window get the window frame as well

colored prompt

 export PS1='\[\033[0;35m\]\h\[\033[0;33m\] \w\[\033[00m\]: '

Unicode Characters ‹↑›

print some unicode characters

 /usr/bin/printf '\u201cdouble\u201D'

The '/usr/bin/printf' is necessary because otherwise the bash built-in printf command is used which doesnt print unicode

install the enjoyable 'unicode' command line program on a debian box

 sudo apt-get install unicode

show all unicode characters relating to chess

 unicode chess

Ansi Escape Sequences ‹↑›

save the current cursor position

 echo -n -e "\033[s"

restore the cursor to its former position

 echo -n -e "\033[u"

doesnt seem to work

    # 2 is green  
    COLOR=`tput setaf 2; tput smso`
    # back to normal screen colors
    NORMAL=`tput sgr0`
    # print the time-date output on the above position
    echo -n $COLOR$cmd$NORMAL

Knowledgable Bash People ‹↑›

"foonly", sydney: http://forums.whirlpool.net.au/user/44690 hoyhoy at stackoverflow.com terson at stackoverflow.com amarillion at stackoverflow.com

Terminals ‹↑›

The term 'terminal' is very ambiguous. I cant supply a good definition at this stage. But it has to do with similar concepts of the console, command-line, shell etc.

a list of colour escape codes
Black 0;30
Dark Gray 1;30
Red 0;31
Bold Red 1;31
Green 0;32
Bold Green 1;32
Yellow 0;33
Bold Yellow 1;33
Blue 0;34
Bold Blue 1;34
Purple 0;35
Bold Purple 1;35
Cyan 0;36
Bold Cyan 1;36
Light Gray 0;37
White 1;37
www: line
wrapping When the text of a line is too long for the computers screen then the line may be broken or 'wrapped' so that the extra text is displayed on the next line in the computer screen. This is called 'line wrapping' or just 'wrapping' @@
clear the screen
 clear

Change the window title of the xterm terminal to 'KungFu'

 echo "^[]0;KungFu^G"

make the text on the screen 'bold' (brighter and thicker than normal)

 tput bold

use tput to save, clear and restore the terminal contents

 tput smcup; echo "Doing some things..."; sleep 2; tput rmcup

The tput command has many options and switches

determine number of lines in terminal

 l=100; while [ $l -gt 1 ]; do echo $l; l=$(($l-1)); done

Change the terminal title to the last shell command executed

 trap 'echo -e "\e]0;$BASH_COMMAND\007"' DEBUG

Start a gnome terminal with three open tabs

 gnome-terminal --tab --tab --tab

print the output of a script on a different terminal

 script -f /dev/pts/3
 script /dev/null | tee /dev/pts/3   another way to do it

stop text lines from wrapping in your terminal

 function nowrap { export COLS=`tput cols` ; cut -c-$COLS ; unset COLS ; }

Using the urxvt terminal daemon

 urxvtd -q -o -f

terminal jargon

change title of terminal window to verbose info useful at login

 echo -ne "\033]0;`id -un`:`id -gn`@`hostname||uname -n|sed 1q` `who -m|sed -e "s%^.* \(pts/[0-9]*\).*(\(.*\))%[\1] (\2)%g"` [`uptime|sed -e "s/.*: \([^,]*\).*/\1/" -e "s/ //g"` / `ps aux|wc -l`]\007"

Changing the terminal title to the last shell command

terminal tools
terminfo - a database about the terminal screen
tput - a utility for the terminal

Resetting The Terminal ‹↑›

Salvage a borked terminal

 echo <ctrl-v><esc>c<enter>

Salvage a borked terminal

 <ctrl+j>stty sane<ctrl+j>

Busybox ‹↑›

Busybox is a minimal set of commands including a version of the bash shell designed to be used on 'small' computers (having limited resources, such as memory, disk storage etc)

Curiosities ‹↑›

implement a daemon using bash

 echo "Starting Daemon"; ( while :; do sleep 15; echo "I am still running =]"; done ) & disown -h -ar $!

A 'daemon' is a program which runs in the background waiting to provide some service or respond to some situation.

Search back through previous commands

 Ctrl-R <search-text>

turn on/off keyboard leds via commandline

 xset led 3

Let your computer lull you to sleep

 echo {1..199}" sheep," | espeak -v english -s 80

Change newlines to space in a file by using echo

 echo $(</tmp/foo)

convert mouse movements into a type of noise (not on BSD)

   while true; do 
     beep -l66 -f`head -c2 /dev/input/mice|hexdump -d|awk 'NR==1{print $2%10000}'`;
   done

a smiley face bash prompt

 PS1="\`if [ \$? = 0 ]; then echo \e[33\;40m\\\^\\\_\\\^\e[0m; else echo \e[36\;40m\\\-\e[0m\\\_\e[36\;40m\\\-\e[0m; fi\` \u \w:\h)"

BASH 4.0 TOPICS

Bash 4.0 is said to be introducing recursive globbing with the '**' operator.

change to the home ($HOME) folder in zsh or the bash v4 shell

 ~

arithmetic series with a 'step' value in bash version 4

 echo {1..8..2}

Things To Investigate ‹↑›

teminator xclip The cli apps site launchy for lauching scripts sam2p image processing SIR simple image resizer webpack for reducing websites pngout /etc/ppp/ip-up for dns makeself httrack

Utas ‹↑›

on apple: <apple-k> smb://alacritas

mount a samba share from mac osx at the university of tasmania

 mount -t smbfs //user@alacritas/user  smb

Unix Culture ‹↑›

Unix embodies a particular type of culture. Here are some of the things it encompasses.

 [[ "x$TERM" == "xrxvt" || "x$XTERM_VERSION" == xXTerm* || "x$COLORTERM" == 'gnome-terminal' && "x$SHELL" == */bin/zsh ]] && preexec () { print -Pn "\e]0;$1\a" }

DOCUMENT-NOTES:

unix culture
use folders for what their designed for put temp stuff in /tmp
lots of pipes
short commands