bumble.sf.net language and parsing

plain text

The Linux Operating System

-----------------------------:

Quote: "I'm doing a (free) operating system, just a hobby,..." (Linus Torvalds)

This booklet is designed to help with common tasks on a Linux system. It is designed to be presentable as a series of "recipes" for accomplishing common tasks. These recipes consist of a plain English one-line description, followed by the Linux command which carries out the task. The document is focused on performing tasks in Linux using the 'command line' or 'console'. The format of the booklet was largely inspired by the "Linux Cookbook" www.dsl.org/cookbook

@@ http://github.com/himanshuc/nixhacker/tree/master a good list of resources @@ http://www.pixelbeat. href="org/cmdline.html">org/cmdline.html some command lines recipes @@ http://www.shell-fu.org/ some command line recipes @@ http://www.commandlinefu.com/ a very good site with lots of command-line tips @@ http://dsl.org/cookbook/cookbook_toc.html A very good Linux User "cookbook" and the primary inspiration for the booklets on this site.

Good books


@@ The UNIX Environment (Andrew Walker, Wiley 1984) @@ The Linux Cookbook (Michael Stutz, No Starch Press) @@ The Unix Programming Environment (Kernighan et al)

Online books


....

@@ http://www.catb.org/~esr/writings/taoup/html/ @@ http://www.faqs.org/docs/artu/ a philosophical book about the unix operating system by Eric Raymond (2003) @@ http://www.linfo. href="org/onlinebooks.html">org/onlinebooks.html a list of online linux books

Getting help


The traditional Unix help system is called 'man' or 'manual' pages. And they can be good. It is one of the ironies and frustrations of Unix that a man page only really becomes helpful and interesting once one already knows what a program does and how to basically use it.

show the short help description for all programs

 whatis -r '.*'
 for i in $(ls $(echo $PATH | tr ':' ' ')); do whatis $i; done | less

search for all programs which have something to do with 'java'

 whatis -r '.*' | grep java
 whatis -r 'java' 
 whatis -r java 

view the 'manual' page for the wc (word/line/character count) command

 man wc  

Sadly, 'man' pages are often written in a cryptic way and are short on examples. The examples, if there are any, are almost always right at the end of the man page.

view the manual page in 'section' 4. See the list of sections, elsewhere

 man 4 command    

show what software is available to install, relating to 'page layout'

 apt-cache search "page layout"
 apt-cache search "page layout" | grep -v '^lib'  ##(exclude code libraries))

Search all man pages for the word 'dig'

 man -k dig  ##(does this search just the short description??)

find documentation for latex packages in pdf format

 find /usr/share/doc/ -name '*.pdf'    ##(on a debian system, at least)

An introduction to the linux command line


Unix culture shock


....

Those users who have no experience with a Unix-style operating system, but are familiar with the Microsoft Windows operating system will experience a kind of 'culture shock' when they begin to use Linux. This is true even if the user is familiar with using the 'command-line' on a Microsoft computer. This section attempts to point out some of the things which may seem baffling and illogical to the new Linux user.

@@ You cant just download an 'exe' and click on it to run it However you can easily install new software with 'sudo apt-get install ' @@ file names dont have to have extensions (such as .txt), but can To an experienced Microsoft command line user, the idea of just calling a text file 'report' rather than 'report.txt' is rather disorientating. "How will the computer know what type of file it is?" The microsoft user thinks. File names can and do begin with a dot '.' File names can begin with a tilde '~' @@ Linux commands are very short and cryptic Why is the 'list files' command, called 'ls' and not 'list'? It would be so much more memorable as 'list' than 'ls'. This is the "Unix way". If you like typing you can do alias list='ls'

@@ The folder hierarchy in Linux seems very cryptic and you cant just put files anywhere

Basic user commands


....

log in to the system with a username of 'bob'

 ubuntu login: bob 

log out of the system

 logout 

switch to the fourth virtual console

 press [ALT]-[F4]

switch from the fourth to the third virtual console, press:

 [ALT]-[<-]

switch from X to the first virtual console, press:

 [CTRL]-[ALT]-[F1]

run the hostname tool to find the name of the computer

 hostname 

output the version of the hostname tool

 hostname --version 

run hostname and specifying that the file 'host.info' be read from

 hostname -F host.info 

change your password

 passwd 

output your username

 whoami 

see who is currently logged in

 who 

see who is currently logged in and what they are doing

 w 

display information about recent use of the system

 last | less

find out when the user 'mjb' last logged in

 last mjb 

Note


: The last tool gets its data from the system file '/var/log/wtmp'; the last line of output tells how far this file goes back.

Basic process commands


....

list the processes in your current shell session

 ps 

list all the processes that user 'hst' has running on the system

 ps -u hst 

This command is useful for listing all of your own processes

list all of the processes and give their user-names

 ps aux  ##(there could be a lot of output, even on single user systems)

display a continually updated display of the current system processes

 top 

list all the processes containing a reference to an 'sbin' directory

 ps aux | grep sbin 

list any processes whose process IDs contain a 13 in them

 ps aux | grep 13 

list the process whose PID is 344

 ps -p 344 

Basic software commands


....

Run skype using your GTK theme

 skype --disable-cleanlooks -style GTK

output a list of programs that pertain to consoles

 apropos consoles 

output a list of all tools whose pages in the system manual contain a reference to consoles

 man -k consoles 

list all of the software packages installed on the system

 dpkg -l 

list all of the packages whose name or description contains the text "edit," regardless of case

 dpkg -l | grep -i edit 

peruse descriptions of the packages that are available

 less /var/lib/dpkg/available 

get a description of the who tool

 whatis who 

view the manual page for w

 man w 

view all of the Info manuals on the system

 info 

read the Info documentation for the tar tool

 info tar 

This command opens a copy of The GNU tar Manual in info. To read the contents of a file written in Info format, give the name of

read 'faq.info', an Info file in the current directory

 info -f faq.info 

read 'faq.info', an Info file in the current directory, beginning with the node Text

 info -n 'Text' -f faq.info 

view the HTML version of the Debian FAQ in the lynx Web browser

 lynx /usr/share/doc/debian/FAQ/debian-faq.html 

The above only works on a Debian flavour of Linux.

view the compressed text version of the Debian FAQ in zless,

 zless /usr/doc/debian/FAQ/debian-faq.txt.gz 

Basic command line usage


....

repeat the last command entered

 [^|] 

The [^|] key moves the last command you typed back to the input line, and executes it.

retrieve the last command you entered with the string 'grep' in it

    ##(press the  key and the 'r' key)
(reverse-i-search)'': grep

put the 3rd-last command you entered with 'grep' in it on the input line

 [Control]-r
(reverse-i-search)'': grep

clear the screen and then log out of the system

 clear; logout 

run the hostname command twice times

 hostname; hostname 

redirect standard input for apropos to file 'keywords'

 apropos < keywords 

redirect standard output of a command to the file 'commands'

 apropos shell bash > commands 

append the standard output of apropos shells to the file 'commands'

 apropos shells >> commands 

redirect the standard error of apropos shell bash to 'command.error'

 apropos shell bash 2> command.error 

perform a long task in the background, saving all messages to 'img.txt'

 find / | xargs file | grep image &>~/img.txt &

In the command above, both error messages (2>) and all normal output of the command will be redirected to the 'img.txt' text file in the users home folder.

append the error output of a command to an existing file 'command.error'

 apropos shells 2>> command.error 

redirect the standard output and standard error to the file 'commands'

 apropos shells &> commands 

view the output of "apropos bash shell shells" in a 'pager' program

 apropos bash shell shells | less 

A pager program allows you to view output from a program one (screen) page at a time. (pagers: more/less/most)

run the command apropos shell > shell-commands as a background job

 apropos shell > shell-commands & 

run job 4 in the background

 bg %4 

Trivia: running a job in the background is sometimes called "backgrounding" or "amping off" a job.

bring the most recent background job to the foreground

 fg 

bring job 3 to the foreground

 fg %3 

list your jobs

 jobs 

kill job number 2

 kill %2 

to interrupt a running command use [control] c

 find / -name '*e*'
 [control] c

search your command history for the text 'apropos'

 history | grep apropos 

specify the second-to-the-last command in your history

 [^|] [^|]

Trivia: '!', the exclamation mark is sometimes called "bang"

run history event number 1 (the last command executed)

 !1 

create a script of a shell session and save it to the file 'log.1'

 script log.1 

Simple file commands


....

create the file 'new.txt' in the current directory

 touch new.txt 

create the file 'new' in the 'work/docs' subdirectory of the current directory

 touch work/docs/new

On a Unix-style computer, such as Apple OSX or Linux, files do not have to have 'extensions' to their names, unlike on Microsoft Windows.

make a new directory called 'work' in the current working directory

 mkdir work 

create the 'work/completed/2001' directory

 mkdir -p work/completed/2001 
 mkdir --parents work/completed/2001    ##(the same)

If the 'work' and 'completed' folders do not exist, then they will be created.

Simple folder commands


....

change the current working directory to '/usr/doc'

 cd /usr/share/doc 

return to the directory you were last in

 cd - 

determine what the current working directory is

 pwd 

list the contents of 'work', a subdirectory in the current directory

 ls work 

list the contents of the '/usr/doc' directory

 ls /usr/doc 

list the contents of the directory so that directories and executables are distinguished from other files

 ls -F 

output a verbose listing of the '/usr/doc/bash' directory

 ls -l /usr/doc/bash 

output a recursive listing of the current folder and sub-folders

 ls -R 

list all of the files on the system

 ls -R / 

list the files in the 'log' directory sorted with newest first

 ls -t /var/log 

The '/var/log' folder usually contains 'log' files which record the activities of software running on the computer. Type 'man hier' for more information about the Linux folder structure.

list all files in the current directory (including 'hidden' files)

 ls -a 

important 'ls' options .. -a - display all files, including hidden ones .. -F - display the file types of files .. -R - list all folders and subfolders .. -t - sort the displayed files by date ..

output a tree graph of your home directory and all its subdirectories

 tree ~       ##(this shows files as well as folders)

show a just the start of a folder tree for the home folder

 tree -d /usr/local | head -20    

[>>0.3 image/eg-tree.png]

peruse a tree graph of the '/usr/local' directory tree

 tree -d /usr/local | less    

Copying files


....

copy the file 'old' to the file `new'

 cp old new 

copy files preserving the file attributes

 cp -p file.txt new-copy.txt

copy the folder 'reports' verbosely (showing what is being done)

 cp -vr reports ~/new/

copy a folder tree verbosely, but only copy files which are newer

 cp -vur docs/office ~/docs

copy the folder 'public_html', and subfolders, to 'private_html'

 cp -R public_html private_html 

The cp '-R' option doesnt copy symbolic links. But the "man" page for cp states that -r and -R are equivalent

make an archive copy of the directory tree 'public' to 'private'

 cp -a public_html private_html 

Moving files


....

move the file 'notes' in the current working directory to '../play'

 mv notes ../play 

move the file '/usr/tmp/notes' to the current working directory,

 mv /usr/tmp/notes . 

This command moves the file '/usr/tmp/notes' to the current working

move the directory 'work' in the current working directory to 'play'

 mv work play 

rename the file 'notes' to `notes.old'

 mv notes notes.old 

Note


: to rename multiple files use 'rename'

Deleting files


....

remove the file 'notes' in the current working directory

 rm notes 

remove the directory 'waste' and all of its contents

 rm -R waste 

remove the directory 'empty'

 rmdir empty 

use tab completion to remove the file 'No Way' in the current directory

 rm No[TAB] Way 

delete the file '^Acat' in a directory that also contains the files 'cat' and `dog'

 rm -i ?cat   ##(rm: remove '^Acat'? y )

remove the file '-cat' from the current directory

 rm -- -cat 

Symbolic links


....

create a hard link from 'seattle' to `emerald-city'

 ln seattle emerald-city 

create a symbolic link from 'seattle' to `emerald-city'

 ln -s seattle emerald-city 

list all files in the '/usr/bin' directory that have the text 'tex' anywhere in their name

 ls /usr/bin/*tex* 

copy all files whose names end with '.txt' to the `doc' subdirectory

 cp *.txt doc 

output a verbose listing of all files whose names end with either a '.txt' or '.text' extension, sorting the list so that newer files are listed first

 ls -lt *.txt *.text 

remove all files in the current working directory that begin with a hyphen and have the text 'out' somewhere else in their file name

 rm -- -*out* 

join text files whose names have an 'a' followed by 2 or more characters

 cat a??* 

The original files are unchanged, but the joined together text files are displayed on the screen.

The bash prompt


....

change your shell prompt to 'Your wish is my command: '

 PS1='Your wish is my command: ' 

change your prompt to the default bash prompt (the current folder)

 PS1='\w $ ' 

change the prompt to the current date, space, the hostname, and a '>'

 PS1='\d (\h)>' 

clear the screen every time you log out,

  clear     ##(put this in the file '.bash_logout')

Installing software


search for games software which are available to install

 apt-cache search game 

update the repository cache

 sudo apt-get update 

Add new repositories to the file '/etc/apt/sources.list'

Sound


A linux system is capable of playing, recording and editing sound (audio) files with a variety of open-source software. This section only provides what should be a very succint overview of the most important sound tasks with linux. For more detailed information, please consult the booklet listed below.

@@ http://bumble.sourceforge.net/books/linux-sound/ A more comprehensive introduction to using audio with the linux operating system.

Synthesize text as speech

 echo "hello world " | festival --tts

find the duration of the audio file 's.wav' in hours/minutes/seconds

 soxi -d s.wav
 soxi s.wav | grep -i duration   ##(the same)

Recording audio


....

record a 'wav' file from the microphone, saving it to 'hello.wav'

 rec hello.wav   

This begins an 8,000 Hz, monaural 8-bit WAV recording, which is not very good quality.

make a high-fidelity recording from the mic and save it to 'goodbye.wav'

 rec -s w -c 2 -r 44100 goodbye.wav 
 rec -s w -c 2 -r 44100 goodbye.wav 

Playing audio


....

play the MP3 stream at the url

 mpg321 http://example.net/broadcast/live.mp3 

Converting audio file formats


....

It is a common task to need to convert sound files from one format to another.

translate an audio file in Sun AU format to a Microsoft WAV file,

 sox recital.au recital.wav

convert 'sound.mp3' into a wav file 'new.wav' (a new file is created)

 mpg321 -w new.wav old.mp3   ##(the file 'old.mp3' is unchanged)
 mpg123 -w new.wav old.mp3   ##(the same) 

encode an MP3 file from a WAV file called 'september-wind.wav'

 lame september-wind.wav september-wind.mp3 

Editing sound


....

tools for editing sound .. audacity - a good graphical sound editor .. sox - a command line audio editor ..

join the audio files 'a.wav' and 'b.wav' together and save as 'new.wav'

 sox a.wav b.wav new.wav

Converting audio files


....

Translation


translation tools .. youtranslate - uses web services such as google ..

Unicode


Find UTF-8 text files misinterpreted as ISO 8859-1 due to Byte

 find . -type f | grep -rl $'\xEF\xBB\xBF'

show the current locale (language and character encoding)

 locale

show a hexdump of a text file

 hd file.txt
 hexdump file.txt    #(the format is a little different)

Text file encodings


Convert a file from ISO-8859-1 (or whatever) to UTF-8 (or

 tcs -f 8859-1 -t utf /some/file

Convert filenames from ISO-8859-1 to UTF-8

 convmv -r -f ISO-8859-1 -t UTF-8 --notest *

Detect encoding of the text file 'file.txt'

 file -i file.txt  ##(-i is the 'mime' switch, but it also shows encoding)

convert file from utf8 (no bom) to utf16 (with 'bom')

 recode UTF8..UTF-16LE linux-utf8-file.txt

convert all '.php' files to the utf-8 text encoding

 find . -name "*.php" -exec iconv -f ISO-8859-1 -t UTF-8 {} -o ../newf/{} \;

find utf-8 encoded text files misinterpreted as iso 8859-1

 find -type f | while read a;do [ "`head -c3 -- "${a}"`" == $'\xef\xbb\xbf' ] && echo "match: ${a}";done

Fix UTF-8 text files misinterpreted as ISO 8859-1 due to Byte

 perl -i -pe 's/\xef\xbb\xbf//g' 

Convert file type to unix utf-8

 ex some_file "+set ff=unix fileencoding=utf-8" "+x"

Convert one file from ISO-8859-1 to UTF-8.

 iconv --from-code=ISO-8859-1 --to-code=UTF-8 iso.txt > utf.txt

Spell checking


spell checking programs .. spell, a non interactive spell checker .. ispell, a veteran program .. aspell, the gnu version .. myspell, the open-office spell checker .. hunspell, based on ispell .. spellutils, debian package to selectively spell check

editors with spell checking .. vim, type ':set spell' to activate, and 'z=' to correct .. emacs, ..

search for all debian packages which have something to do with spelling

 apt-cache search spell 

spell check the file 'lecture'

 spell lecture

prints a list of badly spelled words

print all misspelled words in all ".txt" files with line numbers

 spell -n -o *.txt

spell check the file 'ch.1.txt', with misspellings to the file 'bad.sp'

 spell ch.1.txt > bad.sp 

check the spelling of a word on the command line

 echo 'is this korrect ?' | spell   

This prints 'Korrect' since it is badly spelled

output a sorted list of the misspelled words from 'lecture.draft'

 spell lecture.draft | sort | uniq 

Ispell


....

'ispell' is an older and simpler program than 'aspell'

interactively spell check 'report.txt'

 ispell report.txt 

install a British English dictionary for the "ispell" spell checker

 sudo apt-get install ibritish  

check and correct the spelling interactively in document "report.txt"

 ispell report.txt 
##(when a misspelling is found, type the number of the replacement)

spell check "file.txt" using a british english dictionary

 ispell -d british file.txt 

spell check a document written in spanish (using a spanish dictionary)

 ispell -d spanish archivo.txt 

show what dictionaries are available locally for ispell

 ls /usr/lib/ispell/

the ispell dictionaries are all called "i[language-name]"

 dictionary files: icatalan, ibrazilian ...

spell check and correct "thesis.tex" which is a LaTeX format document

 ispell -t thesis.tex  ##(ispell ignores the latex mark-up codes)

Aspell


aspell is a more modern and capable spell checking program

@@ http://aspell.net/ the official site @@ http://aspell. href="net/man-html/index.html">net/man-html/index.html A usage manual for aspell

show options for aspell and available dictionaries

 aspell help | less

show locally available dictionaries for aspell

 aspell dicts

install a British and American English dictionary for aspell

 sudo apt-get install aspell-en

install a spanish dictionary for aspell

 sudo apt-get install aspell-es

show all debian packages and dictionaries for aspell

 apt-cache search aspell 

interactively check the spelling of the file "chapter.txt"

 aspell -c chapter.txt
 aspell check chapter.txt      ##(the same)

Aspell with other languages


....

check the spelling of "chapter.txt" using British English spelling

 aspell -d british -c chapter.txt
 aspell -d en_GB -c chapter.txt   ##(this is the same)

check the spelling of "chapter.txt" using a Spanish dictionary

 aspell -d spanish -c chapter.txt
 aspell -d es -c chapter.txt   ##(this is the same)

check spelling in the comments in the shell script (lines starting with "#")

 aspell --mode=comment -c script.sh  ##(!!doesnt work on my version)

checking the spelling in the tex/latex file "chapter.tex"

 aspell -t -c chapter.tex 

show available filters for spell-checking particular types of files

 aspell filters
 aspell dump filters   ##(the same) 

spell check a file skipping (ignoring) lines which start with '>'

 aspell --mode=email check book.txt
 aspell --mode=email -c book.txt    ##(the same)
 aspell -e -c book.txt

create a vim "mapping" to use aspell within vim

 map TT :w!:!aspell check %:e! %

spell check a file but only between a "*" character and the end of the line

 aspell --add-filter=context --add-context-delimiters="* \0" -c francisco.txt
##(doesnt really work)

Text files


Viewing text files


....

text file viewing tools .. less - a text file pager .. most - a more capable pager ..

To print a specific line from a file

 awk 'FNR==5' 

set the default pager to be the 'most' program

 update-alternatives --set pager /usr/bin/most

View non-printing characters with cat

 cat -v -t -e

See non printable characters like tabulations, CRLF, LF line

 od -c  | grep --color '\\.'

Less


....

@@ http://www.greenwoodsoftware.com/less the homepage for less

The humble 'less' program is worthy of a second look. Less allows one to peruse and search a text file, but not alter it. I am documenting version 429 (year 2008). Less uses vi-like keys to move around and search.

view the text file 'doc.txt' one screen page at a time

 less doc.txt

view the text file 'days.txt' starting at the end

 less +G days.txt

some common 'less' commands .. [space-bar] - forward one window .. [esc] + [space-bar] - forward one window (with multiple files) .. b - back one window .. j - down one line (the same as 'vim') .. k - up one line (the same as 'vim') .. F - go to end of file and 'follow' new data (like tail -f) .. G - go to the last line of the file .. g - go to the first line of the file .. /pattern - Search forward for (N-th) matching line. .. ?pattern - Search backward for (N-th) matching line. .. n - Repeat previous search (for N-th occurrence). .. N - Repeat previous search in reverse direction. .. ESC-n - Repeat previous search, spanning files. .. v - edit the current file with $VISUAL or $EDITOR ..

some less command line switches .. -i - when searching within less, ignore case, unless search has uppercase .. -I - when searching within less, ignore case. .. -G - dont highlight matches when searching within less ..

Starting less


....

view the file 'long.txt' and make searches within less case-insensitive

 less -I long.txt

view the file 'long.txt', with 'semi' case-insensitive searching

 less -i long.txt  ##(searches with capital letters are case-sensitive)

make an alias which will make less always semi case-insensitive

 alias less='less -i'

within less turn on or off case-insensitive searching

 -I [enter] 

within less see whether searches are case-sensitive or not

 _I

view the output of 'grep' starting at the first line which has 'science' in it

 grep tree forest.txt | less +/science

follow the end of the log file 'tcp.log' showing new data as it enters

 less +F tcp.log   ##(this is like 'tail -f' but allows more perusal) 

Search for a word in less

 /\bTERM\b

go to the 80% position in the file (that is, 80% towards the end)

 p80

display less commands

 h

Less with multiple files


........

search multiple files for the text 'tree'

 less *.txt (then type) /*tree

Less bookmarks


........

Less bookmarks work in the same way as 'vi' or 'vim' bookmarks

mark the current top-of-screen position in the text file as bookmark 'x'

 mx     ##(any single letter can be used as a bookmark)

jump to the bookmark x

 'x

save text from current top-of-screen to the bookmark 'x' in file 'save.txt'

 |x cat > save.txt

jump to where you just were (before going to a bookmark)

 ''

edit the current file (but the variable $EDITOR or $VISUAL must be set)

 v   

Analysing language


....

Dictionaries


....

Look up the definition of a word

 curl dict://dict.org/d:something

Wordnet


....

get help for wordnet

 man wnintro
 man wn

show a list of word senses available for the word 'browse',

 wn browse -over

output a list of words from the dictionary that begin with the string 'homew'

 look homew  ##(prints something like 'homeward' and `homework' ...)

list words in the dictionary containing the string 'dont' regardless of case

 grep -i dont /usr/dict/words 

list all words in the dictionary that end with 'ing'

 grep ing^ /usr/dict/words 

list all of the words that are composed only of vowels

 grep -i '^[aeiou]*$' /usr/dict/words 

output a list of words that rhyme with 'friend', search '/usr/dict/words' for lines ending with `end':

 grep 'end$' /usr/dict/words 

search the WordNet dictionary for nouns that begin with 'homew'

 wn homew -grepn 

search the WordNet dictionary for nouns and adjectives that begin with 'homew'

 wn homew -grepn -grepa 

list the definitions of the word 'slope'

 wn slope -over 

output all of the synonyms (same meaning) for the noun 'break'

 wn break -synsn 

output all of the synonyms for the verb 'break'

 wn break -synsv 

output all of the antonyms (opposite meaning) for the adjective 'sad'

 wn sad -antsa 

A hypernym of a word is a related term whose meaning is more general

output all of the hypernyms for the noun 'cat'

 wn cat -hypen 

Debian 'dict'

check file 'dissertation' for clichés or other misused phrases, type:

 diction dissertation | less 

check file 'dissertation' for clichés or other misused phrases, and write the output to a file called 'dissertation.diction'

 diction dissertation > dissertation.diction 

If you don't specify a file name, diction reads text from the standard

output all lines containing double words in the file 'dissertation'

 diction dissertation | grep 'Double word' 

check the readability of the file 'dissertation'

 style dissertation 

Like diction, style reads text from the standard input if no text is given

output all sentences in the file 'dissertation' whose ARI is greater than a value of 20

 style -r 20 dissertation 

output all sentences longer than 14 words in the file 'dissertation'

 style -l 14 dissertation 

output the number of lines, words, and characters in file 'outline'

 wc outline 

output the number of characters in file 'classified.ad'

 wc -c classified.ad 

Use wc with the '-w' option to specify that just the number of words be

output the number of words in the file 'story'

 wc -w story 

output the combined number of words for all the files with a '.txt' file name extension in the current directory

 cat *.txt | wc -w 

output the number of lines in the file 'outline'

 wc -l outline 

output a word-frequency list of the text file 'naked_lunch',

 tr ' ' '\n' < naked_lunch | sort | uniq -c 

output a count of the number of unique words in the text file 'naked_lunch'

 tr ' ' ' 
> ' < naked_lunch | sort | uniq -c | wc -l

rank the files rep.a, rep.b, rep.c in order of relevance to keywords 'saving' and `profit'

 rel "(saving & profit)" report.a report.b report.c 

output a list of any files containing either 'invitation' or 'request' in the `~/mail' directory, ranked in order of relevancy, type:

 rel "(invitation | request)" ~/mail 

output a list of any files containing 'invitation' and not 'wedding' in the `~/mail' directory, ranked in order of relevancy, type:

 rel "(invitation ! wedding)" ~/mail 

output a list of any files containing 'invitation' and 'party' in the '~/mail' directory, ranked in order of relevancy

 rel "(invitation & party)" ~/mail 

Wrapping text lines


....

format a text file with lines 80 characters long,

 fmt -w 80 textfile   ##(short lines lengthened)
 fmt -s -w 80 textfile   ##(short lines are not lengthened)

use par instead

Splitting text files


....

split a file into a maximum of 10 files on lines containing '#200', '#400', '#600' etc with output files called "zz00", "zz01", etc

 csplit -f zz file.txt "/^#1?[24680]00$/" {8}  
##(the split occurs 'before' the line containing the match)

Merging text files


....

Concatenate lines of to files, one by one

 join file1.txt file2.txt > file3.txt

Merges given files line by line

 paste -d ',:' file1 file2 file3

Converting other formats to text


....

convert from html to text

 lynx -dump http://url > textfile
 links-dump http://url > textfile ##(may render tables)
 w3m -dump http://url > textfile  ##(may tables better)

remove the newline characters from the text file 'autoexec.bat'

 fromdos autoexec.bat 
 dos2unix autoexec.bat ##(the same)

add newline characters to all of '.tex' files in the current directory

 todos *.tex 
 unix2dos *.tex ##(the same)

Converting character encodings


....

@@ http://asis.epfl. href="ch/GNU.MISC/recode-3.6/recode_3.html">ch/GNU.MISC/recode-3.6/recode_3.html

Convert encoding of given files from one encoding to another

 iconv -f utf8 -t utf16 /path/to/file

see also iconv (older)

show possible conversions with the 'recode' tool

 recode -l | less 

convert latin9 (western europe) character encoding to utf8

 recode iso-8859-15..utf8 report.txt    ##(the actual file is changed)

convert from the local character set to the latin1 encoding saving to "new.txt"

 recode ..lat1 < file.txt > new.txt   ##(the original file is unchanged)

convert to html

 recode ..HTML < file.txt > file.html

convert from utf8 to html with verbose output

 recode -v u8..h < file.txt 

convert from MS Windows utf8 to the local character set

 recode utf-8/CRLF.. file-to-change.txt

Comparing and patching text files


....

The process of 'patching' a text or code file is very important in the world of open-source development (and therefore in the development of Linux itself). Patching allows non-linear changes to be made to a file and is usually used in conjuction with 'diff'

Use colordiff in side-by-side mode, and with automatic column

 colordiff -yW"`tput cols`" /path/to/file1 /path/to/file2

Compare a file with the output of a command or compare the output

 vimdiff foo.c <(bzr cat -r revno:-2 foo.c)

remote diff with side-by-side ordering.

 ssh $HOST -l$USER cat /REMOTE/FILE | sdiff /LOCAL/FILE -

Diff files on two remote hosts.

 diff <(ssh alice cat /etc/apt/sources.list) <(ssh bob cat /etc/apt/sources.list)

show lines that appear in both file1 and file2

 comm -1 -2 <(sort file1) <(sort file2)

find the extra lines in file2

 diff file1 file2 | grep ^>

find the extra lines in file1

 diff file1 file2 | grep ^<

Compare a remote file with a local file

 ssh user@host cat /path/to/remotefile | diff /path/to/localfile -

Generate diff of first 500 lines of two files

 diff <(head -500 product-feed.xml) <(head -500 product-feed.xml.old)

compare the files 'manuscript.old' and `manuscript.new'

 diff manuscript.old manuscript.new 

peruse the files 'olive' and 'green' side by side indicating differences

 sdiff olive green | less 

output a difference report for files 'tree', 'bush', and 'hedge',

 diff3 tree bush hedge > arbol 

update the original file 'manuscript.new' with the patchfile 'manuscript.diff'

 patch manuscript.new manuscript.diff 

Colored diff ( via vim ) on 2 remotes files on your local

 vimdiff scp://root@server-foo.com//etc/snmp/snmpd.conf scp://root@server-bar.com//etc/snmp/snmpd.conf

vimdiff to remotehost

 vimdiff tera.py <(ssh -A testserver "cat tera.py")

Searching text


....

gnu grep special characters .. \< - matches beginning of a word .. \> - matches the end of a word .. \b - matches a word boundary .. [:upper:] - matches upper case letters (unicode) .. [:lower:] - matches lower case letters (unicode) .. [:space:] - matches space characters .. ,,,

00- the "--" sequence may be "escaped" in grep. for example: grep "\-\-" file.txt

search interactively and ignoring case all '.txt' files in this folder

 cat *.txt | less -I   ##(then type '/' to search) 
 less -I *.txt         ##(then type '/*' to search, seems better) 

search for lines which begin with "#" in the text file "script"

 grep '^#' 

display lines which begin with an uppercase letter or word in 'doc.txt'

 grep '^[[:upper:]]\+' doc.txt 

display lines in 'doc.txt' which do not have any upper case letters in them

 grep '^[^[:upper:]]*$' doc.txt 

search for lines which dont contain the word "the"

 grep -v the 

show all lines which contain neither of the words "to" nor "green"

 egrep -v "tree|green" tree.txt 

show the names of all files containing the word 'tree' (searches subfolders)

 find . | xargs grep -l "tree" ##(the fastest way) 
 find | xargs grep -l "tree"   ##(the dot doesnt seem important) 
 grep -l tree $(find .)        ##(the same but not for lots of files)
 grep -rl tree *               ##(veeery sloow)

show names of all files Not containing the word 'mall' (searches subfolders)

 find | xargs grep -L "mall"      ##(matches mall, small, ...)    
 find | xargs grep -L "\"  ##(only matches 'mall') 

show the number of lines in a file which start with the word "tree"

 grep "^ *big" book.txt | wc -l

search for "leaf" or "tree" ignoring the case of the words

 grep -i -e "leaf" -e "tree" * 
 egrep -i "leaf|tree" *        ##(the same)
##(shows lines with the word "Leaf", "TREE", "trEE" etc)

show all text files in the books folder which have more than 100 lines

 find books/ | xargs wc -l | sort -rn | awk '$1 > 100 {print $2}'

search for a pattern (regex) in all text files (ignoring binary files)

 find . -type f | perl -lne 'print if -T;' | xargs egrep "somepattern"

Find and replace in text files


notes: turma ?? a graphical tool for search and replace

Single file


....

replace uppercase letters with lower case letters

 tr '[:upper:]' '[:lower:]' 

'squeeze' multiple spaces in the file "test.txt"

 tr -s ' ' test.txt 
 tr -s '[:blank:]' test.txt  ##(the same but better, handles all whitespace)

change aaa for bbb and print each line

 perl -p -e 's/aaa/bbb/' test.txt     ##(the file is not changed)

replace "aaa" with "bbb" and print each line

 perl -pi -e 's/aaa/bbb/' test.txt    ##(the file IS changed)

With a backup


....

replace the word "big" with "small" in .txt files backing up to .bak

 perl -p -i.bak -e 's/\bbig\b/small/g' *.txt

Multiple files


....

Change string in many files at once and more.

 find . -type f -exec grep -l XXX {} \;|tee /tmp/fileschanged|xargs perl -pi.bak -e 's/XXX/YYY/g'

replace the word "big" with "small" in .txt files backing up to .bak

 perl -p -i.bak -e 's/\bbig\b/small/g' *.txt

recursive replacement of text in the current folder and subdirectories

 perl -p -i.bak -e 's/\bbig\b/small/g' $(grep -ril oldstring *)

change 'big' to 'BIG' in all '.txt' files in this folder and subfolders

 set -i.bak 's/big/BIG' $(find . -name '*.txt')
##(the files are edited 'in place' and backed up to '.txt.bak')

find .txt files inside a directory and replace every occurrance

 find . -name '*.txt' -exec sed -ir 's/this/that/g' {} \;

a find and replace within text-based files, to locate and rewrite

 find . -name "*.txt" | xargs perl -pi -e 's/old/new/g'

find and replace with vim 'argdo'

 vim *  ... etc

Analysing text


get line number of all matches in a file

 awk '/match/{print NR}' file

Count the number of characters in each line of a file (??)

 awk '{cc[length]++}END{for(i in cc){printf("%d: %d\n", cc[i], i)}}'

show unique words and the number of times each word occurs

 tr -sc [A-Z][a-z] [\012*] < file.txt | sort | uniq -c | less ##(???)

Count the number of lines in all files in this and subfolders

 wc -l $(find . -name *.php)

plot the frequency distribution of words from files

 cat *.txt | { printf "se te du\nplot '-' t '' w dots\n"; tr '[[:upper:]]' '[[:lower:]]' | tr -s [[:punct:][:space:]] '\n' | sort | uniq -c | sort -nr | head -n 100 | awk '{print $1}END{print "e"}'; } | gnuplot

Analysing text data


....

tools to process text data .. awk, a very capable text data processing language .. perl, an even more capable language .. cut, a simple field based tool .. sort, sort lines of a text file ..

data jargon @@ delimiter This is a something which separate 2 field or marks the beginning or end of some data. If we have 2 numbers separated by a space character as in '23.2 200' then we say that the data is 'delimited' by spaces. @@ field A field is a unit of data in a record. If we have data about people, then 3 fields may be a persons first name, second name and age. However the way that data is broken up into fields is subjective and option culturally dependent. Therefore there is no universally accepted data format @@ data record a record is a set of data fields which relate to a particular entity. For example if we store data about people, then a record will contain data for one person. The way the record is stored is dependant upon the data format. @@ export data The process of exporting data involves converting data to a particular format and saving it in a file or files. @@ import data

related booklets @@ http://bumble.sf. href="net/books/awk/awk-book.txt">net/books/awk/awk-book.txt A small booklet about the awk language, also available as a pdf file. @@ http://bumble.sf. href="net/books/perl/perl-book.txt">net/books/perl/perl-book.txt a booklet about the perl language

The delimiter character for cut can only be a single character, as far as I am aware. If more processing power is needed then the reader should consider using awk, sed, or perl.

show the second "field" of each line with space delimited fields

 cut -d ' ' -f 2 data.txt 

If any extra spaces are in the file, this will fail

with space separated fields, first get rid of unwanted spaces with sed

 sed -r 's/^\s+//; s/\s+/ /g' cgt.txt | cut -d' ' -f2 
if the data were spain 4 5 italy 11 5 england 4 6 then the output would be 4 11 4 ,,,

show every comma delimited field except the second

 cut -d, -f 2 --complement list.txt
if the content of 'list.txt' was a,b,c,d f,g,h,i then the output would be a,c,d f,h,i ,,,

show the 2nd field and all subsequent fields of a comma delimited file

 cut -d, -f2- data.csv

show the 1st, 2nd and 3rd fields

 cut -d, -f-3

show fields 2, 3 and 4

 cut -d, -f2-4

sort alphabetically on the 3rd field of the file ignoring initial blanks

 sort -k 3b

Extraction data from text


....

display all the urls from 'essay.txt' and select to open in browser

 urlview essay.txt

Tidying text files


....

see how many lines in a text file are consecutive and duplicate

 uniq -d file.txt | wc -l   ##(this also counts duplicate blank lines)

show all duplicated (consecutive) lines in a text file

 uniq -d file.txt 

output the file 'paper' with 1 space between words and 2 after fullstops

 fmt -u paper 

show the file 'term-paper' with multiple blank lines as only one blank line

 cat -s term-paper 

display 'term-paper' with multiple blank lines removed and giving the text unified spacing between words

 cat -s term-paper | fmt -u | less 

output the file 'term-paper' with lines up to 75 characters long

 fmt term-paper 

output the file 'doc.txt' with lines up to 80 characters long

 fmt -w 80 doc.txt

wrap long lines in the file 'doc.txt' without disturbing special patterns

 par doc.txt

Stream editing text files


....

'stream editing' or 'batch editing' text files refers to the process of modifying a text file without opening a text editor. In other words the text file is modified via a series of commands which are applied to a file or many files with a program such as sed, awk, perl, etc. This is very useful when a large number of modifications need to be made in many files and where those modifications are similar; For example where a persons name needs to be changed in a large number of files.

Randomize lines (opposite of | sort)

 random -f 

uniq for unsorted data

 awk '!_[$0]++{print}'

Add a line to a file using sudo

 echo "foo bar" | sudo tee -a /path/to/some/file

Remove duplicate entries in the file 'list.txt' without sorting.

 awk '!x[$0]++' list.txt 

Sorting text data


The unix 'sort' utility is a powerful and flexible way to reorder lines of a text file based on the data in one or more of the 'fields' of each line. A field is a chunk of text separated from another chunk by a 'delimiter' character such as a space or a colon. For example if the line is 'italy 3 4/oct/1999' and the delimiter is a space then the fields are 'italy', '3' and '4/oct/1999'

By default, sort divides a line into fields based on spaces or tabs but can use another character (see the -t option)

Viewing help for sort


....

view some examples of using the 'sort' program

 info coreutils 'sort invocation'

view a confusing description about how 'sort' works

 man sort   ##(this is a classic example of an accurate but baffling man page)

Basic usage


....

sort in alphabetical order (abc...) the lines of the text file 'data.txt'

 sort data.txt    ##(the sorted lines are displayed, the file is unchanged)

sort in reverse alphabetical order (zxy...) the lines of the file 'data.txt'

 sort -r data.txt 

sort in alphabetical order, ignoring case, the lines of the file 'data.txt'

 sort -fr data.txt 
 sort -f -r data.txt   ##(the same)

sort in alphabetical order using the 3rd 'field' in each line as the sort key

 sort -k3 data.txt  ##(each 'field' is a bit of text separated by spaces)
 sort +2 data.txt   ##(prehistoric syntax, probably to be avoided like bubonic)

Sort IPV4 ip addresses

 sort -t. -k1,1n -k2,2n -k3,3n -k4,4n

sort lines by length

 perl -lne '$l{$_}=length;END{for(sort{$l{$a}<=>$l{$b}}keys %l){print}}' < /usr/share/dict/words | tail

Sort inplace


....

sort a file and update it

 sort -o data.txt data.txt       ##(is this safe??)
 sort --output=data.txt data.txt

Sorting by multiple fields


....

The lines of a file can be sorted first by one field, and then by a second field when the value of the 1st field is equal

sort a file alphabetically by the 1st field and then numerically by the 2nd

 sort -k1 -nk2 data.txt  

sort the lines alphabetically using the first 3 fields of the file

 sort -k 1,3 data

Numerical sorts


....

perform a numerical sort using the second field

 sort -k2,2n data.txt  ##(2,2 means from the 2nd to the 2nd field)
 sort -n -k2 data.txt  ##(watch out! this may be different ??)

sort lines using the 2nd field as a numeric key with fields delimited by '/'

 sort -n -t/ -k2 data.txt 
 sort -n -t'/' -k2 data.txt    ##(the same)
 sort -nt/ -k2 data.txt      ##(the same, again)

sort in ascending numerical order using the 2nd field as the sorting key

 sort -k 2,2n data   ##(possibly the correct way to do this)
 sort -k 2n,2n data  ##(the same)
 sort -n -k2 data    ##(this may be subtly different)
 sort -nk2 data      ##(options can be written together)

sort the lines of 'data' using the 3rd ':' delimited field as a number key

 sort -n -k3 -t: data
 sort -nk3 -t: data       ##(the same)
 sort -nk3 -t :  data     ##(the same again)

sort the lines of 'data' in descending (reverse) numeric order

 sort -rn data

sort lines in descending numerical order using the 2nd field as the sort key

 sort -k2 -r -n data.txt
 sort -rnk2 data.txt    ##(the same, but better)

sort in ascending numeric order using only the 1st digit of the 2nd field

 sort -nk2.1,2.1 data.txt ##('2.1,2.1' means 'from the 1st to the 1st char)

sort in descending numeric order using the first 3 digits of the 2nd field

 sort -rnk2.1,2.3 data.txt 

Sorting by date values


....

The problem of sorting by a date value is somewhat tricky, mainly because of the daunting array of different formats which a date can appear in, without even thinking about intercultural differences. However 'sort' is capable of sorting by date value, where the dates are in a consitent format.

Where the date string is of a variable length the problem gets harder. It may be possible to use the -t delimiter to divide up each field of the date (eg use '-t/' for '1/january/08' and '02/oct/2001')

Use the 'M' modifier to recognise month names and abbreviated month names

sort lines by month name, assuming that field 3 is an english month name

 sort -k3M data  ##(field 3 can be something like 'feb', 'oct' or 'january')

sort by month names using the 3rd character of the 2nd field as the start point

 sort -k2.3M data  ##(field 2 may be '--jan' or '::august' etc)

sort lines by the 1st field, a date in the format dd/mm/yyyy (eg '09/11/1922')

 sort -k 1.7n -k 1.4n -k 1.1n -k 4.14,4.21 data.txt
##(note that '4/11/1920' will not sort well but '04/11/1920' will)

reverse sort lines by a date field in the format dd/mm/yyyy

 sort -k 1.7nr -k 1.4nr -k 1.1nr data.txt

sort lines with the second field a date in format dd/MON/yyyy '02/Apr/2010'

 sort -k 1.8n -k 1.4M -k 1.1n data.txt
##(the month abbreviations must be 3 letters, and in english ...)

sort lines by a date/time value such as '01/Jan/2004:06:31:51'

 sort -k 1.8n -k 1.4M -k 1.1n -k 1.13,1.21 data.txt
##(this assumes the date is the first field of each line)

Fields and partial fields


....

sort lines starting the key at the second character of the 2nd field

 sort -k2.2 data

sort using characters 2 to 8 as the alphabetic sort key

 sort -k2.2,2.8 data

Other tools


....

tsort is a mysterious tool

 tsort 

sort text files by the number of lines which they contain, reverse order

 file * | grep text | sed "s/:.*//" | xargs wc -l | sort -rn | grep -v "total$" | less

display the lines in a text file 'shuffled'

 shuf file.txt

shuffle lines in a file and update the file

 shuf -o F 
  
 cat F | shuf -o F   ##(the same, no risk of truncation 'apparently')

shuffle some lines entered at the prompt shuf < invasion! eof ,,,

shuffle command line arguments shuf -e clubs hearts diamonds output:
hearts diamonds clubs ,,,

display the lines of a file in reverse order

 tac file     ##(yes 'tac' is the reverse of 'cat')

Typesetting text documents


This section explains how to prepare documents to be printed on paper. This is actually a really large topic. See also the latex-book.txt file for detailed information about the Latex typesetting system.

typesetting tool quick summary .. groff, an old unix system, used for 'man' pages .. latex, extensive system widely used for scientific and maths docs .. docbook, an xml based "super" system .. pod, the perl document system, .. man, the old unix "manual page" system based on groff .. enscript, turns text into pdf in a configurable manner .. markdown, minimal text markup .. pandoc, implements and extends markdown, lots of output format .. phpmarkdown, a php implementation of markdown ..

Overview


....

This section tries to give an overview of the numerous tools available to produce 'typeset' documents in a format ready to be sent to a printer (such as pdf or postscript).

@@ markdown The user writes a plain text document adhering to certain formatting rules ... and markdown produces a formatted html document @@ halibut The user embeds simple 'markup' codes into the text document and the tool can convert the document into a variety of output formats including pdf @@ reStructured text another philosophy similar to markdown

@@ http://docutils.sourceforge. href="net/docs/ref/rst/introduction.html">net/docs/ref/rst/introduction.html a page about 'restructured text'

Latex


....

Latex is for people who have 4 years to write a thesis. If you have less time, use enscript or halibut.

see the latex booklet at http://bumble.sf. href="net/books/latex/latex-book.txt">net/books/latex/latex-book.txt

convert a latex source file (.tex) into opendocument (.odt )

 htlatex MyFile.tex "xhtml,ooffice" "ooffice/! -cmozhtf" "-coo -cvalidate"

Text data formats


store personal contact information in a text file

 vcard

Xml


....

@@ http://xmlstar.source¿forge.net/ The home page for xmlstarlet @@ xmlsoft.org more linux xml software @@ http://xmlstar.sourceforge. href="net/doc/xmlstarlet.txt">net/doc/xmlstarlet.txt xmlstarlet examples

xml tools .. xmlstarlet - useful tools for xml from the command line ..

remove comments from xml

 cat  | perl -e '$/ = ""; $_ = <>; s///gs; print;'

remove comments from the xml file 'page.xhtml'

 xmlstarlet ed -d '//comment()' page.xhtml

show some help information for the 'ed' option of xmlstarlet

 xmlstarlet ed -h

display the xml entity for the & ampersand character

 xmlstarlet esc '&'

Count elements matching XPath expression

 xmlstarlet sel -t -v "count(/xml/table/rec/numField)" xml/table.xml

Count all nodes in XML document

 xmlstarlet sel -t -f -o " " -v "count(//node())" xml/table.xml xml/tab-obj.xml

Delete elements matching XPath expression

 xml ed -d /xml/table/rec[@id='2'] xml/table.xml

Generate HTML from given SGML docbook document

 xml tr --omit-decl --docbook /usr/share/sgml/docbook/yelp/docbook/html/docbook.xsl  sgml/docbook1.sgml | xml fo --html --indent-spaces 2

Validate XML document against a DTD

 xml val --dtd dtd/table.dtd xml/tab-obj.xml >/dev/null 2>&1; echo $?

Prettify an XML file

 tidy -xml -i -m [file]

Csv


....

csv stands for 'comma separated something' and is an old and simple text data format. A simple example of a line in a csv file would be

 "Bob", "Green", 43

Where each of the fields may stand for 'first name', 'second name' and age.

sum all the numbers in the 4th column from a csv data file

 awk -F ',' '{ x = x + $4 } END { print x }' test.csv

turn lines in columns in csv format

 ls | sed -n '1h;2,$H;${g;s/\n/,/g;p}'

convert 'csv' text data files to 'tsv' (tab delimited format)

 sed 's/,/\t/g' report.csv > report.tsv

sum columns from csv column $col

 perl -ne 'split /,/ ; $a+= $_[3]; END {print $a."\n";}' -f ./file.csv

pretty-print a simple csv file on the command line

 column -s, -t 

parse a quoted .csv file

 awk -F'^"|", "|"$' '{ print $2,$3,$4 }' file.csv

extract the 2nd field of a csv file with 'cut'

 cut -d\" -f2 file.csv
 cut '-d"' -f2 file.csv

Text oriented programming languages


The awk language


....

@@ http://www.grymoire. href="com/Unix/Awk.html">com/Unix/Awk.html a tutorial @@ http://www.pement.org/awk/awk1line.txt one line examples of awk

Randomize lines in a file

 awk 'BEGIN{srand()}{print rand(),$0}' SOMEFILE | sort -n | cut -d ' ' -f2-

Print a row of 50 hyphens

 awk 'BEGIN{while (a++<50) s=s "-"; print s}'

Calculating series with awk: add numbers from 1 to 100

 seq 100 | awk '{sum+=$1} END {print sum}'

print file without duplicated lines using awk

 awk '!($0 in a) {a[$0];print}' file

Print line immediately before a matching regex.

 awk '/regex/{print x};{x=$0}'

Grep


....

Grep is the classic and venerable unix tool for searching for text within files.

count how many times a string appears in a tree of files

 grep -or string path/ | wc -l

permanently let 'grep' colorize its output

 echo alias grep=\'grep --color=auto\' >> ~/.bashrc ; . ~/.bashrc

Search through files, ignoring .svn

 grep  -R . --exclude-dir='.svn'

Output files without comments or empty lines

 grep -v "^\($\|#\)" 

Colorize matching string without skipping others

 egrep --color=auto 'usb|' /var/log/messages

find string

 grep -iR find_me ./

Sed stream editor


....

This section provides only a very brief overview to the sed stream editor attempting to cover only the most common tasks which may be performed with sed. For more information see the booklet 'sed-book.txt'

@@ http://sed.sourceforge. href="net/sedfaq.html">net/sedfaq.html The official site for sed containing many examples

perform sed substitution on all but the last line of input

 sed -e "$ ! s/$/,/"

replace 'a' with 'A', 'b' with 'B' etc, similar to the 'tr' tool

 sed "y/abcd/ABCD/g" 

convert words beginning in 'b' or 'c' to upper case

 sed -r "s/\<(b|c)[a-z]+/\U&/g"      

convert upper case words to lower case

 sed -r "s/\<[[:upper:]]+\>/\L&/g" 

match words beginning in 'a' or 'b'

 sed -r "s/\<(a|b)[a-z]+/&/g"    

make changes to files and back up to .bak

 sed -i.bak -r "s/yes/no/g" *.txt    ##(gnused 4)

detailed information about gnu regular expressions (used in sed)

 man 7 regex

delete uppercase letters and commas.

 sed -r 's/[[:upper:],]//g'

Perl


Perl is a language which was originally inspired by the Bash shell syntax, as well as by the idea of writing terse but powerful programs. Perl initially rose to fame through its suitability for writing web-server cgi scripts, since perl, like the unix shells, uses plain text as a kind of "data interchange format".

This section only provides the briefest of introduction to perl, with an emphasis on using short, one-line programs for carrying out a simple task from the bash-shell.

a command line calculator in perl

 perl -e 'for(@ARGV){s/x/*/g;s/v/sqrt /g;s/\^/**/g};print eval(join("",@ARGV)),$/;'

determine if a regular expression has a valid syntax

 perl -we 'my $regex = eval {qr/.*/}; die "$@" if $@;'

a command which will make perl crash

 perl -e '$x = []; push @$x, eval { $x = 1; return $x = 1; }'

transform the contents of a text file to all uppercase.

 perl -i -ne 'print uc $_' $1

print the line numbers of a text file

 perl -ne 'print "$. - $_"' infile.txt

Sort the size usage of a directory tree by gigabytes, kilobytes,

 du -b --max-depth 1 | sort -nr | perl -pe 's{([0-9]+)}{sprintf "%.1f%s", $1>=2**30? ($1/2**30, "G"): $1>=2**20? ($1/2**20, "M"): $1>=2**10? ($1/2**10, "K"): ($1, "")}e'

Perl documentation


....

debian: perl-doc - to use the perldoc tool

query the perl "frequently asked questions" documents for a word or phrase

 perldoc -q eval

show the documentation for the CGI module

 perldoc CGI   ##(these names are case-sensitive, "perldoc cgi" doesnt work)

Perl one line scripts


....

include 2 perl expressions with the -e expression

 perl -e 'print "Hello";' -e 'print " World\n"'

print the 1st and 2nd fields of the input lines

 echo a b c | perl -lane 'print "@F[0..1]"'

print the 3rd line of a file

 perl -nle 'print if $. == 1' file.txt

display the current week number

 perl -e 'use Date::Calc qw(Today Week_Number); $weekn = Week_Number(Today); print "$weekn\n"'

Perl modules


....

Upgrade all perl modules via CPAN

 perl -MCPAN -e 'CPAN::Shell->install(CPAN::Shell->r)'

Upgrade all perl modules via CPAN

 cpan -r

Open Perl module source in your editor

 $EDITOR `perldoc -l Module::Name`

clean up syntax and de-obfuscate perl script

 %! perl -MO=Deparse | perltidy

Python


one-liner to display ip addresses

 python -c "import socket; print '\n'.join(socket.gethostbyname_ex(socket.gethostname())[2])"

An easter egg built into python to give you the Zen of Python

 python -c 'import this'

Cleanup Python bytecode files

 find . -name "*.py[co]" -exec rm -f {} \;

Quick syntax highlighting with multiple output formats

 $ python -m pygments -o source.html source.py

Ruby


Ruby is a language which, along with Python has become increasingly popular. The 'Ruby on Rails' system is a frame-work for creating web-applications (complex websites)

Display command lines visible on commandlinefu.com homepage

 ruby -ropen-uri -e 'require "hpricot";(Hpricot(open("http://commandlinefu.com"))/".command").each{| c| puts c.to_plain_text}'

print a row of 50 hyphens

 ruby -e 'puts "-" * 50'

ruby one-liner to get the current week number

 ruby -rdate -e 'p DateTime.now.cweek'

ruby one-liner to get the current week number

 ruby -e 'require "date"; puts DateTime.now.cweek'

Php


run php code inline from the command line

 php -r 'echo strtotime("2009/02/13 15:31:30")."\n";'

display information about the php configuration

 php -i

test the php configuration

 php -r "phpinfo\(\);"

search a folder to check for syntax errors in php files

 find . -name "*.php" -exec php -l {} \;

The c language


enter and compile a c one line program

 /lib/ld-linux.so.2 =(echo -e '#include \nint main(){printf("c one liners\\n");}' | gcc -x c -o /dev/stdout -)  ##(?? unchecked code)

Source code files


Viewing source code


....

view 'script.sh' in a pager with some syntax highlighting

 over myscript.sh     ##(over is part of the 'enscript' package)

Formatting and indenting source code


....

The program 'indent' is designed for c and cpp but can be used for java as well.

format source code

 astyle, indent, bcpp

indent a c file with no brace indent,

 indent -bli0 

indent 'file.c' but dont put blank lines after procedures

 indent -nbap 

indent Test.java using the 'kernighan & ritchie' style (compact)

 indent -kr Test.java -st | less

indent 'file.c' without lining up parenthesis (). (prevents large indents)

 cat file.c | indent -kr -nlp -st | less
##(the 'k & r' style puts the braces on the same line)

indent Test.java with an indent of 2 spaces, writing to standard output

 indent -gnu --indent-level2  -st Test.java | less
##(the gnu style puts the curly braces underneath the previous line)

bli is the brace indent

indent the java code file 'original', with an indent of 2 spaces

 astyle -js2 < original > new
 astyle --mode=java --indent=2 < original > new  ##(the same)

Converting to syntax highlighted documents


....

The following tools can convert source code in a text file to another format, such as HTML, rtf or LaTeX, with the syntax of the source code highlighted for easier reading. For HTML, the tool 'webcpp' seems to produce the nicest looking output (at least with the default settings). All the tools support a number of programming languages for the input code file.

Syntax highlighting tools @@ webcpp http://webcpp.sourceforge.net outputs only in HTML @@ highlight outputs formats: ansi latex xterm256 rtf tex xhtml xml svg @@ gnu source-highlight output formats: HTML, XHTML, LaTeX, Texinfo, ANSI color escape codes and DocBook @@ the vim editor @@ enscript

convert source code to highlighted html with no tags

 webcpp codefile htmlfile -s

convert 'file' to html, writing to standard out with unix script type

 cat file | webcpp - - -x=sh -s | less

convert java code to html with line numbers and internal css definition

 highlight -i Test.java -o Test.java.html -I -l

convert a c file to LaTeX with the syntax highlighted

 highlight -L -i main.cpp -o main.cpp.tex

'highlight' output formats

 LaTeX (-L), XHTML (-X), TeX (-T), RTF (-R), ANSI Escape Quotes(-A), and XML (-Z). 

convert a c++ file to html from an input stream and with external css

 highlight < main.cpp > main.cpp.html -Sc

convert chom.c to pretty printed postscript with 2 columns, landscape

 enscript -p new.ps -Ec -2 -r -f "Courier8" chom.c
 enscript -p new.ps -2rEc -f Courier8 chom.c     ##(the same)

the enscript options used .. -p - output to the file 'new.ps' .. -r - landscape .. -2 - two columns .. -Ec - syntax highlight with 'c' syntax .. -f "Courier8" - use a 8 point courier font ..

convert 'Assig.java' to a syntax highlighted 6 point pdf file

 enscript -o - -2rE -f Courier6 Assig.java | ps2pdf - Assig.pdf

the enscript options used .. -r - landscape .. -2 - two columns .. -E - syntax highlight the code .. -f "Courier6" - use a 6 point courier font .. -o - sent the output to standard out ..

see what computer languages enscript can handle

 enscript --help-pretty-print

Command history


The 'history' file is a special text file which contains all recent commands which have been entered at the command line. This is invaluable to avoid retyping things.

@@ http://bumble.sf. href="net/books/bash/bash-book.txt">net/books/bash/bash-book.txt Contains more information about the bash history file

put a time stamp in the command history and store more commands

 export HISTTIMEFORMAT='%Y.%m.%d-%T :: ' HISTFILESIZE=50000 HISTSIZE=50000

This will result in the 'history' command displaying something like 514 2010.05.12-09:15:26 :: ls -thor 515 2010.05.12-09:15:50 :: java AssigOne110 516 2010.05.12-09:16:36 :: ls ,,,

A function to display the history and optionally search through it

 h(){ if [ -z "$1" ]; then history; else history | grep "$@"; fi; }; h
 h(){ [ -z "$1" ] && history || history | grep "$@"; }; h

clear your history saved into .bash_history file

 history -c && rm -f ~/.bash_history

clear your history saved into .bash_history file

 history -c

put environment variable in history to edit

 print -s "PATH='$PATH'"

A bit of privacy in .bash_history

 export HISTCONTROL=ignoreboth

List of commands you use most often

 history | awk '{a[$2]++}END{for(i in a){print a[i] " " i}}' | sort -rn | head

find the longest command in your history

 history | perl -lane '$lsize{$_} = scalar(@F); if($longest<$lsize{$_}) { $longest = $lsize{$_}; print "$_"; };' | tail -n1

exit without saving history

 kill -9 $$

See most used commands

 history|awk '{print $2}'|awk 'BEGIN {FS="|"} {print $1}'|sort|uniq -c|sort -r

Avoiding history file to be overwritten

 shopt -s histappend

Add timestamp to history

 export HISTTIMEFORMAT="%F %T "

delete a line from your shell history

 history -d

Text and character recognition


See http://bumble.sf.net/books/linux-image/ for more information about this topic

The process of extracting text data from an image file (which has usually been obtained by scanning a page of typed text) is known as "ocr" for Optical Character Recognition

ocr tools .. tesseract-ocr - Command line OCR tool, said to be mature .. ocropus - google sponsored ocr tool, uses tesseract .. gocr - .. ocrad - .. clara - .. unpaper - improve the quality of scanned text images ..

Images


@@ http://bumble.sf. href="net/books/linux-image/linux-image-book.txt">net/books/linux-image/linux-image-book.txt A more comprehensive treatment of images on linux

Determine an image's dimensions

 identify -format "%wx%h" /path/to/image.jpg

Convert images (jpg, png, ...) into a big PDF 'big.pdf'

 convert images*.* big.pdf

Recipes


....

browse the '.gif' images in the '/usr/doc/imagemagick/examples' directory

 display 'vid:/usr/doc/imagemagick/examples/*.gif' 

browse through all image files in the current directory

 display 'vid:*' 

In the preceding example, only those files with image formats supported by display are read and displayed.

put the image 'tetra.jpeg' in the root window

 display -window root tetra.jpeg 

Use zgv to view images in a virtual console (not in X). You can use zgv

browse the images in the current directory

 zgv 
browse the images in the '/usr/share/gimp/scripts' directory, type:
 zgv /usr/share/gimp/scripts 
Use the arrow keys to navigate through the file display; the red border

view the file '/usr/share/images/mondrian-15.jpeg' file:/usr/share/images/mondrian-15.jpeg Notice that the given [36]file: URL only has one preceding slash, pointing to the root directory, and not two, as in [37 http://.

browse the images on the PhotoCD disc mounted on '/cdrom'

 xpcd /cdrom 
The preceding example will open two new windows -- a small xpcd command bar window, and a larger window containing thumbnails of all PhotoCD

view the PhotoCD file 'driveby-001.pcd'

 xpcd driveby-001.pcd 

Note


: You can also use display to view a '.pcd' PhotoCD image file (see

resize 'phoenix.jpeg' to 480x320 pixels

 mogrify -geometry 480x320 phoenix.jpeg 
This transforms the original 'phoenix.jpeg' file to:
editing-images-scaling-01

resize 'phoenix.jpeg' to exactly 480x320 pixels, regardless of aspect ratio

 mogrify -geometry 640x480! phoenix.jpeg 
This transforms the original 'phoenix.jpeg' to:

increase the height of 'phoenix.jpeg' by 25 percent and decrease its width by 50 percent

 mogrify -geometry 125%x50% phoenix.jpeg 

rotate 'phoenix.jpeg', whose height exceeds its width, by 90 degrees

 mogrify -rotate '90<' phoenix.jpeg 

reduce the colors in 'phoenix.jpeg' to two

 mogrify -colors 2 phoenix.jpeg 
This transforms the original 'phoenix.jpeg' to:
editing-images-colors-01

reduce the colors in 'phoenix.jpeg' to four and apply Floyd-Steinberg error diffusion

 mogrify -colors 4 -dither phoenix.jpeg 
This transforms the original 'phoenix.jpeg' to:

change the colors in the file 'rainbow.jpeg' to those used in the file 'prism.jpeg'

 mogrify -map prism.jpeg rainbow.jpeg 
Use the '-monochrome' option to make a color image black and white. make the color image 'rainbow.jpeg' black and white
 mogrify -monochrome rainbow.jpeg 
If you have a PPM file, use ppmquant to quantize, or reduce to a specified quantity the colors in the image -- see the ppmquant man page

set the gamma correction of the image 'rainbow.jpeg' to .8, type:

 mogrify -gamma .8 rainbow.jpeg 

annotate the image file 'phoenix.jpeg', type (all on one line):

 mogrify -comment "If you can read this,
you're too close!" phoenix.jpeg You won't see the annotation when you view the image; it is added to

read any comments made in the image file 'phoenix.jpeg'

 rdjpgcom phoenix.jpeg 

add a border two pixels wide and four pixels high to 'phoenix.jpeg'

 mogrify -border 2x4 phoenix.jpeg 

add a decorative frame eight pixels wide and eight pixels high to 'phoenix.jpeg'

 mogrify -frame 8x8 phoenix.jpeg 
This transforms the original 'phoenix.jpeg' to:

create a montage from the files 'owl.jpeg', `thrush.jpeg', and 'warbler.jpeg' and write it to `endangered-birds.png'

 montage owl.jpeg thrush.jpeg warbler.jpeg endangered-birds.png 

Note


: In this example, three JPEGs were read and output to a PNG file;

combine two images, 'ashes.jpeg' and `phoenix.jpeg', into a new file 'picture.jpeg'

 combine ashes.jpeg phoenix.jpeg picture.jpeg 
You can specify the percentage to blend two images together with the

combine the image files 'phoenix.jpeg' and `ashes.jpeg' so that the blended image contains 70 percent of the second image

 combine -blend 70 ashes.jpeg phoenix.jpeg picture.jpeg 
This command combines the two images and writes a new image file,

make a morphed image of the files 'ashes.jpeg' and 'phoenix.jpeg', and write it to `picture.jpeg'

 combine -compose difference ashes.jpeg phoenix.jpeg picture.jpeg 
The result in file 'picture.jpeg' is:

convert the JPEG file 'phoenix.jpeg' to a PNG image

 convert phoenix.jpeg phoenix.png 
This command converts the JPEG image 'phoenix.jpeg' to PNG format and writes it to a new file, 'phoenix.png'.

convert the PNM file 'pike.pnm' to non-interlaced JPEG while sharpening the image by 50 percent and adding both a 2x2 border and a copyright comment

 convert -interlace NONE -sharpen 50 -border 2x2

list available scanner devices

 scanimage --list-devices 

device 'umax:/dev/sgb' is a UMAX Astra 1220S flatbed scanner

list available options supported by the device listed in the previous example

 scanimage --help -d 'umax:/dev/sgb' 

Note


: For all scanimage commands, specify the scanner device you want

test the UMAX scanner listed previously

 scanimage --test -d 'umax:/dev/sgb' 

Debian: 'netpbm'

make a 72 dpi scan of a color image 200 pixels wide and 100 pixels tall, using the UNIX scanner from previous examples, and writing to a file called 'scan.ppm'

 scanimage -d umax:/dev/sgb --resolution 72 -x 200 -y 100 >

make a 300 dpi scan of a black and white image 180 pixels wide and 225 pixels tall, using the UMAX scanner from previous examples, and writing to a file called 'scan.pbm'

 scanimage -d umax:/dev/sgb --resolution 300 --mode lineart

extract the highest resolution from the file 'slack.pcd' and save it to a PPM file named 'slack.ppm'

 pcdtoppm -r5 slack.pcd slack.ppm 
[37]Converting PhotoCD: Converting PhotoCD images to other formats.

convert the file 'slack.ppm' to non-interlaced JPEG, sharpen the image, add a two-pixel by two-pixel border, and annotate the image, type (all on one line):

 convert -interlace NONE -sharpen 50 -border 2x2 -comment

remove the "green haze" from a PhotoCD image, do the following: + First, open the extracted image in the GIMP (see [44]Editing Images with the GIMP). + Then, click through the Image menu to the Colors submenu and

Managing images


....

find all image files in the 'icons' folder

 find /usr/share/icons/ | xargs file | grep image | less

find all image files on the computer

 sudo find / | xargs file | grep ' image' | less

Viewing images


....

linux image viewers .. xv - an old x windows viewer .. xli - another one .. aview - ascii image viewer .. asciiview - .. feh - a fast command line driven image viewer .. gthumb - a gnome viewer and simple editor .. geeqie - gtk image viewer .. gpicview - small image viewer .. gqview - simple gtk image viewer, emacs style keys .. eye of gnome - the gnome viewer .. mirage - simple gtk viewer, with thumbnails .. imgseek - image viewer, manager, requires python .. pqiv - similar to qiv, but different, very basic .. qiv - quick image viewer, up to date .. gwenview - kde image viewer .. showfoto - kde ..

show jpegs with thumbnails

 find ~ -name '*.jpg' | xargs mirage 

Feh


....

Feh is a fast command line driven image viewer. maybe just what we need.

show all the images in the /opt/images tree in a slideshow.

 feh -r /opt/images

same again, but sort by image name before showing.

 feh -rSname /opt/image

create a montage from the images in /opt/images/landscapes

 feh -m /opt/images/landscapes

Create a montage from the images in /opt/images/landscapes and all directories below it. Limit the width of the image to 400 and make the thumbnails 30x20, ignoring aspect ratio

 feh -Xrm -W 400 --thumb-width 30 --thumb-height 20 ./landscapes

view jpgs and gifs with thumbnails

 find ~ -name '*.jpg' -o -name '*.gif' | xargs gthumb

open all images specified in the text file 'images.txt' (one per line)

 qiv -F images.txt
 qiv --file images.txt

display all jpg and gif images shrinking to fit them in the window

 find ~ -name '*.jpg' -o -name '*.gif' | xargs qiv -t

browse through all "gif" images in the current folder "gallery" style

 display 'vid:*.gif'

Image metadata


....

Remove EXIF data from images with progress

 i=0; f=$(find . -type f -iregex ".*jpg");c=$(echo $f|sed "s/ /\n/g"| wc -l);for x in $f;do i=$(($i + 1));echo "$x $i of $c"; mogrify -strip $x;done

Screenshots


....

tools for screen capture .. import - makes screen shots, part of 'imagemagick' package. .. scrot - a simple command line screen capture utility .. screencapture - the macosx (jaguar) utility ..

The process of creating an image from what is currently displayed on the computer screen is generally known as "screen capture" or a "screenshot"

take a screenshot of the desktop window after 5 seconds, and display it

 import -pause 5 -window root desk.png; display desk.png
 scrot -d 5 desk.png; display desk.png    ##(the same)

Using a pause like this before taking the screenshot, allows you to rearrange the windows on the desktop if you so desire.

take a screen shot of the whole screen and save it to 'test.png'

 scrot test.png

take a low-quality screen shot in 'test.png' and display it

 scrot -q 10 test.png; feh test.png

on macosx capture the whole screen as a jpg image and display it

 screencapture -tjpg test.jpg; open test.jpg

make an image of the currently focussed window and save as 'test.jpg'

 scrot -u test.jpg

capture a window, with its window frame, save in the file 'i.png'

 import -frame i.png 

After typing this, left-click on the window you want to capture. In other words the technique above, is an 'interactive' technique for capturing an image of a screen

Meta data


....

command to change the exif date time of a image

 exiftool -DateTimeOriginal='2009:01:01 02:03:04' file.jpg

set timestamp in exif of a image

 exiv2 -M"set Exif.Photo.DateTimeOriginal `date "+%Y:%m:%d %H:%M:%S"`" filename.jpg

Image information


....

show the width x height of the image "bird.png"

 identify -format "%wx%h" bird.png  ##(prints "16x20" for example)

rename all "png" files, so that the name includes the width and height

 for f in *.png; do rename "s/\./-"$(identify -format "%wx%h" $f)"./" $f; done

Optimizing images


....

optimize png images

 debian: optipng

image format notes .. png, is a lossless format, has transparency .. jpg, highly compressable, lossy, no transparency .. gif, animations possible, compressed, ..

Transforming images


....

reduce in size all jpeg images in the folder by 50%

 mogrify -resize 50% *.jpg    ##(the images are changed)

reduce by 50% an image and save the reduced image as "rose2.jpg"

 convert rose.jpg -resize 50% rose2.jpg

rotate an image by 90 degrees where the height exceeds the width

 mogrify -rotate '90<' image.jpeg

rotate an image where the width exceeds the height

 mogrify -rotate '90>' image.jpeg

convert an image to black and white

 mogrify -monochrome colourful.jpeg

add a grey border 2px wide and 4px high around an image

 mogrify -border 2x4 cat.jpg
 mogrify -frame 8x8 cat.jpg    ##(a bevelled border)

Converting image formats


....

convert the jpeg image "rose.jpg" to the "png" format.

 convert rose.jpg rose.png

convert all png images in the current folder to the jpeg format

 mogrify -format jpg *.png    ##(a copy of the images is created)

convert a large number of "gif" images to the "png" format

 find . -name "*.ico" | xargs mogrify -format png 
##(this is very fast)

Batch resize all images in the current directory

 mogrify -resize 800\> *

Animation


....

# frame numbers start at 0

show information for all the frames in an animation

 identify tree.ico

display how many frames an animation has

 identify tree.gif | wc -l

extract the 2nd frame from an animated gif and save to "new.gif"

 convert animation.gif[frame1] new.gif

Resizing images


....

Resize photos without changing exif

 mogrify -format jpg -quality 80 -resize 800 *.jpg

resize(1/2) the image using imagemagick

 convert -resize 50%x50% image{,_resize}.jpg

crop a 32x32 pixel block starting at pixel (16,16) and save to "new.gif"

 convert tree.gif -crop 32x32+16+16 new.gif

crop a 20 pixel vertical strip starting at the top left hand corner

 convert tree.png -crop 20x0+0+0 new.png 

create a 1 row montage with a gap in the middle

 montage medical.gif null: present.gif -tile x1 -geometry +2+2 new.jpg

create a montage with the file name under the image, silver background

 montage -label '%f' x.png -font Arial -pointsize 30 -geometry +0+0 -background silver xy.png

rename images according to exif or jpeg metadata

 exiv2 rename *.jpg

draw a label at the bottom of the image in a grey rectangle (superimposed)

 convert tree.png -fill '#0008' -draw 'rectangle 5,128,114,145' \
   -fill white -annotate +10+141 'Tree' new.png 

Images of text


....

create an image, size 165x70, of text "Tree" with text centered in image

 convert -fill blue -font Helvetica -size 165x70  -pointsize 24 \
 -gravity center label:Tree tree.gif

create an image of text, with text font size auto-fitted to image size

 convert -fill blue -font Candice -size 165x70 label:Tree new.gif; feh new.gif

create an image label from text from standard input

 echo "hello!" | convert -fill blue label:@- new.gif; feh new.gif

Compressing images


....

reduce the quality (& file size) of an image, and save in "tree-80.jpg"

 convert tree.jpg -quality 80% tree-80.jpg

reduce a jpeg even more

  -sampling-factor 2x1

Visual art


The open clip art library.

@@ http://openclipart.org/media/tags a tag cloud of open clip art

packages .. openclipart .. openclipart-png .. create-resources - brushes, gradients etc ..

Ascii art


....

a demonstration of the aview tool

 bb

ascii art tools .. aview - .. cadubi - an ascii art editor .. aewan - an ascii art editor .. textdraw - create ascii geometric figures .. figlet - create ascii text banners .. cowsay - create ascii pictures of cows with speach bubble .. banner - show a large ascii banner text ..

Outputs files with ascii art in the intended form.

 iconv -f437 -tutf8 asciiart.nfo

display a text banner with a 50 character width

 banner -w50 'hello'

draw a box with an ascii-art dog design around the text 'hello'

 echo hello | boxes -d dog

choose from a set of ascii art pictures with speech bubbles

 cowsay -f tux 'hello'

A death cow thinking in your fortune cookie

 fortune -s -c -a | cowthink -d -W 45

view a video using only 'ascii art'

 mplayer -vo aa 

bulk dl files based on a pattern

 curl -O http://hosted.met-art.com/generated_gallery/full/061606AnnaUkrainePasha /met-art-free-sample-00[00-19].jpg

create ascii art pictures with various colours

 cadubi

pass a backslash character as an argument to figlet

 figlet $'\\' 

write 'chapter 1' as ascii bubble text

 figlet "Chapter 1"

pass a form feed character followed by a pilcrow sign character (octal character code 266) to figlet

 echo $'\f\266' 

plot the curve 'sin(x)' as ascii text

 echo "set terminal dumb; plot sin(x)" | gnuplot -persist

an ascii art font for your readme text

 toilet -f big ReadMe

vertically scrolling ascii banner text

 while [ 1 ]; do banner 'scroll! ' | while IFS="\n" read l; do echo "$l"; sleep 0.02; done; done

Patterns and tilings


....

kali

Charts and graphs


the 'pic' and 'grap' groff preprocessors debian: rlplot, ygraph, dia

graphing tools .. dot .. gnuplot .. picviz - plotter for parallel co-ordinates ..

@@ graphviz a package with tools, including 'dot' for drawing network type graphs

Create package dependency graph

 apt-cache dotty PKG-NAME | dot -Tpng | display

Gnuplot


....

@@ http://bumble.sf. href="net/books/gnuplot/gnuplot-book.txt">net/books/gnuplot/gnuplot-book.txt A booklet about the gnuplot tool, in the same format as the current booklet.

This section is only meant as a brief demonstration of what gnuplot is capable of. For more detailed information please consult the booklet mentioned above.

create an image of a sin curve graph and display it

 echo "set t png; set o 'o.png'; p sin(x)" | gnuplot; feh o.png

plot a parabola from the bash shell

 echo 'plot x**2' | gnuplot -persist

plot some data from the command line

 echo 'plot "list.txt" using 2:xticlabels(1)' | gnuplot -persist

show the graph of 'parabola' curve (x to the power of 2)

 gnuplot     
 plot x**2  
 quit      

Dot


....

Generate a graph of package dependencies

 apt-cache dotty apache2 | dot -T png | display

Flow charts and figures


for drawing flow charts and other figures use

 xfig

Video


please see http://bumble.sf. href="net/books/linux-video/linux-video-book.txt">net/books/linux-video/linux-video-book.txt for more complete information about using video with the linux operating system.

Log files


In Unix and Linux 'log' files are used by software to record things that it has done or happened to it. In particular, server software makes great use of log files. These log files are always stored as plain text.

Truncate logs in unix

 logs=$(find . -name *.log);for log in $logs; do cat /dev/null > $log;done

Get all possible problems from any log files

 grep -2 -iIr "err\|warn\|fail\|crit" /var/log/*

Follow the most recently updated log files

 ls -drt /var/log/* | tail -n5 | xargs sudo tail -n0 -f

Quickly analyze apache logs for top 25 most common IP addresses.

 cat $(ls -tr | tail -1) | awk '{ a[$1] += 1; } END { for(i in a) printf("%d, %s\n", a[i], i ); }' | sort -n | tail -25

Extract XML from an otherwise plain text log file

 sed -n '//,/<\/Tag>/p' logfile.log

Convert the output of one or more (log, source code ...) files

 enscript -E --color -t "title" -w html --toc -p /PATH/to/output.html /var/log/*log

tail, with specific pattern colored

 tail -F file | egrep --color 'pattern|$'

Watch several log files in a single window

 multitail /var/log/messages /var/log/apache2/access.log /var/log/mail.info

A robust, modular log coloriser

 ccze

useful tail on /var/log to avoid old logs or/and gzipped files

 tail -f *[!.1][!.gz]

Export log to html file

 cat /var/log/auth.log | logtool -o HTML > auth.html

follow the end of the log file 'tcp.log' showing new data as it enters

 less +F tcp.log   ##(press [control] c to stop following)
 tail -f tcp.log   ##(the same, but you cant move up and down the file) 
 tailf tcp.log     ##(the same)

Monitor logs in Linux using Tail

 find /var/log -type f -exec file {} \; | grep 'text' | cut -d' ' -f1 | sed -e's/:$//g' | grep -v '[0-9]$' | xargs tail -f

Ms word documents


extract plain text from MS Word docx files

 unzip -p some.docx word/document.xml | sed -e 's/<[^>]\{1,\}>//g; s/[^[:print:]]\{1,\}//g'

find and grep Word docs

 find . -iname '*filename*.doc' | { while read line; do antiword "$line"; done; } | grep -C4 search_term;

view a microsoft word document

 wv

convert a ms word document to text

 antiword, catdoc

display the 'list.doc' file as plain text

 catdoc list.doc | less

convert the Word file 'resume.doc' to LaTeX

 word2x -f latex resume.doc ##(writes a new file, 'resume.ltx',)

convert all of the '.DOC' Word files in the current directory to LaTeX files with maximum line widths of 40 characters

 word2x -f latex -w 40 *.DOC 

convert the Word file 'resume.doc' to a plain text file called 'resume'

 word2x -f text resume.doc resume 

search the text of the Word file 'resume.doc' for the string 'linux' regardless of case

 word2x resume.doc - | grep -i linux 

Postscript documents


Postscript is a document format very well suited to being printed. However it is more difficult to view than the adobe pdf format (since the acrobat viewer is more widely installed on Microsoft Windows computers)

view a ps file

 evince

view a postscript file

 ghostview file.ps
 gv file.ps       ##(the same)

view a postscript file as plain text

 ps2ascii book.ps | less

Converting postscript to other formats


....

convert a postscript document to the pdf format

 ps2pdf eg.ps   ##(a file called "eg.pdf" is created)

convert a postscript file to plain text

 ps2ascii book.ps book.ps.txt

Pdf documents


debian packages: xpdf, xpdf-utils

@@ http://www.foolabs.com/xpdf/ home page for the pdf-utils programs

useful pdf programs .. evince - view a pdf or postscript file .. xpdf - view a pdf file .. pdftops - convert to postscript .. pdftotext - convert or view a pdf file as plain text .. pdfinfo - display lots of information about a pdf document .. pdfimages - extract images from pdffiles .. pdffonts - show what fonts are used in a pdf document .. pdftoppm - .. xpdfrc - .. pdfcrack - PDF files password cracker .. pdfjam - collection of PDF document handling utilities .. pdftk - A useful tool for manipulating PDF documents .. pdftohtml - Translates PDF documents into HTML format ..

Merge *.pdf files

 gs -q -sPAPERSIZE=letter -dNOPAUSE -dBATCH -sDEVICE=pdfwrite -sOutputFile=out.pdf `ls *.pdf`

Merge several pdf files into a single file

 pdftk $* cat output $merged.pdf

create a pdf of the file 'test.txt'

 enscript test.txt -o - | ps2pdf - ~/tmp/test.pdf 

create a pdf of a directory listing of the current folder

 ls | enscript -o - | ps2pdf - ~/tmp/ls.pdf

Convert images to a multi-page pdf

 convert -adjoin -page A4 *.jpeg multipage.pdf

Get pages number of the pdf file

 pdfinfo file.pdf | awk /Pages/

Save man pages to pdf

 man -t man | ps2pdf - > man.pdf

separate a pdf document into single pages and report its data

 pdftk mydoc.pdf burst

Merge Two or More PDFs into a New Document

 pdftk 1.pdf 2.pdf 3.pdf cat output 123.pdf

Optimize Xsane PDFs

 gs -q -sPAPERSIZE=a4 -dNOPAUSE -dBATCH -sDEVICE=pdfwrite -sOutputFile=test.pdf multipageproject.pdf

Merge several pdf files into a single file

 gs -q -sPAPERSIZE=a4 -dNOPAUSE -dBATCH -sDEVICE=pdfwrite -sOutputFile=out.pdf a.pdf b.pdf c.pdf

Remove security limitations from PDF documents using ghostscript

 gs -q -dNOPAUSE -dBATCH -sDEVICE=pdfwrite -sOutputFile=OUTPUT.pdf -c .setpdfwrite -f INPUT.pdf

Viewing pdf documents


....

view a pdf file as plain text

 pdftotext file.pdf - | less

view a compressed pdf document

 zxpdf book.pdf.gz

view a pdf document

 xpdf book.pdf       ##(xpdf appears much faster than "acroread")
 acroread book.pdf   ##(its easier to select text in acroread) 

view the 10th page of a pdf document with a zoom factor of 200%

 xpdf -z 200 book.pdf 10 

view a pdf document in "continuous" mode starting at the 5th page

 xpdf -cont book.pdf 5  ##(one can scroll smoothly through the whole file)

view a pdf document in full-screen mode

 xpdf -fullscreen book.pdf ##(the -z zoom option doesnt work with fullscreen)

view a pdf document with the colours reversed (usually white text on black)

 xpdf -rv book.pdf

Analyse pdf documents


....

extract the images from the file and save them in jpeg format

 pdfimages -j report.pdf

display information about a pdf document

 pdfinfo book.pdf

show how many pages a pdf document has

 pdfinfo book.pdf | sed -n "/Pages:/s/^ *Pages: *//p"

show the page size of a pdf document

 pdfinfo book.pdf | grep -i "Page size:" 

show what fonts are used in a pdf document

 pdffonts book.pdf

Editing pdf files


....

pdfedit

Converting pdf to images


....

@@ http://www.medicalnerds.com/batch-converting-pdf-to-jpgjpeg-using-free-software/ some hints on converting to another format

The 'convert' tool forms part of the imagemagick suite of tools

convert 'file.pdf' to a low quality 'png' format image

 convert file.pdf file.png

convert 'file.pdf' to a high quality 'png' format image

 convert -density 300 file.pdf file.png

convert 'file.pdf' to a high quality 'jpeg' format image

 convert -density 300 file.pdf file.jpg

convert 'file.pdf' to a high quality jpeg, cropping all whitespace

 convert -density 300 -trim file.pdf file.jpg
 convert -density 300 -trim +repage file.pdf file.jpg  ##(the diff?)

Note: if the pdf file contains page numbers, then the -trim function will probably not work as well as you were hoping.

Convert PDF to JPEG using Ghostscript

 gs -dNOPAUSE -sDEVICE=jpeg -r144 -sOutputFile=p%03d.jpg file.pdf

Pdf to postscript


....

make a copy of 'file.pdf' in postscript format called 'file.ps'

 pdftops file.pdf

Pdf to text


....

convert a pdf document to plain text (without any formatting)

 pdftotext file.pdf  ##(a file called "file.txt" is created)

convert the pdf file "file.pdf" to a plain text file "ouput.txt"

 pdftotext file.pdf output.txt 
 ps2ascii file.pdf output.txt  ##(more or less the same)

pstotext

From other formats


....

convert a web page into a pdf

 touch $2;firefox -print $1 -printmode PDF -printfile $2

The info help reader


Nice info browser

 pinfo

The info reader.

For some incomprehensible reason, the www.gun.org people decided that they didnt like "man" pages and so they invented a whole new help document reader called "info". Its a nasty little second rate hypertext viewer which forces you to learn a whole new set of arbitrary keystrokes in order to move around the document (maybe they're "emacs" keystrokes, who knows). Whats more, instead of just calling "pages" pages, info calls pages "nodes" to try and be as pretentious as it possibly can. But anyway, you may have to use it.

view the "info" help document for Latex

 info latex

Some "info" reader commands .. [space], move forward a page .. [del], move back a page, or to the previous node if at the top of the page .. q, exit the Info document viewer .. ?, show some keystrokes which are available .. h, show an "info" tutorial, if available .. [tab], go to the next link on the page .. [enter], jump to the link under the cursor .. ???, how to go to previous link is anyones guess [M-TAB] ??? .. l, go to the last page that was viewed (like a browser "back button") .. n, open the next page in the document .. p, open the previous page in the document .. u, go "up" one level in the document (usually to a table of contents) .. t, go to the main table of contents for the document .. d, view a list of all Info documents .. b, go to the top of the current page .. e, go to the end of the current page .. m, type the name of a link to jump to (can type partial names) .. i, type the name of any node to jump to .. /, search for a string in the current page ..

Man pages


@@ http://www.schweikhardt.net/man_page_howto.html @@ http://babbage.cs.qc.edu/courses/cs701/Handouts/man_pages.html

'man' (manual) pages are the traditional Unix way to document a program, command or package. They are text files with simple 'markup' codes which are then processed by the troff tool.

Colorful man

 /usr/bin/man man | /usr/bin/col -b | /usr/bin/iconv -c | view -c 'set ft=man nomod nolist nospell nonu

see the conventions for writing a man page

 man 7 man

view a compressed man page in section 1 in its raw plain text format

 zcat /usr/share/man/man1/zless.1.gz | less
##(this may be useful if you wish to write a man page)

see where the 'man' command searches for man pages

 manpath

list all the directories in the man pages path

 ls $(manpath | tr ':' ' ')

man sections .. 1, user commands executable from a shell .. 2, kernel system calls .. 3, library functions and calls (such as libc) .. 4, special files such as in /dev .. 5, file formats and conventions (such as /etc/password) .. 6, games .. 7, conventions and miscellaneous, file system layout. .. 8, system management commands such as like mount(8) .. 9, kernel routines. this is an obsolete manual section. ..

automatically generate man pages from C, C++, Java, Python ... source code

 doxygen   ##(doxygen can also generate documentation in html, latex etc)

a cgi script for generating html man pages

 man2html

save the word count tool (wc) man page as plain text

 man wc | col -b > wc.txt

Documentation systems


@@ http://www.chiark.greenend.org.uk/~sgtatham/halibut/

groff texinfo docbook, halibut

Source code documentation systems


....

doxygen javadoc pod perl documentation

Xml


check if an xml or html document is well formed (valid)

 curl -s 'http://bashcurescancer.com' > bcc.html; xmlwf bcc.html
##('xmlwf' is part of the 'expat' package)

Editing text


In the traditional unix world, the choice of text editor is often seen as a choice between vim and emacs. But many other good choices exist.

@@ tea - ? @@ sam An editor written by Rob Pike which is in turn used by various veteran unix forefathers. Sam makes more use of the mouse. @@ acme Another editor by Rob Pike. @@ joe 'joes own editor' possibly the simplest text editor to use ... @@ nano, pico small reasonably easy to use text editors originating from the pine email program. @@ vim 'visual editor improved' this is a programmers' editor for people who write a great deal of text and preferibly can type without looking at the keyboard. It is strange and annoying for new users. @@ emacs 'editing macros' capable of just about anything but dont use it, because I think its bad.

graphical text editors .. gedit .. kate ..

Batch editing text files


....

Batch editing of text files involves editing several or many text files at once, usually in a non-interactive way.

  1. use vim with the "argdo" function
  2. use perl with the "-i" switch
  3. some versions of "sed" do not have the "-i" switch, so perl must be used
## Before doing a "batch" edit of text files, it is probably a good ## idea to check what changes will take place without doing anything

add the word "green" after the 500th line of the file "report.txt"

 echo -e "500a\ngreen\n.\nw\nq\n" | ed report.txt
 (echo "500a"; echo "green"; echo -e "w\nq\n") | ed report.txt ##(the same)

In all files with a ".txt" extension replace "Daniel" with "Dan"

 grep -l "Daniel" $(find . -name "*.txt") | xargs sed -n "s/Daniel/Dan/p" 
##(the original files are left unchanged)

# using the sed -n and "p" commands, only changed lines will be displayed

In all files with a ".txt" extension replace "Daniel" with "Dan"

 grep -l "Daniel" $(find . -name "*.txt") | xargs sed -i.bak "s/Daniel/Dan/g" 

In "html" files, NOT containing the text "Notes:", add a line with "Notes:"

 grep -L "Daniel" $(find . -name "*.html") | xargs sed -i.bak "$s/$/\nNotes:/" 
##(a copy of the changed files is made with an extension ".bak") ##(all files in the current folder and subfolders are affected)

replace the word "big" with small in all files in this and subfolders

 perl -p -i.bak -e 's/\bbig\b/small/g' $(grep -ril big *)

Unix directory structure


Unix has a standard folder hierarchy in which each folder is designed to serve a specific purpose and contain certain sorts of files.

search google: unix file system standard linux file system standard FS-Stnd

show detailed information about what each folder is used for

 man hier

A very incomplete summary of the Unix directory structure .. .. /usr/bin/ - executable files (binary, shell scripts etc) .. /usr/sbin/ - secure executable files .. /usr/local/bin - executables only on the current machine .. /usr/share/ - files used by many programs, dictionaries, icons etc .. /usr/share/doc - documentation files .. /var/ - data files which change regularly (eg log files) .. /dev/ - peripheral devices (sound cards, usb drives etc) .. /etc/ - configuration files for programs ..

Fonts


Refresh the cache of font directory

 sudo fc-cache -f -v

show which fonts are installed

 fc-list | cut -d ':' -f 1 | sort -u

show what fonts are installed for the japanese language

 fc-list :lang=ja

Installing True-Type fonts

 ttmkfdir mkfontdir fc-cache /usr/share/fonts/miscttf

font tools .. gucharmap - browse the fonts on the computer .. waterfall - show characters from a font in all sizes ..

File systems


Force file system check

 touch /forcefsk

Backup files incremental with rsync to a NTFS-Partition

 rsync -rtvu --modify-window=1 --progress /media/SOURCE/ /media/TARGET/

@@ gparted a graphical partitioner and file-system formatter @@ fdisk a command line tool

Check the age of the filesystem

 df / | awk '{print $1}' | grep dev | xargs tune2fs -l | grep create

currently mounted filesystems in nice layout

 column -t /proc/mounts

Converts ext2 to ext3

 tune2fs -j /dev/sdX

migrate existing ext3 filesystems to ext4

 tune2fs -O extents,uninit_bg,dir_index /dev/yourpartition

Show the UUID of a filesystem or partition

 sudo vol_id -u /dev/sda1

resize a mounted ext3 file system

 v=/dev/vg0/lv0; lvextend -L+200G $v && resize2fs $v

Iso files


....

convert .daa to .iso

 poweriso convert image.daa -o image.iso -ot iso

Create a CD/DVD ISO image from disk.

 readom dev=/dev/scd0 f=/path/to/image.iso

Mount a .iso file in UNIX/Linux

 mount /path/to/file.iso /mnt/cdrom -oloop

convert .bin / .cue into .iso image

 bchunk IMAGE.bin IMAGE.cue IMAGE.iso

Mount and umount iso files

 function miso () { mkdir ~/ISO_CD && sudo mount -o loop "$@" ~/ISO_CD && cd ~/ISO_CD && ls; } function uiso () { cd ~ && sudo umount ~/ISO_CD && rm -r ~/ISO_CD; }

Partitions


....

A partition is a way of dividing a hard-disk or other storage media into several logical 'disks'.

Mount a partition from within a complete disk dump

 INFILE=/path/to/your/backup.img; MOUNTPT=/mnt/foo; PARTITION=1; mount "$INFILE" "$MOUNTPT" -o loop,offset=$[ `/sbin/sfdisk -d "$INFILE" | grep "start=" | head -n $PARTITION | tail -n1 | sed 's/.*start=[ ]*//' | sed 's/,.*//'` * 512 ]

File system types


....

@@ http://en.wikipedia.org/wiki/Comparison_of_file_systems a good chronological table of file systems @@ http://www.osnews.com/story/9681/The_vfat_file_system_and_Linux an article about the vfat file system

a few common file-system types .. fat32 - created 1996, by microsoft, appeared in windows 95b .. vfat - ? .. ntfs v5.1 - created 2001 by microsoft and used in windows xp .. ntfs v6 - 2006 by microsoft for windows vista .. ext2 - created in 1993 and used by linux and hurd .. ext3 - created 1999 used in linux .. reiserfs - created 2001 by namesys, used in linux .. google file system - created 2003 by google for linux .. gfs2 - 2006 by red hat for linux .. ext4 - 2006 for linux ..

view all mounted file system types

 df -T

Mounting and unmounting filesystems


....

Mount proc

 mount -t proc{,,}

currently mounted filesystems in nice layout

 mount | column -t

Mount a temporary ram partition

 mount -t tmpfs tmpfs /mnt -o size=1024m

Mount folder/filesystem through SSH

 sshfs name@server:/path/to/folder /path/to/mount/point

Mount a partition from within a complete disk dump

 lomount -diskimage /path/to/your/backup.img -partition 1 /mnt/foo

umount all nfs mounts on machine

 umount -a -t nfs

mount directories in different locations

 mount --bind /old/directory/path /new/directory/path

Symbolic links


....

Find broken symlinks and delete them

 find -L /path/to/check -type l -delete

Find broken symlinks

 find . -type l ! -exec test -e {} \; -print

Propagate a directory to another and create symlink to content

 lndir sourcedir destdir

show the status of all symlinks in the current folder

 symlinks -r $(pwd)

List all symbolic links in current directory

 find /path -type l

Find dead symbolic links

 find . -type l | perl -lne 'print if ! -e'

Show the disk usage for files pointed to by a symbolic link

 find /usr/lib -maxdepth 1 -type l -print0 | xargs -r0 du -Lh

Eliminate dead symlinks interactively in /usr/ recursevely

 find /usr/ -type l ! -xtype f ! -xtype d -ok rm -f {} \;

Get the canonical, absolute path given a relative and/or

 readlink -f ../super/symlink_bon/ahoy

Directories


Directories, which are also known as 'folders' are a way of organising files in a heirarchical kind of way.

Push your present working directory to a stack that you can pop

 pushd /tmp

display the folder below the current one

 tree -dL 1 | less

display the whole directory tree below the current folder

 tree -d | less

another directory tree

 find . -type d -print | sed -e 's;[^/]*/;..........;g'| awk '{print $0"-("NR-1")"}'

Go to parent directory of filename edited in last command

 cd `dirname $_`

Convert the contents of a directory listing into a colon-separated

 find . -name '*' -printf '%f:'

find and delete empty directories recursively

 find . -depth -type d -empty -exec rmdir -v {} +

Find the 20 biggest directories on the current filesystem

 du -xk | sort -n | tail -20

Get the size of all the directories in current directory

 du --max-depth=1

Replicate a directory structure dropping the files

 for x in `find /path/ -type d | cut -b bytesoffoldername-`; do mkdir -p newpath/$x; done

Go to the next sibling directory in alphabetical order

 for d in `find .. -mindepth 1 -maxdepth 1 -type d | sort`; do if [[ `basename $d` > `basename $PWD` ]]; then cd $d; break; fi; done

maybe the quickest way to get the current program name minus the path

 programname="${0##*/}"

Huh? Where did all my precious space go ?

 ls -la | sort -k 5bn

fast access to any of your favorite directory.

 alias pi='`cat ~/.pi | grep ' ; alias addpi='echo "cd `pwd`" >> ~/.pi'

Remove empty directories

 find . -type d -empty -delete

Count the total number of files in each immediate subdirectory

 find . -type f -printf "%h\n" | cut -d/ -f-2 | sort | uniq -c | sort -rn

Browsing folders


....

Start a file browser in the current directory

 screen -d -m nautilus --no-desktop `pwd`

File browser in the dwm minimalist window manager

 xdg-open $(ls -1 . | dmenu)

browse the current folder with the gnome file manager

 nautilus

Restart nautilus, the gnome file manager

 killall nautilus

Analysing directories


....

list hidden files and folders in the current folder

 ls -d .*

List only the directories

 ls -l | egrep ^d

Sort the size usage of a directory tree by gigabytes, kilobytes,

 dh() { du -ch --max-depth=1 "${@-.}"|sort -h }

disk usage tools .. du .. firelight - a graphical tool ..

Folder size


....

Watch the size of a directory using figlet

 watch -n1 "du -hs /home/$USER | cut -f1 -d'/' | figlet -k"

show the total size of the current directory

 du -sh

display disk usage for the current folder in a readable form

 du -sh *    ##(displays values such as 2G, 43K, 12M etc)

show available disk space in a readable form (2G, 42K etc)

 df -h

Print the 10 deepest directory paths

 find . -type d | perl -nle 'print s,/,/,g," $_"' | sort -n | tail

Comparing folders


....

Recursively compare two directories and output their differences

 diff -urp /originaldirectory /modifieddirectory

Making folders


....

make directory tree

 mkdir -p work/{d1,d2}/{src,bin,bak}

make 100 directories with leading zero, 001...100, using bash3.X

 mkdir $(printf '%03d\n' {1..100})

Changing folders


....

Go up multiple levels of directories quickly and easily.

 alias ..="cd .."; alias ...="cd ../.."; alias ....="cd ../../.."

go to the previous directory

 cd -

go to the users home directory

 cd

go to the previous sibling directory in alphabetical order

 cd ../"$(ls -F ..|grep '/'|grep -B1 `basename $PWD`|head -n 1)"

Copying folders


....

Recursively move folders/files and preserve their permissions and

 cd /source/directory; tar cf - . | tar xf - -C /destination/directory

copy/mkdir and automatically create parent directories

 cp --parents /source/file /target-dir

create a copy of the 'book' folder as 'book-bak'

 cp -r book book-bak 

copy a folder, but only newer files and display all files copied

 cp -vur book book-bak 

copy a directory to another, only where files are newer

 cp -ru sourcefolder targetfolder 
##(or use the 'rsync' tool instead of 'cp')

make a folder and create parent folders if necessary

 mkdir -p ./new/path/  
##(if the folder './new' doesnt exist it will also be created)

Deleting folders


....

take a look to command before action

 find /tmp -type f -printf 'rm "%p";\n'

remove a directory

 rm -r folder      ##(where the dir is not empty: be careful)
 rmdir

make an archive of a the folder 'tree' excluding .exe files

 tar -cvz --exclude *.exe -f folder.tar.gz tree 

File type conversions


File Type Conversion Programs .. .. dvips, tex/latex dvi → postscript .. ps2pdf, postscript → adobe pdf .. ghostscript, postscript → gif image .. groff, groff-pic → postscript ..

Files


Create a bunch of dummy files for testing

 touch {1..10}.txt

Empty a file

 > foobar.txt

list all open files

 lsof   ##(this lists all open file, pipes, sockets etc)
 lsof -i TCP  ##(show open TCP sockets)

show the number of open files for the root user

 lsof | grep ' root ' | awk '{print $NF}' | sort | uniq | wc -l
 lsof | grep ' root ' | awk '{print $NF}' | sort -u | wc -l   ##(the same)

find out what sort of file is 'mys' (i.e. what is the file type)

 file mys 

show the last modification time of a file or folder

 date -r file

File size


....

Alternative size (human readable) of files and directories

 du -ms * | sort -nk1

Another way to calculate sum size of all files matching a pattern

 find . -iname '*.jar' | xargs du -ks | cut -f1 | xargs echo | sed "s/ /+/g" | bc

Analysing files


....

Find files that were modified by a given command

 strace 

Sum size of files returned from FIND

 find [path] [expression] -exec du -ab {} \; | awk '{total+=$0}END{print

list files with last modified at the end

 alias lrt='ls -lart'

Find writable files

 find -writable

list and sort files by size in reverse order (file size in human

 ls -S -lhr

Get file access control list

 getfacl /mydir

Show latest changed files

 ls -ltcrh

Sort files by size

 ls -l | sort -nk5

Find Duplicate Files (based on size first, then MD5 hash)

 find -not -empty -type f -printf "%s\n" | sort -rn | uniq -d | xargs -I{} -n1 find -type f -size {}c -print0 | xargs -0 md5sum | sort | uniq -w32 --all-repeated=separate

File names


....

Substitute spaces in filename with underscore

 ls -1 | rename 's/\ /_/'

Copying files


....

tools for copying files .. cp - the standard unix copy tool .. rsync - copy files across the network .. install - copies files and sets the permissions ..

find and copy scattered mp3 files into the users images folder

 find ~ -iname '*.mp3' -type f -print0 | xargs -I{} -0 cp {} ~/images 

prevents replace an existing file by mistake

 set -o noclobber

Copy all documents PDF in disk for your home directory

 find / -name "*.pdf" -exec cp -t ~/Documents/PDF {} +

Create subdirectory and move files into it

 (ls; mkdir subdir; echo subdir) | xargs mv

File paths


....

For a $FILE, extracts the path, filename, filename without

 FILENAME=`echo ${FILE##*/}`;FILEPATH=`echo ${FILE%/*}`;NOEXT=`echo ${FILENAME%\.*}`;EXT=`echo ${FILE##*.}`

Get the full path to a file

 realpath examplefile.txt

get the top 10 longest filenames

 find | sed -e "s/^.*\///" | awk ' BEGIN { FS=""} { print NF " " $0 } ' | sort -nrf | head -10

Get the absolute path of a file

 absolute_path () { readlink -f "$1"; };

Searching for files and directories


This section covers techniques to search for and find files and folders on a Unix system. This mainly involves using the "find" program from the command line. This program is flexible and useful, but its syntax can initially seem to be unnecessarily complicated. For example "find filename" will not work.

The 'find' tool has an enormous number of options and operators to construct truly abstruse find commands.

@@ http://www.softpanorama.org/Tools/Find/find_mini_tutorial.shtml a find tutorial

Graphical search tools: Beagle, Google desktop

## for some reason find / -name "*.c" doesnt search all subfolders for me ## for this reason I use the syntax ## find /* ....

do a full file listing of every file found with locate

 locate searchstring | xargs ls -l

allow the user to select a file and print the chosen file

 zenity --file-selection --title 'select a file'

Find all directories on filesystem containing more than 99MB

 du -hS / | perl -ne '(m/\d{3,}M\s+\S/ || m/G\s+\S/) && print'

find all files with "tree" in the name in the current folder and subfolders

 find . -iname "*tree*"

Searching files

 find /dir/ -name *name*

best command for searching files

 find / -name \*string\*

Search through files, ignoring .svn

 find . -not \( -name .svn -prune \) -type f -print0 | xargs --null grep 

search the contents of '.c' and '.h' file types recursively

 find . -name "*.[ch]" | xargs grep "TODO"

Searching for folders


....

find all directories in the system

 find /* -type d        ##(the -print action is assumed)
 find /* -type d -print ##(the same as above)
 find / -type d         ##(on many computer the "*" may not be required)

show all folders in the current folder

 find -maxdepth 1 -type d

find all subdirectories and print the full path name

 find $(pwd) -type d | less

find all directories which begin with a '.'

 find / -type d -name .\* -print
 find /* -type d -name .\* -print  
##(for some reason the "/*" idiom is necessary on my computer)

Finding files by name


....

find all c code files on the computer

 find /* -iname "*.c"   ##(-iname does a case insensitive name search)

find files in the current and subfolder with a ".txt" filename extension

 find . -name "*.txt"   ##(this will find "index.txt" but not "index.TXT")

find all files whose names end with '.txt' or '.csv'

 find . -name "*.txt" -o -name "*.csv"  
##(-o is the logical 'or' operator)

 find . -name '*.txt' -o -name '*.csv'  
##(the same, but not always)

 find / -regex ".*\.\(xls\|csv\)"    
##(the same, but a bit messy)

find all files whose names end with '.txt' or '.csv' case insensitive

 find / -iregex ".*\.\(xls\|csv\)"       ##(finds a.CSV, b.Xls, C.csV etc)
 find . -iname "*.txt" -o -iname "*.csv" ##(the same)  

find files which have 'tree' somewhere in the path or name

 find . -wholename '*tree*'
 find . -path '*tree*'  ##(the same, but deprecated)

find all non-html files

 find . -type f ! -name "*html"

Searching for files by size


....

Sort file greater than a specified size in human readeable format

 find ./ -size +10M -type f -print0 | xargs -0 ls -Ssh1 --color

Get the 10 biggest files/folders for the current direcotry

 du -sk * |sort -rn |head

Find all the files more than 10MB, sort in descending order of

 find . -size +10240k -exec ls -l {} \; | awk '{ print $5,"",$9 }'|sort -rn > message.out

find files which are larger than 10 megabytes

 find / -size +10000000c -print
 find / -size +10M     ##(the same but shorter and more readable) 

find files which are larger than 10 kilobytes

 find / -size +10k     ##(the same but better) 

show file sizes in a readable format for all files bigger than 100k

 find . -size +100k -print | xargs du -sh | less 
##(searches this & subfolders)

 find . -size +100k | xargs du -sh | less        ##(the same)

delete all files in the current and subfolder which are bigger than 100k

 find . -size +100k | xargs rm | less   ##(rather dangerous this command!)

a bash function which finds files bigger than an amount of meg function bigger { [ -z "$1" ] && echo "usage: $FUNCNAME filesize" && return 3 find . -size +${1}M | xargs du -sh | less } ,,,

List top ten files/directories sorted by size

 du -sb *|sort -nr|head|awk '{print $2}'|xargs du -sh

Finding files by modification time


....

show files in the home folder which have been modified in the last 24 hours

 find $HOME -mtime 0

find all files which have been modified in the last 7 days

 find / -mtime -7 -print
 find /* -mtime -7 -print   ##(works better for me)

remove files in the /tmp folder which havent been modified in a week

 find /tmp -mtime +7 -exec rm -rf {} \;
 find /tmp -mtime +7 | xargs rm -rf  ##(the same but nicer and maybe faster) 

compress log files which havent been modified for a week

 find /var/log -mtime +7 -exec gzip {} \;

find files which were last modified (the data) more than 30 minutes ago

 find . -mmin +30

find files whose data was modified less than 30 minutes ago

 find . -mmin -30

files which were modified more recently than 'tree.txt'

 find . -newer tree.txt

By access time


....

find all files in the current folder tree accessed exactly 10 minutes ago

 find . -amin 10 

find files accessed more than 10 minutes ago

 find . -amin +10 

find files accessed less than 10 minutes ago

 find . -amin -10 

find files which were accessed after the file 'tree.txt' was modified

 find . -anewer tree.txt 

find files which were last accessed more than 48 hours ago

 find . -atime +2

find files which were last accessed less than 48 hours ago

 find . -atime -2

By file type


....

find all empty files and folder in this and all subfolders

 find . -empty  

find all executable files (not folders)

 find . -type f -executable

find all image files on the computer

 find / | xargs file | grep image | less

Finding text files


....

find files which contain the word 'big' in this folder and subfolders

 find . -type f -print | xargs grep -l big 
 grep -rl big *  ##(this is the same but not all greps have the 'r' option)

find all files having the phrase 'big boy' in the 'doc/books' folder tree

 grep -rl 'big boy' doc/books    ##(this is a case sensitive search) 
 grep -ril 'big boy' doc/books   ##(finds 'big boy', 'BiG Boy' etc)
 grep -sril 'big boy' doc/books  ##(dont show any error messages)
 grep -sri 'big boy' doc/books  ##(show matching lines, not just filenames)

find all ".html" files in "/etc" which do *not* contain the "

 find /etc -iname "*.html" | xargs grep -L "

find all files with names ending in '.html' or '.php' containing the word 'big'

 find / \(-name \*.html -o -name \*.php\) | xargs grep -i "big" 
##(this uses the tool 'xargs' instead of a 'for' loop)

find all text files in the current directory and all subfolders

 find . | perl -lne '-T "$_" and print "$_"' | less
 find . | perl -lne '-T $_ and print $_' | less  ##(the same, I think...)

Finding files by permissions


....

find files not readable by all users

 find -type f ! -perm -444    

find folders which are not accessible for all users

 find -type d ! -perm -111

Complex find expressions


....

Find can use logical operators on each condition. This makes it possible to construct complex, and hopefully powerful find commands.

find all files ending in '.csv' which are real files and list them

 find / -type f -name "*.csv" 
 find / -type f -a -name "*.csv"  ##(the same, '-a' means logical and)

use the 'and' logical operator

 find / -type f -a -name "*.csv"  ##(the -a is not necessary)

use the 'or ' logical operator: find files ending in .txt or .csv

 find / -name '*.txt -o -name "*.csv"  

find files ending in '.xls' or '.csv' which are real files (not folders etc)

 find / -type f \( -name "*.xls" -o -name "*.csv" \) -exec ls -l {} \;

find all files whose names dont end in '.csv'

 find . \! -name '*.csv'

Doing something with found files


....

Find also support the -exec option which performs an action on each found file, but it may be better to use 'xargs' instead for speed

 find / -type f \( -name "*.xls" -o -name "*.csv" \) -exec ls -l {} \;

the -exec option can be used to carry out an action on the found files

 find . -name "*" -exec cp "{}" img \;

delete all files whose names end in '.txt' in this and all subfolders

 find . -name '*.txt' -delete         ##(a bit dangerous, but anyway)
 find . -name '*.txt -exec rm {} \;   ##(the same, if you like typing)
 find . -name '*.txt -exec rm "{}" \; ##(better? but why)
 find . -name '*.txt | xargs rm       ##(the same, might be faster, who knows)

delete each file but ask the user for confirmation

 find . -name '*.txt' -ok rm {} \; 
 find . -name '*.txt' -ok rm "{}" \;         ##(better) 

display the file type information about each file in this folder tree

  find . -type f -exec file '{}' \;

Locating files


The 'locate' tool is faster than find, because it use an 'index' which has to be updated when the filesystem changes.

use locate,

update the file name database for the locate command

 updatedb

find all executable files in the current and subfolder with 'tree' in the name

 find . -name "*tree*" | perl -lne '-x and print'   
 find . -name "*tree*" | perl -lne '-x && print'          ##(the same)
 find . -name "*tree*" | perl -lne '-x $_ and print $_'   ##(the same)

Renaming and moving files


....

smart renaming of files

 ls | sed -n -r 's/banana_(.*)_([0-9]*).asc/mv & banana_\2_\1.asc/gp' | sh

convert filenames in current directory to lowercase

 for i in *; do mv "$i" "$(echo $i|tr A-Z a-z)"; done

recursively change file name from uppercase to lowercase (or

 find . -type f | while read f; do mv $f `echo $f |tr '[:upper:]' '[ :lower:]'`; done

Replace space in filename

 rename "s/ *//g" *.jpg

Rename .JPG to .jpg recursively

 find /path/to/images -name '*.JPG' -exec rename "s/.JPG/.jpg/g" \{\} \;

rename all '.html' files in the folder to '.php'

 rename 's/\.html$/.php/' *.html  ##(index.html will become index.php)

rename uppercase file names to lower-case

 rename 'y/A-Z/a-z/' *

show what renaming would happen but dont actually do anything

 rename -n 's/\.htm$/.html/' *.html

rename all files on the entire computer from "htm" -> "html"

 find /* -name "*.htm" | xargs rename 's/\.htm$/.html/'   
 find /* -name "*.htm" -exec rename 's/\.htm$/.html/' "{}" \;  ##(slower)

# the "xargs" version is much much faster than the "-exec" version # using "xargs -I {} cmd {} " slows down xargs alot # in this case.

Batch file name renaming (copying or moving) w/ glob matching.

 for x in *.ex1; do mv "${x}" "${x%ex1}ex2"; done

get only the first component of a Unix style file name

 f=/user/home/hh.html; echo f  | sed "s*^/**" | cut -d '/' -f 1
##(this prints "user")

Deleting files


....

delete files except some file

 find . |more |grep -v filename |xargs rm

Removes file with a dash in the beginning of the name

 rm -- --myfile

List and delete files older than one year

 find  -mtime +365 -and -not -type d -delete

Delete files if not have some extension

 ls -1 |grep -v .jpg |xargs rm

take a look at what a command will do before doing it

 find /tmp -type f -printf 'rm "%p";\n'

empty the trash folder

 alias trash="rm -fr ~/.local/share/Trash"

may be the optimal way of deleting huge numbers of files

 find /path/to/dir -type f -delete

Verbosely delete files matching specific name pattern, older than x

 find /backup/directory -name "some*" -mtime +15 | xargs rm -vf

Watch how fast the files in a drive are being deleted

 watch "df | grep /path/to/drive"

Remove a file whose name begins with a dash ( - ) character

 rm ./-filename

Erase empty files

 find . -size 0 -print0 | xargs -0 rm

Delete files older than 5 (days?)

 find /dir_name -mtime +5 -exec rm {} \

remove files and directories which havent been accessed in the last 20 minutes

 find -amin +20 -delete

remove all files with the extension ".dvi" and ".log"

 rm *.{dvi,log}  

Remove all but the 5 most recent file in a directory.

 rm `ls -t | awk 'NR>5'`    ##(the old bash syntax)
 rm $(ls -t | awk 'NR>5')   ##(the same, but new syntax)

securely erase unused blocks in a partition

 # cd $partition; dd if=/dev/zero of=ShredUnusedBlocks bs=512M; shred -vzu ShredUnusedBlocks

Copying files


....

copy lots of files from the current folder to the folder "img"

 find . -name "*" | xargs -I {} cp {} img  ##(may be faster than -exec)
 find . -name "*" -exec cp "{}" img \;  
##( "cp * img/" may give the result: file list too long) ##(find with "-exec" is much much faster than a "for" loop)

A rather slow "for" loop for copying files

 for i in *; do cp $i img/; done

Symbolic links to files


....

make a symbolic link to the mounted hard drive in the home folder

 ln -s /mnt/hda1 ~/disk
 ln -s /path/to/original /path/to/link      ##(general format)

make a link to '.bashrc' in the current folder.

 ln -s ~/.bashrc  ##(the symbolic link will be calledd 'bashrc')

delete a symbolic link

 rm link         ##(the original file is not deleted)

see where a symbolic link points to

 ls -l link

File compression


Create a tar archive using 7z compression

 tar cf - /path/to/data | 7z a -si archivename.tar.7z

backup and remove files with access time older than 5 days.

 tar -zcvpf backup_`date +"%Y%m%d_%H%M%S"`.tar.gz `find  -atime

Extract a remote tarball in the current directory without having

 curl http://example.com/foo.tar.gz | tar zxvf -

Extract tarball from internet without local saving

 wget -O - http://example.com/a.gz | tar xz

Gzip files older than 10 days matching *

 find . -type f -name "*" -mtime +10 -print -exec gzip {} \;

Zip all subdirectories into zipfiles

 for f in `find . \( ! -name . -prune \) -type d -print`; do zip $f.zip $f; done

Pack up some files into a tarball on a remote server without

 tar -czf - * | ssh example.com "cat > files.tar.gz"

backup a directory in a timestamped tar.gz

 tar -czvvf backup$(date "+%Y%m%d_%H%M%S").tar.gz /path/to/dir

Create a tar.gz in a single command

 tar cvf - foodir | gzip > foo.tar.gz

Extract tar.gz in a single command

 gunzip < foo.tar.gz | tar xvf -

Create a zip file ignoring .svn files

 zip -r foo.zip DIR -x "*/.svn/*"

 unzip -lt foo.zip | grep testing | awk '{print $2}' | xargs rm -r

Remove all files previously extracted from a tar(.gz) file.

 tar -tf  | xargs rm -r

List contents of tar archive within a compressed 7zip archive

 7z x -so testfile.tar.7z | tar tvf -

Extract neatly a rar compressed file

 unrar e file.part1.rar; if [ $? -eq 0 ]; then rm file.part*.rar; fi

Compare an archive with the file-system

 tar dfz horde-webmail-1.2.3.tar.gz

compress a file and watch the progress

 tar zcf - user | pv /bin/gzip > /tmp/backup.tar.gz

Create a self-extracting archive for win32 using 7-zip

 7z -sfx archive.exe /path/to/archive

Compression formats Benchmark

 for a in bzip2 lzma gzip;do echo -n>$a;for b in $(seq 0 256);do dd if=/dev/zero of=$b.zero bs=$b count=1;c=$(date +%s%N);$a $b.zero;d=$(date +%s%N);total=$(echo $d-$c|bc);echo $total>>$a;rm $b.zero *.bz2 *.lzma *.gz;done;done

list the contents of a .gz compressed tar archive file

 tar ztvf file.tar.gz   ##(nothing is uncompressed or extracted)

list the contents of a .bz2 compressed tar archivve file

 tar jtvf file.tar.bz2

compress stuff and show a progress bar while it is working

 tar zcf - user | pv /bin/gzip > /tmp/backup.tar.gz

uncompress a ".bz2" compressed file

 bunzip2 filename.txt.bz2

for files ending with ".Z" use "uncompress"

 uncompress file.Z

view the contents of the compressed file 'comp.gz'

 zcat comp.gz 

page through the contents of a compressed file

 zless filename 

compress a file as small as possible with gzip

 gzip --best file

browse a compressed directory

 mc      ##(the gnu midnight commander, uses 'curses')

create a compressed archive of the folder 'dir' in file 'dir.tar.gz'

 tar -cvz --exclude *.exe -f dir.tar.gz dir 
##(excludes all filenames ending with '.exe')

Compress files found with find

 find ~/bin/ -name "*sh" -print0 | xargs -0t tar -zcvf foofile.tar.gz

Create a zip archive excluding all SVN folders

 zip -r myfile.zip * -x \*.svn\*

Archives of files


A file archive is a way to combine many different files within one file, optionally compressing (reducing the size) of each file included in the "archive" file. The purpose of creating file archives is to facilitate making backups of files or for transfering files from one computer to another.

Compress archive(s) or directory(ies) and split the output file

 rar a -m5 -v5M -R myarchive.rar /home/

Split a tarball into multiple parts

 tar cf - |split -bM - .tar.

Unrar all files in a directory

 for f in *.rar;do unrar e ?$f?;done

Tar - Compress by excluding folders

 tar -cvzf arch.tgz $(find /path/dir -not -type d)

Extract tarball from internet without local saving

 curl http://example.com/a.gz | tar xz

Slightly better compressed archives

 find . \! -type d | rev | sort | rev | tar c --files-from=- --format=ustar | bzip2 --best > a.tar.bz2

Archive a directory with datestamp on filename

 tar zcvf somedir-$(date +%Y%m%d-%H%M).tar.gz somedir/

display the files contained in the "tar" archive file "archive.tar"

 tar tvf archive.tar
##(the 'v' option prints each file which is added to the archive)

append files with names ending in ".txt" to the archive file "text.tar"

 tar rvfn text.tar *.txt
 tar -rvfn text.tar *.txt   ##(this is the same, the "-" is optional)

create an archive file "new.tar" and append all "c" source files to it

 tar -cvf new.tar *.c
 tar cvf new.tar *.c        ##(the same)  

create an archive file "new.tar" of the folder /etc/lynx

 tar cvvf new.tar /etc/lynx  ##("vv" means "extra verbosity") 

extract all files from a "tar" archive file

 tar xvf myfile.tar

Backups of data and disks


A backup is essentially a copy of data which is stored in a different place from the original data, and which hopefully will combat the laws of entropy.

File backups


....

On unix-style systems, backup files have traditionally had a '~' extension.

add a backup (or any other) suffix to the file 'data'

 mv -vi data{,~}

Create a quick back-up copy of a file

 cp file.txt{,.bak}

Create a backup of the file 'fo/list' with todays timestamp

 cp fo/list{,-$(date +%Y-%m-%d)}

quickly backup or copy a file 'list.txt' with bash

 cp -bfS.bak list.txt list.txt 

remove backup files (with the '~' extension)

 find / -name *~ -delete
 locate '*~' | xargs rm    ##(faster but not as thorough) 

never rewrite a file while copying (or moving)

 cp --backup=t source.file target.file

a function to create backups with a day-time timestamp

 backup() { for i in "$@"; do cp -va $i $i.$(date +%Y%m%d-%H%M%S); done }

Folder backups


....

backup directory. (for bash)

 cp -pr directory-you-want-to-backup{,_`date +%Y%m%d`} # for bash

create a separate backup file for all files in current folder

 find . -maxdepth 1 -type f -print0 | xargs -0 -i cp ./{}{,.bak}

Remote backups


....

Backup a filesystem to a remote machine and use cstream to

 nice -n19 dump -0af - / -z9|gpg -e -r |cstream -v 1 -t 60k|ssh  "cat > backup.img"

Send a backup job to a remote tape drive on another machine over ssh

 tar cvzf - /directory/ | ssh root@host "cat > /dev/nst0"

backup and synchronize entire remote folder locally (curlftpfs and

 curlftpfs ftp://username:password@ftpserver /tmp/remote-website/ && rsync -av /tmp/remote-website/* /usr/local/data_latest && umount /tmp/remote-website

Disk backups


....

check the status of 'dd' in progress

 watch -n 10 killall -USR1 dd

Backup of a partition

 cd /mnt/old && tar cvf - . | ( cd /mnt/new && tar xvf - )

make a backup of an entire hard disk (the first one), saving to a file

  dd if=/dev/hda of=/path/to/image/file

make a backup of the second hard disk, compress it and write to a file

 dd if=/dev/hdb | gzip > /path/to/image.gz

restore a backup of the hard disk /dev/hdb

 dd if=/path/to/image of=/dev/hdb

restore a compressed backup of a hard disk

 gzip -dc /path/to/image.gz | dd of=/dev/hda 

create a backup of the master boot record and partition table of the 'hdb' disk

 dd if=/dev/hdb of=/path/to/image count=1 bs=512 

Restoring a backup


....

Restore a local drive from the image on remote host via ssh

 ssh user@server 'dd if=sda.img' | dd of=/dev/sda

Transfering files


A wide range of programs are available to transfer files between different computers on a network, and each has its own particular speciality

remote file transfer programs .. ftp, unencrypted file transfer (older) .. sftp, encrypted file transfer with user interaction .. scp, encrypted file transfer, non-interactive (faster than sftp) .. wget, http recursive file download, suitable for a bad connection .. curl, like wget but with some more options .. rsync, an efficient and flexible remote copy program .. bittorent, distributed file transfer ..,

share files through the http port 80

 nc -w 5 -v -l -p 80 < file.ext

transfer files using commands in 'ftp.txt' to server.net

 sftp -bc -pw password -l username -b ftp.txt server.net

copy a file over the secure shell 'ssh' without 'scp'

 ssh HOST cat < LOCALFILE ">" REMOTEFILE

share the current directory tree (via http) at http://$HOSTNAME:8000/

 python -m SimpleHTTPServer

Scp


....

The secure copy program is allegedly faster than 'sftp' mode, and uses ssh encryption. For transfering entire folders the 'rsync' program should probably used instead. Scp appears incapable of transfering 2 different files to 2 different folders. For that, see sftp. Without the '-r' switch Scp wont transfer folders.

transfer 'book.txt' to the books folder of a sourceforge site server

 scp books.txt user,project@web.sf.net:htdocs/books/

upload files 'f' and 'g' into the webfolder on a sourceforge server

 scp f g user,project@web.sourceforge.net:htdocs/

download files 'f' and 'g' from a sourceforge server to this folder

 scp user,project@web.sourceforge.net:"htdocs/f htdocs/g" .

upload (overwriting) all files in the current folder to the folder 'doc/books/' to the 'eva' account on the server 'far.net'

 scp * eva@far.net:doc/books/

upload and overwrite all '.txt' files to the 'doc/' folder on far.net

 scp *.txt eva@far.net:doc/

scp a good script from host A which has no public access to host

 cat nicescript |ssh middlehost "cat | ssh -a root@securehost 'cat > nicescript'"

bash function to upload a file via scp to the 'rpike' a/c on the server

 upfile() { scp $1 rpike@server.net:$1 }

Easily scp a file back to the host you're connecting from

 mecp () { scp "$@" ${SSH_CLIENT%% *}:Desktop/; }

download 2 files from the server using the account 'user'

 scp user@server.net:"chap1.txt chap2.txt" ~/books

upload the 'books' folder to the server 'machine.net' with account 'bob'

 scp -r ~/logs bob@machine.net:~/books   ##(consider using rsync instead)

download all the files in the 'books' folder to the current local folder

 scp bob@machine.net:/home/books/* .   ##(no subfolders are downloaded)

copy the file 'file.txt' from the 'server.net' host to the 'server.org'

 scp donkey@server.net:file1.txt donkey@server.org:books

scp file from hostb to hostc while logged into hosta

 scp user@hostb:file user@hostc:

Complex scp


....

create a list of file names from local folders

 d=$(echo ~/sf/htdocs/books/*/); d="basename "${d// /;basename }; d=$(eval $d); d=$(echo $d | sed -r 's:[a-z-]+:htdocs/books/&/&-book.txt:ig'); echo $d

create a list of file names from local folders and download them

 d=$(echo ~/sf/htdocs/books/*/); d="basename "${d// /;basename }; d=$(eval $d); d=$(echo $d | sed -r 's:[a-z-]+:htdocs/books/&/&-book.txt:ig');  echo $d; scp user,project@web.sourceforge.net:"$d" .

Sftp


....

sftp transfers files using the encrypted ssh (secure shell) protocol and is thus 'safer' and more modern than 'ftp'. sftp is designed to be used interactively by the user, but can also run in batch mode with the -b switch

using sftp in batch mode to upload to the sourceforge web server

 sftp -b a.txt user,project@web.sf.net > /tmp/sftp.log
 the file 'a.txt' contains 'cd books; put chap1.txt; bye;'

Rsync


Rsync is a very powerful network file copying program. It is particularly nice because it only transfers the parts of the files which have changed (the 'difference' to the old files). This saves a great deal of bandwidth and is probably the most efficient and pleasant way to develop a website. It is a program with a very large number of options to fine tune the way the file transfers take place.

Note: If the source path ends with "/" then the contents of the folder are copied but the folder itself is not

rsync tools .. grsync - a very basic graphical version of rsync with gtk ..

create an incremental backup of a directory using hard links

 rsync -a --delete --link-dest=../lastbackup $folder $dname/

Simple usage of rsync


....

the general form of rsync is

 rsync [options] source destination

download files from folder 'path' on the server to a local folder rsync -avP user@server.com:path/ /local/folder/ rsync -avP -e ssh user@server.com:path/ /local/folder/ ##(using ssh)

options:
-a archive mode -P keep partially transfered files, and show progress during the transfer of files ,,,

copy recursively all files and folders from src/bar to /data/tmp/bar

 rsync -avz machine:src/bar /data/tmp
##(note that a new directory "bar" may be created within '/data/tmp')

copy recursively all files and folders from src/bar/ to /data/tmp

 rsync -avz foo:src/bar/ /data/tmp

Note that NO directory "/data/tmp/bar" will be created

options used .. -v verbose output .. -a the "archive" mode, which means the same as "-rlptgoD" .. -r recurse into directories .. -l copy symbolic links as symbolic links .. -p preserve file permissions .. -t preserve file timestamps -g preserve the group information of the files (where possible) -o preserve file owner information -D preserve device and special files (where possible) -z compress file data during the transfer ..

copy all files from the current folder to "htdocs/books/" on the server

 rsync -rv --exclude=*.swp . user@server:htdocs/books/

The contents of the current folder are copied but not the folder itself. Files with names ending in '.swp' are not copied because of the --exclude option.

Remote copy directories and files through an SSH tunnel host

 rsync -avz -e 'ssh -A sshproxy ssh' srcdir remhost:dest/path/

copy files displaying a progress bar.

 rsync -av --progress ./file.txt user@host:/path/to/dir

rsync + find

 find . -name "whatever.*" -print0 | rsync -av --files-from=- --from0 ./ ./destination/

Local copying


....

Instead of using 'cp' one may use rsync instead to copy and synchronise large folders. This has the advantage that not all the files need be copied, only those that have changed.

synchronise the users 'web' folder with a folder on a usb key

 rsync -r ~/web/ /media/KINGSTON/web/
 rsync -r ~/web/ /media/KINGSTON/web   ##(the same)
 rsync -r ~/web /media/KINGSTON/       ##(the same)
 rsync -rv ~/web/ /media/KINGSTON/web  ##(the same, but verbose)
 rsync -rvn ~/web/ /media/KINGSTON/web/ ##(just see what would happen)

The trailing '/' on ~/web/ is important since it determined whether rsync will copy just the contents of the folder or the folder itself.

copy the folder 'big' from the current folder to the '/home/user/' folder

 rsync -r big /home/user/  
 cp -r big /home/user/  ##(this is more or less the same)

copy the folder 'big' to the users home folder exclude 'iso' files

 rsync -rv --progress --exclude=*.iso big ~/  ##(progress is shown)

Excluding files from rsync transfers


........

upload files excluding the directory 'images'

 rsync -r --exclude='images' /local/dir/ user@server.com:path/

The local folder /local/dir/images is not uploaded to the remote computer (server.com).

synchronise by uploading all files except 'png' & 'jpg' images

 rsync -r --exclude=*.png --exclude=*.jpg /local/dir/ user@server.com:path/

upload all files except 'pdf' documents and the /local/dir/images folder

 rsync -r --exclude=*.pdf --exclude=images /local/dir/ user@server.com:path/

upload files excluding everthing listed in the file '/path/list.txt'

 rsync --exclude-from '/path/list.txt' /local/dir/ user@server.com:path/
##(an absolute path for the exclude file seems to be necessary, 2008)

try bittorent.

Problems


....

There seems to be no short option for --exclude. It also seems imposible to include a list of files with one --exclude option

Deleting destination files with rsync


........

see what files would be deleted with the "--delete" option

 rsync -rnv --delete /home/user/ user@server:path/
 rsync -rv --delete --dry-run /home/user/ user@server:path/   ##(the same)
##(this should always be done before using "--delete")

recursively upload the contents of /home/ and delete destination files not found in the source folder tree (this can be dangerous!, first use "-n")

 rsync -r --delete /home/ user@server:path/

upload files TO a server from a local folder using rsync and ssh upload the '/home/' folder except c files, deleting c files at the destination

 rsync -rv --delete-excluded --exclude=*.c /home/ user@server:path/

upload recursively the current directory excluding the folder "tree" and deleting the folder "tree" at the destination

 rsync -rv --delete-excluded --exclude=tree  . user@machine:/books/web 

 rsync -avP -e ssh /local/folder/ user@server.com:path/                     

copy all files from the "/home/user/" folder to path/ on the server

 rsync -e ssh /home/user/* user@server.com:path/
##(this is not a recursive copy; folders are not copied; the shell handles "*")

Symbolic links and rsync


....

upload files, copying the target of symbolic links

 rsync -rv --copy-links /local/folder/ user@server.com:path/            
 rsync -rvL /local/folder/ user@server.com:path/ ##(the same)

Sourceforge


....

see what a sourceforge download with rsync would do without doing anything

 rsync -rvn -e ssh user,project@web.sf.net:htdocs/books ~/work/htdocs'

upload to an sf project folder excluding vim swap files

 rsync -rv --exclude='*.swp' -e ssh ~/sf/htdocs/books user,project@web.sf.net:htdocs/'
##(if it doesnt exist, the 'books' folder is created on the server)

get a project web-folder from sourceforge showing progress

 rsync -r --progress -e ssh user,project@web.sf.net:htdocs/books ~/work/htdocs'

Uploading files


use rsync, sftp, ftp,

Downloading files


....

Downloading refers to the process of transfering files from a remote computer (somewhere on the internet, or just in the next room) to the the computer on which you are working.

@@ http://en.wikipedia.org/wiki/Wget

Tools: wget, curl

Wget has the advantage that it works on slow and precarious internet connections and can be scripted and scheduled. Curl is very similar to wget but provides a few extra features.

snarf is a command line resource grabber.

 snarf http://foo.bar.com/picture.jpg

Download an entire ftp directory using wget

 wget -r ftp://user:pass@ftp.example.com

Download from Rapidshare Premium using wget - Part 2

 wget -c -t 1 --load-cookies ~/.cookies/rapidshare 

mirror a yahoo geocities site, accepting only txt, html ... etc

 wget -nH -m -Atxt,html,js,java,class,css -Isitename http://www.geocities.com/sitename

resume downloading a file from the internet

 wget -c url    ##(wget automatically retries getting)

download a file limiting the download rate to 20k/second

 wget --limit-rate=20k url
##(this prevents wget hogging the available network bandwidth)

get all the URLS contained in the file "list.txt"

 wget -i list.txt 

download the html file and save it to the folder "tree"

 wget -P"tree" http://bumble.sf.net 

download all ".gif" files from a web folder

 wget -r -l1 --no-parent -A.gif http://host/folder/

mirror a website without overwriting any existing local files

 wget -nc -r http://www.gnu.ai.mit.edu/

Download an entire website

 wget --random-wait -r -p -e robots=off -U mozilla http://www.example.com

download a file and output to standard output

 wget -O - http://jagor.srce.hr/

get all the pages linked to by the page "links.html"

 wget -O - http://host/links.html | wget --force-html -i

check the validity of links in the bookmarks.html

 wget --spider --force-html -i bookmarks.html

mirror the site http://fa.org/, with 3 retries logging errors in 'mirror.log'

 wget -m -t3 http://fa.org/ -o mirror.log 

use wget but pretend to be the "Konqueror" browser

 wget --user-agent="Mozilla/5.0 (compatible; Konqueror/4.2; Linux) KHTML/4.2.98 (like Gecko)"
##(some sites may block access to wget to reduce server load)

pretend to be the "Lynx" text mode browser

 wget -U "Lynx/2.8.7dev.9 libwww-FM/2.14" http://site

download free ebooks from amazon.com # pretends to be firefox, recurses to 2 levels wget -erobots=off --user-agent="Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.3) Gecko/2008092416 Firefox/3.0.3" -H -r -l2 --max-redirect=1 -w 5 --random-wait -PmyBooksFolder -nd --no-parent -A.pdf http://amazon.com ,,,

Using curl


....

The curl utility is very similar to 'wget' but provides a few extra tricks.

HTTP Get of a web page via proxy server with login credentials

 curl -U username[:password] -x proxyserverIP:proxyserverPort webpageURI

get a file while pretending to be netscape 4.73 browser

 curl -A "Mozilla/4.73 [en] (X11; U; Linux 2.2.15 i686)" [URL]

follow http redirects (where the page is automatically refreshed to another)

 curl -L www.will.redirect.org

use url 'globbing' to get several pages

 curl http://site.{one,two,three}.com 

use arithmetic 'globbing' of urls to retrieve lots of pages

 curl http://www.example.com/file[1-100].txt  
##(this will retrieve file1.txt, file2.txt, file3.txt ...)
 curl http://www.eg.org/file[001-100].txt  
##(this will retrieve file001.txt, file002.txt, ... file099.txt, file100.txt)

specify an alphabetic sequence of urls to retrieve

 curl www.eg.org/[a-g].html  ##(will get a.html, b.html, ... g.html)

various sequences can be include to get lots of files

 curl www.eg.org/archive[1996-1999]/vol[1-4]/part[a-f].html 

specify a sequence of urls to retrieve with a 'step' value (since 7.15.1)

 curl http://www.eg.org/[1-100:10].txt ##(gets 1.txt, 11.txt, 21.txt ...)

access a page which is protected by basic authentication

 curl -u name:password www.secrets.com

get the files a.html, b.html, c.html and save them with their names

 curl -O  www.eg.com/[a-c].html

get a.txt, r.txt and s.txt and save them in a file called dump.txt

 curl -o dump.txt www.eg.com/{a,r,s}.txt

get all the pages of the linux cookbook pretending to be mozilla

 curl -A "Mozilla/4.0" -O http://dsl.org/cookbook/cookbook_[1-45].html
 for i in $(seq 1 45); do curl -A "Mozilla/4.0" -O http://dsl.org/cookbook/cookbook_$i.html; sleep 2; done  ##(the same, but sleeping 2 seconds)

download the linux cookbook, and convert to text at the same time

 for i in $(seq 1 45); do lynx --dump http://dsl.org/cookbook/cookbook_$i.html > cookbook-$i.txt; sleep 2; done  

Mirroring


....

Create a mirror of a local folder, on a remote server

 rsync -e "/usr/bin/ssh -p22" -a --progress --stats --delete -l -z -v -r -p /root/files/ user@remote_server:/root/files/

Developing software


Cpp


Add a newline to the end of a cpp file

 find . -iname "*.cpp" -exec perl -ni -e 'chomp; print "$_\n"' {} \;

Create etags file of .c, .cpp, and .h files in all subdirectories

 find . -regex ".*\.[cChH]\(pp\)?" -print | etags -

display typedefs, structs, unions and functions provided by a library

 cpp /usr/include/stdio.h | grep -v '^#' | grep -v '^$' | less
Display GCC Predefined Macros
 gcc -dM -E - <<<''

Show Shared Library Mappings

 ldconfig -p

Colorize make, gcc, and diff output

 colormake, colorgcc, colordiff

Write and run a quick C program

 cat | gcc -x c -o a.out - && ./a.out && rm a.out

make a patch file using an original file and a modification

 diff -u file.txt.orig file.txt > file.txt.patch 

compile an application

 ./configure; make && make install   ##(use 'sudo' where necessary)

Display a list of code committers sorted by the frequency of commits

 svn log -q|grep "|"|awk "{print \$3}"|sort|uniq -c|sort -nr

List all authors of a particular git project

 git log --format='%aN' | sort -u

Figure out your work output for the day

 git diff --stat `git log --author="XXXXX" --since="12 hours ago" --pretty=oneline | tail -n1 | cut -c1-40` HEAD

git diff of files that have been staged ie 'git add'ed

 git diff --cached

Remote development


....

detach remote console for long running operations

 dtach -c /tmp/wires-mc mc

Execute a sudo command remotely, without displaying the password

 stty -echo; ssh HOSTNAME "sudo some_command"; stty echo

How to run a command on a list of remote servers read from a file

 dsh -M -c -f servers -- "command HERE"

connect to X login screen via vnc

 x11vnc -display :0 -auth $(ps -ef|awk '/xauth/ {print $15}'|head -1) -forever -bg &

Share your terminal session (remotely or whatever)

 screen -x

cvs, subversion

Use a Gmail virtual disk (GmailFS) on Ubuntu

 mount.gmailfs none /mount/path/ [-o username=USERNAME[,password=PASSWORD][,fsname=VOLUME]] [-p]

Ldap


LDAP search to query an ActiveDirectory server

 ldapsearch -LLL -H ldap://activedirectory.example.com:389 -b 'dc=example,dc=com' -D 'DOMAIN\Joe.Bloggs' -w 'p@ssw0rd' '(sAMAccountName=joe.bloggs)'

decoding Active Directory date format

 ldapsearch -v -H ldap:// -x -D cn=,cn=,dc=,dc= -w -b ou=,dc=,dc= -s sub sAMAccountName=* '*' | perl -pne 's/(\d{11})\d{7}/"DATE-AD(".scalar(localtime($1-11644473600)).")"/e'

Dns the domain name system


resolve hostname to IP our vice versa with less output

 resolveip -s www.freshmeat.net

Determine what version of bind is running on a dns server.

 dig -t txt -c chaos VERSION.BIND @

flush cached dns lookups

 ipconfig /flushdns

Check version of DNS Server

 nslookup -q=txt -class=CHAOS version.bind NS.PHX5.NEARLYFREESPEECH.NET

gets the bare ip(s) of a domain

 dig commandlinefu.com | sed -nr 's/^[^;].*?\s([.0-9]{7,15})$/\1/ p'

Get MX records for a domain

 host -t mx foo.org

Reverse DNS lookups

 sed 's/\([0-9]*\)\.\([0-9]*\)\.\([0-9]*\)\.\([0-9]*\).in-addr.arpa domain name pointer\(.*\)\./\4.\3.\2.\1\5/' \ lookups.txt

DNS cache snooping

 for i in `cat names.txt`; do host -r $i [nameserver]; done

Check if a .no domain is available

 check_dns_no() { for i in $* ; do if `wget -O - -q http://www.norid.no/domenenavnbaser/whois/?query=$i.no | grep "no match" &>/dev/null` ; then echo $i.no "available" ; fi ; sleep 1 ;done }

Get your public ip using dyndns

 curl -s http://checkip.dyndns.org/ | grep -o "[[:digit:].]\+"

Get fully qualified domain names (FQDNs) for IP address with

 NAME=$(nslookup $IP | sed -n 's/.*arpa.*name = \(.*\)/\1/p'); test -z "$NAME" && NAME="NO_NAME"; echo "$NAME"

Get your external IP address if your machine has a DNS entry

 host $HOSTNAME|cut -d' ' -f4

Get your external IP address if your machine has a DNS entry

 curl www.whatismyip.com/automation/n09230945.asp

Perform a reverse DNS lookup

 dig -x 74.125.45.100

Get your public ip using dyndns

 curl -s 'http://www.loopware.com/ip.php'

Check reverse DNS

 dig -x {IP}

Check reverse DNS

 dig +short -x {ip}

Update your OpenDNS network ip

 wget -q --user= --password= 'https://updates.opendns.com/nic/update?hostname=your_opendns_hostname& myip=your_ip' -O -

Update dyndns.org with your external IP.

 curl -v -k -u user:password "https://members.dyndns.org/nic/update?hostname= &myip=$(curl -s http://checkip.dyndns.org | sed 's/[a-zA-Z<>/ :]//g')&wildcard=NOCHG&mx=NOCHG&backmx=NOCHG"

Check reverse DNS

 host {checkIp or hostname} [dns server]

Get MX records for a domain

 dig foo.org mx +short

Short and sweet output from dig(1)

 alias ds='dig +noauthority +noadditional +noqr +nostats +noidentify +nocmd +noquestion +nocomments'

Get your external IP address if your machine has a DNS entry

 dig +short $HOSTNAME

List of reverse DNS records for a subnet

 nmap -R -sL 209.85.229.99/27 | awk '{if($3=="not")print"("$2") no PTR";else print$3" is "$2}' | grep '('

Reading usenet news


read news using the server 'nntp.aioe.org' (for example)

 slrn -h nntp.aioe.org

news reading programs .. slrn, in active development .. alpine, a mail and news program .. mutt, another mail and news program .. nn, .. tin, ..

Web servers


set an HTTP redirect to listen on port 80

 while [ 0 ]; do echo -e "HTTP/1.1 302 Found\nLocation: http://www.whatevs.com/index.html" | nc -vvvv -l -p 80; done

Apache


....

Simple list of apache2 virtualhosts

 /usr/sbin/apache2ctl -S

Count how many times a certain referer appears in your apache log

 Q="reddit|digg"; F=*.log; awk -F\" '{print $4}' $F | egrep $Q | wc -l

Search for specific IPs taken from a text file within the apache

 grep -E ":(`cat bnd-ips.txt | sed 's/\./\\./g' | tr '\n' '|'`)" access.log

benchmark web server with apache benchmarking tool

 ab -n 9000 -c 900 localhost:8080/index.php

Analyse compressed Apache access logs for the most commonly

 zcat access_log.*.gz | awk '{print $7}' | sort | uniq -c | sort -n | tail -n 20

Who has the most Apache connections.

 netstat -anl | grep :80 | awk '{print $5}' | cut -d ":" -f 1 | uniq -c | sort -n | grep -c IPHERE

Summarize Apache Extended server-status to show longest running

 links --dump 1 http://localhost/server-status|grep ^[0-9]|awk 'BEGIN {print "Seconds, PID, State, IP, Domain, TYPE, URL\n--"} $4 !~ /[GCRK_.]/ {print $6, $2, $4, $11, $12, $13 " " $14|"sort -n"}'

grep apache access.log and list IP's by hits and date - sorted

 grep Mar/2009 /var/log/apache2/access.log | awk '{ print $1 }' | sort -n | uniq -c | sort -rn | head

show Apache memory usage

 ps auxf | grep httpd | grep -v grep | grep -v defunct | awk '{sum=sum+$6}; END {print sum/1024}'

How much RAM is Apache using?

 ps -o rss -C httpd | tail -n +2 | (sed 's/^/x+=/'; echo x) | bc

Know which modules are loaded on an Apache server

 apache2 -t -D DUMP_MODULES

Get list of all Apache Virtual Host and which is default for each

 httpd -S

web-serve files in the current folder tree accessible at http://$HOSTNAME:8000/

 python -m SimpleHTTPServer ##(this is the simplest webserver setup)

Top 10 requestors arranged by IP address from Apache/NCSA Logs

 awk '{print $1}' /var/log/httpd/access_log | sort | uniq -c | sort -rnk1 | head -n 10

Www the world wide web


view the urls in the file 'eg.txt'

 wget www.dmoz.org/Computers/ -Oeg.txt; urlview eg.txt

convert an html file into valid xml

 tidy -asxhtml -numeric < index.html > index.xml

rapidshare download script in 200 characters

 u=`curl -d 'dl.start=Free' $(curl $1|perl -wpi -e 's/^.*"(http:\/\/rs.*)" method.*$/$1/'|egrep '^http'|head -n1)|grep "Level(3) \#2"|perl -wpi -e 's/^.*(http:\/\/rs[^\\\\]*).*$/$1/'`;sleep 60;wget $u

Check your unread Gmail from the command line

 curl -u username:password --silent "https://mail.google.com/mail/feed/atom" | tr -d '\n' | awk -F '' '{for (i=2; i<=NF; i++) {print $i}}' | sed -n "s/\(.*\)<\/title.*name>\(.*\)<\/name>.*/\2 - \1/p"</pre>
<p>
<strong><em>Twitpic upload and Tweet</em></strong>
  <pre class="codeline"> curl --form username=from_twitter --form password=from_twitter --form media=@/path/to/image --form-string "message=tweet" http://twitpic.com/api/uploadAndPost</pre>
<p>
<strong><em>check site ssl certificate dates</em></strong>
  <pre class="codeline"> echo | openssl s_client -connect www.google.com:443 2>/dev/null |openssl x509 -dates -noout</pre>
<p>
<strong><em>Parallel file downloading with wget</em></strong>
  <pre class="codeline"> wget -nv http://en.wikipedia.org/wiki/Linux -O- | egrep -o "http://[^[:space:]]*.jpg" | xargs -P 10 -r -n 1 wget -nv</pre>
<p>
<strong><em>Search Google from the command line</em></strong>
  <pre class="codeline"> curl -A Mozilla http://www.google.com/search?q=test |html2text -width 80</pre>
<p>
<strong><em>Log your internet download speed</em></strong>
  <pre class="codeline"> echo $(date +%s) > start-time; URL=http://www.google.com; while true; do echo $(curl -L --w %{speed_download} -o/dev/null -s $URL) >> bps; sleep 10; done &</pre>
<p>
<strong><em>Check if a domain is available and get the answer in just one</em></strong>
  <pre class="codeline"> whois domainnametocheck.com | grep match</pre>
<p>
<strong><em>count the appearance of a word or a string in a given webpage</em></strong>
  <pre class="codeline"> wget -q -O- PAGE_URL | grep -o 'WORD_OR_STRING' | wc -w</pre>
<p>
<strong><em>Creating shortened URLs from the command line</em></strong>
  <pre class="codeline"> curl -s http://tinyurl.com/create.php?url=http://<website.url>/ | sed -n 's/.*\(http:\/\/tinyurl.com\/[a-z0-9][a-z0-9]*\).*/\1/p' | uniq</pre>
<p>
<strong><em>Manually Pause/Unpause Firefox Process with POSIX-Signals</em></strong>
  <pre class="codeline"> killall -STOP -m firefox</pre>
<p>
<strong><em>Extract all urls from the last firefox<a href="sessionstore.js"><em><code>sessionstore.js</code></em></a> used.</em></strong>
  <pre class="codeline"> sed -e 's/{"url":/\n&/g' ~/.mozilla/firefox/*/sessionstore.js | cut -d\" -f4</pre>
<p>
<h3>Downloading from the web </h3><hr>....
<p>
<center><h3 class="page-title"></h3></center> web mirroring
.. httrack - more interactive than wget
.. wget 
<p>
<strong><em>Run remote web page, but don't save the results</em></strong>
  <pre class="codeline"> wget -O /dev/null http://www.google.com</pre>
<p>
<h3>Comic strips online </h3><hr>....
<p>
<strong><em>View the newest xkcd comic.</em></strong>
  <pre class="codeline"> wget `lynx --dump http://xkcd.com/|grep png`</pre>
<p>
<strong><em>display the 'dilbert' comic strip of the day </em></strong>
  <pre class="codeline"> display http://dilbert.com$(curl -s dilbert.com|grep -Po '"\K/dyn/str_strip(/0+){4}/.*strip.[^\.]*\.gif')</pre>
<p>
<strong><em>whois surfing my web ?</em></strong>
  <pre class="codeline"> watch lsof -i :80</pre>
<p>
<h3>Css cascading style sheets </h3><hr>....
<p>
<strong><em>Awk one-liner that sorts a css file by selector</em></strong>
  <pre class="codeline"> awk '/.*{$/{s[$1]=z[$1]=j+0}{l[j++]=$0}END{asorti(s);for(v in s){while(l[z[s[v]]]!~/}$/)print l[z[s[v]]++];print"}"ORS}}'</pre>
<p>
<h3>Twitter </h3><hr>....
<p>
The microblogging website has become strangely popular.
<p>
<strong><em>Update twitter from command line without reveal your password</em></strong>
  <pre class="codeline"> curl -n -d status='Hello from cli' https://twitter.com/statuses/update.xml</pre>
<p>
<strong><em>Get your Tweets from the command line</em></strong>
  <pre class="codeline"> curl -s -u user:password 'http://twitter.com/statuses/friends_timeline.xml?count=5' | xmlstarlet sel -t -m '//status' -v 'user/screen_name' -o ': ' -v 'text' -n</pre>
<p>
<strong><em>Print trending topics on Twitter</em></strong>
  <pre class="codeline"> wget http://search.twitter.com/trends.json -O - --quiet | ruby -rubygems -e 'require "json";require "yaml"; puts YAML.dump(JSON.parse($stdin.gets))'</pre>
<p>
<strong><em>check if it is possible to connect to twitter</em></strong>
  <pre class="codeline"> wget http://twitter.com/help/test.json -q -O -</pre>
<p>
<strong><em>another tweet function to view statuses</em></strong>
  <pre class="codeline"> tweet () { curl -u UserName -d status="$*" http://twitter.com/statuses/update.xml; }</pre>
<p>
<strong><em>speak the last 3 tweets on mac os</em></strong>
  <pre class="codeline"> curl -s -u user:password http://twitter.com/statuses/friends_timeline.rss | grep title | sed -ne 's/<\/*title>//gp' | head -n 4 | say -v Bruce</pre>
<p>
<strong><em>send tweets to twitter (and get user details)</em></strong>
  <pre class="codeline"> curl --basic --user "user:pass" --data-ascii "status=tweeting%20from%20%the%20linux%20command%20line" http://twitter.com/statuses/update.json</pre>
<p>
<strong><em>Update twitter via curl</em></strong>
  <pre class="codeline"> curl -u user:pass -d status="Tweeting from the shell" http://twitter.com/statuses/update.xml</pre>
<p>
<strong><em>Print trending topics on Twitter</em></strong>
  <pre class="codeline"> curl -s search.twitter.com | awk -F'</?[^>]+>' '/\/intra\/trend\//{print $2}'</pre>
<p>
<strong><em>Print trending topics on Twitter</em></strong>
  <pre class="codeline"> curl --silent search.twitter.com | sed -n '/div id=\"hot\"/,/div/p' | awk -F\> '{print $2}' | awk -F\< '{print $1}' | sed '/^$/d'</pre>
<p>
<h3>Firefox </h3><hr>....
<p>
The firefox browser is an important piece of software.
<p>
<strong><em>how to run firefox in safe mode from command line</em></strong>
  <pre class="codeline"> firefox --safe-mode</pre>
<p>
<strong><em>run the firefox profile manager (which manages default settings)</em></strong>
  <pre class="codeline"> firefox -no-remote -P</pre>
<p>
<strong><em>view what percentage of memory firefox is using</em></strong>
  <pre class="codeline"> ps -o %mem= -C firefox-bin | sed -s 's/\..*/%/'</pre>
<p>
<strong><em>Extract all urls from last firefox sessionstore</em></strong>
  <pre class="codeline"> perl -lne 'print for /url":"\K[^"]+/g' $(ls -t ~/.mozilla/firefox/*/sessionstore.js | sed q)</pre>
<p>
<strong><em>release firefox from a still running message</em></strong>
  <pre class="codeline"> rm ~/.mozilla/firefox/<profile_dir>/.parentlock</pre>
<p>
<strong><em>List recorded formular fields of Firefox incomplete??</em></strong>
  <pre class="codeline"> cd ~/.mozilla/firefox/ && sqlite3 $(cat profiles.ini | grep Path )</pre>
<p>
<strong><em>Cleanup firefox's database.</em></strong>
  <pre class="codeline"> find ~/Library/Application\ Support/Firefox/ -type f -name "*.sqlite" -exec sqlite3 {} VACUUM \;</pre>
<p>
<strong><em>Speed up launch of firefox</em></strong>
  <pre class="codeline"> find ~ -name '*.sqlite' -exec sqlite3 '{}' 'VACUUM;' \;</pre>
<p>
<h3>Graphical web browsers</h3><hr>
<p>
<center><h3 class="page-title"></h3></center> some graphical browsers
.. galeon - a small browser for gnome
.. seamonkey
.. iceweasel - debian firefox
.. abrowser - unbranded firefox
.. dillo - a very small browser
.. 
<p>
<h3>Text mode web surfing </h3><hr>....
<p>
Text-mode web surfing means using a program running from a 'console'
'terminal' or 'command-line' to view web-pages. This generally means
that it is not possible to view the images contained in the web-pages,
only the text. While this is aesthetically inferior, it is faster and
sometimes more useful and less distracting.
<p>
<center><h3 class="page-title"></h3></center> web browsers
.. lynx - the classic text mode browser, debian: lynx, lynx-cur
.. elinks - displays utf8, menus, bad default colours
.. w3m - very strange key bindings, japanese oriented
.. links - doesnt seem to have utf8 support
<p>
<strong><em>a text mode browser with table support </em></strong>
  <pre class="codeline"> links  ##(press <f9> to set a menu of options)</pre>
<p>
<strong><em>create a lynx macro by recording a session</em></strong>
  <pre class="codeline"> lynx -cmd_log logfilename</pre>
<p>
<strong><em>the '-cmd_log' switch should go after a starting page </em></strong>
  <pre class="codeline"> lynx www.xx.net -cmd_log logfile    ##(this works)</pre>
  <pre class="codeline"> lynx -cmd_log logfile www.xx.net    ##(doesnt work!)</pre>
<p>
<strong><em>replay a macro recorded with the -cmd_log switch</em></strong>
  <pre class="codeline"> lynx -cmd_script=/path/to/logfilename</pre>
<p>
<strong><em>make lynx accept all cookies (also setable in the configuration file)</em></strong>
  <pre class="codeline"> lynx -accept_all_cookies www.xx.net</pre>
<p>
<strong><em>w3m can display tables and with "w3m-img" images</em></strong>
  <pre class="codeline"> w3m</pre>
<p>
<strong><em>output only text, with underscores, of the previous URL, and save it to the file 'winter_dreams', type (all on one line):</em></strong>
  <pre class="codeline"> lynx -dump -nolist -underscore http://www.utas.edu.au/ > winter_dreams </pre>
<p>
<strong><em>print the pure text, with underscores, of the previous URL in a Times Roman font, type (all on one line):</em></strong>
  <pre class="codeline"> lynx -dump -nolist -underscore http://www.sc.edu/fitzgerald/winterd/winter.html | enscript -B</pre>
<p>
<strong><em>view the 'new york times' archive with user-name and password 'cypherpunks'</em></strong>
  <pre class="codeline"> lynx -auth=cypherpunks:cypherpunks http://www.nytimes.com/archive/ </pre>
<p>
<strong><em>save the URL <a href="http://www.nytimes.com/archive/">www.nytimes.com/archive/</a> as an annotated text file, 'mynews' </em></strong>
  <pre class="codeline"> lynx -dump -number_links -auth=cypherpunks:cypherpunks www.nytimes.com/archive/ > mynews </pre>
<p>
If you want a lynx file to go to a webpage through a password form,
then dont press 'q' quit at the end of the recording or else this
will be included in the macro. Quit lynx with control-c instead.
<p>
@@ <a href="http://www.timvw.be/listen-to-online-radio/" <a href="http://www.timvw.be/listen-to-online-radio/</a>">http://www.timvw.be/listen-to-online-radio/</a></a>
<p>
<h3>Email electronic mail</h3><hr>
<p>
<strong><em>Sort a one-per-line list of email address, weeding out duplicates</em></strong>
  <pre class="codeline"> sed 's/[ \t]*$//' < emails.txt | tr 'A-Z' 'a-z' | sort | uniq > emails_sorted.txt</pre>
<p>
<strong><em>extract email adresses from some file (or any other pattern)</em></strong>
  <pre class="codeline"> grep -Eio '([[:alnum:]_.-]+@[[:alnum:]_.-]+?\.[[:alpha:].]{2,6})'</pre>
<p>
<strong><em>Connect to SMTP server using STARTTLS</em></strong>
  <pre class="codeline"> openssl s_client -starttls smtp -crlf -connect 127.0.0.1:25</pre>
<p>
<strong><em>Email HTML content</em></strong>
  <pre class="codeline"> mailx bar@foo.com -s "HTML Hello" -a "Content-Type: text/html" < body.htm</pre>
<p>
<strong><em>Block the 6700 worst spamhosts</em></strong>
  <pre class="codeline"> wget -q -O - http://someonewhocares.org/hosts/ | grep ^127 >> /etc/hosts</pre>
<p>
<h3>Alpine </h3><hr>....
<p>
Alpine is a 'console' based (curses) email client, allegedly used
by linus torvalds. It uses the pico text editor to edit mail.
<p>
<h3>Mutt </h3><hr>....
<p>
@@ <a href="http://www.mutt.org" <a href="http://www.mutt.org</a>">http://www.mutt.org</a></a>
the official site.
<p>
Mutt is apparently in active development. (Mutt 1.5.20 was released on
June 14, 2009). Mutt uses an external text editor to edit mail
<p>
<strong><em>Send email with one or more binary attachments</em></strong>
  <pre class="codeline"> echo "Body goes here" | mutt -s "A subject" -a /path/to/file.tar.gz</pre>
<p>
<strong><em>Create mails array from .mutt-alias file.</em></strong>
  <pre class="codeline"> muttlst(){ for i in $*;do mails+=($(grep -wi "$i" .mutt-alias|awk '{print $NF}'));done;}</pre>
<p>
<h3>Webmail </h3><hr>....
<p>
<strong><em>Check your unread Gmail from the command line</em></strong>
  <pre class="codeline"> curl -u username --silent "https://mail.google.com/mail/feed/atom" | perl -ne 'print "\t" if /<name>/; print "$2\n" if /<(title|name)>(.*)<\/\1>/;'</pre>
<p>
<h3>Attachments </h3><hr>....
<p>
<strong><em>Decode a MIME message</em></strong>
  <pre class="codeline"> munpack file.txt</pre>
<p>
<strong><em>Send a local file via email</em></strong>
  <pre class="codeline"> echo "see attached file" | mail -a filename -s "subject" email@address</pre>
<p>
<strong><em>Send a binary file as an attachment to an email</em></strong>
  <pre class="codeline"> uuencode archive.tar.gz archive.tar.gz | mail -s "Emailing: archive.tar.gz" user@example.com</pre>
<p>
<strong><em>Send a local file as an attachment via email</em></strong>
  <pre class="codeline"> mutt you@mailserver.com -s "Message Subject Here" -a attachment.jpg </dev/null</pre>
<p>
<strong><em>Send a local file via email</em></strong>
  <pre class="codeline"> mpack -s "Backup: $file" "$file" email@id.com</pre>
<p>
<h3>Email addresses </h3><hr>....
<p>
<strong><em>extract email adresses from some file (or any other pattern)</em></strong>
  <pre class="codeline"> grep -Eio '([[:alnum:]_.]+@[[:alnum:]_]+?\.[[:alpha:].]{2,6})' file.html</pre>
<p>
<strong><em>Move all but the newest 100 emails to a gzipped archive</em></strong>
  <pre class="codeline"> find $MAILDIR/ -type f -printf '%T@ %p\n' | sort --reverse | sed -e '{ 1,100d; s/[0-9]*\.[0-9]* \(.*\)/\1/g }' | xargs -i sh -c "cat {}&&rm -f {}" | gzip -c >>ARCHIVE.gz</pre>
<p>
<h3>Smtp protocol </h3><hr>....
<p>
<strong><em>python smtp server</em></strong>
  <pre class="codeline"> python -m smtpd -n -c DebuggingServer localhost:1025</pre>
<p>
<strong><em>Create AUTH PLAIN string to test SMTP AUTH session</em></strong>
  <pre class="codeline"> printf '\!:1\0\!:1\0\!:2' | mmencode | tr -d '\n' | sed 's/^/AUTH PLAIN /'</pre>
<p>
<h3>Older mail systems </h3><hr>....
<p>
<strong><em>send an email message to lisa@example.com</em></strong>
  <pre class="codeline"> mail lisa@example.com </pre>
Subject: Hello 
Hi there, long time no talk! I'm just learning how to use 
,,,
<p>
<strong><em>send an email message to user mrs on your local system</em></strong>
mail mrs 
Subject: are you going to the party tonight? 
C-d
,,,
<p>
<strong><em>mail the contents of the text file 'trades' to the email address terrapin@example.com</em></strong>
  <pre class="codeline"> mail terrapin@example.com < trades </pre>
<p>
<strong><em>mail the text of the URL [28 <a href="http://etext.org/" <a href="http://etext.org/</a>">http://etext.org/</a></a> as annotated text</em></strong>
to the email address droneon@example.com
  <pre class="codeline"> mail droneon@example.com < lynx -dump -number_links</pre>
http://etext.org/ 
<p>
<strong><em>insert a copy of the current mail message into the body of the</em></strong>
message you are writing, and then open the message in the default
text editor
~f 
<p>
<strong><em>output the location of your INBOX</em></strong>
  <pre class="codeline"> echo $MAIL </pre>
Usually, the INBOX location is in the '/var/spool/mail' directory, and
has the same name as your username -- so if your username is mrs, your
<p>
<strong><em>see if you have mail</em></strong>
  <pre class="codeline"> mail </pre>
Mail version 8.1 6/6/93. Type ? for help.
"/var/spool/mail/m": 3 messages 3 new
,,,
<p>
<strong><em>read the next unread message in mail</em></strong>
& 
<p>
<strong><em>read message number three in mail</em></strong>
  <pre class="codeline"> & 3 </pre>
<p>
<strong><em>exit mail and revert your INBOX to its state before you started mail</em></strong>
  <pre class="codeline"> & x </pre>
<p>
<strong><em>delete the message you just read</em></strong>
  <pre class="codeline"> & d </pre>
<p>
<strong><em>delete message 3</em></strong>
  <pre class="codeline"> & d3 </pre>
<p>
<strong><em>delete messages 10 through 14</em></strong>
  <pre class="codeline"> & d10-14 </pre>
<p>
<strong><em>view the mail folder '~/email/mrs' in elm</em></strong>
  <pre class="codeline"> elm -f ~/email/mrs </pre>
<p>
<strong><em>view the contents of all of the email folders in your '~/email' directory</em></strong>
  <pre class="codeline"> cat ~/email/* > allmessages </pre>
  <pre class="codeline"> elm -f allmessages </pre>
<p>
<strong><em>turn biff on</em></strong>
  <pre class="codeline"> biff y    ##(or put in .bashrc)</pre>
<p>
<strong><em>see what biff is set to</em></strong>
  <pre class="codeline"> biff </pre>
<p>
<strong><em>see also xbiff,</em></strong>
<p>
<strong><em>see how many email messages you have waiting</em></strong>
  <pre class="codeline"> messages </pre>
<p>
<strong><em>count the number of email messages in the mail folder</em></strong>
'~/email/saved'
  <pre class="codeline"> messages ~/email/saved </pre>
<p>
<strong><em>output a list showing sender names and subjects of your incoming mail</em></strong>
  <pre class="codeline"> frm </pre>
<p>
<strong><em>output a list with sender names and subjects in the file '~/email/saved'</em></strong>
  <pre class="codeline"> frm ~/email/saved </pre>
<p>
<strong><em>verify that the email address user@example.edu is valid</em></strong>
  <pre class="codeline"> vrfy user@example.edu </pre>
<p>
<strong><em>verify all of the email addresses contained in the file 'mail-list'</em></strong>
  <pre class="codeline"> vrfy -f mail-list </pre>
<p>
<strong><em>mail the JPEG file 'dream.jpeg' in the current directory to dali@example.org</em></strong>
metasend 
To: dali@example.org 
,,,
<p>
<strong><em>view the current history log with lynx</em></strong>
  <pre class="codeline"> lynx ~/.browser-history/history-log.html </pre>
<p>
<strong><em>find URLs visited in the year 2000 titles containing the word 'Confessions'</em></strong>
  <pre class="codeline"> zgrep Confessions ~/.browser-history/history-log-2000* </pre>
<p>
<strong><em>open the URL <a href="http://www.drudgereport.com/">www.drudgereport.com/</a> in Mozilla from a shell script</em></strong>
  <pre class="codeline"> mozilla -remote 'openURL(http://www.drudgereport.com/)'</pre>
<p>
<strong><em>go back to the last URL you visited</em></strong>
  <pre class="codeline">  type [ALT] [<-],</pre>
<p>
<strong><em>forward to the next URL in your history,</em></strong>
  <pre class="codeline">  type [ALT] [->].</pre>
<p>
<strong><em>open your bookmarks file in a new window, type [ALT]-b.</em></strong>
<p>
<strong><em>archive the Web site at <a href="http://dougal.bris.ac.uk/~mbt/," <a href="http://dougal.bris.ac.uk/~mbt/,</a>">http://dougal.bris.ac.uk/~mbt/,</a></a> only archiving the '/~mbt' directory, and writing log messages to a file called 'uk.log'</em></strong>
  <pre class="codeline"> wget -m -t3 -I /~mbt http://dougal.bris.ac.uk/~mbt/</pre>
<p>
<strong><em>add 'HEIGHT' and `WIDTH' parameters to the file `index.html',</em></strong>
  <pre class="codeline"> imgsizer index.html </pre>
<p>
<strong><em>peruse the file 'index.html' with its HTML tags removed</em></strong>
  <pre class="codeline"> unhtml index.html | less </pre>
<p>
<strong><em>remove the HTML tags from 'index.html' and put the output in 'index.txt'</em></strong>
  <pre class="codeline"> unhtml index.html > index.txt </pre>
<p>
<strong><em>print a copy of <a href="http://example.com/essay/" <a href="http://example.com/essay/</a>">http://example.com/essay/</a></a> in typescript manuscript form</em></strong>
  <pre class="codeline"> lynx -dump -underscore -nolist http://example.com/essay/ | pr -d | enscript -B </pre>
<p>
<strong><em>print a PostScript copy of the document at the URL</em></strong>
  <pre class="codeline"> html2ps http://example.com/essay/ | lpr </pre>
<p>
<strong><em>write a copy of the document at the URL with all hypertext links underlined</em></strong>
  <pre class="codeline"> html2ps -u -o submission.ps http://example.com/essay/ </pre>
<p>
<strong><em>validate the HTML in the file 'index.html'</em></strong>
  <pre class="codeline"> weblint index.html </pre>
<p>
<strong><em>connect to the system kanga.ins.cwru.edu</em></strong>
  <pre class="codeline"> telnet kanga.ins.cwru.edu </pre>
Trying 129.22.8.32...
Connected to kanga.INS.CWRU.Edu.
<p>
<strong><em>disconnect from a remote Linux system</em></strong>
  <pre class="codeline"> C-d</pre>
<p>
<strong><em>temporarily return to a local shell prompt</em></strong>
faraway-system$ C-[
telnet> z 
<p>
<strong><em>return to the remote system</em></strong>
  <pre class="codeline"> fg </pre>
faraway-system$
In the first of the two preceding examples, the escape character C-[
<p>
<strong><em>make an anonymous ftp connection to ftp.leo.org</em></strong>
  <pre class="codeline"> ftp ftp.leo.org </pre>
Connected to ftp.leo.org.
220-Welcome to LEO.ORG.
<p>
<strong><em>change to the '/pub' directory on the remote system and look at</em></strong>
the files that are there
ftp> cd /pub 
250 Directory changed to /pub.
<p>
<strong><em>put a copy of the file 'thyme.rcp' from the current directory on</em></strong>
the local system to the current directory of the remote system,
type:<br/>
ftp> put thyme.rcp 
<p>
<strong><em>change to the parent directory of the current directory on the</em></strong>
local system
ftp> lcd .. 
Local directory now /home/james/demos
<p>
<strong><em>download the file 'INDEX.gz' in the current directory on the</em></strong>
remote system, saving it to your '~/tmp' directory
ftp> lcd ~/tmp 
Local directory now /home/james/tmp
<p>
<strong><em>output a list of all newsgroups that match the pattern</em></strong>
'society'
  <pre class="codeline"> nngrep society </pre>
Use the '-u' option to only search through unsubscribed groups. This is
<p>
<strong><em>output a list of all unsubscribed-to newsgroups that match the</em></strong>
pattern 'society'
  <pre class="codeline"> nngrep society </pre>
In the previous example, if you were already subscribed to the group
<p>
<p>
<h3>The ppp protocol</h3><hr>
<p>
<strong><em>start a PPP connection</em></strong>
  <pre class="codeline"> pon </pre>
<p>
<strong><em>stop a PPP session</em></strong>
  <pre class="codeline"> poff </pre>
<p>
<h3>Sending and receiving faxes</h3><hr>
<p>
Debian: 'efax'
<p>
<strong><em>fax a copy of the file 'resume.txt' to the number `555-9099', using DTMF tone dialing</em></strong>
  <pre class="codeline"> efax -d /dev/modem -t T555-9099 resume.txt </pre>
<p>
<strong><em>fax all of the files with the '.fax' extension in the current directory to the number '555-9099', using DTMF tone dialing</em></strong>
  <pre class="codeline"> efax -d /dev/modem -t T555-9099 *.fax </pre>
<p>
<strong><em>fax all of the files listed in the file 'fax.list' to the number '555-9099', dialing `9' first to obtain an outside line, and using DTMF tone dialing</em></strong>
  <pre class="codeline"> efax -d /dev/modem -t T9,555-9099 $(cat fax.list) </pre>
<p>
<strong><em>set up efax to receive an incoming fax, saving the session log</em></strong>
to a file, 'faxlog'
  <pre class="codeline"> efax -d /dev/modem -kZ -w -iS0=1 2>&1 >> faxlog </pre>
This command starts efax and sets up the modem to wait for an incoming
<p>
<strong><em>automatically receive any incoming fax messages</em></strong>
  <pre class="codeline"> faxon </pre>
efax: Wed Feb 24 08:38:52 1999 efax v 0.8a (Debian release 08a-6)
Copyright 1996 Ed Casas
<p>
<strong><em>convert the file 'chart.pbm' for faxing</em></strong>
  <pre class="codeline"> efix -i pbm chart.pbm > chart.fax </pre>
This command converts a copy of the file 'chart.pbm' to the `tiffg3'
fax format, writing it to a file called 'chart.fax'. The original PBM
<p>
<strong><em>convert the PostScript file 'resume.ps' to fax format</em></strong>
  <pre class="codeline"> gs -q -sDEVICE=tiffg3 -dSAFER -dNOPAUSE -sOutputFile=resume.fax resume.ps < /dev/null </pre>
<p>
<strong><em>convert '19990325.001', a received fax file, to a PostScript file</em></strong>
  <pre class="codeline"> efix -o ps 19990325.001 > received.ps </pre>
<p>
<strong><em>dial the number '368-2208'</em></strong>
  <pre class="codeline"> ATDT3682208 </pre>
<p>
<h3>Remote shells</h3><hr>
<p>
<strong><em>Create a backdoor on a machine to allow remote connection to bash</em></strong>
  <pre class="codeline"> /bin/bash | nc -l 1234   ##(rather unwise...)</pre>
<p>
<h3>Ssh </h3><hr>....
<p>
<strong><em>Script executes itself on another host with one ssh command</em></strong>
  <pre class="codeline"> [ $1 == "client" ] && hostname || cat $0 | ssh $1 /bin/sh -s client</pre>
<p>
<strong><em>connect via ssh using mac address</em></strong>
  <pre class="codeline"> sudo arp -s 192.168.1.200 00:35:cf:56:b2:2g temp && ssh root@192.168.1.200</pre>
<p>
<strong><em>ssh and attach to a screen in one line.</em></strong>
  <pre class="codeline"> ssh -t user@host screen -x <screen name></pre>
<p>
<strong><em>Forward port 8888 to remote machine for SOCKS Proxy</em></strong>
  <pre class="codeline"> ssh -D 8888 user@site.com</pre>
<p>
<strong><em>Copy ssh keys to user@host to enable password-less ssh logins.</em></strong>
  <pre class="codeline"> ssh-copy-id user@host</pre>
<p>
<strong><em>Attach screen over ssh</em></strong>
  <pre class="codeline"> ssh -t remote_host screen -r</pre>
<p>
<strong><em>login to an ssh server as root 'root@server.net'</em></strong>
  <pre class="codeline"> alias s='ssh -l root'</pre>
<p>
<h3>Ssh tunnels </h3><hr>....
<p>
<strong><em>start a tunnel from some machine's port 80 to your local port 2001</em></strong>
  <pre class="codeline"> ssh -N -L2001:localhost:80 somemachine</pre>
<p>
<strong><em>plink ssh connect</em></strong>
  <pre class="codeline"> plink lyu0@mysshserver -pw 123456</pre>
<p>
<strong><em>Transfer large files/directories with no overhead over ssh</em></strong>
  <pre class="codeline"> ssh user@host "cd targetdir; tar cfp - *" | dd of=file.tar</pre>
<p>
<strong><em>Create an SSH connection (reverse tunnel) through your firewall.</em></strong>
  <pre class="codeline"> ssh -R 2001:localhost:22 username@<remote server ip></pre>
<p>
<h3>Ssh keys </h3><hr>....
<p>
<strong><em>create an alias to logon to the sourceforge shell</em></strong>
  <pre class="codeline"> alias sfshell='ssh -t user,project@shell.sourceforge.net create'</pre>
<p>
<strong><em>Copy your SSH public key on a remote machine for passwordless access</em></strong>
  <pre class="codeline"> cat ~/.ssh/*.pub | ssh user@remote-system 'umask 077; cat >>.ssh/authorized_keys'</pre>
<p>
<strong><em>Remove invalid key from the known_hosts file for the IP address</em></strong>
  <pre class="codeline"> ssh-keygen -R `host hostname | cut -d " " -f 4`</pre>
<p>
<strong><em>Remove invalid host keys from ~/.ssh/known_hosts</em></strong>
  <pre class="codeline"> ssh-keygen -R \[localhost\]:8022</pre>
<p>
<strong><em>SSH connection through host in the middle</em></strong>
  <pre class="codeline"> ssh -t reachable_host ssh unreachable_host</pre>
<p>
<strong><em>ssh autocomplete on the known hosts</em></strong>
  <pre class="codeline"> complete -W "$(echo `cut -f 1 -d ' ' ~/.ssh/known_hosts | sed -e s/,.*//g | uniq | grep -v "\["`;)" ssh</pre>
<p>
<strong><em>find the difference between two nodes</em></strong>
  <pre class="codeline"> diff <(ssh nx915000 "rpm -qa") <(ssh nx915001 "rpm -qa")</pre>
<p>
<strong><em>Copy something to multiple SSH hosts with a Bash loop</em></strong>
  <pre class="codeline"> for h in host1 host2 host3 host4 ; { scp file user@h$:/path/; }</pre>
<p>
<strong><em>Setup a persistant SSH tunnel w/ pre-shared key authentication</em></strong>
  <pre class="codeline"> autossh -f -i /path/to/key -ND local-IP:PORT User@Server</pre>
<p>
<strong><em>live ssh network throughput test</em></strong>
  <pre class="codeline"> pv /dev/zero|ssh $host 'cat > /dev/null'</pre>
<p>
<strong><em>setup a tunnel from destination machine port 80 to localhost 2001,</em></strong>
  <pre class="codeline"> ssh -N -L2001:localhost:80 -o "ProxyCommand ssh someuser@hubmachine nc -w 5 %h %p" someuser@destinationmachine</pre>
<p>
<strong><em>Enter your ssh password one last time</em></strong>
  <pre class="codeline"> cat .ssh/id_dsa.pub | ssh server.net "[ -d .ssh ] || mkdir .ssh ; cat >> .ssh/authorized_keys"</pre>
<p>
<strong><em>open a seperate konsole tab and ssh to each of N servers</em></strong>
  <pre class="codeline"> for i in $(cat listofservers.txt); do konsole --new-tab -e ssh $i; done</pre>
<p>
<strong><em>browse files on an ssh server</em></strong>
  <pre class="codeline"> mc</pre>
<p>
<p>
<h3>Expect</h3><hr>
<p>
Expect is a useful tool to allow the scripting of tasks 
which would normally require some user interaction. Expect
includes its own simple scripting language. Expect is available
on the majority of unix type operating systems, including 
Mac OSX. 
<p>
Entire books exist about expect, Expect is an extension to 
the 'tcl' language.
<p>
<center><h3 class="page-title"></h3></center> expect related tools
.. expectk - a graphical front end
.. autoexpect - create a script from an interactive session
..
<p>
<strong><em>set the variable 'x' to the value 30</em></strong>
  <pre class="codeline"> set x 30</pre>
<p>
<strong><em>make a script timeout after 20 seconds</em></strong>
  <pre class="codeline"> set timeout 20</pre>
<p>
The above is useful when interacting with servers which
may not respond.
<p>
<strong><em>start the 'ssh' program from within an expect script</em></strong>
  <pre class="codeline"> spawn ssh</pre>
<p>
<strong><em>wait for the program spawn to say exactly "hello!"</em></strong>
  <pre class="codeline"> expect "hello!"</pre>
<p>
<strong><em>wait for the program spawned to say anything with boggle in it</em></strong>
  <pre class="codeline"> expect "*boggle*"</pre>
<p>
<strong><em>send the response "yes" to the spawned program</em></strong>
  <pre class="codeline"> send "yes\n"</pre>
  <pre class="codeline"> send "yes\r"     ##(more or less the same)</pre>
  <pre class="codeline"> send yes\r       ##(also possible, but not always)</pre>
<p>
<strong><em>escape a '-' character in a send command</em></strong>
  <pre class="codeline"> send "/-4.5\n"</pre>
<p>
If we just wrote 'send "-4.5\n"' then expect would think
that -4 was a flag and would crash
<p>
<strong><em>a simple expect script</em></strong>
#!/usr/bin/expect
set timeout 20 
set name [lindex $argv 0] 
set user [lindex $argv 1]
set password [lindex $argv 2] 
spawn telnet $name
expect "login:"
send "$user " 
expect "Password:"
send "$password "
interact # hands interaction to the user
,,,
<p>
<h3>The shell</h3><hr>
<p>
<strong><em>set command line editing in the 'vi' mode </em></strong>
  <pre class="codeline"> set -o vi      ##(gives "vi" like command line editing keystrokes)</pre>
  <pre class="codeline"> set -o emacs   ##(return to the default mode) </pre>
<p>
<h3>Ram memory </h3><hr>
<p>
<strong><em>View memory utilisation</em></strong>
  <pre class="codeline"> sar -r</pre>
<p>
<strong><em>free swap</em></strong>
  <pre class="codeline"> free -b | grep "Swap:" | sed 's/ * / /g' | cut -d ' ' -f2</pre>
<p>
<strong><em>Find the ratio between ram usage and swap usage.</em></strong>
  <pre class="codeline"> sysctl -a | grep vm.swappiness</pre>
<p>
<strong><em>Monitor memory usage</em></strong>
  <pre class="codeline"> watch vmstat -sSM</pre>
<p>
<h3>Virtual memory </h3><hr>....
<p>
<strong><em>Add temporary swap space</em></strong>
  <pre class="codeline"> dd if=/dev/zero of=/swapfile bs=1M count=64; chmod 600 /swapfile; mkswap /swapfile; swapon /swapfile</pre>
<p>
<strong><em>Clean swap area after using a memory hogging application</em></strong>
  <pre class="codeline"> swapoff -a ; swapon -a</pre>
<p>
<strong><em>vmstat/iostat with timestamp</em></strong>
  <pre class="codeline"> vmstat 1 | awk '{now=strftime("%Y-%m-%d %T "); print now $0}'</pre>
<p>
<h3>Disks</h3><hr>
<p>
<strong><em>check free disk space and display sizes in an easy to read format</em></strong>
  <pre class="codeline"> df -h</pre>
<p>
<strong><em>show file and directory sizes for the current directory</em></strong>
  <pre class="codeline"> du -sh * </pre>
<p>
<h3>Hard disk partitions </h3><hr>....
<p>
<strong><em>list all hd partitions</em></strong>
  <pre class="codeline"> awk '/d.[0-9]/{print $4}' /proc/partitions</pre>
<p>
<strong><em>show disk partitions and sizes</em></strong>
  <pre class="codeline"> sudo fdisk -l</pre>
<p>
<strong><em>gparted: to repartition a disk</em></strong>
<p>
<h3>Serial connections </h3><hr>
<p>
<strong><em>Test a serial connection</em></strong>
  <pre class="codeline"> host A: cat /proc/dev/ttyS0 host B: echo hello > /dev/ttyS0</pre>
<p>
<h3>The network</h3><hr>
<p>
<strong><em>Network Information</em></strong>
  <pre class="codeline"> ntop</pre>
<p>
<strong><em>restart network manager</em></strong>
  <pre class="codeline"> sudo /etc/init.d/networking restart</pre>
<p>
<strong><em>Consolle based network interface monitor</em></strong>
  <pre class="codeline"> ethstatus -i eth0</pre>
<p>
<strong><em>Display ncurses based network monitor</em></strong>
  <pre class="codeline"> nload -u m eth0</pre>
<p>
<strong><em>Show apps that use internet connection at the moment.</em></strong>
  <pre class="codeline"> netstat -lantp | grep -i stab | awk -F/ '{print $2}' | sort | uniq</pre>
<p>
<strong><em>Monitor RX/TX packets and any subsquent errors</em></strong>
  <pre class="codeline"> watch 'netstat -aniv'</pre>
<p>
<strong><em>Most simple way to get a list of open ports</em></strong>
  <pre class="codeline"> netstat -lnp</pre>
<p>
<strong><em>eth-tool summary of eth# devices</em></strong>
  <pre class="codeline"> for M in 0 1 2 3 ; do echo eth$M ;/sbin/ethtool eth$M | grep -E "Link|Speed" ; done</pre>
<p>
<strong><em>Check the status of a network interface</em></strong>
  <pre class="codeline"> mii-tool [if]</pre>
<p>
<strong><em>find all active IP addresses in a network</em></strong>
  <pre class="codeline"> arp-scan -l</pre>
<p>
<strong><em>Create a persistent connection to a machine</em></strong>
  <pre class="codeline"> ssh -MNf <user>@<host></pre>
<p>
<strong><em>directly ssh to host B that is only accessible through host A</em></strong>
  <pre class="codeline"> ssh -t hostA ssh hostB</pre>
<p>
<strong><em>Show all programs on UDP and TCP ports with timer information</em></strong>
  <pre class="codeline"> netstat -putona</pre>
<p>
<strong><em>ping the host bfi.org</em></strong>
  <pre class="codeline"> ping bfi.org </pre>
<p>
<strong><em>mtr, better than traceroute and ping combined</em></strong>
  <pre class="codeline"> mtr google.com</pre>
<p>
<strong><em>Lists all listening ports together with the PID of the associated</em></strong>
  <pre class="codeline"> netstat -tlnp</pre>
<p>
<strong><em>finger the user bradley@ap.spl.org</em></strong>
finger bradley@ap.spl.org 
[ap.spl.org]
Login: bradley Name: Bradley J Milton
,,,
<p>
<strong><em>output the users who are currently logged in to the system ap.spl.org</em></strong>
  <pre class="codeline"> finger @ap.spl.org </pre>
<p>
<strong><em>find the IP address of the host linart.net</em></strong>
<p>
dig linart.net 
...output messages...
;; ANSWER SECTION:<br/>
,,,
<p>
<strong><em>find the host name that corresponds to the IP address 216.92.9.215</em></strong>
  <pre class="codeline"> dig -x 216.92.9.215 </pre>
<p>
<strong><em>output the name of the Whois Server for linart.net</em></strong>
  <pre class="codeline"> whois linart.net </pre>
<p>
<strong><em>view the domain record for linart.net, using the whois.networksolutions.com </em></strong>
  <pre class="codeline"> whois -h whois.networksolutions.com linart.net </pre>
<p>
<strong><em>send the message 'get up!' to the terminal where user 'sleepy' is logged in</em></strong>
  <pre class="codeline"> write sleepy get up </pre>
<p>
<strong><em>output the contents of '/etc/motd' to all logged-in terminals,</em></strong>
  <pre class="codeline"> wall /etc/motd </pre>
<p>
<strong><em>output the text 'hello?' to all logged-in terminals</em></strong>
  <pre class="codeline"> wall hello? </pre>
<p>
<strong><em>disallow messages to be written to your terminal</em></strong>
  <pre class="codeline"> mesg n </pre>
<p>
<strong><em>output the current access state of your terminal</em></strong>
  <pre class="codeline"> mesg </pre>
<p>
<strong><em>request a chat with the user kat@kent.edu</em></strong>
  <pre class="codeline"> talk kat@kent.edu </pre>
<p>
<strong><em>view network traffic with protocols: wireshark</em></strong>
<p>
<strong><em>find which tcp ports are currently in use</em></strong>
  <pre class="codeline"> lsof | grep TCP   ##(lsof lists all open unix 'files' including pipes)</pre>
<p>
<strong><em>lots of detailed information</em></strong>
  <pre class="codeline"> lshw -C Network</pre>
<p>
<strong><em>restart the gnome graphical network manager apple</em></strong>
  <pre class="codeline"> sudo restart network-manager</pre>
<p>
<h3>Analysing the network </h3><hr>....
<p>
<strong><em>make an alias (command) to ping a yahoo server without any dns</em></strong>
   <pre class="codeline"> alias testnet='ping 69.147.114.224' </pre>
##(this can determine if the problem is dns or tcpip)
<p>
<strong><em>Router discovery</em></strong>
  <pre class="codeline"> sudo arp-scan 192.168.1.0/24 -interface eth0</pre>
<p>
<strong><em>Show all machines on the network</em></strong>
  <pre class="codeline"> nmap 192.168.0-1.0-255 -sP</pre>
<p>
<strong><em>Localize provenance of current established connections</em></strong>
  <pre class="codeline"> for i in $(netstat --inet -n|grep ESTA|awk '{print $5}'|cut -d: -f1);do geoiplookup $i;done</pre>
<p>
<strong><em>List the number and type of active network connections</em></strong>
  <pre class="codeline"> netstat -ant | awk '{print $NF}' | grep -v '[a-z]' | sort | uniq -c</pre>
<p>
<h3>Sniffing network traffic</h3><hr>
<p>
<strong><em>Remotely sniff traffic and pass to snort</em></strong>
  <pre class="codeline"> ssh root@pyramid \ "tcpdump -nn -i eth1 -w -" | snort -c /etc/snort/snort.conf -r -</pre>
<p>
<h3>Tcpip ports </h3><hr>....
<p>
<strong><em>List all TCP opened ports on localhost in LISTEN mode</em></strong>
  <pre class="codeline"> netstat -nptl</pre>
<p>
<strong><em>which program is this port belongs to ?</em></strong>
  <pre class="codeline"> lsof -i tcp:80</pre>
<p>
<strong><em>list all opened ports on host</em></strong>
  <pre class="codeline"> nmap -p 1-65535 --open localhost</pre>
<p>
<strong><em>list all opened ports on host</em></strong>
  <pre class="codeline"> sudo lsof -P -i -n -sTCP:LISTEN</pre>
<p>
<strong><em>Show apps that use internet connection at the moment.</em></strong>
  <pre class="codeline"> netstat -lantp | grep -i establ | awk -F/ '{print $2}' | sort | uniq</pre>
<p>
<strong><em>Port Knocking!</em></strong>
  <pre class="codeline"> knock <host> 3000 4000 5000 && ssh -p <port> user@host && knock <host> 5000 4000 3000</pre>
<p>
<strong><em>Show apps that use internet connection at the moment.</em></strong>
  <pre class="codeline"> ss -p</pre>
<p>
<strong><em>netcat as a portscanner</em></strong>
  <pre class="codeline"> nc -v -n -z -w 1 127.0.0.1 22-1000</pre>
<p>
<h3>Ip addresses </h3><hr>....
<p>
<strong><em>Get My Public IP Address</em></strong>
  <pre class="codeline"> wget -qO - http://myip.dk/ | egrep -m1 -o '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}'</pre>
<p>
<strong><em>What is my ip?</em></strong>
  <pre class="codeline"> curl -s checkip.dyndns.org | grep -Eo '[0-9\.]+'</pre>
<p>
<strong><em>What is my ip? (hardened)</em></strong>
  <pre class="codeline"> curl --connect-timeout 3 http://www.whatismyip.org/</pre>
<p>
<strong><em>geoip information</em></strong>
  <pre class="codeline"> GeoipLookUp(){ curl -A "Mozilla/5.0" -s "http://www.geody.com/geoip.php?ip=$1" | grep "^IP.*$1" | html2text; }</pre>
<p>
<strong><em>Get My Public IP Address</em></strong>
  <pre class="codeline"> curl -s http://whatismyip.org/</pre>
<p>
<strong><em>display ip address</em></strong>
  <pre class="codeline"> curl -s http://myip.dk | grep '<title>' | sed -e 's/<[^>]*>//g'</pre>
<p>
<strong><em>Filter IPs out of files</em></strong>
  <pre class="codeline"> egrep -o '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' file.txt</pre>
<p>
<h3>Dhcp dynamic host control protocol </h3><hr>....
<p>
<center><h3 class="page-title"></h3></center> tools
.. dhclient - sets up a dhcp client from the command line
..
<p>
<strong><em>get IPs with a DHCP lease</em></strong>
  <pre class="codeline"> egrep "^lease" /var/lib/dhcp/db/dhcpd.leases |awk '{ print $2 }'</pre>
<p>
<strong><em>randomize hostname and mac address, force dhcp renew. (for</em></strong>
  <pre class="codeline"> dhclient -r && rm -f /var/lib/dhcp3/dhclient* && sed "s=$(hostname)=REPLACEME=g" -i /etc/hosts && hostname "$(echo $RANDOM | md5sum | cut -c 1-7 | tr a-z A-Z)" && sed "s=REPLACEME=$(hostname)=g" -i /etc/hosts && macchanger -e eth0 && dhclient</pre>
<p>
<h3>Ping </h3><hr>....
<p>
<strong><em>ping a range of IP addresses</em></strong>
  <pre class="codeline"> nmap -sP 192.168.1.100-254</pre>
<p>
<strong><em>ping a host until it responds, then play a sound, then exit</em></strong>
  <pre class="codeline"> beepwhenup () { echo 'Enter host you want to ping:'; read PHOST; if [[ "$PHOST" == "" ]]; then exit; fi; while true; do ping -c1 -W2 $PHOST 2>&1 >/dev/null; if [[ "$?" == "0" ]]; then for j in $(seq 1 4); do beep; done; ping -c1 $PHOST; break; fi; done; }</pre>
<p>
<h3>Ifconfig </h3><hr>....
<p>
@@ <a href="http://linuxhelp.blogspot.com/2006/11/ifconfig-dissected-and-demystified.html" <a href="http://linuxhelp.blogspot.com/2006/11/ifconfig-dissected-and-demystified.html</a>">http://linuxhelp.blogspot.com/2006/11/ifconfig-dissected-and-demystified.html</a></a>
how to use ifconfig
<p>
<strong><em>setup ethernet network device specifying an ip address</em></strong>
  <pre class="codeline">  ifconfig eth0 192.168.0.1 netmask 255.255.255.0 broadcast 192.168.0.255 up</pre>
<p>
<h3>Proxy servers </h3><hr>....
<p>
<strong><em>Create a single-use TCP proxy with debug output to stderr</em></strong>
  <pre class="codeline"> socat -v tcp4-l:<port> tcp4:<host>:<port></pre>
<p>
<strong><em>Create a single-use TCP (or UDP) proxy</em></strong>
  <pre class="codeline"> nc -l -p 2000 -c "nc example.org 3000"</pre>
<p>
<h3>Mac addresses </h3><hr>....
<p>
A 'mac' address is a unique number associated with a given 
network interface on a computer. A network interface is a wireless card,
ethernet card, modem, etc. No 2 network devices in the world have
the same mac address, or at least shouldnt.
<p>
@@ <a href="http://linuxhelp.blogspot.com/2005/09/how-to-change-mac-address-of-your.html" <a href="http://linuxhelp.blogspot.com/2005/09/how-to-change-mac-address-of-your.html</a>">http://linuxhelp.blogspot.com/2005/09/how-to-change-mac-address-of-your.html</a></a>
how to change ('spoof') the mac address for a given network device
on the local computer.
<p>
@@ deb: macchanger
a package to change the mac address for a given network 
interface. this may not work with all wireless cards.
<p>
<strong><em>Determine MAC address of remote host when you know its IP address</em></strong>
  <pre class="codeline"> arping 192.168.1.2</pre>
<p>
<strong><em>Insert a colon between every two digits</em></strong>
  <pre class="codeline"> sed 's/\(..\)/\1:/g;s/:$//' mac_address_list</pre>
<p>
<strong><em>find the mac address of all active network interfaces (wireless, ethernet ...)</em></strong>
  <pre class="codeline"> ifconfig -a | grep HWaddr</pre>
<p>
<strong><em>show the mac address to ip table for the local computer</em></strong>
  <pre class="codeline"> arp</pre>
<p>
<strong><em>Resets your MAC to a random MAC address to make you harder to</em></strong>
  <pre class="codeline"> ran=$(head /dev/urandom | md5sum); MAC=00:07:${ran:0:2}:${ran:3:2}:${ran:5:2}:${ran:7:2}; sudo ifconfig wlan0 down hw ether $MAC; sudo ifconfig wlan0 up; echo ifconfig wlan0:0</pre>
<p>
<strong><em>change or 'spoof' the mac address of the ethernet network device</em></strong>
ifconfig eth0 down
ifconfig eth0 hw ether 00:80:48:BA:d1:30
ifconfig eth0 up
ifconfig eth0 | grep HWaddr
,,,
<p>
<strong><em>change the mac address for the wireless card 'wlan0' to the specified</em></strong>
  <pre class="codeline"> macchanger --mac 00:00:13:37:00:00 wlan0 </pre>
##(this only works if the wireless card driver supports the operation)
<p>
<h3>Monitoring network bandwidth </h3><hr>....
<p>
<strong><em>watch data usage on eth0</em></strong>
  <pre class="codeline"> watch ifconfig eth0</pre>
<p>
<strong><em>monitor bandwith and data transfer</em></strong>
  <pre class="codeline"> iptables, bmon, vnstat, ntop, bwm-ng, iftop, iptraf, darkstat, mrtg ...</pre>
<p>
<strong><em>output bmon data as html</em></strong>
  <pre class="codeline"> bmon -O html:path=/home/var/www/html/data/bmon</pre>
<p>
<strong><em>create a new database with data about the eth0 (ethernet interface)</em></strong>
  <pre class="codeline"> vnstat -u -i eth0</pre>
<p>
<strong><em>store data about the 'ppp0' interface (a modem, or wireless usb dongle) etc</em></strong>
  <pre class="codeline"> sudo vnstat -u -i ppp0</pre>
<p>
<strong><em>show an ascii graph of data transfer by the hour </em></strong>
  <pre class="codeline"> vnstat -h</pre>
<p>
<strong><em>look for the number of RX bytes and TX bytes of the active interface</em></strong>
  <pre class="codeline"> ifconfig</pre>
<p>
<h3>Network configuration </h3><hr>....
<p>
udp uses no acknowledgement
used by: tftp (trivial file transfer protocol)
, snmp (simple network manage protocol), dhcp, dns
uses port numbers
uses ip
the only improvement on ip are port numbers
<p>
<strong><em>tcpdump, ethereal to see network traffic</em></strong>
<p>
<strong><em>monitor the ethernet network traffic</em></strong>
  <pre class="codeline"> while :; do netstat -in | grep ath0 ; sleep 1 ; done </pre>
<p>
<strong><em>show network interfaces</em></strong>
  <pre class="codeline"> ip link show</pre>
  <pre class="codeline"> netstat -in </pre>
<p>
<strong><em>list the routing table</em></strong>
  <pre class="codeline"> ip route show</pre>
<p>
<strong><em>configure a network connection via dhcp</em></strong>
  <pre class="codeline"> dhclient</pre>
<p>
<h3>Wireless networks </h3><hr>....
<p>
<center><h3 class="page-title"></h3></center> tools
.. iwlist - show available wlans
.. iwgetid - shows interfaces
.. iwpriv
.. iwspy - show other lan clients
.. iwevent - monitor wlans
.. dhclient - set up dhcp clients
..
<p>
<strong><em>connect to a network called 'beach cafe' </em></strong>
  <pre class="codeline"> iwconfig etho essid "beach cafe"</pre>
<p>
<strong><em>connect to the 'uconnect' network</em></strong>
  <pre class="codeline"> iwconfig eth0 essid uconnect</pre>
<p>
<strong><em>connect with a wep key</em></strong>
  <pre class="codeline"> iwconfig eth0 essid uconnect key s:Password</pre>
  <pre class="codeline"> iwconfig eth0 essid uconnect key 1fa24 ##(hex notation)</pre>
<p>
<strong><em>get a dhcp ip address from the network</em></strong>
  <pre class="codeline"> dhclient   ##(run this after 'iwconfig')</pre>
<p>
<strong><em>show the mac address of the access point</em></strong>
  <pre class="codeline"> iwgetid -ap</pre>
<p>
<strong><em>show available encryption algorithms</em></strong>
  <pre class="codeline"> iwlist encryption</pre>
<p>
<strong><em>show transmitter power </em></strong>
  <pre class="codeline"> iwlist power</pre>
<p>
<strong><em>show the status of the wireless interface eth1</em></strong>
  <pre class="codeline"> iwconfig eth1</pre>
<p>
If an ESSID:"name" appears then connected
<p>
<strong><em>show information about available wireless networks in range</em></strong>
  <pre class="codeline"> iwlist scan</pre>
<p>
<strong><em>view all networks reachable from the ra0 interface (wireless card) </em></strong>
  <pre class="codeline"> iwlist ra0 scan</pre>
<p>
<strong><em>very mysterious</em></strong>
  <pre class="codeline"> iwpriv  ??</pre>
<p>
<h3>Wireless security </h3><hr>....
<p>
@@ <a href="http://www.aircrack-ng.org" <a href="http://www.aircrack-ng.org</a>">http://www.aircrack-ng.org</a></a>
The official site
<p>
<center><h3 class="page-title"></h3></center> wireless security auditing programs
.. aircrack-ng, obtain wireless network passwords
.. kismet, see wireless networks in range and clients
..
<p>
<strong><em>sniff wireless traffic using the 'ra0' interface </em></strong>
  <pre class="codeline"> sudo airodump-ng ra0</pre>
<p>
<strong><em>Put the wireless card into monitor mode</em></strong>
  <pre class="codeline"> airmon-ng start <interface> <channel></pre>
<p>
<strong><em>capture wireless packets on channel 11 from the given mac address </em></strong>
  <pre class="codeline"> airodump-ng -c 11 --bssid 00:01:02:03:04:05 -w dump rausb0</pre>
##(the captured packets get written to a file prefixed 'dump')
##(>40000 packets should be captured in order to crack the key)
<p>
<strong><em>attempt to crack the wep key using captured data in 'dump-01.cap' </em></strong>
  <pre class="codeline"> aircrack-ng -b 00:01:02:03:04:05 dump-01.cap</pre>
<p>
<strong><em>use several captured data packet files using a wildcard</em></strong>
  <pre class="codeline"> aircrack-ng -b 00:01:02:03:04:05 dump*.cap </pre>
<p>
<strong><em>test if the 'ra0' wireless card supports 'injection'</em></strong>
  <pre class="codeline"> aireplay-ng -9 ra0      ##(or run with sudo) </pre>
##(injection is required for speeding up wireless attacks)
<p>
<h3>User and group accounts</h3><hr>
<p>
Unix style operating systems use user and group accounts to attempt to
control access to files, folders and other resources. The super-user on a
unix-style operating system is called the 'root' user
<p>
<h3>Users </h3><hr>....
<p>
<strong><em>find out who you logged onto the machine as</em></strong>
  <pre class="codeline"> who am i</pre>
<p>
<strong><em>find out what users are logged onto the system </em></strong>
  <pre class="codeline"> who</pre>
<p>
<strong><em>display which user is running a process from a given port name</em></strong>
  <pre class="codeline"> fuser -nu tcp 3691</pre>
<p>
<strong><em>list what rights you will have after a 'sudo' command</em></strong>
  <pre class="codeline"> sudo -l</pre>
<p>
<strong><em>quickly add user accounts to the system and force a password</em></strong>
  <pre class="codeline"> for name in larry moe schemp; do useradd $name; echo 'password' | passwd --stdin $name; chage -d 0 $name; done</pre>
<p>
<strong><em>Add existing user to a group</em></strong>
  <pre class="codeline"> usermod -a -G groupname username</pre>
<p>
<strong><em>list files not owned by any user or group</em></strong>
  <pre class="codeline"> find / -nouser -o -nogroup -print</pre>
<p>
<strong><em>list your group memberships</em></strong>
  <pre class="codeline"> groups </pre>
##(prints 'steward galley crew')
<p>
<strong><em>list the group memberships of user blackbeard</em></strong>
  <pre class="codeline"> groups blackbeard </pre>
<p>
<strong><em>output a list of the members of the galley group</em></strong>
  <pre class="codeline"> members galley </pre>
<p>
<strong><em>change the group ownership of file 'cruise' to bridge</em></strong>
  <pre class="codeline"> chgrp bridge cruise </pre>
<p>
<strong><em>give group ownership of the 'maps' directory and all the files it contains to the bridge group</em></strong>
  <pre class="codeline"> chgrp -R bridge maps </pre>
<p>
<strong><em>write-protect the file 'cruise' so that no other users can change it</em></strong>
  <pre class="codeline"> chmod go-w cruise </pre>
<p>
<strong><em>make the file 'cruise' private from all users but yourself</em></strong>
  <pre class="codeline"> chmod go= cruise </pre>
<p>
<strong><em>make the file 'cruise' both world readable and world writable</em></strong>
  <pre class="codeline"> chmod a+rw cruise </pre>
<p>
<strong><em>give execute permission to all users for the file 'myscript' </em></strong>
  <pre class="codeline"> chmod a+x myscript </pre>
<p>
<strong><em>Get a quick list of all user and group owners of files and dirs</em></strong>
  <pre class="codeline"> find -printf '%u %g\n' | sort | uniq</pre>
<p>
Packages
@@ deb: acct
show information about users
@@ deb: quota
allows the administrator to place disk quotas on 
users account
<p>
<strong><em>Remove executable bit from all files in the current directory</em></strong>
  <pre class="codeline"> find . ! -type d -exec chmod -x {}\;</pre>
<p>
<strong><em>Switch to a user with "nologin" shell</em></strong>
  <pre class="codeline"> sudo -u username bash</pre>
<p>
<strong><em>Recursive chmod all files and directories within the current folder</em></strong>
  <pre class="codeline"> chmod -R 774 .</pre>
<p>
<strong><em>Create a listing of all possible permissions and their octal</em></strong>
  <pre class="codeline"> touch /tmp/$$;for N in `seq -w 0 7777|grep -v [89]`; do chmod $N /tmp/$$; P=`ls -l /tmp/$$ | awk '{print $1}'`; echo $N $P; done;rm /tmp/$$</pre>
<p>
<strong><em>show all user accounts on the local computer</em></strong>
  <pre class="codeline"> cat /etc/passwd    </pre>
  <pre class="codeline"> cat /etc/passwd | grep /home  ##(shows only non-application users)</pre>
<p>
<strong><em>List all groups and the user names that were in each group</em></strong>
  <pre class="codeline"> for u in `cut -f1 -d: /etc/passwd`; do echo -n $u:; groups $u; done | sort</pre>
<p>
<strong><em>add a new user account</em></strong>
  <pre class="codeline"> adduser</pre>
<p>
<strong><em>add a new group account</em></strong>
  <pre class="codeline"> addgroup</pre>
<p>
<strong><em>change ownership of a file to another user</em></strong>
  <pre class="codeline"> chown</pre>
<p>
<strong><em>change the group ownership of a file or files</em></strong>
  <pre class="codeline"> chgrp</pre>
<p>
<strong><em>print what groups a user belongs to </em></strong>
  <pre class="codeline"> groups</pre>
<p>
<strong><em>change the password for the 'mjb' user account</em></strong>
  <pre class="codeline"> passwd mjb</pre>
<p>
<strong><em>find all files on the computer owned by the 'bob' user</em></strong>
  <pre class="codeline"> find / -user bob -ls</pre>
<p>
<h3>File permissions </h3><hr>....
<p>
<strong><em>give any files that don't already have it group read permission</em></strong>
  <pre class="codeline"> find . -type f ! -perm /g=r -exec chmod g+r {} +</pre>
<p>
<strong><em>remove executable bit from all files in the current directory</em></strong>
  <pre class="codeline"> chmod -R -x+X *</pre>
<p>
<strong><em>recursively reset file or folder permissions</em></strong>
  <pre class="codeline"> find public_html/stuff -type d -exec chmod 755 {} + -or -type f -exec chmod 644 {} +</pre>
<p>
<strong><em>Change the ownership of all files owned by one user.</em></strong>
  <pre class="codeline"> find /home -uid 1056 -exec chown 2056 {} \;</pre>
<p>
<strong><em>create directory and set owner/group/mode in one shot</em></strong>
  <pre class="codeline"> install -o user -g group -m 0700 -d /path/to/newdir</pre>
<p>
<h3>Security</h3><hr>
<p>
<strong><em>Unix security checker</em></strong>
  <pre class="codeline"> tiger</pre>
<p>
<h3>Net security </h3><hr>....
<p>
<strong><em>find running binary executables that were installed irregularly </em></strong>
  <pre class="codeline"> cat /var/lib/dpkg/info/*.list > /tmp/listin ; ls /proc/*/exe |xargs -l readlink | grep -xvFf /tmp/listin; rm /tmp/listin</pre>
<p>
<strong><em>Scan Network for Rogue APs.</em></strong>
  <pre class="codeline"> nmap -A -p1-85,113,443,8080-8100 -T4 --min-hostgroup 50 --max-rtt-timeout 2000 --initial-rtt-timeout 300 --max-retries 3 --host-timeout 20m --max-scan-delay 1000 -oA wapscan 10.0.0.0/8</pre>
<p>
<strong><em>Create a backdoor on a machine to allow remote connection to bash</em></strong>
  <pre class="codeline"> nc -vv -l -p 1234 -e /bin/bash</pre>
<p>
<strong><em>Find brute force attempts on SSHd</em></strong>
<pre class="codeline"> cat /var/log/secure | grep sshd | grep Failed | sed 's/invalid//' | sed 's/user//' | awk '{print $11}' | sort | uniq -c | sort -n</pre>
<p>
<p>
<strong><em>Use a decoy while scanning ports to avoid getting caught by the</em></strong>
  <pre class="codeline"> sudo nmap -sS 192.168.0.10 -D 192.168.0.2</pre>
<p>
<strong><em>Conficker Detection with NMAP</em></strong>
  <pre class="codeline"> nmap -PN -d -p445 --script=smb-check-vulns --script-args=safe=1 IP-RANGES</pre>
<p>
<strong><em>Find files with root setuids settings</em></strong>
  <pre class="codeline"> sudo find / -user root -perm -4000 -print</pre>
<p>
<strong><em>Trojan inverse shell</em></strong>
  <pre class="codeline"> nc -l -p 2000 -e /bin/bash</pre>
<p>
<strong><em>Check for login failures and summarize</em></strong>
  <pre class="codeline"> zgrep "Failed password" /var/log/auth.log* | awk '{print $9}' | sort | uniq -c | sort -nr | less</pre>
<p>
<strong><em>Block known dirty hosts from reaching your machine</em></strong>
  <pre class="codeline"> wget -qO - http://infiltrated.net/blacklisted|awk '!/#|[a-z]/&&/./{print "iptables -A INPUT -s "$1" -j DROP"}'</pre>
<p>
<strong><em>retrieve top ip threats from <a href="http://isc.sans.<a" <a href="http://isc.sans.<a</a>">http://isc.sans.<a</a></a> href="org/sources.html"><em><code>org/sources.html</code></em></a> </em></strong>
  <pre class="codeline"> curl -s http://isc.sans.org/sources.html|grep "ipinfo.html"|awk -F"ip="</pre>
<p>
<strong><em>Locking and unlocking files and mailboxes</em></strong>
  <pre class="codeline"> lockfile</pre>
<p>
<strong><em>Add a line for your username in the /etc/sudoers file</em></strong>
  <pre class="codeline"> echo 'loginname ALL=(ALL) ALL' >> /etc/sudoers</pre>
<p>
<h3>Shredding </h3><hr>....
<p>
Shredding data means deleting data in a way that makes that
data not recoverable, by analogy with the process of 
shredding a document.
<p>
<strong><em>Securely destroy data (including whole hard disks)</em></strong>
  <pre class="codeline"> shred targetfile</pre>
<p>
<strong><em>securely overwrite a file with random junk, rename it to clear</em></strong>
  <pre class="codeline"> shred -vzu /tmp/junk-file-to-be-shredded</pre>
<p>
<strong><em>Shred an complete disk, by overwritting its content 10 times</em></strong>
  <pre class="codeline"> sudo shred -zn10 /dev/sda</pre>
<p>
<strong><em>Securely destroy data on given device</em></strong>
  <pre class="codeline"> # for i in $(seq 1 25); do dd if=/dev/urandom of=<your disk> bs=1M ; done</pre>
<p>
<h3>Firewalls </h3><hr>....
<p>
<strong><em>Redirect incoming traffic to SSH, from a port of your choosing</em></strong>
  <pre class="codeline"> iptables -t nat -A PREROUTING -p tcp --dport [port of your choosing] -j REDIRECT --to-ports 22</pre>
<p>
<strong><em>Tired of switching between proxy and no proxy? here's the solution</em></strong>
  <pre class="codeline"> iptables -t nat -A OUTPUT -d ! 10.0.0.0/8 -p tcp --dport 80 -j DNAT --to-destination 10.1.1.123:3128</pre>
<p>
<strong><em>Save iptables firewall info</em></strong>
  <pre class="codeline"> sudo iptables-save > /etc/iptables.up.rules</pre>
<p>
<strong><em>Remove an IP address ban that has been errantly blacklisted by</em></strong>
  <pre class="codeline"> denyhosts-remove $IP_ADDRESS</pre>
<p>
<strong><em>watch iptables counters</em></strong>
  <pre class="codeline"> watch 'iptables -vL'</pre>
<p>
<strong><em>Block all IP addresses and domains that have attempted brute</em></strong>
  <pre class="codeline"> (bzcat BZIP2_FILES && cat TEXT_FILES) | grep -E "Invalid user|PAM" | grep -o -E "from .+" | awk '{print $2}' | sort | uniq >> /etc/hosts.deny</pre>
<p>
<h3>Privacy</h3><hr>
<p>
@@ bleachbit
eliminate all sorts of caches, for the sake of privacy
<p>
@@ gpa
a graphical user interface to 'gpg'
@@ seahorse
the gnome graphical front-end for gpg
@@ kgpg
the kde front end
@@ firegpg
use gpg with webmail in firefox
@@ psi
use gpg with jabber xmpp im clients
@@ enigmail
integrate gpg with thunderbird email client
<p>
<center><h3 class="page-title"></h3></center> the 2 keys
.. your public - other people use it to send you stuff
.. your private - decrypt things sent to you
.. other persons public - you use it to send things to them
.. other person private - they use it to decrypt things sent to them
<p>
<strong><em>see if gpg is installed </em></strong>
  <pre class="codeline"> which gpg</pre>
  <pre class="codeline"> which gpg2    ##(the newer version)</pre>
<p>
<strong><em>make a public key for somebody to send you stuff</em></strong>
  <pre class="codeline"> gpg --gen-key</pre>
  <pre class="codeline"> input name and email address</pre>
<p>
The new key gets put on a key-ring
<p>
<strong><em>see what keys are on your keyring (like the one you just made)</em></strong>
  <pre class="codeline"> gpg --list-keys</pre>
<p>
<strong><em>show the fingerprint for a key</em></strong>
  <pre class="codeline"> gpg --fingerprint</pre>
<p>
'fingerprints' are used to make sure that the public key for 
somebody really is from them.
<p>
<strong><em>export the key with the given id to a file 'key.asc' and send to people</em></strong>
  <pre class="codeline"> gpg --armor --export 86d68542 --output key.asc</pre>
<p>
<strong><em>send the public key with given id to the mit 'public key server'</em></strong>
  <pre class="codeline"> gpg --keyserver=x-hkp://pgp.mit.edu --send-keys 86d68542 </pre>
<p>
or use the web interface at <a href="http://pgp.mit.edu" <a href="http://pgp.mit.edu</a>">http://pgp.mit.edu</a></a> to submit a public key
paste the '.asc' file created before with --armor
<p>
<strong><em>import a key file containing somebodies public key</em></strong>
  <pre class="codeline"> gpg --import key.asc</pre>
<p>
Then you use this key to encrypt things which you are going to 
send to them.
<p>
<strong><em>search the 'mit' public key server for a public key belonging to Ben stewart</em></strong>
  <pre class="codeline"> gpg --keyserver=x-hkp://pgp.mit.edu --search-keys Ben Stewart</pre>
<p>
Then enter a number corresponding to the desired user to enter that
key into your keyring
<p>
<h3>Keyring </h3><hr>....
<p>
<strong><em>private keys normally stored in</em></strong>
  <pre class="codeline"> /home/user/.gnupg/pubring.pgp</pre>
<p>
<strong><em>public keys stored in</em></strong>
  <pre class="codeline"> /home/user/.gnupg/secring.pgp</pre>
<p>
<h3>Signing keys </h3><hr>....
<p>
In order to verify that a public key really is from somebody, signing
is used. The more genuine people sign a key as authentic the more likely
it is that it will be authentic
<p>
<strong><em>sign the public key for 'bob' </em></strong>
  <pre class="codeline"> pgp --edit-key bob@stew.net</pre>
<p>
Then send this signed key to the public key server with the same command
as before.
<p>
<strong><em>list who has signed the public key for 'bob stewart'</em></strong>
  <pre class="codeline"> gpg --list-sigs bob@stew.net</pre>
<p>
<h3>Encrypting </h3><hr>....
<p>
<strong><em>encrypt 'file.pdf' for sending to user bob@stew.net (using his public key)</em></strong>
  <pre class="codeline"> gpg --encrypt --recipient 'bob@stew.net' file.pdf</pre>
<p>
creates a file 'file.pdf.gpg' which is encrypted 
<p>
<strong><em>encrypt a file for yourself (just use your own public key to encrypt it)</em></strong>
  <pre class="codeline"> gpg --encrypt --recipient 'me@here.net' file.pdf</pre>
<p>
<h3>Decrypting </h3><hr>....
<p>
<strong><em>decrypt a file sent to you (uses your private key)</em></strong>
  <pre class="codeline"> gpg --output f.txt --decrypt f.txt.asc</pre>
<p>
this prompts for your keyring passphrase
<p>
<strong><em>create a file 'leaks.txt.asc' which is signed as being from you but not encrypted</em></strong>
  <pre class="codeline"> gpg --clearsign leaks.txt</pre>
<p>
<strong><em>verify that a 'signed' file is from the person who it says it is</em></strong>
  <pre class="codeline"> gpg --verify leaks.txt</pre>
<p>
<strong><em>create a revocation certificate which is useful if you forget a password</em></strong>
  <pre class="codeline"> gpg --gen-revoke 8657698</pre>
<p>
Send the revocation certificate to the public key server if you forget
a password for a key
<p>
<h3>Encryption</h3><hr>
<p>
<center><h3 class="page-title"></h3></center> encryption tools
.. password gorilla - a graphical cross-platform encryption tool
.. gpg -
.. gpg2 -
.. cccrypt -
.. mcrypt - replacement for crypt, unrecommends itself
..
<p>
<strong><em>Quickly generate an MD5 hash for a text string using OpenSSL</em></strong>
  <pre class="codeline"> echo -n 'text to be encrypted' | openssl md5</pre>
<p>
<strong><em>gpg decrypt several files</em></strong>
  <pre class="codeline"> gpg --allow-multiple-messages --decrypt-files *</pre>
<p>
<strong><em>Safely store your gpg key passphrase.</em></strong>
  <pre class="codeline"> pwsafe -qa "gpg keys"."$(finger `whoami` | grep Name | awk '{ print $4" "$5 }')"</pre>
<p>
<strong><em>quickly encrypt a file with gnupg and email it with mailx</em></strong>
  <pre class="codeline"> cat file.txt | gpg2 --encrypt --armor --recipient "Disposable Key" | mailx -s "Email Subject" user@email.com</pre>
<p>
<strong><em>Mount a truecrypt drive from a file from the command line</em></strong>
  <pre class="codeline"> su -c "truecrypt --non-interactive truecrypt-file cryptshare -p PASSWORD"</pre>
<p>
<strong><em>Create/open/use encrypted directory</em></strong>
  <pre class="codeline"> encfs ~/.crypt ~/crypt</pre>
<p>
<strong><em>Encrypted archive with openssl and tar</em></strong>
  <pre class="codeline"> tar c folder_to_encrypt | openssl enc -aes-256-cbc -e > secret.tar.enc</pre>
<p>
<strong><em>Encrypted archive with openssl and tar</em></strong>
  <pre class="codeline"> openssl des3 -salt -in unencrypted-data.tar -out encrypted-data.tar.des3</pre>
<p>
<strong><em>rot13 simple substitution cipher via command line</em></strong>
  <pre class="codeline"> alias rot13='perl -pe "y/A-Za-z/N-ZA-Mn-za-m/;"'</pre>
<p>
<strong><em>Cracking a password protected .rar file</em></strong>
  <pre class="codeline"> for i in $(cat dict.txt);do unrar e -p$i protected.rar; if [ $? = 0 ];then echo "Passwd Found: $i";break;fi;done</pre>
<p>
MD5 SUMS ....
<p>
<strong><em>Create md5sum of files under the current dir excluding some</em></strong>
  <pre class="codeline"> find . -type d \( -name DIR1 -o -name DIR2 \) -prune -o -type f -print0 | xargs -r0 md5sum</pre>
<p>
<strong><em>recursively md5 all files in a tree</em></strong>
  <pre class="codeline"> find ./backup -type f -print0 | xargs -0 md5sum > /checksums_backup.md5</pre>
<p>
<strong><em>Verify MD5SUMS but only print failures</em></strong>
  <pre class="codeline"> md5sum --check MD5SUMS | grep -v ": OK"</pre>
<p>
<h3>Gpg </h3><hr>....
<p>
gpg stands for gnu pretty good privacy, more or less.
<p>
<strong><em>Receive, sign and send GPG key id</em></strong>
  <pre class="codeline"> caff <keyid></pre>
<p>
<strong><em>import gpg key from the web</em></strong>
  <pre class="codeline"> curl -s http://defekt.nl/~jelle/pubkey.asc | gpg --import</pre>
<p>
<strong><em>gpg decrypt a file</em></strong>
  <pre class="codeline"> gpg --output foo.txt --decrypt foo.txt.pgp</pre>
<p>
<strong><em>add a gpg key to aptitute package manager in a ubuntu system</em></strong>
  <pre class="codeline"> wget -q http://xyz.gpg -O- | sudo apt-key add -</pre>
<p>
<strong><em>encrypt the file 'log.txt' </em></strong>
  <pre class="codeline"> gpg -c log.txt  </pre>
<p>
<strong><em>gpg encrypt a file</em></strong>
  <pre class="codeline"> gpg --encrypt --recipient 'Foo Bar' foo.txt</pre>
<p>
<strong><em>decrypt file</em></strong>
  <pre class="codeline"> gpg log.txt  </pre>
<p>
<strong><em>encrypts or decrypts files in a specific directory</em></strong>
  <pre class="codeline"> for a in path/* ; do ccenrypt -K <password> $a; done</pre>
<p>
<h3>Passwords </h3><hr>....
<p>
<strong><em>Use md5 to generate a pretty hard to crack password</em></strong>
  <pre class="codeline"> echo "A great password" | md5sum</pre>
<p>
<strong><em>Generate a random password 30 characters long</em></strong>
  <pre class="codeline"> strings /dev/urandom | grep -o '[[:alnum:]]' | head -n 30 | tr -d '\n'; echo</pre>
<p>
<strong><em>Hiding password while reading it from keyboard</em></strong>
  <pre class="codeline"> save_state=$(stty -g);echo -n "Password: ";stty -echo;read password;stty "$save_state";echo "";echo "You inserted $password as password"</pre>
<p>
<strong><em>generate a unique and secure password for every website that you</em></strong>
  <pre class="codeline"> sitepass() { echo -n "$@" | md5sum | sha1sum | sha224sum | sha256sum | sha384sum | sha512sum | gzip - | strings -n 1 | tr -d "[:space:]" | tr -s '[:print:]' | tr '!-~' 'P-~!-O' | rev | cut -b 2-11; history -d $(($HISTCMD-1)); }</pre>
<p>
<h3>Recovering passwords </h3><hr>....
<p>
<strong><em>password recovery on debian</em></strong>
  <pre class="codeline"> init=/bin/bash; mount -o remount,rw /</pre>
<p>
<h3>Generating passwords </h3><hr>....
<p>
<strong><em>generate random password</em></strong>
  <pre class="codeline"> pwgen -Bs 10 1</pre>
<p>
<strong><em>Password Generation</em></strong>
  <pre class="codeline"> pwgen --alt-phonics --capitalize 9 10</pre>
<p>
<strong><em>a homemade password generator</em></strong>
  <pre class="codeline"> genpass(){local i x y z h;h=${1:-8};x=({a..z} {A..Z} {0..9});for ((i=0;i<$h;i++));do y=${x[$((RANDOM%${#x[@]}))]};z=$z$y;done;echo $z ;}</pre>
<p>
<strong><em>Generate Random Passwords</em></strong>
  <pre class="codeline"> dd if=/dev/urandom count=200 bs=1 2>/dev/null | tr "\n" " " | sed 's/[^a-zA-Z0-9]//g' | cut -c-16</pre>
<p>
<strong><em>generate random password</em></strong>
  <pre class="codeline"> openssl rand -base64 6</pre>
<p>
<strong><em>Generate 10 pronunciable passwords</em></strong>
  <pre class="codeline"> apg -a 0 -n 10</pre>
<p>
<strong><em>password generator</em></strong>
  <pre class="codeline"> genpass() { local h x y;h=${1:-8};x=( {a..z} {A..Z} {0..9} );y=$(echo ${x[@]} | tr ' ' '\n' | shuf -n$h | xargs);echo -e "${y// /}"; }</pre>
<p>
<strong><em>generate random password</em></strong>
  <pre class="codeline"> tr -dc 'a-zA-Z0-9' < /dev/urandom | head -c10</pre>
<p>
<strong><em>Creates a random password from /dev/urandom [0-9A-za-z]</em></strong>
  <pre class="codeline"> head -c $((<pw-lenght>-2)) /dev/urandom | uuencode -m - | sed -e '1d' -e '3d' | sed -e 's/=.*$//g'</pre>
<p>
<h3>Changing passwords </h3><hr>....
<p>
<strong><em>force change password for all users</em></strong>
  <pre class="codeline"> for i in `cat /etc/passwd | awk -F : '{ print $1 }';`; do passwd -e $i; done</pre>
<p>
<h3>Services</h3><hr>
<p>
<p>
'services' are a special type of process which are generally 
always running and are often started when the computer starts up.
Examples of services are a web-server, an ftp server ...
<p>
@@ debian: sysvconfig
allows a user to use the redhat-style 'service' command
<p>
<strong><em>show what services are available on the computer</em></strong>
  <pre class="codeline"> ls /etc/init.d</pre>
<p>
<strong><em>start or stop a service on a debian-style linux</em></strong>
  <pre class="codeline"> sudo /etc/init.d/servicename start|stop</pre>
<p>
<strong><em>restart the 'sshd' (secure shell) service</em></strong>
  <pre class="codeline"> sudo /etc/init.d/sshd restart</pre>
<p>
<strong><em>another way</em></strong>
  <pre class="codeline"> update-rc.d ... invoke-rc.d</pre>
<p>
<strong><em>stop the 'apache' service (with the sysvconfig package, or on redhat style)</em></strong>
  <pre class="codeline"> sudo service apache stop</pre>
<p>
<strong><em>other tools: rcconf, update-rc.d</em></strong>
<p>
<h3>Starting services at computer start up </h3><hr>....
<p>
Linux has a concept of 'run-levels' which are stages the 
computer reaches as it boots up, and as it shuts down. The higher the 
level the more 'booted-up' is. One can configure a server to start
at any particular 'run-level'.
<p>
<strong><em>configure apache to start up when the computer starts</em></strong>
  <pre class="codeline"> update-rc.d apache2 defaults   ##(this is debian-specific) </pre>
  <pre class="codeline"> update-rc.d apache2 start 20 2 3 4 5 . stop 80 0 1 6 .  ##(the same)</pre>
<p>
<strong><em>disable the apache webserver from starting when the computer starts</em></strong>
  <pre class="codeline"> update-rc.d -f apache2 remove</pre>
<p>
<strong><em>create automatic startup links manually (this is the older way)</em></strong>
  <pre class="codeline"> cd /etc/rc5.d/</pre>
  <pre class="codeline"> ln -s /etc/init.d/apache2 S20apache2</pre>
# when the computer enters run-level 5 (rc5.d) is will start (S) the 
# 'apache2' service with a priority of '20', that is, before all other
# services which have a priority number greater than 20
<p>
<h3>Pipes</h3><hr>
<p>
The unix pipeline is possibly the most important concept
in the unix world
<p>
<strong><em>cat large file to clipboard with speed-o-meter</em></strong>
  <pre class="codeline"> pv large.xml | xclip</pre>
<p>
<h3>Processes </h3><hr>
<p>
At the heart of a unix-type operating system is the concept of the 
'process'. Processes allow the computer to do more than one thing 
at a time.
<p>
Processes are essentially running programs (applications, software). 
Some applications, when they run, are visible because they use
a 'window' or they display information or data on the command-line.
However other processes are 'invisible'; that is, they are running, but
you as the user doesnt see any visible activity.
<p>
<strong><em>Close shell keeping all subprocess running</em></strong>
  <pre class="codeline"> disown -a && exit</pre>
<p>
<center><h3 class="page-title"></h3></center> process tools
.. ps - the basic tool
.. top - views processes in real time
.. lsof - displays open files
.. pstree - a hierarchical tree display
.. pmap -
.. pgrep - 
..
<p>
<h3>Viewing processes </h3><hr>....
<p>
It may be possible to view processes also with 'zenity' etc.
<p>
<strong><em>view processes (jobs) for only the current user/ shell</em></strong>
  <pre class="codeline"> ps</pre>
<p>
<strong><em>show information for all processes</em></strong>
  <pre class="codeline"> ps -e</pre>
<p>
<strong><em>display all your processes which have 'eave' in their name </em></strong>
  <pre class="codeline"> leave +30; ps | grep -v grep | grep -i eave</pre>
<p>
<strong><em>make a bash function 'pss' to display processes with 'ave' in the name </em></strong>
  <pre class="codeline"> pss(){ ps | grep -v grep | grep -i "$1"; }; pss ave</pre>
<p>
<strong><em>view all processes in real time</em></strong>
  <pre class="codeline"> top</pre>
<p>
<strong><em>show verbose information for all processes, not just for the current user</em></strong>
  <pre class="codeline"> ps -ef</pre>
  <pre class="codeline"> ps aux  ##(more or less the same)</pre>
<p>
<strong><em>Show a 4-way scrollable process tree with full details.</em></strong>
  <pre class="codeline"> ps awwfux | less -S</pre>
<p>
<strong><em>Alias for displaying a process tree nicely</em></strong>
  <pre class="codeline"> alias pst='pstree -Alpha'; pst</pre>
<p>
<strong><em>show all processes in a 'tree' format (parent and child processes linked)</em></strong>
  <pre class="codeline"> pstree</pre>
<p>
<strong><em>Displays process tree of all running processes</em></strong>
  <pre class="codeline"> pstree -Gap</pre>
<p>
<strong><em>Check if a process is running</em></strong>
  <pre class="codeline"> kill -0 <process-id> </pre>
<p>
<strong><em>order processes by cpu usage</em></strong>
  <pre class="codeline"> ps -e -o pcpu,cpu,nice,state,cputime,args --sort pcpu | sed "/^ 0.0 /d"</pre>
<p>
<strong><em>Show the 20 most CPU/Memory hungry processes</em></strong>
  <pre class="codeline"> ps aux | sort +2n | tail -20</pre>
<p>
<strong><em>show process numbers for the root user</em></strong>
  <pre class="codeline"> pgrep -u root</pre>
<p>
<strong><em>show running processes ordered by the amount of CPU usage</em></strong>
  <pre class="codeline"> ps -eo pcpu,pid,args | sort -n</pre>
<p>
<strong><em>show all processes using a directory/file/etc</em></strong>
  <pre class="codeline"> lsof | grep /Volumes/Lexar </pre>
<p>
<strong><em>sort all running processes by their memory & cpu usage</em></strong>
  <pre class="codeline"> ps aux --sort=%mem,%cpu</pre>
<p>
<h3>Managing processes </h3><hr>....
<p>
<strong><em>ionice limits process I/O, to keep it from swamping the system</em></strong>
  <pre class="codeline"> ionice -c3 find /</pre>
<p>
<strong><em>Restart command if it dies.</em></strong>
  <pre class="codeline"> ps -C program_name || { program_name & }</pre>
<p>
<h3>Analysing processes </h3><hr>....
<p>
<strong><em>Determining the excat memory usages by certain PID</em></strong>
  <pre class="codeline"> pmap -d [pid]</pre>
<p>
<strong><em>count processes with status "D" uninterruptible sleep</em></strong>
  <pre class="codeline"> top -b -n 1 | awk '{if (NR <=7) print; else if ($8 == "D") {print; count++} } END {print "Total status D: "count}'</pre>
<p>
<strong><em>Find the processes that are on the runqueue. Processes with a</em></strong>
  <pre class="codeline"> ps -eo stat,pid,user,command | egrep "^STAT|^D|^R"</pre>
<p>
<strong><em>Pulls total current memory usage, including SWAP being used, by all processes</em></strong>
  <pre class="codeline"> ps aux | awk '{sum+=$6} END {print sum/1024}'</pre>
<p>
<strong><em>find out current working directory of a process</em></strong>
  <pre class="codeline"> echo COMMAND | xargs -ixxx ps -C xxx -o pid= | xargs -ixxx ls -l /proc/xxx/cwd</pre>
<p>
<strong><em>Show top running processes by the number of open filehandles they</em></strong>
  <pre class="codeline"> lsof | awk '{print $1}' | sort | uniq -c | sort -rn | head</pre>
<p>
<strong><em>Return threads count of a process</em></strong>
  <pre class="codeline"> ps -o thcount -p <process id></pre>
<p>
<strong><em>List all process running on port 80</em></strong>
  <pre class="codeline"> sudo lsof -i :80</pre>
<p>
<strong><em>catch a process from a user and strace it.</em></strong>
  <pre class="codeline"> x=1; while [ $x = 1 ]; do process=`pgrep -u username`; if [ $process ]; then x=0; fi; done; strace -vvtf -s 256 -p $process</pre>
<p>
<h3>Killing processes </h3><hr>....
<p>
In order to stop a running program (which doesnt have a window) it
is necessary to 'kill' (or stop) the associated process. In order
to do this first it may be necessary to find out the process
identification number ('pid') of the running application.
<p>
<strong><em>start and then kill the 'leave' process using its exact name</em></strong>
  <pre class="codeline"> leave +20; killall -9 leave; ps </pre>
<p>
The command above will permanently stop ('kill') all processes which
are named exactly 'leave', that is, all processes which were started
by the 'leave' reminder program.
<p>
<strong><em>make a bash function 'psk' to kill processes with 'ave' in their name </em></strong>
  <pre class="codeline"> psk(){ ps | grep -v grep | grep -i "$1" | awk '{print $1;}' | xargs kill -9; }; psk ave</pre>
<p>
<strong><em>another way to kill a process using its name</em></strong>
  <pre class="codeline"> leave +20; pkill leave; ps</pre>
<p>
<strong><em>Kill a background job</em></strong>
  <pre class="codeline"> kill %1</pre>
<p>
<strong><em>Kills a process that is locking the file 'eg.txt'</em></strong>
  <pre class="codeline"> fuser -k eg.txt </pre>
<p>
<strong><em>find out the 'pid' number of a process associated with 'leave'</em></strong>
  <pre class="codeline"> leave +20; ps aux | grep leave</pre>
<p>
<strong><em>show the process id of a running program</em></strong>
  <pre class="codeline"> ls | at 1156; pidof at ; ps </pre>
<p>
you have to know the exact name of the program
<p>
<strong><em>Kill all processes that don't belong to root/force logoff</em></strong>
  <pre class="codeline"> for i in $(pgrep -v -u root);do kill -9 $i;done</pre>
<p>
<strong><em>Kill all processes belonging to a user</em></strong>
  <pre class="codeline"> ps -ef | grep $USERNAME | awk {'print $2'} | xargs kill [-9]</pre>
<p>
<strong><em>Kill all processes beloging to a single user.</em></strong>
  <pre class="codeline"> kill -9 `ps -u <username> -o "pid="`</pre>
<p>
<strong><em>Kill most recently created process with the name 'firefox'</em></strong>
  <pre class="codeline"> pkill -n firefox</pre>
<p>
<strong><em>kill all processes using a directory/file/etc</em></strong>
  <pre class="codeline"> lsof | grep /Volumes/Lexar | awk '{print $2}' | xargs kill</pre>
<p>
<strong><em>stop a program or process with 'sshd' in its name</em></strong>
  <pre class="codeline"> kill $(ps -ef | awk '/sshd/ { print $2 }') </pre>
  <pre class="codeline"> kill $(ps -ef | grep sshd | awk '{ print $2 }')   ##(the same)</pre>
<p>
<strong><em>trace the system calls made by a process (and its children)</em></strong>
  <pre class="codeline"> strace -f -s 512 -v ls -l</pre>
<p>
<h3>Zombie processes </h3><hr>....
<p>
<strong><em>display all 'zombie' processese</em></strong>
  <pre class="codeline"> ps aux | awk '{ print $8 " " $2 " " $11}' | grep -w Z</pre>
<p>
<strong><em>Get a regular updated list of zombies</em></strong>
  <pre class="codeline"> watch "ps auxw | grep [d]efunct"</pre>
<p>
<strong><em>Get a regular updated list of zombies</em></strong>
  <pre class="codeline"> watch "ps auxw | grep 'defunct' | grep -v 'grep' | grep -v 'watch'"</pre>
<p>
<h3>Environment configurations</h3><hr>
<p>
<strong><em>show the values of the environment variables</em></strong>
  <pre class="codeline"> env </pre>
  <pre class="codeline"> printenv</pre>
<p>
<strong><em>Executes a command changing an environment variable 'var'</em></strong>
  <pre class="codeline"> var="value" command </pre>
<p>
For example ...
<p>
<strong><em>show what time and date it is in new york</em></strong>
  <pre class="codeline"> TZ=America/New_York date</pre>
<p>
<h3>Aliases </h3><hr>....
<p>
<strong><em>put aliases in the ~/.bashrc file to save them</em></strong>
  <pre class="codeline"> alias dir='ls -la | less'</pre>
<p>
<strong><em>Make alias pemanent fast</em></strong>
  <pre class="codeline"> PERMA () { echo "$@" >> ~/.bashrc; }</pre>
<p>
<strong><em>a variable can be used in an alias</em></strong>
  <pre class="codeline"> alias say='echo $1'   ##(but alias say='echo $1; echo 1' doesnt work...)</pre>
<p>
<strong><em>reload the .bashrc file to make a new alias take effect</em></strong>
  <pre class="codeline"> source ~/.bashrc</pre>
<p>
<strong><em>add a folder to the executable path (put in ~/.bash_profile file to save it)</em></strong>
  <pre class="codeline"> export PATH=/path/to/folder:"${PATH}"</pre>
##(programs in this folder can then be executed with 'programname')
<p>
<strong><em>make all terminals write to the same history file</em></strong>
  <pre class="codeline"> shopt -s histappend       ##(put in .bashrc, on single user systems)</pre>
<p>
<strong><em>enable changing to a folder by typing only the folder name (not path)</em></strong>
  <pre class="codeline"> export CDPATH='.:~:/some/folder:/path/to/folder'</pre>
##(place this in .bashrc with commonly used folders)
<p>
<h3>Timing performance</h3><hr>
<p>
<strong><em>time how long a "grep" command takes to execute</em></strong>
  <pre class="codeline"> time grep -rl big *</pre>
<p>
<strong><em>time the execution time of 2 commands at the same time</em></strong>
  <pre class="codeline"> time { find / -name '*what*'; locate '*.cc' ; }</pre>
<p>
<h3>Scheduling</h3><hr>
<p>
<center><h3 class="page-title"></h3></center> tools
.. batch - 
.. cron - a scheduling tools
.. crontab - a file of scheduled tasks
.. at - run a command at a certain time
.. nice - run with a certain priority
..
<p>
<h3>At </h3><hr>....
<p>
<strong><em>retrieve the page 'elpais.com' at 12:00 oclock</em></strong>
  <pre class="codeline"> echo 'wget http://elpais.com' | at 12:00</pre>
<p>
<strong><em>Schedule a script or command to run in 2 hours</em></strong>
  <pre class="codeline"> ( ( sleep 2h; your-command your-args ) & )</pre>
<p>
<strong><em>Execute a command at a given time</em></strong>
  <pre class="codeline"> echo "ls -l" | at midnight</pre>
<p>
<strong><em>Run a command only when load average is below a certain threshold</em></strong>
  <pre class="codeline"> echo "rm -rf /unwanted-but-large/folder" | batch</pre>
<p>
<h3>Cron </h3><hr>....
<p>
Cron is the traditional unix tool for scheduling tasks; that is for
instructing the computer to automatically carry out a task at a given time,
or at given intervals, without the user explicitly starting the task.
Each task is placed in the 'crontab' file.
<p>
<strong><em>view the help page for the crontab file</em></strong>
  <pre class="codeline"> man 5 crontab</pre>
<p>
<strong><em>run the script 'upload.sh' every 15 mins past the hour</em></strong>
  <pre class="codeline"> */15 * * * * /usr/local/bin/upload.sh  ##(put this in the cron file)</pre>
<p>
<strong><em>add the new entry in the file 'crontab' to the cron schedule,</em></strong>
  <pre class="codeline"> crontab</pre>
<p>
<strong><em>update your system every day at lunch time (12:00)</em></strong>
  <pre class="codeline"> (crontab -e) 00 12 * * * apt-get update (/etc/init.d/cron restart)</pre>
<p>
<strong><em>edit the crontab file</em></strong>
  <pre class="codeline"> crontab -e</pre>
<p>
<strong><em>edit crontab</em></strong>
  <pre class="codeline"> vi ~/.crontab && crontab ~/.crontab</pre>
<p>
<strong><em>Log output from a cronjob to a file, but also e-mail if a string</em></strong>
  <pre class="codeline"> some_cronjobed_script.sh 2>&1 | tee -a output.log | grep -C 1000 ERROR</pre>
<p>
<strong><em>print crontab entries for all the users that actually have a file</em></strong>
  <pre class="codeline"> for USER in `cut -d ":" -f1 </etc/passwd`; do crontab -u ${USER} -l 1>/dev/null 2>&1; if [ ! ${?} -ne 0 ]; then echo -en "--- crontab for ${USER} ---\n$(crontab -u ${USER} -l)\n"; fi; done</pre>
<p>
<h3>Notifications </h3><hr>....
<p>
<strong><em>Set an alarm to wake up [2]</em></strong>
  <pre class="codeline"> echo "aplay path/to/song" | at [time]</pre>
<p>
<strong><em>Set an alarm to wake up</em></strong>
  <pre class="codeline"> sleep 5h && rhythmbox path/to/song</pre>
<p>
<strong><em>beep when a server goes offline</em></strong>
  <pre class="codeline"> while true; do [ "$(ping -c1W1w1 server-or-ip.com | awk '/received/ {print $4}')" != 1 ] && beep; sleep 1; done</pre>
<p>
<strong><em>Remind yourself to leave in 15 minutes</em></strong>
  <pre class="codeline"> leave +15</pre>
<p>
<strong><em>An alarm clock using xmms2 and at</em></strong>
  <pre class="codeline"> echo "xmms2 play" | at 6:00</pre>
<p>
<strong><em>Send pop-up notifications on Gnome</em></strong>
  <pre class="codeline"> notify-send ["<title>"] "<body>"</pre>
<p>
<strong><em>Will email user@example.com when all Rsync processes have</em></strong>
  <pre class="codeline"> $(while [ ! -z "$(pgrep rsync)" ]; do echo; done; echo "rsync done" | mailx user@example.com) > /dev/null &</pre>
<p>
<strong><em>Run a long job and notify me when it's finished</em></strong>
  <pre class="codeline"> ./my-really-long-job.sh && notify-send "Job finished"</pre>
<p>
<strong><em>Set audible alarm when an IP address comes online</em></strong>
  <pre class="codeline"> ping -i 60 -a IP_address</pre>
<p>
<strong><em>Notify me when users log in</em></strong>
  <pre class="codeline"> notifyme -C `cat /etc/passwd | cut -d: -f1`</pre>
<p>
<strong><em>display a (gtk) window with the text 'command finished'</em></strong>
  <pre class="codeline"> zenity --info --text="command finished!" </pre>
<p>
<h3>Alarms </h3><hr>.... 
<p>
<strong><em>A snooze button for xmms2 alarm clock</em></strong>
  <pre class="codeline"> xmms2 pause && echo "xmms2 play" | at now +5min</pre>
<p>
<strong><em>An alarm clock using xmms2 and at</em></strong>
  <pre class="codeline"> at 6:00 <<< "xmms2 play"</pre>
<p>
<h3>Databases</h3><hr>
<p>
<center><h3 class="page-title"></h3></center> popular database software
.. postgresql -
.. mysql - 
.. berkeley db -
..
<p>
see the book:<br/>
 <a href="http://bumble.sf.<a" <a href="http://bumble.sf.<a</a>">http://bumble.sf.<a</a></a> href="net/books/linux-database/linux-database-book.txt'"><em><code>net/books/linux-database/linux-database-book.txt'</code></em></a> 
<p>
<h3>Configuring linux</h3><hr>
<p>
<center><h3 class="page-title"></h3></center> tools
.. update-alternatives - maintains default programs for tasks
.. getent - gets data from an adminstrative database
..
<p>
<strong><em>Get contents from hosts, passwd, groups </em></strong>
  <pre class="codeline"> getent [group|hosts|networks|passwd|protocols|services] [keyword]</pre>
<p>
<strong><em>set the default pager used for man pages</em></strong>
  <pre class="codeline"> update-alternatives --set pager /usr/bin/most</pre>
<p>
Environment variables etc
<p>
<strong><em>show the current path</em></strong>
  <pre class="codeline"> echo $PATH</pre>
<p>
<h3>The operating system</h3><hr>
<p>
<strong><em>Find distro name and/or version/release</em></strong>
  <pre class="codeline"> cat /etc/*-release</pre>
<p>
<strong><em>When was your OS installed?</em></strong>
  <pre class="codeline"> ls -lct /etc/ | tail -1 | awk '{print $6, $7, $8}'</pre>
<p>
<h3>Kernel </h3><hr>....
<p>
<strong><em>send kernel log (dmesg) notifications to root via cron</em></strong>
  <pre class="codeline"> (crontab -l; echo '* * * * * dmesg -c'; ) | crontab -</pre>
<p>
<strong><em>Short Information about loaded kernel modules</em></strong>
  <pre class="codeline"> awk '{print $1}' "/proc/modules" | xargs modinfo | awk '/^(filename|desc|depends)/'</pre>
<p>
<strong><em>Short Information about loaded kernel modules</em></strong>
  <pre class="codeline"> modinfo $(cut -d' ' -f1 /proc/modules) | sed '/^dep/s/$/\n/; /^file\|^desc\|^dep/!d'</pre>
<p>
<strong><em>find your release version of your ubuntu / debian distro</em></strong>
  <pre class="codeline"> lsb_release -a</pre>
<p>
<strong><em>When was your OS installed?</em></strong>
  <pre class="codeline"> ls -ldct /lost+found |awk '{print $6, $7}'</pre>
<p>
<h3>Modules </h3><hr>....
<p>
Modules in some cases serve the purpose of device drivers. In some
cases they need to be compiled from source code, and then installed
in the kernel with 'modprobe'.
<p>
<strong><em>Disable beep sound from your computer</em></strong>
  <pre class="codeline"> echo "blacklist pcspkr"|sudo tee -a /etc/modprobe.d/blacklist.conf</pre>
<p>
<h3>Swap files </h3><hr>....
<p>
<strong><em>create an emergency swapfile when the existing swap space is too small</em></strong>
  <pre class="codeline"> sudo dd if=/dev/zero of=/swapfile bs=1024 count=1024000;sudo mkswap /swapfile; sudo swapon /swapfile</pre>
<p>
<h3>Hardware configuration</h3><hr>
<p>
<strong><em>set your ssd disk as a non-rotating medium</em></strong>
  <pre class="codeline"> sudo echo 0 > /sys/block/sdb/queue/rotational</pre>
<p>
<strong><em>32 bits or 64 bits?</em></strong>
  <pre class="codeline"> getconf LONG_BIT</pre>
<p>
<strong><em>Getting information about model no. of computer</em></strong>
  <pre class="codeline"> dmidecode | grep -i prod</pre>
<p>
<strong><em>print indepth hardware info</em></strong>
  <pre class="codeline"> sudo dmidecode | more</pre>
<p>
<strong><em>Generate the CPU utilization report</em></strong>
  <pre class="codeline"> sar -u 2 5</pre>
<p>
<strong><em>hard disk information - Model/serial no.</em></strong>
  <pre class="codeline"> hdparm -i[I] /dev/sda</pre>
<p>
<strong><em>Create an html page of information about your harddisk</em></strong>
  <pre class="codeline"> lshw -C disk -html > /tmp/diskinfo.html</pre>
<p>
<strong><em>Create a nifty html overview of the hardware in your computer</em></strong>
  <pre class="codeline"> lshw -html > hardware.html  ##('hardware.html' can be viewed in a browser)</pre>
<p>
<strong><em>show the linux kernel version</em></strong>
  <pre class="codeline"> uname -r</pre>
<p>
<strong><em>show kernel startup messages</em></strong>
  <pre class="codeline"> dmesg</pre>
<p>
<strong><em>show usb devices</em></strong>
  <pre class="codeline"> lsusb</pre>
<p>
<h3>Modules and device drivers</h3><hr>
<p>
In Linux, 'device drivers' (which make bits of hardware work) are also
known as 'kernel modules'. If a piece of hardware is not working, then
usually a kernel module must be loaded. This may involve; Find the
product code for the device (looks like 0341:4561). Then find the module
for that product. Then download the module source, compiling the module
and installing it.
<p>
<strong><em>list your device drivers</em></strong>
  <pre class="codeline"> lspci -vv | less</pre>
<p>
<strong><em>show all loaded kernel modules</em></strong>
  <pre class="codeline"> lsmod</pre>
<p>
<strong><em>find module files in or below the current folder </em></strong>
  <pre class="codeline"> find . -name *.ko    ##(kernel module files end in 'ko')</pre>
<p>
<strong><em>build the dependencies between various kernel modules </em></strong>
  <pre class="codeline"> depmod -a</pre>
<p>
<strong><em>load the 'rt3090sta' module (a wireless card driver)</em></strong>
  <pre class="codeline"> sudo modprobe rt3090sta</pre>
<p>
<strong><em>show pci devices</em></strong>
  <pre class="codeline"> lspci</pre>
<p>
<strong><em>show ram memory information</em></strong>
  <pre class="codeline"> cat /proc/meminfo</pre>
<p>
<strong><em>show cpu information</em></strong>
  <pre class="codeline"> cat /proc/cpuinfo</pre>
<p>
<strong><em>Short information about loaded kernel modules</em></strong>
  <pre class="codeline"> lsmod | cut -d' ' -f1 | xargs modinfo | egrep '^file|^desc|^dep' | sed -e'/^dep/s/$/\n/g'</pre>
<p>
<strong><em>Short information about loaded kernel modules</em></strong>
  <pre class="codeline"> lsmod | sed -e '1d' -e 's/\(\([^ ]*\) \)\{1\}.*/\2/' | xargs modinfo | sed -e '/^dep/s/$/\n/g' -e '/^file/b' -e '/^desc/b' -e '/^dep/b' -e d</pre>
<p>
<strong><em>Lists installed kernels</em></strong>
  <pre class="codeline"> dpkg --get-selections | grep linux-image</pre>
<p>
<h3>Keyboard </h3><hr>....
<p>
<strong><em>Replace Caps-lock with Control-key</em></strong>
  <pre class="codeline"> xmodmap -e 'remove Lock = Caps_Lock' && xmodmap -e 'add control = Caps_Lock'</pre>
<p>
<strong><em>Change the console keyboard layout</em></strong>
  <pre class="codeline"> loadkeys uk</pre>
<p>
<strong><em>show all key and mouse events</em></strong>
  <pre class="codeline"> xev</pre>
<p>
<h3>Monitor </h3><hr>....
<p>
<strong><em>Give information about your graphic chipset</em></strong>
  <pre class="codeline"> lshw -C display</pre>
<p>
<strong><em>Show display adapter, available drivers, and driver in use</em></strong>
  <pre class="codeline"> lspci -v | perl -ne '/VGA/../^$/ and /VGA|Kern/ and print'</pre>
<p>
<h3>Devices </h3><hr>
<p>
<h3>Ethernet card </h3><hr>....
<p>
<strong><em>Get ethernet card information.</em></strong>
  <pre class="codeline"> ethtool eth0</pre>
<p>
<h3>Monitors </h3><hr>....
<p>
<strong><em>Configure second monitor to sit to the right of laptop</em></strong>
  <pre class="codeline"> xrandr --output LVDS --auto --output VGA --auto --right-of LVDS</pre>
<p>
<h3>Cdroms </h3><hr>....
<p>
<strong><em>Add audio CD to xmms2 playlist</em></strong>
  <pre class="codeline"> xmms2 addpls cdda://</pre>
<p>
<strong><em>Decreasing the cdrom device speed</em></strong>
  <pre class="codeline"> eject -x 4</pre>
<p>
<strong><em>limit the cdrom driver to a specified speed</em></strong>
  <pre class="codeline"> eject -x 8 /dev/cdrom</pre>
<p>
<strong><em>save a compressed copy of data from a cdrom</em></strong>
  <pre class="codeline"> gzip < /dev/cdrom > cdrom.iso.gz     </pre>
<p>
<strong><em>rip audio tracks from CD to wav files in current dir</em></strong>
  <pre class="codeline"> cdparanoia -B </pre>
<p>
<strong><em>clear a rewritable compact disk (CDRW)</em></strong>
  <pre class="codeline"> cdrecord -v dev=/dev/cdrom blank=fast   </pre>
<p>
<strong><em>Make audio CD from all wavs in current dir (see also cdrdao)</em></strong>
  <pre class="codeline"> cdrecord -v dev=/dev/cdrom -audio *.wav</pre>
<p>
<strong><em>how to copy cd/dvd onto the hard disk (.iso)</em></strong>
  <pre class="codeline"> dd if=/dev/cdrom of=whatever.iso</pre>
<p>
<h3>Burning cds and dvds </h3><hr>....
<p>
<strong><em>erase content from a cdrw</em></strong>
  <pre class="codeline"> cdrecord -v -blank=all -force</pre>
<p>
<strong><em>Create an ISO Image from a folder and burn it to CD</em></strong>
  <pre class="codeline"> hdiutil makehybrid -o CDname.iso /Way/to/folder ; hdiutil burn CDname.iso</pre>
<p>
<strong><em>burn an ISO image to writable CD</em></strong>
  <pre class="codeline"> wodim cdimage.iso</pre>
<p>
<strong><em>burn an iso to cd or dvd</em></strong>
  <pre class="codeline"> cdrecord -v path_to_iso_image.iso</pre>
<p>
<strong><em>add files to existing growable DVD using growisofs</em></strong>
  <pre class="codeline"> growisofs -M /dev/dvd -J -r "directory name with files to add to DVD"</pre>
<p>
<strong><em>Blank/erase a DVD-RW</em></strong>
  <pre class="codeline"> dvd+rw-format -force /dev/dvd1</pre>
<p>
<strong><em>Burn a directory of mp3s to an audio cd.</em></strong>
  <pre class="codeline"> alias burnaudiocd='mkdir ./temp && for i in *.[Mm][Pp]3;do mpg123 -w "./temp/${i%%.*}.wav" "$i";done;cdrecord -pad ./temp/* && rm -r ./temp'</pre>
<p>
<strong><em>backup your playstation game using rip</em></strong>
  <pre class="codeline"> cdrdao read-cd --read-raw --datafile FILE_NAME.bin --device /dev/cdrom --driver generic-mmc-raw FILE_NAME.toc</pre>
<p>
<h3>Scsi </h3><hr>....
<p>
<strong><em>Shows physically connected drives (SCSI or SATA)</em></strong>
  <pre class="codeline"> ls /sys/bus/scsi/devices</pre>
<p>
<strong><em>scan for new scsi devices</em></strong>
  <pre class="codeline"> echo "- - -" > /sys/class/scsi_host/host0/scan</pre>
<p>
<h3>Usb </h3><hr>....
<p>
<strong><em>remount a usb disk in gnome without physically removing and reinserting</em></strong>
  <pre class="codeline"> eject /dev/sdb; sleep 1; eject -t /dev/sdb</pre>
<p>
<strong><em>Get names of files in /dev, a USB device is attached to</em></strong>
  <pre class="codeline"> ls -la /dev/disk/by-id/usb-*</pre>
<p>
To relabel a usb thumb drive, first find out the type 
of file system on the drive. 
<p>
<center><h3 class="page-title"></h3></center> packages for relabeling usb thumb drives
.. FAT16 and FAT32 partitions, use mtools.
.. NTFS partitions, use ntfsprogs.
.. ext2, ext3, or ext4 partitions, use e2fsprogs.
.. JFS partitions, use jfsutils.
.. ReiserFS (v3) partitions, use reiserfsprogs.
.. XFS partitions, use xfsprogs
..
<p>
<strong><em>unmount the usb device before re-labeling it</em></strong>
  <pre class="codeline"> sudo umount /dev/sdb1</pre>
<p>
<strong><em>check the current device label</em></strong>
  <pre class="codeline"> sudo mlabel -i /dev/sdb1 -s ::</pre>
<p>
<strong><em>edit the<a href="mtools.conf"><em><code>mtools.conf</code></em></a></em></strong>
  <pre class="codeline"> vim /etc/mtools.conf</pre>
<p>
<strong><em>relabel the drive</em></strong>
  <pre class="codeline"> mlabel p:newlabel</pre>
<p>
<h3>Cpu central processing unit </h3><hr>....
<p>
<strong><em>List the CPU model name</em></strong>
  <pre class="codeline"> grep "model name" /proc/cpuinfo</pre>
<p>
<strong><em>List the CPU model name</em></strong>
  <pre class="codeline"> sed -n 's/^model name[ \t]*: *//p' /proc/cpuinfo</pre>
<p>
<h3>Xwindows </h3><hr>
<p>
<center><h3 class="page-title"></h3></center> tools
.. wmctrl - control a window manager from scripts
..
<p>
<strong><em>Start an X app remotely</em></strong>
  <pre class="codeline"> ssh -f user@remote.ip DISPLAY=:0.0 smplayer movie.avi</pre>
<p>
<strong><em>Run any GUI program remotely</em></strong>
  <pre class="codeline"> ssh -fX <user>@<host> <program></pre>
<p>
<strong><em>Send keypresses to an X application</em></strong>
  <pre class="codeline"> xvkbd -xsendevent -text "Hello world"</pre>
<p>
<strong><em>Click on a GUI window and show its process ID and command used to</em></strong>
  <pre class="codeline"> xprop | awk '/PID/ {print $3}' | xargs ps h -o pid,cmd</pre>
<p>
<strong><em>Start another X session in a window</em></strong>
  <pre class="codeline"> startx -- /usr/bin/Xephyr :2</pre>
<p>
<h3>The clipboard </h3><hr>....
<p>
<strong><em>get xclip to own the clipboard contents</em></strong>
  <pre class="codeline"> xclip -o -selection clipboard | xclip -selection clipboard</pre>
<p>
<strong><em>copy the <a href="list.xml"><em><code>list.xml</code></em></a> to the clipboard</em></strong>
  <pre class="codeline"> cat list.xml | xclip</pre>
<p>
<strong><em>copoy a large file to the clipboard with a progress meter</em></strong>
  <pre class="codeline"> pv large.xml | xclip</pre>
<p>
<strong><em>print to standard output the text or data in the x clipboard</em></strong>
  <pre class="codeline"> xsel -o </pre>
<p>
If a piece of text is currently 'selected' or 'highlighted' in
some window on the computer, then this text will be printed
with 'xsel -o'
<p>
<h3>Kde</h3><hr>
<p>
Kde is an alternative to 'gnome' and uses the qt windowing
toolkit. qt is not a completely free toolkit.
<p>
<strong><em>Unlock your KDE4.3 session remotely</em></strong>
  <pre class="codeline"> qdbus org.kde.screenlocker /MainApplication quit</pre>
<p>
<h3>Gnome</h3><hr>
<p>
Gnome uses the gtk windowing toolkit to draw its graphical
user interfaces. gtk is a completely opensource toolkit.
<p>
<h3>User interfaces</h3><hr>
<p>
<strong><em>Read a keypress without echoing it</em></strong>
  <pre class="codeline"> stty cbreak -echo; KEY=$(dd bs=1 count=1 2>/dev/null); stty -cbreak echo</pre>
<p>
<strong><em>On screen display of command results</em></strong>
  <pre class="codeline"> date | osd_cat</pre>
<p>
<h3>Select menus </h3><hr>....
<p>
The linux 'select' command is the simplest way of
making a menu for the user to select from.
<p>
<h3>Whiptail and dialog </h3><hr>....
<p>
Both whiptail and dialog create 'windows' in a console
terminal.
<p>
<h3>Kdialog </h3><hr>....
<p>
Kdialog seems to be more or less equivalent to zenity
for the KDE desktop. 
<p>
<strong><em>Show a passive popup in KDE which times out in 30 seconds</em></strong>
  <pre class="codeline"> kdialog --passivepopup <text> 30</pre>
<p>
<strong><em>an SSH monitor using kdialog</em></strong>
  <pre class="codeline"> ssh root@server 'tail --max-unchanged-stats=10 -n0 -F /var/log/auth.log ' | grep Accepted | while read l ; do kdialog --title "SSH monitor" --passivepopup "$l" 3; done</pre>
<p>
<h3>Zenity</h3><hr>
<p>
zenity is a simple way to present the user with a graphical user interface
without needing to use a programming language. Zenity uses the 
gtk windowing library, so is aimed to the gnome linux desktop
<p>
@@ <a href="http://bumble.sf.<a" <a href="http://bumble.sf.<a</a>">http://bumble.sf.<a</a></a> href="net/books/bash/bash-book.txt"><em><code>net/books/bash/bash-book.txt</code></em></a>
Some other zenity examples.
<p>
<strong><em>display a window with a calender and output the chosen date</em></strong>
  <pre class="codeline"> zenity --calendar</pre>
<p>
<strong><em>allow the user to (graphically) choose a file</em></strong>
  <pre class="codeline"> zenity --file-selection --title 'select a file'</pre>
<p>
<strong><em>make a window with a list-box with the files from current folder</em></strong>
  <pre class="codeline"> ls | zenity --list --column="test" </pre>
<p>
<strong><em>try to jump to a folder with zenity </em></strong>
  <pre class="codeline">  find -type d -name '*er*' | zenity --list --column= | xargs cd</pre>
<p>
The file which the user selects in the list box is printed to
the standard output.
<p>
<strong><em>display a text box and a label in a window</em></strong>
  <pre class="codeline"> zenity --title "Select Host" --entry --text "Select a server"</pre>
<p>
<strong><em>display a yes/no box with the question 'really quit now?'</em></strong>
  <pre class="codeline"> zenity --question --text "really quit now?"</pre>
<p>
<strong><em>show a question dialog and exit the script if the user says yes</em></strong>
  <pre class="codeline"> [ $(zenity --question --text "really quit?") ] && echo quiting && exit</pre>
<p>
<strong><em>show a notification icon in the gnome task bar</em></strong>
  <pre class="codeline"> zenity --notification --window-icon=update.png --text "System update necessary!"</pre>
<p>
<h3>Progress bars with zenity </h3><hr>....
<p>
<strong><em>show a progress bar while a command is executing</em></strong>
   <pre class="codeline"> find $HOME -name '*.mp3' | zenity --progress --pulsate</pre>
<p>
<h3>Checkboxes with zenity </h3><hr>....
<p>
<strong><em>display a list with check boxes with the column labels </em></strong>
  <pre class="codeline"> zenity --list --checklist --column "Buy" --column "Item" TRUE Apples TRUE Oranges FALSE Pears FALSE Toothpaste </pre>
<p>
##(this writes 'Apples|Oranges' to standard out)
<p>
<h3>Windows and gui applications </h3><hr>....
<p>
@@ gtk-apps.org
graphical applications which run with the gnome desktop.
<p>
<strong><em>open a file with its appropriate window application</em></strong>
  <pre class="codeline"> xdg-open file.txt </pre>
<p>
<strong><em>show the x window for a graphical app running on a remote computer</em></strong>
  <pre class="codeline"> ssh -X user@server.ext</pre>
<p>
<h3>Starting x </h3><hr>....
<p>
<strong><em>start X from a virtual console</em></strong>
  <pre class="codeline"> startx </pre>
<p>
<strong><em>run startx and redirect its output to a log file</em></strong>
  <pre class="codeline"> startx >$HOME/startx.log 2>&1 </pre>
<p>
<strong><em>start X from a virtual console, and specify 16-bit color depth,</em></strong>
  <pre class="codeline"> startx -- -bpp 16 </pre>
<p>
<strong><em>end your X session if you are running the fvwm2 window manager,</em></strong>
click the left mouse button in the root window to pull up the start
menu, and then choose Really quit? from the Exit Fvwm submenu.
<p>
<strong><em>end your X session if you are running the afterstep window</em></strong>
manager, click the left mouse button in the root window to pull up the
start menu, and then choose Really quit? from the Exit Fvwm submenu.
<p>
<strong><em>exit X immediately</em></strong>
  <pre class="codeline"> [CTRL]-[ALT]-[BKSP]</pre>
<p>
<strong><em>run a digital clock from a shell window</em></strong>
  <pre class="codeline"> xclock -digital & </pre>
<p>
<strong><em>start a small xclock, 48 pixels wide and 48 pixels high</em></strong>
  <pre class="codeline"> xclock -geometry 48x48 </pre>
<p>
<strong><em>start a large xclock, 480 pixels wide and 500 pixels high</em></strong>
  <pre class="codeline"> xclock -geometry 480x500 </pre>
<p>
<strong><em>start an xclock with a width of 48 pixels and the default height</em></strong>
  <pre class="codeline"> xclock -geometry 48 </pre>
<p>
<strong><em>start an xclock with a height of 48 pixels and the default width</em></strong>
  <pre class="codeline"> xclock -geometry x48 </pre>
<p>
<strong><em>list the available colors</em></strong>
  <pre class="codeline"> xcolors  ##(Press [Q] to exit xcolors.)</pre>
<p>
<strong><em>switch to the desktop to the left of the current one while running fvw2,</em></strong>
  <pre class="codeline">  type [ALT]-[{<-]}.</pre>
<p>
<strong><em>switch to the desktop directly to the left of the current one while running afterstep,</em></strong>
  <pre class="codeline">  type [CTRL]-[{<-]}.</pre>
<p>
<strong><em>switch to the next-lowest video mode</em></strong>
  <pre class="codeline"> [CTRL]-[ALT]-[+]</pre>
<p>
<strong><em>switch to the next-highest video mode</em></strong>
  <pre class="codeline"> [CTRL]-[ALT]-[-]</pre>
<p>
<strong><em>change the root window color to blue violet</em></strong>
  <pre class="codeline"> xsetroot -solid blueviolet </pre>
<p>
<strong><em>tile the root window with a star pattern</em></strong>
  <pre class="codeline"> xsetroot -bitmap /usr/X11R6/include/bitmaps/star </pre>
<p>
<strong><em>tile the root window with a light slate gray star pattern on a black background</em></strong>
  <pre class="codeline"> xsetroot -fg slategray2 -bg black -bitmap /usr/X11R6/include/bitmaps/star </pre>
<p>
<strong><em>make the root window a gray color with no pattern</em></strong>
  <pre class="codeline"> xsetroot -gray </pre>
<strong><em>browse the system documentation files in the '/usr/doc' directory</em></strong>
  <pre class="codeline"> lynx /usr/doc </pre>
<p>
<strong><em>browse the system documentation files in the '/usr/doc' directory in Mozilla, type the following in Mozilla's Location window:  <a href="usr/doc</em></strong>">usr/doc</em></strong></a>
<p>
<strong><em>find all files on the system that have 'audio' anywhere in their name</em></strong>
  <pre class="codeline"> locate audio </pre>
<p>
<strong><em>find all the files on the system whose file names end with the text 'ogg'</em></strong>
  <pre class="codeline"> locate *ogg </pre>
<p>
<strong><em>find all hidden "dotfiles" on the system</em></strong>
  <pre class="codeline"> locate /. </pre>
<p>
<h3>Note</h3><hr>: locate searches are not case sensitive.
<p>
<strong><em>list all files on the system whose file name is 'top', regardless of case</em></strong>
  <pre class="codeline"> find / -iname top </pre>
<p>
<strong><em>list all files whose names begin with the three characters 'top' followed by exactly three more characters</em></strong>
  <pre class="codeline"> find / -name 'top???' </pre>
<p>
<strong><em>list all files in the current directory tree whose names have either the string 'net' or `comm' anywhere in their file names, type:</em></strong>
  <pre class="codeline"> find . -regex '.*\(net\|comm\).*' </pre>
<p>
<strong><em>list all files in the '/usr/local' directory tree that are greater than 10,000 kilobytes in size</em></strong>
  <pre class="codeline"> find /usr/local -size +10000k </pre>
<p>
<strong><em>list all files in your home directory tree less than 300 bytes in size</em></strong>
  <pre class="codeline"> find ~ -size -300b </pre>
<p>
<strong><em>list all files on the system whose size is exactly 42 512-byte blocks</em></strong>
  <pre class="codeline"> find / -size 42 </pre>
<p>
<strong><em>find all empty files in your home directory tree</em></strong>
  <pre class="codeline"> find ~ -empty </pre>
<p>
<strong><em>list the files in the '/usr/local' directory tree that were modified exactly 24 hours ago</em></strong>
  <pre class="codeline"> find /usr/local -mtime 1 </pre>
<p>
<strong><em>list the files in the '/usr' directory tree that were modified exactly five minutes ago</em></strong>
  <pre class="codeline"> find /usr -mmin 5 </pre>
<p>
<strong><em>list the files in the '/usr/local' directory tree that were modified within the past 24 hours</em></strong>
  <pre class="codeline"> find /usr/local -mtime -1 </pre>
<p>
<strong><em>list the files in the '/usr' directory tree that were modified within the past five minutes</em></strong>
  <pre class="codeline"> find /usr -mmin -5 </pre>
<p>
<strong><em>list all of the files in your home directory tree that were modified yesterday</em></strong>
  <pre class="codeline"> find ~ -mtime 1 -daystart </pre>
<p>
<strong><em>list all of the files in the '/usr' directory tree that were modified one year or longer ago</em></strong>
  <pre class="codeline"> find /usr -mtime +356 -daystart </pre>
<p>
<strong><em>list all of the files in your home directory tree that were modified from two to four days ago</em></strong>
  <pre class="codeline"> find ~ -mtime 2 -mtime -4 -daystart </pre>
<p>
<strong><em>find files in the '/etc' directory tree that are newer than the file '/etc/motd'</em></strong>
  <pre class="codeline"> find /etc -newer /etc/motd </pre>
<p>
<strong><em>list all files in your home directory tree that were modified after May 4 of the current year</em></strong>
  <pre class="codeline"> touch -t 05040000 /tmp/timestamp </pre>
  <pre class="codeline"> find ~ -newer /tmp/timestamp </pre>
<p>
<strong><em>list all files in the '/usr/local/fonts' directory tree owned by the user warwick</em></strong>
  <pre class="codeline"> find /usr/local/fonts -user warwick </pre>
<p>
<strong><em>list all files in the '/dev' directory tree owned by the audio group</em></strong>
  <pre class="codeline"> find /dev -group audio </pre>
<p>
<strong><em>find all files in the '~/html/' directory tree with an `.html' extension, and output lines from these files that contain the string 'organic'</em></strong>
  <pre class="codeline"> find ~/html/ -name '*.html' -exec grep organic '{}' ';' </pre>
<p>
<strong><em>remove files from your home directory tree that were accessed more than one year after they were last modified, pausing to confirm before each removal</em></strong>
  <pre class="codeline"> find ~ -used +365 -ok rm '{}' ';' </pre>
<p>
<strong><em>list files in your home directory tree whose names begin with the string 'top', and that are newer than the file `/etc/motd', type:</em></strong>
  <pre class="codeline"> find ~ -name 'top*' -newer /etc/motd </pre>
<p>
<strong><em>compress all the files in your home directory tree that are two megabytes or larger, and that are not already compressed with gzip (having a '.gz' file name extension)</em></strong>
  <pre class="codeline"> find ~ -size +2000000c -regex '.*[^gz]' -exec gzip '{}' ';' </pre>
<p>
<strong><em>list the files in the current directory, with their attributes, sorted with the largest files first</em></strong>
  <pre class="codeline"> ls -lS </pre>
<p>
<strong><em>list the files in the current directory and their attributes, sorted from smallest to largest</em></strong>
  <pre class="codeline"> ls -lSr </pre>
<p>
<strong><em>output a list of the subdirectories of the current directory tree, sorted in ascending order by size</em></strong>
  <pre class="codeline"> du -S . | sort -n </pre>
<p>
<strong><em>output a list of the subdirectories in the current directory tree, sorted in descending order by size</em></strong>
  <pre class="codeline"> du -S . | sort -nr </pre>
<p>
<strong><em>output a list of the subdirectories in the '/usr/local' directory tree, sorted in descending order by size</em></strong>
  <pre class="codeline"> du -S /usr/local | sort -nr </pre>
<p>
<strong><em>output the number of files in the current directory</em></strong>
  <pre class="codeline"> ls | wc -l </pre>
19
<p>
<strong><em>count the number of files -- including dot files -- in the</em></strong>
current directory
  <pre class="codeline"> ls -A | wc -l </pre>
81
<p>
<strong><em>list the number of files in the '/usr/share' directory tree,</em></strong>
type:<br/>
  <pre class="codeline"> find /usr/share \! -type d | wc -l </pre>
<strong><em>list the number of files and directories in the '/usr/share'</em></strong>
directory tree
  <pre class="codeline"> find /usr/share | wc -l </pre>
<strong><em>list the number of directories in the '/usr/share' directory</em></strong>
tree
  <pre class="codeline"> find /usr/share \! -type f | wc -l </pre>
<p>
<strong><em>find out whether perl is installed on your system, and, if so, where it resides</em></strong>
  <pre class="codeline"> which perl  ##(prints something like '/usr/bin/perl')</pre>
<p>
<strong><em>determine the format of the file '<a href="/usr/doc/HOWTO/README.gz',</em></strong>"><em><code>/usr/doc/HOWTO/README.gz',</em></strong></code></em></a>
  <pre class="codeline"> file /usr/doc/HOWTO/README.gz </pre>
<a href="/usr/doc/HOWTO/README.gz:"><em><code>/usr/doc/HOWTO/README.gz:</code></em></a> gzip compressed data, deflated, original
<p>
<strong><em>determine the compression format of the file '<a href="/usr/doc/HOWTO/README.gz'</em></strong>"><em><code>/usr/doc/HOWTO/README.gz'</em></strong></code></em></a>
  <pre class="codeline"> file -z /usr/doc/HOWTO/README.gz </pre>
<p>
<h3>File timestamps </h3><hr>
<p>
<strong><em>change the timestamp of file 'pizzicato' to the current date and time</em></strong>
  <pre class="codeline"> touch pizzicato </pre>
<p>
<strong><em>change the timestamp of file 'pizzicato' to '17 May 1999 14:16'</em></strong>
  <pre class="codeline"> touch -d '17 May 1999 14:16' pizzicato </pre>
<p>
<strong><em>change the timestamp of file 'phone' to '14:16'</em></strong>
  <pre class="codeline"> touch -d '14:16' phone </pre>
<p>
<strong><em>split 'large.mp3' into separate files of one megabyte each, whose names begin with 'large.mp3.'</em></strong>
  <pre class="codeline"> split -b1m large.mp3 large.mp3. </pre>
<p>
<strong><em>reconstruct the original file from the split files</em></strong>
  <pre class="codeline"> cat large.mp3.* > large.mp3 </pre>
  <pre class="codeline"> rm large.mp3.* </pre>
<p>
<strong><em>determine whether the files 'master' and `backup' differ</em></strong>
  <pre class="codeline"> cmp master backup </pre>
<p>
<h3>Version control systems</h3><hr>
<p>
A version or revision control system is designed to keep track of 
different versions of a text document, or other file or files. 
These systems are often employed when writers or programmers want
to be able to revert to a previous version of a document or code file.
<p>
<h3>Cvs </h3><hr>....
<p>
<strong><em>Override and update your locally modified files through cvs..</em></strong>
  <pre class="codeline"> cvs update -C</pre>
<p>
<strong><em>List only locally modified files with CVS</em></strong>
  <pre class="codeline"> cvs -Q status | grep -i locally</pre>
<p>
<h3>Subversion </h3><hr>....
<p>
Subversion is a reasonably modern versioning system which was
supposed to replace 'cvs'. 
<p>
<strong><em>Prints total line count contribution per user for an SVN</em></strong>
  <pre class="codeline"> svn ls -R | egrep -v -e "\/$" | xargs svn blame | awk '{print $2}' | sort | uniq -c | sort -r</pre>
<p>
<strong><em>Add new files/directory to subversion repository</em></strong>
  <pre class="codeline"> svn status | grep '^\?' | sed -e 's/^\?//g' | xargs svn add</pre>
<p>
<strong><em>Skip over .svn directories when using the "find" command.</em></strong>
  <pre class="codeline"> find . -not \( -name .svn -prune \)</pre>
<p>
<strong><em>Archive all SVN repositories in a platform independent form</em></strong>
  <pre class="codeline"> find repMainPath -maxdepth 1 -mindepth 1 -type d | while read dir; do echo processing $dir; sudo svnadmin dump --deltas $dir >dumpPath/`basename $dir`; done</pre>
<p>
<strong><em>Ignore a directory in SVN, permanently</em></strong>
  <pre class="codeline"> svn propset svn:ignore "*" tool/templates_c; svn commit -m "Ignoring tool/templates_c"</pre>
<p>
<strong><em>Add all files in current directory to SVN</em></strong>
  <pre class="codeline"> svn add --force *</pre>
<p>
<strong><em>Get a range of SVN revisions from svn diff and tar gz them</em></strong>
  <pre class="codeline"> tar cvfz changes.tar.gz --exclude-vcs `svn diff -rM:N --summarize . | grep . | awk '{print $2}' | grep -E -v '^\.$'`</pre>
<p>
<strong><em>get colorful side-by-side diffs of files in svn with vim</em></strong>
  <pre class="codeline"> vimdiff <(svn cat "$1") "$1"</pre>
<p>
<strong><em>sync svn working copy and remote repository (auto adding new</em></strong>
  <pre class="codeline"> svn status | grep '^?' | awk '{ print $2; }' | xargs svn add</pre>
<p>
<strong><em>Commit only newly added files to subversion repository</em></strong>
  <pre class="codeline"> svn ci `svn stat |awk '/^A/{printf $2" "}'`</pre>
<p>
<strong><em>Add all unversioned files to svn</em></strong>
  <pre class="codeline"> svn st | grep "^\?" | awk "{print \$2}" | xargs svn add $1</pre>
<p>
<strong><em>Add all files not under subversion control</em></strong>
  <pre class="codeline"> for i in $(svn st | grep "?" | awk '{print $2}'); do svn add $i; done;</pre>
<p>
<strong><em>Deleting Files from svn which are missing</em></strong>
  <pre class="codeline"> svn status | grep '!' | sed 's/!/ /' | xargs svn del --force</pre>
<p>
<strong><em>Have subversion ignore a file pattern in a directory</em></strong>
  <pre class="codeline"> svn propset svn:ignore "*txt" log/</pre>
<p>
<strong><em>output list of modifications for an svn revision</em></strong>
  <pre class="codeline"> svn log $url -r $revision -v | egrep " [RAMD] \/" | sed s/^.....//</pre>
<p>
<strong><em>Recursively Add Changed Files to Subversion</em></strong>
  <pre class="codeline"> svn status | grep "^\?" | awk '{print $2}' | xargs svn add</pre>
<p>
<strong><em>Output a list of svn repository entities to xml file</em></strong>
  <pre class="codeline"> svn list -R https://repository.com --xml >> svnxxmlinfo.xml</pre>
<p>
<strong><em>fetch all revisions of a specific file in an SVN repository</em></strong>
  <pre class="codeline"> svn log fileName|cut -d" " -f 1|grep -e "^r[0-9]\{1,\}$"|awk {'sub(/^r/,"",$1);print "svn cat fileName@"$1" > /tmp/fileName.r"$1'}|sh</pre>
<p>
<strong><em>Create subversion undo point</em></strong>
  <pre class="codeline"> function svnundopoint() { if [ -d .undo ]; then r=`svn info | grep Revision | cut -f 2 -d ' '` && t=`date +%F_%T` && f=${t}rev${r} && svn diff>.undo/$f && svn stat>.undo/stat_$f; else echo Missing .undo directory; fi }</pre>
<p>
<strong><em>gets all files committed to svn by a particular user since a given date</em></strong>
  <pre class="codeline"> svn log -v -r{2009-05-21}:HEAD | awk '/^r[0-9]+ / {user=$3} /yms_web/ {if (user=="george") {print $2}}' | sort | uniq</pre>
<p>
<h3>Git </h3><hr>....
<p>
This system seems to be used for linux development and apparently
was written by Mr Torvalds himself after dissatisfaction with
other tools
<p>
<strong><em>Add forgotten changes to the last git commit</em></strong>
  <pre class="codeline"> git commit --amend</pre>
<p>
<p>
<strong><em>Move all files untracked by git into a directory</em></strong>
  <pre class="codeline"> git clean -n | sed 's/Would remove //; /Would not remove/d;' | xargs mv -t stuff/</pre>
<p>
<strong><em>Show git branches by date - useful for showing active branches</em></strong>
  <pre class="codeline"> for k in `git branch|sed s/^..//`;do echo -e `git log -1 --pretty=format:"%Cgreen%ci %Cblue%cr%Creset" "$k"`\\t"$k";done|sort</pre>
<p>
<strong><em>List all authors of a particular git project</em></strong>
  <pre class="codeline"> git shortlog -s | cut -c8-</pre>
<p>
<strong><em>grep across a git repo and open matching files in gedit</em></strong>
  <pre class="codeline"> git grep -l "your grep string" | xargs gedit</pre>
<p>
<strong><em>Show (only) list of files changed by commit</em></strong>
  <pre class="codeline"> git show --relative --pretty=format:'' --name-only HASH</pre>
<p>
<strong><em>git remove files which have been deleted</em></strong>
  <pre class="codeline"> git add -u</pre>
<p>
<strong><em>Prints per-line contribution per author for a GIT repository</em></strong>
  <pre class="codeline"> git ls-files | xargs -n1 -d'\n' -i git-blame {} | perl -n -e '/\s\((.*?)\s[0-9]{4}/ && print "$1\n"' | sort -f | uniq -c -w3 | sort -r</pre>
<p>
<strong><em>github push-ing behind draconian proxies!</em></strong>
  <pre class="codeline"> git remote add origin git@SSH-HOST:<USER>/<REPOSITORY>.git</pre>
<p>
<strong><em>Display summary of git commit ids and messages for a given branch</em></strong>
  <pre class="codeline"> git log --pretty='format:%Cgreen%H %Cred%ai %Creset- %s'</pre>
<p>
<strong><em>Makes a project directory, unless it exists; changes into the dir,</em></strong>
  <pre class="codeline"> gitstart () { if ! [[ -d "$@" ]]; then mkdir -p "$@" && cd "$@" && git init; else cd "$@" && git init; fi }</pre>
<p>
<strong><em>Display summary of git commit ids and messages for a given branch</em></strong>
  <pre class="codeline"> git log master | awk '/commit/ {id=$2} /\s+\w+/ {print id, $0}'</pre>
<p>
<strong><em>Display condensed log of changes to current git repository</em></strong>
  <pre class="codeline"> git log --pretty=oneline</pre>
<p>
<strong><em>Undo several commits by committing an inverse patch.</em></strong>
  <pre class="codeline"> git diff HEAD..rev | git apply --index; git commit</pre>
<p>
<strong><em>Stage only portions of the changes to a file.</em></strong>
  <pre class="codeline"> git add --patch <filename></pre>
<p>
<h3>Rcs an old version control system </h3><hr>....
<p>
<strong><em>check in the file 'novel' with RCS</em></strong>
  <pre class="codeline"> ci novel </pre>
<p>
<strong><em>deposit this revision in RCS</em></strong>
  <pre class="codeline"> ci novel </pre>
<p>
<strong><em>check out the latest revision of the file 'novel' for editing,</em></strong>
  <pre class="codeline"> co -l novel </pre>
<p>
<strong><em>check out the current revision of file 'novel', but dont permit any changes</em></strong>
  <pre class="codeline"> co novel </pre>
<p>
<strong><em>check out revision 1.14 of file 'novel'</em></strong>
  <pre class="codeline"> co -l -r1.14 novel ##(check-in the latest changes first)</pre>
<p>
<strong><em>view the revision log for file 'novel'</em></strong>
  <pre class="codeline"> rlog novel </pre>
<p>
<h3>Reading text files </h3><hr>....
<p>
<strong><em>page through the text file 'README'</em></strong>
  <pre class="codeline"> less README </pre>
<p>
<strong><em>page through all of the Unix FAQ files in '/usr/doc/FAQ'</em></strong>
  <pre class="codeline"> less /usr/doc/FAQ/unix-faq-part* </pre>
This command starts less, opens in it all of the files that match the
given pattern '/usr/doc/FAQ/unix-faq-part*', and begins displaying the
<p>
<strong><em>peruse the file 'translation' with non-printing characters displayed </em></strong>
  <pre class="codeline"> cat -v translation | less  ##(non-printing characters are shown with 'hats') </pre>
<p>
<strong><em>output the first ten lines of file 'placement-list'</em></strong>
  <pre class="codeline"> head placement-list </pre>
<p>
<strong><em>output the first line of file 'placement-list'</em></strong>
  <pre class="codeline"> head -1 placement-list </pre>
<p>
<strong><em>output the first sixty-six lines of file 'placement-list'</em></strong>
  <pre class="codeline"> head -66 placement-list </pre>
<p>
<strong><em>output the first character in the file 'placement-list'</em></strong>
  <pre class="codeline"> head -c1 placement-list </pre>
<p>
<strong><em>output the last ten lines of file 'placement-list'</em></strong>
  <pre class="codeline"> tail placement-list </pre>
<p>
<strong><em>output the last fourteen lines of file 'placement-list'</em></strong>
  <pre class="codeline"> tail -14 placement-list </pre>
<p>
<strong><em>follow the end of the file 'access_log'</em></strong>
  <pre class="codeline"> tail -f access_log </pre>
<p>
<strong><em>output line 47 of file 'placement-list'</em></strong>
  <pre class="codeline"> sed '47!d' placement-list </pre>
<p>
<strong><em>output lines 47 to 108 of file 'placement-list'</em></strong>
  <pre class="codeline"> sed '47,108!d' placement-list </pre>
<p>
<strong><em>output the tenth line in the file 'placement-list'</em></strong>
  <pre class="codeline"> head placement-list | tail -1 </pre>
<p>
<strong><em>output the fifth and fourth lines from the bottom of file 'placement-list'</em></strong>
  <pre class="codeline"> tail -5 placement-list | head -2 </pre>
<p>
<strong><em>output the 500th character in 'placement-list'</em></strong>
  <pre class="codeline"> head -c500 placement-list | tail -c1 </pre>
<p>
<strong><em>output the first character on the fifth line of the file 'placement-list'</em></strong>
  <pre class="codeline"> head -5 placement-list | tail -1 | head -c1 </pre>
<p>
<strong><em>output all the text from file 'book-draft' between `Chapter 3' and 'Chapter 4'</em></strong>
  <pre class="codeline"> sed -n '/Chapter 3/,/Chapter 4/p' book-draft </pre>
<p>
<strong><em>output all the text from file 'book-draft', except that which lies between the text 'Chapter 3' and `Chapter 4'</em></strong>
  <pre class="codeline"> sed '/Chapter 3/,/Chapter 4/p' book-draft </pre>
<p>
<strong><em>apply the kraut filter to the text in the file '/etc/motd'</em></strong>
  <pre class="codeline"> cat /etc/motd | kraut </pre>
<p>
<strong><em>view the contents of the text file 'alice-springs' in sview</em></strong>
  <pre class="codeline"> sview alice-springs </pre>
<p>
<strong><em>view an ASCII character set</em></strong>
  <pre class="codeline"> man ascii </pre>
<p>
<strong><em>view the ISO 8859-1 character set</em></strong>
  <pre class="codeline"> man iso_8859_1 </pre>
<p>
<strong><em>run the vi tutorial, type the following from your home directory:</em></strong>
  <pre class="codeline"> cp /usr/doc/nvi/vi.beginner.gz . </pre>
  <pre class="codeline"> gunzip vi.beginner </pre>
<p>
<strong><em>concatenate these files into a new file, 'novels'</em></strong>
  <pre class="codeline"> cat early later > novels </pre>
<p>
<strong><em>make a file, 'novels', with some text in it</em></strong>
cat > novels 
This Side of Paradise 
The Beautiful and Damned 
,,,
<p>
<strong><em>add a line of text to the bottom of file 'novels'</em></strong>
  <pre class="codeline"> cat >> novels </pre>
The Last Tycoon 
C-d
<p>
<strong><em>insert several lines of text at the beginning of the file 'novels'</em></strong>
  <pre class="codeline"> ins novels </pre>
The Novels of F. Scott Fitzgerald 
<p>
<strong><em>process the file and write to the file 'monday.txt'</em></strong>
  <pre class="codeline"> m4 menu > monday.txt </pre>
Debian: 'an'
<p>
<strong><em>output all anagrams of the word 'lake'</em></strong>
  <pre class="codeline"> an lake </pre>
<p>
<strong><em>output all anagrams of the phrase 'lakes and oceans'</em></strong>
  <pre class="codeline"> an 'lakes and oceans' </pre>
<p>
<strong><em>output only anagrams of the phrase 'lakes and oceans' which contain the string 'seas'</em></strong>
  <pre class="codeline"> an -c seas 'lakes and oceans' </pre>
<p>
<strong><em>output all of the words that can be made from the letters of the word 'seas'</em></strong>
  <pre class="codeline"> an -w seas </pre>
<p>
<strong><em>output all of the palindromes in the system dictionary</em></strong>
  <pre class="codeline"> perl -lne 'print if $_ eq reverse' /usr/dict/words </pre>
<p>
<strong><em>make a cut-up from a file called 'nova'</em></strong>
  <pre class="codeline"> cutup nova </pre>
<p>
Debian: 'dadadodo'
<p>
<strong><em>output random text based on the text in the file 'nova'</em></strong>
  <pre class="codeline"> dadadodo nova </pre>
<p>
<strong><em>output all non-empty lines from the file 'term-paper'</em></strong>
  <pre class="codeline"> grep . term-paper </pre>
<p>
<strong><em>output only the lines from the file 'term-paper' that contain more than just space characters</em></strong>
  <pre class="codeline"> grep '[^ ].' term-paper </pre>
<p>
<strong><em>output only the odd lines from file 'term-paper'</em></strong>
  <pre class="codeline"> sed 'n;d' term-paper </pre>
<p>
<strong><em>double-space the file 'term-paper' and and save to 'term-paper.print'</em></strong>
  <pre class="codeline"> pr -d -t term-paper > term-paper.print </pre>
<p>
<strong><em>triple-space the file 'term-paper' and save to the file 'term-paper.print'</em></strong>
  <pre class="codeline"> sed 'G;G' term-paper > term-paper.print </pre>
<p>
<strong><em>quadruple-space the file 'term-paper', and save to the file 'term-paper.print'</em></strong>
  <pre class="codeline"> sed 'G;G;G' term-paper > term-paper.print </pre>
<p>
<strong><em>output the file 'owners-manual' with a five-space (or five-column) margin to a new file, 'owners-manual.pr'</em></strong>
  <pre class="codeline"> pr -t -o 5 -w 77 owners-manual > owners-manual.pr </pre>
This command is almost always used for printing, so the output is
<p>
<strong><em>print the file 'owners-manual' with a 5-column margin and 80 columns of text</em></strong>
  <pre class="codeline"> pr -t -o 5 -w 85 owners-manual | lpr </pre>
<p>
<strong><em>print the file 'owners-manual' with a 5-column margin and 75 columns of text</em></strong>
  <pre class="codeline"> pr -t -o 5 -w 80 owners-manual | lpr </pre>
<p>
<strong><em>convert all tab characters to spaces in 'list', and write the output to 'list2'</em></strong>
  <pre class="codeline"> expand list > list2 </pre>
<p>
<strong><em>convert initial tab characters to spaces in 'list', and write the output to the standard output</em></strong>
  <pre class="codeline"> expand -i list </pre>
<p>
<strong><em>convert every 8 leading space characters to tabs in 'list2', saving in 'list'</em></strong>
  <pre class="codeline"> unexpand list2 > list </pre>
<p>
<strong><em>convert all occurrences of eight space characters to tabs in file 'list2', and write the output to the standard output</em></strong>
  <pre class="codeline"> unexpand -a list2 </pre>
<p>
<strong><em>convert every leading space character to a tab character in 'list2', and write the output to the standard output</em></strong>
  <pre class="codeline"> unexpand -t 1 list2 </pre>
<p>
<strong><em>paginate the file 'listings' and write the output to a file called 'listings.page'</em></strong>
  <pre class="codeline"> pr -f -h "" listings > listings.page </pre>
By default, pr outputs pages of 66 lines each. You can specify the page
<p>
<strong><em>paginate the file 'listings' with 43-line pages, and write the</em></strong>
output to a file called 'listings.page'
  <pre class="codeline"> pr -f -h "" -l 43 listings > listings.page </pre>
<h3>Note</h3><hr>: If a page has more lines than a printer can fit on a physical
<p>
<strong><em>print the file 'duchess' with the default pr preparation</em></strong>
  <pre class="codeline"> pr duchess | lpr </pre>
<p>
You can also use pr to put text in columns -- give the number of
<p>
<strong><em>print the file 'news.update' in four columns with no headers or</em></strong>
footers
  <pre class="codeline"> pr -4 -t news.update | lpr </pre>
<p>
<strong><em>replace plaintext-style italics with TeX '\it' commands</em></strong>
M-x replace-regular-expression 
_\([^_]+\)_ 
\{\\it \1} 
<strong><em>replace TeX-style italics with plaintext _underscores_</em></strong>
M-x replace-regular-expression 
\{\\it \{\([^\}]+\)\} 
_\1_ 
<p>
<strong><em>output the file 'term-paper' so that you can view underbars, type:</em></strong>
  <pre class="codeline"> ul term-paper </pre>
<p>
<strong><em>output the file 'term-paper' with all backspace characters stripped out</em></strong>
  <pre class="codeline"> col -u term-paper </pre>
<p>
<strong><em>sort the file 'provinces' and output all lines in ascending order</em></strong>
  <pre class="codeline"> sort provinces </pre>
<p>
<strong><em>sort the file 'provinces' and output all lines in descending order</em></strong>
  <pre class="codeline"> sort -r provinces </pre>
<p>
<strong><em>peruse the file 'report' with each line of the file preceded by line numbers</em></strong>
  <pre class="codeline"> nl report | less ##(set the numbering style with the '-b')</pre>
<p>
<strong><em>show 'report', each line preceded by line numbers, starting with 2 step 4</em></strong>
  <pre class="codeline"> nl -v 2 -i 4 report </pre>
<p>
<strong><em>peruse the text file 'report' with each line of the file numbered</em></strong>
  <pre class="codeline"> cat -n report | less </pre>
<p>
<strong><em>peruse the text file 'report' with each non-blank line of the file numbered</em></strong>
  <pre class="codeline"> cat -b report | less </pre>
<p>
<strong><em>write a line-numbered version of file 'report' to file 'report.lines'</em></strong>
  <pre class="codeline"> cat -n report > report.lines </pre>
<p>
<strong><em>output the file 'prizes' in line-for-line reverse order</em></strong>
  <pre class="codeline"> tac prizes </pre>
<p>
<strong><em>output 'prizes' in page-for-page reverse order</em></strong>
  <pre class="codeline"> tac -s $'\f' prizes  ##(using the form-feed as the delimiter)</pre>
<p>
<strong><em>output 'prizes' in word-for-word reverse order</em></strong>
  <pre class="codeline"> tac -r -s '[^a-zA-z0-9\-]' prizes </pre>
<p>
<strong><em>output 'prizes' in character-for-character reverse order</em></strong>
  <pre class="codeline"> tac -r -s '.\| </pre>
  <pre class="codeline">  ' prizes </pre>
<p>
<strong><em>output 'prizes' with the characters on each line reversed</em></strong>
  <pre class="codeline"> rev prizes </pre>
<p>
<strong><em>output lines in the file 'catalog' containing the word `CD', type:</em></strong>
  <pre class="codeline"> grep CD catalog </pre>
<p>
<strong><em>output lines in the file 'catalog' containing the word `Compact Disc'</em></strong>
  <pre class="codeline"> grep 'Compact Disc' catalog </pre>
<p>
<strong><em>output lines in all files in the current directory containing the word 'CD'</em></strong>
  <pre class="codeline"> grep CD * </pre>
<p>
<strong><em>output lines in the file 'catalog' that contain a `$' character, type:</em></strong>
  <pre class="codeline"> grep '\$' catalog </pre>
<p>
<strong><em>output lines in the file 'catalog' that contain the string '$1.99'</em></strong>
  <pre class="codeline"> grep '\$1\.99' catalog </pre>
<p>
<strong><em>output all lines in '/usr/dict/words' beginning with `pre', type:</em></strong>
  <pre class="codeline"> grep '^pre' /usr/dict/words </pre>
<p>
<strong><em>output all lines in the file 'book' that begin with the text `in the beginning', regardless of case</em></strong>
  <pre class="codeline"> grep -i '^in the beginning' book </pre>
<p>
<strong><em>output lines in the file 'sayings' ending with an exclamation point</em></strong>
  <pre class="codeline"> grep '!$' sayings </pre>
<p>
<strong><em>output all lines in '/usr/dict/words' that are exactly two characters wide</em></strong>
  <pre class="codeline"> grep '^..$' /usr/dict/words </pre>
<p>
<strong><em>output all lines in '/usr/dict/words' that are 17 characters wide</em></strong>
  <pre class="codeline"> grep '^.\{17\}$' /usr/dict/words </pre>
<p>
<strong><em>output all lines in '/usr/dict/words' that are 25 or more characters wide</em></strong>
  <pre class="codeline"> grep '^.\{25,\}$' /usr/dict/words </pre>
<p>
<strong><em>output all lines in 'playlist' containing either 'the sea' or 'cake'</em></strong>
  <pre class="codeline"> grep 'the sea\|cake' playlist </pre>
<p>
<strong><em>output all lines in '/usr/dict/words' that are not three characters wide</em></strong>
  <pre class="codeline"> grep -v '^...$' </pre>
<p>
<strong><em>output all lines in 'access_log' that do not contain the string 'http'</em></strong>
  <pre class="codeline"> grep -v http access_log </pre>
<p>
<strong><em>output lines in '/usr/dict/words' that only contain vowels, type:</em></strong>
  <pre class="codeline"> grep -i '^[aeiou]*$' /usr/dict/words </pre>
<p>
<strong><em>search across line breaks for the string 'at the same time as' in the file 'notes'</em></strong>
  <pre class="codeline"> cat notes | tr -d '\r\n:\>\|-' | fmt -u | grep 'at the same time as' </pre>
<p>
<strong><em>list lines from the file 'email-archive' that contain the word 'narrative' only when it is quoted</em></strong>
  <pre class="codeline"> grep '^>' email-archive | grep narrative </pre>
<p>
<strong><em>list lines of 'archive' containing the word 'narrative', but notquoted</em></strong>
  <pre class="codeline"> grep narrative archive | grep -v '^>' </pre>
<p>
<strong><em>show lines in '/usr/dict/words' containing any of the words in the file 'swear'</em></strong>
  <pre class="codeline"> grep -f swear /usr/dict/words </pre>
<p>
<strong><em>output lines in '/usr/dict/words' not containing any of the words in 'swear'</em></strong>
  <pre class="codeline"> grep -v -i -f swear /usr/dict/words </pre>
<p>
<strong><em>search through the compressed file 'README.gz' for the text 'Linux'</em></strong>
  <pre class="codeline"> zgrep Linux README.gz </pre>
<p>
<strong><em>search the contents of the URL <a href="http://example.com/" <a href="http://example.com/</a>">http://example.com/</a></a> for lines containing the text 'gonzo' or `hunter'</em></strong>
  <pre class="codeline"> lynx -dump http://example.com/ | grep 'gonzo\|hunter' </pre>
<p>
<strong><em>search '/usr/dict/words' for lines matching `tsch' and output two lines of context before and after each line of output</em></strong>
  <pre class="codeline"> grep -C tsch /usr/dict/words </pre>
<p>
<strong><em>search '/usr/dict/words' for lines matching `tsch' and output six lines of context before and after each line of output</em></strong>
  <pre class="codeline"> grep -6 tsch /usr/dict/words </pre>
<p>
<strong><em>search '/usr/dict/words' for lines matching `tsch' and output two lines of context before each line of output</em></strong>
  <pre class="codeline"> grep -B tsch /usr/dict/words </pre>
<p>
<strong><em>search '/usr/dict/words' for lines matching `tsch' and output six lines of context after each line of output</em></strong>
  <pre class="codeline"> grep -A6 tsch /usr/dict/words </pre>
<p>
<strong><em>search '/usr/dict/words' for lines matching `tsch' and output ten lines of context before and three lines of context after each line of output</em></strong>
  <pre class="codeline"> grep -B10 -A3 tsch /usr/dict/words </pre>
<p>
<strong><em>replace the string 'helpless' with the string `helpful' in all</em></strong>
files in the current directory
  <pre class="codeline"> perl -pi -e "s/helpless/helpful/g;" * </pre>
<p>
<strong><em>search forward through the text you are perusing for the word 'cat'</em></strong>
/cat 
<p>
<strong><em>convert the text file 'saved-mail' to PostScript, with default formatting, and spool the output right to the printer</em></strong>
  <pre class="codeline"> enscript saved-mail </pre>
<p>
<strong><em>write the text file 'saved-mail' to a PostScript file, 'saved-mail.ps', and then preview it in X</em></strong>
  <pre class="codeline"> enscript -p report.ps saved-mail </pre>
  <pre class="codeline"> ghostview saved-mail.ps </pre>
<p>
<strong><em>print the contents of the text file 'saved-mail' on a PostScript printer, with text set in the Helvetica font at 12 points</em></strong>
  <pre class="codeline"> enscript -B -f "Helvetica12" saved-mail </pre>
<p>
<strong><em>make a PostScript file called 'saved-mail.ps' containing the</em></strong>
contents of the text file 'saved-mail', with text set in the
Helvetica font at 12 points
  <pre class="codeline"> enscript -B -f "Helvetica12" -p saved-mail.ps saved-mail </pre>
<p>
<strong><em>print the contents of the text file 'saved-mail' to a PostScript</em></strong>
printer, with text set in 10-point Times Roman and header text set
in 18-point Times Bold
  <pre class="codeline"> enscript -f "Times-Roman10" -F "Times-Bold18" saved-mail </pre>
<p>
<strong><em>make a PostScript file called 'saved-mail.ps' containing the</em></strong>
contents of the text file 'saved-mail', with text and headers both
set in 16-point Palatino Roman
  <pre class="codeline"> enscript -f "Palatino-Roman16" -F "Palatino-Roman16" -p</pre>
<p>
<strong><em>print a sign in 72-point Helvetica Bold type to a PostScript</em></strong>
printer
  <pre class="codeline"> enscript -B -f "Helvetica-Bold72" </pre>
<p>
<strong><em>print a sign in 63-point Helvetica Bold across the long side of</em></strong>
the page
  <pre class="codeline"> enscript -B -r --word-wrap -f "Helvetica-Bold63" </pre>
<p>
<strong><em>pretty-print the HTML file 'index.html'</em></strong>
  <pre class="codeline"> enscript -Ehtml index.html </pre>
<p>
<strong><em>pretty-print an email message saved to the file</em></strong>
'important-mail', and output it with no headers to a file named
'important-mail.ps'
  <pre class="codeline"> enscript -B -Email -p important-mail.ps important-mail </pre>
<p>
<strong><em>peruse a list of currently supported languages</em></strong>
  <pre class="codeline"> enscript --help-pretty-print | less </pre>
<p>
<strong><em>print the contents of the text file 'saved-mail' with fancy headers on a PostScript printer</em></strong>
  <pre class="codeline"> enscript -G saved-mail </pre>
<p>
<strong><em>make a PostScript file called 'saved-mail.ps' containing the</em></strong>
contents of the text file 'saved-mail', with fancy headers
  <pre class="codeline"> enscript -G -p saved-mail.ps saved-mail </pre>
<p>
<strong><em>print the contents of the text file 'saved-mail' with a custom header label containing the current page number</em></strong>
  <pre class="codeline"> enscript -b "Page $% of the saved email archive" saved-mail </pre>
<p>
<strong><em>determine whether the file 'gentle.tex' is a TeX or LaTeX file, type:</em></strong>
  <pre class="codeline"> grep '\\document' gentle.tex </pre>
<p>
<strong><em>print a copy of the PostScript version of the SGML-Tools guide to the default printer</em></strong>
  <pre class="codeline"> zcat /usr/doc/sgml-tools/guide.ps.gz | lpr </pre>
<p>
<strong><em>check the SGML file 'myfile.sgml'</em></strong>
  <pre class="codeline"> sgmlcheck myfile.sgml </pre>
<p>
<strong><em>make a plain text file from 'myfile.sgml'</em></strong>
  <pre class="codeline"> sgml2txt myfile.sgml </pre>
<p>
<strong><em>make a PostScript file from 'myfile.sgml'</em></strong>
  <pre class="codeline"> sgml2latex myfile.sgml </pre>
  <pre class="codeline"> latex myfile.latex </pre>
  <pre class="codeline"> dvips -t letter -o myfile.ps myfile.dvi </pre>
<p>
<strong><em>list all the X fonts on the system</em></strong>
  <pre class="codeline"> xlsfonts </pre>
<p>
<strong><em>list all the X fonts on the system whose name contains the text 'rea'</em></strong>
  <pre class="codeline"> xlsfonts '*rea*' </pre>
<p>
<strong><em>list all the bold X fonts on the system</em></strong>
  <pre class="codeline"> xlsfonts '*bold*' </pre>
<p>
<strong><em>display the characters in a medium Courier X font</em></strong>
  <pre class="codeline"> xfd -fn '-*-courier-medium-r-normal--*-100-*-*-*-*-iso8859-1' </pre>
<p>
<strong><em>set the console font to the scrawl_w font</em></strong>
  <pre class="codeline"> consolechars -f scrawl_w </pre>
<p>
<strong><em>set the console font to the 8x8 size sc font</em></strong>
  <pre class="codeline"> consolechars -H 8 -f sc </pre>
<p>
<strong><em>list all of the characters in the current console font</em></strong>
  <pre class="codeline"> showcfont </pre>
<p>
<strong><em>output the text 'news alert' in the default figlet font</em></strong>
  <pre class="codeline"> figlet news alert </pre>
<p>
<strong><em>output the text of the file 'poster' in the figlet `bubble' font</em></strong>
  <pre class="codeline"> cat poster | figlet -f bubble </pre>
<h3>Note</h3><hr>: The 'bubble' font is installed at `<a href="/usr/lib/figlet/bubble.flf'."><em><code>/usr/lib/figlet/bubble.flf'.</code></em></a>
<p>
<strong><em>make a banner saying 'Happy Birthday Susan'</em></strong>
  <pre class="codeline"> banner 'Happy Birthday Susan' </pre>
<p>
<strong><em>preview the file '<a href="/usr/doc/gs/examples/tiger.ps'</em></strong>"><em><code>/usr/doc/gs/examples/tiger.ps'</em></strong></code></em></a>
  <pre class="codeline"> ghostview /usr/doc/gs/examples/tiger.ps </pre>
<strong><em>extract only the first page from the file 'abstract.dvi' and</em></strong>
send the PostScript output to the printer
  <pre class="codeline"> dvips -pp1 abstract.dvi </pre>
<p>
By default, dvips will output to the printer; to save the PostScript
<p>
<strong><em>output as PostScript the pages 137 to 146 of the file 'abstract.dvi' to the file `abstract.ps'</em></strong>
  <pre class="codeline"> dvips -pp137-146 -o abstract.ps abstract.dvi </pre>
<p>
<strong><em>select the first ten pages, page 104, pages 23 through 28, and</em></strong>
page 2 from the file 'newsletter.ps' and write it to the file
'selection.ps'
  <pre class="codeline"> psselect -p1-10,104,23-28,2 newsletter.ps selection.ps </pre>
<p>
<strong><em>write the 2nd-to-last to the tenth pages from the PostScript file </em></strong>
<strong><em>'newsletter.ps' and output them to the file 'selection.ps'</em></strong>
  <pre class="codeline"> psselect -p_2-10 newsletter.ps selection.ps </pre>
<p>
<strong><em>write all even pages in 'newsletter.ps' to a new file, 'even.ps'</em></strong>
  <pre class="codeline"> psselect -e newsletter.ps even.ps </pre>
<p>
<strong><em>select all of the odd pages in the file 'newsletter.ps' and</em></strong>
write them to a new file, 'odd.ps'
  <pre class="codeline"> psselect -o newsletter.ps odd.ps </pre>
Use an underscore ('_') alone to insert a blank page, and use `-r' to
<p>
<strong><em>select the last ten pages of file 'newsletter.ps', followed by a</em></strong>
blank page, followed by the first ten pages, and output them to a
new file, 'selection.ps'
  <pre class="codeline"> psselect -p_1-_10,_,1-10 newsletter.ps selection.ps </pre>
<strong><em>select the pages 59, 79, and 99 in the file 'newsletter.ps', and</em></strong>
output them in reverse order (with the 99th page first) to a new
file, 'selection.ps'
  <pre class="codeline"> psselect -p59,79,99 -r newsletter.ps selection.ps </pre>
<p>
<strong><em>make a new PostScript file, 'double.ps', putting two pages from</em></strong>
the file 'single.ps' on each page
  <pre class="codeline"> psnup -2 single.ps double.ps </pre>
To specify the paper size, give the name of a standard paper size as an
<p>
<strong><em>rearrange the pages of file 'newsletter.ps' into a signature and</em></strong>
write it to the file 'newsletter.bound.ps'
  <pre class="codeline"> psbook newsletter.ps newsletter.bound.ps </pre>
By default, psbook uses one signature for the entire file. If the file
<p>
<strong><em>rearrange the pages of file 'newsletter.ps' into an eight-sided</em></strong>
signature and write it to 'newsletter.bound.ps'
  <pre class="codeline"> psbook -s8 newsletter.ps newsletter.bound.ps </pre>
<p>
<strong><em>resize the file 'double.ps' to US letter-sized paper, saving to 'new.ps'</em></strong>
  <pre class="codeline"> psresize -pletter double.ps new.ps </pre>
<p>
<strong><em>merge the files 's1.ps', 's2.ps', and 's3.ps' into a postscript file, 'slideshow.ps'</em></strong>
  <pre class="codeline"> psmerge -oslideshow.ps s1.ps s2.ps s3.ps </pre>
<p>
<strong><em>make a booklet from the file 'newsletter.ps':</em></strong>
  <pre class="codeline"> psbook newsletter.ps newsletter.signature.ps </pre>
<p>
<strong><em>make a double-sized booklet on letter-sized paper in landscape</em></strong>
orientation, from a file using letter-sized portrait orientation,
type:<br/>
  <pre class="codeline"> psbook input.ps > temp1.ps </pre>
<p>
<strong><em>make a text file, 'sutra.txt', from the input file `sutra.ps', type:</em></strong>
  <pre class="codeline"> ps2ascii sutra.ps sutra.txt </pre>
<p>
<strong><em>see how much free space is left on the system's disks</em></strong>
  <pre class="codeline"> df </pre>
<p>
<strong><em>output the disk usage for the folder tree whose root is the current directory</em></strong>
  <pre class="codeline"> du </pre>
<p>
<strong><em>output the disk usage, in kilobytes, of the '/usr/local' directory tree</em></strong>
  <pre class="codeline"> du -k /usr/local </pre>
<p>
<strong><em>show the number of megabytes used by the file '/tmp/cache'</em></strong>
  <pre class="codeline"> du -m /tmp/cache </pre>
<p>
<strong><em>output only the total disk usage of the '/usr/local' directory tree</em></strong>
  <pre class="codeline"> du -s /usr/local </pre>
<p>
<strong><em>output only the total disk usage, in kilobytes, of the '/usr/local' folder tree</em></strong>
  <pre class="codeline"> du -s -k /usr/local </pre>
<p>
<strong><em>format a floppy disk in the first removable floppy drive</em></strong>
  <pre class="codeline"> mke2fs /dev/fd0 </pre>
<p>
<h3>Mounting and unmounting media</h3><hr>
<p>
In order for a some piece of media to be used in Linux it
normally needs to be 'mounted' or 'unmounted'
<p>
<strong><em>mount a floppy</em></strong>
  <pre class="codeline"> mount /floppy </pre>
<p>
<strong><em>mount the floppy in the first floppy drive to '~/tmp'</em></strong>
  <pre class="codeline"> mount /dev/fd0 ~/tmp </pre>
<p>
Once you have mounted a floppy, its contents appear in the directory
you specify, and you can use any file command on them.
<p>
<strong><em>list the contents of the base directory of the floppy mounted on '/floppy'</em></strong>
  <pre class="codeline"> ls /floppy </pre>
<p>
<strong><em>list the contents of the whole folder tree on the floppy mounted on '/floppy'</em></strong>
  <pre class="codeline"> ls -lR /floppy </pre>
<p>
<strong><em>umount the usb stick that is mounted on '/floppy'</em></strong>
  <pre class="codeline"> umount /floppy  </pre>
<p>
You cant unmount the drive if you are in the mounted folder
<p>
<strong><em>mount a CD-ROM on the system</em></strong>
  <pre class="codeline"> mount /cdrom   ##(the contents are then available at '/cdrom/')</pre>
<p>
<strong><em>mount the disc in the CD-ROM drive to the '/usr/local/share/clipart' directory</em></strong>
  <pre class="codeline"> mount /dev/cdrom /usr/local/share/clipart </pre>
<p>
<strong><em>peruse a directory tree graph of the CD-ROM's contents</em></strong>
  <pre class="codeline"> tree /usr/local/share/clipart | less </pre>
<p>
<strong><em>change to the root directory of the CD-ROM</em></strong>
  <pre class="codeline"> cd /usr/local/share/clipart </pre>
<p>
<strong><em>list the contents of the root directory of the CD-ROM</em></strong>
  <pre class="codeline"> ls /usr/local/share/clipart </pre>
<p>
<strong><em>unmount the disc in the CD-ROM drive mounted on '/cdrom'</em></strong>
  <pre class="codeline"> umount /cdrom   ##(if files are in use, it may be impossible to unmount)</pre>
<p>
<h3>Printing </h3><hr>
<p>
<strong><em>print the file 'invoice'</em></strong>
  <pre class="codeline"> lpr invoice </pre>
<p>
<strong><em>type a message with banner and send it to the printer</em></strong>
  <pre class="codeline"> banner "Bon voyage!" | lpr </pre>
<p>
<strong><em>print a verbose, recursive listing of the '/usr/doc/HOWTO' directory</em></strong>
  <pre class="codeline"> ls -lR /usr/doc/HOWTO | lpr </pre>
<p>
<strong><em>send the file 'nightly-report' to the printer called bossomatic, type:</em></strong>
  <pre class="codeline"> lpr -P bossomatic nightly-report </pre>
<p>
<strong><em>print a dozen copies of the file 'nightly-report'</em></strong>
  <pre class="codeline"> lpr -#12 nightly-report </pre>
<p>
<h3>Printing queues </h3><hr>....
<p>
<strong><em>view the spool queue for the default printer</em></strong>
  <pre class="codeline"> lpq </pre>
<p>
<strong><em>view the spool queue for the printer called bossomatic</em></strong>
  <pre class="codeline"> lpq -P bossomatic </pre>
<p>
<strong><em>list the print jobs for user harpo</em></strong>
  <pre class="codeline"> lpq harpo </pre>
<p>
<h3>Canceling print jobs </h3><hr>....
<p>
<strong><em>cancel print job 83</em></strong>
  <pre class="codeline"> lprm 83 </pre>
<p>
<strong><em>cancel all of your print jobs, but already spooled pages still print</em></strong>
  <pre class="codeline"> lprm - </pre>
<p>
<strong><em>print the current buffer with page numbers and headers</em></strong>
  <pre class="codeline">  M-x print-buffer </pre>
<p>
<strong><em>print the current buffer with no additional print formatting done to the text</em></strong>
  <pre class="codeline"> M-x lpr-buffer </pre>
<p>
<strong><em>print a PostScript image of the current buffer</em></strong>
  <pre class="codeline"> M-x ps-print-buffer </pre>
<p>
<strong><em>print the DVI file 'list.dvi'</em></strong>
  <pre class="codeline"> dvips list.dvi </pre>
<p>
<strong><em>print 'home.dvi' on an envelope loaded in the manual feed tray of the printer</em></strong>
  <pre class="codeline"> dvips -m -t landscape home.dvi </pre>
<p>
<strong><em>list the available printer formats</em></strong>
  <pre class="codeline"> gs -? </pre>
<p>
<h3>Gnu g</h3><hr>hostscript 5.10 (1998-12-17)
...more output messages...
<p>
<strong><em>convert 'tiger.ps' to a format suitable for printing on an HP Color DeskJet 500 printer</em></strong>
  <pre class="codeline"> gs -sDEVICE=cdj500 -sOutputFile=tiger.dj -dSAFER -dNOPAUSE tiger.ps < /dev/null </pre>
<p>
<strong><em>convert the file 'abstract.dvi' to PostScript</em></strong>
  <pre class="codeline"> dvips -o abstract.ps abstract.dvi </pre>
<p>
this command reads the dvi file 'abstract.dvi' and writes a postscript
version of it to the file 'abstract.ps'; the original file is not
<p>
<strong><em>output only pages 14 and 36 from file 'abstract.dvi' to a PostScript file, 'abstract.ps'</em></strong>
  <pre class="codeline"> dvips -pp14,36 -o abstract.ps abstract.dvi </pre>
<p>
<strong><em>output pages 2 - 100 from 'abstract.dvi' to a postscript file, 'abstract.ps'</em></strong>
  <pre class="codeline"> dvips -pp2-100 -o abstract.ps abstract.dvi </pre>
<p>
<strong><em>output page 1 and pages 5 to 20 from file 'abstract.dvi' to a PostScript file, 'abstract.ps'</em></strong>
  <pre class="codeline"> dvips -pp1,5-20 -o abstract.ps abstract.dvi </pre>
<p>
<strong><em>output 'abstract.dvi' as a PostScript file, 'abstract.ps', with a paper size of `legal'</em></strong>
  <pre class="codeline"> dvips -t legal -o abstract.ps abstract.dvi </pre>
<p>
<strong><em>print the file 'abstract.dvi' to the default printer in landscape mode</em></strong>
  <pre class="codeline"> dvips -t landscape abstract.dvi </pre>
<p>
<strong><em>generate a pdf file from the dvi file 'abstract.dvi'</em></strong>
  <pre class="codeline"> dvips -Ppdf -o abstract.pdf abstract.dvi </pre>
<p>
This command writes a new file, 'abstract.pdf', in PDF format.
<p>
<strong><em>convert the PDF file 'pricelist.pdf'</em></strong>
  <pre class="codeline"> pdf2ps pricelist.pdf pricelist.ps </pre>
<p>
<strong><em>output the man page for psbook as PostScript and send it to the default printer</em></strong>
  <pre class="codeline"> man -t psbook | lpr </pre>
<p>
<strong><em>output the man page for psbook to the file 'psbook.ps'</em></strong>
  <pre class="codeline"> man -t psbook > psbook.ps </pre>
<p>
In the preceding example, you can then use gs to convert the file to a
format your non-PostScript printer understands.
<p>
<strong><em>get a directory listing of the DOS disk currently in the primary floppy drive</em></strong>
  <pre class="codeline"> mdir a: </pre>
<p>
<strong><em>introduce the floppy disk in the first floppy drive as an HFS volume for the 'hfsutils'</em></strong>
  <pre class="codeline"> hmount /dev/fd0 </pre>
After you run this command, the other tools in the hfsutils package
<p>
<strong><em>get a directory listing of the currently specified Macintosh disk</em></strong>
  <pre class="codeline"> hls </pre>
Give the name of a directory as a quoted argument.
<p>
<strong><em>get a directory listing of the 'Desktop Folder' directory in the currently specified Macintosh disk</em></strong>
  <pre class="codeline"> hls 'Desktop Folder' </pre>
<p>
<strong><em>copy the file 'readme.txt' to the `Desktop Folder' directory in the current Mac disk</em></strong>
  <pre class="codeline"> hcopy readme.txt 'Desktop Folder' </pre>
<p>
<strong><em>copy the file 'Desktop Folder:Readme' from the current Mac disk</em></strong>
to the current directory
  <pre class="codeline"> hcopy 'Desktop Folder:Readme' . </pre>
<p>
<strong><em>delete the file 'Desktop Folder:Readme' on the current Mac disk,</em></strong>
type:<br/>
  <pre class="codeline"> hdel 'Desktop Folder:Readme' </pre>
<p>
<strong><em>format the disk in the first floppy drive with a Macintosh HFS filesystem</em></strong>
  <pre class="codeline"> hformat /dev/fd0 </pre>
<p>
<strong><em>format the disk in the second floppy drive with a Mac HFS filesystem, giving it a volume label of 'Work Disk'</em></strong>
  <pre class="codeline"> hformat -l 'Work Disk' /dev/fd1 </pre>
<p>
<strong><em>format the second partition of the SCSI disk at '/dev/sd2' with a Mac HFS filesystem</em></strong>
  <pre class="codeline"> hformat /dev/sd2 2 </pre>
<p>
<strong><em>format the entire SCSI disk at '/dev/sd2' with a Mac HFS filesystem, overwriting any existing Mac filesystem and giving it a label of 'Joe's Work Disk'</em></strong>
  <pre class="codeline"> hformat -f -l "Joe's Work Disk" /dev/sd2 0  ##(Dangerous!!)</pre>
<p>
<h3>Times and dates</h3><hr>
<p>
<strong><em>Convert UNIX time to human readable date</em></strong>
  <pre class="codeline"> awk 'BEGIN{print strftime("%c",1238387636)}'</pre>
<p>
<strong><em>show a calender in english</em></strong>
  <pre class="codeline"> LC_TIME=c cal -y</pre>
<p>
<strong><em>When was your OS installed?</em></strong>
  <pre class="codeline"> ls -lct /etc | tail -1 | awk '{print $6, $7}'</pre>
<p>
<strong><em>add the current time to your bash shell prompt</em></strong>
  <pre class="codeline"> export PS1="${PS1%\\\$*}"' \t \$ '</pre>
<p>
<strong><em>Binary Clock</em></strong>
  <pre class="codeline"> watch -n 1 'echo "obase=2;`date +%s`" | bc'</pre>
<p>
<strong><em>output the numeric day of the year that 21 June falls on in the current year</em></strong>
  <pre class="codeline"> date -d '21 Jun' +%j   ##(may print '172' for example)</pre>
<p>
<strong><em>hear the current system time</em></strong>
  <pre class="codeline"> saytime </pre>
<p>
<strong><em>remind yourself to leave at 8:05 p.m.</em></strong>
  <pre class="codeline"> leave 2005 </pre>
<p>
<strong><em>ring the bell in five seconds</em></strong>
  <pre class="codeline"> sleep 5; echo -e '\a' </pre>
<p>
<strong><em>announce the time in exactly five minutes</em></strong>
  <pre class="codeline"> sleep 5m; saytime & </pre>
<p>
<strong><em>announce the time in thirty seconds</em></strong>
  <pre class="codeline"> sleep 30; saytime </pre>
<p>
<h3>Display the current time and date </h3><hr>....
<p>
<strong><em>print todays time and date in the format '12:32am 23 July 2009'</em></strong>
  <pre class="codeline"> date "+%l:%M%p %d %B %Y" </pre>
<p>
<strong><em>display todays date in the format '3mar2009'</em></strong>
  <pre class="codeline"> date "+%d%b%Y" </pre>
<p>
<strong><em>find out what format specifiers you can use with the 'date' command</em></strong>
  <pre class="codeline"> man strftime</pre>
<p>
<strong><em>?? perl one-liner to get the current week number</em></strong>
  <pre class="codeline"> date +%V</pre>
<p>
<strong><em>output the current date and time in RFC822 format</em></strong>
  <pre class="codeline"> date -R   ##(prints 'Fri, 11 May 2001 11:10:29 -0400' for example)</pre>
<p>
<strong><em>output the current system date and time in a standard format</em></strong>
  <pre class="codeline"> date   </pre>
<p>
This prints something like 'Fri May 11 11:10:29 EST 2001' where 'EST'
signifies the timezone (in this case 'Eastern Standard Time')
<p>
<strong><em>output the current date and time in UTC format</em></strong>
  <pre class="codeline"> date -u   ##(prints 'Fri May 11 15:10:29 UTC 2001')</pre>
<p>
<strong><em>Display a cool clock on your terminal</em></strong>
  <pre class="codeline"> watch -t -n1 "date +%T|figlet"</pre>
<p>
<h3>Using date time stamps </h3><hr>....
<p>
<strong><em>backup the file 'eg.txt' to 'eg.txt.13may2010' (using the current date)</em></strong>
  <pre class="codeline"> cp eg.txt{,.$(date "+%d%b%Y")}</pre>
<p>
<strong><em>backup the file 'eg.txt' with a readable format date-time stamped file</em></strong>
  <pre class="codeline"> cp eg.txt{,.$(date "+%d%b%Y.%I-%M%p")}</pre>
<p>
This will back up the file 'eg.txt' to 'eg.txt.13may2010.10-45am' for
example where the filename used represents the current date and time.
<p>
<strong><em>a function to back up a file with a date-time stamp name</em></strong>
  <pre class="codeline"> bak(){ [ -z "$1" ] || cp $1{,.$(date "+%d%b%Y.%I-%M%p")}; } </pre>
<p>
<h3>Setting the system date and time </h3><hr>....
<p>
There are a number of ways to set the date and time on the computer.
One way is to set the time just locally. Another way is to use a 
network time server to set the time, which should be more accurate in
the long run
<p>
<strong><em>advance the clock by 3 minutes</em></strong>
  <pre class="codeline"> date -s '+3 mins'</pre>
<p>
<strong><em>set the hardware clock</em></strong>
  <pre class="codeline"> hwclock</pre>
<p>
<strong><em>set the system date</em></strong>
  <pre class="codeline"> rdate -s time-A.timefreq.bldrdoc.gov</pre>
<p>
<strong><em>use a network time server to set the date</em></strong>
  <pre class="codeline"> man ntp</pre>
<p>
<h3>Converting time and date formats </h3><hr>....
<p>
Date and times come in many different text formats, depending
on the country and whim of the date-time writer. This is very 
thorny issue, but often must be dealt with. The problem is confounded
by cultural differences and ways of measuring time.
<p>
Unix-type operating systems such as Linux and Mac OSX have traditionally
used the number of milliseconds since 1970 as a universal date-time 
format, but this has obvious limitations.
<p>
<strong><em>Unix time to local time</em></strong>
  <pre class="codeline"> date -R -d @1234567890</pre>
<p>
<h3>Converting date and time timezones </h3><hr>....
<p>
If you need to know the equivalent date and time in a different
timezone, then you will have to do a conversion.
<p>
<h3>Foreign and non local times and dates </h3><hr>....
<p>
<strong><em>show what time and date it is in new york</em></strong>
  <pre class="codeline"> TZ=America/New_York date</pre>
<p>
<strong><em>print the time and date in the 'Indian/Maldives' timezone </em></strong>
  <pre class="codeline"> TZ=Indian/Maldives date</pre>
<p>
<strong><em>show what time and date it is in bogota, colombia </em></strong>
  <pre class="codeline"> TZ=America/Bogota date</pre>
<p>
<strong><em>show what general time zones are available</em></strong>
  <pre class="codeline"> ls /usr/share/zoneinfo/ | less</pre>
<p>
<strong><em>show all available specific time-zones </em></strong>
  <pre class="codeline"> f=/usr/share/zoneinfo; find $f | sed "s,$f,," | less</pre>
<p>
<strong><em>a function which searches for a particular time-zone by name</em></strong>
  <pre class="codeline"> tiz(){ f=/usr/share/zoneinfo; find $f | sed "s,$f,," | grep -i "$1"; }; tiz perth </pre>
<p>
<strong><em>show all the specific time-zones for the Australian zone</em></strong>
  <pre class="codeline"> ls /usr/share/zoneinfo/Australia</pre>
<p>
<strong><em>get time in other timezones</em></strong>
  <pre class="codeline"> let utime=$offsetutc*3600+$(date --utc +%s)+3600; date --utc --date=@${utime}</pre>
<p>
<strong><em>show the date in the german language</em></strong>
  <pre class="codeline"> LC_TIME=de_DE.utf8 date</pre>
<p>
<h3>Timezones </h3><hr>....
<p>
<strong><em>show the time in various timezones</em></strong>
  <pre class="codeline"> gworldclock</pre>
<p>
<strong><em>show the time in other timezones</em></strong>
  <pre class="codeline"> tzwatch</pre>
<p>
<h3>Greenwich mean time </h3><hr>....
<p>
<strong><em>print greenwich mean time in the format 'Fri Jul 17 16:36:24 2009'</em></strong>
  <pre class="codeline"> perl -e "print scalar(gmtime(1247848584))"</pre>
<p>
<h3>Date and time offsets </h3><hr>....
<p>
An date offset is something like '2 days ago' or 'tomorrow'.
<p>
<strong><em>retrieve GMT time from websites ( generally accruate )</em></strong>
  <pre class="codeline"> w3m -dump_head www.fiat.com | awk '/Date+/{print $6, $7}'</pre>
<p>
<strong><em>Get the time from NIST.GOV</em></strong>
  <pre class="codeline"> cat </dev/tcp/time.nist.gov/13</pre>
<p>
<strong><em>print the date 14 days ago. </em></strong>
  <pre class="codeline"> date --date="1 fortnight ago"</pre>
  <pre class="codeline"> date -d "1 fortnight ago"     ##(the same)</pre>
<p>
<strong><em>print date 24 hours ago</em></strong>
  <pre class="codeline"> date --date=yesterday</pre>
<p>
<strong><em>Get yesterday's date or a previous time</em></strong>
  <pre class="codeline"> date -d '1 day ago'; date -d '11 hour ago'; date -d '2 hour ago - 3 minute'; date -d '16 hour'</pre>
<p>
<h3>Unix time </h3><hr>....
<p>
Unix time is a number which represent the number of seconds since 1970-01-01
00:00:00 UTC
<p>
<strong><em>Easily decode unix-time (function)</em></strong>
  <pre class="codeline"> utime(){ perl -e "print localtime($1).\"\n\"";}</pre>
<p>
<strong><em>show the number of seconds since 1970</em></strong>
  <pre class="codeline"> date +%s</pre>
<p>
<strong><em>Unix alias for date command that lets you create timestamps</em></strong>
  <pre class="codeline"> alias timestamp='date "+%Y%m%dT%H%M%S"'</pre>
<p>
<h3>Convert to unix time </h3><hr>....
<p>
It can be useful to convert to unix-time when arithmetic needs to be performed
on dates or times, or when dates need to be sorted. The 'date' command does a
very good job of parsing diverse date strings (such as feb 7 2009 or
01/11/2001) with its '-d' option, but it has its limitations
<p>
<strong><em>convert a date string to the number of seconds since 1970</em></strong>
  <pre class="codeline"> date -d "Sat Feb 7 00:37:06 EST 2009" +%s</pre>
<p>
<strong><em>convert tomorrows date (midnight, I suppose) into unix-time</em></strong>
  <pre class="codeline"> date -d'tomorrow' +%s  ##(prints something like '1264168894')</pre>
<p>
<strong><em>print the final unix date</em></strong>
  <pre class="codeline"> date -ud @$[2**31-1]</pre>
<p>
<strong><em>convert the date 'february 7, 2009' into unix-time</em></strong>
  <pre class="codeline"> date -d "Sat Feb 7 00:00:00 EST 2009" +%s</pre>
  <pre class="codeline"> date -d "feb 7 2009" +%s</pre>
  <pre class="codeline"> date -d "7 feb 2009" +%s       ##(the same)</pre>
  <pre class="codeline"> date -d "february 7 2009" +%s  ##(the same)</pre>
  <pre class="codeline"> date -d "7/2/2009" +%s         ##(the same, expects mm/dd/yyyy)</pre>
  <pre class="codeline"> date -d "7/2/09" +%s           ##(the same)</pre>
<p>
<strong><em>these formats do NOT work (gnu date version 7.4)</em></strong>
  <pre class="codeline"> date -d "feb/7/2009" +%s       ##(NO!! doesnt work, at least for me)</pre>
  <pre class="codeline"> date -d "7/feb/2009" +%s       ##(Nor does this work!)</pre>
  <pre class="codeline"> date -d "feb 2009" +%s         ##(Nor does this work!)</pre>
<p>
<strong><em>convert the date 'Dec 11 01:25:00 2008' to a unix time value</em></strong>
  <pre class="codeline"> perl -e 'use Time::Local; print timelocal(0,25,1,11,11,2008), "\n";'</pre>
##(should print '1228919100')
<p>
<p>
<h3>Convert from unix time </h3><hr>....
<p>
<strong><em>convert unix time (seconds since 1970) to something human readable</em></strong>
  <pre class="codeline"> date -d @1234567890</pre>
<p>
<strong><em>convert unix-time (seconds since 1970) to something more human-readable</em></strong>
  <pre class="codeline"> perl -e 'print scalar(gmtime(1234567890)), "\n"'</pre>
<p>
<strong><em>convert unix-time to human-readable with awk</em></strong>
  <pre class="codeline"> echo 1234567890 | awk '{ print strftime("%c", $0); }'</pre>
<p>
<h3>Calendars </h3><hr>....
<p>
<strong><em>output a calendar for the current month</em></strong>
  <pre class="codeline"> cal </pre>
<p>
<strong><em>output a calendar for the year 2001</em></strong>
  <pre class="codeline"> cal 2001 </pre>
<p>
<strong><em>output a calendar for the current year</em></strong>
  <pre class="codeline"> cal -y </pre>
<p>
<strong><em>output a calendar for June 1991</em></strong>
  <pre class="codeline"> cal 06 1991 </pre>
<p>
<strong><em>Printing multiple years with Unix cal command</em></strong>
  <pre class="codeline"> for y in $(seq 2009 2011); do cal $y; done</pre>
<p>
<strong><em>Show this month's calendar, with today's date highlighted</em></strong>
  <pre class="codeline"> cal | grep --before-context 6 --after-context 6 --color -e " $(date +%e)" -e "^$(date +%e)"</pre>
<p>
<strong><em>Show the date of easter</em></strong>
  <pre class="codeline"> ncal -e</pre>
<p>
<strong><em>Display holidays in UK/England for 2009 (with week numbers)</em></strong>
  <pre class="codeline"> gcal -K -q GB_EN 2009 </pre>
<p>
<h3>Clocks </h3><hr>....
<p>
<strong><em>Display clock in terminal</em></strong>
  <pre class="codeline"> watch -n 1 :</pre>
<p>
<strong><em>StopWatch, simple text, hh:mm:ss using Unix Time</em></strong>
  <pre class="codeline"> export I=$(date +%s); watch -t -n 1 'T=$(date +%s);E=$(($T-$I));hours=$((E / 3600)) ; seconds=$((E % 3600)) ; minutes=$((seconds / 60)) ; seconds=$((seconds % 60)) ; echo $(printf "%02d:%02d:%02d" $hours $minutes $seconds)'</pre>
<p>
<h3>Arithmetic and numbers </h3><hr>
<p>
<strong><em>hexadecimal2decimal</em></strong>
  <pre class="codeline"> printf "%d\n" \0x64</pre>
<p>
<strong><em>floating point operations in shell scripts</em></strong>
  <pre class="codeline"> echo "5 k 3 5 / p" | dc</pre>
<p>
<strong><em>formatting number with comma</em></strong>
  <pre class="codeline"> printf "%'d\n" 1234567</pre>
<p>
<strong><em>output the result of 50 times 10</em></strong>
  <pre class="codeline"> calc 50*10    ##(prints '500')</pre>
<p>
<strong><em>output the result of 100 times the sum of 4 plus 420</em></strong>
  <pre class="codeline"> calc '100*(4+420)'  ##(prints '42400')</pre>
<p>
<strong><em>output the remainder of 10 divided by 3</em></strong>
  <pre class="codeline"> calc 10%3           ##(prints '1')</pre>
<p>
<strong><em>use bc to compute the result of 10 divided by 3, using 20 digits after the decimal point</em></strong>
  <pre class="codeline"> bc </pre>
<p>
<strong><em>Count to 65535 in binary (for no apparent reason)</em></strong>
  <pre class="codeline"> a=`printf "%*s" 16`;b=${a//?/{0..1\}}; echo `eval "echo $b"`</pre>
<p>
<h3>Bc the binary calculator </h3><hr>....
<p>
<strong><em>start the bc binary calculator in interactive mode</em></strong>
  <pre class="codeline"> bc</pre>
<p>
<strong><em>assign values to variables in bc and print the sum of 2 variables</em></strong>
  <pre class="codeline"> a=4; b=7; a+b  ##(prints '11' on a new line)</pre>
<p>
<strong><em>raise 4 to the power of 3 and multiply the result by 10</em></strong>
  <pre class="codeline"> 4^3; .*10    ##(you can use the '.' to access the last result)</pre>
<p>
<strong><em>set the number of decimal places shown to 4 and compute 112 divided by 111</em></strong>
  <pre class="codeline"> scale=4; 112/111;  ##(this prints '1.0090' on a new line)</pre>
<p>
<strong><em>set the number of decimal places show to 10</em></strong>
  <pre class="codeline"> scale=10</pre>
<p>
<strong><em>multiply the variable 'a' by 4</em></strong>
  <pre class="codeline"> a*=4</pre>
<p>
<strong><em>Floating point power p of x</em></strong>
  <pre class="codeline"> bc -l <<< "x=2; p=0.5; e(l(x)*p)"</pre>
<p>
<h3>Random numbers </h3><hr>....
<p>
see: jot 
<p>
<strong><em>lotto generator</em></strong>
  <pre class="codeline"> shuf -i 1-49 | head -n6 | sort -n| xargs</pre>
<p>
<strong><em>Outputs a 10-digit random number</em></strong>
  <pre class="codeline"> echo $RANDOM$RANDOM$RANDOM |cut -c3-12</pre>
<p>
<strong><em>Outputs a 10-digit random number</em></strong>
  <pre class="codeline"> head -c10 <(echo $RANDOM$RANDOM$RANDOM)</pre>
<p>
<strong><em>Outputs a 10-digit random number</em></strong>
  <pre class="codeline"> head -c4 /dev/urandom | od -N4 -tu4 | sed -ne '1s/.* //p'</pre>
<p>
<strong><em>Display rows and columns of random numbers with awk</em></strong>
  <pre class="codeline"> seq 6 | awk '{for(x=1; x<=5; x++) {printf ("%f ", rand())}; printf ("\n")}'</pre>
<p>
<strong><em>Print a random 8 digit number</em></strong>
  <pre class="codeline"> jot -r -n 8 0 9 | rs -g 0</pre>
<p>
<strong><em>Print a random 8 digit number</em></strong>
  <pre class="codeline"> jot -s '' -r -n 8 0 9</pre>
<p>
<strong><em>Random number generation within a range N, here N=10</em></strong>
  <pre class="codeline"> echo $(( $RANDOM % 10 + 1 ))</pre>
<p>
<strong><em>output a random number from 0 to 9</em></strong>
  <pre class="codeline"> random 10 </pre>
<p>
<strong><em>Outputs a 10-digit random number</em></strong>
  <pre class="codeline"> tr -c -d 0-9 < /dev/urandom | head -c 10</pre>
<p>
<strong><em>Printable random characters</em></strong>
  <pre class="codeline"> tr -dc '[:print:]' < /dev/urandom</pre>
<p>
<strong><em>a random number guessing game</em></strong>
  <pre class="codeline"> A=1;B=100;X=0;C=0;N=$[$RANDOM%$B+1];until [ $X -eq $N ];do read -p "N between $A and $B. Guess? " X;C=$(($C+1));A=$(($X<$N?$X:$A));B=$(($X>$N?$X:$B));done;echo "Took you $C tries, Einstein";</pre>
<p>
<strong><em>sort the lines of a file in a random way </em></strong>
  <pre class="codeline"> cat test.txt | while read f ; do printf "$RANDOM\t%s\n" "$f"; done | sort -n | cut -f2-</pre>
<p>
<strong><em>print a random number with printf </em></strong>
  <pre class="codeline"> printf "$RANDOM %s\n" number </pre>
<p>
<h3>Sequences </h3><hr>....
<p>
<strong><em>output the sequence of numbers from one to seven</em></strong>
  <pre class="codeline"> seq 7 </pre>
<p>
<strong><em>output the sequence of numbers from one to negative seven</em></strong>
  <pre class="codeline"> seq -7 </pre>
<p>
<strong><em>output the sequence of numbers from nine to zero</em></strong>
  <pre class="codeline"> seq 9 0 </pre>
<p>
<strong><em>output the sequence of numbers from negative one to negative twenty</em></strong>
  <pre class="codeline"> seq -1 -20 </pre>
<p>
<strong><em>output the sequence of numbers from -1 to 14, incrementing by 3,</em></strong>
  <pre class="codeline"> seq -1 3 14 </pre>
<p>
<strong><em>output from 9 to 999, stepping by 23, with numbers padded with zeros </em></strong>
  <pre class="codeline"> seq -w 9 23 999   ##(all numbers will have an equal 'width') </pre>
<p>
<strong><em>output the sequence of numbers from 1 to 23, with a space character between each</em></strong>
  <pre class="codeline"> seq -s ' ' 1 23 </pre>
<p>
<strong><em>concatenate all the files in this folder, whose names are numbers 25 to 75, into a new file called 'selected-mail'</em></strong>
  <pre class="codeline"> cat $(seq -s " " 25 75) > selected-mail </pre>
<p>
<strong><em>output the prime factors of 2000</em></strong>
  <pre class="codeline"> factor 2000  ##(prints '2000: 2 2 2 2 5 5 5')</pre>
<p>
<strong><em>output the number of ounces in 50 grams</em></strong>
  <pre class="codeline"> units '50 grams' 'ounces' </pre>
<strong><em>1.7636981</em></strong>
/ 0.56699046
,,,
<p>
<strong><em>determine the location of the units database</em></strong>
  <pre class="codeline"> units -V </pre>
units version 1.55 with readline, units database in
<a href="/usr/share/misc/units.dat"><em><code>/usr/share/misc/units.dat</code></em></a>
<p>
<strong><em>output the English text equivalent of 100,000</em></strong>
  <pre class="codeline"> number 100000 </pre>
<p>
<h3>Mathematics</h3><hr>
<p>
<strong><em>Find out how to say the first 66 digits of pi as a word</em></strong>
  <pre class="codeline"> pi 66 | number</pre>
<p>
<strong><em>Fibonacci numbers with awk</em></strong>
  <pre class="codeline"> awk 'BEGIN {a=1;b=1;for(i=0;i<'${NUM}';i++){print a;c=a+b;a=b;b=c}}'</pre>
<p>
<strong><em>Fibonacci numbers with sh</em></strong>
  <pre class="codeline"> prev=0;next=1;echo $prev;while(true);do echo $next;sum=$(($prev+$next));prev=$next;next=$sum;sleep 1;done</pre>
<p>
<strong><em>A handy mathematical calculator</em></strong>
  <pre class="codeline"> bc</pre>
<p>
<strong><em>Draw a Sierpinski triangle</em></strong>
  <pre class="codeline"> perl -e 'print "P1\n256 256\n", map {$_&($_>>8)?1:0} (0..0xffff)' | display</pre>
<p>
<h3>Weather</h3><hr>
<p>
<strong><em>get Hong Kong weather infomation from HK Observatory</em></strong>
  <pre class="codeline"> wget -q -O - 'http://wap.weather.gov.hk/' | sed -r 's/<[^>]+>//g;/^UV/q' | grep -v '^$'</pre>
<p>
<strong><em>check the weather</em></strong>
  <pre class="codeline"> ZIP=48104; curl http://thefuckingweather.com/?zipcode=$ZIP 2>/dev/null|grep -A1 'div class="large"'|tr '\n' ' '|sed 's/^.*"large" >\(..\)/\1/;s/&d.* <br \/>/ - /;s/<br \/>//;s/<\/div.*$//'</pre>
<p>
<strong><em>Show current weather for any US city or zipcode</em></strong>
  <pre class="codeline"> weather() { lynx -dump "http://mobile.weather.gov/port_zh.php?inputstring=$*" | sed 's/^ *//;/ror has occ/q;2h;/__/!{x;s/\n.*//;x;H;d};x;s/\n/ -- /;q';}</pre>
<p>
<h3>Money </h3><hr>
<p>
<strong><em>Get Dollar-Euro exchage rate</em></strong>
  <pre class="codeline"> curl -s wap.kitco.com/exrate.wml | awk ' BEGIN { x=0; FS = "<" } { if ($0~"^<br/>") {x=0} if (x==1) {print $1} if ($0~"EUR/US") {x=1} }'</pre>
<p>
<h3>Science</h3><hr>
<p>
<strong><em>Mirror the NASA Astronomy Picture of the Day Archive</em></strong>
  <pre class="codeline"> wget -t inf -k -r -l 3 -p -m http://apod.nasa.gov/apod/archivepix.html</pre>
<p>
<strong><em>View the latest astronomy picture of the day from NASA.</em></strong>
  <pre class="codeline"> apod(){ local x=http://antwrp.gsfc.nasa.gov/apod/;feh $x$(curl -s ${x}astropix.html|grep -Pom1 'image/\d+/.*\.\w+');}</pre>
<p>
<strong><em>Real time satellite weather wallpaper</em></strong>
  <pre class="codeline"> curl http://www.cpa.unicamp.br/imagens/satelite/ult.gif | xli -onroot -fill stdin</pre>
<p>
<h3>Geography </h3><hr>....
<p>
<strong><em>find geographical location of an ip address</em></strong>
  <pre class="codeline"> lynx -dump http://www.ip-adress.com/ip_tracer/?QRY=$1|grep address|egrep 'city|state|country'|awk '{print $3,$4,$5,$6,$7,$8}'|sed 's\ip address flag \\'|sed 's\My\\'</pre>
<p>
<strong><em>find geographical location of an ip address</em></strong>
  <pre class="codeline"> lynx -dump http://www.ip-adress.com/ip_tracer/?QRY=$1|sed -nr s/'^.*My IP address city: (.+)$/\1/p'</pre>
<p>
<strong><em>geoip lookup, the geographical location of an ip address</em></strong>
  <pre class="codeline"> geoip(){curl -s "http://www.geody.com/geoip.php?ip=${1}" | sed '/^IP:/!d;s/<[^>][^>]*>//g' ;}</pre>
<p>
<h3>Gps global positioning system </h3><hr>....
<p>
<strong><em>send a .loc file to a garmin gps over usb</em></strong>
  <pre class="codeline"> gpsbabel -D 0 -i geo -f "/path/to/.loc" -o garmin -F usb:</pre>
<p>
<h3>Shutting down the computer</h3><hr>
<p>
<strong><em>immediately shut down and halt the system</em></strong>
  <pre class="codeline"> shutdown -h now </pre>
<p>
<strong><em>immediately shutdown the system, and then reboot</em></strong>
  <pre class="codeline"> shutdown -r now </pre>
<p>
<strong><em>immediately shut down and halt the system, sending a warning to all users</em></strong>
  <pre class="codeline"> shutdown -h now "The system is being shut down now!" </pre>
<p>
<strong><em>shut down and then reboot the system at 4:23 a.m.</em></strong>
  <pre class="codeline"> shutdown -r 4:23 </pre>
<p>
<strong><em>shut down and halt the system at 8:00 p.m.</em></strong>
  <pre class="codeline"> shutdown -h 20:00 </pre>
<p>
<strong><em>shut down and halt the system in five minutes</em></strong>
  <pre class="codeline"> shutdown -h +5 </pre>
<p>
<strong><em>shut down and halt the system at midnight, and warn all logged-in users</em></strong>
  <pre class="codeline"> shutdown -h 00:00 "The system is going down for maintenance at midnight" </pre>
<p>
<strong><em>cancel any pending shutdown</em></strong>
  <pre class="codeline"> shutdown -c </pre>
<p>
<strong><em>cancel any pending shutdown and send a message to all logged in users</em></strong>
  <pre class="codeline"> shutdown -c "Sorry, I hit the wrong key!" </pre>
<p>
<strong><em>find out where perl is installed on your system</em></strong>
  <pre class="codeline"> which perl </pre>
<p>
<strong><em>create a new user with a username of bucky</em></strong>
  <pre class="codeline"> adduser bucky ##(By default, the user's home directory will be their name)</pre>
<p>
<strong><em>add the user doug to the audio group</em></strong>
  <pre class="codeline"> addgroup doug audio </pre>
<p>
<strong><em>find out how long the system has been up</em></strong>
  <pre class="codeline"> uptime </pre>
##(prints 3:34pm up 4:31, 4 users, load average: 0.01, 0.05, 0.07)
<p>
<strong><em>output a list of times when the system was rebooted</em></strong>
  <pre class="codeline"> last reboot </pre>
<p>
<strong><em>output the name of the operating system</em></strong>
  <pre class="codeline"> uname </pre>
<p>
<strong><em>output the release number of the operating system</em></strong>
  <pre class="codeline"> uname -r </pre>
<p>
<strong><em>output the CPU processor type of the system</em></strong>
  <pre class="codeline"> uname -m </pre>
<p>
<strong><em>output all of the uname information for the system you are on,</em></strong>
  <pre class="codeline"> uname -a </pre>
<p>
<strong><em>output the release name of the Debian system you are on</em></strong>
  <pre class="codeline"> cat /etc/debian_version </pre>
<p>
Debian releases have historically been named after characters
from the motion picture Toy Story.
<p>
<h3>Microsoft windows </h3><hr>
<p>
<strong><em>Change Windows Domain password from Linux</em></strong>
  <pre class="codeline"> smbpasswd -r <domain-server> -U <user name></pre>
<p>
<strong><em>Use Cygwin to talk to the Windows clipboard</em></strong>
  <pre class="codeline"> cat /dev/clipboard; $(somecommand) > /dev/clipboard</pre>
<p>
<strong><em>Mount a Windows shared folder on the local network (Ubuntu) with user</em></strong>
  <pre class="codeline"> sudo mount -t cifs -o user,username="samba username"</pre>
<p>
<strong><em>automount samba shares as devices in /mnt/</em></strong>
  <pre class="codeline"> sudo vi /etc/fstab; Go//smb-share/gino /mnt/place smbfs defaults,username=gino,password=pass 0 0<esc>:wq; mount //smb-share/gino</pre>
<p>
<strong><em>mount a windows partition</em></strong>
sudo mkdir /media/nt/
sudo fdisk -l # find out where is the nt partition
sudo mount -t ntfs -o nls=utf8,umask=0222 /dev/hdb1 /media/nt
<p>
<h3>Miscelaneous</h3><hr>
<p>
<h3>Making a simple debian package </h3><hr>....
<p>
The following is a procedure for creating a debian package.
<p>
<strong><em>create a 'debian/usr/bin' folder and copy the executable there</em></strong>
  <pre class="codeline"> mkdir -p ./debian/usr/bin; cp someprogram ./debian/usr/bin</pre>
<p>
<strong><em>create a man page for the program</em></strong>
  <pre class="codeline"> vim someprogram.1</pre>
##(this is an optional step)
<p>
<strong><em>create a man page folder and copy the man page there,</em></strong>
  <pre class="codeline"> mkdir -p ./debian/usr/share/man/man1</pre>
  <pre class="codeline"> cp ./man/man1/someprogram.1 ./debian/usr/share/man/man1</pre>
<p>
<strong><em>compress the man page with gzip (an option step)</em></strong>
<p>
<strong><em>create a copyright file to include in the package</em></strong>
  <pre class="codeline"> find /usr/share/doc -name "copyright" ##(see examples of copyright files)</pre>
<p>
<strong><em>create a folder for the copyright file and copy the file there</em></strong>
  <pre class="codeline"> mkdir -p ./debian/usr/share/doc/someprogram; ... </pre>
  <pre class="codeline"> cp ./copyright ./debian/usr/share/doc/someprogram</pre>
<p>
<strong><em>find out the dependencies of all programs which your program uses</em></strong>
  <pre class="codeline"> dkpg -S /bin/cat ##(this will display 'coreutils: /bin/cat)</pre>
<p>
<strong><em>find out the version numbers of dependencies</em></strong>
  <pre class="codeline"> apt-cache showpkg coreutils</pre>
##(this information is needed for the 'Dependencies' field of the control file)
<p>
make a control file..
<p>
Package: someprogram 
Version: 1.1-1
Section: base
Priority: optional
Architecture: all
Depends: bash (>= 2.05a-11), textutils (>= 2.0-12), awk, procps (>= \
1:2.0.7-8), sed (>= 3.02-8), grep (>= 2.4.2-3), coreutils (>= 5.0-5)
Maintainer: James Tree <james@tree.org>
Description: parses a text stream 
This script parses a text stream using a stack virtual machine 
,,,
<p>
<strong><em>copy the control file to the 'DEBIAN' folder</em></strong>
  <pre class="codeline"> mkdir -p debian/DEBIAN; cp control debian/DEBIAN</pre>
##(on some debians: find ./debian -type d | xargs chmod 755)
<p>
<strong><em>make a change log file (an optional step)</em></strong>
xlinuxstatus (1.2-1)
<strong><em>Made Debian package lintian clean.</em></strong>
-- James Tree <james@tree.org> 2002-12-13
,,,
<p>
<strong><em>make a<a href="changelog.Debian"><em><code>changelog.Debian</code></em></a> (optional)</em></strong>
<p>
linuxstatus Debian maintainer and upstream author are identical.
Therefore see also normal changelog file for Debian changes.
,,,
<p>
<strong><em>create and rename the package file</em></strong>
  <pre class="codeline"> dpkg-deb --build debian; mv debian.deb someprogram_1.1-1_all.deb</pre>
##(or build as root 'fakeroot dpkg-deb --build debian')
<p>
<strong><em>install the newly created package </em></strong>
  <pre class="codeline"> dpkg -i ./someprogram_1.1-1_all.deb</pre>
<p>
<strong><em>remove the installed package</em></strong>
  <pre class="codeline"> dpkg -r someprogram</pre>
<p>
=;
<p>
<h3>Unix program naming </h3><hr>....
<p>
The names for Unix programs appear cryptic and seem to have no relation to
their function. However there is usually an explanation for the name. Unix
commands are traditionally very short to save typing.
<p>
<center><h3 class="page-title"></h3></center> Some Unix names explained
.. unix, a pun on the <mul>tics operating system
.. linux, <linu>s torvalds version of the uni<x> operating system
.. cat, con<cat>enate files, display all mentioned files
.. less, is a successor to the 'more' program, a file page 
.. sed, <s>tream <ed>itor
.. grep, <g>lobal <r>egular <e>x<p>ression search, 
.. ed, <ed>itor
.. vi, <vi>sual editor
.. vim, <v>isual editor <im>proved
.. emacs, <e>diting <mac>ro<s>
.. wc, <w>ord <c>ount
.. tar, write to a <t>ape <ar>chive
.. ls, <l>i<s>t files,
.. troff, <t>ypesetting <r>un <off> system, formats documents
.. groff, <g>nu version of the <r>un <off> system
.. perl, <p>ractical <e>xtraction and <r>reporting <l>anguage
.. awk, text processing language by <A>ho, <W>einberger, and <K>ernighan
.. cd, <c>hange <d>irectory
.. gcc, <g>nu <c> <c>ompiler
.. wget, <w>eb <get>ter
.. curl, <c>apture <url>
.. scp, <s>ecure <c>opy <p>rogram
..
<p>
<p>
<h3>Curiosities </h3><hr>....
<p>
<strong><em>A death cow thinking in your fortune cookie</em></strong>
  <pre class="codeline"> fortune -s -c -a | cowthink -d -W 45</pre>
<p>
<strong><em>Find the cover image for an album</em></strong>
  <pre class="codeline"> albumart(){ local y="$@";awk '/View larger image/{gsub(/^.*largeImagePopup\(.|., .*$/,"");print;exit}' <(curl -s 'http://www.albumart.org/index.php?srchkey='${y// /+}'&itempage=1&newsearch=1&searchindex=Music');}</pre>
<p>
<strong><em>random xkcd comic</em></strong>
  <pre class="codeline"> display "$(wget -q http://dynamic.xkcd.com/comic/random/ -O - | grep -Po '(?<=")http://imgs.xkcd.com/comics/[^"]+(png|jpg)')"</pre>
<p>
<strong><em>Auto Rotate Cube (compiz)</em></strong>
  <pre class="codeline"> wmctrl -o 2560,0 ;sleep 2 ; echo "FIRE 001" | osd_cat -o 470 -s 8 -c red -d 10 -f -*-bitstream\ vera\ sans-*-*-*--250-*-*-*-*-*-*-* ; sleep 1; wmctrl -o 0,0</pre>
<p>
<strong><em>Matrix Style</em></strong>
  <pre class="codeline"> LC_ALL=C tr -c "[:digit:]" " " < /dev/urandom | dd cbs=$COLUMNS conv=unblock | GREP_COLOR="1;32" grep --color "[^ ]"</pre>
<p>
<strong><em>read the useless use of cat awards page</em></strong>
  <pre class="codeline"> lynx http://partmaps.org/era/unix/award.html</pre>
<p>
<strong><em>dolphins on the desktop (compiz)</em></strong>
  <pre class="codeline"> xwinwrap -ni -argb -fs -s -st -sp -nf -b -- /usr/libexec/xscreensaver/atlantis -count 20 -window-id WID &</pre>
<p>
<strong><em>for all who don't have the watch command</em></strong>
  <pre class="codeline"> watch() { while test :; do clear; date=$(date); echo -e "Every "$1"s: $2 \t\t\t\t $date"; $2; sleep $1; done }</pre>
<p>
<strong><em>Get Futurama quotations from slashdot.org servers</em></strong>
  <pre class="codeline"> echo -e "HEAD / HTTP/1.1\nHost: slashdot.org\n\n" | nc slashdot.org 80 | egrep "Bender|Fry" | sed "s/X-//"</pre>
<p>
<strong><em>View the newest xkcd comic.</em></strong>
  <pre class="codeline"> xkcd(){ local f=$(curl -s http://xkcd.com/);display $(echo "$f"|grep -Po '(?<=")http://imgs.xkcd.com/comics/[^"]+(png|jpg)');echo "$f"|awk '/<img src="http:\/\/imgs\.xkcd\.com\/comics\/.*?" title=.*/{gsub(/^.*title=.|".*?$/,"");print}';}</pre>
<p>
<strong><em>The absolutely fastest nth fibonacci number</em></strong>
  <pre class="codeline"> time echo 'n=70332;m=(n+1)/2;a=0;b=1;i=0;while(m){e[i++]=m%2;m/=2};while(i--){c=a *a;a=c+2*a*b;b=c+b*b;if(e[i]){t=a;a+=b;b=t}};if(n%2)a*a+b*b;if(!n%2)a*( a+2*b)' | bc</pre>
<p>
<strong><em>The 1 millionth fibonacci number</em></strong>
  <pre class="codeline"> time echo 'n=1000000;m=(n+1)/2;a=0;b=1;i=0;while(m){e[i++]=m%2;m/=2};while(i--){c =a*a;a=c+2*a*b;b=c+b*b;if(e[i]){t=a;a+=b;b=t}};if(n%2)a*a+b*b;if(!n%2)a *(a+2*b)' | bc</pre>
<p>
<strong><em>Another Matrix Style Implementation</em></strong>
  <pre class="codeline"> echo -ne "\e[32m" ; while true ; do echo -ne "\e[$(($RANDOM % 2 + 1))m" ; tr -c "[:print:]" " " < /dev/urandom | dd count=1 bs=50 2> /dev/null ; done</pre>
<p>
<strong><em>useless load</em></strong>
  <pre class="codeline"> cat /dev/urandom | gzip -9 > /dev/null &</pre>
<p>
<strong><em>Gets a random Futurama quote from /.</em></strong>
  <pre class="codeline"> curl -Is slashdot.org | egrep '^X-(F|B|L)' | cut -d \- -f 2</pre>
<p>
<strong><em>Get Futurama quotations from slashdot.org servers</em></strong>
  <pre class="codeline"> lynx -head -dump http://slashdot.org|egrep 'Bender|Fry'|sed 's/X-//'</pre>
<p>
<strong><em>put nothing nowhere</em></strong>
  <pre class="codeline"> cat /dev/zero > /dev/null &</pre>
<p>
<strong><em>Random line from bash.org (funny IRC quotes)</em></strong>
  <pre class="codeline"> curl -s http://bash.org/?random1</pre>
<p>
<strong><em>decode html entities with perl</em></strong>
  <pre class="codeline"> perl -ne 'use HTML::Entities;print decode_entities($_),"\n"'</pre>
<p>
<p>
<h3>Zsh</h3><hr>
<p>
<strong><em>Get length of array in zsh</em></strong>
  <pre class="codeline"> $foo[(I)$foo[-1]]</pre>
<p>
<strong><em>List the top ten commands you use for the 'zsh' shell</em></strong>
  <pre class="codeline"> perl -pe 's/.+;//' ~/.zsh_history | sort | uniq -c | sort -r|head -10</pre>
<p>
<strong><em>ZSH prompt. ':)' after program execution with no error, ':(' after</em></strong>
  <pre class="codeline"> PROMPT=$'%{\e[0;32m%}%B[%b%{\e[0m%}%n%{\e[0;32m%}@%{\e[0m%}%(4c,./%1~,% ~)%{\e[0;32m%}%B]%b% %(?,%{\e[0;32m%}:%)%{\e[0m%},%{\e[0;31m%}:(%{\e[0m%}) %# '</pre>
<p>
<strong><em>Show every subdirectory (zsh)</em></strong>
  <pre class="codeline"> ls -ld **/*(/)</pre>
<p>
<strong><em>Delete empty directories with zsh</em></strong>
  <pre class="codeline"> rm -d **/*(/^F)</pre>
<p>
<strong><em>Postpone a command [zsh]</em></strong>
  <pre class="codeline"> <alt+q></pre>
<p>
<strong><em>zsh only: access a file when you don't know the path, if it is in</em></strong>
  <pre class="codeline"> file =top</pre>
<p>
<h3>Mac osx</h3><hr>
<p>
This book focusses more on Linux but here is a little
information about OSX
<p>
<strong><em>Enable Hibernate in OS X</em></strong>
  <pre class="codeline"> sudo pmset -a hibernatemode 1</pre>
<p>
<strong><em>On Mac OS X, runs System Profiler Report and e-mails it to</em></strong>
  <pre class="codeline"> system_profiler | mail -s "$HOSTNAME System Profiler Report" user@domain.com</pre>
<p>
<strong><em>Paste OS X clipboard contents to a file on a remote machine</em></strong>
  <pre class="codeline"> pbpaste | ssh user@hostname 'cat > ~/my_new_file.txt'</pre>
<p>
<strong><em>Throttling Bandwidth On A Mac</em></strong>
  <pre class="codeline"> sudo ipfw pipe 1 config bw 50KByte/s;sudo ipfw add 1 pipe 1 src-port 80</pre>
<p>
<strong><em>OSX command to take badly formatted xml from the clipboard, cleans</em></strong>
  <pre class="codeline"> pbpaste | tidy -xml -wrap 0 | pbcopy</pre>
<p>
<strong><em>Paste the contents of OS X clipboard into a new text file</em></strong>
  <pre class="codeline"> pbpaste > newfile.txt</pre>
<p>
<strong><em>Zip a directory on Mac OS X and ignore .DS_Store (meta-data)</em></strong>
  <pre class="codeline"> zip -vr example.zip example/ -x "*.DS_Store"</pre>
<p>
<strong><em>Open up a man page as PDF (#OSX)</em></strong>
  <pre class="codeline"> function man2pdf(){ man -t ${1:?Specify man as arg} | open -f -a preview; }</pre>
<p>
@@ <a href="http://www.sveinbjorn.org/cljg2macosx" <a href="http://www.sveinbjorn.org/cljg2macosx</a>">http://www.sveinbjorn.org/cljg2macosx</a></a>
a good macosx command line site
<p>
<strong><em>on a bsd system view information about the folder structure</em></strong>
 <pre class="codeline"> man hier</pre>
<p>
<h3>Newsreaders </h3><hr>
<p>
@@ slrn
lightweight
@@ pan
graphical, good
@@ gnus
within emacs
@@ thunderbird
@@ xpn
graphical
@@ knode
the kde app
<p>
<h3>File usage</h3><hr>
<p>
@@ filelight
shows disk usage in a graphical manner
@@ gadmin-rsync
a gui wrapper for rsync with ssh, scp etc
<p>
<h3>Terminals </h3><hr>
<p>
@@ slugterm
light weight terminal for small linuxes
@@ grdc.sf.net
remote desktop for sys admins
<p>
<h3>Graphs </h3><hr>
<p>
A simple pie chart in 'gnuplot' (and with a 3d thickness)
<p>
@@ freemind.sf.net
mind mapping charting software
<p>
<h3>Wiki</h3><hr>
<p>
@@ docuwiki
a wiki with no database backend 
<p>
<h3>Music</h3><hr>
<p>
@@ amarok
@@ songbird
music playlist software
<p>
@@ frescobaldi
create musical scores from a text input file. gui application
<p>
<h3>Gps</h3><hr>
<p>
@@ gpx
an xml format for storing gps data
@@ prune
a tool for manipulating gps data
<p>
<h3>Art</h3><hr>
<p>
@@ synfig studio
a program to create animations using stick figures,
paths etc.
@@ blender 
3d animations
@@ wwww.pencil-animation.org
a tool for creating traditional drawn animation
@@ <a href="http://www.morevnaproject.org">www.morevnaproject.org</a>
an open source animation (anime)
<p>
<h3>Games</h3><hr>
<p>
@@ <a href="http://www.schoolsplay.org">www.schoolsplay.org</a>
education games. a spelling pacman, flashcards etc
'childsplay'
@@ pyspace war
an asteroid shootem up
<p>
<h3>Game engines </h3><hr>....
<p>
@@ pygame
@@ sdl
@@ irrlicht.sf.net
an open source game engine, written in c++, with bindings
for other languages
<p>
<p>
<h3>Geneology</h3><hr>
<p>
@@ gramps
gui geneology builder
<p>
<h3>Blog engines </h3><hr>
<p>
@@ bilboblog
a simple blog engine
@@ lifeograph
simple diary application
<p>
<h3>Jargon</h3><hr>
<p>
@@ easter egg
Is an unexpected function hidden in a piece of software
@@ Unix-style operating system, 
This is an operating system which is somewhat 'unix-like'
an included Apple OSX, Linux, FreeBSD, etc
@@ Linux flavour, Linux Distribution
Linux comes in a number of different distributions, such as
Debian, Fedora, and Ubuntu to name only some of many. 
@@ regular expression
A search pattern
@@ recursive
Recursive means to do an action for some object, and then
repeat the same action for all objects which that object
contains. 
@@ concatenate
join together two or more things. For example join together
2 files with 'cat a.txt b.txt > c.txt'
@@ command line
@@ shell
A shell is a piece of software which allows you to communicate
with the computer by typing (rather cryptic) commands, and 
watching the output of those commands. This book focusses on
the 'Bash' shell, which is probably the most popular shell
on Unix, Linux and Apple OsX.
@@ terminal
The terminal is a piece of software which allows you to use
the 'shell'. Examples are 'Konsole' for KDE
@@ program
A program is really just a piece of software which does 
something on the computer.
@@ parse, parsing
parsing is the process of analysing and splitting into its
structure a text pattern. This is a very important task on all
computer systems, since a great deal of data processing involves
the parsing of one text pattern, and then the transformation of 
that pattern into another format.
@@ thread
@@ daemon
@@ process
These are similar but important concepts
<p>
<center><h3 class="page-title"></h3></center> printing jargon
@@ pretty-print 
This refers to the process of preparing some piece of text 
automatically in some way which is pleasing to read once printed
on a piece of paper or on a computer screen. The term is 
ambiguous since it can refer to the application of colour to 
a text file or to the changing and formatting of indentation and
paragraph width
<p>
<h3>Notes and ideas</h3><hr>
<p>
This section should contain references to things which I
think would be worthwhile investigating but which I havent
and may well never get round to doing.
<p>
grsync. gimp thresholds. sea horse for encrypt
<p>
vnc and vino for remote control
Tweetdeck, bluefish, dropbox, 
dialog, gdialog,
<p>
xxd - hexdumps
<p>
kompozer, 
<p>
<h3>Organisation</h3><hr>
<p>
<center><h3 class="page-title"></h3></center>
.. zim - note taker
.. tomboy - note taker
.. fzapper
..
<p>
<h3>Document</h3><hr>-NOTES:<br/>
<p>
# this section contains information about the document and
# will not normally be printed.
<p>
# a small icon image, filename relative to the "htdocs" folder
document-icon: <a href="/img/icon/linux-penguin.png"><em><code>/img/icon/linux-penguin.png</code></em></a>
#
document-image:<br/>
# what sort of document is this
document-type: book
# in what kind of state (good or bad) is this document 
document-quality: good
# when was this document last updated
last-revision: jan 2010
<p>
# a history of work on the document 
document-history:<br/>
@@ 2009
document begun
@@ jan 2010 revision
@@ 5 jan 2010
incorporated rewritten recipes from dsl.org
@@ 8 jan 2010 
got 4032 recipes from commandlinefu.com 
@@ early jan 2010
more detailed enscript description. This enscript chapter
has now (april 2010) been moved to the 'linux-documentation-book.txt' 
although a few enscript recipes remain.
@@ 24 jan 2010
added information about the 'halibut' and 'markdown' text
transformation systems.
@@ 25 jan 2010
removed a large quantity of unclassified recipes which had
been incorporated from 'commandlinefu.com'
@@ 26 march 2010
removed linux sound chapters into a separate book since it
was getting to big
@@ 27 march 2010
some general tidying up, classifying recipes. The task of
cleaning and verifying all the commandfu recipes is a 
big one
@@ 30 march 2010
added some zenity info
@@ 2 april 2010
continuing with the rather overwhelming task of attempting
to analyse all the recipes from commandlinefu.com and 
classify and weed them out. 
@@ 21 april 2010
some tidying up. wrote a vim command (:Bsi) to create screenshots of
oneline commands and save them in the 'image' subfolder. Also the
':Bci' command just creates and displays a test image without saving it
into the image folder. When building with the 'booktolatex.sed',
'booktolatex.cgi' scripts, we still have the problem that right aligned
images 'push' the recipes out of alignment.
@@ 6 may 2010
a little clean up of the 'introduction to the command line'. I
had the idea of writing a 'culture shock' section for Microsoft
users who are coming to Linux.
<p>
# work that needs to be done on this document
to-do:<br/>
A great deal of work needs to be carried out
@@ enscript
reorganise the enscript recipes which came from the 
linux cook-book.
<p>
#
# who wrote this
authors: mjb ~ <a href="http://bumble.sf.net" <a href="http://bumble.sf.net</a>">http://bumble.sf.net</a></a>
<p>
# a short description of the contents, possible used for doc lists
short-description: recipes for using linux from the command line
<p>
# a computer language used in the document (this helps latex pretty print)
code-language: bash
<p>
# the script which will be used to produce html (a webpage)
make-html: ./book-html.sh
<p>
# the script which will produce 'LaTeX' output (for printing, pdf etc)
make-latex:<br/>
<p>
# one liners which may be useful
# book-scripts:<br/>
<p>
<strong><em>section and subsection headings can be viewed with</em></strong>
  <pre class="codeline"> sed -n '/\.\{4\}$/s/^/  /;/^ *[[:upper:]]\{2\}[^[:lower:]]*$/p' book.txt</pre>
<p>
<h3>Command fu notes</h3><hr>
<p>
This section contains commands got from 'commandfu.com' 
and not yet classified or analysed.
<p>
<strong><em>Like top, but for files</em></strong>
  <pre class="codeline"> watch -d -n 2 'df; ls -FlAt;'</pre>
<p>
<strong><em>Quick case-insensitive partial filename search</em></strong>
  <pre class="codeline"> alias lg='ls --color=always | grep --color=always -i'</pre>
<p>
<strong><em>Watch the progress of 'dd'</em></strong>
  <pre class="codeline"> dd if=/dev/urandom of=file.img bs=4KB& pid=$!</pre>
<p>
<strong><em>Processor / memory bandwidthd? in GB/s</em></strong>
  <pre class="codeline"> dd if=/dev/zero of=/dev/null bs=1M count=32768</pre>
<p>
<strong><em>Share a terminal screen with others</em></strong>
  <pre class="codeline"> % screen -r someuser/</pre>
<p>
<strong><em>Show directories in the PATH, one per line</em></strong>
  <pre class="codeline"> ( IFS=:; for p in $PATH; do echo $p; done )</pre>
<p>
<strong><em>Tweak system files without invoking a root shell</em></strong>
  <pre class="codeline"> echo "Whatever you need" | sudo tee [-a] /etc/system-file.cfg</pre>
<p>
<strong><em>Look for IPv4 address in files.</em></strong>
  <pre class="codeline"> alias ip4grep "grep -E '([0-9]{1,3}\.){3}[0-9]{1,3}'"</pre>
<p>
<strong><em>fuman, an alternative to the 'man' command that shows commandlinefu tips</em></strong>
  <pre class="codeline"> fuman(){ lynx -width=$COLUMNS -nonumbers -dump "http://www.commandlinefu.com/commands/using/$1" | less; }</pre>
<p>
<strong><em>Reconnect to screen without disconnecting other sessions</em></strong>
  <pre class="codeline"> screen -xR</pre>
<p>
<strong><em>Generate a binary file with all ones (0xff) in it</em></strong>
  <pre class="codeline"> tr '\000' '\377' < /dev/zero | dd of=allones bs=1024 count=2k</pre>
<p>
<strong><em>Print stack trace of a core file without needing to enter gdb</em></strong>
  <pre class="codeline"> alias gdbbt="gdb -q -n -ex bt -batch"</pre>
<p>
<strong><em>Remove color codes (special characters) with sed</em></strong>
  <pre class="codeline"> sed -r "s/\x1B\[([0-9]{1,3}((;[0-9]{1,3})*)?)?[m|K]//g</pre>
<p>
<strong><em>calulate established tcp connection of local machine</em></strong>
  <pre class="codeline"> netstat -an|grep -ci "tcp.*established"</pre>
<p>
<strong><em>Using column to format a directory listing</em></strong>
  <pre class="codeline"> (printf "PERMISSIONS LINKS OWNER GROUP SIZE MONTH DAY HH:MM PROG-NAME\n" \ ; ls -l | sed 1d) | column -t</pre>
<p>
<strong><em>Port scan a range of hosts with Netcat.</em></strong>
  <pre class="codeline"> for i in {21..29}; do nc -v -n -z -w 1 192.168.0.$i 443; done</pre>
<p>
<strong><em>calulate established tcp connection of local machine</em></strong>
  <pre class="codeline"> netstat -an | awk '$1 ~ /[Tt][Cc][Pp]/ && $NF ~ /ESTABLISHED/{i++}END{print "Connected:\t", i}'</pre>
<p>
<strong><em>To get internet connection information .</em></strong>
  <pre class="codeline"> sudo /bin/netstat -tpee</pre>
<p>
<strong><em>Copy via tar pipe while preserving file permissions </em></strong>
  <pre class="codeline"> cp -pr olddirectory newdirectory</pre>
<p>
<strong><em>Show GCC-generated optimization commands when using the</em></strong>
  <pre class="codeline"> cc -march=native -E -v - </dev/null 2>&1 | grep cc1</pre>
<p>
<strong><em>Show in a web server, running in the port 80, how many ESTABLISHED</em></strong>
  <pre class="codeline"> netstat -ant | grep :80 | grep ESTABLISHED | awk '{print $5}' | awk -F: '{print $1}' | sort | uniq -c | sort -n</pre>
<p>
<strong><em>Count all conections estabilished on gateway</em></strong>
  <pre class="codeline"> cat /proc/net/ip_conntrack | grep ESTABLISHED | grep -c -v ^#</pre>
<p>
<strong><em>List programs with open ports and connections</em></strong>
  <pre class="codeline"> netstat -ntauple</pre>
<p>
<strong><em>Alternative size (human readable) of files and directories</em></strong>
  <pre class="codeline"> du -ms * .[^.]*| sort -nk1</pre>
<p>
<strong><em>Uniformly correct filenames in a directory</em></strong>
  <pre class="codeline"> for i in *;do mv "$i" "$(echo $i | sed s/PROBLEM/FIX/g)";done</pre>
<p>
<strong><em>Get a shell with a not available account</em></strong>
  <pre class="codeline"> su - <user> -s /bin/sh -c "/bin/sh"</pre>
<p>
<strong><em>Size(k) of directories(Biggest first)</em></strong>
  <pre class="codeline"> find . -depth -type d -exec du -s {} \; | sort -k1nr</pre>
<p>
<strong><em>purge installed but unused linux headers, image, or modules</em></strong>
  <pre class="codeline"> dpkg -l 'linux-*' | sed '/^ii/!d;/'"$(uname -r | sed "s/\(.*\)-\([^0-9]\+\)/\1/")"'/d;s/^[^ ]* [^ ]* \([^ ]*\).*/\1/;/[0-9]/!d' | xargs sudo apt-get -y purge</pre>
<p>
<strong><em>Sort lines using the Xth characted as the start of the sort string</em></strong>
  <pre class="codeline"> sort -k1.x</pre>
<p>
<strong><em>Poor man's nmap for a class C network from rfc1918</em></strong>
  <pre class="codeline"> ( nw=192.168.0 ; h=1; while [ $h -lt 255 ] ; do ( ping -c2 -i 0.2 -W 0.5 -n $nw.$h & ); h=$[ $h + 1 ] ; done ) | awk '/^64 bytes.*/ { gsub( ":","" ); print $4 }' | sort -u</pre>
<p>
<strong><em>Random numbers with Ruby</em></strong>
  <pre class="codeline"> ruby -e "puts (1..20).map {rand(10 ** 10).to_s.rjust(10,'0')}"</pre>
<p>
<strong><em>find listening ports by pid</em></strong>
  <pre class="codeline"> lsof -nP +p 24073 | grep -i listen | awk '{print $1,$2,$7,$8,$9}'</pre>
<p>
<strong><em>Lock your KDE4 remotely (via regular KDE lock)</em></strong>
  <pre class="codeline"> DISPLAY=:0 /usr/lib/kde4/libexec/krunner_lock --forcelock >/dev/null 2>&1 &</pre>
<p>
<strong><em>List last opened tabs in firefox browser</em></strong>
  <pre class="codeline"> F="$HOME/.moz*/fire*/*/session*.js" ; grep -Go 'entries:\[[^]]*' $F | cut -d[ -f2 | while read A ; do echo $A | sed s/url:/\n/g | tail -1 | cut -d\" -f2; done</pre>
<p>
<strong><em>Break lines after, for example 78 characters, but don't break</em></strong>
  <pre class="codeline"> fold -w 78 -s file-to-wrap</pre>
<p>
<strong><em>Function that counts recursively number of lines of all files in</em></strong>
  <pre class="codeline"> count() { find $@ -type f -exec cat {} + | wc -l; }</pre>
<p>
<strong><em>Go to the next sibling directory in alphabetical order, version 2</em></strong>
  <pre class="codeline"> cd ../"$(ls -F ..|grep '/'|grep -A1 `basename $PWD`|tail -n 1)"</pre>
<p>
<strong><em>find largest file in /var</em></strong>
  <pre class="codeline"> find /var -mount -ls -xdev | /usr/bin/sort -nr +6 | more</pre>
<p>
<strong><em>Show established network connections</em></strong>
  <pre class="codeline"> lsof -i | grep -i estab</pre>
<p>
<strong><em>Start screen with name and run command</em></strong>
  <pre class="codeline"> screen -dmS "name_me" echo "hi"</pre>
<p>
<strong><em>Replace Solaris vmstat numbers with human readable format</em></strong>
  <pre class="codeline"> vmstat 1 10 | /usr/xpg4/bin/awk -f ph-vmstat.awk</pre>
<p>
<strong><em>Browse shared folder when you're the only Linux user</em></strong>
  <pre class="codeline"> smbclient -U userbob //10.1.1.75/Shared</pre>
<p>
<strong><em>connect to all screen instances running</em></strong>
  <pre class="codeline"> screen -ls | grep pts | gawk '{ split($1, x, "."); print x[1] }' | while read i; do gnome-terminal -e screen\ -dx\ $i; done</pre>
<p>
<strong><em>Use ImageMagick to get an image's properties</em></strong>
  <pre class="codeline"> identify -ping imageName.png</pre>
<p>
<strong><em>Show apps that use internet connection at the moment.</em></strong>
  <pre class="codeline"> lsof -P -i -n | cut -f 1 -d " "| uniq | tail -n +2</pre>
<p>
<strong><em>send echo to socket network</em></strong>
  <pre class="codeline"> echo "foo" > /dev/tcp/192.168.1.2/25</pre>
<p>
<strong><em>Colorized grep in less</em></strong>
  <pre class="codeline"> grep --color=always | less -R</pre>
<p>
<strong><em>Search recursively to find a word or phrase in certain file</em></strong>
  <pre class="codeline"> find . -name "*.[ch]" -exec grep -i -H "search pharse" {} \;</pre>
<p>
<strong><em>Recursively change permissions on files, leave directories alone.</em></strong>
  <pre class="codeline"> find ./ -type f -exec chmod 644 {} \;</pre>
<p>
<strong><em>ensure your ssh tunnel will always be up (add in crontab)</em></strong>
  <pre class="codeline"> [[ $(COLUMNS=200 ps faux | awk '/grep/ {next} /ssh -N -R 4444/ {i++} END {print i}') ]] || nohup ssh -N -R 4444:localhost:22 user@relay &</pre>
<p>
<strong><em>Download from Rapidshare Premium using wget - Part 1</em></strong>
  <pre class="codeline"> wget --save-cookies ~/.cookies/rapidshare --post-data "login=USERNAME&password=PASSWORD" -O - https://ssl.rapidshare.com/cgi-bin/premiumzone.cgi > /dev/null</pre>
<p>
<strong><em>Recursively lists all files in the current directory, except the</em></strong>
  <pre class="codeline"> find . -wholename './.snapshot' -prune -o -print</pre>
<p>
<strong><em>Monitor Linux/MD RAID Rebuild</em></strong>
  <pre class="codeline"> watch -n 5 -d cat /proc/mdstat</pre>
<p>
<strong><em>memcache affinity: queries local memcached for stats, calculates</em></strong>
  <pre class="codeline"> echo -en "stats\r\n" "quit\r\n" | nc localhost 11211 | tr -s [:cntrl:] " "| cut -f42,48 -d" " | sed "s/\([0-9]*\)\s\([0-9]*\)/ \2\/\1*100/" | bc -l</pre>
<p>
<strong><em>nmap IP block and autogenerate comprehensive Nagios service</em></strong>
  <pre class="codeline"> nmap -sS -O -oX /tmp/nmap.xml 10.1.1.0/24 -v -v && perl nmap2nagios.pl -v -r /tmp/10net.xml -o /etc/nagios/10net.cfg</pre>
<p>
<strong><em>Convert df command to posix; uber GREPable</em></strong>
  <pre class="codeline"> df -P</pre>
<p>
<strong><em>List top 20 IP from which TCP connection is in SYN_RECV state</em></strong>
  <pre class="codeline"> netstat -pant 2> /dev/null | grep SYN_ | awk '{print $5;}' | cut -d: -f1 | sort | uniq -c | sort -n | tail -20</pre>
<p>
<strong><em>Show what a given user has open using lsof</em></strong>
  <pre class="codeline"> lsof -u www-data</pre>
<p>
<strong><em>show all programs connected or listening on a network port</em></strong>
  <pre class="codeline"> alias nsl 'netstat -f inet | grep -v CLOSE_WAIT | cut -c-6,21-94 | tail +2'</pre>
<p>
<strong><em>Show number of NIC's, ports per nic and PCI address</em></strong>
  <pre class="codeline"> lspci | grep Ether | awk '{ VAR=$1; split(VAR,ARR,"."); count[ARR[1]]++; LINE=$0; split(LINE,LINEARR,":"); LINECOUNT[ARR[1]]=LINEARR[3]; } END { for(i in count) { printf("PCI address: %s\nPorts: %d\nCard Type: %s\n", i, count[i], LINECOUNT[i]) } }'</pre>
<p>
<strong><em>locate bin, src, and man file for a command</em></strong>
  <pre class="codeline"> whereis somecommand</pre>
<p>
<strong><em>Display all readline bindings that use CTRL</em></strong>
  <pre class="codeline"> bind -p | grep -F "\C"</pre>
<p>
<strong><em>Synchronise a file from a remote server</em></strong>
  <pre class="codeline"> rsync -av -e ssh user@host:/path/to/file.txt .</pre>
<p>
<strong><em>Is it a terminal?</em></strong>
  <pre class="codeline"> isatty(){ test -t $1; }</pre>
<p>
<strong><em>create disk copy over the net without temp files</em></strong>
  <pre class="codeline"> SOURCE: dd if=/dev/sda bs=16065b | netcat ip-target 1234 TARGET: netcat -l -p 1234 | dd of=/dev/mapper/laptop bs=16065b STATS on target: watch -n60 -- kill -USR1 $(pgrep dd)</pre>
<p>
<strong><em>Write a listing of all directories and files on the computer to an archive</em></strong>
  <pre class="codeline"> sudo ls -RFal / | gzip > all_files_list.txt.gz</pre>
<p>
<strong><em>Collect a lot of icons from /usr/share/icons (may overwrite some,</em></strong>
  <pre class="codeline"> mkdir myicons && find /usr/share/icons/ -type f | xargs cp -t myicons</pre>
<p>
<strong><em>copy ACL of one file to another using getfacl and setfacl</em></strong>
  <pre class="codeline"> getfacl <file-with-acl> | setfacl -f - <file-with-no-acl></pre>
<p>
<strong><em>determine if tcp port is open</em></strong>
  <pre class="codeline"> nmap -p 80 hostname</pre>
<p>
<strong><em>determine if tcp port is open</em></strong>
  <pre class="codeline"> if (nc -zw2 www.example.com 80); then echo open; fi</pre>
<p>
<strong><em>Strace all signals processes based on a name ( The processes</em></strong>
  <pre class="codeline"> straceprocessname(){ x=( $(pgrep "$@") ); [[ ${x[@]} ]] || return 1; strace -vf ${x[@]/#/-p }; }</pre>
<p>
<strong><em>last.fm rss parser</em></strong>
  <pre class="codeline"> awk '/<link>/{gsub(/.*<link>|<\/link>.*/,"");print "<li><a href=\042"$0"\042> "t"</a>" } /<title>/{gsub(/.*<title>|<\/title>.*/,"");t=$0 }' file</pre>
<p>
<strong><em>last.fm rss parser</em></strong>
  <pre class="codeline"> egrep "<link>|<title>" recenttracks.rss | awk 'ORS=NR%2?" ":"\n"' | awk -F "" '{print $2, $1}' | sed -e 's/\/\\/\">/' -e 's/\//' -e 's/$/\<\/a\>\<\/li\>/g' -e '1,1d' -e 's/^[ \t]*//'

Mount a partition from dd disk image

 mount -o loop,offset=$((512*x)) /path/to/dd/image /mount/path

Sorted list of established destination connections

 netstat | awk '/EST/{print $5}' | sort

List all symbolic links in current directory

 \ls -1 | xargs -l readlink

remove files and directories with acces time older than a given

 touch -t "YYYYMMDDhhmm.ss" dummy ; find . -anewer dummy

Show sorted list of files with sizes more than 1MB in the current

 du | sort -nr | cut -f2- | xargs du -hs

exit if another instance is running

 if [ `fuser $0|wc -w` -gt "1" ];then exit; fi

Find files recursively that were updated in the last hour ignoring

 find . -mmin -60 -not -path "*svn*" -print|more

Clean your broken terminal

 reset

umount all nfs mounts on machine

 mount | grep : | awk {'print $3'} | xargs sudo umount -v

Get your external IP address with a random commandlinefu.com

 IFS=$'\n';cl=($(curl -s http://www.commandlinefu.com/commands/matching/external/ZXh0ZXJuYWw=/so rt-by-votes/plaintext|sed -n '/^# Get your external IP address$/{n;p}'));c=${cl[$(( $RANDOM % ${#cl[@]} ))]};eval $c;echo "Command used: $c"

Counts number of lines (in source code excluding comments)

 find . -name '*.java' | xargs -L 1 cpp -fpreprocessed | grep . | wc -l

Empty a file

 truncate -s0 file

grep -v with multiple patterns.

 grep test somefile | grep -v -e error -e critical -e warning

Set creation timestamp of a file to the creation timestamp of

 touch -r "$FILE1" "$FILE2"

Commit command to history file immedeately after execution

 PROMPT_COMMAND="history -a"

Make directories for and mount all iso files in a folder

 for file in *.iso; do mkdir `basename $file | awk -F. '{print $1}'`; sudo mount -t iso9660 -o loop $file `basename $file | awk -F. '{print $1}'`; done

disk space email alert

 [ $(df / | perl -nle '/([0-9]+)%/ && print $1') -gt 90 ] && df -hP | mutt -s "Disk Space Alert -- $(hostname)" admin@example.com

Stop long commands wrapping around and over-writing itself in the

 shopt -s checkwinsize

download all the presentations from UTOSC2009

 b="http://2009.utosc.com"; for p in $( curl -s $b/presentation/schedule/ | grep /presentation/[0-9]*/ | cut -d"\"" -f2 ); do f=$(curl -s $b$p | grep "/static/slides/" | cut -d"\"" -f4); if [ -n "$f" ]; then echo $b$f; curl -O $b$f; fi done

Functions to display, save and restore $IFS

 ifs () { echo -n "${IFS}"|hexdump -e '"" 10/1 "'\''%_c'\''\t" "\n"' -e '"" 10/1 "0x%02x\t" "\n\n"'|sed "s/''\|\t0x[^0-9]//g; $,/^$/d"

Schedule Nice Background Commands That Won't Die on Logout -

 ( trap '' 1; ( nice -n 19 sleep 2h && command rm -v -rf /garbage/ &>/dev/null && trap 1 ) & )

Go get those photos from a Picasa album

 wget 'link of a Picasa WebAlbum' -O - |perl -e'while(<>){while(s/"media":{"content":\[{"url":"(.+?\.JPG)//){print "$1\n"}}' |wget -w1 -i -

32 bits or 64 bits?

 sudo lshw -C cpu|grep width

Show current pathname in title of terminal

 export PROMPT_COMMAND='echo -ne "\033]0;${PWD/#$HOME/~}\007";'

print all except first collumn

 cut -f 2- -d " "

Search for a string inside all files in the current directory

 find . -type f -exec grep -i  \;

Find and display most recent files using find and perl

 find $HOME -type f -print0 | perl -0 -wn -e '@f=<>; foreach $file (@f){ (@el)=(stat($file)); push @el, $file; push @files,[ @el ];} @o=sort{$a->[9]<=>$b->[9]} @files; for $i (0..$#o){print scalar localtime($o[$i][9]), "\t$o[$i][-1]\n";}'|tail

create SQL-statements from textfile with awk

 $ awk '{printf "select * from table where id = %c%s%c;\n",39,$1,39; }' inputfile.txt

Send a local file via email

 cat filename | mail -s "Email subject" user@example.com

Track X Window events in chosen window

 xev -id `xwininfo | grep 'Window id' | awk '{print $4}'`

Test file system performance

 bonnie++ -n 0 -u 0 -r  -s <2 x physical ram> -f -b -d 

(Git) Revert files with changed mode, not content

 git diff --numstat | awk '{if ($1 == "0" && $1 == "0") print $3}' | xargs git checkout HEAD

Instant mirror from your laptop + webcam

 mplayer tv:// -vf mirror

Instant mirror from your laptop + webcam

 cvlc v4l2:// :vout-filter=transform :transform-type=vflip :v4l2-width=320 :v4l2-height=240 -f &

Shorten any Url using bit.ly API, using your API Key which

 curl "http://api.bit.ly/shorten?version=2.0.1&longUrl=&login=&apiKey="

Archive all SVN repositories in platform indepenent form

 budir=/tmp/bu.$$;for name in repMainPath/*/format;do dir=${name%/format};bufil=dumpPath/${dir##*/};svnadmin hotcopy --clean-logs $dir $budir;svnadmin dump --delta $budir>$bufil;rm -rf $budir;done

preprocess code to be posted in comments on this site

 sed 's/^/$ /' "$script" | xclip

send echo to socket network

 echo foo | netcat 192.168.1.2 25

Display the standard deviation of a column of numbers with awk

 awk '{delta = $1 - avg; avg += delta / NR; mean2 += delta * ($1 - avg); } END { print sqrt(mean2 / NR); }'

Compute running average for a column of numbers

 awk '{avg += ($1 - avg) / NR;} END { print avg; }'

Route outbound SMTP connections through a addtional IP address

 iptables -t nat -A POSTROUTING -p tcp --dport 25 -j SNAT --to-source IP_TO_ROUTE_THROUGH

Check if x509 certificate file and rsa private key match

 diff <(openssl x509 -noout -modulus -in server.crt ) <( openssl rsa -noout -modulus -in server.key )

clone ide hard disk

 sudo dd if=/dev/hda1 of=/dev/hdb2

Update Ping.fm status

 curl -d api_key="$api_key" -d user_app_key="$user_app_key -d body="$body" -d post_method="default" http://api.ping.fm/v1/user.post

Send a signed and encrypted email from the command line

 echo "SECRET MESSAGE" | gpg -e --armor -s | sendmail USER@DOMAIN.COM

fetch all revisions of a specific file in an SVN repository

 svn log fileName | sed -ne "/^r\([0-9][0-9]*\).*/{;s//\1/;s/.*/svn cat fileName@& > fileName.r&/p;}" | sh -s

Ultimate current directory usage command

 du -a --max-depth=1 | sort -n | cut -d/ -f2 | sed '$d' | while read i; do if [ -f $i ]; then du -h "$i"; else echo "$(du -h --max-depth=0 "$i")/"; fi; done

Reinstall Grub

 sudo grub-install --recheck /dev/sda1

xxhere

dstat- this command is powerful one to monitor system activity .

 dstat -afv

Search for an active process without catching the search-process

 ps -ef | awk '/process-name/ && !/awk/ {print}'

Replace Every occurrence of a word in a file

 perl -p -i -e 's/this/that/g' filename

Get a list of all your VirtualBox virtual machines by name and

 VBoxManage list vms

run a VirtualBox virtual machine without a gui

 VBoxHeadless -s 

Start a command on only one CPU core

 taskset -c 0 your_command

Resume an emerge, and keep all object files that are already

 FEATURES=keepwork emerge --resume

Synchronize both your system clock and hardware clock and

 ntpdate pool.ntp.org && hwclock --systohc && hwclock --adjust

'Fix' a typescript file created by the 'script' program to remove

 cat typescript | perl -pe 's/\e([^\[\]]|\[.*?[a-zA-Z]|\].*?\a)//g' | col -b > typescript-processed

Replace all tabs with spaces in an application

 grep -PL "\t" -r . | grep -v ".svn" | xargs sed -i 's/\t/ /g'

Getting Screen's Copy Buffer Into X's Copy Buffer (on Linux)

 Type "c-a b" in gnu screen after updating your .screenrc (See Description below).

using scanner device from command line

 scanimage -d mustek_usb --resolution 100 --mode Color > image.pnm

Make a directory named with the current date

 mkdir `date --iso`

Duplicating service runlevel configurations from one server to

 chkconfig --list | fgrep :on | sed -e 's/\(^.*\)*0:off/\1:/g' -e 's/\(.\):on/\1/g' -e 's/.:off//g' | tr -d [:blank:] | awk -F: '{print$2,$1}' | ssh host 'cat > foo'

Validate and pretty-print JSON expressions.

 echo '{"json":"obj"}' | python -m simplejson.tool

Find and delete oldest file of specific types in directory tree

 find / \( -name "*.log" -o -name "*.mylogs" \) -exec ls -lrt {} \; | sort -k6,8 | head -n1 | cut -d" " -f8- | tr -d '\n' | xargs -0 rm

Remove all backup files in my home directory

 find ~user/ -name "*~" -exec rm {} \;

Get an IP address out of fail2ban jail

 iptables -D fail2ban-SSH -s  -j DROP

Safe Delete

 shred -n33 -zx file; rm file

Search for files older than 30 days in a directory and list only

 find /var/www/html/ -type f -mtime +30 -exec basename {} \;

Working random fact generator

 lynx -dump randomfunfacts.com | grep -A 3 U | sed 1D

Mac OS X: Change Color of the ls Command

 export LSCOLORS=gxfxcxdxbxegedabagacad

split source code to page with numbers

 pr -l 40 bitree.c > printcode; split -40 printcode -d page_

Pick a random image from a directory (and subdirectories) every

 while true; do xfconf-query -c xfce4-desktop -p /backdrop/screen0/monitor0/image-path -s "$(find  -type f -iregex '.*\.\(bmp\|gif\|jpg\|png\)$' | sort -R | head -1)"; sleep 30m; done

Change wallpaper for xfce4 >= 4.6.0

 xfconf-query -c xfce4-desktop -p /backdrop/screen0/monitor0/image-path -s 

Prevent non-root users from logging in

 touch /etc/nologin

Display time of accounts connection on a system

 ac -p

Change Gnome wallpaper

 gconftool-2 -t string -s /desktop/gnome/background/picture_filename 

Clone current directory into /destination verbosely

 find . | cpio -pumdv /destination

Mount a disk image (dmg) file in Mac OSX

 hdiutil attach somefile.dmg

Connect to all running screen instances

 for i in `screen -ls | perl -ne'if(/^\s+\d+\.([^\s]+)/){print $1, " "}'`; do gnome-terminal -e "screen -x $i"; done

compare two Microsoft Word documents

 meld <(antiword microsoft_word_a.doc) <(antiword microsoft_word_b.doc)

Mount important virtual system directories under chroot'ed

 for i in sys dev proc; do sudo mount --bind /$i /mnt/xxx/$i; done

Set Time Zone in Ubuntu

 sudo dpkg-reconfigure tzdata

Summarise the size of all files matching a simple regex

 find /path/to/my/files/ -type f -name "*txt*" | xargs du -k | awk 'BEGIN{x=0}{x=x+$1}END{print x}'

Root shell

 sudo -i

Remove all hidden files in a directory

 rm -r .??*

Prints any IP out of a file

 perl -ne 'while (/([0-9]+\.){3}[0-9]+/g) {print "$&\n"};' file.txt

Dump HTTP header using wget

 wget --server-response --spider http://www.example.com/

backup your entire hosted website using cPanel backup interface

 wget --http-user=YourUsername --http-password=YourPassword http://YourWebsiteUrl:2082/getbackup/backup-YourWebsiteUrl-`date +"%-m-%d-%Y"`.tar.gz

Copy input sent to a command to stderr

 rev <<< 'lorem ipsum' | tee /dev/stderr | rev

create a simple version of ls with extended output

 alias l='ls -CFlash'

Make info pages much less painful

 pinfo date

print battery , thermal , and cooling info

 acpi -tc

View a man page on a nice interface

 yelp man:foo

Emptying a text file in one shot

 :1,$d

find an unused unprivileged TCP port

 netstat -atn | awk ' /tcp/ {printf("%s\n",substr($4,index($4,":")+1,length($4) )) }' | sed -e "s/://g" | sort -rnu | awk '{array [$1] = $1} END {i=32768; again=1; while (again == 1) {if (array[i] == i) {i=i+1} else {print i; again=0}}}'

Change files case, without modify directories, recursively

 find ./ -name '*.JPG' -type f -execdir rename -f 'y/A-Z/a-z/' {} \+

force unsupported i386 commands to work on amd64

 setarch i386 [command [args]]

Combine all .mpeg files in current directory into one big one.

 cat *.mpg > all.mpg

Output system statistics every 5 seconds with timestamp

 while [ 1 ]; do echo -n "`date +%F_%T`" ; vmstat 1 2 | tail -1 ; sleep 4; done

Tail a log file with long lines truncated

 tail -f logfile.log | cut -b 1-80

Spell check the text in clipboard (paste the corrected clipboard

 xclip -o > /tmp/spell.tmp; aspell check /tmp/spell.tmp ; cat /tmp/spell.tmp | xclip

Display information sent by browser

 nc -l 8000

Determine the version of a specific package with RPM

 rpm -q --qf "%{VERSION}\n" redhat-release

add a gpg key to aptitute package manager in a ubuntu system

 wget -q http://xyz.gpg -O- | sudo apt-key add -

Quick calculator at the terminal

 echo "$math_expr" | bc -l

Remove embedded fonts from a pdf.

 gs -sDEVICE=pswrite -sOutputFile=- -q -dNOPAUSE With-Fonts.pdf -c quit | ps2pdf - > No-Fonts.pdf

Quick plotting of a function

 seq 0 0.1 20 | awk '{print $1, cos(0.5*$1)*sin(5*$1)}' | graph -T X

Add thousand separator with sed, in a file or within pipe

 sed -e :a -e 's/\(.*[0-9]\)\([0-9]\{3\}\)/\1,\2/;ta' filename

Downsample mp3s to 128K

 for f in *.mp3 ; do lame --mp3input -b 128 "$f" ./resamp/"$f" ; done

start vim in diff mode

 vimdiff file{1,2}

Time Synchronisation with NTP

 ntpdate ntp.ubuntu.com pool.ntp.org

Lists architecture of installed RPMs

 rpm -qa --queryformat "%{NAME} %{ARCH}\n"

Display usb power mode on all devices

 for i in `find /sys/devices/*/*/usb* -name level` ; do echo -n "$i: " ; cat $i ; done

Display a list of RPMs installed on a particular date

 rpm -qa --queryformat '%{installtime} \"%{vendor}\" %{name}-%{version}-%{release} %{installtime:date}\n' | grep "Thu 05 Mar"

create a backup for all directories from current dir

 find -maxdepth 1 -type d -print0 | xargs -0 -I {} tar -cvzf {}.tar.gz {}

Display 6 largest installed RPMs sorted by size (descending)

 rpm -qa --qf '%{SIZE} %{NAME}\n' | sort -nr | nl | head -6 # six largest RPMs

Convert embedded spaces in filenames to "_" (underscore)

 ls -1 | grep " " | awk '{printf("mv \"%s\" ",$0); gsub(/ /,"_",$0); printf("%s\n",$0)}' | sh # rename filenames: spaces to "_"

Display kernel profile of currently executing functions in

 lockstat -I -i 977 -s 30 -h sleep 1 > /tmp/profile.out

List all the files that have been deleted while they were still

 lsof | egrep "^COMMAND|deleted"

Search through all installed packages names (on RPM systems)

 rpm -qa \*code\*

Binary injection

 echo -n $HEXBYTES | xxd -r -p | dd of=$FILE seek=$((0x$OFFSET)) bs=1 conv=notrunc

Forwards connections to your port 2000 to the port 22 of a remote

 ssh -NL 2000:remotehost:22 remotehost

Change the extension of a filename by using rename to convert

 rename .JPG .jpg *.JPG

Repeatedly send a string to stdout-- useful for going through

 yes "text" | annoying_installer_program # "text" defaults to the letter y

Remove annoying OS X DS_Store folders

 find . -name .DS_Store -exec rm {} \;

Copy the text from the 3rd line to the 9th line into a new file

 :3,9w new_file

cat stdout of multiple commands

 cat <( command1 arg arg ) <( command2 arg ) ...

Find commets in jpg files.

 find / -name "*.jpg" -print -exec rdjpgcom '{}' ';'

View a file with less, starting at the end of the file

 less +G 

vi case insensitive search

 :set ic

"hidden" remote shell

 ssh -T user@host /bin/bash -i

Find all files with root SUID or SGID executables

 sudo find / -type f \( -perm /4000 -a -user root \) -ls -o \( -perm /2000 -a -group root \) -ls

Delete empty directories recursively

 find  -depth -type d -empty -exec rmdir -v {} \;

Remove several files with ease

 rm file{1..10}

Prints new content of files

 tail -f file1 (file2 .. fileN)

Extract icons from windows exe/dll

 wrestool -x --output . -t14 /path/to/your-file.exe

Know which version dpkg/apt considers more recent

 dpkg --compare-versions 1.0-2ubuntu5 lt 1.1-1~raphink3 && echo y || echo n

Replace "space" char with "dot" char in current directory file

 ls -1 | while read a; do mv "$a" `echo $a | sed -e 's/\ /\./g'`; done

sort ugly text

 sort -bdf

Recursive Search and Replace

 perl -pi -e's///g' `grep -Rl  //*`

remove all snapshots from all virtual machines in vmware esx

 time vmware-cmd -l | while read x; do printf "$x"; vmware-cmd "$x" removesnapshots; done

Print the last modified file

 ls -t1 | head -n1

Symlink all files from a base directory to a target directory

 for f in $(ls -d /base/*); do ln -s $f /target; done && ls -al /target

Create passwords and store safely with gpg

 tr -dc "a-zA-Z0-9-_\$\?" < /dev/urandom | head -c 10 | gpg -e -r medha@nerdish.de > password.gpg

Type strait into a file from the terminal.

 cat /dev/tty > FILE

convert a pdf to jpeg

 sips -s format jpeg Bild.pdf --out Bild.jpg

Cleanly manage tempfiles in scripts

 TMPROOT=/tmp; TMPDIR=$(mktemp -d $TMPROOT/somedir.XXXXXX); TMPFILE=$(mktemp $TMPROOT/somefile.XXXXXX); trap "rm -rf $TMPDIR $TMPFILE; exit" INT TERM EXIT; some treatment using $TMPDIR and $TMPFILE; exit 0

1:1 copy of a volume

 find / -xdev -print | cpio -pdmuv /mnt/mydisk

sort through source to find most common authors

 find . -type f -name "*.java" -print0 | xargs -0 -n 1 svn blame | sed -n 's/^[^a-z]*\([a-z]*\).*$/\1/p' | sort | uniq -c | sort -n

Drop or block attackers IP with null routes

 sudo route add xxx.xxx.xxx.xxx gw 127.0.0.1 lo

split a string (3)

 OLD_IFS="$IFS"; IFS=: ARRAY=($PATH); echo ${ARRAY[2]}; IFS="$OLD_IFS"

get your terminal back after it's been clobbered

 reset

Set file access control lists

 setfacl -m u:john:r-- myfile

Compress logs older than 7 days

 find /path/to/files -type f -mtime +7 | grep -v \.gz | xargs gzip

restore the contents of a deleted file for which a descriptor is

 N="filepath" ; P=/proc/$(lsof +L1 | grep "$N" | awk '{print $2}')/fd ; ls -l $P | sed -rn "/$N/s/.*([0-9]+) ->.*/\1/p" | xargs -I_ cat $P/_ > "$N"

Quickest way to sort/display # of occurences

 "some line input" | sort | uniq -c | sort -nr

Copy a file from a remote server to your local box using

 rsync -Pz user@remotehost:/path/file.dat .

Show current iptables rules, with line numbers

 iptables -nL -v --line-numbers

Find out the last times your system was rebooted (for the duration

 last reboot

Follow a new friend on twitter

 curl -u USERNAME:PASSWORD -d "" http://twitter.com/friendships/create/NAMEOFNEWFRIEND.xml?follow=true

checking space availabe on all /proc/mounts points (using Nagios

 check_disk -w 15% -c 10% $(for x in $(cat /proc/mounts |awk '{print $2}')\; do echo -n " -p $x "\; done)

doing some floating point math

 echo "8000000/(20*6*86400)" | bc -l

List all active access_logs for currently running Apache or

 lsof -p $(netstat -ltpn|awk '$4 ~ /:80$/ {print substr($7,1,index($7,"/")-1)}')| awk '$9 ~ /access.log$/ {print $9| "sort -u"}'

Empty the linux buffer cache

 sync && echo 3 > /proc/sys/vm/drop_caches

Delete .svn directories and content recursively

 `find . -iname ".svn" -type d | sed -e "s/^/rm -rfv /g"`

Clean up after a poorly-formed tar file

 tar ztf tar-lacking-subdirectory.tar.gz | xargs rm

Simple complete system backup excluding files or directories

 tar zcpf backup.tgz --exclude=/proc --exclude=backup.tgz /

Comma insertions

 perl -pe '$_=reverse;s/\d{3}(?=\d)(?!.*?\.)/$&,/g;$_=reverse'

Double your disk read performance in a single command

 blockdev --setra 1024 /dev/sdb

Check ps output to see if file is running, if not start it

 ps -C thisdaemon || { thisdaemon & }

Generate a Random Password

 dd if=/dev/urandom bs=1 count=32 2>/dev/null | base64 -w 0 | rev | cut -b 2- | rev

decode base64

 echo -n $@ | openssl base64

Make a high definition VNC

 vncserver -nohttpd -name hidef-server -depth 24 -geometry 1440x900

Finding the number of cpu's

 grep -c -e '^cpu[0-9]\+' /proc/stat

Search inside a folder of jar/zip files

 find . -name "*.jar" | xargs -tn1 jar tvf | grep --color "SearchTerm"

Netstat Connection Check

 netstat -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n | tail

Email a file to yourself

 uuencode $file $file | /usr/bin/mailx -s "$file" ${USER}

shut of the screen.

 xset dpms force standby

Find files that have been modified on your system in the past 60

 sudo find / -mmin 60 -type f

Intercept, monitor and manipulate a TCP connection.

 mkfifo /tmp/fifo; cat /tmp/fifo | nc -l -p 1234 | tee -a to.log | nc machine port | tee -a from.log > /tmp/fifo

Use tee + process substitution to split STDOUT to multiple

 some_command | tee >(command1) >(command2) >(command3) ... | command4

Copy your ssh public key to a server from a machine that doesn't

 cat ~/.ssh/id_rsa.pub | ssh user@machine "mkdir ~/.ssh; cat >> ~/.ssh/authorized_keys"

Execute a command with a timeout

 timeout 10 sleep 11

Search for a string inside all files in the current directory

 grep -RnisI  *

redirect stdout and stderr each to separate files and print both

 (some_command 2>&1 1>&3 | tee /path/to/errorlog ) 3>&1 1>&2 | tee /path/to/stdoutlog

Find files that were modified by a given command

 touch /tmp/file ; $EXECUTECOMMAND ; find /path -newer /tmp/file

Have an ssh session open forever

 autossh -M50000 -t server.example.com 'screen -raAd mysession'

throttle bandwidth with cstream

 tar -cj /backup | cstream -t 777k | ssh host 'tar -xj -C /backup'

replace spaces in filenames with underscores

 rename 'y/ /_/' *

Save an HTML page, and covert it to a .pdf file

 wget $URL | htmldoc --webpage -f "$URL".pdf - ; xpdf "$URL".pdf &

Compare two directory trees.

 diff <(cd dir1 && find | sort) <(cd dir2 && find | sort)

Resume scp of a big file

 rsync --partial --progress --rsh=ssh $file_source $user@$host:$destination_file

Install a Firefox add-on/theme to all users

 sudo firefox -install-global-extension /path/to/add-on

iso-8859-1 to utf-8 safe recursive rename

 detox -r -s utf_8 /path/to/old/win/files/dir

recursive search and replace old with new string, inside files

 $ grep -rl oldstring . |xargs sed -i -e 's/oldstring/newstring/'

Terminate a frozen SSH-session

 RETURN~.

Get Cisco network information

 tcpdump -nn -v -i eth0 -s 1500 -c 1 'ether[20:2] == 0x2000'

Copy stdin to your X11 buffer

 ssh user@host cat /path/to/some/file | xclip

Check if running in an X session

 if [ ! -z "${DISPLAY}" ]; then someXcmd ; fi

delete unversioned files in a checkout from svn

 svn st | grep "^\?" | awk "{print \$2}" | xargs rm -rf

Rsync between two servers

 rsync -zav --progress original_files_directory/ root@host(IP):/path/to/destination/

replace XX by YY in the the current directory and cd to it. ( in

 cd XX YY

Show a line when a "column" matchs

 awk '{ FS = OFS = "#" } { if ($9==1234) print }' filename*.log > bigfile.log

Split and join with split and cat.

 split -b 1k file ; cat x* > file

run command on a group of nodes in parallel

 echo -n m{1..5}.cluster.net | xargs -d' ' -n1 -P5 -I{} ssh {} 'uptime'

Sort IP addresses

 sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4 /file/of/ip/addresses

Check executable shared library usage

 ldd 

fix broken permissions

 find /path -type d -perm 777 -exec chmod 755 {} \;

Use color grep by default

 alias grep 'gnu grep -i --color=auto'

finding cr-lf files aka dos files with ^M characters

 find $(pwd) -type f -exec grep -l "$(echo "\r")" {} \;

Make a thumbnail image of first page of a PDF.

 convert -resize 200 -sharpen 40 some_file.pdf[0] some_file.jpg

Import/clone a Subversion repo to a git repo

 git svn --authors-file=some-authors-file clone svn://address/of/svn/repo new-git-dir

Gives you what's between first string and second string included.

 sed "s/^ABC/+ABC/" 

Get My Public IP Address

 curl -s http://myip.dk/ | egrep -m1 -o '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}'

Getting GnuPG Public Keys From KeyServer

 gpg --keyserver pgp.surfnet.nl --recv-key 19886493

Binary search/replace

 xxd < orig | sed 's/A/B/' | sed 's/HEXA/HEXB/' | xxd -r > new

"I Feel Lucky" for Google Images

 echo -n "search> ";read QUERY && wget -O - `wget -O - -U "Mozilla/5.0" "http://images.google.com/images?q=${QUERY}" 2>/dev/null |sed -e 's/","http/\n","http/g' |awk -F \" '{print $3}' |grep -i http: |head -1` > "$QUERY"

modify a file in place with perl

 perl -pi -e 's/THIS/THAT/g' fileglob*

Create a DOS floppy image

 dd if=/dev/zero bs=1024 count=1440 > floppy.img && mkdosfs floppy.img

Test for Weak SSL Ciphers

 openssl s_client -connect [host]:[sslport] -cipher LOW

Translate your terminal into Swedish Chef

 perl -e '$b="bork"; while(){$l=`$_ 2>&1`; $l=~s/[A-Za-z]+/$b/g;

Change every instance of OLD to NEW in file FILE

 sed -i 's/OLD/NEW/g' FILE

Getting started with tcpdump

 tcpdump -nli eth0; tcpdump -nli eth0 src or dst w.x.y.z; tcpdump -nli eth0 port 80; tcpdump -nli eth0 proto udp

Dump root ext3 fs over ssh

 dump 0f - / | bzip -c9 | ssh user@host "cat > /home/user/root.dump.bz2"

Resets a terminal that has been messed up by binary input

 reset

Display total Kb/Mb/Gb of a folder and each file

 du -hc *

easily convert one unit to another

 units "2048 bytes" "kibibytes"

show the date every rpm was installed

 rpm -qa --last

To find the count of each open file on a system (that supports

 sudo lsof | awk '{printf("%s %s %s\n", $1, $3, $NF)}' | grep -v "(" | sort -k 4 | gawk '$NF==prv{ct++;next} {printf("%d %s\n",ct,$0);ct=1;prv=$NF}' | uniq | sort -nr

rgrep: recursive grep without .svn

 alias rgrep="find . \( ! -name .svn -o -prune \) -type f -print0 | xargs -0 grep"

get a directory from one machine to another using tar and ssh

 ssh somemachine "cd some dir; tar zcpf - somedirname" |tar zxpf -

A nice command for summarising repeated information

 alias counts=sort | uniq -c | sort -nr

Command to keep an SSH connection open

 watch -n 30 uptime

Run a command on a remote machine

 ssh user@host "ps aux | grep httpd | wc -l"

Display a list of all PHP classes that are called statically

 find . -name "*\.php" | xargs grep -o --color "\w\+::\w\+" | cut -d":" -f2 | sort | uniq -c

Show recent earthquakes in Bay Area

 lynx --width=200 --dump 'http://quake.usgs.gov/recenteqs/Maps/San_Francisco_eqs.htm'|sed -ne '/MAG.*/,/^References/{;s/\[[0-9][0-9]*\]//;1,/h:m:s/d;/Back to map/,$d;/^$/d;/^[ \t][ \t]*[3-9]\.[0-9][0-9]*[ \t][ \t]*/p; }'|sort -k1nr

Every Nth line position # (AWK)

 awk 'NR%3==1' file

Find all files <10MB and sum up their size

 i=0; for f in $(find ./ -size -10M -exec stat -c %s {} \; ); do i=$(($i + $f)); done; echo $i

Every Nth line position # (SED)

 sed -n '1,${p;n;n;}' foo > foo_every3_position1; sed -n '2,${p;n;n;}' foo > foo_every3_position2; sed -n '3,${p;n;n;}' foo > foo_every3_position3

Rename .JPG to .jpg recursively

 find /path/to/images -name '*.JPG' -exec bash -c 'mv "$1" "${1/%.JPG/.jpg}"' -- {} \;

Prints line numbers

 grep -n "^" 

Print a row of 50 hyphens

 python -c 'print "-" * 50'

Resize a Terminal Window

 printf "\e[8;70;180;t"

Pretty man pages under X

 function manpdf() {man -t $1 | ps2pdf - - | epdfview -}

Pretty man pages under X

 vman(){ T=/tmp/$$.pdf;man -t $1 |ps2pdf - >$T; xpdf $T; rm -f $T; }

Print a row of 50 hyphens

 perl -le'print"-"x50'

Do a search-and-replace in a file after making a backup

 sed -i.bak 's/old/new/g' file

view certificate details

 openssl x509 -in filename.crt -noout -text

analyze traffic remotely over ssh w/ wireshark

 mkfifo /tmp/fifo; ssh-keygen; ssh-copyid root@remotehostaddress; sudo ssh root@remotehost "tshark -i eth1 -f 'not tcp port 22' -w -" > /tmp/fifo &; sudo wireshark -k -i /tmp/fifo;

Solaris get PID socket

 pfiles -F /proc/* 2>/dev/null | awk '/^[0-9]+/{proc=$1};/[s]ockname: AF_INET/{print proc $0}'

Batch convert PNG to JPEG

 for i in *.png; do convert "$i" "${i%.png}.jpg" && rm "$i" && echo "$i is converted."; done

pipe output to notify-send

 echo 'Desktop SPAM!!!' | while read SPAM_OUT; do notify-send "$SPAM_OUT"; done

Display _something_ when an X app fails

 xlaunch(){ T=/tmp/$$;sh -c "$@" >$T.1 2>$T.2;S=$?;[ $S -ne 0 ]&&{ echo -e "'$@' failed with error $S\nSTDERR:\n$(cat $T.2)\nSTDOUT:\n$(cat $T.1)\n"|xmessage -file -;};rm -f $T.1 $T.2;}

mount an iso

 mount -o loop -t iso9660 my.iso /mnt/something

monitor network traffic and throughput in real time

 iptraf

find out how much space are occuipied by files smaller than 1024K

 find dir -size -1024k -type f -print0 | du --files0-from - -bc

reload config

 source .bashrc

Create aliases for common vim minibuffer/cmd typos

 command! -nargs=1 Vs vs 

find out how much space is occupied by files smaller than 1024K

 find dir -size -1024k -type f | xargs -d $'\n' -n1 ls -l | cut -d ' ' -f 5 | sed -e '2,$s/$/+/' -e '$ap' | dc

List folders containing only PNGs

 find . -name '*png' -printf '%h\0' | xargs -0 ls -l --hide=*.png | grep -ZB1 ' 0$'

Find the location of the currently loadedphp.ini

 php -i | grep php.ini

Test file system type before further commands execution

 DIR=. ; FSTYPE=$(df -TP ${DIR} | grep -v Type | awk '{ print $2 }') ; echo "${FSTYPE}"

See entire packet payload using tcpdump.

 tcpdump -nnvvXSs 1514 -i  

Sniffing network to generate a pcap file in CLI mode on a remote

 tcpdump -v -i  -s 0 -w /tmp/sniff.pcap port  # On the

Read just the IP address of a device

 /sbin/ip -f inet addr | sed -rn 's/.*inet ([^ ]+).*(eth[[:digit:]]*(:[[:digit:]]+)?)/\2 \1/p' | column -t

Random colours at random locations

 p(){ printf "\033[%d;%dH\033[4%dm \033[m" $((RANDOM%LINES+1)) $((RANDOM%COLUMNS+1)) $((RANDOM%8)); }; clear;while :;do p; sleep .001;done

output stats from a running dd command to see its progress

 watch -n60 --kill -USR1 $(pgrep dd)

Freshening up RKhunter

 rkhunter --versioncheck --update --propupd --check

Expedient hard disk temprature and load cycle stats

 watch -d 'sudo smartctl -a /dev/sda | grep Load_Cycle_Count ; sudo smartctl -a /dev/sda | grep Temp'

Show permissions of current directory and all directories upwards

 dir=$(pwd); while [ ! -z "$dir" ]; do ls -ld "$dir"; dir=${dir%/*}; done; ls -ld /

Display IP adress of the given interface in a most portable and

 x=IO::Interface::Simple; perl -e 'use '$x';' &>/dev/null || cpan -i "$x"; perl -e 'use '$x'; my $ip='$x'->new($ARGV[0]); print $ip->address,$/;' 

Collect a lot of icons from /usr/share/icons (may overwrite some,

 mkdir myicons; find /usr/share/icons/ -type f -exec cp {} ./myicons/ \;

list all opened ports on host

 time { i=0; while [ $(( i < 65535 )) -eq 1 ] ; do nc -zw2 localhost $((++i)) && echo port $i opened ; done; }

Get your public ip

 curl -s http://sputnick-area.net/ip

random xkcd comic as xml

 curl -sL 'dynamic.xkcd.com/comic/random/' | awk -F\" '/^\n\n\n %s\n %s\n %s\n\n\n", $6, $4, $2)}'

View a random xkcd comic

 wget -q http://dynamic.xkcd.com/comic/random/ -O-| sed -n '/

Provide information on IPC (Inter-process communication)

 ipcs

recursive search and replace old with new string, inside files

 $rpl -R oldstring newstring folder

recursive search and replace old with new string, inside files

 find . -type f -exec sed -i s/oldstring/newstring/g {} +

Find and copy scattered mp3 files into one directory

 find . -type f -iname '*.mp3' -exec cp {} ~/mp3/ \;

Check your hard drive for bad blocks (destructive)

 badblocks -c 65536 -o /tmp/badblocks.out -p 2 -s -v -w /dev/hdX > /tmp/badblocks.stdout 2> /tmp/badblocks.stderr

Find and copy scattered mp3 files into one directory

 find . -name '*.mp3' -type f -exec sh -c 'exec cp -f "$@" /home/user/dir' find-copy {} +

Quickly batch resize images

 mogrify -geometry 800x600 *.jpg

Update twitter via curl (and also set the "from" bit)

 curl -u twitter-username -d status="Hello World, Twitter!" -d source="cURL" http://twitter.com/statuses/update.xml

Monitoring sessions that arrive at your server

 watch -n 1 -d "finger"

Ruby - nslookup against a list of IP`s or FQDN`s

 while read n; do host $n; done < list

Fill up disk space (for testing)

 tail $0 >> $0

Show local/public IP adresses with or without interface argument

 MyIps(){ echo -e "local:\n$(ifconfig $1 | grep -oP 'inet (add?r:)?\K(\d{1,3}\.){3}\d{1,3}')\n\npublic:\n$(curl -s sputnick-area.net/ip)"; }

Audible warning when a downloading is finished

 while [ "$(ls $filePart)" != "" ]; do sleep 5; done; mpg123 /home/.../warning.mp3

mount a cdrom

 mount -t iso9660 /dev/cdrom /media/cdrom

Return IP Address

 perl -e '$_=`ifconfig eth0`;/\d+.\d+.\d+.\d+ /; print $&,"\n";'

Return IP Address

 ifconfig|while read i;do [[ $i =~ inet.*B ]]&&r=${i%%B*}&&echo ${r/*[tr]:/};done

Capture data in ASCII. 1500 bytes

 tcpdump -ieth0 -n tcp port 80 -A -s1500

Find file containing namespace in a directory of jar files.

 for f in *.jar; do if jar -tf $f | grep -q javax.servlet; then echo $f; fi; done

View the newest xkcd comic.

 curl -s 'xkcd.com' | awk -F\" '/^\n\n\n %s\n %s\n %s\n\n\n", $6, $4, $2)}'

find all writable (by user) files in a directory tree (use 4 for

 find . -type f -perm +200 -print

show how many regex you use in your vim today

 cat ~/.viminfo | sed -n '/^:[0-9]\+,\([0-9]\+\|\$\)s/p'

find read write traffic on disk since startup

 iostat -m -d /dev/sda1

urldecoding

 perl -pe 's/%([0-9a-f]{2})/sprintf("%s", pack("H2",$1))/eig'

Pull up remote desktop for other than gnome/kde eg fluxbox

 rdp() { ssh $1 sh -c 'PATH=$PATH:/usr/local/bin; x11vnc -q -rfbauth ~/.vnc/passwd -display :0' & sleep 4; vncviewer $1:0 & }

Get your bash scripts to handle options (-h, --help etc) and spit

 process-getopt

urldecoding

 printf $(echo -n $1 | sed 's/\\/\\\\/g;s/\(%\)\([0-9a-fA-F][0-9a-fA-F]\)/\\x\2/g')

In (any) vi, add a keystroke to format the current paragraph.

 map ^A !}fmt

Convert decimal numbers to binary

 function decToBin { echo "ibase=10; obase=2; $1" | bc; }

Find all dotfiles and dirs

 find -mindepth 1 -maxdepth 1 -name .\*

watch the previous command

 watch -n1 -d !!

List all symbolic links in current directory

 ls -lah | grep ^l

Find all dot files and directories

 ls -d .*

Debian Runlevel configuration tool

 rcconf

Checks apache's access_log file, strips the search queries and

 awk '/q=/{print $11}' /var/log/httpd/access_log.4 | awk -F 'q=' '{print $2}' | sed 's/+/ /g;s/%22/"/g;s/q=//' | cut -d "&" -f 1

Show sorted list of files with sizes more than 1MB in the current

 ls -l | awk '$5 > 1000000' | sort -k5n

rotate the compiz cube via command line

 wmctrl -o 1280,0

Watch a TiVo File On Your Computer

 curl -s -c /tmp/cookie -k -u tivo:$MAK --digest http://$tivo/download/$filename | tivodecode -m $MAK -- - | mplayer - -cache-min 50 -cache 65536

check the status of 'dd' in progress

 while killall -USR1 dd; do sleep 5; done

Both view and pipe the file without saving to disk

 cat /path/to/some/file.txt | tee /dev/pts/0 | wc -l

Get ethX mac addresses

 ip link | grep 'link/ether' | awk '{print $2}'

Grab an interface's IP from ifconfig without screen clutter

 ifconfig eth1 | grep inet\ addr | awk '{print $2}' | cut -d: -f2 | sed s/^/eth1:\ /g

Display top 5 processes consuming CPU

 ps -eo pcpu,user,pid,cmd | sort -r | head -5

List the files any process is using

 lsof +p xxxx

Kill any lingering ssh processes

 for i in `ps aux | grep ssh | grep -v grep | awk {'print $2'}` ; do kill $i; done

get a process list by listen port

 netstat -ntlp | grep 80 | awk '{print $7}' | cut -d/ -f1

Show current working directory of a process

 pwdx pid

Open-iscsi target discovery

 iscsiadm -m discovery -t sendtargets -p 192.168.20.51

Creates Solaris alternate boot environment on another zpool.

 lucreate -n be1 [-c be0] -p zpool1

one-liner mpc track changer using dmenu

 mpc play $(sed -n "s@^[ >]\([0-9]\+\)) $(mpc playlist|cut -d' ' -f3-|dmenu -i -p 'song name'||echo void)@\1@p" < <(mpc playlist))

A function to find the newest file of a set.

 newest () { candidate=''; for i in "$@"; do [[ -f $i ]] || continue; [[

Erase empty files

 find . -type f -size 0 -delete

Monitor a file's size

 watch -n 60 du /var/log/messages

Remove CR LF from a text file

 tr -d '\r\n' < input_file.txt > output_file.txt

Check a server is up. If it isn't mail me.

 curl -fs brandx.jp.sme 2&>1 > /dev/null || echo brandx.jp.sme ping failed | mail -ne -s'Server unavailable' joker@jp.co.uk

On Screen micro display for battery and CPU temperature.

 acpi -t | osd_cat -p bottom

set wallpaper on windowmaker in one line

 wmsetbg -s -u path_to_wallpaper

List your largest installed packages (on Debian/Ubuntu)

 sed -ne '/^Package: \(.*\)/{s//\1/;h;};/^Installed-Size: \(.*\)/{s//\1/;G;s/\n/ /;p;}' /var/lib/dpkg/status | sort -rn

Get decimal ascii code from character

 echo -n a | od -d | sed -n "s/^.* //gp"

get the ascii number with bash builtin printf

 printf "%d\n" "'A" "'B"

Function to output an ASCII character given its decimal

 chr () { echo -en "\0$(printf %x $1)"}

Function to output an ASCII character given its decimal

 chr() { printf \\$(printf %o $1); }

Less a grep result, going directly to the first match in the

 argv=("$@"); rest=${argv[@]:1}; less -JMN +"/$1" `grep -l $1 $rest`

Execute a command before display the bash prompt

 PROMPT_COMMAND=command

gzip all files in a directory separately

 gzip *

gzip all files in a directory separately

 ls | xargs -n1 gzip

MoscowML with editable input-line and history

 rlwrap mosml

retrieve the source address used to contact a given host

 python -c 'import socket; s = socket.socket(socket.AF_INET, socket.SOCK_STREAM); s.connect(("", )); print s.getsockname()[0] ; s.close() ;' 2> /dev/null

Get absolut path to your bash-script

 PATH=$(cd ${0%/*}; pwd)

Get full URL via http://untr.im/api/ajax/api

 URL=[target.URL]; curl -q -d "url=$URL" http://untr.im/api/ajax/api | awk -F 'href="' '{print $3}' | awk -F '" rel="' '{print $1}'

remove the last of all html files in a directory

 for f in *.html; do sed '$d' -i "$f"; done

Makefile argument passing

 make [target] VAR=foobar

convert filenames in current directory to lowercase

 find my_root_dir -depth -exec rename 's/(.*)\/([^\/]*)/$1\/\L$2/' {} \;

Print a list of the 30 last modified mp3s sorted by last first

 find ~/Music -daystart -mtime -60 -name *mp3 -printf "%T@\t%p\n" | sort -f -r | head -n 30 | cut -f 2

Print text string vertically, one character per line.

 echo "vertical text" | fold -1

Generate SHA1 hash for each file in a list

 find . -type f -exec sha1sum {} >> SHA1SUMS \;

Benchmark report generator

 hardinfo -am benchmark.so -f html > report.html

Changing the terminal title to the last shell command

 if [ "$SHELL" = '/bin/zsh' ]; then case $TERM in rxvt|*term|linux) preexec () { print -Pn "\e]0;$1\a" };; esac; fi

Force an fsck on reboot

 shutdown -rF now

Grep auth log and print ip of attackers

 egrep 'Failed password for invalid' /var/log/secure | awk '{print $13}' | uniq

Generate SHA1 hash for each file in a list

 ls [FILENAME] | xargs openssl sha1

Notify Gnome user of files modified today

 OLDIFS=$IFS; IFS=$(echo -en "\n\b"); for f in `find -daystart -mtime 0 -type f -printf "%f\n"`; do notify-send -t 0 "$f downloaded" ; done; IFS=$OLDIFS

Download a TiVo Show

 curl -s -c /tmp/cookie -k -u tivo:$MAK --digest "$(curl -s -c

Create a single-use TCP proxy with copy to stdout

 gate() { mkfifo /tmp/sock1 /tmp/sock2 &> /dev/null && nc -p $1 -l < /tmp/sock1 | tee /tmp/sock2 & PID=$! && nc $2 $3 < /tmp/sock2 | tee /tmp/sock1; kill -KILL $PID; rm -f /tmp/sock1 /tmp/sock2 ; }

Print all members of US House of Representatives

 curl "http://www.house.gov/house/MemberWWW.shtml" 2>/dev/null | sed -e :a -e 's/<[^>]*>//g;/

Remove blank lines from a file using grep and save output to new

 grep -v "^$" filename > newfilename

Search for a string inside all files in the current directory

 grep -r  * .[!.]*

Search for a string inside all files in the current directory

 ack 

Search for a string inside all files in the current directory

 find . -type f -print0 | xargs -0 grep -i 

calulate established tcp connection of local machine

 netstat -an | grep -Ec '^tcp.+ESTABLISHED$'

Adding Color Escape Codes to global CC array for use by echo -e

 declare -ax CC; for i in `seq 0 7`;do ii=$(($i+7)); CC[$i]="\033[1;3${i}m"; CC[$ii]="\033[0;3${i}m"; done

shorten url using curl, sed and is.gd

 curl -s -d URL="$1" http://is.gd/create.php | sed '/Your new shortened/!d;s/.*value="\([^"]*\)".*/\1/'

Delete all but the latest 5 files, ignoring directories

 ls -lt|grep ^-|awk 'NR>5 { print $8 }'|xargs -r rm

Delete all but the latest 5 files

 ls -t | tail +6 | xargs rm

convert a,b,c to ('a','b','c') for use in SQL in-clauses

 echo a,b,c | sed -e s/,/\',\'/g -e s/^/\(\'/ -e s/$/\'\)/

Remove all files but one starting with a letter(s)

 rm -rf [a-bd-zA-Z0-9]* c[b-zA-Z0-9]*

Setting reserved blocks percentage to 1%

 sudo tune2fs -m 1 /dev/sda4

find only current directory (universal)

 find . \( ! -name . -prune \) \( -type f -o -type l \)

Ultimate current directory usage command

 find . -maxdepth 1 ! -name '.' -execdir du -0 -s {} + | sort -znr | gawk 'BEGIN{ORS=RS="\0";} {sub($1 "\t", ""); print $0;}' | xargs -0 du -hs

Output files without comments or empty lines

 function catv { egrep -v "^$|^#" ${*} ; }

List hostnames of all IPs

 for IP in $(/sbin/ifconfig | fgrep addr: | sed 's/.*addr:\([[0-9.]*\) .*/\1/') ; do host $IP | awk '{print $5}'; done

A DESTRUCTIVE command to render a drive unbootable

 dd if=/dev/zero of=/dev/fd0 bs=512 count=1

Ripping VCD in Linux

 cdrdao read-cd --device ATA:1,1,0 --driver generic-mmc-raw --read-raw image.toc

create date-stamped tar archive of files in a directory tree (with subfolders)

 tar czf /path/archive_of_foo.`date -I`.tgz /path/foo

Creae a tar file for backup info

 tar --create --file /path/$HOSTNAME-my_name_file-$(date -I).tar.gz --atime-preserve -p -P --same-owner -z /path/

Force logout after 24 hours idle

 fuser -k `who -u | awk '$6 == "old" { print "/dev/"$2'}`

List out classes in of all htmls in directory

 find . -name '*.html' -exec 'sed' 's/.*class="\([^"]*\?\)".*/\1/ip;d' '{}' ';' |sort -su

Ultimate current directory usage command

 find . -maxdepth 1 -type d|xargs du -a --max-depth=0|sort -rn|cut -d/ -f2|sed '1d'|while read i;do echo "$(du -h --max-depth=0 "$i")/";done;find . -maxdepth 1 -type f|xargs du -a|sort -rn|cut -d/ -f2|sed '$d'|while read i;do du -h "$i";done

bash function to check for something every 5 seconds

 function checkfor () { while :; do $*; sleep 5; done; }

Copy a file structure without files

 find * -type d -exec mkdir /where/you/wantem/\{\} \;

Efficiently print a line deep in a huge log file

 sed '1000000!d;q' < massive-log-file.log

Harder, Faster, Stronger SSH clients

 ssh -4 -C -c blowfish-cbc

Duplicate several drives concurrently

 dd if=/dev/sda | tee >(dd of=/dev/sdb) | dd of=/dev/sdc

backup all your commandlinefu.com favourites to a plaintext file

 clfavs(){ URL="http://www.commandlinefu.com";wget -O - --save-cookies c --post-data "username=$1&password=$2&submit=Let+me+in" $URL/users/signin;for i in `seq 0 25 $3`;do wget -O - --load-cookies c $URL/commands/favourites/plaintext/$i >>$4;done;rm -f c;}

Cleanup firefox's database.

 find ~/.mozilla/firefox/ -type f -name "*.sqlite" -exec sqlite3 {} VACUUM \;

Terminal - Show directories in the PATH, one per line with sed and

 tr : '\n' <<<$PATH

get all pdf and zips from a website using wget

 wget --reject html,htm --accept pdf,zip -rl1 url

Release memory used by the Linux kernel on caches

 free && sync && echo 3 > /proc/sys/vm/drop_caches && free

find and delete empty dirs, start in current working dir

 find . -empty -type d -exec rmdir {} +

Triple monitoring in screen

 tmpfile=$(mktemp) && echo -e 'startup_message off\nscreen -t top htop\nsplit\nfocus\nscreen -t nethogs nethogs wlan0\nsplit\nfocus\nscreen -t iotop iotop' > $tmpfile && sudo screen -c $tmpfile

Show me a histogram of the busiest minutes in a log file:

 cat /var/log/secure.log | awk '{print substr($0,0,12)}' | uniq -c | sort -nr | awk '{printf("\n%s ",$0) ; for (i = 0; i<$1 ; i++) {printf("*")};}'

Print a great grey scale demo !

 yes "$(seq 232 255;seq 254 -1 233)" | while read i; do printf "\x1b[48;5;${i}m\n"; sleep .01; done

DELETE all those duplicate files but one based on md5 hash

 find . -type f -print0|xargs -0 md5sum|sort|perl -ne 'chomp;$ph=$h;($h,$f)=split(/\s+/,$_,2);print "$f"."\x00" if ($h eq $ph)'|xargs -0 rm -v --

restoring some data from a corrupted text file

 ( cat badfile.log ; tac badfile.log | tac ) > goodfile.log

How to run X without any graphics hardware

 startx -- `which Xvfb` :1 -screen 0 800x600x24 && DISPLAY=:1 x11vnc

Run a bash script in debug mode, show output and save it on a file

 bash -x test.sh 2>&1 | tee out.test

Add user to group on OS X 10.5

 sudo dscl localhost -append /Local/Default/Groups/admin GroupMembership username

to see about php configure

 $php_dir/bin/php -i | grep configure

Terminal - Show directories in the PATH, one per line with sed and

 sed 's/:/\n/g' <<<$PATH

Verbosely delete files matching specific name pattern, older than

 rm -vf /backup/directory/**/FILENAME_*(m+15)

Copy via tar pipe while preserving file permissions (run this

 tar -C /oldirectory -cvpf - . | tar -C /newdirector -xvf -

watch iptables counters

 watch --interval 0 'iptables -nvL | grep -v "0 0"'

Remove empty directories

 rmdir **/*(/^F)

Ping sweep without NMAP

 for i in `seq 1 255`; do ping -c 1 10.10.10.$i | tr \\n ' ' | awk '/1 received/ {print $2}'; done

follow the content of all files in a directory

 find dir/ -type f | xargs tail -fqn0

View Processeses like a fu, fu

 pstree -p

Not a kismet replacement...

 watch -n .5 "iwlist wlan0 scan"

clear the cache from memory

 sync; echo 3 > /proc/sys/vm/drop_caches

Show changed files, ignoring permission, date and whitespace

 git diff --numstat -w --no-abbrev | perl -a -ne '$F[0] != 0 && $F[1] !=0 && print $F[2] . "\n";'

Scrape commands from commandline fu's 1st page

 curl -s http://www.commandlinefu.com/commands/browse|egrep '("Fin.*and"|
.*
)'|sed 's/<[^<]*>//g'|ruby -rubygems -pe 'require "cgi"; $_=sprintf("\n\n%-100s\n\t#%-20s",CGI.unescapeHTML($_).chomp.strip, gets.lstrip) if $.%2'

The command used by applications in OS X to determine whether a

 plutil -lint plist-file

Resolve a list of domain names to IP addresses

 awk < file.name '{ system("resolveip -s " $1) }'

Unarchive entire folder

 for f in *;do case "$(echo $f|sed "s/.*\.\([a-z\.]*\)/\1/g")" in zip)unzip -qqo $f&&rm $f;;tar.gz|tar.bz2)tar xf $f&&rm $f;;rar)unrar e -o+ -r -y $f&&rm $f;;7z)7z e -qqo $f;;esac;done

Find out when your billion-second anniversary is (was).

 date -j -v +1000000000S -f %m%d%Y mmddYYYY

Find out when your billion-second anniversary is (was). (on OS X)

 date -j -v +1000000000S -f %m%d%Y mmddyyyy

Find out when your billion-second anniversary is (was).

 date -d12/31/1970+1000000000sec

Shrink more than one blank lines to one in VIM.

 :%v/./,/./-j

Have netcat listening on your ports and use telnet to test

 SERVER: nc -l p 666 CLIENT: telnet -l -p 666

return external ip

 host -t a dartsclink.com | sed 's/.*has address //'

Searches $PATH for files using grep

 IFS=:; find $PATH | grep pattern

Find

 xwininfo

open in textmate sidebar files (recursively) with names matching

 mate - `find * -type f -regex 'REGEX_A' | grep -v -E 'REGEX_B'`

View a colorful logfile using less

 < /var/log/syslog ccze -A | less -R

Generate random IP addresses

 nmap -n -iR 0 -sL | cut -d" " -f 2

Get the size of all the directories in current directory

 du -hd 1

Convert every eps in a directory to pdf

 for f in *.eps;do ps2pdf -dEPSCrop $f `basename $f .eps`.pdf; done

Remove newlines from output

 awk /./ filename

Remove newlines from output

 cat filename | grep .

A fun thing to do with ram is actually open it up and take a peek.

 strings /dev/mem|less

complete extraction of a debian-package

 dpkg-deb -x $debfile $extractdir; dpkg-deb -e $debfile $extractdir/DEBIAN;

Get the size of all the directories in current directory

 sudo du -sh $(ls -d */) 2> /dev/null

Get the ip registered to a domain on OpenWRT

 nslookup commandlinefu.com|sed 's/[^0-9. ]//g'|tail -n 1|awk -F " " '{print $2}'

Test http request every second, fancy display.

 watch -n 1 nc localhost 80 '<

Display the definition of a shell function

 typeset -f 

tar copy

 tar cf - dir_to_cp/ | (cd path_to_put/ && tar xvf -)

Nicely display permissions in octal format with filename

 stat -f '%Sp %p %N' * | rev | sed -E 's/^([^[:space:]]+)[[:space:]]([[:digit:]]{4})[^[:space:]]*[[:space:]]( [^[:space:]]+)/\1 \2 \3/' | rev

Shows how many percents of all avaliable packages are installed in

 echo $((`eix --only-names -I | wc -l` * 100 / `eix --only-names | wc -l`))%

Installing debian on fedora (chrooted)

 debootstrap --arch i386 lenny /opt/debian ftp://debian.das.ufsc.br/pub/debian/

Updating to Fedora 11

 yum clean all ; rpm -Uvh http://download.fedora.redhat.com/pub/fedora/linux/releases/11/Fedora/i 386/os/Packages/fedora-release-11-1.noarch.rpm ; yum -y upgrade ; reboot

Which files/dirs waste my disk space

 du -aB1m|awk '$1 >= 100'

Query ip pools based on successive netnames via whois

 net=DTAG-DIAL ; for (( i=1; i<30; i++ )); do whois -h whois.ripe.net $net$i | grep '^inetnum:' | sed "s;^.*:;$net$i;" ; done

Get Futurama quotations from slashdot.org servers

 curl -sI http://slashdot.org/ | sed -nr 's/X-(Bender|Fry)(.*)/\1\2/p'

Get all IPs via ifconfig

 ifconfig | awk '/ddr:[0-9]/ {sub(/addr:/, ""); print $2}'

Make a statistic about the lines of code

 find . -type f -name "*.c" -exec cat {} \; | wc -l

txt2html

 recode ..HTML < file.txt > file.html

List all installed Debian packages

 dpkg --get-selections | grep -v deinstall | cut -f 1

Tail the most recently modified file

 ls -t1 | head -n1 | xargs tail -f

Join a folder full of split files

 for file in *.001; do NAME=`echo $file | cut -d. -f1,2`; cat "$NAME."[0-9][0-9][0-9] > "$NAME"; done

Combine two mp3's or more into 1 long mp3

 cat 1.mp3 2.mp3 > combined.mp3

Get the SUM of visual blocked digits in vim

 vmap  y:$omap:'a,$!awk '{sum+=$0}END{print "SUM:" sum}'dd'>p

Recursive replace of directory and file names in the current

 for i in `find -name '*oldname*'`; do "mv $i ${i/oldname/newname/}"; done

Recursive replace of directory and file names in the current

 find . *oldname* | grep oldname | perl -p -e 's/^(.*)(oldname)(.*$)/mv $1$2$3 $1newname$3/' | sh

Get all IPs via ifconfig

 ifconfig | awk -F':| +' '/ddr:/{print $4}'

Output a SSL certificate start or end date

 date --date="$(openssl x509 -in xxxxxx.crt -noout -startdate | cut -d= -f 2)" --iso-8601

Efficient count files in directory (no recursion)

 perl -e 'if(opendir D,"."){@a=readdir D;print $#a-1,"\n"}'

get events from google calendar for a given dates range

 wget -q -O - 'URL/full?orderby=starttime&singleevents=true&start-min=2009-06-01&star t-max=2009-07-31' | perl -lane '@m=$_=~m/(.+?)</g;@a=$_=~m/startTime=.(2009.+?)T/g;shift @m;for ($i=0;$i<@m;$i++){ print $m[$i].",".$a[$i];}';</pre>
<p>
<strong><em>get delicious bookmarks on your shell (text version :-))</em></strong>
  <pre class="codeline"> curl -u 'username' https://api.del.icio.us/v1/posts/all | sed 's/^.*href=//g;s/>.*$//g;s/"//g' | awk '{print $1}' | grep 'http'</pre>
<p>
<strong><em>search string in _all_ revisions</em></strong>
  <pre class="codeline"> for i in `git log --all --oneline --format=%h`; do git grep SOME_STRING $i; done</pre>
<p>
<strong><em>Read just the IP address of a device</em></strong>
  <pre class="codeline"> /sbin/ifconfig | awk -F'[ :]+' '/inet addr/{print $4}'</pre>
<p>
<strong><em>Add temporary entry to authorized_keys</em></strong>
  <pre class="codeline"> Keys=$HOME/.ssh/authorized_keys;Back=$Keys.tmp.bak;Time=${1:-15};cp $Keys $Back;cat /dev/stdin >>$Keys;echo mv $Back $Keys|at now+${Time}minutes;</pre>
<p>
<strong><em>A command's package details</em></strong>
  <pre class="codeline"> dpkg -S `which nm` | cut -d':' -f1 | (read PACKAGE; echo "[${PACKAGE}]"; dpkg -s "${PACKAGE}"; dpkg -L "${PACKAGE}") | less</pre>
<p>
<strong><em>Test your total disk IO capacity, regardless of caching, to find</em></strong>
  <pre class="codeline"> time (dd if=/dev/zero of=blah.out bs=256M count=1 ; sync )</pre>
<p>
<strong><em>git remove files which have been deleted</em></strong>
  <pre class="codeline"> git ls-files -z --deleted | xargs -0 git rm</pre>
<p>
<strong><em>Find files with lines that do not match a pattern</em></strong>
  <pre class="codeline"> fmiss() { grep -RL "$*" * }</pre>
<p>
<strong><em>Copy 3 files from 3 different servers and adds server name tag to</em></strong>
  <pre class="codeline"> for i in `seq 1 3`; do scp finku@server$i:file.txt server$i-file.txt; done</pre>
<p>
<strong><em>extract all tgz in current dir</em></strong>
  <pre class="codeline"> ls *tgz | xargs -n1 tar xzf</pre>
<p>
<strong><em>Search commandlinefu from the command line</em></strong>
  <pre class="codeline"> (curl -d q=grep http://www.commandlinefu.com/search/autocomplete) | egrep 'autocomplete|votes|destination' | perl -pi -e 's/a style="display:none" class="destination" href="//g;s/<[^>]*>//g;s/">$/\n\n/g;s/^ +//g;s/^\//http:\/\/commandlinefu.com\//g'</pre>
<p>
<strong><em>Check if you need to run LaTeX to update the TOC</em></strong>
  <pre class="codeline"> cp texfile.toc texfile.toc.bak; latex texfile.tex; cmp -s texfile.toc texfile.toc.bak; if [ $? -ne 0 ]; then latex texfile.tex; fi</pre>
<p>
<strong><em>clone a hard drive to a remote directory via ssh tunnel, and</em></strong>
  <pre class="codeline"> # dd if=/dev/sda | gzip -c | ssh user@ip 'dd of=/mnt/backups/sda.dd'</pre>
<p>
<strong><em>Solaris - check ports/sockets which process has opened</em></strong>
  <pre class="codeline"> /usr/proc/bin/pfiles $PID | egrep "sockname|port"</pre>
<p>
<strong><em>Access partitions inside a LVM volume</em></strong>
  <pre class="codeline"> kpartx -a /dev/mapper/space-foobar</pre>
<p>
<strong><em>Log the current memory statistics frequently to syslog</em></strong>
  <pre class="codeline"> while true; do { $(which logger) -p local4.notice `free -m | grep Mem`; sleep 60; } done &</pre>
<p>
<strong><em>Record camera's output to a avi file</em></strong>
  <pre class="codeline"> mencoder -tv device=/dev/video1 tv:// -ovc copy -o video.avi</pre>
<p>
<strong><em>list all crontabs for users</em></strong>
  <pre class="codeline"> cut -d: -f1 /etc/passwd | grep -vE "#" | xargs -i{} crontab -u {} -l</pre>
<p>
<strong><em>Find Files That Exceed a Specified Size Limit</em></strong>
  <pre class="codeline"> find directory -size +nnn</pre>
<p>
<strong><em>get newest jpg picture in a folder</em></strong>
  <pre class="codeline"> cp `ls -x1tr *.jpg | tail -n 1` newest.jpg</pre>
<p>
<strong><em>Use QuickLook from the command line without verbose output</em></strong>
  <pre class="codeline"> qlook() { qlmanage -p "$@" >& /dev/null & }</pre>
<p>
<strong><em>Skip to next selection in playlist</em></strong>
  <pre class="codeline"> killall -2 mpg321</pre>
<p>
<strong><em>Show all usernames and passwords for Plesk email addresses</em></strong>
  <pre class="codeline"> mysql -uadmin -p` cat /etc/psa/.psa.shadow` -Dpsa -e"select mail_name,name,password from mail left join domains on mail.dom_id = domains.id inner join accounts where mail.account_id = accounts.id;"</pre>
<p>
<strong><em>The Chronic: run a command every N seconds in the background</em></strong>
  <pre class="codeline"> chronic () { t=$1; shift; while true; do $@; sleep $t; done & }</pre>
<p>
<strong><em>eDirectory LDAP Search for Statistics</em></strong>
  <pre class="codeline"> ldapsearch -h ldapserver.willeke.com -p389 -b "" -s base -D cn=admin,ou=administration,dc=willeke,dc=com -w secretpwd "(objectclass=*)" chainings removeEntryOps referralsReturned listOps modifyRDNOps repUpdatesIn repUpdatesOut strongAuthBinds addEntryOps</pre>
<p>
<strong><em>Pipe the result of a command to IRC (channel or query)</em></strong>
  <pre class="codeline"> function my_irc { tmp=`mktemp`; cat > $tmp; { echo -e "USER $username x x :$ircname\nNICK $nick\nJOIN $target"; while read line; do echo -e "PRIVMSG $target :$line"; done < $tmp; } | nc $server > /dev/null ; rm $tmp; }</pre>
<p>
<strong><em>Phrack 66 is out, but the .tar.gz is not there yet on</em></strong>
  <pre class="codeline"> mkdir phrack66; (cd phrack66; for n in {1..17} ; do echo "http://www.phrack.org/issues.html?issue=66&id=$n&mode=txt" ; done | xargs wget)</pre>
<p>
<strong><em>Recursively create a TAGS file for an entire source tree. TAGS</em></strong>
  <pre class="codeline"> ctags -R</pre>
<p>
<strong><em>rsync over ssh via non-default ssh port</em></strong>
  <pre class="codeline"> rsync -e 'ssh -p PORT' user@host:SRC DEST</pre>
<p>
<strong><em>Configuring proxy client on terminal</em></strong>
  <pre class="codeline"> export http_proxy=<user>:<pass>@<server>:<port> ftp_proxy=<user>:<pass>@<server>:<port></pre>
<p>
<strong><em>Check wireless link quality with dialog box</em></strong>
  <pre class="codeline"> while [ i != 0 ]; do sleep 1 | dialog --clear --gauge "Quality: " 0 0 $(cat /proc/net/wireless | grep $WIRELESSINTERFACE | awk '{print $3}' | tr -d "."); done</pre>
<p>
<strong><em>change ownership en masse of files owned by a specific user,</em></strong>
  <pre class="codeline"> find . -uid 0 -print0 | xargs -0 chown foo:foo</pre>
<p>
<strong><em>Clear history</em></strong>
  <pre class="codeline"> history -c</pre>
<p>
<strong><em>Change timestamp on a file</em></strong>
  <pre class="codeline"> touch -amct [[CC]YY]MMDDhhmm[.ss] FILE</pre>
<p>
<strong><em>Show the last 20 sessions logged on the machine</em></strong>
  <pre class="codeline"> last -n 20</pre>
<p>
<strong><em>Show LAN IP with ip(8)</em></strong>
  <pre class="codeline"> ip route show dev eth0 | awk '{print $7}'</pre>
<p>
<strong><em>Unlock your KDE4 session remotely (for boxes locked by KDE lock</em></strong>
  <pre class="codeline"> killall -s 9 krunner_lock</pre>
<p>
<strong><em>Using scapy to get the IP of the iface used to contact local gw</em></strong>
  <pre class="codeline"> python -c "import scapy.all; print [x[4] for x in scapy.all.conf.route.routes if x[2] != '0.0.0.0'][0]"</pre>
<p>
<strong><em>Backup with SSH in a archive</em></strong>
  <pre class="codeline"> ssh -i $PRIVATEKEY $HOST -C 'cd $SOURCE; tar -cz --numeric-owner .' | tee $DESTINATION/backup.tgz | tar -tz</pre>
<p>
<strong><em>Listen to a file</em></strong>
  <pre class="codeline"> while true; do cat /usr/src/linux/kernel/signal.c > /dev/dsp; done</pre>
<p>
<strong><em>Updating the status on identi.ca using curl.</em></strong>
  <pre class="codeline"> curl -u USER:PASS -d status="NEW STATUS" http://identi.ca/api/statuses/update.xml</pre>
<p>
<strong><em>Copy data using gtar</em></strong>
  <pre class="codeline"> gtar cpf - . | (cd /dest/directory; gtar xpf -)</pre>
<p>
<strong><em>burn initial session on a growable DVD using growisofs</em></strong>
  <pre class="codeline"> growisofs -Z /dev/dvd -J -r "directory name to burn on DVD"</pre>
<p>
<strong><em>dump 1KB of data from ram to file</em></strong>
  <pre class="codeline"> dd if=/dev/mem of=file.dump bs=1024 skip=0 count=1</pre>
<p>
<strong><em>Change framebuffer font</em></strong>
  <pre class="codeline"> setfont cybercafe</pre>
<p>
<strong><em>Report information about executable launched on system</em></strong>
  <pre class="codeline"> aureport -x</pre>
<p>
<strong><em>Print summary of referers with X amount of occurances</em></strong>
  <pre class="codeline"> awk -F\" '{print $4}' *.log | grep -v "eviljaymz\|\-" | sort | uniq -c | awk -F\ '{ if($1>500) print $1,$2;}' | sort -n</pre>
<p>
<strong><em>cpu and memory usage top 10 under Linux</em></strong>
  <pre class="codeline"> ps -eo user,pcpu,pmem | tail -n +2 | awk '{num[$1]++; cpu[$1] += $2; mem[$1] += $3} END{printf("NPROC\tUSER\tCPU\tMEM\n"); for (user in cpu) printf("%d\t%s\t%.2f%\t%.2f%\n",num[user], user, cpu[user], mem[user]) }'</pre>
<p>
<strong><em>Convert from octal format to umask</em></strong>
  <pre class="codeline"> perm=( 6 4 4 ) ; for elem in ${perm[@]}; do echo `expr 7 - $elem` ;</pre>
<p>
<strong><em>mail with attachment</em></strong>
  <pre class="codeline"> tar cvzf - data1 data2 | uuencode data.tar.gz | mail -s 'data' you@host.fr</pre>
<p>
<strong><em>find/edit your forgotten buddy pounces for pidgin</em></strong>
  <pre class="codeline"> vim ~/.purple/pounces.xml</pre>
<p>
<strong><em>Display formatted routes</em></strong>
  <pre class="codeline"> routel</pre>
<p>
<strong><em>For Gentoo users : helping with USE / emerge</em></strong>
  <pre class="codeline"> emerge -epv world | grep USE | cut -d '"' -f 2 | sed 's/ /\n/g' | sed '/[(,)]/d' | sed s/'*'//g | sort | uniq > use && grep ^- use | sed s/^-// | sed ':a;N;$!ba;s/\n/ /g' > notuse && sed -i /^-/d use && sed -i ':a;N;$!ba;s/\n/ /g' use</pre>
<p>
<strong><em>Burn CD/DVD from an iso, eject disc when finished.</em></strong>
  <pre class="codeline"> cdrecord dev=0,0,0 -v -eject yourimage.iso</pre>
<p>
<strong><em>Add a line from 1 file after every line of another (shuffle files</em></strong>
  <pre class="codeline"> sed '/^/R addfile' targetfile > savefile</pre>
<p>
<strong><em>Find the modified time (mtime) for a file</em></strong>
  <pre class="codeline"> date -r foo</pre>
<p>
<strong><em>Jump to a song in your XMMS2 playlist, based on song title/artist</em></strong>
  <pre class="codeline"> function jumpTo { xmms2 jump `xmms2 list | grep -i '$1' | head -n 1 | tail -n 1 | sed -re 's@.+\[(.+)/.+\] (.+)@\1@'`; }</pre>
<p>
<strong><em>Use mtr to create a text file report</em></strong>
  <pre class="codeline"> mtr --report --report-cycles 10 www.google.com > google_net_report.txt</pre>
<p>
<strong><em>Get the current svn branch/tag (Good for PS1/PROMPT_COMMAND cases)</em></strong>
  <pre class="codeline"> svn info | grep '^URL:' | egrep -o '(tags|branches)/[^/]+|trunk' | egrep -o '[^/]+$'</pre>
<p>
<strong><em>Change the default editor for modifying the sudoers list.</em></strong>
  <pre class="codeline"> sudo update-alternatives --config editor</pre>
<p>
<strong><em>Simple word scramble</em></strong>
  <pre class="codeline"> shuf -n1 /usr/share/dict/words | tee >(sed -e 's/./&\n/g' | shuf | tr -d '\n' | line) > /tmp/out</pre>
<p>
<strong><em>Show all mergeinfo for a svn subtree</em></strong>
  <pre class="codeline"> find . \( -type d -name .svn -prune \) -o -print | while read file ; do mergeinfo=`svn propget svn:mergeinfo $file` ; [ "$mergeinfo" != "" ] && echo -e "$file\n $mergeinfo\n" ; done</pre>
<p>
<strong><em>SVN Command line branch merge</em></strong>
  <pre class="codeline"> /usr/local/bin/svn merge -r {rev_num}:HEAD https://{host}/{project}/branches/{branch_name} .</pre>
<p>
<strong><em>Propagate X session cookies on a different user and login as that</em></strong>
  <pre class="codeline"> read -p 'Username: ' u;sudo -H -u $u xauth add $(xauth list|grep :$(echo ${DISPLAY: -4:2}));sudo su - $u</pre>
<p>
<strong><em>Display only hosts up in network</em></strong>
  <pre class="codeline"> nmap -sP your network/submask | awk "/^Host/"'{ print $2 }'</pre>
<p>
<strong><em>Recursive Line Count</em></strong>
  <pre class="codeline"> find ./ -not -type d | xargs wc -l | cut -c 1-8 | awk '{total += $1} END {print total}'</pre>
<p>
<strong><em>sorting file contents into individual files with awk</em></strong>
  <pre class="codeline"> awk '{print > $3".txt"}' FILENAME</pre>
<p>
<strong><em>Copy files and directories from a remote machine to the local</em></strong>
  <pre class="codeline"> ssh user@host "(cd /path/to/remote/top/dir ; tar cvf - ./*)" | tar xvf -</pre>
<p>
<strong><em>Then end of the UNIX epoch</em></strong>
  <pre class="codeline"> date -d @$(echo $((2 ** 31 - 1)))</pre>
<p>
<strong><em>View an info page on a nice interface</em></strong>
  <pre class="codeline"> yelp info:foo</pre>
<p>
<strong><em>Flush DNS cache in MacOS 10.5</em></strong>
  <pre class="codeline"> dscacheutil -flushcache</pre>
<p>
<strong><em>list services running (as root)</em></strong>
  <pre class="codeline"> service --status-all | grep running</pre>
<p>
<strong><em>Watch your freebox flux, through a other internet connection (for</em></strong>
  <pre class="codeline"> vlc -vvv http://mafreebox.freebox.fr/freeboxtv/playlist.m3u --sout</pre>
<p>
<strong><em>Watch the National Debt clock</em></strong>
  <pre class="codeline"> watch -n 10 "wget -q http://www.brillig.com/debt_clock -O - | grep debtiv.gif | sed -e 's/.*ALT=\"//' -e 's/\".*//' -e 's/ //g'"</pre>
<p>
<strong><em>Use find to get around Argument list too long problem</em></strong>
  <pre class="codeline"> find . -name 'junkfiles-*' -print0 | xargs -0 rm</pre>
<p>
<strong><em>Easily run a program in the background without losing output</em></strong>
  <pre class="codeline"> function fork () { tf=$(tempfile -d /tmp -p $1.);echo -n "$tf "; $@ &>$tf& }</pre>
<p>
<strong><em>Remove apps with style: nuke it from orbit</em></strong>
  <pre class="codeline"> function nuke() { if [ $(whoami) != "root" ] ; then for x in $@; do sudo apt-get autoremove --purge $x; done; else for x in $@; do apt-get autoremove --purge $x; done; fi }</pre>
<p>
<strong><em>which procs have $PATH_REGEX open?</em></strong>
  <pre class="codeline"> find /proc -regex '/proc/[0-9]+/smaps' -exec grep -l "$PATH_REGEX" {} \; | cut -d'/' -f2</pre>
<p>
<strong><em>add static arp entry to default gateway, arp poison protection</em></strong>
  <pre class="codeline"> arp -s $(route -n | awk '/^0.0.0.0/ {print $2}') \ $(arp -n | grep `route -n | awk '/^0.0.0.0/ {print $2}'`| awk '{print $3}')</pre>
<p>
<strong><em>burn a isofile to cd or dvd</em></strong>
  <pre class="codeline"> cdrecord -v dev=/dev/cdrom yourimage.iso</pre>
<p>
<strong><em>keep an eye on system load changes</em></strong>
  <pre class="codeline"> watch -n 7 -d 'uptime | sed s/.*users,//'</pre>
<p>
<strong><em>Mount an smb share on linux</em></strong>
  <pre class="codeline"> mount -t smbfs //$server/share /local/mount -o rw,username=$USER</pre>
<p>
<strong><em>Display the standard deviation of a column of numbers with awk</em></strong>
  <pre class="codeline"> awk '{sum+=$1; sumsq+=$1*$1} END {print sqrt(sumsq/NR - (sum/NR)**2)}' file.dat</pre>
<p>
<strong><em>Averaging columns of numbers</em></strong>
  <pre class="codeline"> awk '{sum1+=$1; sum2+=$2} END {print sum1/NR, sum2/NR}' file.dat</pre>
<p>
<strong><em>Count number of bytes that are different between 2 binary files</em></strong>
  <pre class="codeline"> cmp -l file1.bin file2.bin | wc -l</pre>
<p>
<strong><em>Stream the latest offering from your fave netcasts/podcasts</em></strong>
  <pre class="codeline"> vlc --one-instance --playlist-enqueue -q $(while read netcast; do wget -q $netcast -O - |grep enclosure | tr '\r' '\n' | tr \' \" | sed -n 's/.*url="\([^"]*\)".*/\1/p'|head -n1; done <netcast.txt)</pre>
<p>
<strong><em>Display email addresses that have been sent to by a postfix</em></strong>
  <pre class="codeline"> sed -n -e '/postfix\/smtp\[.*status=sent/s/^.*to=<\([^>]*\).*$/\1/p' /var/log/mail.log | sort -u</pre>
<p>
<strong><em>Compress a file or directory keeping the owner and permissions</em></strong>
  <pre class="codeline"> tar -jcvf /folder/file.tar.bz2 --same-owner --same-permissions /folder/</pre>
<p>
<strong><em>A signal trap that logs when your script was killed and what other</em></strong>
  <pre class="codeline"> trap "echo \"$0 process $$ killed on $(date).\" | tee ${0##*/}_$$_termination.log; echo 'Active processes at the time were logged to ${0##*/}_$$_termination.log'; ps u >> ${0##*/}_$$_termination.log; exit " HUP INT QUIT ABRT TERM STOP</pre>
<p>
<strong><em>listen to ram</em></strong>
  <pre class="codeline"> cat /dev/mem > /dev/audio</pre>
<p>
<strong><em>Big (four-byte) $RANDOM</em></strong>
  <pre class="codeline"> printf %d 0x`dd if=/dev/urandom bs=1 count=4 2>/dev/null | od -x | awk 'NR==1 {print $2$3}'`</pre>
<p>
<strong><em>HDD Performance Write Test</em></strong>
  <pre class="codeline"> dd if=/dev/zero of=10gb bs=1M count=10240</pre>
<p>
<strong><em>Copy a file and force owner/group/mode</em></strong>
  <pre class="codeline"> install -o user -g group -m 755 /path/to/file /path/to/dir/</pre>
<p>
<strong><em>Delete newline</em></strong>
  <pre class="codeline"> tr -d "\n" < file1 > file2</pre>
<p>
<strong><em>Make the Mac OS X Dock 2D once more (10.5 and above only)</em></strong>
  <pre class="codeline"> defaults write com.apple.Dock no-glass -boolean YES; killall Dock</pre>
<p>
<strong><em>Find files with the same names in several directories.</em></strong>
  <pre class="codeline"> ls -1 . dir2 dir3|sort|uniq -d</pre>
<p>
<strong><em>reset an hanging terminal session</em></strong>
  <pre class="codeline"> ^J tput sgr0 ^J</pre>
<p>
<strong><em>Create a 100MB file for testing transfer speed</em></strong>
  <pre class="codeline"> dd if=/dev/random of=bigfile bs=1024 count=102400</pre>
<p>
<strong><em>Number of CPU's in a system</em></strong>
  <pre class="codeline"> grep "processor" /proc/cpuinfo | wc -l</pre>
<p>
<strong><em>send substituted text to a command without echo, pipe</em></strong>
  <pre class="codeline"> nc localhost 10000 <<< "message"</pre>
<p>
<strong><em>phpinfo from the command line</em></strong>
  <pre class="codeline"> echo "<?php phpinfo(); ?>" | php > phpinfo.txt</pre>
<p>
<strong><em>read Windows ACLs from Linux</em></strong>
  <pre class="codeline"> smbcacls //server/sharename file -U username</pre>
<p>
<strong><em>Umount only the NFS related to 'string'</em></strong>
  <pre class="codeline"> for i in `df -P |grep string|cut -f2 -d%|cut -c2-100`; do umount -l -f $i;done</pre>
<p>
<strong><em>Find files with at least one exec bit set</em></strong>
  <pre class="codeline"> find . -type f -perm +0111 -print</pre>
<p>
<strong><em>Get your default route</em></strong>
  <pre class="codeline"> ip route | grep default | awk '{print $3}'</pre>
<p>
<strong><em>List all files/folders in working directory with their total size</em></strong>
  <pre class="codeline"> du --max-depth=1 -m</pre>
<p>
<strong><em>Show a calendar</em></strong>
  <pre class="codeline"> cal [[month] year]</pre>
<p>
<strong><em>Check for Firewall Blockage.</em></strong>
  <pre class="codeline"> iptables -L -n --line-numbers | grep xx.xx.xx.xx</pre>
<p>
<strong><em>Finds all files from / on down over specified size.</em></strong>
  <pre class="codeline"> find / -type f -size +25M -exec ls -lh {} \; | awk '{ print $5 " " $6$7 ": " $9 }'</pre>
<p>
<strong><em>Mount a windows partition in a dual boot linux installation with</em></strong>
  <pre class="codeline"> mount -o -t ntfs-3g /dev/sda1 /mnt/windows/c force</pre>
<p>
<strong><em>tar pipe to copy files, alternate to cp -Ra</em></strong>
  <pre class="codeline"> (cd /orignl/path tar -cf - . ) | (cd /dst/dir;tar -xvf -)</pre>
<p>
<strong><em>burn backed up xbox 360 games</em></strong>
  <pre class="codeline"> growisofs -use-the-force-luke=dao -use-the-force-luke=break:1913760 -dvd-compat -speed=2 -Z /dev/cdrom=XBOX360GAMEHERE.iso</pre>
<p>
<strong><em>find all files containing a pattern, open them using vi and place</em></strong>
  <pre class="codeline"> find . -type f -exec grep -l pattern {} \; | xargs vi +/pattern</pre>
<p>
<strong><em>converts a directory full of source tarballs into a bzr repository</em></strong>
  <pre class="codeline"> bzr init .;for file in `ls *.bz2`; do bzr import $file; bzr ci -m $file; done</pre>
<p>
<strong><em>Test python regular expressions</em></strong>
  <pre class="codeline"> rgx_match() { python -c "import re; print re.search('$1','$2').groups()"; }</pre>
<p>
<strong><em>Perform a C-style loop in Bash.</em></strong>
  <pre class="codeline"> for (( i = 0; i < 100; i++ )); do echo "$i"; done</pre>
<p>
<strong><em>Pipe a textfile to vim and move the cursor to a certain line</em></strong>
  <pre class="codeline"> zcat /usr/share/doc/vim-common/README.gz | vim -g +23 -</pre>
<p>
<strong><em>Convert PNG to GIF</em></strong>
  <pre class="codeline"> for file in *.png; do convert "$file" "$(basename $file .png).gif"; done</pre>
<p>
<strong><em>Turns hidden applications transparent in the Mac OS X dock.</em></strong>
  <pre class="codeline"> defaults write com.apple.Dock showhidden -bool YES</pre>
<p>
<strong><em>Creat a new user with no shell. Useful to provide other services</em></strong>
  <pre class="codeline"> useradd -s /sbin/nologin nicdev</pre>
<p>
<strong><em>Search specified $TEXT1 and Replace that by specified arg ($TEXT2)</em></strong>
  <pre class="codeline"> find "$DIR" -regex "$FILENAME" -type f -print0 | xargs -0 sed -i _`date "+%y%m%d%H%M%S"` -E "s/$TEXT1/$TEXT2/g"</pre>
<p>
<strong><em>It decripts all pgp files in a selection folder and move the</em></strong>
  <pre class="codeline"> for x in *.pgp do `cat /file_with_the_passphrase.dat|(gpg --batch --no-tty --yes --passphrase-fd=0 --decrypt `basename $x`; ) > 'dump_content.dat'` done;</pre>
<p>
<strong><em>Show device drivers and their properties (Windows XP)</em></strong>
  <pre class="codeline"> driverquery /si /fo table</pre>
<p>
<strong><em>relabel current konsole tab</em></strong>
  <pre class="codeline"> alias rk='d=$(dcop|grep $PPID) && s=$(dcop $d konsole currentSession) && dcop $d $s renameSession'</pre>
<p>
<strong><em>Mortality Countdown</em></strong>
  <pre class="codeline"> while [ 0 ]; do expr 2365200000 \- `date +%s` \- `date --date "YYYY-mm-dd HH:MM:ss" +%s`; sleep 1; clear; done</pre>
<p>
<strong><em>set prompt and terminal title to display hostname, user ID and</em></strong>
  <pre class="codeline"> export PS1='\[\e]0;\h \u \w\a\]\n\[\e[0;34m\]\u@\h \[\e[33m\]\w\[\e[0;32m\]\n\$ '</pre>
<p>
<strong><em>Show one line summaries of all DEB packages installed on Ubuntu</em></strong>
  <pre class="codeline"> dpkg --list '*linux*' | grep '^ii'</pre>
<p>
<strong><em>Create a git alias that will pull and fast-forward the current</em></strong>
  <pre class="codeline"> git config --global --add alias.ff "pull --no-commit -v" ; git ff</pre>
<p>
<strong><em>Getting the ip address of eth0</em></strong>
  <pre class="codeline"> ifconfig eth0 | awk '/inet addr/ {split ($2,A,":"); print A[2]}'</pre>
<p>
<strong><em>tunnel vnc port</em></strong>
  <pre class="codeline"> ssh -L 5900:localhost:5900 user@exampleserver.com</pre>
<p>
<strong><em>Find public IP when behind a random router (also see description)</em></strong>
  <pre class="codeline"> alias pubip='GET http://www.whatismyip.com/automation/n09230945.asp && echo'</pre>
<p>
<strong><em>Remove multiple same rpm packages</em></strong>
  <pre class="codeline"> rpm -e --allmatches filename.rpm</pre>
<p>
<strong><em>KDE Mixer Master Mute/Unmute</em></strong>
  <pre class="codeline"> alias mute="dcop kmix Mixer0 toggleMasterMute\(\) ; dcop kmix Mixer0</pre>
<p>
<strong><em>Automatically create a rar archive</em></strong>
  <pre class="codeline"> rar a -m0 "${PWD##*/}.rar" *</pre>
<p>
<strong><em>Display the linux host infomation.</em></strong>
  <pre class="codeline"> hostinfo.sh</pre>
<p>
<strong><em>Recover cvs ": no such repository" error</em></strong>
  <pre class="codeline"> find ./* -name 'CVS' | awk '{print "dos2unix " $1 "/*"}' | awk '{system($0)}'</pre>
<p>
<strong><em>Recursive Ownership Change</em></strong>
  <pre class="codeline"> chown -cR --from=olduser:oldgroup newuser:newgroup *</pre>
<p>
<strong><em>Name a backup/archive file based on current date and time</em></strong>
  <pre class="codeline"> archivefile=filename-$(date +%Y%m%d-%H%M).tar.gz</pre>
<p>
<strong><em>Erase DVD RW</em></strong>
  <pre class="codeline"> dvd+rw-format /dev/dvd</pre>
<p>
<strong><em>Separates each frame of a animated gif file to a counted file,</em></strong>
  <pre class="codeline"> convert +adjoin animatedImage.gif test.gif ; convert +append test*.gif</pre>
<p>
<strong><em>convert hex to decimal ; decimal to hex</em></strong>
  <pre class="codeline"> echo 16i `echo "F" | tr '[a-f]' '[A-F]'` p | dc ; echo 16o "15" p | dc</pre>
<p>
<strong><em>Better recursive grep with pretty colors... requires ruby and gems</em></strong>
  <pre class="codeline"> rak "what you're searching for" dir/path</pre>
<p>
<strong><em>Change size of lots of image files.</em></strong>
  <pre class="codeline"> for File in *.jpg; do mogrify -resize 1024 -quality 96 $File; done</pre>
<p>
<strong><em>list file descriptors opened by a process</em></strong>
  <pre class="codeline"> ls -al /proc/<PID>/fd</pre>
<p>
<strong><em>finding more large files</em></strong>
  <pre class="codeline"> find / -xdev -size +1024 -exec ls -al {} \; | sort -r -k 5</pre>
<p>
<strong><em>cloning partition tables under Solaris</em></strong>
  <pre class="codeline"> prtvtoc /dev/rdsk/c0t0d0s2 | fmthard -s - /dev/rdsk/c0t1d0s2</pre>
<p>
<strong><em>E-mail a traditional Berkeley mbox to another recipient as</em></strong>
  <pre class="codeline"> formail -Y -s /usr/sbin/sendmail bar@example.com < /var/mail/foo</pre>
<p>
<strong><em>Prepare B&W scans for clean looking, searchable PDF</em></strong>
  <pre class="codeline"> convert pagexxx.png -filter Cubic -resize 200% -threshold 50% -compress Group4 pagexxx.tiff; tiff2pdf -z -p letter -ro -x 1200 -y 1200 -o pagexxx.pdf pagexxx.tiff</pre>
<p>
<strong><em>Periodic Log Deletion</em></strong>
  <pre class="codeline"> find /path/to/dir -type f -mtime +[#] -exec rm -f {} \;</pre>
<p>
<strong><em>in-place search/replace with datestamped backup</em></strong>
  <pre class="codeline"> sed -i.`date +%Y%m%d` -e 's/pattern/replace' [filename]</pre>
<p>
<strong><em>Unrar multiple directories into current working directory</em></strong>
  <pre class="codeline"> for x in */*.rar; do unrar x $x; done</pre>
<p>
<strong><em>Get line count for any file ending with extension recursively</em></strong>
  <pre class="codeline"> find . -name "*.py" | xargs wc -l</pre>
<p>
<strong><em>Search for in which package the specified file is included.</em></strong>
  <pre class="codeline"> /bin/rpm -qf /etc/passwd /etc/issue /etc/httpd/conf/httpd.conf</pre>
<p>
<strong><em>List just the executable files (or directories) in current</em></strong>
  <pre class="codeline"> ls -dF `find . -maxdepth 1 \( -perm -1 -o \( -perm -10 -o -perm -100 \) \) -print`</pre>
<p>
<strong><em>[WinXP]Use as a shortcut in the SendTo menu to open a cmd window</em></strong>
  <pre class="codeline"> C:\WINDOWS\system32\cmd.exe /t:0A /k cd /d</pre>
<p>
<strong><em>Function to remove a directory from your PATH</em></strong>
  <pre class="codeline"> pathrm() { PATH=`echo $PATH | sed -e "s=^${1}:==;s=:${1}$==;s=:${1}:=:="`; }</pre>
<p>
<strong><em>move all the .bak backup copies to their original names (rename</em></strong>
  <pre class="codeline"> for i in *.bak ; do nuname=`echo $i | sed 's/\.[^\.]*$//'`; echo renaming $i to $nuname;mv $i $nuname; done</pre>
<p>
<strong><em>Print all lines in a file that are not a certain length</em></strong>
  <pre class="codeline"> awk 'length($0)!=12 {print}' your_file_name</pre>
<p>
<strong><em>Creates a SSHFS volume on MacOS X (better used as an alias). Needs</em></strong>
  <pre class="codeline"> mkdir /Volumes/sshdisk 2> /dev/null; sshfs user@server:/</pre>
<p>
<strong><em>Get me yesterday's date, even if today is 1-Mar-2008 and yesterday</em></strong>
  <pre class="codeline"> TZ=XYZ24 date</pre>
<p>
<strong><em>look for a function reference in a library set</em></strong>
  <pre class="codeline"> nm --defined-only --print-file-name lib*so 2>/dev/null | grep ' pthread_create$'</pre>
<p>
<strong><em>diff directories, quick cut and paste to view the changes</em></strong>
  <pre class="codeline"> diff -q dir1/ dir2/ | grep differ | awk '{ print "vimdiff " $2 " " $4 }'</pre>
<p>
<strong><em>Flush and then immediately start watching a file</em></strong>
  <pre class="codeline"> (> errors.log) && tail -f !^</pre>
<p>
<strong><em>Every Nth line position # (AWK)</em></strong>
  <pre class="codeline"> awk '{if (NR % 3 == 1) print $0}' foo > foo_every3_position1; awk '{if (NR % 3 == 2) print $0}' foo > foo_every3_position2; awk '{if (NR % 3 == 0) print $0}' foo > foo_every3_position3</pre>
<p>
<strong><em>Fix borked character coding in a tty.</em></strong>
  <pre class="codeline"> LC_ALL=C man -c man</pre>
<p>
<strong><em>use sed to simulate rpad and lpad</em></strong>
  <pre class="codeline"> ls / | sed -e :a -e 's/^.\{1,15\}$/&_/;ta'</pre>
<p>
<strong><em>Print a row of 50 hyphens</em></strong>
  <pre class="codeline"> echo - | sed -e :a -e 's/^.\{1,50\}$/&-/;ta'</pre>
<p>
<strong><em>Print a row of 50 hyphens</em></strong>
  <pre class="codeline"> jot -s '' -b '-' 50</pre>
<p>
<strong><em>Prints line numbers</em></strong>
  <pre class="codeline"> nl <filename></pre>
<p>
<strong><em>Determine what process is listening on a port on Solaris, without</em></strong>
  <pre class="codeline"> for x in `ptree | awk '{print $1}'`; do pfiles $x | grep ${PORT} > /dev/null 2>&1; if [ x"$?" == "x0" ]; then ps -ef | grep $x | grep -v grep; fi; done 2> /dev/null</pre>
<p>
<strong><em>Change your swappiness Ratio under linux</em></strong>
  <pre class="codeline"> echo 50 > /proc/sys/vm/swappiness</pre>
<p>
<strong><em>paged 'ls' in color</em></strong>
  <pre class="codeline"> ls -lah --color=always | most</pre>
<p>
<strong><em>Manage "legacy" service run control links</em></strong>
  <pre class="codeline"> sudo find /etc/rc{1..5}.d -name S99myservice -type l -exec sh -c 'NEWFN=`echo {} | sed 's/S99/K99/'` ; mv -v {} $NEWFN' \;</pre>
<p>
<strong><em>Replace spaces in filename</em></strong>
  <pre class="codeline"> for i in *\ *; do if [ -f "$i" ]; then mv "$i" ${i// /_}; fi; done</pre>
<p>
<strong><em>Replace strings in files</em></strong>
  <pre class="codeline"> sed -i -e 's/war/peace/g' *</pre>
<p>
<strong><em>Shorten url using bit.ly API</em></strong>
  <pre class="codeline"> curl -s --data-urlencode 'longUrl='$1 --data-urlencode 'login='$login --data-urlencode 'apiKey='$apikey 'http://api.bit.ly/shorten?version=2.0.1&format=xml' | xmlstarlet sel -T -t -m "//shortUrl" -v "." | line</pre>
<p>
<strong><em>Find and print pattern location from all files on command line</em></strong>
  <pre class="codeline"> find . -exec grep $foo {} \; -print</pre>
<p>
<strong><em>Display the output of a command from the first line until the</em></strong>
  <pre class="codeline"> command | sed '/regex/q'</pre>
<p>
<strong><em>Read just the IP address of a device</em></strong>
  <pre class="codeline"> ifconfig -l | xargs -n1 ipconfig getifaddr 2> /dev/null</pre>
<p>
<strong><em>Display the list of all opened tabs from Firefox via a python</em></strong>
  <pre class="codeline"> python <<< $'import minjson\nf = open("sessionstore.js", "r")\njdata = minjson.read(f.read())\nf.close()\nfor win in jdata.get("windows"):\n\tfor tab in win.get("tabs"):\n\t\ti = tab.get("index") - 1\n\t\tprint tab.get("entries")[i].get("url")'</pre>
<p>
<strong><em>Pause and Resume Processes</em></strong>
  <pre class="codeline"> stop () { ps -ec | grep $@ | kill -SIGSTOP `awk '{print $1}'`; }</pre>
<p>
<strong><em>upload a file via ftp</em></strong>
  <pre class="codeline"> curl -u user:passwd -T /home/dir/local_file_to_upload ftp://your_host.com/subdir/</pre>
<p>
<strong><em>floating point shell calculator</em></strong>
  <pre class="codeline"> calc() { awk 'BEGIN { OFMT="%f"; print '"$*"'; exit}'; }</pre>
<p>
<strong><em>show how much diskspace all images in a given directory need</em></strong>
  <pre class="codeline"> find /home/bubo/ -type f \( -iname \*.jpg -print0 , -iname \*.png -print0 , -iname \*gif -print0 \) | du -cm --files0-from - | tail -1</pre>
<p>
<strong><em>floating point bash calculator w/o precision</em></strong>
  <pre class="codeline"> b(){ echo "scale=${2:-2}; $1" | bc -l; }</pre>
<p>
<strong><em>copy partition table from /dev/sda to /dev/sdb</em></strong>
  <pre class="codeline"> sfdisk -d /dev/sda | sed 's/sda/sdb/g' | sfdisk /dev/sdb</pre>
<p>
<strong><em>Display the output of a command from the first line until the</em></strong>
  <pre class="codeline"> <your command here> | perl -n -e 'print "$_" if 1 ... /<regex>/;'</pre>
<p>
<strong><em>Randomize lines (opposite of | sort), or use 'shuf' better.</em></strong>
  <pre class="codeline"> perl -wl -e '@f=<>; for $i (0 .. $#f) { $r=int rand ($i+1); @f[$i, $r]=@f[$r,$i] if ($i!=$r); } chomp @f; print join $/, @f;' try.txt</pre>
<p>
<strong><em>Sum process information using awk</em></strong>
  <pre class="codeline"> ps -ylC httpd --sort:rss | awk '{ SUM += $8 } END { print SUM/1024 }'</pre>
<p>
<strong><em>monitor the kernel ring buffer</em></strong>
  <pre class="codeline"> watch 'dmesg | tail -15'</pre>
<p>
<strong><em>parse an rpm name into its components - fast</em></strong>
  <pre class="codeline"> parse_rpm() { RPM=$1;B=${RPM##*/};B=${B%.rpm};A=${B##*.};B=${B%.*};R=${B##*-};B=${B%- *};V=${B##*-};B=${B%-*};N=$B;echo "$N $V $R $A"; }</pre>
<p>
<strong><em>display embeded comments for every --opt, usefull for auto</em></strong>
  <pre class="codeline"> vim -n -es -c 'g/# CommandParse/+2,/^\s\+esac/-1 d p | % d | put p | %<' -c 'g/^\([-+]\+[^)]\+\))/,/^\(\s\+[^- \t#]\|^$\)/-1 p' -c 'q!' $0</pre>
<p>
<strong><em>p is for pager</em></strong>
  <pre class="codeline"> p() { l=$LINES; case $1 in do) shift; IFS=$'\n' _pg=( $("$@") ) && _pgn=0 && p r;; r) echo "${_pg[*]:_pgn:$((l-4))}";; d) (( _pgn+=l-4 )); (( _pgn=_pgn>=${#_pg[@]}?${#_pg[@]}-l+4:_pgn )); p r;; u) (( _pgn=_pgn<=l-4?0:_pgn-$l-4 )); p r;; esac; }</pre>
<p>
<strong><em>Sometimes you just want a quick way to find out if a certain user</em></strong>
  <pre class="codeline"> getent shadow | grep '^[^:]\+:!' | cut -d: -f1</pre>
<p>
<strong><em>Sometimes you just want a quick way to find out if a certain user</em></strong>
  <pre class="codeline"> getent shadow | while IFS=: read a b c; do grep -q '!' <<< "$b" && echo "$a LOCKED" || echo "$a not locked"; done</pre>
<p>
<strong><em>Sometimes you just want a quick way to find out if a certain user ??</em></strong>
  <pre class="codeline"> awk -F":" '{ print $1 }' /etc/passwd | while read UU ; do STATUS=$(passwd -S ${UU} | grep locked 2>/dev/null) ; if [[ ! -z ${STATUS} ]] ; then echo "Account ${UU} is locked." ; fi ; done</pre>
<p>
<strong><em>report what tape is in autoloader mailslot (using barcode label)</em></strong>
  <pre class="codeline"> mtx -f /dev/sg13 status | grep EXPORT | cut -c 56-63</pre>
<p>
<strong><em>count and number lines of output, useful for counting number of</em></strong>
  <pre class="codeline"> ps aux | grep [h]ttpd | cat -n</pre>
<p>
<strong><em>Create SSH key exchange from one host to the other</em></strong>
  <pre class="codeline"> cat ~/.ssh/id_rsa.pub | ssh <remote_host> "xargs --null echo >> ~/.ssh/authorized_keys"</pre>
<p>
<strong><em>Check general system error on AIX</em></strong>
  <pre class="codeline"> errpt -a | more</pre>
<p>
<strong><em>Send a local file via email</em></strong>
  <pre class="codeline"> { echo -e "$body"; uuencode "$outfile" "$outfile"; } | mail -s "$subject" "$destaddr" ;</pre>
<p>
<strong><em>display / view the contents of the manifest within a java jar</em></strong>
  <pre class="codeline"> $ unzip -p some-jar-file.jar META-INF/MANIFEST.MF</pre>
<p>
<strong><em>Optimal way of deleting huge numbers of files</em></strong>
  <pre class="codeline"> find /path/to/dir/ -type f -exec rm {} +</pre>
<p>
<strong><em>Extract all urls from the last firefox<a href="sessionstore.js"><em><code>sessionstore.js</code></em></a> used.</em></strong>
  <pre class="codeline"> grep -oP '"url":"\K[^"]+' $(ls -t ~/.mozilla/firefox/*/sessionstore.js | sed q)</pre>
<p>
<strong><em>Router discovery</em></strong>
  <pre class="codeline"> traceroute 2>/dev/null -n google.com | awk '/^ *1/{print $2;exit}'</pre>
<p>
<strong><em>prints line numbers</em></strong>
  <pre class="codeline"> perl -pe 'print "$. "' <file></pre>
<p>
<strong><em>Use curl on Windows to bulk-download the Savitabhabhi Comic Strip</em></strong>
  <pre class="codeline"> for /L %%x in (1,1,16) do mkdir %%x & curl -R -e http://www.kirtu.com</pre>
<p>
<strong><em>Know when you will type :q in your term instead of vi(m), the</em></strong>
  <pre class="codeline"> alias :q='tput setaf 1; echo >&2 "this is NOT vi(m) :/"; tput sgr0'</pre>
<p>
<strong><em>Salvage a borked terminal</em></strong>
  <pre class="codeline"> echo <ctrl+v><ctrl+o><enter></pre>
<p>
<strong><em>Search trought pidgin's conversation logs for "searchterm", and</em></strong>
  <pre class="codeline"> grep -Ri searchterm ~/.purple/logs/* | sed -e 's/<.*?>//g'</pre>
<p>
<strong><em>Shorten url with is.gd using curl, perl</em></strong>
  <pre class="codeline"> curl -s "http://is.gd/api.php?longurl=[long_url]"</pre>
<p>
<strong><em>get MAC address of default gateway</em></strong>
  <pre class="codeline"> netstat -nr | awk 'BEGIN {while ($1!="default") getline; g=$2;} (g==$1) {print $2; exit;}'</pre>
<p>
<strong><em>Finding all numbers that are bigger then 1 in vim</em></strong>
  <pre class="codeline"> /^\([2-9]\d*\|1\d+\)</pre>
<p>
<strong><em>Show Network IP and Subnet</em></strong>
  <pre class="codeline"> ipcalc $(ifconfig eth0 | grep "inet addr:" | cut -d':' -f2,4 | sed 's/.+Bcast:/\//g') | awk '/Network/ { print $2 } '</pre>
<p>
<strong><em>Router discovery</em></strong>
  <pre class="codeline"> awk 'NR==2 {print $1}' /proc/net/arp</pre>
<p>
<strong><em>Return IP Address</em></strong>
  <pre class="codeline"> ifconfig -a| awk 'BEGIN{FS="[ :]+"} /Bcast/{print $4}'</pre>
<p>
<strong><em>Make ABBA better (requires firefox)</em></strong>
  <pre class="codeline"> wget -O - -q http://www.azlyrics.com/lyrics/abba/takeachanceonme.html | sed -e 's/[cC]hance/dump/g' > ~/tdom.htm && firefox ~/tdom.htm</pre>
<p>
<strong><em>get daily wizard of id comic</em></strong>
  <pre class="codeline"> curl -o id.gif `date +http://d.yimg.com/a/p/uc/%Y%m%d/largeimagecrwiz%y%m%d.gif`</pre>
<p>
<strong><em>View the newest xkcd comic.</em></strong>
  <pre class="codeline"> lynx --dump --source http://www.xkcd.com | grep `lynx --dump http://www.xkcd.com | egrep '(png|jpg)'` | grep title | cut -d = -f2,3 | cut -d '"' -f2,4 | sed -e 's/"/|/g' | awk -F"|" ' { system("display " $1);system("echo "$2); } '</pre>
<p>
<strong><em>Watch those evil Red Hat states code D Uninterruptible sleep</em></strong>
  <pre class="codeline"> watch -n 1 "ps aux | sed -n 's/ D /&/p'"</pre>
<p>
<strong><em>lists contents of a tar file</em></strong>
  <pre class="codeline"> tar -tf /path/to/file.tar</pre>
<p>
<strong><em>generate random password</em></strong>
  <pre class="codeline"> cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 10 | sed 1q</pre>
<p>
<strong><em>use the short username by default for network authentication</em></strong>
  <pre class="codeline"> defaults write /Library/Preferences/com.apple.NetworkAuthorization UseShortName -bool YES</pre>
<p>
<strong><em>Match a URL</em></strong>
  <pre class="codeline"> echo "(Something like http://foo.com/blah_blah)" | grep -oP "\b(([\w-]+://?|www[.])[^\s()<>]+(?:\([\w\d]+\)|([^[:punct:]\s]|/)))"</pre>
<p>
<strong><em>Run command in an ftp session</em></strong>
  <pre class="codeline"> ftp>!w</pre>
<p>
<strong><em>configify the list of gems on ur machine. the quick hack</em></strong>
  <pre class="codeline"> gem list --local | python -c "import sys;import re;l=sys.stdin.readlines();x=['config.gem \"'+line[:-1][:line.index(' ')] + '\" , ' +line[:-1][line.index(' '):].replace('(',':version => \"').replace(')','')+'\"' for line in l];print '\n'.join(x)"</pre>
<p>
<strong><em>configify the list of gems on ur machine. the quick hack</em></strong>
  <pre class="codeline"> gem list --local | python -c "import sys;import</pre>
<p>
<strong><em>Find in all files in the current directory, just a find shorthand</em></strong>
  <pre class="codeline"> grep -H -n "pattern" *</pre>
<p>
<strong><em>Find all dot files and directories</em></strong>
  <pre class="codeline"> printf "%s\n" .*</pre>
<p>
<strong><em>List the biggest accessible files/dirs in current directory,</em></strong>
  <pre class="codeline"> du -ms * 2>/dev/null |sort -nr|head</pre>
<p>
<strong><em>Make all GUI stuff show up on the display connected to the</em></strong>
  <pre class="codeline"> DISPLAY=:0.0; export DISPLAY</pre>
<p>
<strong><em>Show sorted list of files with sizes more than 1MB in the current</em></strong>
  <pre class="codeline"> find . -maxdepth 1 -type f -size +1M -printf "%f:%s\n" | sort -t":" -k2</pre>
<p>
<strong><em>Get your external IP address</em></strong>
  <pre class="codeline"> wget -qO - http://www.sputnick-area.net/ip;echo</pre>
<p>
<strong><em>force change password for all user</em></strong>
  <pre class="codeline"> while IFS=: read u x; do passwd -e "$u"; done < /etc/passwd</pre>
<p>
<strong><em>Find all dot files and directories</em></strong>
  <pre class="codeline"> printf "%s\n" .*</pre>
<p>
<strong><em>Testing reading speed with dd</em></strong>
  <pre class="codeline"> sync; time `dd if=/dev/cciss/c0d1p1 of=/dev/null bs=1M count=10240`</pre>
<p>
<strong><em>Testing writing speed with dd</em></strong>
  <pre class="codeline"> sync; time `dd if=/dev/zero of=bigfile bs=1M count=2048 && sync`</pre>
<p>
<strong><em>Better "hours of video" summary for each file/dir in the current</em></strong>
  <pre class="codeline"> for item in *;do echo -n "$item - ";find "$item" -type f -print0 | xargs -0 file -iNf - | grep video | cut -d: -f1 | xargs -d'\n' /usr/share/doc/mplayer/examples/midentify | grep ID_LENGTH | awk -F= '{sum+=$2} END {print(sum/60)}'; done | grep -v ' - 0$'</pre>
<p>
<strong><em>Find all videos under current directory</em></strong>
  <pre class="codeline"> find ./ -type f -print0 | xargs -0 file -iNf - | grep video | cut -d: -f1</pre>
<p>
<strong><em>force change password for all user</em></strong>
  <pre class="codeline"> getent passwd|cut -d: -f1|xargs -n1 passwd -e</pre>
<p>
<strong><em>system beep off</em></strong>
  <pre class="codeline"> setterm -bfreq 0</pre>
<p>
<strong><em>Provides external IP, Country and City in a formated manner.</em></strong>
  <pre class="codeline"> geoip () { curl -s "http://www.geoiptool.com/?IP=$1" | html2text | egrep --color 'City:|IP Address:|Country:' }</pre>
<p>
<strong><em>geoip information</em></strong>
  <pre class="codeline"> geoiplookup www.commandlinefu.com</pre>
<p>
<strong><em>shell function to turn start and length in to a range suitable</em></strong>
  <pre class="codeline"> range () { end=$(echo "$1 + $2 - 1" | bc); echo "$1-$end"; }</pre>
<p>
<strong><em>slice a fixed number of characters from the output of a command,</em></strong>
  <pre class="codeline"> slice(){ cut -c$((${#1}+1))-; }; ls -l | slice "-rw-r--r--"</pre>
<p>
<strong><em>slice a fixed number of characters from the output of a command,</em></strong>
  <pre class="codeline"> slice="-rw-r--r-- "; ls -l | cut -c $(echo "$slice" | wc -c)-</pre>
<p>
<strong><em>Be notified about overheating of your CPU and/or motherboard</em></strong>
  <pre class="codeline"> sensors | grep "Core 1" | [[ `sed -e 's/^.*+\([0-9]\{2,3\}\).*(.*/\1/'` -gt 50 ]] && notify-send "Core 1 temperature exceeds 50 degrees"</pre>
<p>
<strong><em>Start the x11vnc server</em></strong>
  <pre class="codeline"> x11vnc -display :0 -scale 6/7 -rfbauth vncpass -forever</pre>
<p>
<strong><em>Create x11vnc server authentication file</em></strong>
  <pre class="codeline"> x11vnc -storepasswd your_new_apssword ~/my_vnc_pass</pre>
<p>
<strong><em>List all available commands</em></strong>
  <pre class="codeline"> in bash hit "tab" twice and answer y</pre>
<p>
<strong><em>geoip information</em></strong>
  <pre class="codeline"> geo(){ curl -s "http://www.geody.com/geoip.php?ip=$(dig +short $1)"| sed '/^IP:/!d;s/<[^>][^>]*>//g'; }</pre>
<p>
<strong><em>Show a script or config file without comments</em></strong>
  <pre class="codeline"> egrep -v "^[[:blank:]]*($|#|//|/\*| \*|\*/)" somefile</pre>
<p>
<strong><em>Create a simple backup</em></strong>
  <pre class="codeline"> tar pzcvf /result_path/result.tar.gz /target_path/target_folder</pre>
<p>
<strong><em>Sort movies by length, longest first</em></strong>
  <pre class="codeline"> for i in *.avi; do echo -n "$i:";mediainfo $i|head | grep PlayTime | cut -d: -f2 ; done | sort -t: -k2 -r</pre>
<p>
<strong><em>Get ethX mac addresses</em></strong>
  <pre class="codeline"> ifconfig | awk '/HW/ {print $5}'</pre>
<p>
<strong><em>Get ethX mac addresses</em></strong>
  <pre class="codeline"> ip link show eth0 | grep "link/ether" | awk '{print $2}'</pre>
<p>
<strong><em>Build an exhaustive list of maildir folders for mutt</em></strong>
  <pre class="codeline"> find ~/Maildir/ -mindepth 1 -type d | egrep -v '/cur$|/tmp$|/new$' | xargs</pre>
<p>
<strong><em>Sum of the total resident memory Stainless.app is using.</em></strong>
  <pre class="codeline"> ps -ec -o command,rss | grep Stainless | awk -F ' ' '{ x = x + $2 } END { print x/(1024) " MB."}'</pre>
<p>
<strong><em>Attach all discovered iscsi nodes</em></strong>
  <pre class="codeline"> iscsiadm -m node -l</pre>
<p>
<strong><em>Add the sbin directories to your PATH if it doesn't already exist</em></strong>
  <pre class="codeline"> path+=( /sbin /usr/sbin /usr/local/sbin ); path=( ${(u)path} );</pre>
<p>
<strong><em>Search vmware vmx files if Linux guests are set to sync time to</em></strong>
  <pre class="codeline"> for x in `find /vmfs/volumes/ -name *vmx -exec grep -H linux.iso {} \; |cut -d : -f 1`; do echo $x; grep -i sync $x; done;</pre>
<p>
<strong><em>Empty a file</em></strong>
  <pre class="codeline"> > [filename].txt</pre>
<p>
<strong><em>Erase empty files</em></strong>
  <pre class="codeline"> find . -size 0 -exec rm '{}' \;</pre>
<p>
<strong><em>Receiving alerts about commands who exit with failure</em></strong>
  <pre class="codeline"> export PROMPT_COMMAND='( x=$? ; let x!=0 && echo shell returned $x )'</pre>
<p>
<strong><em>Tweet my ip ( see your machine ip on twitter )</em></strong>
  <pre class="codeline"> STAT=`curl http://www.whatismyip.org/`; curl -u YourUserName:YourPassword -d status=$STAT http://twitter.com/statuses/update.xml</pre>
<p>
<strong><em>Calculating series with awk: add numbers from 1 to 100</em></strong>
  <pre class="codeline"> awk 'BEGIN {for(i=1;i<=100;i++)sum+=i}; END {print sum}' /dev/null</pre>
<p>
<strong><em>Get ssh server fingerprints</em></strong>
  <pre class="codeline"> ssh-keygen -l -f /etc/ssh/ssh_host_rsa_key.pub && ssh-keygen -l -f /etc/ssh/ssh_host_dsa_key.pub</pre>
<p>
<strong><em>Remove CR LF from a text file</em></strong>
  <pre class="codeline"> flip -u $FILE</pre>
<p>
<strong><em>Prints per-line contribution per author for a GIT repository</em></strong>
  <pre class="codeline"> git ls-files | while read i; do git blame $i | sed -e 's/^[^(]*(//' -e 's/^\([^[:digit:]]*\)[[:space:]]\+[[:digit:]].*/\1/'; done | sort | uniq -ic | sort -nr</pre>
<p>
<strong><em>grep -v with multiple patterns.</em></strong>
  <pre class="codeline"> sed -n '/test/{/error\|critical\|warning/d;p}' somefile</pre>
<p>
<strong><em>view the system console remotely</em></strong>
  <pre class="codeline"> sudo cat /dev/vcs1 | fold -w 80</pre>
<p>
<strong><em>Download all Delicious bookmarks</em></strong>
  <pre class="codeline"> curl -u username -o bookmarks.xml https://api.del.icio.us/v1/posts/all</pre>
<p>
<strong><em>Schedule a download at a later time</em></strong>
  <pre class="codeline"> echo 'wget url' | at 01:00</pre>
<p>
<strong><em>Add calendar to desktop wallpaper</em></strong>
  <pre class="codeline"> convert -font -misc-fixed-*-*-*-*-*-*-*-*-*-*-*-* -fill black -draw "text 270,260 \" `cal` \"" testpic.jpg newtestpic.jpg</pre>
<p>
<strong><em>create dir tree</em></strong>
  <pre class="codeline"> mkdir -p doc/{text/,img/{wallpaper/,photos/}}</pre>
<p>
<strong><em>Backup your hard drive with dd</em></strong>
  <pre class="codeline"> sudo dd if=/dev/sda of=/media/disk/backup/sda.backup</pre>
<p>
<strong><em>Quick glance at who's been using your system recently</em></strong>
  <pre class="codeline"> last | grep -v "^$" | awk '{ print $1 }' | sort -nr | uniq -c</pre>
<p>
<strong><em>Get Dell Service Tag Number from a Dell Machine</em></strong>
  <pre class="codeline"> sudo dmidecode | grep Serial\ Number | head -n1</pre>
<p>
<strong><em>change directory to actual path instead of symlink path</em></strong>
  <pre class="codeline"> cd `pwd -P`</pre>
<p>
<strong><em>Generat a Random MAC address</em></strong>
  <pre class="codeline"> MAC=`(date; cat /proc/interrupts) | md5sum | sed -r 's/^(.{10}).*$/\1/; s/([0-9a-f]{2})/\1:/g; s/:$//;'`</pre>
<p>
<strong><em>Get list of servers with a specific port open</em></strong>
  <pre class="codeline"> nmap -sT -p 80 -oG - 192.168.1.* | grep open</pre>
<p>
<strong><em>ssh tunnel with auto reconnect ability</em></strong>
  <pre class="codeline"> while [ ! -f /tmp/stop ]; do ssh -o ExitOnForwardFailure=yes -R 2222:localhost:22 target "while nc -zv localhost 2222; do sleep 5; done"; sleep 5;done</pre>
<p>
<strong><em>Use last argument of last command</em></strong>
  <pre class="codeline"> file !$</pre>
<p>
<strong><em>A function to output a man page as a pdf file</em></strong>
  <pre class="codeline"> function man2pdf(){ man -t ${1:?Specify man as arg} | ps2pdf -dCompatibility=1.3 - - > ${1}.pdf; }</pre>
<p>
<strong><em>Save current layout of top</em></strong>
  <pre class="codeline"> <Shift + W></pre>
<p>
<strong><em>Mount the first NTFS partition inside a VDI file (VirtualBox Disk</em></strong>
  <pre class="codeline"> mount -t ntfs-3g -o ro,loop,uid=user,gid=group,umask=0007,fmask=0117,offset=0x$(hd -n 1000000 image.vdi | grep "eb 52 90 4e 54 46 53" | cut -c 1-8) image.vdi /mnt/vdi-ntfs</pre>
<p>
<strong><em>Use all the cores or CPUs when compiling</em></strong>
  <pre class="codeline"> make -j 4</pre>
<p>
<strong><em>Create a list of binary numbers</em></strong>
  <pre class="codeline"> echo {0..1}{0..1}{0..1}{0..1}</pre>
<p>
<strong><em>Create a system overview dashboard on F12 key</em></strong>
  <pre class="codeline"> bind '"\e[24~"':"\"ps -elF;df -h;free -mt;netstat -lnpt;who -a\C-m"""</pre>
<p>
<strong><em>Find last reboot time</em></strong>
  <pre class="codeline"> who -b</pre>
<p>
<strong><em>Start screen in detached mode</em></strong>
  <pre class="codeline"> screen -d -m [<command>]</pre>
<p>
<strong><em>Ctrl+S Ctrl+Q terminal output lock and unlock</em></strong>
  <pre class="codeline"> Ctrl+S Ctrl+Q</pre>
<p>
<strong><em>StopWatch, toilet version, amazing format inside terminal</em></strong>
  <pre class="codeline"> export I=$(date +%s); watch -t -n 1 'T=$(date</pre>
<p>
<strong><em>StopWatch, OnScreen version, blinking shily on all desktops</em></strong>
  <pre class="codeline"> export I=$(date +%s); watch -t -n 1 'T=$(date +%s);E=$(($T-$I));hours=$((E / 3600)) ; seconds=$((E % 3600)) ; minutes=$((seconds / 60)) ; seconds=$((seconds % 60)) ; echo $(printf "%02d:%02d:%02d" $hours $minutes $seconds) | osd_cat -o 20 -d 1 -p bottom'</pre>
<p>
<strong><em>Show bash's function definitions you defined in .bash_profile or</em></strong>
  <pre class="codeline"> declare -f [ function_name ]</pre>
<p>
<strong><em>List bash functions defined in .bash_profile or .bashrc</em></strong>
  <pre class="codeline"> set | fgrep " ()"</pre>
<p>
<strong><em>Show current folder permission recursively from /, useful for</em></strong>
  <pre class="codeline"> pushd .> /dev/null; cd /; for d in `echo $OLDPWD | sed -e 's/\// /g'`; do cd $d; echo -n "$d "; ls -ld .; done; popd >/dev/null</pre>
<p>
<strong><em>Convert Windows/DOS Text Files to Unix</em></strong>
  <pre class="codeline"> flip -u <filenames></pre>
<p>
<strong><em>create a motion jpeg (MJPEG) with the jpg file from current</em></strong>
  <pre class="codeline"> mencoder mf://image1.jpg,image2.jpg,image3.jpg -mf w=800:h=600:fps=25:type=jpeg -ovc copy -oac copy -o output.avi</pre>
<p>
<strong><em>Create a mpeg4 video from a jpeg picture sequence (e.g. for pencil</em></strong>
  <pre class="codeline"> mencoder mf://*.jpg -mf w=800:h=600:fps=25:type=jpeg -ovc lavc -lavcopts vcodec=mpeg4 -oac copy -o output.avi</pre>
<p>
<strong><em>Command to build one or more network segments - with for</em></strong>
  <pre class="codeline"> seg() { for b in $(echo $1); do for x in $(seq 10); do echo $b.$x; done; done }</pre>
<p>
<strong><em>Command to build one or more network segments - with while</em></strong>
  <pre class="codeline"> seg() { echo -e "$1" | while read LINE; do for b in $(seq 10); do echo $LINE.$b; done; done; }</pre>
<p>
<strong><em>Partition a new disk as all one partition tagged as "LInux LVM"</em></strong>
  <pre class="codeline"> echo -e "n\np\n1\n\n\nt\n8e\nw" | fdisk /dev/sdX</pre>
<p>
<strong><em>Set default "New Page" as HTML in TextMate</em></strong>
  <pre class="codeline"> defaults write com.macromates.textmate OakDefaultLanguage 17994EC8-6B1D-11D9-AC3A-000D93589AF6</pre>
<p>
<strong><em>encode image to base64 and copy to clipboard</em></strong>
  <pre class="codeline"> uuencode -m $1 /dev/stdout | sed '1d' | sed '$d' | tr -d '\n' | xclip -selection clipboard</pre>
<p>
<strong><em>Battery real life energy vs predicted remaining plotted</em></strong>
  <pre class="codeline"> echo start > battery.txt; watch -n 60 'date >> battery.txt ; acpi -b >> battery.txt'</pre>
<p>
<strong><em>Do you really believe on Battery Remaining Time? Confirm it from</em></strong>
  <pre class="codeline"> echo start > battery.txt; watch -n 60 'date >> battery.txt'</pre>
<p>
<strong><em>Get decimal ascii code from character</em></strong>
  <pre class="codeline"> ord() { printf "%d\n" "'$1"; }</pre>
<p>
<strong><em>Testing hard disk writing speed</em></strong>
  <pre class="codeline"> time dd if=/dev/zero of=TEST bs=4k count=512000</pre>
<p>
<strong><em>record alexa traffic stats of your website</em></strong>
  <pre class="codeline"> x=1 ; while [ $x -le 10 ] ; do lynx -dump http://www.alexa.com/siteinfo/http://[YOUR WEBSITE] | grep Global | sed 's/ \|Global\|\,//g' >> /var/log/alexa-stats.txt ; sleep 5h ; done &</pre>
<p>
<strong><em>Get decimal ascii code from character</em></strong>
  <pre class="codeline"> ord () { seq 1 127 | while read i; do echo `chr $i` $i; done | grep "^$1 " | cut -c '3-' }</pre>
<p>
<strong><em>google search</em></strong>
  <pre class="codeline"> perl -e '$i=0;while($i<10){open(WGET,qq/|xargs lynx -dump/);printf WGET qq{http://www.google.com/search?q=site:g33kinfo.com&hl=en&start=$i&sa=N },$i+=10}'|grep '\/\/g33kinfo.com\/'</pre>
<p>
<strong><em>Set gnome wallpaper to a random jpg from the specified directory</em></strong>
  <pre class="codeline"> gconftool -t str -s /desktop/gnome/background/picture_filename "`find /DIR_OF_JPGS -name '*.jpg' | shuf -n 1`"</pre>
<p>
<strong><em>A quick shell command to weed out the small wallpapers</em></strong>
  <pre class="codeline"> for i in ~/Desktop/Personal/Wallpapers/*.jpg ; { size=$((`identify -format "%wx%h" $i | sed 's/x/*/'`)) ; if [[ $size -lt 800001 ]] then ; rm -f "$i" ; fi; }</pre>
<p>
<strong><em>GZip all files in a directory separately</em></strong>
  <pre class="codeline"> for file in *.foo; do gzip "$file"; done</pre>
<p>
<strong><em>Command to resolve name from Ip address, passing only the last</em></strong>
  <pre class="codeline"> seq 4|xargs -n1 -i bash -c "echo -n 164.85.216.{} - ; nslookup 164.85.216.{} |grep name"|tr -s ' ' ' '|awk '{print $1" - "$5}'|sed 's/.$//'</pre>
<p>
<strong><em>Monitor a file's size</em></strong>
  <pre class="codeline"> while [ 1 ]; do du /var/log/messages;sleep 60; done</pre>
<p>
<strong><em>Check if variable is a number</em></strong>
  <pre class="codeline"> echo $X | egrep "^[0-9]+$"</pre>
<p>
<strong><em>View firewall config including devices on linux w/netfilter</em></strong>
  <pre class="codeline"> iptables -L -n -v</pre>
<p>
<strong><em>sort lines by length</em></strong>
  <pre class="codeline"> awk '{print length, $0;}' | sort -nr</pre>
<p>
<strong><em>Get last changed revision to all eclipse projects in a SVN working</em></strong>
  <pre class="codeline"> find . -iname ".project"| xargs -I {} dirname {} | LC_ALL=C xargs -I {} svn info {} | grep "Last Changed Rev\|Path" | sed "s/Last Changed Rev: /;/" | sed "s/Path: //" | sed '$!N;s/\n//'</pre>
<p>
<strong><em>Convert unix timestamp to date</em></strong>
  <pre class="codeline"> date -ud "1970-01-01 + 1234567890 seconds"</pre>
<p>
<strong><em>List only locally modified files with CVS</em></strong>
  <pre class="codeline"> cvs -n update 2>null | grep -i "M " | sed s/"M "//</pre>
<p>
<strong><em>Bashbuiltin printf</em></strong>
  <pre class="codeline"> cat file.txt | while read line; do printf "%7.2f -> %7.2f\n" $line; done</pre>
<p>
<strong><em>List open TCP/UDP ports</em></strong>
  <pre class="codeline"> netstat -ltun</pre>
<p>
<strong><em>List open TCP/UDP ports</em></strong>
  <pre class="codeline"> netstat -anp --tcp --udp | grep LISTEN</pre>
<p>
<strong><em>Consistent Oracle Datapump Export</em></strong>
  <pre class="codeline"> expdp user/password FLASHBACK_SCN=$(echo -e "select current_scn from v\$database;" | sqlplus / as sysdba 2>/dev/null| grep [0-9][0-9][0-9][0-9][0-9][0-9]*)</pre>
<p>
<strong><em>Are 64-bit applications supported on my Solaris OS?</em></strong>
  <pre class="codeline"> isainfo -vb</pre>
<p>
<strong><em>convert string to array</em></strong>
  <pre class="codeline"> s=124890; array=($(echo $s | sed 's/./& /g')); echo ${array[@]}; echo</pre>
<p>
<strong><em>convert string to array</em></strong>
  <pre class="codeline"> s="124890";for i in $(seq 0 1 $((${#s}-1))); do arr[$i]=${s:$i:1}; done</pre>
<p>
<strong><em>Open your application to a specific size and location</em></strong>
  <pre class="codeline"> command -geometry 120x30+1280+0</pre>
<p>
<strong><em>Log a command's votes</em></strong>
  <pre class="codeline"> while true; do curl -s http://www.commandlinefu.com/commands/view/3643/log-a-commands-votes | grep 'id="num-votes-' | sed 's;.*id="num-votes-[0-9]*">\([0-9\-]*\)</div>;\1;' >> votes; sleep 10; done</pre>
<p>
<strong><em>Colorized grep in less</em></strong>
  <pre class="codeline"> ack --pager='less -r'</pre>
<p>
<strong><em>futz.me - Send yourself notes from the command line</em></strong>
  <pre class="codeline"> lynx "futz.me/xxx hey this is a test"</pre>
<p>
<strong><em>Find duplicate dir in path</em></strong>
  <pre class="codeline"> echo $PATH|tr : '\n'|sort|uniq -d</pre>
<p>
<strong><em>Randomize lines (opposite of | sort)</em></strong>
  <pre class="codeline"> cat ~/SortedFile.txt | perl -wnl -e '@f=<>; END{ foreach $i (reverse 0 .. $#f) { $r=int rand ($i+1); @f[$i, $r]=@f[$r,$i] unless ($i==$r); } chomp @f; foreach $line (@f){ print $line; }}'</pre>
<p>
<strong><em>find . -name "*.txt" | xargs sed -i "s/old/new/"</em></strong>
  <pre class="codeline"> find . -name "*.txt" | xargs sed -i "s/old/new/"</pre>
<p>
<strong><em>svn diff $* | colordiff | lv -c</em></strong>
  <pre class="codeline"> svn diff $* | colordiff | lv -c</pre>
<p>
<strong><em>Recursive Ownership Change</em></strong>
  <pre class="codeline"> sudo chown -R user2:user2 /../../somedirectory</pre>
<p>
<strong><em>Execute a command with a timeout</em></strong>
  <pre class="codeline"> perl -e "alarm 10; exec @ARGV" "somecommand"</pre>
<p>
<strong><em>Show which include directories your installation of Perl is</em></strong>
  <pre class="codeline"> perl -le 'print join $/, @INC'</pre>
<p>
<strong><em>Dump all of perl's config info</em></strong>
  <pre class="codeline"> perl -le 'use Config; foreach $i (keys %Config) {print "$i : @Config{$i}"}'</pre>
<p>
<strong><em>Go get those photos from a Picasa album</em></strong>
  <pre class="codeline"> echo 'Enter Picasa album RSS URL:"; read -e feedurl; GET "$feedurl" |sed 's/</\n</g' | grep media:content |sed 's/.*url='"'"'\([^'"'"']*\)'"'"'.*$/\1/' > wgetlist</pre>
<p>
<strong><em>List all text files (exclude binary files)</em></strong>
  <pre class="codeline"> find . | xargs file | grep ".*: .* text" | sed "s;\(.*\): .* text.*;\1;"</pre>
<p>
<strong><em>Learn how to stop mistyping "ls" the fun way</em></strong>
  <pre class="codeline"> apt-get install sl; sl</pre>
<p>
<strong><em>List the size (in human readable form) of all sub folders from the</em></strong>
  <pre class="codeline"> du -h --max-depth=1</pre>
<p>
<strong><em>Query Wikipedia via console over DNS</em></strong>
  <pre class="codeline"> dig +short txt <keyword>.wp.dg.cx</pre>
<p>
<strong><em>Display the top ten running processes - sorted by memory usage</em></strong>
  <pre class="codeline"> ps aux | sort -nk +4 | tail</pre>
<p>
<strong><em>A very simple and useful stopwatch</em></strong>
  <pre class="codeline"> time read (ctrl-d to stop)</pre>
<p>
<strong><em>Place the argument of the most recent command on the shell</em></strong>
  <pre class="codeline"> 'ALT+.' or '<ESC> .'</pre>
<p>
<strong><em>Quick access to the ascii table.</em></strong>
  <pre class="codeline"> man ascii</pre>
<p>
<strong><em>Shutdown a Windows machine from Linux</em></strong>
  <pre class="codeline"> net rpc shutdown -I ipAddressOfWindowsPC -U username%password</pre>
<p>
<strong><em>Execute a command without saving it in the history</em></strong>
  <pre class="codeline"> <space>command</pre>
<p>
<strong><em>Jump to a directory, execute a command and jump back to current</em></strong>
  <pre class="codeline"> (cd /tmp && ls)</pre>
<p>
<strong><em>Clear the terminal screen</em></strong>
  <pre class="codeline"> ctrl-l</pre>
<p>
<strong><em>Backticks are evil</em></strong>
  <pre class="codeline"> echo "The date is: $(date +%D)"</pre>
<p>
<strong><em>Reboot machine when everything is hanging</em></strong>
  <pre class="codeline"> <alt> + <print screen/sys rq> + <R> - <S> - <E> - <I> - <U> - <B></pre>
<p>
<strong><em>Easily search running processes (alias).</em></strong>
  <pre class="codeline"> alias 'ps?'='ps ax | grep '</pre>
<p>
<strong><em>Rip audio from a video file.</em></strong>
  <pre class="codeline"> mplayer -ao pcm -vo null -vc dummy -dumpaudio -dumpfile <output-file> <input-file></pre>
<p>
<strong><em>Matrix Style</em></strong>
  <pre class="codeline"> tr -c "[:digit:]" " " < /dev/urandom | dd cbs=$COLUMNS conv=unblock | GREP_COLOR="1;32" grep --color "[^ ]"</pre>
<p>
<strong><em>Watch Star Wars via telnet</em></strong>
  <pre class="codeline"> telnet towel.blinkenlights.nl</pre>
<p>
<strong><em>Stream YouTube URL directly to mplayer.</em></strong>
  <pre class="codeline"> mplayer -fs $(echo "http://youtube.com/get_video.php?$(curl -s $youtube_url | sed -n "/watch_fullscreen/s;.*\(video_id.\+\)&title.*;\1;p")")</pre>
<p>
<strong><em>Use heading subtitle file as watermark using mencoder</em></strong>
  <pre class="codeline"> mencoder -sub heading.ssa -subpos 0 -subfont-text-scale 4 -utf8 -oac</pre>
<p>
<strong><em>Send a local file via email</em></strong>
  <pre class="codeline"> cat filename | uuencode filename | mail -s "Email subject" user@example.com</pre>
<p>
<strong><em>Bulk copy large blocks of data between File Systems (run as root</em></strong>
  <pre class="codeline"> tar cpof - src |( cd des; tar xpof -)</pre>
<p>
<strong><em>list the last week's added files in xmms2's library</em></strong>
  <pre class="codeline"> xmms2 mlib search added \> $(echo $(date +%s) - 604800|bc)</pre>
<p>
<strong><em>View a sopcast stream</em></strong>
  <pre class="codeline"> (sp-sc sop://broker.sopcast.com:3912/6002 3900 8900 &>/dev/null &); sleep 10; mplayer http://localhost:8900/tv.asf</pre>
<p>
<strong><em>Show only existing executable dirs in PATH using only builtin bash</em></strong>
  <pre class="codeline"> for p in ${PATH//:/ }; do [[ -d $p && -x $p ]] && echo $p; done</pre>
<p>
<strong><em>for loop with leading zero in bash 3</em></strong>
  <pre class="codeline"> printf "%02u " {3..20}; echo</pre>
<p>
<strong><em>for loop with leading zeros</em></strong>
  <pre class="codeline"> for s in `seq -f %02.0f 5 15`; do echo $s; done</pre>
<p>
<strong><em>Substitution cipher</em></strong>
  <pre class="codeline"> echo "Decode this"| tr [a-zA-Z] $(echo {a..z} {A..Z}|grep -o .|sort -R|tr -d "\n ")</pre>
<p>
<strong><em>regex to match an ip</em></strong>
  <pre class="codeline"> echo 254.003.032.3 | grep -P '^((25[0-4]|2[0-4]\d|[01]?[\d]?[1-9])\.){3}(25[0-4]|2[0-4]\d|[01]?[\d]? [1-9])$'</pre>
<p>
<strong><em>replace @ symbol with new line character, to get new line</em></strong>
  <pre class="codeline"> %s/@/^v[M]/g</pre>
<p>
<strong><em>Tar a directory and its sub-directory</em></strong>
  <pre class="codeline"> tar cvfz dir_name.tgz dir/</pre>
<p>
<strong><em <a href="http://www.merriam-webster.com/dictionary/ultimate</em></strong>" <a href="http://www.merriam-webster.com/dictionary/ultimate</em></strong></a>">http://www.merriam-webster.com/dictionary/ultimate</em></strong></a></a>
  <pre class="codeline"> ultimate</pre>
<p>
<strong><em>Run remote web page, but don't save the results</em></strong>
  <pre class="codeline"> wget -q --spider http://server/cgi/script</pre>
<p>
<strong><em>Mount an ISO image on Mac OS X</em></strong>
  <pre class="codeline"> hdiutil mount sample.iso</pre>
<p>
<strong><em>When need to compress the Zope Database</em></strong>
  <pre class="codeline"> python fsrecovery.py -P 0 -f <path-to-instance>/Data.fs <path-to-instance-destination>/Data.fs.packed</pre>
<p>
<strong><em>find files in $PATH that were not installed through dpkg</em></strong>
  <pre class="codeline"> echo -e "${PATH//://\n}" >/tmp/allpath; grep -Fh -f /tmp/allpath</pre>
<p>
<strong><em>Create a zip file ignoring .svn files</em></strong>
  <pre class="codeline"> find . -not \( -name .svn -prune \) -type f | xargs zip XXXXX.zip</pre>
<p>
<strong><em>Search OpenSolaris packages and show only the pkg names</em></strong>
  <pre class="codeline"> pkg search SEARCH_TERM | awk '{print $NF}' | sed -e 's;.*/\(.*\)\@.*;\1;' | sort -u</pre>
<p>
<strong><em>Put the machine to sleep after the download(wget) is done</em></strong>
  <pre class="codeline"> while [ -n "`pgrep wget`" ]; do sleep 2 ;done; [ -e "/tmp/nosleep"] || echo mem >/sys/power/state</pre>
<p>
<strong><em>View the octal dump of a file</em></strong>
  <pre class="codeline"> od -vt x1 /tmp/spaghettifile</pre>
<p>
<strong><em>Add another tty device using mknod command</em></strong>
  <pre class="codeline"> sudo mknod /dev/ttyS4 c 4 68</pre>
<p>
<strong><em>badblocks for floppy</em></strong>
  <pre class="codeline"> /sbin/badblocks -v /dev/fd0 1440</pre>
<p>
<strong><em>How to watch files</em></strong>
  <pre class="codeline"> watch -d 'ls -l'</pre>
<p>
<strong><em>Watching Command</em></strong>
  <pre class="codeline"> watch 'cat /proc/loadavg'</pre>
<p>
<strong><em>See if your mac can run 64-bit && if it the kernel is loaded</em></strong>
  <pre class="codeline"> ioreg -l -p IODeviceTree | grep -o EFI[0-9]. && system_profiler SPSoftwareDataType |grep 64</pre>
<p>
<strong><em>bash/ksh function: given a file, cd to the directory it lives</em></strong>
  <pre class="codeline"> function fcd () { [ -f $1 ] && { cd $(dirname $1); } || { cd $1 ; } pwd }</pre>
<p>
<strong><em>Edit all files found having a specific string found by grep</em></strong>
  <pre class="codeline"> grep -ir 'foo' * | awk -F '{print $1}' | xargs vim</pre>
<p>
<strong><em>Edit all files found having a specific string found by grep</em></strong>
  <pre class="codeline"> grep -ir 'foo' * | awk '{print $1}' | sed -e 's/://' | xargs vim</pre>
<p>
<strong><em>list all hd partitions</em></strong>
  <pre class="codeline"> fdisk -l |grep -e '^/' |awk '{print $1}'|sed -e "s|/dev/||g"</pre>
<p>
<strong><em>find files containing text</em></strong>
  <pre class="codeline"> grep -lir "sometext" * > sometext_found_in.log</pre>
<p>
<strong><em>Another way to see the network interfaces</em></strong>
  <pre class="codeline"> ip addr show</pre>
<p>
<strong><em>scp a good script from host A which has no public access to host</em></strong>
  <pre class="codeline"> ssh middlehost "ssh -a root@securehost '> nicescript'" < nicescript</pre>
<p>
<strong><em>Download entire commandlinefu archive to single file</em></strong>
  <pre class="codeline"> for x in `jot - 0 \`curl "http://www.commandlinefu.com/commands/browse"|grep "Terminal - All commands" |perl -pe 's/.+(\d+),(\d+).+/$1$2/'|head -n1\` 25`; do curl "http://www.commandlinefu.com/commands/browse/sort-by-votes/plaintext/$ x" ; done >a.txt</pre>
<p>
<strong><em>Do a search-and-replace in a file after making a backup</em></strong>
  <pre class="codeline"> for file in <filename>; do cp $file{,.bak} && sed 's/old/new/g' $file.bak > $file; done</pre>
<p>
<strong><em>Resolve the "all display buffers are busy, please try later" error</em></strong>
  <pre class="codeline"> dm display-buffer reset</pre>
<p>
<strong><em>Get number of diggs for a news URL</em></strong>
  <pre class="codeline"> curl -s "http://services.digg.com/stories?link=$NEWSURL&appkey=http://www.whate ver.com&type=json" | python -m simplejson.tool | grep diggs</pre>
<p>
<strong><em>Verbosely delete files matching specific name pattern, older than</em></strong>
  <pre class="codeline"> find /backup/directory -name "FILENAME_*" -mtime +15 -exec rm -vf {};</pre>
<p>
<strong><em>locate a filename, make sure it exists and display it with full</em></strong>
  <pre class="codeline"> locate -e somefile | xargs ls -l</pre>
<p>
<strong><em>List files and sizes</em></strong>
  <pre class="codeline"> find / -type f -exec wc -c {} \; | sort -nr | head -100</pre>
<p>
<strong><em>power off system in X hours form the current time, here X=2</em></strong>
  <pre class="codeline"> echo init 0 | at now + 2 hours</pre>
<p>
<strong><em>purge old stale messages on a qmail queue</em></strong>
  <pre class="codeline"> for i in `grep "unable to stat" /var/log/syslog | cut -d "/" -f 3 | sort | uniq`; do find /var/qmail/queue -name $i -type f -exec rm -v {} \; ; done</pre>
<p>
<strong><em>delete all DrWeb status, failure and other messages on a postfix</em></strong>
  <pre class="codeline"> mailq | grep DrWEB | awk {'print $1'} | sed s/*//g | postsuper -d -</pre>
<p>
<strong><em>less an message on a postfix mailsystem with a specific</em></strong>
  <pre class="codeline"> id=<XXXX>; find /var/spool/postfix/ -name $id -exec less {} \;</pre>
<p>
<strong><em>replace old htaccess php AddHandler values with new one</em></strong>
  <pre class="codeline"> find /var/www/ -type f -name ".htaccess" -exec perl -pi -e 's/AddHandler[\s]*php(4|5)-cgi/AddHandler x-httpd-php\1/' {} \;</pre>
<p>
<strong><em>Have a list of directories in a file, ending with newlines and</em></strong>
  <pre class="codeline"> cat filename | tr '\n' '\0' | du -hsc ?files0-from=-</pre>
<p>
<strong><em>Download all Phrack .tar.gzs</em></strong>
  <pre class="codeline"> for ((i=1; i<67; i++)) do wget http://www.phrack.org/archives/tgz/phrack${i}.tar.gz -q; done</pre>
<p>
<strong><em>Remove CR from Windows- / DOS-textfiles</em></strong>
  <pre class="codeline"> dos2unix file.txt</pre>
<p>
<strong><em>Watch changeable interrupts continuously</em></strong>
  <pre class="codeline"> watch -n1 'cat /proc/interrupts</pre>
<p>
<strong><em>Merge ( directories [looking for improvement]</em></strong>
  <pre class="codeline"> (cd SRC; find . -type d -exec mkdir TARGET/{} ";"; find . -type f -exec mv {} TARGET/{} ";")</pre>
<p>
<strong><em>total text files in current dir</em></strong>
  <pre class="codeline"> file -i * | grep 'text/plain' | wc -l</pre>
<p>
<strong><em>An easter egg built into python to give you the Zen of Python</em></strong>
  <pre class="codeline"> echo "import this" | python</pre>
<p>
<strong><em>checks if host /service is up on a host that doesn't respond to</em></strong>
  <pre class="codeline"> while true; do clear; nmap ${hostname} -PN -p ${hostport}; sleep 5; done</pre>
<p>
<strong><em>print latest (top 10, top 3 or *) commandlinefu.com commands</em></strong>
  <pre class="codeline"> wget -qO - http://www.commandlinefu.com/feed/tenup | xmlstarlet sel -T -t -o '<x>' -n -t -m rss/channel/item -o '<y>' -n -v description -o '</y>' -n -t -o '</x>' | xmlstarlet sel -T -t -m x/y -v code -n</pre>
<p>
<strong><em>Analyze awk fields</em></strong>
  <pre class="codeline"> tr " " "\n" | nl</pre>
<p>
<strong><em>Add "prefix" on a bunch of files</em></strong>
  <pre class="codeline"> for a in *; do mv $a prefix${a}; done</pre>
<p>
<strong><em>Get Interface's IP on Mac</em></strong>
  <pre class="codeline"> ipconfig getifaddr <Interface></pre>
<p>
<strong><em>Picture Renamer</em></strong>
  <pre class="codeline"> ls -1 *.jpg | while read fn; do export pa=`exiv2 "$fn" | grep timestamp | awk '{ print $4 " " $5 ".jpg"}' | tr ":" "-"`; mv "$fn" "$pa"; done</pre>
<p>
<strong><em>pipe commands from a textfile to a telnet-server with netcat</em></strong>
  <pre class="codeline"> nc $telnetserver 23 < $commandfile</pre>
<p>
<strong><em>Copy files from list with hierarchy</em></strong>
  <pre class="codeline"> cat files.txt | xargs tar -cv | tar -x -c $DIR/</pre>
<p>
<strong><em>Send SNMP traps</em></strong>
  <pre class="codeline"> sudo snmptrap -m ALL -v 2c -c public trapserver "" UCD-DEMO-MIB::ucdDemoPublic SNMPv2-MIB::sysLocation.0 s "Just here"</pre>
<p>
<strong><em>Create more threads with less stack space</em></strong>
  <pre class="codeline"> ulimit -s 64</pre>
<p>
<strong><em>modify (mozldap) with proxy authentication and no other controls</em></strong>
  <pre class="codeline"> ldapmodify -Y "dn:uid=rob,dc=example.com" -g -R -J 2.16.840.1.113730.3.4.16 ...</pre>
<p>
<strong><em>Convert a batch of images to a Video</em></strong>
  <pre class="codeline"> mencoder "mf://frame_*.bmp" -mf w=720:h=480:fps=30:type=bmp -ovc lavc</pre>
<p>
<strong><em>Command to display how much resource is taken by cpu and which</em></strong>
  <pre class="codeline"> pidstat -C "ffmpeg" -u</pre>
<p>
<strong><em>Lists unambigously names of all xml elements used in files in</em></strong>
  <pre class="codeline"> grep -Eho '<[a-ZA-Z_][a-zA-Z0-9_-:]*' * | sort -u | cut -c2-</pre>
<p>
<strong><em>use curl to resume a failed download</em></strong>
  <pre class="codeline"> cat file-that-failed-to-download.zip | curl -C - http://www.somewhere.com/file-I-want-to-download.zip >successfully-downloaded.zip</pre>
<p>
<strong><em>Batch image resize</em></strong>
  <pre class="codeline"> for a in `ls`; do echo $a && convert $a -resize <Width>x<Height> $a; done</pre>
<p>
<strong><em>skipping five lines, at top, then at bottom</em></strong>
  <pre class="codeline"> seq 1 12 | sed 1,5d ; seq 1 12 | head --lines=-5</pre>
<p>
<strong><em>Generate Files with Random Content and Size in Bash</em></strong>
  <pre class="codeline"> no_of_files=10; counter=1; while [[ $counter -le $no_of_files ]]; do echo Creating file no $counter; dd bs=1024 count=$RANDOM skip=$RANDOM if=/dev/sda of=random-file.$counter; let "counter += 1"; done</pre>
<p>
<strong><em>Gentoo: Get the size of all installed packets, sorted</em></strong>
  <pre class="codeline"> equery s | sed 's/(\|)/ /g' | sort -n -k 9 | gawk '{print $1" "$9/1048576"m"}'</pre>
<p>
<strong><em>Get all IPs via ifconfig</em></strong>
  <pre class="codeline"> ipconfig getpacket en0 | grep yi| sed s."yiaddr = "."en0: ". ipconfig getpacket en1 | grep yi| sed s."yiaddr = "."en1: ".</pre>
<p>
<strong><em>Bypass 1000 Entry limit of Active Directory with ldapsearch</em></strong>
  <pre class="codeline"> ldapsearch -LLL -H ldap://${HOST}:389 -b 'DC=${DOMAIN},DC=${TLD}' -D '${USER}' -w 'password' objectclass=* -E pr=2147483647/noprompt</pre>
<p>
<strong><em>Change a text files contents without opening it, or intermediate</em></strong>
  <pre class="codeline"> print 'g/'delete this line'/delete\nwq' | ex file.txt</pre>
<p>
<strong><em>Change open file descriptors limit.</em></strong>
  <pre class="codeline"> ulimit -n <value></pre>
<p>
<strong><em>ignore hidden directory in bash completion (e.g. .svn)</em></strong>
  <pre class="codeline"> Add to ~/.inputrc: set match-hidden-files off</pre>
<p>
<strong><em>Send Reminders from your Linux Server to Growl on a Mac</em></strong>
  <pre class="codeline"> remind -z1 -k'echo %s |ssh <user>@<host> "growlnotify"' ~/.reminders &</pre>
<p>
<strong><em>Test your total disk IO capacity, regardless of caching, to find</em></strong>
  <pre class="codeline"> time dd if=/dev/zero of=blah.out oflag=direct bs=256M count=1</pre>
<p>
<strong><em>Create a file list of all package files installed on your</em></strong>
  <pre class="codeline"> for i in `rpm -qva | sort ` ; do ; echo "===== $i =====" ; rpm -qvl $i ; done > /tmp/pkgdetails</pre>
<p>
<strong><em>Test network performance, copying from the mem of one box, over</em></strong>
  <pre class="codeline"> dd if=/dev/zero bs=256M count=1 | nc [remoteIP] [remotePort] and on the other host nc -l port >/dev/null</pre>
<p>
<strong><em>Fast grepping (avoiding UTF overhead)</em></strong>
  <pre class="codeline"> export LANG=C; grep string longBigFile.log</pre>
<p>
<strong><em>Watch number of lines being processed on a clear screen</em></strong>
  <pre class="codeline"> cat /dev/urandom|awk 'BEGIN{"tput cuu1" | getline CursorUp; "tput clear" | getline Clear; printf Clear}{num+=1;printf CursorUp; print num}'</pre>
<p>
<strong><em>Sorting by rows</em></strong>
  <pre class="codeline"> infile=$1 for i in $(cat $infile) do echo $i | tr "," "\n" | sort -n | tr "\n" "," | sed "s/,$//" echo done</pre>
<p>
<strong><em>kill some process (same as others) but parsing to a variable</em></strong>
  <pre class="codeline"> pkill -9 -f program</pre>
<p>
<strong><em>Test if the given argument is a valid ip address.</em></strong>
  <pre class="codeline"> perl -e '$p=qr!(?:0|1\d{0,2}|2(?:[0-4]\d?|5[0-5]?|[6-9])?|[3-9]\d?)!;print((shi ft=~m/^$p\.$p\.$p\.$p$/)?1:0);' 123.123.123.123</pre>
<p>
<strong><em>SSH monitor</em></strong>
  <pre class="codeline"> ssh root@server 'tail --max-unchanged-stats=10 -n0 -F /var/log/auth.log ' | grep Accepted | while read l ; do kdialog --title "SSH monitor" --passivepopup "$l" 3; done</pre>
<p>
<strong><em>Printing multiple years with Unix cal command</em></strong>
  <pre class="codeline"> for y in 2009 2010 2011; do cal $y; done</pre>
<p>
<strong><em>Matrix Style</em></strong>
  <pre class="codeline"> while $t; do for i in `seq 1 30`;do r="$[($RANDOM % 2)]";h="$[($RANDOM % 4)]";if [ $h -eq 1 ]; then v="\e[1m $r";else v="\e[2m $r";fi;v2="$v2 $v";done;echo -e $v2;v2="";done;</pre>
<p>
<strong><em>Migrate gems from one ruby installation to another</em></strong>
  <pre class="codeline"> /originalInstall/gem list | tr -d '(),' | xargs -L 1 sudo ./gemInst.sh</pre>
<p>
<strong><em>Find files with lines that do not match a pattern</em></strong>
  <pre class="codeline"> fmiss() { grep -cR "$*" * | grep -E ':0$' | cut -d: -f1 ; }</pre>
<p>
<strong><em>Scan computers OS and open services on all network</em></strong>
  <pre class="codeline"> nmap -O 192.168.1.1/24</pre>
<p>
<strong><em>bbs in utf8 console</em></strong>
  <pre class="codeline"> luit -encoding gbk telnet bbs.sysu.edu.cn</pre>
<p>
<strong><em>Puts every word from a file into a new line</em></strong>
  <pre class="codeline"> sed -r 's/[ \t\r\n\v\f]+/\^J/g' INFILE > OUTFILE</pre>
<p>
<strong><em>Overwrite local files from copies in a flat directory, even if</em></strong>
  <pre class="codeline"> for f in $(find * -maxdepth 0 -type f); do file=$(find ~/target -name $f); if [ -n "$file" ]; then cp $file ${file}.bak; mv $f $file; fi; done</pre>
<p>
<strong><em>Frequency Sweep</em></strong>
  <pre class="codeline"> l=500; x=500; y=200; d=-15;for i in `seq $x $d $y`; do beep -l $l -f $i;done</pre>
<p>
<strong><em>get a list of running virtual machines from the command line</em></strong>
  <pre class="codeline"> vmrun list</pre>
<p>
<strong><em>run vmware virtual machine from the command line without the gui</em></strong>
  <pre class="codeline"> vmrun start /path/to/virtual_machine.vmx nogui</pre>
<p>
<strong><em>Check if you need to run LaTeX more times to get the refefences</em></strong>
  <pre class="codeline"> egrep "(There were undefined references|Rerun to get (cross-references|the bars) right)" texfile.log</pre>
<p>
<strong><em>Quickly clean log files (assuming you don't want them anymore)</em></strong>
  <pre class="codeline"> for file in `find /var/log/ -type f -size +5000k`; do > $file; done</pre>
<p>
<strong><em>Quickly clean log files (assuming you don't want them anymore)</em></strong>
  <pre class="codeline"> for file in `find /var/log/ -type f -size +5000k`; do echo " " > $file; done</pre>
<p>
<strong><em>my command for downloading delicious web links,</em></strong>
  <pre class="codeline"> wget -r --wait=5 --quota=5000m --tries=3 --directory-prefix=/home/erin/Documents/erins_webpages --limit-rate=20k --level=1 -k -p -erobots=off -np -N --exclude-domains=del.icio.us,doubleclick.net -F -i ./delicious-20090629.htm</pre>
<p>
<strong><em>Extract the emoticons regex from a running skype process</em></strong>
  <pre class="codeline"> S=`pidof skype`;grep heap /proc/$S/maps|cut -f1 -d' '|awk -F- '{print "0x" $1 " 0x" $2}'|xargs echo "du me t ">l;gdb -batch -p $S -x l>/dev/null 2>&1;strings t|grep \(smirk|head -n1</pre>
<p>
<strong><em>Untar file with absolute pathname to relative location</em></strong>
  <pre class="codeline"> pax -r -s ',^/,,' -f file.tar</pre>
<p>
<strong><em>Open a Remote Desktop (RDP) session with a custom resolution.</em></strong>
  <pre class="codeline"> mstsc /w:1500 /h:900 /v:www.example.com</pre>
<p>
<strong><em>get linkspeed, ip-adress, mac-address and processor type from osx</em></strong>
  <pre class="codeline"> echo "-------------" >> nicinfo.txt; echo "computer name x" >> nicinfo.txt; ifconfig | grep status >> nicinfo.txt; ifconfig | grep inet >> nicinfo.txt; ifconfig | grep ether >> nicinfo.txt; hostinfo | grep type >> nicinfo.txt;</pre>
<p>
<strong><em>String Capitalization</em></strong>
  <pre class="codeline"> echo "${STRING}" | tr '[A-Z]' '[a-z]' | awk '{print toupper(substr($0,1,1))substr($0,2);}'</pre>
<p>
<strong><em>vim insert current filename</em></strong>
  <pre class="codeline"> :r! echo %</pre>
<p>
<strong><em>Match non-empty lines</em></strong>
  <pre class="codeline"> grep -v "^\W$" <filename></pre>
<p>
<strong><em>find large files</em></strong>
  <pre class="codeline"> ls -s | sort -nr | more</pre>
<p>
<strong><em>Create a random file of a specific size</em></strong>
  <pre class="codeline"> dd if=/dev/zero of=testfile.txt bs=1M count=10</pre>
<p>
<strong><em>Extract the daily average number of iops</em></strong>
  <pre class="codeline"> for x in `seq -w 1 30`; do sar -b -f /var/log/sa/sa$x | gawk '/Average/ {print $2}'; done</pre>
<p>
<strong><em>Send multiple attachments using mailx</em></strong>
  <pre class="codeline"> (uuencode foo.txt foo.txt; uuencode /etc/passwd passwd.txt)|mailx -s "Pandaren!" someone@cmdfu.com</pre>
<p>
<strong><em>Extract multiple file in a directory</em></strong>
  <pre class="codeline"> for i in *.tar.gz; do tar -xzf $i; done</pre>
<p>
<strong><em>Finds all of the mailers being used in your rails app</em></strong>
  <pre class="codeline"> egrep -r '(render_message|multipart).*('`find app/views -name '*.erb' | grep mailer | sed -e 's/\..*//' -e 's/.*\///' | uniq | xargs | sed 's/ /|/g'`')' app/models</pre>
<p>
<strong><em>I hate `echo X | Y`</em></strong>
  <pre class="codeline"> base64 -d <<<</pre>
<p>
<strong><em>Rsync remote data as root using sudo</em></strong>
  <pre class="codeline"> rsync --rsync-path 'sudo rsync' username@source:/folder/ /local/</pre>
<p>
<strong><em>find the process that is using a certain port e.g. port 3000</em></strong>
  <pre class="codeline"> lsof -P | grep ':3000'</pre>
<p>
<strong><em>Check Ram Speed and Type in Linux</em></strong>
  <pre class="codeline"> sudo dmidecode --type 17 | more</pre>
<p>
<strong><em>Grep for word in directory (recursive)</em></strong>
  <pre class="codeline"> grep --color=auto -iRnH "$search_word" $directory</pre>
<p>
<strong><em>List of commands you use most often</em></strong>
  <pre class="codeline"> history | awk '{print $2}' | sort | uniq -c | sort -rn | head</pre>
<p>
<strong><em>Recursively remove .svn directories from the current location</em></strong>
  <pre class="codeline"> find . -type d -name '.svn' -print0 | xargs -0 rm -rdf</pre>
<p>
<strong><em>find process associated with a port</em></strong>
  <pre class="codeline"> fuser [portnumber]/[proto]</pre>
<p>
<strong><em>Get http headers for an url</em></strong>
  <pre class="codeline"> curl -I www.commandlinefu.com</pre>
<p>
<strong><em>copy working directory and compress it on-the-fly while showing</em></strong>
  <pre class="codeline"> tar -cf - . | pv -s $(du -sb . | awk '{print $1}') | gzip > out.tgz</pre>
<p>
<strong><em>Clean your broken terminal</em></strong>
  <pre class="codeline"> stty sane</pre>
<p>
<strong><em>geoip information</em></strong>
  <pre class="codeline"> curl -s "http://www.geody.com/geoip.php?ip=$(curl -s icanhazip.com)" | sed '/^IP:/!d;s/<[^>][^>]*>//g'</pre>
<p>
<strong><em>Unbelievable Shell Colors, Shading, Backgrounds, Effects for</em></strong>
  <pre class="codeline"> for c in `seq 0 255`;do t=5;[[ $c -lt 108 ]]&&t=0;for i in `seq $t 5`;do echo -e "\e[0;48;$i;${c}m|| $i:$c `seq -s+0 $(($COLUMNS/2))|tr -d '[0-9]'`\e[0m";done;done</pre>
<p>
<strong><em>How fast is the connexion to a URL, some stats from curl</em></strong>
  <pre class="codeline"> URL="http://www.google.com";curl -L --w "$URL\nDNS %{time_namelookup}s conn %{time_connect}s time %{time_total}s\nSpeed %{speed_download}bps Size %{size_download}bytes\n" -o/dev/null -s $URL</pre>
<p>
<strong><em>Emulate a dual-screen using vnc</em></strong>
  <pre class="codeline"> x2vnc {-west|-east|-north|-south} computer-ip:display-number</pre>
<p>
<strong><em>Remove the boot loader from a usb stick</em></strong>
  <pre class="codeline"> dd if=/dev/zero of=/dev/sdb bs=446 count=1</pre>
<p>
<strong><em>Archive every file in /var/logs</em></strong>
  <pre class="codeline"> find /var/logs -name * | xargs tar -jcpf logs_`date +%Y-%m-%e`.tar.bz2</pre>
<p>
<strong><em>Check syntax of all PHP files before an SVN commit</em></strong>
  <pre class="codeline"> for i in `svn status | egrep '^(M|A)' | sed -r 's/\+\s+//' | awk '{ print $2 }'` ; do if [ ! -d $i ] ; then php -l $i ; fi ; done</pre>
<p>
<strong><em>Remove an old gmetric statistic</em></strong>
  <pre class="codeline"> gmetric -n $METRIC_NAME -v foo -t string -d 10</pre>
<p>
<strong><em>Today's date on a yearly calendar...</em></strong>
  <pre class="codeline"> cal -y | tr '\n' '|' | sed "s/^/ /;s/$/ /;s/ $(date +%e) / $(date +%e | sed 's/./#/g') /$(date +%m | sed s/^0//)" | tr '|' '\n'</pre>
<p>
<strong><em>Eclipse needs to know the path to the local maven repository.</em></strong>
  <pre class="codeline"> mvn -Declipse.workspace=<path-to-eclipse-workspace> eclipse:add-maven-repo</pre>
<p>
<strong><em>Do a quick check on the harware specifications on a set of Linux</em></strong>
  <pre class="codeline"> clear; for i in `cat thehosts` ; do ssh $i "cat uname -a ; /etc/redhat-release; cat /proc/cpuinfo | tail -n 25 | egrep '^processor|^model name' "; free ; df -h ;done</pre>
<p>
<strong><em>Get max number of arguments</em></strong>
  <pre class="codeline"> getconf ARG_MAX</pre>
<p>
<strong><em>get the list of temps for your hard-drives</em></strong>
  <pre class="codeline"> hddtemp /dev/sda /dev/sdb /dev/hda /dev/hdb | gawk '{print $NF}' | perl -n -e '$_ =~ s/(\d+)/print "$1 "/eg }{ print "\n"'</pre>
<p>
<strong><em>Shows the torrent file name along with the trackers url</em></strong>
  <pre class="codeline"> grep -ao -HP "http://[^/]*/" *</pre>
<p>
<strong><em>Delete Empty Directories</em></strong>
  <pre class="codeline"> find . -type d -exec rmdir {} \;</pre>
<p>
<strong><em>my command for downloading delicious web links,</em></strong>
  <pre class="codeline"> wget -H -r -nv --level=1 -k -p -erobots=off -np -N --exclude-domains=del.icio.us,doubleclick.net --exclude-directories=</pre>
<p>
<strong><em>How to Find the Block Size</em></strong>
  <pre class="codeline"> /sbin/dumpe2fs /dev/hda2 | grep 'Block size'</pre>
<p>
<strong><em>automatically add and remove files in subversion</em></strong>
  <pre class="codeline"> svn st | grep '^\?' | awk '{print $2}' | xargs svn add; svn st | grep '^\!' | awk '{print $2}' | xargs svn rm</pre>
<p>
<strong><em>alias for etckeeper, to commit changes after moification of etc</em></strong>
  <pre class="codeline"> function ec() { ec_var="`pwd`" && cd /etc/ && sudo bzr commit -m "$@" && cd $ec_var; }</pre>
<p>
<strong><em>Syslog System Reporting in a shell</em></strong>
  <pre class="codeline"> tail -f --retry /var/log/syslog /var/log/auth.log | ccze -A</pre>
<p>
<strong><em>view the system memory in clear text</em></strong>
  <pre class="codeline"> hexdump -e '90/1 "%_p" "\n"' /dev/mem | less</pre>
<p>
<strong><em>prints message in given argument on on center of screen</em></strong>
  <pre class="codeline"> function echox { echo `tput cup $(($(tput lines))) $(( ($(tput cols) - $(echo "${#1}"))/2 ))`"$1"`tput cup $(tput lines) $(( $(tput cols)-1 ))`; }</pre>
<p>
<strong><em>fetch 1600 jokes from robsjokes.com into a single file, which is</em></strong>
  <pre class="codeline"> for i in `seq -w 1600` ; do links -dump http://www.robsjokes.com/$i/index.html | sed '/Random Joke/,/Next Joke/!d' | sed '/^$/,/^$/!d' >> ~/temp/Rob.jokes ; echo '%' >> ~/temp/Rob.jokes ; done</pre>
<p>
<strong><em>sync a directory of corrupted jpeg with a source directory</em></strong>
  <pre class="codeline"> for i in *jpg; do jpeginfo -c $i | grep -E "WARNING|ERROR" | cut -d " " -f 1 | xargs -I '{}' find /mnt/sourcerep -name {} -type f -print0 | xargs -0 -I '{}' cp -f {} ./ ; done</pre>
<p>
<strong><em>Reads in the ~/.Xdefaults</em></strong>
  <pre class="codeline"> alias xdef_load='xrdb -merge ~/.Xdefaults'</pre>
<p>
<strong><em>Search gdb help pages</em></strong>
  <pre class="codeline"> gdb command: apropos <keyword></pre>
<p>
<strong><em>One liner gdb attach to Acrobat</em></strong>
  <pre class="codeline"> (acroread &);sleep 2;gdb /opt/Adobe/Reader8/Reader/intellinux/bin/acroread `pidof ld-linux.so.2`</pre>
<p>
<strong><em>Convert ip address in hexadecimal</em></strong>
  <pre class="codeline"> gethostip 208.69.34.230 -x</pre>
<p>
<strong><em>Find your graphics chipset</em></strong>
  <pre class="codeline"> lspci |grep VGA</pre>
<p>
<strong><em>Laminate files line by line</em></strong>
  <pre class="codeline"> lam -f 1.4 myfile</pre>
<p>
<strong><em>Convert Raw pictures to jpg</em></strong>
  <pre class="codeline"> for img in $( ls *.CR2 ); do convert $img $img.jpg; done</pre>
<p>
<strong><em>Get a summary of network devices in the system</em></strong>
  <pre class="codeline"> $ for i in `ls --color=none /sys/class/net`; do echo "# $i"; sudo ethtool $i | grep -E "Link|Speed" ; done</pre>
<p>
<strong><em>Install a library to a remote repository</em></strong>
  <pre class="codeline"> mvn deploy:deploy-file -DgroupId=groupId -DartifactId=artifactId -Dversion=1.0 -Dpackaging=jar -Dfile=pathtolib -DrepositoryId=repository -Durl=url</pre>
<p>
<strong><em>Get thread count for process on Solaris</em></strong>
  <pre class="codeline"> ps -L -p <pid> | wc -l</pre>
<p>
<p>
<strong><em>Start a vnc session on the currently running X session</em></strong>
  <pre class="codeline"> x0vnc4server -display :0 -PasswordFile ~/.vnc/passwd</pre>
<p>
<strong><em>Send Disk usage via email</em></strong>
  <pre class="codeline"> #!/bin/sh #du.sh i=`hostname -i` df -h > /tmp/space.txt echo "server $i " >> /tmp/space.txt uuencode /tmp/space.txt space.txt | mail -s "HDD usage $i" email@email.com</pre>
<p>
<strong><em>List installed rpm named and arquitecture.</em></strong>
  <pre class="codeline"> rpm -qa --queryformat "%{NAME} %{ARCH}\n"</pre>
<p>
<strong><em>Execute a command with the last parameter of a previous command</em></strong>
  <pre class="codeline"> ls !$</pre>
<p>
<strong><em>Number of seconds to certain unix date</em></strong>
  <pre class="codeline"> echo $( (( $( (2**31 -1) ) - $(date +%s) )) )</pre>
<p>
<strong><em>Get the version of sshd on a remote system</em></strong>
  <pre class="codeline"> ssh -vN hostname 2>&1 | grep "remote software version"</pre>
<p>
<strong><em>log your PC's motherboard and CPU temperature along with the</em></strong>
  <pre class="codeline"> date +%m/%d/%y%X|tr -d 'n' >>datemp.log&& sensors|grep +5V|cut -d "(" -f1|tr -d 'n'>> datemp.log && sensors |grep Temp |cut -d "(" -f1|tr -d 'n'>>datemp.log</pre>
<p>
<strong><em>Delete more than one month old thumbnails from home directory</em></strong>
  <pre class="codeline"> find ~/.thumbnails/ -type f -atime +30 -print0 | xargs -0 rm</pre>
<p>
<strong><em>Clean up after improper deletes in subversion</em></strong>
  <pre class="codeline"> svn rm `svn status | grep "\!" | cut -c 8-`</pre>
<p>
<strong><em>Create a symbolic link tree that shadows a directory structure</em></strong>
  <pre class="codeline"> find /home/user/doc/ -type d -printf "mkdir -vp '/home/user/Dropbox%p'\n" -o -type f -printf "ln -vs '%p' '/home/user/Dropbox%p'\n" | sh</pre>
<p>
<strong><em>Given NOPASSWD privileges on a remote SSH server, sftp as root</em></strong>
  <pre class="codeline"> sftp -s "sudo /usr/lib/sftp-server" user@host</pre>
<p>
<strong><em>Simple countdown from a given date</em></strong>
  <pre class="codeline"> watch --no-title -d -n 1 'echo `date -d "next Thursday" +%s` "-" `date +%s` | bc -l'</pre>
<p>
<strong><em>Clean the /boot directory</em></strong>
  <pre class="codeline"> rpm -q kernel-2* | grep -v $(uname -r) | xargs yum erase -y</pre>
<p>
<strong><em>Apply all pending updates to Mandriva Linux system (2008.0 and</em></strong>
  <pre class="codeline"> urpmi --auto-update --force # apply all pending updates (Mandriva Linux)</pre>
<p>
<strong><em>Dns zone transfer</em></strong>
  <pre class="codeline"> host -la domain.com</pre>
<p>
<strong><em>Adds characters at the beginning of the name of a file</em></strong>
  <pre class="codeline"> rename 's/.*/[it]$&/' *.pdf</pre>
<p>
<strong><em>Disable all iptables rules without disconnecting yourself</em></strong>
  <pre class="codeline"> iptables -F && iptables -X && iptables -P INPUT ACCEPT && iptables -OUTPUT ACCEPT</pre>
<p>
<strong><em>Show the files that you've modified in an SVN tree</em></strong>
  <pre class="codeline"> svn status | egrep '^(M|A)' | egrep -o '[^MA\ ].*$'</pre>
<p>
<strong><em>Show the ordered header line (with field names) of a CSV file</em></strong>
  <pre class="codeline"> function headers { head -1 $* | tr ',' '\12' | pr -t -n ; }</pre>
<p>
<strong><em>A bash function to show the files most recently modified in the</em></strong>
  <pre class="codeline"> function t { ls -ltch $* | head -20 ; }</pre>
<p>
<strong><em>Send an http HEAD request w/curl</em></strong>
  <pre class="codeline"> curl -i -X HEAD http://localhost/</pre>
<p>
<strong><em>Lists open ports</em></strong>
  <pre class="codeline"> netstat -antuwp | egrep "(^[^t])|(^tcp.*LISTEN)"</pre>
<p>
<strong><em>alias to show my own configured ip</em></strong>
  <pre class="codeline"> alias showip="ifconfig eth0 | grep 'inet addr:' | sed 's/.*addr\:\(.*\) Bcast\:.*/\1/'"</pre>
<p>
<strong><em>do a release upgrade in ubuntu</em></strong>
  <pre class="codeline"> do-release-upgrade</pre>
<p>
<strong><em>List all rpms on system by name, version and release numbers, and</em></strong>
  <pre class="codeline"> rpm -qa --qf '%{name}-%{version}-%{release}.%{arch}\n'</pre>
<p>
<strong><em>Remove leading zeros in multiple columns with sed</em></strong>
  <pre class="codeline"> sed 's/\b\(0*\)//g' filename</pre>
<p>
<strong><em>How to know if your NIC receive link</em></strong>
  <pre class="codeline"> watch ethtool eth0</pre>
<p>
<strong><em>Strip out time difference entries when verifying rpms on x86_64</em></strong>
  <pre class="codeline"> rpm -Va | grep -v "\.\.\.\.\.\.\.T"</pre>
<p>
<strong><em>host - DNS lookup utility</em></strong>
  <pre class="codeline"> host google.com</pre>
<p>
<strong><em>Copy all files. All normal files, all hidden files and all files</em></strong>
  <pre class="codeline"> cp ./* .[!.]* ..?* /path/to/dir</pre>
<p>
<strong><em>Create a P12 file, using OpenSSL</em></strong>
  <pre class="codeline"> openssl pkcs12 -export -in /dir/CERTIFICATE.pem -inkey /dir/KEY.pem -certfile /dir/CA-cert.pem -name "certName" -out /dir/certName.p12</pre>
<p>
<strong><em>View the list of files and directories in an archive with less.</em></strong>
  <pre class="codeline"> less file.tar.gz</pre>
<p>
<strong><em>Generate secure password to userwith chpasswd</em></strong>
  <pre class="codeline"> echo "encryptedpassword"|openssl passwd -1 -stdin</pre>
<p>
<strong><em>Edit the /etc/sudoers config file the right way.</em></strong>
  <pre class="codeline"> visudo</pre>
<p>
<strong><em>package most recent files in project</em></strong>
  <pre class="codeline"> find ~/project -mtime -1 -type f -print | tar jcvf myfiles.tar.bz2 -T -</pre>
<p>
<strong><em>ls to show hidden file, but not . or ..</em></strong>
  <pre class="codeline"> ls -A</pre>
<p>
<strong><em>Scan your LAN for unauthorized IPs</em></strong>
  <pre class="codeline"> diff <(nmap -sP 192.168.1.0/24 | grep ^Host | sed 's/.appears to be up.//g' | sed 's/Host //g') auth.hosts | sed 's/[0-9][a-z,A-Z][0-9]$//' | sed 's/</UNAUTHORIZED IP -/g'</pre>
<p>
<strong><em>search manpages on the internets</em></strong>
  <pre class="codeline"> manview() { lynx -dump -accept_all_cookies 'http://www.csuglab.cornell.edu/cgi-bin/adm/man.cgi?section=all&topic=' "$1" | less; }</pre>
<p>
<strong><em>AIX: Determine what filesets are missing to reach a TL</em></strong>
  <pre class="codeline"> instfix -icq | grep 5300-07_AIX_ML | grep ":-:"</pre>
<p>
<strong><em>Extract the contents of an RPM package to your current directory</em></strong>
  <pre class="codeline"> rpm2cpio /path/to/file.rpm | cpio -i -d</pre>
<p>
<strong><em>SVN Clean</em></strong>
  <pre class="codeline"> svn status | grep ^? | awk '{print $2}' | xargs rm -rf</pre>
<p>
<strong><em>Use the page up key to complete the command.</em></strong>
  <pre class="codeline"> echo "\"\e[5~\": history-search-backward" >> ~/.inputrc</pre>
<p>
<strong><em>Generate a specification file for file integrity scanning.</em></strong>
  <pre class="codeline"> mtree -c -K sha256digest -X mtree.exclude -p /path > host.mtree</pre>
<p>
<strong><em>Show hidden files in OS X</em></strong>
  <pre class="codeline"> defaults write com.apple.Finder AppleShowAllFiles TRUE</pre>
<p>
<strong><em>Diff with colour highlighting</em></strong>
  <pre class="codeline"> svn diff ARGUMENTS_FOR_DIFF | source-highlight --out-format=esc --src-lang=diff</pre>
<p>
<strong><em>Summarize size of all files of given type in all subdirectories</em></strong>
  <pre class="codeline"> SUM=0; for FILESIZE in `find /tmp -type f -iname \*pdf -exec du -b {}</pre>
<p>
<strong><em>Remount root in read-write mode.</em></strong>
  <pre class="codeline"> sudo mount -o remount,rw /</pre>
<p>
<strong><em>search the pattern from bzip2'ed file</em></strong>
  <pre class="codeline"> bzgrep -i "pattern" pattern.bz2</pre>
<p>
<strong><em>exclude file(s) from rsync</em></strong>
  <pre class="codeline"> rsync -vazuK --exclude "*.mp3" --exclude "*.svn*" * user@host:/path</pre>
<p>
<strong><em>CSV list of infected URLS detected by ClamAV</em></strong>
  <pre class="codeline"> grep "FOUND" /var/log/squidclamav.log | awk '{print $5"-"$2"-"$3","$4","$11}' | sed -e 's/\,http.*url=/\,/g' | sed -e 's/&/\,/g' | sed -e 's/source=//g' |sed -e 's/user=//g' | sed -e 's/virus=//g' | sed -e 's/stream\:+//g' | sed -e 's/\+FOUND//g'</pre>
<p>
<strong><em>Parse tektronic csv files</em></strong>
  <pre class="codeline"> awk 'BEGIN {FS=","} {loc = $4, val=$5; getline < "f0001ch1.csv"; print loc,val,$5}' f0001ch2.csv > data</pre>
<p>
<strong><em>find distro name / release version</em></strong>
  <pre class="codeline"> $ cat /etc/*-release</pre>
<p>
<strong><em>search installed files of package, that doesn't remember his name</em></strong>
  <pre class="codeline"> rpm -qa | grep PACKAGENAME | xargs rpm -q --filesbypkg</pre>
<p>
<strong><em>Allows incoming traffic from specific IP address to port 80</em></strong>
  <pre class="codeline"> sudo ufw allow proto tcp from 1.2.3.4 to any port 80</pre>
<p>
<strong><em>Send current job to the background</em></strong>
  <pre class="codeline"> ^Z then bg</pre>
<p>
<strong><em>Add all not commited files to svn</em></strong>
  <pre class="codeline"> svn st | grep ^? | xargs svn add 2> /dev/null</pre>
<p>
<strong><em>Grep the process excluding the grep itself.</em></strong>
  <pre class="codeline"> ps -ef | grep [t]clsh</pre>
<p>
<strong><em>this toggles mute on the Master channel of an alsa soundcard</em></strong>
  <pre class="codeline"> on="off"; off="on"; now=$(amixer get Master | tr -d '[]' | grep "Playback.*%" |head -n1 |awk '{print $7}'); amixer sset Master ${!now}</pre>
<p>
<strong><em>make a zip file containing all files with the openmeta tag "data"</em></strong>
  <pre class="codeline"> mdfind "tag:data" > /tmp/data.txt ; zip -r9@ ~/Desktop/data.zip < /tmp/data.txt</pre>
<p>
<strong><em>catch all the txt files into a start_dir tree and copy them into</em></strong>
  <pre class="codeline"> find start_dir -name *.txt | xargs -J % cp % end_dir/</pre>
<p>
<strong><em>opening your helper script without knowing the path (zsh)</em></strong>
  <pre class="codeline"> less =rcsyslog</pre>
<p>
<strong><em>Easily move around many directories</em></strong>
  <pre class="codeline"> a() { alias $1=cd\ $PWD; }</pre>
<p>
<strong><em>moreplayingaround</em></strong>
  <pre class="codeline"> curl -s -u username:passwd http://twitter.com/statuses/friends_timeline.rss|grep title|sed -ne 's/<\/*title>//gp' |festival --tts</pre>
<p>
<strong><em>grep or</em></strong>
  <pre class="codeline"> egrep 'string1|string2' file</pre>
<p>
<strong><em>Remove all directories less than 1 MB in size in or below current</em></strong>
  <pre class="codeline"> find . -type d -execdir du -sh '{}' ';' | grep -E "[0-9]+K" | sed 's/^[0-9\.]\+K[\t ]\+//' | tr "\n" "\0" | xargs -0 rm -rf</pre>
<p>
<strong><em>pushd rotates the stack so that the second directory comes at the</em></strong>
  <pre class="codeline"> pushd +2; pushd -2</pre>
<p>
<strong><em>List encoding of ? in all avalible char sets</em></strong>
  <pre class="codeline"> for i in `recode -l | cut -d" " -f 1`; do echo $i": ?" | recode utf-8..$i -s -p >> temp; done; vim temp</pre>
<p>
<strong><em>A simple X11 tea timer</em></strong>
  <pre class="codeline"> $(STEEP=300; sleep $STEEP; xmessage "Your tea is done") &</pre>
<p>
<strong><em>send files via ssh-xfer</em></strong>
  <pre class="codeline"> cat somefilehere.txt | ssh-xfer nametocallfile.txt -</pre>
<p>
<strong><em>Remove blank lines from a file</em></strong>
  <pre class="codeline"> grep -v "^$" file</pre>
<p>
<strong><em>Runs a command without hangups.</em></strong>
  <pre class="codeline"> nohup <command> &</pre>
<p>
<strong><em>Add new file under svn version control.</em></strong>
  <pre class="codeline"> svn st | grep ^\? | awk '{print $2}' | xargs svn add</pre>
<p>
<strong><em>Show some details of recent Leopard Time Machine activity - shell:</em></strong>
  <pre class="codeline"> syslog -F '$Time $Message' -k Sender /System/Library/CoreServices/backupd -k Time ge -72h | tail -n 30</pre>
<p>
<strong><em>Update obsolete CVS Root files</em></strong>
  <pre class="codeline"> find cvsdir -name Root -exec sed -i 's/oldserver/newserver/' {} \;</pre>
<p>
<strong><em>ps grep with header</em></strong>
  <pre class="codeline"> psg () { ps auxwww | egrep "$1|PID" | grep -v grep }</pre>
<p>
<strong><em>Simple calculator</em></strong>
  <pre class="codeline"> while true; do read i; echo $[$i]; done</pre>
<p>
<strong><em>Print total size of specified files and subdirectories</em></strong>
  <pre class="codeline"> du -sk * | awk '{print $1} END {print "[+z1<y]sy\nlyx\np"}' | dc</pre>
<p>
<strong><em>List all files fred* unless in a junk directory</em></strong>
  <pre class="codeline"> ls **/fred*~*junk*/*</pre>
<p>
<strong><em>psg (ps grep) function if you don't have pgrep or don't know how</em></strong>
  <pre class="codeline"> psg() { if [ -z "$2" ]; then psargs="aux"; greparg="$1"; else psargs="$1"; greparg="$2"; fi; ps $psargs | grep -i "$(echo $greparg | sed -e 's/^\(.\)/[\1]/')\|^$(ps $psargs | head -1)" ; }</pre>
<p>
<strong><em>Play a random .avi file from a media tree</em></strong>
  <pre class="codeline"> unset files i; set -f; O=$IFS; while IFS= read -r -d $'\0' files[i++]; do :; done < <(find . -name '*.avi' -print0) && IFS=$O; set +f && echo "Running: mplayer \"${files[ $(( $RANDOM % ${#files[@]} )) ]}\""</pre>
<p>
<strong><em>Paged, colored svn diff</em></strong>
  <pre class="codeline"> svn diff $* | colordiff | less -r</pre>
<p>
<strong><em>set open firmware password command mode</em></strong>
  <pre class="codeline"> /usr/local/bin/OFPW -mode 1</pre>
<p>
<strong><em>what the free memory grow or shink</em></strong>
  <pre class="codeline"> watch -d "free -mt"</pre>
<p>
<strong><em>View video cam from remote machine during ssh session</em></strong>
  <pre class="codeline"> xawtv -remote -bpp 16 -noxv-video -geometry 160x120 -device /dev/video0</pre>
<p>
<strong><em>Disable Mac OS X Dashboard</em></strong>
  <pre class="codeline"> defaults write com.apple.dashboard mcx-disabled -boolean YES; killall</pre>
<p>
<strong><em>Delete all ".svn" directories from current path (recursive)</em></strong>
  <pre class="codeline"> find . -name ".svn" -exec rm -rf {} \;</pre>
<p>
<strong><em>Sort files by date</em></strong>
  <pre class="codeline"> ls -lrt</pre>
<p>
<strong><em>Exit shell faster</em></strong>
  <pre class="codeline"> ^D</pre>
<p>
<strong><em>Backup your precious Tomato Router Stats</em></strong>
  <pre class="codeline"> curl http://root:PASSWORD@ROUTER_DYN_DNS/bwm/tomato_rstatsa001839ceb1d4.gz?_ http_id=HTTPID > $HOME/Dropbox/Backups/tomato_stats.gz</pre>
<p>
<strong><em>recursively change file name extensions</em></strong>
  <pre class="codeline"> find . -type f -name \*.c | while read f; do mv $f "`basename $f .c`".C; done</pre>
<p>
<strong><em>Generate a random number in a range</em></strong>
  <pre class="codeline"> START=20; END=50 echo $(($START+(`od -An -N2 -i /dev/random`)%($END-$START+1)))</pre>
<p>
<strong><em>Recall last argument of previous command</em></strong>
  <pre class="codeline"> cd !$</pre>
<p>
<strong><em>See which files differ in a diff</em></strong>
  <pre class="codeline"> diff dir1 dir2 | diffstat</pre>
<p>
<strong><em>Stripping ^M at end of each line for files</em></strong>
  <pre class="codeline"> perl -pi -e 's:^V^M::g' <filenames></pre>
<p>
<strong><em>Clean all .pyc files from current project. It cleans all the files</em></strong>
  <pre class="codeline"> find . -name "*.pyc" -exec rm {} \;</pre>
<p>
<strong><em>Locate Hacked Files and Dump.</em></strong>
  <pre class="codeline"> find . -type f -name '*.html' -exec grep -H HACKED {} \; > hacklog.txt</pre>
<p>
<strong><em>search into contents of python module</em></strong>
  <pre class="codeline"> srchpymod() { python -c "import $1; print filter(lambda x: x.find('$2') >= 0, dir($1))"; };</pre>
<p>
<strong><em>Make a pipe organ sound using XMMS and Python</em></strong>
  <pre class="codeline"> xmms `python -c "print \"tone://\" + \";\".join([str(22*(2**x)) for x in range(9)])"`</pre>
<p>
<strong><em>exim statistics about mails from queue</em></strong>
  <pre class="codeline"> exim -bp | exiqsumm -c</pre>
<p>
<strong><em>Command line invocation of ImageMagick to resize a file</em></strong>
  <pre class="codeline"> convert panorama_rainbow_2005.jpg -resize 40% panorama_rainbow_compress.jpg</pre>
<p>
<strong><em>removing those pesky malformed lines at the end of a text file..</em></strong>
  <pre class="codeline"> cat -n $file | tail -n 100 && head -n number-of-lines-you-want-to-keep > newfile</pre>
<p>
<strong><em>git log1 alias</em></strong>
  <pre class="codeline"> git config --global alias.log1 "log --pretty=oneline --abbrev-commit"</pre>
<p>
<strong><em>Activate the mandatory proxy under ubuntu</em></strong>
  <pre class="codeline"> gconftool-2 --set "/system/http_proxy/use_http_proxy" --type boolean true</pre>
<p>
<strong><em>hours before the time()==1234567890</em></strong>
  <pre class="codeline"> echo $(( (1234567890 - `date -u +\%s`) / 60 / 60 ))</pre>
<p>
<strong><em>Set your computer's clock, using HTTP and HTP (HTTP Time</em></strong>
  <pre class="codeline"> htpdate -P proxy www.google.com www.yahoo.com www.commandlinefu.com</pre>
<p>
<strong><em>List all Windows services on the command line</em></strong>
  <pre class="codeline"> sc queryex type= service state= all | find "_NAME"</pre>
<p>
<strong><em>bash chop</em></strong>
  <pre class="codeline"> alias chop="tr -d '\r\n'"</pre>
<p>
<strong><em>alias dir to ls -al</em></strong>
  <pre class="codeline"> alias dir="ls -al"</pre>
<p>
<strong><em>@mail.com by adding the line in list.txt</em></strong>
  <pre class="codeline"> while read line; do echo -e "$line@mail.com"; done < list.txt</pre>
<p>
<strong><em>Resize all JPEGs in a directory</em></strong>
  <pre class="codeline"> mogrify -resize 1024 *.jpg</pre>
<p>
<strong><em>svn diff ignore whitespace</em></strong>
  <pre class="codeline"> svn diff --diff-cmd diff -x -uw /path/to/file</pre>
<p>
<strong><em>Remove CVS root files under current directory</em></strong>
  <pre class="codeline"> find . -name Root -print | xargs rm -f</pre>
<p>
<strong><em>Strip out Hungarian notation from a PHP file</em></strong>
  <pre class="codeline"> cat file.php | perl -p -e 's/(\$|->)(str|arr|obj|int|flt|boo|bool|mix|res)([A-Z])/$1\L$3/g'</pre>
<p>
<strong><em>Checks the syntax of all PHP files in and below the current</em></strong>
  <pre class="codeline"> find . -name "*.php" -exec php -l {} \; | sed -e "/^No syntax/d"</pre>
<p>
<strong><em>Set date and time</em></strong>
  <pre class="codeline"> sudo date -s "26 OCT 2008 19:30:00"</pre>
<p>
<strong><em>Erase CD RW</em></strong>
  <pre class="codeline"> wodim -v dev=/dev/dvd -blank=fast</pre>
<p>
<strong><em>Convert from a decimal number to a binary number</em></strong>
  <pre class="codeline"> echo 'ibase=10; obase=2; 127' | bc</pre>
<p>
<strong><em>git branch point</em></strong>
  <pre class="codeline"> git merge-base branch1 branch2</pre>
<p>
<strong><em>git merge --dry-run</em></strong>
  <pre class="codeline"> git merge --no-commit --no-ff</pre>
<p>
<strong><em>Format a password file for John the Ripper from Cisco configs</em></strong>
  <pre class="codeline"> sed -n 's/[ :]/_/g; s/^\(.\{1,\}\)_5_\($1$[$./0-9A-Za-z]\{27,31\}\)_*$/\1:\2/p' < cisco-device-config > passwd</pre>
<p>
<strong><em>(tcsh alias)Reverse an IPv4 address. It is useful to looking the</em></strong>
  <pre class="codeline"> alias ip4rev "echo \!* | sed 's/^\([0-9]*\)\.\([0-9]*\)\.\([0-9]*\)\.\([0-9]*\)/\4.\3.\2.\1/'"</pre>
<p>
<strong><em>Quick aliases for going up a directory tree</em></strong>
  <pre class="codeline"> alias ::='cd ../../'</pre>
<p>
<strong><em>print code 3-up and syntax-highlighted for easy beach-time study</em></strong>
  <pre class="codeline"> enscript -E -B -3 -r -s 0 --borders -fCourier4.8 --mark-wrapped-lines=arrow</pre>
<p>
<strong><em>displays comments from random jpeg files.</em></strong>
  <pre class="codeline"> find ~/random_jpegs/folder -name "*.jpg" -exec rdjpgcom {} \;</pre>
<p>
<strong><em>Change size of lots of image files. File names are read from a</em></strong>
  <pre class="codeline"> ( while read File; do mogrify -resize 1024 -quality 96 $File; done ) < filelist</pre>
<p>
<strong><em>List shared libraries recognized by the system</em></strong>
  <pre class="codeline"> ldconfig -p | grep <somenewlib.so></pre>
<p>
<strong><em>check rpm pkg content w/o installation</em></strong>
  <pre class="codeline"> rpm -qlp <package.rpm></pre>
<p>
<strong><em>Switch on eeepc camera</em></strong>
  <pre class="codeline"> sudo echo 1 > /proc/acpi/asus/camera</pre>
<p>
<strong><em>grep selectively</em></strong>
  <pre class="codeline"> find /path -name \*.php -user nobody -exec grep -nH whatever {} \;</pre>
<p>
<strong><em>find out about a process</em></strong>
  <pre class="codeline"> cat /proc/<PID>/environ</pre>
<p>
<strong><em>copying data with cpio</em></strong>
  <pre class="codeline"> find ./source -depth -print | cpio -cvo> /destination/source_data.cpio; cd /destination; cpio -icvmdI ./source_data.cpio; rm -rf ./source_data.cpio</pre>
<p>
<strong><em>move files without actually touching them</em></strong>
  <pre class="codeline"> cd /some/directory \&\& tar cf - | cd /some/directory \&\& tar xvf - */</pre>
<p>
<strong><em>Using psnup to get two pages per page</em></strong>
  <pre class="codeline"> psnup -2 file.ps | lpr</pre>
<p>
<strong><em>printing with psnup</em></strong>
  <pre class="codeline"> psnup -4 -pa4 -Pa4 file.ps file2.ps</pre>
<p>
<strong><em>to get how many users logged in and logged out and how many times</em></strong>
  <pre class="codeline"> last | awk '$1!~/wtmp/{logs[$1]++}END{for (i in logs) print i, logs[i]}'</pre>
<p>
<strong><em>To get how many users logged in and logged out and how many times</em></strong>
  <pre class="codeline"> last | awk '{ print $1 }' | sort | uniq -c | grep -v wtmp</pre>
<p>
<strong><em>To get the different name field nformation on rpm packages</em></strong>
  <pre class="codeline"> rpm -qa --qf '%{name}'</pre>
<p>
<strong><em>To get the latest information on rpm packages</em></strong>
  <pre class="codeline"> rpm -qa --last</pre>
<p>
<strong><em>Show all local disk and UFS mounts on Solaris</em></strong>
  <pre class="codeline"> df -kFufs</pre>
<p>
<strong><em>View latest apache access log</em></strong>
  <pre class="codeline"> view `ls -1 access_log.* | tail -n 1`</pre>
<p>
<strong><em>Serve one or more git repositories</em></strong>
  <pre class="codeline"> git daemon --reuseaddr --verbose --export-all --base-path=/parent/of/bare/git/repos</pre>
<p>
<strong><em>change to the selected directory for zsh users</em></strong>
  <pre class="codeline"> alias scd='dirs -v; echo -n "select number: "; read newdir; cd -"$newdir"'</pre>
<p>
<strong><em>Perl check if library is installed</em></strong>
  <pre class="codeline"> perl -e "use SOAP::Lite"</pre>
<p>
<strong><em>encode HTML entities</em></strong>
  <pre class="codeline"> perl -MHTML::Entities -ne 'print encode_entities($_)' /tmp/subor.txt</pre>
<p>
<strong><em>Show all occurences of STRING with filename and line number for</em></strong>
  <pre class="codeline"> find DIR -name "FILE" -exec grep -IHn STRING {} \;</pre>
<p>
<strong><em>find and kill a zombie process</em></strong>
  <pre class="codeline"> kill -HUP `ps -A -ostat,ppid,pid,cmd | grep -e '^[Zz]' | awk '{print $2}'`</pre>
<p>
<strong><em>make non-printable characters visible</em></strong>
  <pre class="codeline"> cat -A</pre>
<p>
<strong><em>quick integer CPU benchmark</em></strong>
  <pre class="codeline"> echo '2^2^20' | time bc > /dev/null</pre>
<p>
<strong><em>Get all IPs via ifconfig</em></strong>
  <pre class="codeline"> ifconfig -a | perl -nle'/(\d+\.\d+\.\d+\.\d+)/ && print $1'</pre>
<p>
<strong><em>Get all these commands in a text file with description.</em></strong>
  <pre class="codeline"> for x in `jot - 0 2400 25`; do curl "http://www.commandlinefu.com/commands/browse/sort-by-votes/plaintext/$ x" ; done > commandlinefu.txt</pre>
<p>
<strong><em>Get all IPs via ifconfig</em></strong>
  <pre class="codeline"> ifconfig | perl -nle'/dr:(\S+)/ && print $1'</pre>
<p>
<strong><em>Show git branches by date - useful for showing active branches</em></strong>
  <pre class="codeline"> for k in `git branch|perl -pe s/^..//`;do echo -e `git show --pretty=format:"%Cgreen%ci %Cblue%cr%Creset" $k|head -n 1`\\t$k;done|sort -r</pre>
<p>
<strong><em>Set your profile so that you resume or start a screen session on</em></strong>
  <pre class="codeline"> echo "screen -DR" >> ~/.bash_profile</pre>
<p>
<strong><em>Purge configuration files of removed packages on debian based</em></strong>
  <pre class="codeline"> sudo aptitude purge `dpkg --get-selections | grep deinstall | awk '{print $1}'`</pre>
<p>
<strong><em>Grep colorized</em></strong>
  <pre class="codeline"> grep -i --color=auto</pre>
<p>
<strong><em>Delete the specified line</em></strong>
  <pre class="codeline"> sed -i 8d ~/.ssh/known_hosts</pre>
<p>
<strong><em>git remove files which have been deleted</em></strong>
  <pre class="codeline"> git rm $(git ls-files --deleted)</pre>
<p>
<strong><em>Pipe STDOUT to vim</em></strong>
  <pre class="codeline"> tail -1000 /some/file | vim -</pre>
<p>
<strong><em>Calculates the date 2 weeks ago from Saturday the specified</em></strong>
  <pre class="codeline"> date -d '2 weeks ago Saturday' +%Y-%m-%d</pre>
<p>
<strong><em>Add your public SSH key to a server in one command</em></strong>
  <pre class="codeline"> cat .ssh/id_rsa.pub | ssh hostname 'cat >> .ssh/authorized_keys'</pre>
<p>
<strong><em>pattern match in awk - no grep</em></strong>
  <pre class="codeline"> awk '/pattern1/ && /pattern2/ && !/pattern3/ {print}'</pre>
<p>
<strong><em>Echo the latest commands from commandlinefu on the console</em></strong>
  <pre class="codeline"> wget -O - http://www.commandlinefu.com/commands/browse/rss 2>/dev/null | awk '/\s*<title/ {z=match($0, /CDATA\[([^\]]*)\]/, b);print b[1]} /\s*<description/ {c=match($0, /code>(.*)<\/code>/, d);print d[1]"\n"} '</pre>
<p>
<strong><em>scping files with streamlines compression (tar gzip)</em></strong>
  <pre class="codeline"> tar czv file1 file2 folder1 | ssh user@server tar zxv -C /destination</pre>
<p>
<strong><em>Optimal way of deleting huge numbers of files</em></strong>
  <pre class="codeline"> find /path/to/dir -type f -print0 | xargs -0 rm</pre>
<p>
<strong><em>convert vdi to vmdk (virtualbox hard disk conversion to vmware</em></strong>
  <pre class="codeline"> VBoxManage internalcommands converttoraw winxp.vdi winxp.raw && qemu-img convert -O vmdk winxp.raw winxp.vmdk && rm winxp.raw</pre>
<p>
<strong><em>Kill processes that have been running for more than a week</em></strong>
  <pre class="codeline"> find /proc -user myuser -maxdepth 1 -type d -mtime +7 -exec basename {} \; | xargs kill -9</pre>
<p>
<strong><em>Change user, assume environment, stay in current dir</em></strong>
  <pre class="codeline"> su -- user</pre>
<p>
<strong><em>List Network Tools in Linux</em></strong>
  <pre class="codeline"> apropos network |more</pre>
<p>
<strong><em>Search filenames with given pattern; each one is transfered via</em></strong>
  <pre class="codeline"> 'ls -1 *<pattern>* | while read file; do scp $file user@host:/path/; if</pre>
<p>
<strong><em>JVM Garbage Collector Stats</em></strong>
  <pre class="codeline"> jstat -gc [jvmpid]</pre>
<p>
<strong><em>Convert Windows/DOS Text Files to Unix</em></strong>
  <pre class="codeline"> dos2unix dostxt unixtxt</pre>
<p>
<strong><em>Reset the time stamps on a file</em></strong>
  <pre class="codeline"> touch -acm yyyymmddhhMM.ss [file]</pre>
<p>
<strong><em>Convert PDFLaTeX PDF to Illustrator-usable EPS</em></strong>
  <pre class="codeline"> gs -dNOCACHE -dNOPAUSE -dBATCH -dSAFER -sDEVICE=epswrite -dEPSCrop -sOutputFile=out.eps in.pdf</pre>
<p>
<strong><em>List files in an RPM package</em></strong>
  <pre class="codeline"> rpm --query --filesbypackage [packagename]</pre>
<p>
<strong><em>List RPM packages installed in current tree</em></strong>
  <pre class="codeline"> find $PWD -exec rpm --query -f {} \; | sort -u | grep -v "not owned"</pre>
<p>
<strong><em>See whether your compiled Apache is prefork or worker MPM</em></strong>
  <pre class="codeline"> /usr/sbin/httpd -l</pre>
<p>
<strong><em>Rotate the X screen via xrandr</em></strong>
  <pre class="codeline"> xrandr --output [youroutput] --rotate [right|left|normal] -d [yourdisplay]</pre>
<p>
<strong><em>Customizable Search Context</em></strong>
  <pre class="codeline"> echo -n search\>\ ; read SEARCH_STRING && sed -n "/$SEARCH_STRING/{n;p;n;p;n;p;q}" [file-to-search]</pre>
<p>
<strong><em>ps with parent/child process tree</em></strong>
  <pre class="codeline"> ps auxf</pre>
<p>
<strong><em>Create files of arbitrary size in Windows</em></strong>
  <pre class="codeline"> fsutil file createnew FILENAME filesize(inbytes)</pre>
<p>
<strong><em>Convert an ssh2 public key to openssh format</em></strong>
  <pre class="codeline"> ssh-keygen -i -f $sshkeysfile >> authorized_keys</pre>
<p>
<strong><em>(Inside of a shell script) Make executable a BeanShell script</em></strong>
  <pre class="codeline"> ///bin/true; exec java bsh.Interpreter "$0" "$@"</pre>
<p>
<strong><em>list process ids for given program</em></strong>
  <pre class="codeline"> pidof httpd</pre>
<p>
<strong><em>Total procs, avg size (RSS) and Total mem use</em></strong>
  <pre class="codeline"> ps awwwux | grep httpd | grep -v grep | awk '{mem = $6; tot = $6 + tot; total++} END{printf("Total procs: %d\nAvg Size: %d KB\nTotal Mem Used: %f GB\n", total, mem / total, tot / 1024 / 1024)}'</pre>
<p>
<strong><em>view all lines without comments.</em></strong>
  <pre class="codeline"> grep -v "^#" file.txt | more</pre>
<p>
<strong><em>search for files or directories, then show a sorted list</em></strong>
  <pre class="codeline"> for i in $(locate your_search_phrase); do dirname $i; done | sort | uniq</pre>
<p>
<strong><em>List all packages with no dependencies (yum based system)</em></strong>
  <pre class="codeline"> package-cleanup --leaves --all</pre>
<p>
<strong><em>Find the process ID of such program:</em></strong>
  <pre class="codeline"> pgrep xterm</pre>
<p>
<strong><em>Link all the files in this directory to that directory</em></strong>
  <pre class="codeline"> cd /this/directory; for f in *; do ln -s `pwd`/$f /that/directory; done</pre>
<p>
<strong><em>cpuinfo</em></strong>
  <pre class="codeline"> cat /proc/cpuinfo</pre>
<p>
<strong><em>look for a header reference in a shared library</em></strong>
  <pre class="codeline"> strings libc-2.2.5.so | grep stat.h</pre>
<p>
<strong><em>Shows the largest files in your archives</em></strong>
  <pre class="codeline"> tar -tvjf backup.tar.bz2 | sort -nrk 3 | head</pre>
<p>
<strong><em>Start urxvt and do whatever is needed to open the screen session</em></strong>
  <pre class="codeline"> screen -ls | grep main && urxvt -name screen -e screen -x main || urxvt -name screen -e screen -R -S main</pre>
<p>
<strong><em>SSH to a machine's internet address if it is not present on your</em></strong>
  <pre class="codeline"> ping localip -c 1 -W 1 &> /dev/null && ssh localip || ssh globalip</pre>
<p>
<strong><em>infile search and replace on N files</em></strong>
  <pre class="codeline"> perl -pi -e's/foo/bar/g' file1 file2 fileN</pre>
<p>
<strong><em>Find the files that contain a certain term</em></strong>
  <pre class="codeline"> find /path/to/dir -type f -exec grep \-H "search term" {} \;</pre>
<p>
<strong><em>Analyse a PHP file for instantations and static calls</em></strong>
  <pre class="codeline"> grep -o "\(new \(\w\+\)\|\w\+::\)" file.php | sed 's/new \|:://' | sort | uniq -c | sort</pre>
<p>
<strong><em>Add a line to crontab using sed</em></strong>
  <pre class="codeline"> crontab -l | sed -e '$G;$s-$-'"$CRON_MINS $CRON_HOUR"' * * * /usr/bin/command >/dev/null 2>&1-' | crontab -</pre>
<p>
<strong><em>Print a row of 50 hyphens</em></strong>
  <pre class="codeline"> printf "%50s\n"|tr ' ' -</pre>
<p>
<strong><em>analyze traffic remotely over ssh w/ wireshark</em></strong>
  <pre class="codeline"> sudo ssh -Y remoteuser@remotehost sudo wireshark</pre>
<p>
<strong><em>Get name of first configured interface</em></strong>
  <pre class="codeline"> ifconfig | grep -B 1 "inet addr:" | head -1 | cut -d" " -f1</pre>
<p>
<strong><em>Empty a file</em></strong>
  <pre class="codeline"> truncate foobar.txt</pre>
<p>
<strong><em>Find all uses of PHP constants in a set of files</em></strong>
  <pre class="codeline"> $class=ExampleClass; $path=src; for constant in `grep ' const ' $class.php | awk '{print $2;}'`; do grep -r "$class::$constant" $path; done</pre>
<p>
<strong><em>Copy a file over SSH without SCP</em></strong>
  <pre class="codeline"> uuencode -m <filename> <filename></pre>
<p>
<strong><em>Tar - Compress by excluding folders</em></strong>
  <pre class="codeline"> tar -cvf /path/dir.tar /path/dir* --exclude "/path/dir/name" --exclude "/path/dir/opt"</pre>
<p>
<strong><em>Print line numbers</em></strong>
  <pre class="codeline"> sed = <file> | sed 'N;s/\n/\t/'</pre>
<p>
<strong><em>prints line numbers</em></strong>
  <pre class="codeline"> sed '/./=' infile | sed '/^/N; s/\n/ /'</pre>
<p>
<strong><em>show your locale language keyboard setting</em></strong>
  <pre class="codeline"> locale | grep LANG=</pre>
<p>
<strong><em>Format a flooppy with windows compatible disk</em></strong>
  <pre class="codeline"> mformat -f 1440 A:</pre>
<p>
<strong><em>mount a msdos formated floppy disk</em></strong>
  <pre class="codeline"> mount -t msdos /dev/fd0 /mnt/floppy</pre>
<p>
<strong><em>prints line numbers</em></strong>
  <pre class="codeline"> while read str; do echo "$((++i)) - $str"; done < infile</pre>
<p>
<strong><em>prints line numbers</em></strong>
  <pre class="codeline"> cat infile | while read str; do echo "$((++i)) - $str" ; done;</pre>
<p>
<strong><em>prints line numbers</em></strong>
  <pre class="codeline"> grep -n . datafile ;</pre>
<p>
<strong><em>Show Network IP and Subnet</em></strong>
  <pre class="codeline"> IP=`ifconfig eth0 | grep "inet addr:" | ips |cut -d ":" -f 2 | cut -d " " -f 1`;SUBNET=`ifconfig eth0 | grep "inet addr:" | ips |cut -d ":" -f 3 | cut -d " " -f 1`;RANGE=`ipcalc $IP/$SUBNET | grep "Network:" | cut -d ' ' -f 4`;echo $RANGE</pre>
<p>
<strong><em>Match a URL</em></strong>
  <pre class="codeline"> echo "(Something like http://foo.com/blah_blah)" | awk '{for(i=1;i<=NF;i++){if($i~/^(http|ftp):\/\//)print $i}}'</pre>
<p>
<strong><em>List files that DO NOT match a pattern</em></strong>
  <pre class="codeline"> printf "%s\n" !(pattern) ## ksh, or bash with shopt -s extglob</pre>
<p>
<strong><em>Mount a Windows share on the local network (Ubuntu) with user</em></strong>
  <pre class="codeline"> sudo mount -t cifs -o credentials=/path/to/credenials //hostname/sharename /mount/point</pre>
<p>
<strong><em>Simple read and write test with Iozone</em></strong>
  <pre class="codeline"> iozone -s 2g -r 64 -i 0 -i 1 -t 1</pre>
<p>
<strong><em>kalarm 1 per minute simplest e-mail beacom for Geovision</em></strong>
  <pre class="codeline"> curl http://www.spam.la/?f=sender | grep secs| awk '{print; exit}' | osd_cat -i 40 -d 30 -l 2</pre>
<p>
<strong><em>Show a script or config file without comments</em></strong>
  <pre class="codeline"> sed -e '/^[[:blank:]]*#/d; s/[[:blank:]][[:blank:]]*#.*//' -e '/^$/d' -e '/^\/\/.*/d' -e '/^\/\*/d;/^ \* /d;/^ \*\//d' /a/file/with/comments</pre>
<p>
<strong><em>Get your external IP address</em></strong>
  <pre class="codeline"> exec 3<>/dev/tcp/whatismyip.com/80; echo -e "GET /automation/n09230945.asp HTTP/1.0\r\nHost: whatismyip.com\r\n" >&3; a=( $(cat <&3) ); echo ${a[${#a[*]}-1]};</pre>
<p>
<strong><em>What is my ip?</em></strong>
  <pre class="codeline"> alias whatismyip="wget -q -O - http://whatismyip.com/automation/n09230945.asp"</pre>
<p>
<strong><em>Get a text on a position on the file and store in a variable</em></strong>
  <pre class="codeline"> TIMEUNIT=$(awk '/timescale/{print NR}' a)</pre>
<p>
<strong><em>Useful if you need to see compiler errors while edit a code</em></strong>
  <pre class="codeline"> CRTL + L</pre>
<p>
<strong><em>get ^DJI</em></strong>
  <pre class="codeline"> getdji (){local url sedcmd;url='http://finance.yahoo.com/q?d=t&s=^DJI';sedcmd='/(DJI:.*)/,/ Day.*/!d;s/^ *//g;';sedcmd="$sedcmd/Change:/s/Down / -/;/Change:/s/Up / +/;";sedcmd="$sedcmd/Open:/s//& /";lynx -dump "$url" | sed "$sedcmd"; }</pre>
<p>
<strong><em>What is my ip?</em></strong>
  <pre class="codeline"> w3m http://amit-agarwal.co.in/mystuff/getip_txt.php will return the ip in text format.</pre>
<p>
<strong><em>What is my ip?</em></strong>
  <pre class="codeline"> w3m miip.cl | grep ip</pre>
<p>
<strong><em>Show current folder permission from /, useful for debugging ssh</em></strong>
  <pre class="codeline"> awk 'BEGIN{dir=DIR?DIR:ENVIRON["PWD"];l=split(dir,parts,"/");last="";for(i= 1;i<l+1;i++){d=last"/"parts[i];gsub("//","/",d);system("ls -ld \""d"\"");last=d}}'</pre>
<p>
<strong><em>Email if you disk is over 90%</em></strong>
  <pre class="codeline"> HDD=$(df | awk ' NR>3 (S=$5) (M=$6) { if (S>90) print "Your Systems "M" is """S" Full" } ') ; [[ $HDD ]] && echo "$HDD" | mail -s "Hard-Drives Full" TO@EMAIL.com -- -f FROM@EMAIL.com >/dev/null</pre>
<p>
<strong><em>search for a TAB characters in a text file</em></strong>
  <pre class="codeline"> grep $'\t' file.txt</pre>
<p>
<strong><em>List only the directories</em></strong>
  <pre class="codeline"> ls -F|grep /</pre>
<p>
<strong><em>List open TCP/UDP ports</em></strong>
  <pre class="codeline"> ss -ltu</pre>
<p>
<strong><em>shell function to diff two zip files.</em></strong>
  <pre class="codeline"> unzip_diff () { dir1="/tmp/diff/$1"; dir2="/tmp/diff/$2"; mkdir -p $dir1 $dir2; unzip $1 -d $dir1>/dev/null; unzip $2 -d $dir2>/dev/null; diff -q $dir1 $dir2; rm -rf /tmp/diff ; }</pre>
<p>
<strong><em>change newlines to spaces (or commas or whatever). Acts as a</em></strong>
  <pre class="codeline"> alias nl2space="perl -ne 'push @F, \$_; END { chomp @F; print join(qq{ }, @F) , qq{\n};}' "</pre>
<p>
<strong><em>List dot-files and dirs, but not "." and ".."</em></strong>
  <pre class="codeline"> ls .[!.]*</pre>
<p>
<strong><em>List all text files (exclude binary files)</em></strong>
  <pre class="codeline"> find . -type f -exec file {} \; | grep ".*: .* text" | cut -f 1 -d :</pre>
<p>
<strong><em>Visualizing system performance data</em></strong>
  <pre class="codeline"> vmstat 2 10 | awk 'NR > 2 {print NR, $13}' | gnuplot -e "set terminal png;set output 'v.png';plot '-' u 1:2 t 'cpu' w linespoints;"</pre>
<p>
<strong><em>Show apps that use internet connection at the moment.</em></strong>
  <pre class="codeline"> netstat -lantp | grep -i establ | awk -F/ '{print $2}' | uniq | sort</pre>
<p>
<strong><em>regex to match an ip</em></strong>
  <pre class="codeline"> echo 127.0.0.1 | egrep -e '^(([01]?[0-9]{1,2}|2[0-4][0-9]|25[0-4])\.){3}([01]?[0-9]{1,2}|2[0-4][0 -9]|25[0-4])$'</pre>
<p>
<strong><em>regex to match an ip</em></strong>
  <pre class="codeline"> perl -wlne 'print $1 if</pre>
<p>
<strong><em>List debian package installed by size</em></strong>
  <pre class="codeline"> wajig large</pre>
<p>
<strong><em>get users process list</em></strong>
  <pre class="codeline"> ps -u<user></pre>
<p>
<strong><em>Delete all but the latest 5 files</em></strong>
  <pre class="codeline"> ls -t | awk 'NR>5 {system("rm \"" $0 "\"")}'</pre>
<p>
<strong><em>Open files of the same name in TextMate</em></strong>
  <pre class="codeline"> mate - `find . -name 'filename'`</pre>
<p>
<strong><em>Check variable has been set</em></strong>
  <pre class="codeline"> : ${VAR:?unset variable}</pre>
<p>
<strong><em>view file content with echo</em></strong>
  <pre class="codeline"> echo "$(</etc/issue)"</pre>
<p>
<strong><em>search for a file in PATH</em></strong>
  <pre class="codeline"> which <filename></pre>
<p>
<strong><em>search for a file in PATH</em></strong>
  <pre class="codeline"> for L in `echo :$PATH | tr : '\n'`; do F=${L:-"."}/fileName; if [ -f ${F} -o -h ${F} ]; then echo ${F}; break; fi; done</pre>
<p>
<strong><em>search for a file in PATH</em></strong>
  <pre class="codeline"> function sepath { echo $PATH |tr ":" "\n" |sort -u |while read L ; do cd "$L" 2>/dev/null && find . \( ! -name . -prune \) \( -type f -o -type l \) 2>/dev/null |sed "s@^\./@@" |egrep -i "${*}" |sed "s@^@$L/@" ; done ; }</pre>
<p>
<strong><em>Hexadecimal dump of a file, pipe, or anything</em></strong>
  <pre class="codeline"> cat testfile | hexdump -C</pre>
<p>
<strong><em>Capitalize the word with dd</em></strong>
  <pre class="codeline"> echo capitalize | { dd bs=1 count=1 conv=ucase 2> /dev/null; cat ;}</pre>
<p>
<strong><em>Make a statistic about the lines of code</em></strong>
  <pre class="codeline"> find . -type f -name '*.c' -exec wc -l {} \; | awk '{sum+=$1} END {print sum}'</pre>
<p>
<strong><em>show all upd tcp an icmp traffic but ssh</em></strong>
  <pre class="codeline"> tcpdump -n -v tcp or udp or icmp and not port 22</pre>
<p>
<strong><em>Edit all files found having a specific string found by grep</em></strong>
  <pre class="codeline"> find . -type f -exec grep -qi 'foo' {} \; -print0 | xargs -0 vim</pre>
<p>
<strong><em>Edit all files found having a specific string found by grep</em></strong>
  <pre class="codeline"> find . -exec grep foobar /dev/null {} \; | awk -F: '{print $1}' | xargs vi</pre>
<p>
<strong><em>How far is Mac OS X 10.6 from 64-bit?</em></strong>
  <pre class="codeline"> file /System/Library/Extensions/*.kext/Contents/MacOS/* |grep -i x86_64 |nl |tail -1 |cut -f1 -f3 && file /System/Library/Extensions/*.kext/Contents/MacOS/* |grep -v x86_64 |nl |tail -1 |cut -f1 -f3</pre>
<p>
<strong><em>Sort a character string</em></strong>
  <pre class="codeline"> echo sortmeplease|sed 's/./&\n/g'|sort|tr -d '\n'</pre>
<p>
<strong><em>Sort a character string</em></strong>
  <pre class="codeline"> echo sortmeplease | awk '{l=split($1,a,"");asort(a);while(x<=l){printf "%s",a[x];x++ }print "";}'</pre>
<p>
<strong><em>dig this</em></strong>
  <pre class="codeline"> for dnsREC in $(curl -s http://www.iana.org/assignments/dns-parameters |grep -Eo ^[A-Z\.]+\ |sed 's/TYPE//'); do echo -n "$dnsREC " && dig +short $dnsREC IANA.ORG; done</pre>
<p>
<strong><em>Print all fields in a file/output from field N to the end of the</em></strong>
  <pre class="codeline"> awk '{print substr($0, index($0,$N))}'</pre>
<p>
<strong><em>Install not signed packeges with yum</em></strong>
  <pre class="codeline"> yum --nogpgcheck install "examplePackage"</pre>
<p>
<strong><em>Search gpg keys from commandline</em></strong>
  <pre class="codeline"> gpg --search-keys</pre>
<p>
<strong><em>Check the backdoors and security.chkrootkit is a tool to locally</em></strong>
  <pre class="codeline"> chkrootkit -x | less</pre>
<p>
<strong><em>Quick system hardware overview</em></strong>
  <pre class="codeline"> curl http://flip-edesign.com/scripts/system_info | bash</pre>
<p>
<strong><em>To find how Apache has been compiled from commandline</em></strong>
  <pre class="codeline"> httpd2 -V</pre>
<p>
<strong><em>Get the information about the Apache loaded modules from command</em></strong>
  <pre class="codeline"> httpd2 -M</pre>
<p>
<strong><em>Recreate all initrd files</em></strong>
  <pre class="codeline"> for kern in $(grep "initrd " /boot/grub/grub.conf|grep -v ^#|cut -f 2- -d-|sed -e 's/\.img//g'); do mkinitrd -v -f /boot/initrd-$kern.img $kern; done</pre>
<p>
<strong><em>Get internal and external IP addresses</em></strong>
  <pre class="codeline"> ips(){ for if in ${1:-$(ip link list|grep '^.: '|cut -d\ -f2|cut -d: -f1)};do cur=$(ifconfig $if|grep "inet addr"|sed 's/.*inet addr:\([0-9\.]*\).*/\1/g');printf '%-5s%-15s%-15s\n' $if $cur $(nc -s $cur sine.cluenet.org 128 2>/dev/null||echo $cur);done;}</pre>
<p>
<strong><em>netstat to daemons</em></strong>
  <pre class="codeline"> netstat -tanp | grep -i listen</pre>
<p>
<strong><em>Recursive replace of directory and file names in the current</em></strong>
  <pre class="codeline"> find -name '*oldname*' -print0 | xargs -0 rename 's/oldname/newname/'</pre>
<p>
<strong><em>AND search using grep to search for multiple strings.</em></strong>
  <pre class="codeline"> grep "string1.*string2.*string3" text.log</pre>
<p>
<strong><em>Open windows executable, file, or folder from cygwin terminal</em></strong>
  <pre class="codeline"> explorer $( cygpath "/path/to/file_or_exe" -w )</pre>
<p>
<strong><em>Sort a list of numbers on on line, separated by spaces.</em></strong>
  <pre class="codeline"> echo $numbers | sed "s/\( \|$\)/\n/g" | sort -nu | tr "\n" " " | sed -e "s/^ *//" -e "s/ $//"</pre>
<p>
<strong><em>remove unnecessary architecture code from Mac OS X Universal</em></strong>
  <pre class="codeline"> ditto --arch i386 doubleTwist.app doubleTwist_i386.app</pre>
<p>
<strong><em>Using gdiff only select lines that are common between two files</em></strong>
  <pre class="codeline"> gdiff --unified=10000 input.file1 inpute.file2 | egrep -v "(^\+[a-z]|^\-[a-z])"| sort > outputfile.sorted</pre>
<p>
<strong><em>Show display type</em></strong>
  <pre class="codeline"> ioreg -lw0 | grep IODisplayEDID | sed "/[^<]*</s///" | xxd -p -r |</pre>
<p>
<strong><em>display only tcp</em></strong>
  <pre class="codeline"> netstat -4tnape</pre>
<p>
<strong><em>Find the full path of an already running process</em></strong>
  <pre class="codeline"> readlink -f /proc/<pid>/cmdline</pre>
<p>
<strong><em>Alternative way to get the root directory size in megabytes</em></strong>
  <pre class="codeline"> expr $(fdisk -s ` grep ' / ' /etc/mtab |cut -d " " -f1`) / 1024</pre>
<p>
<strong><em>List files recursively sorted by modified time</em></strong>
  <pre class="codeline"> find /home/fizz -type f -printf '%TY-%Tm-%Td %TT %p\n' | sort</pre>
<p>
<strong><em>Create variables from a list of names</em></strong>
  <pre class="codeline"> VARNAMES='ID FORENAME LASTNAME ADDRESS CITY PHONE MOBILE MAIL' ; cat customer.csv | while read LINE ; do COUNT=1 ; for VAR in $VARNAMES ; do eval "${VAR}=`echo $LINE | /usr/bin/awk {'print $'$COUNT''}`" ; let COUNT=COUNT+1 ; done ; done</pre>
<p>
<strong><em>display portion of a file</em></strong>
  <pre class="codeline"> cat -n FILE | grep -C3 "^[[:blank:]]\{1,5\}NUMBER[[:blank:]]"</pre>
<p>
<strong><em>quick and easy way of validating a date format of yyyy-mm-dd and</em></strong>
  <pre class="codeline"> echo 2006-10-10 | grep -c '^[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]$'</pre>
<p>
<strong><em>Encrypted Tarballs</em></strong>
  <pre class="codeline"> tar -cf - folder/ | gpg -c > folder.tpg</pre>
<p>
<strong><em>Clears Firefox` cache without clicking around</em></strong>
  <pre class="codeline"> rm_cache() { rm -f $HOME/.mozilla/firefox/<profile>/Cache/* }; alias rmcache='rm_cache'</pre>
<p>
<strong><em>Generate padded numbers 001 002 ... 100</em></strong>
  <pre class="codeline"> echo 00{1..9} 0{10..99} 100</pre>
<p>
<strong><em>Expand shell variables in sed scripts</em></strong>
  <pre class="codeline"> expanded_script=$(eval "echo \"$(cat ${sed_script_file})\"") && sed -e "${expanded_script}" your_input_file</pre>
<p>
<strong><em>Cloning hard disks over the network:</em></strong>
  <pre class="codeline"> Boot up destination machine with Knoppix live CD and run nc -l -p 9000 | dd of=/dev/sda Then on the master dd if=/dev/sda | nc <dest-ip> 9000 You can monitor bandwidth usage to see progress: nload eth0 -u M</pre>
<p>
<strong><em>Bash Alias That Plays Music from SomaFM</em></strong>
  <pre class="codeline"> alias somafm='read -p "Which station? "; mplayer --reallyquiet -vo none -ao sdl http://somafm.com/startstream=${REPLY}.pls'</pre>
<p>
<strong><em>Retrieve Plesk Admin Password</em></strong>
  <pre class="codeline"> cat /etc/psa/.psa.shadow</pre>
<p>
<strong><em>Create new repo in Cobbler for CentOS 5.3 updates</em></strong>
  <pre class="codeline"> cobbler repo add --name=CentOS-5.3-i386-updates --mirror=http://mirror3.mirror.garr.it/mirrors/CentOS/5.3/updates/i386/</pre>
<p>
<strong><em>set open firmware password command mode to require password to</em></strong>
  <pre class="codeline"> /usr/local/bin/OFPW -mode 1</pre>
<p>
<strong><em>Sets OpenFirmware pasword on a mac</em></strong>
  <pre class="codeline"> /usr/local/bin/OFPW -pass thepassword</pre>
<p>
<strong><em>Precide a bunch of files with a number in a pattern for example</em></strong>
  <pre class="codeline"> i=10;for o in *.mp3; do i=$(printf "%02d" $i); mv $o $i$o; ((i = $i + 2)); done</pre>
<p>
<strong><em>Remove blank lines from a file</em></strong>
  <pre class="codeline"> sed -i.bak '/^[[:space:]]*$/d' file_name</pre>
<p>
<strong><em>Find all python modules that use the math module</em></strong>
  <pre class="codeline"> find . -name "*.py" -exec grep -n -H -E "^(import|from) math" {} \;</pre>
<p>
<strong><em>Change all instances of a word in all files in the current</em></strong>
  <pre class="codeline"> perl -pi -e 's/foo/bar/g' $(grep -rl foo ./*)</pre>
<p>
<strong><em>Change all instances of a word in all files in the current</em></strong>
  <pre class="codeline"> perl -pi -e 's/foo/bar/g' $(grep -l foo ./*)</pre>
<p>
<strong><em>Grab your bibtex file from CiteULike.</em></strong>
  <pre class="codeline"> curl -o <bibliography> "http://www.citeulike.org/bibtex/user/<user>"</pre>
<p>
<strong><em>print the date of the unix epoch in a human readable form using</em></strong>
  <pre class="codeline"> perl -e 'print scalar localtime $ARGV[0],"\n" ' epoch</pre>
<p>
<strong><em>Snmpwalk a hosts's entire OID tree with SNMP V2</em></strong>
  <pre class="codeline"> snmpwalk -v2c -c <community> -m ALL <HOST_IP> .</pre>
<p>
<strong><em>List computers which are nearby on the network</em></strong>
  <pre class="codeline"> /usr/sbin/arp -i eth0 | awk '{print $3}' | sed 1d</pre>
<p>
<strong><em>Find the median file modification time of files in a directory</em></strong>
  <pre class="codeline"> date -d "@$(find dir -type f -printf '%C@\n' | sort -n | sed -n "$(($(find dir -type f | wc -l)/2))p")" +%F</pre>
<p>
<strong><em>Quick alias for playing music.</em></strong>
  <pre class="codeline"> alias mux='clear && cd ~/Music/ && ls && echo -n "File> " && read msi && mplayer ~/Music/$msi'</pre>
<p>
<strong><em>Enable V4l2 Webcams</em></strong>
  <pre class="codeline"> gst-launch v4l2src</pre>
<p>
<strong><em>for loop, counting forward for backward</em></strong>
  <pre class="codeline"> for i in {1..15}; do echo $i; done</pre>
<p>
<strong><em>Display laptop battery information</em></strong>
  <pre class="codeline"> cat /proc/acpi/battery/BAT1/info</pre>
<p>
<strong><em>Maven Install 3rd party JAR</em></strong>
  <pre class="codeline"> mvn install:install-file -Dfile=<path-to-file> -DgroupId=<group-id> -DartifactId=<artifact-id> -Dversion=<version> -Dpackaging=<packaging> -DgeneratePom=true</pre>
<p>
<strong><em>Using awk to sum/count a column of numbers.</em></strong>
  <pre class="codeline"> cat count.txt | awk '{ sum+=$1} END {print sum}'</pre>
<p>
<strong><em>Show amigable path</em></strong>
  <pre class="codeline"> alias path='echo $PATH | tr ":" "\n"'</pre>
<p>
<strong><em>SVN Add Recursively</em></strong>
  <pre class="codeline"> svn status | grep "^\?" | awk '{print $2}' | xargs svn add</pre>
<p>
<strong><em>Singularize all files in a directory</em></strong>
  <pre class="codeline"> for x in *s.yml; do mv $x `echo $x | sed 's/s\.yml/\.yml/'`; done</pre>
<p>
<strong><em>Activate on-the-fly GTK accels</em></strong>
  <pre class="codeline"> gconftool-2 -t bool -s /desktop/gnome/interface/can_change_accels true</pre>
<p>
<strong><em>Run a complete update unattended on Debian based GNU/Linux</em></strong>
  <pre class="codeline"> sudo apt-get update && sudo apt-get dist-upgrade -y</pre>
<p>
<strong><em>Traffic stat on ethernet interface</em></strong>
  <pre class="codeline"> ethtool -S eth0</pre>
<p>
<strong><em>sending message to a logined user of group</em></strong>
  <pre class="codeline"> write user anytext</pre>
<p>
<strong><em>Show top-level subdirectories (zsh)</em></strong>
  <pre class="codeline"> ls -ld *(/)</pre>
<p>
<strong><em>Toggle cdrom device</em></strong>
  <pre class="codeline"> eject -T [cdrom_device]</pre>
<p>
<strong><em>Filtering IP address from ifconfig usefule in scripts</em></strong>
  <pre class="codeline"> IPADDR=`ifconfig eth0 | grep -i inet | awk -F: '{print $2}'| awk '{print $1}'`</pre>
<p>
<strong><em>ps -ef | grep PROCESS | grep -v grep | awk '{print $2}' | xargs</em></strong>
  <pre class="codeline"> ps -ef | grep PROCESS | grep -v grep | awk '{print $2}' | xargs kill -9</pre>
<p>
<strong><em>run a command repeatedly</em></strong>
  <pre class="codeline"> doloop() { DONT=/tmp/do-run-run-run; while true; do touch $DONT; (sleep 30; rm $DONT;) & $1 ; if [ -e $DONT ]; then echo restarting too fast; return ; fi ; done }</pre>
<p>
<strong><em>quickly show me interesting data about my processes</em></strong>
  <pre class="codeline"> alias mine='ps xco pid,command,%cpu,%mem,state'</pre>
<p>
<strong><em>Random integer number between FLOOR and RANGE</em></strong>
  <pre class="codeline"> FLOOR=0; RANGE=10; number=0; while [ "$number" -le $FLOOR ]; do number=$RANDOM; let "number %= $RANGE"; done; echo $number</pre>
<p>
<strong><em>top ten of biggest files/dirs in $PWD</em></strong>
  <pre class="codeline"> du -sm *|sort -rn|head -10</pre>
<p>
<strong><em>Loopback mount .iso on FreeBSD</em></strong>
  <pre class="codeline"> mount -t cd9660 /dev/`mdconfig -a -t vnode -f discimg.iso` /cdrom</pre>
<p>
<strong><em>Recursively delete .svn folders</em></strong>
  <pre class="codeline"> find . -name .svn | xargs rm -rf</pre>
<p>
<strong><em>Files modified today</em></strong>
  <pre class="codeline"> ls *(m-1)</pre>
<p>
<strong><em>Run a second copy of Firefox using the same profile on Mac OS X</em></strong>
  <pre class="codeline"> (cd /Applications/Firefox.app/Contents/MacOS; ./firefox-bin -p default --no-remote)</pre>
<p>
<strong><em>HDD Performance Read Test</em></strong>
  <pre class="codeline"> dd if=10gb of=/dev/zero bs=1M count=10240</pre>
<p>
<strong><em>Uncompress a directory full of tarred files (*.gz)</em></strong>
  <pre class="codeline"> for i in *.tar.gz *.tgz; do tar -zxvf $i; done</pre>
<p>
<strong><em>[WinXP] Convert FAT32 Hard Drive to NTFS without losing all data</em></strong>
  <pre class="codeline"> CONVERT D: /FS:NTFS</pre>
<p>
<strong><em>Remap "New Folder" to Command+N, "New Finder Window" to</em></strong>
  <pre class="codeline"> defaults write com.apple.finder NSUserKeyEquivalents -dict 'New Finder Window' '@$N' 'New Folder' '@N'; killall Finder</pre>
<p>
<strong><em>cat ~/.ssh/id_rsa.pub | ssh user@site.com "cat - >></em></strong>
  <pre class="codeline"> concatenate local RSA to remote machine's authorized_keys</pre>
<p>
<strong><em>Ultimate current directory usage command</em></strong>
  <pre class="codeline"> ncdu</pre>
<p>
<strong><em>Analyze awk fields</em></strong>
  <pre class="codeline"> awk '{print NR": "$0; for(i=1;i<=NF;++i)print "\t"i": "$i}'</pre>
<p>
<strong><em>track flights from the command line</em></strong>
  <pre class="codeline"> flight_status() { if [[ $# -eq 3 ]];then offset=$3; else offset=0; fi; curl "http://mobile.flightview.com/TrackByRoute.aspx?view=detail&al="$1"&fn= "$2"&dpdat=$(date +%Y%m%d -d ${offset}day)" 2>/dev/null |html2text |grep ":"; }</pre>
<p>
<strong><em>Backup a local drive into a file on the remote host via ssh</em></strong>
  <pre class="codeline"> dd if=/dev/sda | ssh user@server 'dd of=sda.img'</pre>
<p>
<strong><em>Kill the terminal(window/tab) you work in [suicide]</em></strong>
  <pre class="codeline"> kill -9 $$</pre>
<p>
<strong><em>backup file. (for bash)</em></strong>
  <pre class="codeline"> cp -p file-you-want-backup{,_`date +%Y%m%d`} # for bash</pre>
<p>
<strong><em>Renaming jpg extension files at bunch</em></strong>
  <pre class="codeline"> find . -name "*.jpg" | perl -ne'chomp; $name = $_; $quote = chr(39); s/[$quote\\!]/_/ ; print "mv \"$name\" \"$_\"\n"'</pre>
<p>
<strong><em>Get to the user for using system.</em></strong>
  <pre class="codeline"> ps awwux|awk '{print $1}'|sort|uniq</pre>
<p>
<strong><em>Import an entire directory into clearcase</em></strong>
  <pre class="codeline"> ct mkelem -nc `find ./ -name "*" | xargs`</pre>
<p>
<strong><em>Check out hijacked files in clearcase</em></strong>
  <pre class="codeline"> cleartool co -nc `cleartool ls -recurse | grep "hijacked" | sed s/\@\@.*// | xargs`</pre>
<p>
<strong><em>ktadd</em></strong>
  <pre class="codeline"> kadmin -p admin@NOC.NBIRN.NET -q "ktadd -k /etc/krb5.keytab host/hostname"</pre>
<p>
<strong><em>addprinc</em></strong>
  <pre class="codeline"> kadmin -p admin@NOC.NBIRN.NET -q "addprinc -randkey host/host"</pre>
<p>
<strong><em>Clearcase find branch</em></strong>
  <pre class="codeline"> ct find -avobs -nxname -element 'brtype(branch_name)' -print 2>/dev/null</pre>
<p>
<strong><em>Convert a PKCS#8 private key to PEM format</em></strong>
  <pre class="codeline"> openssl pkcs8 -inform DER -nocrypt -in [priv key] -out [pem priv key]</pre>
<p>
<strong><em>list all file-types (case-insensitive extensions) including</em></strong>
  <pre class="codeline"> find /path/to/dir -type f |sed 's/^.*\///'|grep -o '\.[^.]*$'|sort -f|uniq -i</pre>
<p>
<strong><em>creates a bash function to remove certain lines from SSH</em></strong>
  <pre class="codeline"> function sshdel { perl -i -n -e "print unless (\$. == $1)" ~/.ssh/known_hosts; }</pre>
<p>
<strong><em>Check if file is greater than 20 bytes, such as an empty gzip</em></strong>
  <pre class="codeline"> BACKUP_FILE_SIZE=`eval ls -l ${BACKUP_FILE} | awk {'print $5'}`; if [</pre>
<p>
<strong><em>Press Any Key to Continue</em></strong>
  <pre class="codeline"> read enterKey</pre>
<p>
<strong><em>oneliner to open several times same application</em></strong>
  <pre class="codeline"> for i in $(seq 5); do xpenguins & done</pre>
<p>
<strong><em>View entire process string</em></strong>
  <pre class="codeline"> /usr/ucb/ps -auxgww</pre>
<p>
<strong><em>oneliner to open several times same application</em></strong>
  <pre class="codeline"> i="0"; while [ $i -lt 5 ] ; do xpenguins & i=$[$i+1] ; done</pre>
<p>
<strong><em>wget ? server to server files transfer</em></strong>
  <pre class="codeline"> wget -H -r ?level=1 -k -p http://www.domain.com/folder/</pre>
<p>
<strong><em>copy partition table from /dev/sda to /dev/sdb</em></strong>
  <pre class="codeline"> sfdisk /dev/sdb <(sfdisk -d /dev/sda| perl -pi -e 's/sda/sdb/g')</pre>
<p>
<strong><em>Open a RemoteDesktop from terminal</em></strong>
  <pre class="codeline"> rdesktop -a 16 luigi:3052</pre>
<p>
<strong><em>Save VM running as headless</em></strong>
  <pre class="codeline"> VBoxManage controlvm ServidorProducao savestate</pre>
<p>
<strong><em>Running VirtualBox as headless</em></strong>
  <pre class="codeline"> nohup VBoxHeadless -p 3052 -startvm ServidorProducao &</pre>
<p>
<strong><em>Ruby - nslookup against a list of IP`s or FQDN`s</em></strong>
  <pre class="codeline"> ruby -e 'File.foreach("list") {|i| print `nslookup #{i}`}'</pre>
<p>
<strong><em>Killing multiplpe process for one program like apache, wget,</em></strong>
  <pre class="codeline"> ps ax| awk '/[h]ttpd/{print $1}'| xargs kill -9</pre>
<p>
<strong><em>Killing multiplpe process for one program like apache, wget,</em></strong>
  <pre class="codeline"> ps aux| grep -v grep| grep httpd| awk {'print $2'}| xargs kill -9</pre>
<p>
<strong><em>Return IP Address</em></strong>
  <pre class="codeline"> ifconfig -a|grep Bcast:|cut -d\: -f2|awk '{print $1}'</pre>
<p>
<strong><em>Return IP Address</em></strong>
  <pre class="codeline"> ifconfig | awk '/inet /{sub(/^[^:]+:/,"",$2); print $2}'</pre>
<p>
<strong><em>Return IP Address</em></strong>
  <pre class="codeline"> /usr/sbin/ifconfig -a|awk -F" " 'NR==4{print $2}'</pre>
<p>
<strong><em>search for a pattern (regex) in all text files (ignoring binary</em></strong>
  <pre class="codeline"> egrep -i "somepattern" `find . -type f -print`</pre>
<p>
<strong><em>Mail text file (/tmp/scream-dump) contents from linux box with</em></strong>
  <pre class="codeline"> mail -s scream-dump user@example.com < /tmp/scream-dump</pre>
<p>
<strong><em <a href="http://xname.<a" <a href="http://xname.<a</a>">http://xname.<a</a></a> href="cc/text/video-streaming-on-wan.pdf"><em><code>cc/text/video-streaming-on-wan.pdf</code></em></a> (encode.sh)</em></strong>
  <pre class="codeline"> ./encode.sh [ h264 | xvid | theora | mpeg4 ]</pre>
<p>
<strong><em>generate random password</em></strong>
  <pre class="codeline"> tr -dc 'a-zA-Z0-9' < /dev/urandom | fold -w 10 | sed 1q</pre>
<p>
<strong><em>getting your current ip on your e-mail</em></strong>
  <pre class="codeline"> mail -s "`curl -s whatismyip.org`" youremail@yourserver.com <</pre>
<p>
<strong><em>Sorted list of established destination connections</em></strong>
  <pre class="codeline"> netstat | grep EST | awk '{print $5}' | sort</pre>
<p>
<strong><em>Find in all files in the current directory, just a find shorthand</em></strong>
  <pre class="codeline"> find ./ -name $1 -exec grep -H -n $2 '{}' ';'</pre>
<p>
<strong><em>List all symbolic links in current directory</em></strong>
  <pre class="codeline"> ls -F | sed -n 's/@$//p'</pre>
<p>
<strong><em>Mount a Windows share on the local network (Ubuntu)</em></strong>
  <pre class="codeline"> sudo mount -t cifs //$ip_or_host/$sharename /mnt</pre>
<p>
<strong><em>Checks apache's access_log file, strips the search queries and</em></strong>
  <pre class="codeline"> cat /var/log/httpd/access_log | grep q= | awk '{print $11}' | awk -F 'q=' '{print $2}' | sed 's/+/ /g;s/%22/"/g;s/q=//' | cut -d "&" -f 1 | mail youremail@isp.com -s "[your-site] search strings for `date`"</pre>
<p>
<strong><em>remove files and directories with acces time older than a given</em></strong>
  <pre class="codeline"> find <dir> -printf '%p : %A@\n' | awk '{FS=" : " ; if($2 < <time in epoc> ) print $1 ;}' | xargs rm --verbose -fr ;</pre>
<p>
<strong><em>Remove an unnecessary suffix from a file name for all files in a</em></strong>
  <pre class="codeline"> for f in $(ls *.xml.skippy); do mv $f `echo $f | sed 's|.skippy||'`; done</pre>
<p>
<strong><em>Write and run a quick C program</em></strong>
  <pre class="codeline"> alias cstdin='echo "Ctrl-D when done." && gcc -Wall -o ~/.stdin.exe ~/.stdin.c && ~/.stdin.exe'</pre>
<p>
<strong><em>Pipe ls output into less</em></strong>
  <pre class="codeline"> function lsless() { ls "$@" | less; }</pre>
<p>
<strong><em>Find all dot files and directories</em></strong>
  <pre class="codeline"> ls -a | egrep "^\.\w"</pre>
<p>
<strong><em>List files in tarballs</em></strong>
  <pre class="codeline"> for F in $(find ./ -name "*.tgz") ; do tar -tvzf $F ; done</pre>
<p>
<strong><em>bash / vim workflow</em></strong>
  <pre class="codeline"> zsh$ M-v</pre>
<p>
<strong><em>List files in tarballs</em></strong>
  <pre class="codeline"> find <path> -name "*.tgz" -or -name "*.tar.gz" | while read file; do echo "$file: "; tar -tzf $file; done</pre>
<p>
<strong><em>Count emails in an MBOX file</em></strong>
  <pre class="codeline"> grep -c '^From ' mbox_file</pre>
<p>
<strong><em>Press Any Key to Continue</em></strong>
  <pre class="codeline"> echo -n "Press any key to continue..." && read</pre>
<p>
<strong><em>umount all nfs mounts on machine</em></strong>
  <pre class="codeline"> mount | awk '/:/ { print $3 } ' | xargs sudo umount</pre>
<p>
<strong><em>Get ethX mac addresses</em></strong>
  <pre class="codeline"> sudo ifconfig -a | grep eth | grep HW | cut -d' ' -f11</pre>
<p>
<strong><em>Get your external IP address with the best commandlinefu.com</em></strong>
  <pre class="codeline"> eval $(curl -s http://www.commandlinefu.com/commands/matching/external/ZXh0ZXJuYWw=/so rt-by-votes/plaintext|sed -n '/^# Get your external IP address$/{n;p;q}')</pre>
<p>
<strong><em>Go up multiple levels of directories quickly and easily.</em></strong>
  <pre class="codeline"> alias ..="cd .." ...="cd ../.." ....="cd ../../.."</pre>
<p>
<strong><em>Get a text on a position on the file and store in a variable with</em></strong>
  <pre class="codeline"> TIMEUNIT=$( cat a | grep -n "timescale" | awk -F ":" '{ print $1 } ' )</pre>
<p>
<strong><em>Useful if you need to see compiler errors while edit a code</em></strong>
  <pre class="codeline"> alias clear='( for ((i=1;i<$LINES;i++)) ; do echo "" ; done ) ; clear'</pre>
<p>
<strong><em>Mount an external FAT32 USB HDD</em></strong>
  <pre class="codeline"> sudo mount -t vfat /dev/sdb1 /mnt/sdb1</pre>
<p>
<strong><em>Forward connections</em></strong>
  <pre class="codeline"> ssh -g -L 8080:localhost:80 root@$HOST</pre>
<p>
<strong><em>tail, with specific pattern colored</em></strong>
  <pre class="codeline"> tail -f file | egrep --color=always $\|PATTERN</pre>
<p>
<strong><em>Compress files and delete the uncompressed ones</em></strong>
  <pre class="codeline"> for i in `ls`; do tar cvjf $i.tar.bz2 $i && rm -r $i; done</pre>
<p>
<strong><em>Outputs a 10-digit random number</em></strong>
  <pre class="codeline"> n=$RANDOM$RANDOM$RANDOM; let "n %= 10000000000"; echo $n</pre>
<p>
<strong><em>Simple Find</em></strong>
  <pre class="codeline"> find / -name FILENAME</pre>
<p>
<strong><em>print crontab entries for all the users that actually have a</em></strong>
  <pre class="codeline"> for USER in `ls /var/spool/cron`; do echo "=== crontab for $USER ==="; echo $USER; done</pre>
<p>
<strong><em>Transforms a file to all uppercase.</em></strong>
  <pre class="codeline"> perl -pi -e 's/([[:lower:]]+)/uc $1/gsex' file</pre>
<p>
<strong><em>Laminate a file</em></strong>
  <pre class="codeline"> awk '{print(substr($0,1,5))}' file</pre>
<p>
<strong><em>Get all members from one AD group and put them in another AD group</em></strong>
  <pre class="codeline"> for /F "DELIMS=""" %i in ('dsquery group -name SourceGroupName ^| dsget group -members') do dsquery group -name TargetGroupName | dsmod group -addmbr %i</pre>
<p>
<strong><em>Remove blank lines from a file and save output to new file</em></strong>
  <pre class="codeline"> sed '/^$/d' file >newfile</pre>
<p>
<strong><em>regex to match an ip</em></strong>
  <pre class="codeline"> echo "123.32.12.134" | grep -P '([01]?\d\d?|2[0-4]\d|25[0-5])\.([01]?\d\d?|2[0-4]\d|25[0-5])\.([01]?\d \d?|2[0-4]\d|25[0-5])\.([01]?\d\d?|2[0-4]\d|25[0-5])'</pre>
<p>
<strong><em>Checks your unread Gmail from the command line (or use imap etc)</em></strong>
  <pre class="codeline"> curl -u username --silent "https://mail.google.com/mail/feed/atom" | perl -ne 'print "\t" if /<name>/; print "$2\n" if /<(title|name)>(.*)<\/\1>/;</pre>
<p>
<strong><em>Search through files, ignoring .svn</em></strong>
  <pre class="codeline"> ack -ai 'searchterm'</pre>
<p>
<strong><em>show current directory</em></strong>
  <pre class="codeline"> xdg-open .</pre>
<p>
<strong><em>show current directory</em></strong>
  <pre class="codeline"> gnome-open .</pre>
<p>
<strong><em>Show directories in the PATH, one per line</em></strong>
  <pre class="codeline"> echo src::${PATH} | awk 'BEGIN{pwd=ENVIRON["PWD"];RS=":";FS="\n"}!$1{$1=pwd}$1!~/^\//{$1=pwd"/" $1}{print $1}'</pre>
<p>
<strong><em>bash function to check for something every 5 seconds</em></strong>
  <pre class="codeline"> watch -n <seconds> <command></pre>
<p>
<strong><em>Sort a character string</em></strong>
  <pre class="codeline"> echo sortmeplease | grep -o . | sort | tr -d '\n'; echo</pre>
<p>
<strong><em>free swap</em></strong>
  <pre class="codeline"> free -m | awk '/Swap/ {print $4}'</pre>
<p>
<strong><em>rkhunter (Rootkit Hunter) is a Unix-based tool that scans for</em></strong>
  <pre class="codeline"> rkhunter --check</pre>
<p>
<strong><em>Securely edit the sudo file over the network</em></strong>
  <pre class="codeline"> visudo</pre>
<p>
<strong><em>Securely look at the group file over the network</em></strong>
  <pre class="codeline"> vigr</pre>
<p>
<strong><em>Securely seeing the password file over the network</em></strong>
  <pre class="codeline"> vipw</pre>
<p>
<strong><em>dos2unix recursively</em></strong>
  <pre class="codeline"> find . -type f -exec dos2unix {} +</pre>
<p>
<strong><em>Get the information about the internet usage from the commandline.</em></strong>
  <pre class="codeline"> vnstat</pre>
<p>
<strong><em>find dis1k space</em></strong>
  <pre class="codeline"> du -s `find . -maxdepth 1 \! -name '.'` | sort -n | tail</pre>
<p>
<strong><em>Print man pages to PDF (yes, another one)</em></strong>
  <pre class="codeline"> man -t [command] | lp -d PDF -t [command].pdf</pre>
<p>
<strong><em>Find the average QTime for all queries ran within the last hour</em></strong>
  <pre class="codeline"> cat /service/solr/log/main/current | tai64nlocal | grep "\(`date '+%F %H'`\|`date '+%F %H %M' | awk '{print $1" 0"$2-1":"$3}'`\)" | grep QTime | awk '{print $NF}' | awk -F\= '{ s += $2} END {print s/NR}'</pre>
<p>
<strong><em>Echo the local IP addresses of the machines on your local network</em></strong>
  <pre class="codeline"> for i in 192.168.1.{61..71};do ping -c 1 $i &> /dev/null && echo $i;fi;done</pre>
<p>
<strong><em>YES = NO</em></strong>
  <pre class="codeline"> yes | tr 'y' 'n'</pre>
<p>
<strong><em>Update iptables firewall with a temp ruleset</em></strong>
  <pre class="codeline"> sudo iptables-restore < /etc/iptables.test.rules</pre>
<p>
<strong><em>Emulate sleep in DOS/BAT</em></strong>
  <pre class="codeline"> echo sleep() begins: %TIME% && FOR /l %a IN (10,-1,1) do (ECHO 1 >NUL %as&ping -n 2 -w 1 127.0.0.1>NUL) && echo sleep() end: %TIME%</pre>
<p>
<strong><em>Remove CR from Windows- / DOS-textfiles</em></strong>
  <pre class="codeline"> tr -d '\r' < ${input_txt} > ${output_txt}</pre>
<p>
<strong><em>Emptying a text file in one shot</em></strong>
  <pre class="codeline"> ggdG</pre>
<p>
<strong><em>CPU model</em></strong>
  <pre class="codeline"> cat /proc/cpuinfo</pre>
<p>
<strong><em>Multiple Timed Execution of subshells sleeping in the background</em></strong>
  <pre class="codeline"> S=$SSH_TTY && (sleep 3 && echo -n 'Peace... '>$S & ) && (sleep 5 && echo -n 'Love... '>$S & ) && (sleep 7 && echo 'and Intergalactic Happiness!'>$S & )</pre>
<p>
<strong><em>parses the BIOS memory and prints information about all structures</em></strong>
  <pre class="codeline"> biosdecode</pre>
<p>
<strong><em>x86info</em></strong>
  <pre class="codeline"> x86info</pre>
<p>
<strong><em>Lists the size of certain file in every 10 seconds</em></strong>
  <pre class="codeline"> while true ; do du -sk testfile ; sleep 10 ; done</pre>
<p>
<strong><em>Turn Regular quotes ("") into curly quotes (??)</em></strong>
  <pre class="codeline"> smartypants | php -r "echo mb_decode_numericentity(file_get_contents('php://stdin'),array(0x0000,0 xFFFF,0x0000,0xFFFF),'UTF-8');"</pre>
<p>
<strong><em>List path of binaries</em></strong>
  <pre class="codeline"> echo $PATH|awk -F: ' { for (i=1; i <= NF; i++) print $i }'</pre>
<p>
<strong><em>Get your external IP address</em></strong>
  <pre class="codeline"> html2text http://checkip.dyndns.org | grep -i 'Current IP Address:'|cut -d' ' -f4</pre>
<p>
<strong><em>find . -name "*.gz" | xargs -n 1 -I {} bash -c "gunzip -c {} |</em></strong>
  <pre class="codeline"> Rezip a bunch of files</pre>
<p>
<strong><em>ssh autocomplete</em></strong>
  <pre class="codeline"> complete -W "$(echo `cat .bash_history | egrep '^ssh ' | sort | uniq | sed 's/^ssh //'`;)" ssh</pre>
<p>
<strong><em>Get all IPs via ifconfig</em></strong>
  <pre class="codeline"> ifconfig | grep "inet addr" | cut -d: -f2 | cut -d' ' -f1</pre>
<p>
<strong><em>Read just the IP address of a device</em></strong>
  <pre class="codeline"> /sbin/ifconfig eth0 | grep "inet addr" | sed -e 's/.*inet addr:\(.*\) B.*/\1/g'</pre>
<p>
<strong><em>Read just the IP address of a device</em></strong>
  <pre class="codeline"> ifconfig $DEVICE | perl -lne '/inet addr:([\d.]+)/ and print $1'</pre>
<p>
<strong><em>Command for getting the list of files with perms, owners, groups</em></strong>
  <pre class="codeline"> find / | xargs ls -l | tr -s ' ' | cut -d ' ' -f 1,3,4,9</pre>
<p>
<strong><em>count how many times a string appears in a (source code) tree</em></strong>
  <pre class="codeline"> grep -rc logged_in app/ | cut -d : -f 2 | awk '{sum+=$1} END {print sum}'</pre>
<p>
<strong><em>kill some process (same as others) but parsing to a variable</em></strong>
  <pre class="codeline"> tokill=`ps -fea|grep process|awk '{ printf $2" "}'`; kill -9 $tokill;</pre>
<p>
<strong><em>Move files around local filesystem with tar without wasting space</em></strong>
  <pre class="codeline"> tar -C <source> -cf - . | tar -C <destination> -xf -</pre>
<p>
<strong><em>find a word in multiple files avoiding svn</em></strong>
  <pre class="codeline"> grep -r 'keyword keyword2' your/path/ | grep -v svn</pre>
<p>
<strong><em>Display which distro is installed</em></strong>
  <pre class="codeline"> test `uname` = Linux && lsb_release -a || ( test `uname` = SunOS && cat /etc/release || uname -rms )</pre>
<p>
<strong><em>Alternative size (human readable) of directories (biggest last)</em></strong>
  <pre class="codeline"> function duf { du -sk "$@" | sort -n | while read size fname; do for unit in k M G T P E Z Y; do if [ $size -lt 1024 ]; then echo -e "${size}${unit}\t${fname}"; break; fi; size=$((size/1024)); done; done; }</pre>
<p>
<strong><em>Use FileMerge to compare two files</em></strong>
  <pre class="codeline"> opendiff <file1> <file2></pre>
<p>
<strong><em>tar per directory</em></strong>
  <pre class="codeline"> cd <YOUR_DIRECTORY>; for i in `ls ./`; do tar czvf "$i".tar.gz "$i" ; done</pre>
<p>
<strong><em>search for groups in ldap</em></strong>
  <pre class="codeline"> ldapsearch -H ldap://localhost:389 -D cn=username,ou=users,dc=domain -x -W -b ou=groups,dc=domain '(member=cn=username,ou=users,dc=domain)' | grep ^dn | sed "s/dn\: cn=\([^,]*\),ou=\([^,]*\),.*/\2 \1/"</pre>
<p>
<strong><em>Mac, ip, and hostname change - sweet!</em></strong>
  <pre class="codeline"> ifconfig eth0 down hw ether (newmacaddresshere) && ifconfig eth0 up && ifconfig eth0 (newipaddresshere) netmask 255.255.255.0 up && /bin/hostname (newhostnamehere)</pre>
<p>
<strong><em>Uncompress a CSS file</em></strong>
  <pre class="codeline"> cat somefile.css | awk '{gsub(/{|}|;/,"&\n"); print}' >> uncompressed.css</pre>
<p>
<strong><em>print random commandlinefu.com submission</em></strong>
  <pre class="codeline"> lynx -source http://www.commandlinefu.com/commands/random | sed 's/<[^>]*>//g' | head -1037 | tail -10 | sed -e 's/^[ \t]*//' | sed '/^$/d' | head -2</pre>
<p>
<strong><em>Mount a CD-ROM on Solaris (SPARC)</em></strong>
  <pre class="codeline"> mkdir -p /cdrom/unnamed_cdrom ; mount -F hsfs -o ro `ls -al /dev/sr* |awk '{print "/dev/" $11}'` /cdrom/unnamed_cdrom</pre>
<p>
<strong><em>Kill an orphan console</em></strong>
  <pre class="codeline"> skill -KILL -t ttyS0</pre>
<p>
<strong><em>Run Remote GUI Programs Using SSH Forwarding</em></strong>
  <pre class="codeline"> ssh -C -X user@remotehost gui_command</pre>
<p>
<strong><em>Scan for viruses</em></strong>
  <pre class="codeline"> clamscan -ir --bell ~user/</pre>
<p>
<strong><em>Get Tomorrow's Date</em></strong>
  <pre class="codeline"> TOM=`perl -w -e '@tom=localtime(time+86400);printf "%d%.2d%.2d",$tom[5]+1900,$tom[4]+1,$tom[3];'`</pre>
<p>
<strong><em>Get Yesterday's Date</em></strong>
  <pre class="codeline"> YEST=`perl -w -e '@yest=localtime(time-86400);printf "%d%.2d%.2d",$yest[5]+1900,$yest[4]+1,$yest[3];'`</pre>
<p>
<strong><em>Creating a ZFS Storage Pool by Using Files</em></strong>
  <pre class="codeline"> zpool create tank /path/to/file/a /path/to/file/b</pre>
<p>
<strong><em>Creating a Mirrored Storage Pool using Zpool</em></strong>
  <pre class="codeline"> zpool create tank mirror c0t0d0 c0t1d0 mirror c0t2d0 c0t3d0</pre>
<p>
<strong><em>Creating a RAID-Z Storage Pool</em></strong>
  <pre class="codeline"> zpool create tank raidz c0t0d0 c0t1d0 c0t2d0 c0t3d0 c0t4d0 c0t5d0</pre>
<p>
<strong><em>Remove all .svn folders inside a folder</em></strong>
  <pre class="codeline"> find . -name "\.svn" -exec rm -rf {} ";"</pre>
<p>
<strong><em>Do an OR search using grep to look for more than one search term</em></strong>
  <pre class="codeline"> grep -i '<searchTerm>\|<someOtherSearchTerm>' <someFileName></pre>
<p>
<strong><em>IP address of current host</em></strong>
  <pre class="codeline"> hostname -i</pre>
<p>
<strong><em>Find the svn directory that a commit was made in. Usefull if you</em></strong>
  <pre class="codeline"> echo "12345,12346" |sed -e's/ //'|tr "," "\n"| while read line; do echo -n $line" "; svn log -vr $line https://url/to/svn/repository/|grep "/"|head -1|cut -d"/" -f2; done</pre>
<p>
<strong><em>Append output to the beginning of a file.</em></strong>
  <pre class="codeline"> command > tmp && cat logfile.txt >> tmp && tmp > logfile.txt && rm tmp</pre>
<p>
<strong><em>gzip vs bzip2 at compressing random strings?</em></strong>
  <pre class="codeline"> < /dev/urandom tr -dc A-Za-z0-9_ | head -c $((1024 * 1024)) | tee >(gzip -c > out.gz) >(bzip2 -c > out.bz) > /dev/null</pre>
<p>
<strong><em>Show live HTTP requests being made on OS X</em></strong>
  <pre class="codeline"> sudo tcpdump -i en1 -n -s 0 -w - | grep -a -o -E "Host\: .*|GET \/.*"</pre>
<p>
<strong><em>Display all installed ISO/IEC 8859 manpages</em></strong>
  <pre class="codeline"> for i in $(seq 1 11) 13 14 15 16; do man iso-8859-$i; done</pre>
<p>
<strong><em>List SMTP connections by host</em></strong>
  <pre class="codeline"> cat /var/log/secure | grep smtp | awk '{print $9}' | cut -f2 -d= | sort | uniq -c | sort -n | tail</pre>
<p>
<strong><em>Not so simple countdown from a given date</em></strong>
  <pre class="codeline"> watch -tn1 'bc<<<"`date -d'\''friday 21:00'\'' +%s`-`date +%s`"|perl -ne'\''@p=gmtime($_);printf("%dd %02d:%02d:%02d\n",@p[7,2,1,0]);'\'</pre>
<p>
<strong><em>Get own public IP address</em></strong>
  <pre class="codeline"> wget -qO- whatismyip.org</pre>
<p>
<strong><em>Add some color to ls file and folder listings</em></strong>
  <pre class="codeline"> eval "`dircolors -b`"</pre>
<p>
<strong><em>Quickly make schema changes in Django</em></strong>
  <pre class="codeline"> while true ; do scripts/bootstrap.py ; ./manage.py runserver ; done</pre>
<p>
<strong><em>Convert files with CR-terminated lines (as created by Mac OS X</em></strong>
  <pre class="codeline"> function crtonl { perl -i -ape 's/\r/\n/g;' $* ; }</pre>
<p>
<strong><em>Find files modified in the last 5 days, no more than 2 levels</em></strong>
  <pre class="codeline"> find . -type f -depth -3 -mtime -5</pre>
<p>
<strong><em>easy regex based mass renaming</em></strong>
  <pre class="codeline"> ls /some/directory | sed -rn -e 's/inpattern/mv -v & outpattern/p' | sh</pre>
<p>
<strong><em>Shows your WAN IP, when you`re sitting behind a router</em></strong>
  <pre class="codeline"> alias myip='curl -s www.wieistmeineip.de | egrep -o "[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}"'</pre>
<p>
<strong><em>Kills all processes for a certain program</em></strong>
  <pre class="codeline"> kill -9 $(pidof process)</pre>
<p>
<strong><em>startx output to log file</em></strong>
  <pre class="codeline"> startx > startx.log 2>&1</pre>
<p>
<strong><em>Copy files from one dir to another using tar.</em></strong>
  <pre class="codeline"> tar cf - . | (cd /new/dir; tar xvf -)</pre>
<p>
<strong><em>read a column of a file with table like data</em></strong>
  <pre class="codeline"> echo 1 2 3 > FILE; while read -a line; do echo ${line[2]}; done < FILE</pre>
<p>
<strong><em>Recursively deletes folders named DIR </em></strong>
  <pre class="codeline"> find . -type d -name DIR -exec rm -r {} \;</pre>
<p>
<strong><em>remove file named 1 after fat fingeriing :w! in vi</em></strong>
  <pre class="codeline"> :rm 1</pre>
<p>
<strong><em>Generic date format</em></strong>
  <pre class="codeline"> date --iso</pre>
<p>
<strong><em>Query for installed packages on RHEL boxes, and format the output</em></strong>
  <pre class="codeline"> rpm -qa --queryformat 'Installed on %{INSTALLTIME:date}\t%{NAME}-%{VERSION}-%{RELEASE}: %{SUMMARY}\n'</pre>
<p>
<strong><em>Say the current time (Mac OS X)</em></strong>
  <pre class="codeline"> date "+The time is %H:%M" | say</pre>
<p>
<strong><em>Show "Max" settings for PHP</em></strong>
  <pre class="codeline"> php -i|grep -i max</pre>
<p>
<strong><em>determine if CPU is 32-bit or 64-bit</em></strong>
  <pre class="codeline"> grep lm /proc/cpuinfo</pre>
<p>
<strong><em>More precise BASH debugging</em></strong>
  <pre class="codeline"> env PS4=' ${BASH_SOURCE}:${LINENO}(${FUNCNAME[0]}) ' sh -x /etc/profile</pre>
<p>
<strong><em>is today the end of the month?</em></strong>
  <pre class="codeline"> [ `date --date='next day' +'%B'` == `date +'%B'` ] || echo 'end of month'</pre>
<p>
<strong><em>Create a directory and change into it at the same time</em></strong>
  <pre class="codeline"> md () { mkdir -p "$@" && cd "$@"; }</pre>
<p>
<strong><em>Show Directories in the PATH Which does NOT Exist</em></strong>
  <pre class="codeline"> (IFS=:;for p in $PATH; do test -d $p || echo $p; done)</pre>
<p>
<strong><em>Testing hard disk reading speed</em></strong>
  <pre class="codeline"> hdparm -t /dev/sda</pre>
<p>
<strong><em>Identify differences between directories (possibly on different</em></strong>
  <pre class="codeline"> diff <(ssh server01 'cd config; find . -type f -exec md5sum {} \;| sort -k 2') <(ssh server02 'cd config;find . -type f -exec md5sum {} \;| sort -k 2')</pre>
<p>
<strong><em>Print text string vertically, one character per line.</em></strong>
  <pre class="codeline"> echo "vertical text" | grep -o '.'</pre>
<p>
<strong><em>Show directories in the PATH, one per line</em></strong>
  <pre class="codeline"> echo $PATH | tr \: \\n</pre>
<p>
<strong><em>display an embeded help message from bash script header</em></strong>
  <pre class="codeline"> [ "$1" == "--help" ] && { sed -n -e '/^# Usage:/,/^$/ s/^# \?//p' < $0; exit; }</pre>
<p>
<strong><em>Send data securly over the net.</em></strong>
  <pre class="codeline"> cat /etc/passwd | openssl aes-256-cbc -a -e -pass pass:password | netcat -l -p 8080</pre>
<p>
<strong><em>Replace spaces in filenames with underscorees</em></strong>
  <pre class="codeline"> rename -v 's/ /_/g' *</pre>
<p>
<strong><em>bash: hotkey to put current commandline to text-editor</em></strong>
  <pre class="codeline"> bash-hotkey: <CTRL+x+e></pre>
<p>
<strong><em>move a lot of files over ssh</em></strong>
  <pre class="codeline"> tar -cf - /home/user/test | gzip -c | ssh user@sshServer 'cd /tmp; tar xfz -'</pre>
<p>
<strong><em>print file without duplicated lines using awk</em></strong>
  <pre class="codeline"> awk '!a[$0]++' file</pre>
<p>
<strong><em>Cleanup firefox's database.</em></strong>
  <pre class="codeline"> pgrep -u `id -u` firefox-bin || find ~/.mozilla/firefox -name '*.sqlite'|(while read -e f; do echo 'vacuum;'|sqlite3 "$f" ; done)</pre>
<p>
<strong><em>vim easter egg</em></strong>
  <pre class="codeline"> $ vim ... :help 42</pre>
<p>
<strong><em>Change proccess affinity.</em></strong>
  <pre class="codeline"> taskset -cp <core> <pid></pre>
<p>
<strong><em>Download file with multiple simultaneous connections</em></strong>
  <pre class="codeline"> aria2c -s 4 http://my/url</pre>
<p>
<strong><em>A fun thing to do with ram is actually open it up and take a</em></strong>
  <pre class="codeline"> sudo strings /dev/mem</pre>
<p>
<strong><em>Pick a random line from a file</em></strong>
  <pre class="codeline"> shuf -n1 file.txt</pre>
<p>
<strong><em>Use xdg-open to avoid hard coding browser commands</em></strong>
  <pre class="codeline"> xdg-open http://gmail.com</pre>
<p>
<strong><em>Search commandlinefu from the CLI</em></strong>
  <pre class="codeline"> curl -sd q=Network http://www.commandlinefu.com/search/autocomplete |html2text -width 100</pre>
<p>
<strong><em>pipe output of a command to your clipboard</em></strong>
  <pre class="codeline"> some command|xsel --clipboard</pre>
<p>
<strong><em>Using bash inline "here document" with three less-than symbols on</em></strong>
  <pre class="codeline"> <<<"k=1024; m=k*k; g=k*m; g" bc</pre>
<p>
<strong><em>Colored SVN diff</em></strong>
  <pre class="codeline"> svn diff <file> | vim -R -</pre>
<p>
<strong><em>show open ports on computer</em></strong>
  <pre class="codeline"> netstat -an | grep -i listen</pre>
<p>
<strong><em>Checking total connections to each Ip inserver</em></strong>
  <pre class="codeline"> netstat -alpn | grep :80 | awk '{print $4}' |awk -F: '{print $(NF-1)}' |sort | uniq -c | sort -n</pre>
<p>
<strong><em>Convert images (foo.gif => foo.jpg)</em></strong>
  <pre class="codeline"> for i in **/*.gif; convert $i $i:r.jpg</pre>
<p>
<strong><em>Updating twitter with from curl</em></strong>
  <pre class="codeline"> curl -u userid:password -d status="New Twitter Message" http://twitter.com/statuses/update.xml</pre>
<p>
<strong><em>Inserting a decimal every third digit</em></strong>
  <pre class="codeline"> perl -lpe'1 while s/^([-+]?\d+)(\d{3})/$1.$2/'</pre>
<p>
<strong><em>Forget fortunes in your terminal this grabs a random</em></strong>
  <pre class="codeline"> wget -qO - snubster.com|sed -n '65p'|awk 'gsub(/<span><br>.*/,"")&&1'|perl -p -e 's:myScroller1.addItem\("<span class=atHeaderOrange>::g;s:</span> <span class=snubFontSmall>::g;s:":":g;s:^:\n:g;s:$:\n:'</pre>
<p>
<strong><em>Double Compile system and world on gentoo</em></strong>
  <pre class="codeline"> emerge -e system && emerge -e system && emerge -e world && emerge -e world</pre>
<p>
<strong><em>Finding hostname and the IP Address of your machine</em></strong>
  <pre class="codeline"> host `hostname`</pre>
<p>
<strong><em>Force log creation when running an msi install</em></strong>
  <pre class="codeline"> msiexec.exe /i product.msi /l* c:\folder\LogFileName.txt</pre>
<p>
<strong><em>Execute a command if a file exists</em></strong>
  <pre class="codeline"> grep -sq "" /etc/lsb-release && lsb_release -rd</pre>
<p>
<strong><em>strip ^M character from files in VI</em></strong>
  <pre class="codeline"> :%s/<control-VM>//g</pre>
<p>
<strong><em>Cleanly quit KDE4 apps</em></strong>
  <pre class="codeline"> kquitapp plasma</pre>
<p>
<strong><em>Remove files/directory with spaces.</em></strong>
  <pre class="codeline"> find -name "*.exe" -printf "\"%p\"\n" | xargs /bin/rm</pre>
<p>
<strong><em>Calculates fake folder checksum based on folder's files' md5sums</em></strong>
  <pre class="codeline"> find path/to/folder/ -type f -print0 | xargs -0 -n 1 md5sum | awk '{print $1}' | sort | md5sum | awk '{print $1}'</pre>
<p>
<strong><em>use perl instead of sed</em></strong>
  <pre class="codeline"> echo "sed -e"|perl -pe 's/sed -e/perl -pe/'</pre>
<p>
<strong><em>Contextual Menu Cleanup (OSX)</em></strong>
  <pre class="codeline"> /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks /LaunchServices.framework/Versions/A/Support/lsregister -kill -r -domain local -domain system -domain user</pre>
<p>
<strong><em>Outputs current folder svn revision</em></strong>
  <pre class="codeline"> LC_ALL=C svn info | grep Revision | awk '{print $2}'</pre>
<p>
<strong><em>Current directory files and subdirectories ordered by size</em></strong>
  <pre class="codeline"> du -ks * | sort -n</pre>
<p>
<strong><em>Print the current battery status</em></strong>
  <pre class="codeline"> acpi | cut -d '%' -f1 | cut -d ',' -f2</pre>
<p>
<strong><em>Mount a windows partition in a dual boot linux</em></strong>
  <pre class="codeline"> mount -o auto -t ntfs /dev/hda1 /windows</pre>
<p>
<strong><em>Unmount locked filesystems.</em></strong>
  <pre class="codeline"> umount -l /media/foo</pre>
<p>
<strong><em>Extract .daa files with PowerISO</em></strong>
  <pre class="codeline"> ./poweriso extract $USER/file.daa / -od $USER/file_extracted</pre>
<p>
<strong><em>Takes a multi line df or bdf and turns it into just one line</em></strong>
  <pre class="codeline"> bdf | awk '(NF<5){f=$1; next} (NF>5){f=$1} {print f, $2, $3, $NF}'</pre>
<p>
<strong><em>Launch an Explorer window with a file selected</em></strong>
  <pre class="codeline"> explorer /select,[file]</pre>
<p>
<strong><em>Broadcast message to all logged in terminal users.</em></strong>
  <pre class="codeline"> cat welcome | wall</pre>
<p>
<strong><em>To retrieve a normal prompt</em></strong>
  <pre class="codeline"> PS1='$PWD$ '</pre>
<p>
<strong><em>kill all processes of a program</em></strong>
  <pre class="codeline"> kill -9 $(pidof *program*)</pre>
<p>
<strong><em>Open the Windows Explorer from the current directory</em></strong>
  <pre class="codeline"> explorer /e,.</pre>
<p>
<strong><em>Save the network interface info into a text file, so that you can</em></strong>
  <pre class="codeline"> netsh interface ip dump > current-interfaces.txt</pre>
<p>
<strong><em>how to like to know if a host is ON</em></strong>
  <pre class="codeline"> for ip in $(seq 1 25); do ping -c 1 192.168.0.$ip>/dev/null; [ $? -eq 0 ] && echo "192.168.0.$ip UP" || : ; done</pre>
<p>
<strong><em>edit files in current and subdir, remove all lines that containing</em></strong>
  <pre class="codeline"> grep -r "sampleString" . |uniq | cut -d: -f1 | xargs sed -i "/sampleString/d"</pre>
<p>
<strong><em>get line#1000 from text.</em></strong>
  <pre class="codeline"> head -1000 < lines.txt | tail -1</pre>
<p>
<strong><em>watch filesizes (c.f. logfiles, file downloading, etc.)</em></strong>
  <pre class="codeline"> while [ 1 ]; do date; ls -l /path/to/dir; sleep 1; done</pre>
<p>
<strong><em>Fast searh Ubntu software repo</em></strong>
  <pre class="codeline"> alias acs='apt-cache search'</pre>
<p>
<strong><em>color grep with specification of colors with GREP_COLOR env</em></strong>
  <pre class="codeline"> setenv GREP_COLOR '1;37;41'</pre>
<p>
<strong><em>reset the bizzarre gone junk terminal to normal</em></strong>
  <pre class="codeline"> echo "Xc" | tr "Xo" "\033\017</pre>
<p>
<strong><em>example usage of sar</em></strong>
  <pre class="codeline"> sar -g 5 5</pre>
<p>
<strong><em>umount all nfs mounts on machine</em></strong>
  <pre class="codeline"> mount | grep : | tr -s ' ' -d 3 | xargs umount -v</pre>
<p>
<strong><em>Display a list of upgradeable packages (apt)</em></strong>
  <pre class="codeline"> apt-show-versions -u</pre>
<p>
<strong><em>GUID generator</em></strong>
  <pre class="codeline"> guid(){ lynx -nonumbers -dump http://www.famkruithof.net/uuid/uuidgen | grep "\w\{8\}-" | tr -d ' '; }</pre>
<p>
<strong><em>search for the content in a directory</em></strong>
  <pre class="codeline"> find . -exec grep "test" '{}' /dev/null \; -print</pre>
<p>
<strong><em>Get the size of every directories and files in a path recursively</em></strong>
  <pre class="codeline"> for i in $(ls /the/path); do du -hs /the/path/$i; done</pre>
<p>
<strong><em>shows the space of a folder in bytes ever two seconds.</em></strong>
  <pre class="codeline"> watch "df | grep /this/folder/"</pre>
<p>
<strong><em>Ring the system bell after finishing a long script/compile</em></strong>
  <pre class="codeline"> myLongScript && echo -e '\a' || (echo -e '\a'; sleep 1; echo -e '\a')</pre>
<p>
<strong><em>Print only the even lines of a file</em></strong>
  <pre class="codeline"> awk '{if (NR % 2 == 0) print $0}' file.txt</pre>
<p>
<strong><em>kill all processes with name or argument</em></strong>
  <pre class="codeline"> pkill -f foo</pre>
<p>
<strong><em>find and kill a pid for APP</em></strong>
  <pre class="codeline"> ps -ef | grep APP | awk '/grep/!{print$2}' | xargs -i kill {}</pre>
<p>
<strong><em>Search for a running process through grep</em></strong>
  <pre class="codeline"> ps -e | grep SearchStringHere</pre>
<p>
<strong><em>Print a row of 50 hyphens</em></strong>
  <pre class="codeline"> ALT-50 -</pre>
<p>
<strong><em>generate random password</em></strong>
  <pre class="codeline"> openssl rand -base64 1000 | tr "[:upper:]" "[:lower:]" | tr -cd "[:alnum:]" | tr -d "lo" | cut -c 1-8 | pbcopy</pre>
<p>
<strong><em>oneliner to open several times same application</em></strong>
  <pre class="codeline"> for ((i=0;i<5;i++)) ; do xpenguins & done</pre>
<p>
<strong><em>Set Permission to user and group</em></strong>
  <pre class="codeline"> chown -R webuser:webgroup /var/www/vhosts/domain.com/httpdocs</pre>
<p>
<strong><em>Get your public ip</em></strong>
  <pre class="codeline"> wget -qO - http://cfaj.freeshell.org/ipaddr.cgi</pre>
<p>
<strong><em>Show the meta information on a package (dependency , statuts ..)</em></strong>
  <pre class="codeline"> aptitude show packages_name</pre>
<p>
<strong><em>Return IP Address</em></strong>
  <pre class="codeline"> awk '/inet end/ {print $3}' <(ifconfig eth0)</pre>
<p>
<strong><em>Show local IP</em></strong>
  <pre class="codeline"> ifconfig eth0 | grep "inet:" | cut -d ":" -f2 | cut -d " " -f1</pre>
<p>
<strong><em>Fill up disk space (for testing)</em></strong>
  <pre class="codeline"> dd if=/dev/zero of=/fs/to/fill/dummy00 bs=8192 count=$(df --block-size=8192 / | awk 'NR!=1 {print $4-100}')</pre>
<p>
<strong><em>List files that DO NOT match a pattern</em></strong>
  <pre class="codeline"> ls *[^p][^a][^t]* ; # or shopt -s extglob; ls !(*pattern*)</pre>
<p>
<strong><em>List all symbolic links in current directory</em></strong>
  <pre class="codeline"> ls -l `ls -l |awk '/^l/ {print $8}'`</pre>
<p>
<strong><em>Random password generating function</em></strong>
  <pre class="codeline"> mkpasswd() { head -c $(($1)) /dev/urandom | uuencode - | sed -n 's/.//;2s/\(.\{'$1'\}\).*/\1/p' ;}</pre>
<p>
<strong><em>Compress and Backup a disk image</em></strong>
  <pre class="codeline"> dd if=/dev/<device location> | gzip -c /<path to backup location>/<disk image name>.img.gz</pre>
<p>
<strong><em>bash / vim workflow</em></strong>
  <pre class="codeline"> vim -</pre>
<p>
<strong><em>Delete everything on hda</em></strong>
  <pre class="codeline"> dd if=/dev/zero of=/dev/hda bs=16M</pre>
<p>
<strong><em>Remove lines that contain a specific pattern($1) from file($2).</em></strong>
  <pre class="codeline"> function rmatch { echo "$(grep -v "$1" "$2")">"$2";}</pre>
<p>
<strong><em>Get your external IP address</em></strong>
  <pre class="codeline"> wget -q -O - checkip.dyndns.org|sed -e 's/.*Current IP Address: //' -e 's/<.*$//'</pre>
<p>
<strong><em>Get your external IP address</em></strong>
  <pre class="codeline"> curl whatismyip.org</pre>
<p>
<strong><em>Get your external IP address</em></strong>
  <pre class="codeline"> echo -e "GET /automation/n09230945.asp HTTP/1.0\r\nHost: whatismyip.com\r\n" | nc whatismyip.com 80 | tail -n1</pre>
<p>
<strong><em>Clear your history saved into .bash_history file!</em></strong>
  <pre class="codeline"> echo "" > .bash_history</pre>
<p>
<strong><em>remove the last of all html files in a directory</em></strong>
  <pre class="codeline"> a=($(ls *html)) && a=${a[$(expr ${#a[@]} - 1)]} && rm $a</pre>
<p>
<strong><em>Checking DNS zone on Name servers directly</em></strong>
  <pre class="codeline"> host <domain> <nameserver name></pre>
<p>
<strong><em>convert .rpm to .deb using alien</em></strong>
  <pre class="codeline"> sudo alien --to-deb Your_PackAge.rpm</pre>
<p>
<strong><em>Create Bootable USB from ISO file</em></strong>
  <pre class="codeline"> xcopy D:\*.* /s/e/f E:\</pre>
<p>
<strong><em>floating point operations in shell scripts</em></strong>
  <pre class="codeline"> echo $((3.0/5.0))</pre>
<p>
<strong><em>Ultimate current directory usage command</em></strong>
  <pre class="codeline"> O=$IFS;IFS=$'\n'; D=$(for f in *;do [[ -d $f ]] && du -sh "$f";done |</pre>
<p>
<strong><em>Ultimate current directory usage command</em></strong>
  <pre class="codeline"> ls -shF --color</pre>
<p>
<strong><em>Seach google from the command line in Unofficial google shell</em></strong>
  <pre class="codeline"> http://goosh.org</pre>
<p>
<strong><em>This command will tell the last login and reboot related</em></strong>
  <pre class="codeline"> last</pre>
<p>
<strong><em>How to run a specific command in remote server by ssh</em></strong>
  <pre class="codeline"> ssh user@remotehost [anycommand](i.e uptime,w)</pre>
<p>
<strong><em>securely locate file and dir</em></strong>
  <pre class="codeline"> slocate filename/dirname</pre>
<p>
<strong><em>get debian version number</em></strong>
  <pre class="codeline"> lsb_release -a</pre>
<p>
<strong><em>System load information alongside process information in a similar</em></strong>
  <pre class="codeline"> atop</pre>
<p>
<strong><em>detect the Super I/O chip on your computer, tell you at which</em></strong>
  <pre class="codeline"> superiotool</pre>
<p>
<strong><em>get basic information out of your computer</em></strong>
  <pre class="codeline"> lspci</pre>
<p>
<strong><em>Output all Files in Directory w/ Details to Filelist</em></strong>
  <pre class="codeline"> ls -laR > /path/to/filelist</pre>
<p>
<strong><em>Testing php configuration</em></strong>
  <pre class="codeline"> echo "<?php phpinfo(); ?>" >> /srv/www/htdocs/test.php</pre>
<p>
<strong><em>Install NMAP 5.0 ,Short and sweet command to do it</em></strong>
  <pre class="codeline"> sudo wget -c "http://nmap.org/dist/nmap-5.00.tar.bz2" && bzip2 -cd nmap-5.00.tar.bz2 | tar xvf - && cd nmap-5.00 && ./configure && make && sudo make install</pre>
<p>
<strong><em>Pick a random line from a file</em></strong>
  <pre class="codeline"> shuf file.txt | head -n 1</pre>
<p>
<strong><em>Pick a random line from a file</em></strong>
  <pre class="codeline"> head -$(($RANDOM % $(wc -l < file.txt) +1 )) file.txt | tail -1</pre>
<p>
<strong><em>Read just the IP address of a device</em></strong>
  <pre class="codeline"> ip addr|grep "inet "</pre>
<p>
<strong><em>create tar.gz archive</em></strong>
  <pre class="codeline"> tar -pczf archive_name.tar.gz /path/to/dir/or/file</pre>
<p>
<strong><em>let w3m usecookie</em></strong>
  <pre class="codeline"> alias w3m='w3m -cookie'</pre>
<p>
<strong><em>Move files around local filesystem with tar without wasting space</em></strong>
  <pre class="codeline"> tar -C <source_dir> -cf . | tar -C <dest_dir> -xf -</pre>
<p>
<strong><em>Outputs size of /example/folder in human readable format.</em></strong>
  <pre class="codeline"> du -hs /example/folder/</pre>
<p>
<strong><em>Remove all files previously extracted from a tar(.gz) file.</em></strong>
  <pre class="codeline"> for i in $(tar -tf <file.tar.gz>); do rm $i; done;</pre>
<p>
<strong><em>Quickly assess quality of project by greping the SVN commit logs</em></strong>
  <pre class="codeline"> svn log | grep "bodge\|fudge\|hack\|dirty"</pre>
<p>
<strong><em>Search and replace in multiple files and save them with the same</em></strong>
  <pre class="codeline"> for files in $(ls -A directory_name); do sed 's/search/replaced/g' $files > $files.new && mv $files.new $files; done;</pre>
<p>
<strong><em>remove unneeded configuration files in debian</em></strong>
  <pre class="codeline"> dpkg-query -l| grep -v "ii " | grep "rc " | awk '{print $2" "}' | tr -d "\n" | xargs aptitude purge -y</pre>
<p>
<strong><em>Remove packages by pattern on debian and based systems</em></strong>
  <pre class="codeline"> sudo apt-get remove --purge `dpkg -l | awk '{print $2}' | grep gnome` && apt-get autoremove</pre>
<p>
<strong><em>Check cobbler environment</em></strong>
  <pre class="codeline"> cobbler check</pre>
<p>
<strong><em>How many Non-free software is on your machine ?</em></strong>
  <pre class="codeline"> vrms</pre>
<p>
<strong><em>Randomize the order of lines in a text file.</em></strong>
  <pre class="codeline"> awk 'BEGIN {srand()} {print int(rand()*1000000) "\t" $0}' FILE | sort -n | cut -f 2-</pre>
<p>
<strong><em>Find 'foo' in located files</em></strong>
  <pre class="codeline"> locate searchstring | xargs grep foo</pre>
<p>
<strong><em>Join lines</em></strong>
  <pre class="codeline"> echo -e "aa\nbb\ncc\ndd\n123" | sed -e :a -e "/$/N; s/\n/;/; ta"</pre>
<p>
<strong><em>remove exact phrase from multiple files</em></strong>
  <pre class="codeline"> grep -r "mystring" . |uniq | cut -d: -f1 | xargs sed -i "s/mystring//"</pre>
<p>
<strong><em>Quickly backup your current directory</em></strong>
  <pre class="codeline"> alias backup_dir='mkdir -p .backup && cp * .backup'</pre>
<p>
<strong><em>Print out "string" between "match1" and "match2"</em></strong>
  <pre class="codeline"> echo "string" | sed -e 's/.*match1//' -e 's/match2.*$//'</pre>
<p>
<strong><em>Look for a string in one of your codes, excluding the files with</em></strong>
  <pre class="codeline"> find . -type f -exec grep StringToFind \{\} --with-filename \;|sed -e '/svn/d'|sed -e '/~/d'</pre>
<p>
<strong><em>Copy files over network using compression</em></strong>
  <pre class="codeline"> on the listening side: sudo nc -lp 2022 | sudo tar -xvf - and on the sending side: tar -cvzf - ./*| nc -w 3 name_of_listening_host 2022</pre>
<p>
<strong><em>bind a web server in $PWD</em></strong>
  <pre class="codeline"> python -c "import SimpleHTTPServer;SimpleHTTPServer.test()"</pre>
<p>
<strong><em>recursively delete .svn folders from a directory</em></strong>
  <pre class="codeline"> rm -rf `find . -type d -name .svn`</pre>
<p>
<strong><em>Binary editor</em></strong>
  <pre class="codeline"> bvi [binary-file]</pre>
<p>
<strong><em>find a class or file within a number of jar files</em></strong>
  <pre class="codeline"> for i in `find . | grep jar$`; do echo $i; jar tvf $i | grep 'search-string'; done;</pre>
<p>
<strong><em>Recursive file content search</em></strong>
  <pre class="codeline"> find . -name *.php | xargs grep -i -n 'TERM'</pre>
<p>
<strong><em>extract a certain number of lines from a file and dump them to</em></strong>
  <pre class="codeline"> grep '' -m X file1 > file2</pre>
<p>
<strong><em>Read a gzipped text file directly with less.</em></strong>
  <pre class="codeline"> less textfile.gz</pre>
<p>
<strong><em>Removes Apple "garbage"</em></strong>
  <pre class="codeline"> find . -name *DS_Store -exec echo rm {} \;</pre>
<p>
<strong><em>rcs - local backup of any text configuration file before</em></strong>
  <pre class="codeline"> ci -l /etc/rc.conf</pre>
<p>
<strong><em>Generate MD5 hash for a string</em></strong>
  <pre class="codeline"> echo -n "string" | md5sum -</pre>
<p>
<strong><em>Listen to the OS X system's voices</em></strong>
  <pre class="codeline"> for person in Alex Bruce Fred Kathy Vicki Victoria ; do say -v $person "Hello, my name is $person"; sleep 1; done</pre>
<p>
<strong><em>Move a file up a directory.</em></strong>
  <pre class="codeline"> mv file_name.extension ..</pre>
<p>
<strong><em>Basic Local Port Scan</em></strong>
  <pre class="codeline"> [command too long]</pre>
<p>
<strong><em>Read Nth column (e.g. 2nd column) of a row of data in a file that</em></strong>
  <pre class="codeline"> grep 'HOME.*' data.txt | awk '{print $2}' | awk '{FS="/"}{print $NF}' OR USE ALTERNATE WAY awk '/HOME/ {print $2}' data.txt | awk -F'/' '{print $NF}'</pre>
<p>
<strong><em>Get top 10 largest directories under cwd</em></strong>
  <pre class="codeline"> du | sort -n | tail -11 | head</pre>
<p>
<strong><em>Download a complete podcast</em></strong>
  <pre class="codeline"> wget -c -v -S -T 100 --tries=0 `curl -s http://ms1.espectador.com/ podcast/espectador/la_venganza_sera_terrible.xml | grep -v xml | grep link | sed 's/]*>//g'`</pre>
<p>
<strong><em>Edit a file using vi or vim in read-only mode</em></strong>
  <pre class="codeline"> vi -R filename</pre>
<p>
<strong><em>Creates a minimalist xorg.conf</em></strong>
  <pre class="codeline"> dpkg-reconfigure -phigh xserver-xorg</pre>
<p>
<strong><em>Delete tens of thousans of files at one go</em></strong>
  <pre class="codeline"> rm -rf `ls | head -5000`</pre>
<p>
<strong><em>stores the number of lines of "file" in a variable to use in a</em></strong>
  <pre class="codeline"> count=`wc -l file | cut -d ' ' -f1`</pre>
<p>
<strong><em>Run the previous command with sudo</em></strong>
  <pre class="codeline"> sudo !!</pre>
<p>
<strong><em>uppercase</em></strong>
  <pre class="codeline"> tr [a-z] [A-Z]</pre>
<p>
<strong><em>sets volume via command line</em></strong>
  <pre class="codeline"> amixer -c 0 set PCM 2dB+</pre>
<p>
<strong><em>ctrl -k and ctrl -y for cut and paste</em></strong>
  <pre class="codeline"> ctrl k , ctrl y</pre>
<p>
<strong><em>xxcopy everything from one Windows box to another</em></strong>
  <pre class="codeline"> xxcopy x:\folder1 y:\folder2 /s /h /tca /tcc /tcw /yy</pre>
<p>
<strong><em>Quick scrape of recent mobile home dir file sync for Mac Admins -</em></strong>
  <pre class="codeline"> tail -n 20 ~/Library/Logs/FileSyncAgent.log</pre>
<p>
<strong><em>Generate load on your CPU</em></strong>
  <pre class="codeline"> while true; do /bin/true; done</pre>
<p>
<strong><em>LSD: List directory files in current directory</em></strong>
  <pre class="codeline"> ls -l !* | /usr/bin/grep '^d'</pre>
<p>
<strong><em>rm all files you grep</em></strong>
  <pre class="codeline"> find . | grep deleteme | while read line; do rm $line; done</pre>
<p>
<strong><em>prints long line</em></strong>
  <pre class="codeline"> sed -n '/^.\{10\}/p'</pre>
<p>
<strong><em>View your machine firewall settings</em></strong>
  <pre class="codeline"> iptables -L -n -v</pre>
<p>
<strong><em>Directly change directory without having to specify drive letter</em></strong>
  <pre class="codeline"> cd /d d:\Windows</pre>
<p>
<strong><em>backup the old files</em></strong>
  <pre class="codeline"> tar -zcps <dir> -X <(find <dir> -type f -mtime -<days>) |ssh user@backuphost tar -xzpsC /data/bkup</pre>
<p>
<strong><em>Run ADSL connection</em></strong>
  <pre class="codeline"> pon dsl-provider</pre>
<p>
<strong><em>Stage added, updated, and DELETED files for commit</em></strong>
  <pre class="codeline"> git add -u</pre>
<p>
<strong><em>Display directory hierarchy listing as a tree</em></strong>
  <pre class="codeline"> ls -R | grep : | sed -e '\''s/:$//'\'' -e '\''s/[^-][^\/]*\//--/g'\'' -e '\''s/^/ /'\'' -e '\''s/-/|/'\''</pre>
<p>
<strong><em>set fan speed (ATI cards)</em></strong>
  <pre class="codeline"> aticonfig --pplib-cmd "set fanspeed 0 <number>"</pre>
<p>
<strong><em>Find files containing "text"</em></strong>
  <pre class="codeline"> grep -lir "text to find" *</pre>
<p>
<strong><em>Execute AccuRev pop command to retrieve missing files from a</em></strong>
  <pre class="codeline"> accurev stat -M -fl | awk '{print "\"" $0 "\""}' | xargs accurev pop</pre>
<p>
<strong><em>Place a filename at the beginning of the line to make it easier</em></strong>
  <pre class="codeline"> Place a filename at the beginning of the line to make it easier to edit the search at the end of the command.</pre>
<p>
<strong><em>UNIX one-liner to kill a hanging Firefox process</em></strong>
  <pre class="codeline"> kill -HUP ` ps -aef | grep -i firefox | sort -k 2 -r | sed 1d | awk ' { print $2 } ' `</pre>
<p>
<strong><em>Backup the first 1MB of your volume</em></strong>
  <pre class="codeline"> dd if=/dev/sdX of=/root/sdX.bin bs=1M count=1</pre>
<p>
<strong><em>Recursive chmod all files and directories within the current</em></strong>
  <pre class="codeline"> chmod -R 777 *</pre>
<p>
<strong><em>List the size (in human readable form) of all sub folders from the</em></strong>
  <pre class="codeline"> ls | xargs du -sh</pre>
<p>
<strong><em>copying files using sed to rename based on regular expression</em></strong>
  <pre class="codeline"> ls "*.mp3" | awk '{print("cp "$1" /folder/"$1)}' | sed -e 's/foo/bar/' | /bin/bash</pre>
<p>
<strong><em>Get the 10 biggest files/folders for the current direcotry</em></strong>
  <pre class="codeline"> ls -1rSA | tail</pre>
<p>
<strong><em>Add a directory with a name to /etc/passwd to get quick access</em></strong>
  <pre class="codeline"> function qcd { useradd -No -s /bin/false -u 999 -g 999 -lf -1 -d $1 $2 ; }</pre>
<p>
<strong><em>Monitor Applications application that are connected/new</em></strong>
  <pre class="codeline"> while true; do netstat -p |grep "tcp"|grep --color=always "/[a-z]*";sleep 1;done</pre>
<p>
<strong><em>Duplicate a directory tree using tar and pipes</em></strong>
  <pre class="codeline"> (cd /source/dir ; tar cvf - .)|(cd /dest/dir ; tar xvpf -)</pre>
<p>
<strong><em>unzip all .zip files in /example/directory</em></strong>
  <pre class="codeline"> cd /example/directory && unzip \*.zip</pre>
<p>
<strong><em>Search through files, ignoring .svn (less accurate/slower but</em></strong>
  <pre class="codeline"> find . ! -path \*.svn\*</pre>
<p>
<strong><em>Quick findstring recursively in dirs (Alias from long find with</em></strong>
  <pre class="codeline"> alias findstring="find . -type f -print | xargs grep $1"</pre>
<p>
<strong><em>Kill a daemon by name, not by PID</em></strong>
  <pre class="codeline"> kill_daemon() { echo "Daemon?"; read dm; kill -15 $(netstat -atulpe | grep $dm | cut -d '/' -f1 | awk '{print $9}') }; alias kd='kill_daemon</pre>
<p>
<strong><em>Delete all files more t han 7 days old</em></strong>
  <pre class="codeline"> rm -rf `find -maxdepth 1 -mindepth 1 -mtime +7`</pre>
<p>
<strong><em>Skipping tests in Maven</em></strong>
  <pre class="codeline"> mvn -Dmaven.test.skip=true install</pre>
<p>
<strong><em>Creating a Maven project</em></strong>
  <pre class="codeline"> mvn archetype:create -DgroupId=my.work -DartifactId=MyProject</pre>
<p>
<strong><em>Rename files that have number, space and hyphen</em></strong>
  <pre class="codeline"> for f in * ; do mv -- "$f" "${f/[0-9][0-9] \- /}" ; done</pre>
<p>
<strong><em>zip all files in a directory, one file per zip</em></strong>
  <pre class="codeline"> for i in $( find . ); do echo zipping file: $i zip $i.zip $i done</pre>
<p>
<strong><em>Chmod directories to add executable & read permission to the group</em></strong>
  <pre class="codeline"> sudo chmod -R g=u-w,g+X *</pre>
<p>
<strong><em>Print the ten largest files</em></strong>
  <pre class="codeline"> ls -Sl * | head</pre>
<p>
<strong><em>[re]verify a disc with very friendly output</em></strong>
  <pre class="codeline"> dd if=/dev/cdrom | pv -s 700m | md5sum | tee test.md5</pre>
<p>
<strong><em>Traceroute w/TCP to get through firewalls.</em></strong>
  <pre class="codeline"> tcptraceroute www.google.com</pre>
<p>
<strong><em>List only hidden files</em></strong>
  <pre class="codeline"> ls -ad .*</pre>
<p>
<strong><em>List all executable files in the current directory</em></strong>
  <pre class="codeline"> ls -F | grep '\''\*'\'' | sed '\''s/\*$//'\</pre>
<p>
<strong><em>Copy directories and files just like</em></strong>
  <pre class="codeline"> xcopy /e/h/y /z/i /k /f src dest</pre>
<p>
<strong><em>get size of dir or file</em></strong>
  <pre class="codeline"> du -h</pre>
<p>
<strong><em>View new log messages in real time</em></strong>
  <pre class="codeline"> tail -f /var/log/messages</pre>
<p>
<strong><em>test connection if ICMP is disabled</em></strong>
  <pre class="codeline"> telnet <ip> <port></pre>
<p>
<strong><em>Always run apt-get as root</em></strong>
  <pre class="codeline"> alias apt-get='sudo apt-get'</pre>
<p>
<strong><em>Open a list of files in VIM using separate terminal windows</em></strong>
  <pre class="codeline"> find . -name "*.java" -exec gnome-terminal \-x vim {} \;</pre>
<p>
<strong><em>find all processes named hunger and force kill, minus the grep</em></strong>
  <pre class="codeline"> ps -auwx|egrep hunger|grep -v grep| awk '{print "kill -9",$1}' > ~/fu.bar</pre>
<p>
<strong><em>Search gzipped files</em></strong>
  <pre class="codeline"> zcat /usr/share/man/man1/grep.1.gz | grep "color"</pre>
<p>
<strong><em>Recursive chmod all files and directories within the current</em></strong>
  <pre class="codeline"> find . -print -exec chmod 777 {} \;</pre>
<p>
<strong><em>Recursive chmod all files and directories within the current</em></strong>
  <pre class="codeline"> find | xargs chmod 777</pre>
<p>
<strong><em>Check reverse DNS</em></strong>
  <pre class="codeline"> nslookup {ip}</pre>
<p>
<strong><em>List the size (in human readable form) of all sub folders from</em></strong>
  <pre class="codeline"> find . -maxdepth 1 -type d -not -name . -exec du -sh {} +</pre>
<p>
<strong><em>SELinux Status</em></strong>
  <pre class="codeline"> getenforce</pre>
<p>
<strong><em>Sneaky logout</em></strong>
  <pre class="codeline"> rm ~/.bash_history; ln -s /dev/null ~/.bash_history</pre>
<p>
<strong><em>get kernel version</em></strong>
  <pre class="codeline"> uname -a</pre>
<p>
<strong><em>"Reset" directories permissions</em></strong>
  <pre class="codeline"> find . -type d -exec chmod 0755 {} \;</pre>
<p>
<strong><em>Wordpress - download latest, extract, and rename config file.</em></strong>
  <pre class="codeline"> alias wordpress='mkdir wordpress && cd wordpress && wget http://wordpress.org/latest.tar.gz && tar -xvzf latest.tar.gz && mv wordpress/* . && rm -rf latest.tar.gz wordpress && cp wp-config-sample.php wp-config.php'</pre>
<p>
<strong><em>"Reset" file permissions</em></strong>
  <pre class="codeline"> find . -type f -exec chmod 0644 {} \;</pre>
<p>
<strong><em>Equivalent of alias in cmd.exe: doskey (macros)</em></strong>
  <pre class="codeline"> doskey l=dir /OD $*</pre>
<p>
<strong><em>Altera texto dentro dos arquivos retornados pelo comando 'find'</em></strong>
  <pre class="codeline"> find ./wp-content/themes/rotce2009/ -name '*.php' -type f | xargs sed -i 's/<? /<?php /g'</pre>
<p>
<strong><em>Telnet to Free Internet Chess Server</em></strong>
  <pre class="codeline"> telnet fics.freechess.org 5000</pre>
<p>
<strong><em>List files contained within a zip archive</em></strong>
  <pre class="codeline"> unzip -l <filename></pre>
<p>
<strong><em>List all file and directory on user's home with details</em></strong>
  <pre class="codeline"> ls /home/user | xargs ls -lhR | less</pre>
<p>
<strong><em>Make an iso file out of your entire hard drive</em></strong>
  <pre class="codeline"> cat /dev/hda > ~/hda.iso</pre>
<p>
<strong><em>Monitor a file's size</em></strong>
  <pre class="codeline"> watch -n60 du /var/log/messages</pre>
<p>
<strong><em>FAST and NICE Search and Replace for Strings in Files</em></strong>
  <pre class="codeline"> nice -n19 sh -c 'S=askapache && R=htaccess; find . -type f|xargs -P5 -iFF grep -l -m1 "$S" FF|xargs -P5 -iFF sed -i -e s%${S}%${R}% FF'</pre>
<p>
<strong><em>Find the process you are looking for minus the grepped one</em></strong>
  <pre class="codeline"> pgrep command_name</pre>
<p>
<strong><em>Stripping ^M at end of each line for files</em></strong>
  <pre class="codeline"> dos2unix <filenames></pre>
<p>
<strong><em>IFS - use entire lines in your for cycles</em></strong>
  <pre class="codeline"> export IFS=$(echo -e "\n")</pre>
<p>
<strong><em>Attempt an XSS exploit on commandlinefu.com</em></strong>
  <pre class="codeline"> perl -pi -e 's/<a href="#" onmouseover="console.log('xss! '+document.cookie)" style="position:absolute;height:0;width:0;background:transparent;font-w eight:normal;">xss</a>/<\/a>/g'</pre>
<p>
<strong><em>exit if another instance is running</em></strong>
  <pre class="codeline"> pidof -x -o $$ ${0##*/} && exit</pre>
<p>
<strong><em>dstat - a mix of vmstat, iostat, netstat, ps, sar...</em></strong>
  <pre class="codeline"> dstat -ta</pre>
<p>
<strong><em>launch a virtualbox virtual machine</em></strong>
  <pre class="codeline"> VBoxManage startvm "name"</pre>
<p>
<strong><em>Create a large test file (taking no space).</em></strong>
  <pre class="codeline"> dd bs=1 seek=2TB if=/dev/null of=ext3.test</pre>
<p>
<strong><em>Determine what an process is actually doing what</em></strong>
  <pre class="codeline"> sudo strace -pXXXX -e trace=file</pre>
<p>
<strong><em>Adding formatting to an xml document for easier reading</em></strong>
  <pre class="codeline"> xmllint --format <filename> > <output file></pre>
<p>
<p>
<strong><em>Adding formatting to an xml document for easier reading</em></strong>
  <pre class="codeline"> xmllint --format <filename> > <output file></pre>
<p>
</div>
</body>
</html>