&& The Linux Operating System -----------------------------------. quote: "I'm doing a (free) operating system, just a hobby,..." (Linus Torvalds) This booklet is designed to help with common tasks on a Linux system. The book is designed to be presentable as a series of "recipes" for accomplishing common tasks. These recipes consist of a plain English one-line description, followed by the Linux command which carries out the task. The document is focussed on performing tasks in Linux using the 'command line' or 'console'. The format of the booklet was largely inspired by the "Linux Cookbook" www.dsl.org/cookbook @@ http://github.com/himanshuc/nixhacker/tree/master a good list of resources @@ http://www.pixelbeat.org/cmdline.html some command lines recipes @@ http://www.shell-fu.org/ some command line recipes @@ http://www.commandlinefu.com/ a very good site with lots of commandline tips @@ http://dsl.org/cookbook/cookbook_toc.html A very very good Linux User "cookbook" and the primary inspiration for the booklets on this site. * retrieve some recipes for commandlinefu in plain text format >> lynx -dump "http://www.commandlinefu.com/commands/browse/sort-by-votes/plaintext" | less ##(commandlinefu supports an 'api' for retrieval formats) lynx -dump http://www.commandlinefu.com/commands/browse/sort-by-votes/25/25 | less for i in $(seq 1 1000); do lynx -dump http://www.commandlinefu.com/commands/tagged/$i/plaintext; sleep 2; done GOOD BOOKS The UNIX Environment (Andrew Walker, Wiley 1984) The Linux Cookbook (Michael Stutz, No Starch Press) The Unix Programming Environment (Kernighan et al) GETTING HELP * show the short help description for all programs >> whatis -r '.*' >> for i in $(ls $(echo $PATH | tr ':' ' ')); do whatis $i; done | less * search for a program by its name or short help >> whatis -r '.*' | grep searchword * view the 'manual' page for the wc command >> man wc ##(sadly, man pages rarely have examples ...) * view the manual page in section 4 >> man 4 command * show what programs a available to install, on a debian linux distribution >> apt-cache search "search terms" ##(requires an internet connection) INSTALLING PROGRAMS * download a file to install and then check the md5sum >> md5sum filename ##(compare this with the given value) LINUX PACKAGES .... @@ other software: dselect, aptitude, synaptic The simple installation of new software on a linux system is achieved via a 'packaging' system. On a Debian-style linux distribution (such as Ubuntu) these package files have a '.deb' extension, where as on a redhat-style Linux distribution (such as Fedora), these files have a '.rpm' filename extension DEBIAN PACKAGES .... Dpkg may be used to install '.deb' files which are already present on the local machine, where as apt-get is capable of downloading the package from a repository. * update the repository cache >> sudo apt-get update ##(add repositories to /etc/apt/sources.list) * show all packages which are currently installed >> dpkg -l | less * show just the names and descriptions of packages installed >> dpkg -l | awk '{$1=""; $2=$2 ","; $3=""; print }' | less * show all files which were installed as part of 'package' >> dpkg -L package * find out which package a file belongs to >> dpkg -S /path/to/file * install the package in the 'miscfiles-1.1.7.deb' file >> dpkg -i miscfiles-1.1.7.deb * install a program on a linux system >> sudo apt-get install program * install a particular version of a program / package >> apt-get install program=X.Y.Z-n * check which versions of a program or package are available >> apt-cache policy programname * search for a program to install which has something to do with 'web' >> apt-cache search web * show all programs whose names do not begin with 'lib' >> apt-cache search '.*' | grep -v '^lib' | sort | less * show the details for a particular program/ package >> apt-cache show programname * uninstall a program >> apt-get remove --purge program * get the source code for a program or package >> sudo apt-get source program ##(the files get put in the current folder) ##(uncomment the 'deb-src' line in 'sources.list') * put a 'source line' in /etc/apt/sources.list >> deb-src http://http.us.debian.org/debian etch main contrib >> sudo apt-get update ##(to update the package lists) * upgrade your Debian system to the most recent release >> apt-get update >> apt-get dist-upgrade COMPILING AND INSTALLING PROGRAMS .... If a 'package' is not available for the Linux distribution which you have on your computer, then you may have to download and install the source code and any libraries which it depends on. This is also the case when you want the absolutely latest cutting-edge version of a piece of open-source software. * download the source code >> wget ... * unpack the downloaded source code >> tar xvzf source.tar.gz >> tar jvzf source.tar.bz2 ##( ?? ) * look in the unpacked folder to see if there is a 'configure' script >> cd sourcefolder; ls * make sure that a compiler (gcc for example) is installed on the computer >> sudo apt-get install build-essentials * configure, compile and install a program >> ./configure && make && make install >> sudo ./configure && make && make install MAKING DEBIAN PACKAGES .... @@ http://tldp.org/HOWTO/html_single/Debian-Binary-Package-Building-HOWTO/ Instructions about how to make a debian (binary) package that can be installed with the 'apt-get' command. * list all files which a debian package will install >> dpkg-deb -c var/cache/apt/archives/somepackage.deb * show information about a debian package >> dpkg-deb -I var/cache/apt/archives/somepackage.deb * list all the files within a debian package file >> ar tv somepackage.deb * show information about the format of the 'control' file of a debian package >> man 5 deb-control ##(the control file determines dependencies etc) * find problems in a debian package >> lintian somepackage.deb =** how to make a simple debian binary package ----------------------------------------------: * create a 'debian/usr/bin' folder and copy the executable there >> mkdir -p ./debian/usr/bin; cp someprogram ./debian/usr/bin * create a man page for the program >> vim someprogram.1 ##(this is an optional step) * create a man page folder and copy the man page there, >> mkdir -p ./debian/usr/share/man/man1 >> cp ./man/man1/someprogram.1 ./debian/usr/share/man/man1 * compress the man page with gzip (an option step) * create a copyright file to include in the package >> find /usr/share/doc -name "copyright" ##(see examples of copyright files) * create a folder for the copyright file and copy the file there >> mkdir -p ./debian/usr/share/doc/someprogram; ... >> cp ./copyright ./debian/usr/share/doc/someprogram * find out the dependencies of all programs which your program uses >> dkpg -S /bin/cat ##(this will display 'coreutils: /bin/cat) * find out the version numbers of dependencies >> apt-cache showpkg coreutils ##(this information is needed for the 'Dependencies' field of the control file) make a control file.. ---------------------- Package: someprogram Version: 1.1-1 Section: base Priority: optional Architecture: all Depends: bash (>= 2.05a-11), textutils (>= 2.0-12), awk, procps (>= \ 1:2.0.7-8), sed (>= 3.02-8), grep (>= 2.4.2-3), coreutils (>= 5.0-5) Maintainer: James Tree Description: parses a text stream This script parses a text stream using a stack virtual machine ,,, * copy the control file to the 'DEBIAN' folder >> mkdir -p debian/DEBIAN; cp control debian/DEBIAN ##(on some debians: find ./debian -type d | xargs chmod 755) *make a change log file (an optional step) ------------------------------------------- xlinuxstatus (1.2-1) * Made Debian package lintian clean. -- James Tree 2002-12-13 ,,, * make a changelog.Debian file (optional) ----------------------------------------- linuxstatus Debian maintainer and upstream author are identical. Therefore see also normal changelog file for Debian changes. ,,, * create and rename the package file >> dpkg-deb --build debian; mv debian.deb someprogram_1.1-1_all.deb ##(or build as root 'fakeroot dpkg-deb --build debian') * install the newly created package >> dpkg -i ./someprogram_1.1-1_all.deb * remove the installed package >> dpkg -r someprogram =; SCREEN CAPTURE # The process of creating an image from which is currently displayed # on the computer screen is generally known as "screen capture" * command line screen capture >> scrot TEXT AND CHARACTER RECOGNITION # the process of extracting text data from an image file # (which has usually been obtained by scanning a page of typed text) is # known as "ocr" for Optical Character Recognition == ocr tools .. tesseract-ocr - Command line OCR tool .. gocr - the same .. ocrad - . .. clara - . .. unpaper - improve the quality of scanned text images .. IMAGES VIEWING IMAGES .... @@ http://www.linux.org/apps/all/Graphics/Image_Viewing.html a list of all image viewers * view images from a console >> zgv * the imagemagick image viewer >> display * a quick image viewer for X windows >> qiv * another image viewer >> showimg * browse through all "gif" images in the current folder "gallery" style >> display 'vid:*.gif' * browse all "gif" images in the imagemagick/examples folder >> display 'vid:/usr/doc/imagemagick/examples/*.gif == imagemagick "display" viewer commands .. .. [SPC], Display the next image specified on the command line. .. [BKSP], Display the previous image specified on the command line. .. C-q, Quit displaying the image and exit display. .. C-s, Write the image to a file. .. <, Halve the image size. .. >, Double the image size. .. -, Return the image to its original size. .. /, Rotate image 90 degrees clockwise. .. \, Rotate image 90 degrees counter-clockwise. .. ?, Open a new window with information about the image .. h, Toggle a horizontal mirror image. .. v, Toggle a vertical mirror image. .. * create an html and thumbnail image gallery >> galrey * debian: imseek, imview, paul IMAGE INFORMATION .... @@ http://www.imagemagick.org/script/escape.php lists all possible image information codes # use the image magick "identify" command with the -format switch * show all image formats supported by image magick >> identify -list format * show the width x height of the image "bird.png" >> identify -format "%wx%h" bird.png ##(prints "16x20" for example) * rename all "png" files, so that the name includes the width and height >> for f in *.png; do rename "s/\./-"$(identify -format "%wx%h" $f)"./" $f; done o- - For example "tree.png" --> "tree-16x30.png" - this assumes the file name has only 1 "." character) - use the "rename -n" switch to just see what would be renamed, but do nothing) - this is rather slow on my bash shell - * another way to do the same thing ------------------------------------------------------------------------ for f in *.png do mv $f ${f/.*/}-$(identify -format "%wx%h" $f)${f/*./.}; done ,,, OPTIMIZING IMAGES .... * optimize png images >> debian: optipng IMAGE MAGICK BUILT IN PATTERNS .... @@ http://www.imagemagick.org/script/formats.php a list of all built in patterns which can be referenced as if they were an image pattern:checkerboard IMAGE FORMATS .... == image format notes .. .. png, is a lossless format, has transparency .. jpg, highly compressable, lossy, no transparency .. gif, animations possible, compressed, .. TRANSFORMING IMAGES .... ## generally "mogrify" modifies the image itself whereas "convert" ## creates a copy of the changed image. The exception is where the ## -format option is used with "mogrify" @@ http://www.imagemagick.org/Usage/ lots of usage examples @@ http://dsl.org/cookbook/cookbook_23.html#SEC338 * scale and transform images >> mogrify ##(changes the image) >> convert ##(doesnt change) * reduce in size all jpeg images in the folder by 50% >> mogrify -resize 50% *.jpg ##(the images are changed) * reduce by 50% an image and save the reduced image as "rose2.jpg" >> convert rose.jpg -resize 50% rose2.jpg * rotate an image by 90 degrees where the height exceeds the width >> mogrify -rotate '90<' image.jpeg * rotate an image where the width exceeds the height >> mogrify -rotate '90>' image.jpeg * reduce the colors of an image to 4 with color diffusion >> mogrify -colors 4 -dither image.jpeg * convert an image to black and white >> mogrify -monochrome colourful.jpeg * change the brightness of the image "cat.jpg" >> mogrify -gamma .8 cat.jpg * add a grey border 2px wide and 4px high around an image >> mogrify -border 2x4 cat.jpg >> mogrify -frame 8x8 cat.jpg ##(a bevelled border) CONVERTING BETWEEN IMAGE FORMATS .... * convert the jpeg image "rose.jpg" to the "png" format. >> convert rose.jpg rose.png * convert all png images in the current folder to the jpeg format >> mogrify -format jpg *.png ##(a copy of the images is created) * convert a large number of "gif" images to the "png" format >> find . -name "*.ico" | xargs mogrify -format png ##(this is very fast) ANIMATION .... # frame numbers start at 0 * show information for all the frames in an animation >> identify tree.ico * display how many frames an animation has >> identify tree.gif | wc -l * extract the 2nd frame from an animated gif and save to "new.gif" >> convert animation.gif[frame1] new.gif * delete all frames except the first 3 (0, 1, 2) >> convert animation.gif -delete 3--1 frames012.gif ##(-1 refers to the last frame in an animation) * split an animation into its constituent frames >> convert canvas_prev.gif -scene 1 +adjoin frame_%03d.gif * convert a set of frames to an animation >> convert frame_???.gif anim_rebuilt.gif ##(combines frame_001.gif frame_002.gif etc) * show only image files which have only one frame (are not animations) >> for i in $(find . -name "*" | head -200); do echo -n $i "#"; identify $i | wc -l ; done | grep "#1$" ##(this is rather horrifyingly slow, about 10 file per second) ANIMATED GIFS .... * gifsicle CROPPING IMAGES .... * crop a 32x32 pixel block starting at pixel (16,16) and save to "new.gif" >> convert tree.gif -crop 32x32+16+16 new.gif * divide an image into 20x20 pixel blocks and save to zz-nn.png etc >> convert tree.gif -crop 20x20 zz.png ##(creates files zz-1.png, zz-2.png etc) * crop a 32x32 pixel block from the center of the image and save in "new.png" >> convert tree.png -gravity Center -crop 32x32+0+0 new.png * crop a 20x20 block, 5 pixels from the bottom and centered horizontally >> convert tree.png -gravity South -crop 20x20+0+5 new.gif * crop the top left hand quarter of the image >> convert tree.png -crop 50%x+0+0 new.png ##(the height of the new image is 1/2 the height of the old image) * crop a 20 pixel vertical strip starting at the top left hand corner >> convert tree.png -crop 20x0+0+0 new.png * crop a 30 pixel horizontal strip starting at the top left hand corner >> convert tree.png -crop 0x30+0+0 new.png * convert "tree.png" into a series of 20 pixel wide vertical strips >> convert tree.png -crop 20x strip-%d.png * remove a 10 pixel horizontal strip from the top of "tree.png" >> convert tree.png -crop +0+10 new.png * remove a 10 pixel horizontal strip from the bottom of "tree.png" >> convert tree.png -gravity South -crop +0+10 new.png >> convert tree.png -crop -0-10 new.png ##(the same) * remove a 10 pixel horizontal strip from the bottom of all png files >> mogrify -gravity South -crop +0+10 *.png ##(the actual image files are modified, no copies are made) * remove 20 pixels from the top and bottom of the image "tree.png" >> convert tree.png -shave 0x20 new.png SPLICING .... * splice a 10 pixel white horizontal band in the image, at vertical pixel 30 >> convert tree.png -background white -splice 0x10+0+30 new.png * add a 10 pixel row of blue space at the bottom of the image. >> convert tree.png -gravity south -background blue -splice 0x10 new.png CREATING IMAGE MONTAGES .... # An image montage is combining several images side by side, resizing # the images and arranging in blocks with spacing. # In an image montage all the images are the same size. * create a horizontal strip of images "tree-1.gif" etc resizing to 16x16 pixels >> montage tree-[1-7].gif -tile 9x1 -geometry 16x16+1+1 new.gif ##(images are laid out "row-wise") * create a 3x3 block of images resized to 16 pixels with 1 pixel spacing >> montage tree-[1-7].gif -tile 3x3 -geometry 16x16+1+1 new.gif ##(if there are not enough images to fill the block, white space is created) * create a two row montage with an unknown number of images >> montage tree-*.gif -tile x2 -geometry 16x16+1+1 new.gif ##(the necessary columns are calculated by imagemagick) * create a 1 row montages, 5 pixel spaces, blue background and image shadow >> montage balloon.gif medical.gif -tile x1 -shadow \ >> -geometry +5+5 -background lightblue new.jpg * create a 1 row montage putting a 3 pixel frame around each image >> montage balloon.gif medical.gif -tile x1 -frame 3 -geometry +5+5 new.jpg * create a 1 row montage with image-magic "built in" images >> montage logo: rose: -tile x1 -frame 3 -geometry +5+5 new.jpg * create a 1 row montage with a title above the montage >> montage balloon.gif medical.gif -tile x1 -geometry '60x60+2+2>' \ >> -title 'My Images' titled.jpg * create a 1 row montage with a gap in the middle >> montage medical.gif null: present.gif -tile x1 -geometry +2+2 new.jpg * create a montage with the file name under the image, silver background >> montage -label '%f' x.png -font Arial -pointsize 30 -geometry +0+0 -background silver xy.png * create an overlapped and rotated image montage >> montage null: font_*.gif null: -background none -rotate 30 \ >> -background white -tile x1 -geometry -8+2 montage_rot_overlap.jp HTML IMAGE MAPS .... * create an image map with a 4 pixel gap between images, with 10 columns >> montage '*.png' -geometry +4+4 -tile 10x map.html ##(this produces 3 files, an html file, shtml file, and a png file) ##(note that "montage" expands the file list, not the shell) * create an html image map in "png" format and convert to "jpg" to reduce size ------------------------------------------------------------------------------ montage '../img_photos/*_orig.*[120x90]' -auto-orient \ -geometry +5+5 -tile 5x -frame 5 -shadow photo_jpeg.html convert photo_jpeg.png photo_jpeg.jpg perl -i -lpe 's/src="photo_jpeg.png"/src="photo_jpeg.jpg"/' photo_jpeg.html rm -f photo_jpeg.png photo_jpeg_map.shtml ,,, CREATING A MONTAGE OF LOTS OF IMAGES .... * create a montage of the first 300 files in the current folder >> montage $(ls | head -300) -tile 45x -geometry +3+3 new.png * create montage of 300 icons each of the files in the current folder --------------------------------------------------------------------- for j in $(seq 300 300 36000); do montage $(ls | head -$j | tail -300) -tile 40x -geometry +3+3 fav-$i.html; done ,,, * quote the file list to avoid shell file argument limits >> montage '*.png' -tile 45x -geometry +3+3 new.png ##(the "montage" tool expands the file list instead of the shell) HTML THUMBNAIL GALLERIES .... @@ http://www.imagemagick.org/Usage/thumbnails/ * create an html thumbnail gallery with image map >> montage -label '%t\n%[width]x%[height]' \ >> -size 512x512 '../img_photos/*_orig.*[120x90]' -auto-orient \ >> -geometry +5+5 -tile 5x -frame 5 -shadow photo_index.html UNICODE CHARACTERS IN IMAGES .... printf "\u201Cdouble\u2 01D" | \ convert -background lightblue -fill blue -pointsize 36 \ label:@- label_quotes.gif IMAGE CAPTIONS .... * add a caption at the bottom of an image in 15px of white space >> convert tree.png -gravity south -background LimeGreen -splice 0x15 \ >> -annotate 0x0 'Tree' new.png * draw a label at the bottom of the image in a grey rectangle (superimposed) >> convert tree.png -fill '#0008' -draw 'rectangle 5,128,114,145' \ >> -fill white -annotate +10+141 'Tree' new.png * the same >> convert tree.png -fill white -undercolor '#00000080' -gravity South \ >> -annotate +0+5 ' Faerie Dragon ' tree.jpg * label an image with its file name and size in pixels on a blue background >> montage balloon.gif -tile x1 -geometry '90x32>' -pointsize 10 \ >> -set label '%f\n%wx%h' -background SkyBlue new.jpg COMBINING IMAGES .... * overlay "tree.gif" in the center of "mountain.gif" and save to "new.gif" >> composite -gravity center tree.gif mountain.gif new.gif * position "point.gif" exactly within "tree.gif" and save as "new.gif" >> composite -geometry +31+105 point.gif tree.gif new.jpg IMAGE BORDERS .... * add 10 pixel red side bars to an image >> convert tree.png -bordercolor Red -border 10x0 new.png IMAGES OF TEXT .... * show what fonts are available with image magick (for writing images) >> convert -list type ##(for IM older than v6.3.5-7) >> convert -list font ##(for newer versions) * divide an image of text into text lines >> divide_vert * create an image of "Tree" blue on white background >> convert -background white -fill blue -font Candice \ >> -pointsize 72 label:Tree label.gif ##(check that the font is available with the command above) * create an image, size 165x70, of text "Tree" with text centered in image >> convert -fill blue -font Helvetica -size 165x70 -pointsize 24 \ >> -gravity center label:Tree tree.gif * create an image of text, with text font size auto-fitted to image size >> convert -fill blue -font Candice -size 165x70 label:Tree new.gif * create image of text with the width of the image autofitted to the text width >> convert -fill blue -font Candice -size 160x label:Tree new.gif ##(in this way the text fills the image box well) * for multiline labels IM version 6.2.5 or later is required * create a label with text from the file "/etc/motd" >> convert -background lightblue -fill blue label:@/etc/motd new.gif * create an image label from text from standard input >> echo "hello!" | convert -fill blue label:@- new.gif * create an image label in which the text wraps on long lines >> convert -fill blue -font Courier -pointsize 36 -size 320x \ >> caption:'This is long caption line.' new.gif * create an image label in which the text wraps on long lines and is centered >> convert -fill blue -font Courier -pointsize 36 -size 320x \ >> -gravity center caption:'This is long caption line.' new.gif ##( >= IM version 6.2.0) * image label with text size autofitted to the image size >> convert -background lightblue -fill blue -font Candice -size 320x140 \ >> caption:'This text is resized to best fill the space given.' \ >> caption_filled.gif ##( >= IM v6.3.2) * create an image of the word "shadow" with a shadow, magenta and red * with a transparent background. --------------------------------------------------------------------- convert -size 320x85 xc:transparent -font Bookman-DemiItalic -pointsize 72 \ -draw "text 25,60 'shadow'" -channel RGBA -gaussian 0x6 -fill darkred \ -stroke magenta -draw "text 20,55 'shadow'" fuzzy-magick.png ,,, * create white text with a hard shadow on a transparent background ------------------------------------------------------------------ convert -size 320x100 xc:transparent -font Bookman -pointsize 72 \ -fill black -draw "text 28,68 'Tree'" \ -fill white -draw "text 25,65 'Tree'" new.jpg ##(the trick is to draw the text twice with a slight displacement) ,,, * create a "bevelled" font using the "shade operator, white with black edges ---------------------------------------------------------------------------- convert -size 320x100 xc:black -font Bookman -pointsize 72 \ -fill white -annotate +25+65 'Tree' -shade 140x60 new.jpg ,,, DRAWING COMMANDS TO CREATE IMAGES .... * draw a white rectangle with a black border in a 100x60 image >> convert -size 100x60 xc:skyblue -fill white -stroke black \ >> -draw "rectangle 20,10 80,50" new.gif COMPRESSING IMAGES .... * reduce the quality (and file size) of an image, and save in "tree-80.jpg" >> convert tree.jpg -quality 80% tree-80.jpg * reduce a jpeg even more >> -sampling-factor 2x1 IMAGE FORMATS .... * convert a jpg image to the "png" format >> convert tree.jpeg tree.png ##(a copy in the new format is made) IMAGE EDITORS .... * xmorph, gimp * a simple paint program >> xpaint * edit bitmaps, pixmaps >> tkpaint, bitmap PATTERNS AND TILINGS .... * kali THREE D IMAGE EDITORS .... * sced, moonlight MAKING CHARTS AND GRAPHS * the 'pic' and 'grap' groff preprocessors * gnuplot * debian: rlplot, ygraph, dia * an example of creating a graph with gnuplot >> http://www.pixelbeat.org/docs/web/access_log/analyzing.html * for drawing flow charts and other figures use >> xfig VIDEO ----------------------------------------------------------------- VIDEO EDITING PROGRAMS .... Kino, Cinelerra, Avidemux, Kdelive, Lives, Lumiera, Pitivi, Open Movie Editor == video tools .. .. vcdimager - A VideoCD (VCD) image mastering and ripping too .. VIEWING VIDEO .... * debian: avifile-player - video player for AVI/ASF/WMF files COMPRESSING VIDEO .... * convert an avi video into a gif animation >> convert -quiet -delay 1 plane.avi plane.gif * convert an avi to gif with a colour map and no dithering for more compression >> convert -quiet -delay 1 plane.avi +dither -map colormap_332.png plane_ugc_nd.gif * use ordered dithering for good compression and quality >> convert -quiet -delay 1 plane.avi -ordered-dither o8x8,8,8,4 +map plane_od.gif UNICODE * show the current locale (language and character encoding) >> locale * show a hexdump of a text file >> hd file.txt >> hexdump file.txt #(the format is a little different) ------------------------------------------------------------------ TEXT FILES ------------------------------------------------------------------ ANALYSING LANGUAGE .... * output a list of word senses available for the word 'browse', >> wn browse * output a list of words from the dictionary that begin with the string 'homew' >> look homew ##(prints something like 'homeward' and `homework' ...) * list all words in the dictionary containing the string 'dont' regardless of case >> grep -i dont /usr/dict/words * list all words in the dictionary that end with 'ing' >> grep ing^ /usr/dict/words * list all of the words that are composed only of vowels >> grep -i '^[aeiou]*$' /usr/dict/words * output a list of words that rhyme with 'friend', search '/usr/dict/words' for lines ending with `end': >> grep 'end$' /usr/dict/words * search the WordNet dictionary for nouns that begin with 'homew' >> wn homew -grepn * search the WordNet dictionary for nouns and adjectives that begin with 'homew' >> wn homew -grepn -grepa * list the definitions of the word 'slope' >> wn slope -over * output all of the synonyms (same meaning) for the noun 'break' >> wn break -synsn * output all of the synonyms for the verb 'break' >> wn break -synsv * output all of the antonyms (opposite meaning) for the adjective 'sad' >> wn sad -antsa A hypernym of a word is a related term whose meaning is more general * output all of the hypernyms for the noun 'cat' >> wn cat -hypen Debian 'dict' * check file 'dissertation' for clichés or other misused phrases, type: >> diction dissertation | less * check file 'dissertation' for clichés or other misused phrases, and write the output to a file called 'dissertation.diction' >> diction dissertation > dissertation.diction * If you don't specify a file name, diction reads text from the standard * output all lines containing double words in the file 'dissertation' >> diction dissertation | grep 'Double word' * check the readability of the file 'dissertation' >> style dissertation Like diction, style reads text from the standard input if no text is given * output all sentences in the file 'dissertation' whose ARI is greater than a value of 20 >> style -r 20 dissertation * output all sentences longer than 14 words in the file 'dissertation' >> style -l 14 dissertation * output the number of lines, words, and characters in file 'outline' >> wc outline * output the number of characters in file 'classified.ad' >> wc -c classified.ad Use wc with the '-w' option to specify that just the number of words be * output the number of words in the file 'story' >> wc -w story * output the combined number of words for all the files with a '.txt' file name extension in the current directory >> cat *.txt | wc -w * output the number of lines in the file 'outline' >> wc -l outline * output a word-frequency list of the text file 'naked_lunch', >> tr ' ' '\n' < naked_lunch | sort | uniq -c * output a count of the number of unique words in the text file 'naked_lunch' >> tr ' ' ' > ' < naked_lunch | sort | uniq -c | wc -l * rank the files rep.a, rep.b, rep.c in order of relevance to keywords 'saving' and `profit' >> rel "(saving & profit)" report.a report.b report.c * output a list of any files containing either 'invitation' or 'request' in the `~/mail' directory, ranked in order of relevancy, type: >> rel "(invitation | request)" ~/mail * output a list of any files containing 'invitation' and not 'wedding' in the `~/mail' directory, ranked in order of relevancy, type: >> rel "(invitation ! wedding)" ~/mail * output a list of any files containing 'invitation' and 'party' in the '~/mail' directory, ranked in order of relevancy >> rel "(invitation & party)" ~/mail SPELL CHECKING .... == spell checking programs .. spell, a non interactive spell checker .. ispell, a veteran program .. aspell, the gnu version .. myspell, .. * search for all debian packages which have something to do with spelling >> apt-cache search spell * print all words mispelled in "chapter.txt" with line numbers and file name >> spell -n -o chapter.txt * spell check the file 'fall-lecture.draft' >> spell fall-lecture.draft occurance willl * spell check the file 'chapter.1.txt', with misspellings put in the file 'bad.sp' >> spell chapter.1.txt > bad.sp * output a sorted list of the misspelled words that are in the file 'fall-lecture.draft' >> spell fall-lecture.draft | sort | uniq * interactively spell check 'fall-lecture.notes' >> ispell fall-lecture.notes ISPELL .... # ispell is an older and simpler program than aspell * install a British English dictionary for the "ispell" spell checker >> sudo apt-get install ibritish * check and correct the spelling interactively in document "report.txt" >> ispell report.txt ##(when a misspelling is found, type the number of the replacement) * spell check "file.txt" using a british english dictionary >> ispell -d british file.txt * spell check a document written in spanish (using a spanish dictionary) >> ispell -d spanish archivo.txt * show what dictionaries are available locally for ispell >> ls /usr/lib/ispell/ * the ispell dictionaries are all called "i[language-name]" >> dictionary files: icatalan, ibrazilian ... * spell check and correct "thesis.tex" which is a LaTeX format document >> ispell -t thesis.tex ##(ispell ignores the latex mark-up codes) ASPELL .... # aspell is a more modern and capable spell checking program # @@ http://aspell.net/ the official site @@ http://aspell.net/man-html/index.html A usage manual for aspell * show options for aspell and available dictionaries >> aspell help | less * show locally available dictionaries for aspell >> aspell dicts * install a British and American English dictionary for aspell >> sudo apt-get install aspell-en * install a spanish dictionary for aspell >> sudo apt-get install aspell-es * show all debian packages and dictionaries for aspell >> apt-cache search aspell * interactively check the spelling of the file "chapter.txt" >> aspell -c chapter.txt >> aspell check chapter.txt ##(the same) * check the spelling of "chapter.txt" using British English spelling >> aspell -d british -c chapter.txt >> aspell -d en_GB -c chapter.txt ##(this is the same) * check the spelling of "chapter.txt" using a Spanish dictionary >> aspell -d spanish -c chapter.txt >> aspell -d es -c chapter.txt ##(this is the same) * check spelling in the comments in the shell script (lines starting with "#") >> aspell --mode=comment -c script.sh ##(!!doesnt work on my version) * checking the spelling in the tex/latex file "chapter.tex" >> aspell -t -c chapter.tex * show available filters for spell-checking particular types of files >> aspell filters >> aspell dump filters ##(the same) * create a vim "mapping" to use aspell within vim >> map TT :w!:!aspell check %:e! %> aspell --add-filter=context --add-context-delimiters="* \0" -c francisco.txt ##(doesnt really work) WRAPPING TEXT LINES .... * format a text file with lines 80 characters long, >> fmt -w 80 textfile ##(short lines lengthened) >> fmt -s -w 80 textfile ##(short lines are not lengthened) SPLITTING TEXT FILES .... * split a file into a maximum of 10 files on lines containing * '#200', '#400', '#600' etc with output files called "zz00", "zz01", etc >> csplit -f zz file.txt "/^#1?[24680]00$/" {8} ##(the split occurs 'before' the line containing the match) CONVERTING OTHER FORMATS TO TEXT .... * convert from html to text >> lynx -dump http://url > textfile >> links-dump http://url > textfile ##(may render tables) >> w3m -dump http://url > textfile ##(may tables better) * remove the newline characters from the text file 'autoexec.bat' >> fromdos autoexec.bat * add newline characters to all of '.tex' files in the current directory >> todos *.tex COMPARING AND PATCHING TEXT FILES .... The process of 'patching' a text or code file is very important in the world of open-source development (and therefore in the development of Linux itself). Patching allows non-linear changes to be made to a file and is usually used in conjuction with 'diff' * compare the files 'manuscript.old' and `manuscript.new' >> diff manuscript.old manuscript.new * peruse the files 'olive' and 'green' side by side indicating differences >> sdiff olive green | less * output a difference report for files 'tree', 'bush', and 'hedge', >> diff3 tree bush hedge > arbol * update the original file 'manuscript.new' with the patchfile 'manuscript.diff' >> patch manuscript.new manuscript.diff MICROSOFT WORD DOCUMENTS * view a microsoft word document >> wv * convert a ms word document to text >> antiword, catdoc * convert the Word file 'resume.doc' to LaTeX >> word2x -f latex resume.doc ##(writes a new file, 'resume.ltx',) * convert all of the '.DOC' Word files in the current directory to LaTeX files with maximum line widths of 40 characters >> word2x -f latex -w 40 *.DOC * convert the Word file 'resume.doc' to a plain text file called 'resume' >> word2x -f text resume.doc resume * search the text of the Word file 'resume.doc' for the string 'linux' regardless of case >> word2x resume.doc - | grep -i linux POSTSCRIPT DOCUMENTS Postscript is a document format very well suited to being printed. However it is more difficult to view than the adobe pdf format (since the acrobat viewer is more widely installed on Microsoft Windows computers) * view a ps file >> evince * view a postscript file >> ghostview file.ps >> gv file.ps ##(the same) * view a postscript file as plain text >> ps2ascii book.ps | less CONVERTING POSTSCRIPT TO OTHER FORMATS .... * convert a postscript document to the pdf format >> ps2pdf eg.ps ##(a file called "eg.pdf" is created) * convert a postscript file to plain text >> ps2ascii book.ps book.ps.txt PDF DOCUMENTS debian packages: xpdf, xpdf-utils @@ http://www.foolabs.com/xpdf/ home page for the pdf-utils programs == useful pdf programs .. evince - view a pdf file .. xpdf - view a pdf file .. pdftops - convert to postscript .. pdftotext - convert or view a pdf file as plain text .. pdfinfo - display lots of information about a pdf document .. pdfimages - extract images from pdffiles .. pdffonts - show what fonts are used in a pdf document .. pdftoppm - .. xpdfrc - .. pdfcrack - PDF files password cracker .. pdfjam - collection of PDF document handling utilities .. pdftk - A useful tool for manipulating PDF documents .. pdftohtml - Translates PDF documents into HTML format .. VIEWING PDF DOCUMENTS .... * view a pdf file as plain text >> pdftotext file.pdf - | less * view a compressed pdf document >> zxpdf book.pdf.gz * view a pdf document >> xpdf book.pdf ##(xpdf appears much faster than "acroread") >> acroread book.pdf ##(its easier to select text in acroread) * view the 10th page of a pdf document with a zoom factor of 200% >> xpdf -z 200 book.pdf 10 * view a pdf document in "continuous" mode starting at the 5th page >> xpdf -cont book.pdf 5 ##(one can scroll smoothly through the whole file) * view a pdf document in full-screen mode >> xpdf -fullscreen book.pdf ##(the -z zoom option doesnt work with fullscreen) * view a pdf document with the colours reversed (usually white text on black) >> xpdf -rv book.pdf ANALYSE PDF DOCUMENTS .... * extract the images from the file and save them in jpeg format >> pdfimages -j report.pdf * display information about a pdf document >> pdfinfo book.pdf * show how many pages a pdf document has >> pdfinfo book.pdf | sed -n "/Pages:/s/^ *Pages: *//p" * show the page size of a pdf document >> pdfinfo book.pdf | grep -i "Page size:" * show what fonts are used in a pdf document >> pdffonts book.pdf EDITING PDF FILES .... * pdfedit CONVERTING PDF TO OTHER FORMATS .... * convert a pdf document to plain text (without any formatting) >> pdftotext file.pdf ##(a file called "file.txt" is created) * convert the pdf file "file.pdf" to a plain text file "ouput.txt" >> pdftotext file.pdf output.txt >> ps2ascii file.pdf output.txt ##(more or less the same) * pstotext TYPESETTING TEXT DOCUMENTS # This section explains how to prepare documents to be printed # on paper. USING ENSCRIPT ........ * enscript is the simplest solution for creating a printed document * view a postscript file >> gv file.ps * convert plain text 'file.txt' to pdf >> enscript -p file.ps file.txt; ps2pdf file.ps * create a pdf file with no "headers" called "output.pdf" >> enscript -B -p output.ps file.txt; pd2pdf output.ps * create a postscript file "new.ps" using the font "Helvetica12" >> enscript -f "Helvetica12" -p new.ps file.txt * create and view a postscript file with 2 columns with surrounding boxes >> enscript -2 -j -p file.ps file.txt; gv file.ps * convert "file.txt" to pdf "file.pdf" landscape format with 2 columns >> enscript -2r -p file.ps file.txt; ps2pdf file.ps * create a postscript file with fancy grey headings and view it with "gv" >> enscript -G -p file.ps file.txt; gv file.ps ##(the header contains the date, file name and the page number) * create a postscript file with a "watermark" large font text diagonally * down the page. >> enscript -u"Resume" -p file.ps file.txt; gv file.ps * create a colourised webpage of the file "Point.java" with syntax highlighting >> enscript -whtml --color -E -p file.html Point.java * print with a gaudy header, two columns, landscape, code highlighting, 2-up printing. >> enscript -G2rE -U2 foo.c * create a rich text file from a text file >> enscript -wrtf -B -p file.rtf file.txt ##(rtf files can be opened in wysiwyg editors like MS Word) == other typesetting tools .. groff, an old unix system .. latex, widely used for scientific and maths docs .. docbook, an xml based "super" system .. pod, the perl document system, .. man, the old unix "manual page" system based on groff .. CONVERTING CHARACTER ENCODINGS .... @@ http://asis.epfl.ch/GNU.MISC/recode-3.6/recode_3.html * see also iconv (older) * show possible conversions with the 'recode' tool >> recode -l | less * convert latin9 (western europe) character encoding to utf8 >> recode iso-8859-15..utf8 report.txt ##(the actual file is changed) * convert from the local character set to the latin1 encoding saving to "new.txt" >> recode ..lat1 < file.txt > new.txt ##(the original file is unchanged) * convert to html >> recode ..HTML < file.txt > file.html * convert from utf8 to html with verbose output >> recode -v u8..h < file.txt * convert from MS Windows utf8 to the local character set >> recode utf-8/CRLF.. file-to-change.txt VIEWING TEXT FILES .... * view a text file starting at the end >> less +G COMPARING TEXT FILES .... * find the extra lines in file2 >> diff file1 file2 | grep ^> * find the extra lines in file1 >> diff file1 file2 | grep ^< SORTING TEXT .... * perform a numerical sort on the file, using the 3rd field, colon delimited >> sort -n -k3 -t: /etc/passwd * sort 'file.txt' in reverse alphabetical order >> sort -r file.txt * sort text files by the number of lines which they contain, reverse order >> file * | grep text | sed "s/:.*//" | xargs wc -l | sort -rn | grep -v "total$" | less ANALYSING TEXT .... * show unique words and the number of times each word occurs >> tr -sc ’[A-Z][a-z]’ ’[\012*]’ < file.txt | sort | uniq -c | less SEARCHING TEXT .... gnu grep special characters ------------------------------- .. \< - matches beginning of a word .. \> - matches the end of a word .. \b - matches a word boundary .. [:upper:] - matches upper case letters (unicode) .. [:lower:] - matches lower case letters (unicode) .. [:space:] - matches space characters .. ,,, 00- the "--" sequence may be "escaped" in grep. for example: grep "\-\-" file.txt - * search for lines which begin with "#" in the text file "script" >> grep '^#'