% latex generated by the script '/home/project-web/bumble/cgi-bin/text2latex.cgi' % with help from the sed script 'text2latex.sed' % query string: 'books/sed/sed-book' % sFileName= 'sed-book' % sDirName = 'books/sed' % sName = '' --> % latex by http://bumble.sourceforge.net/scripts/text2latex.sed % this script is a work in progress \documentclass[11pt]{article} \usepackage[hmargin=2cm, vmargin=2cm]{geometry} \usepackage{url} \usepackage{hyperref} \begin{document} \\&\\& Sed, The Unix Stream Editor ------------------------------: quote: "Should be part of every gentlemans toolbox" (me) The gnu sed has some features not found in other versions of sed. But many of the examples should work for all versions of sed. Sed is actually a virtual machine, which has 2 registers, the 'work-space' and the 'hold-space' and a set of "instructions" or commands which manipulate these two registers. This document contains an almost complete copy of the "sed oneliners" document available at sed.sf.net/sed1line.html since that document is so complete and well compiled that there is not much point reinventing the wheel. I have only reformatted the document slightly so that it can be compiled into pdf using the scripts at bumble.sf.net/books/format-book @@ \url{http://www.grymoire.com/Unix/Sed.html} A concise introduction @@ \url{http://sed.sourceforge.net/} The "home page" for sed. @@ \url{http://sed.sourceforge.net/sedfaq.html} comprehensive recipes @@ \url{http://sed.sourceforge.net/sed1line.txt} a long list of one line sed scripts covering all the advanced techniques BOOKS ABOUT SED @@ sed and awk, 2nd Edition Dale Dougherty and Arnold Robbins (O'Reilly, 1997; \url{http://www.ora.com),} @@ UNIX Text Processing Dale Dougherty and Tim O'Reilly (Hayden Books, 1987) @@ the tutorials by Mike Arst distributed in U-SEDIT2.ZIP. @@ "Mastering Regular Expressions" Jeffrey Friedl (O'Reilly, 1997). HISTORY The sed stream editor has been around since the very early days of unix. One of its original motivations was that the computer in use (pdp?) did not have the capability to load large text files into memory in order to edit them "interactively". GETTING HELP FOR SED * view the man page for sed >> man sed * view the man page for the line editing program 'ed' a forerunner of 'sed' >> man ed * view information about regular expression >> man regexp SIMPLE USAGE Sed takes one or more editing commands and applies all of them, in sequence, to each line of input. After all the commands have been applied to the first input line, that line is output and a second input line is taken for processing, and the cycle repeats. The preceding examples assume that input comes from the standard input device (i.e, the console, normally this will be piped input). One or more filenames can be appended to the command line if the input does not come from stdin. Output is sent to stdout (the screen). Thus >> cat filename | sed '10q' \\#\\#( uses piped input ) >> sed '10q' filename \\#\\#( same effect, avoids a useless "cat" ) >> sed '10q' filename > newfile \\#\\#( redirects output to disk ) * transliterate characters >> sed "y/abcd/ABCD/g" GOTCHAS These are tricky things that can cause the poor uncautious sed-user head-scratching beffudlement and a pervading existential 'ennui'. * check for 'dos' carriage return characters '\\$\\backslash\\$r' which are not desireable >> sed -n l file.txt \\#\\#('l' prints unprintable characters visibly) >> sed -n 'l' file.txt \\#\\#(the same) * if you saw any '\\$\\backslash\\$r' characters in the previous, then get rid of them >> fromdos file.txt \\#\\#(those '\\$\\backslash\\$r' can cause annoying silent bugs) >> dos2unix file.txt \\#\\#(another way to say the same thing, I think) * duplicate words, using groupings and backreferences. >> s/\\$\\backslash\\$([a-z]+\\$\\backslash\\$)/\\$\\backslash\\$1\\$\\backslash\\$1/g; >> s/([a-z]+)/\\$\\backslash\\$1\\$\\backslash\\$1/g; \\#\\#(gnu sed with the -r flag, this is 'gotcha') * the 'e' gnu command seems to have to be the last thing on the line >> echo ls | "e;p" \\#\\#(doesnt work) >> echo ls | "e" \\#\\#(works) * the 'q' quit command can only have one address >> 1,4 q \\#\\#(doesnt work) * if a one-line script contains a "!", it must use single quotes >> sed -n "/\\^ *\\#/!p" \\#\\#(Incorrect: the bash shell uses the "!" character) >> sed -n '/\\^ *\\#/!p' \\#\\#(Correct) * if an 'a' (append) or 'i' (insert) command has a space after the backslash (in any line) then the command doesnt work. This is a really annoying one. use 'sed -i "s/\\$\\backslash\\$s\\$\\backslash\\$+\\$//" scriptname' to fix this problem * 'd' delete immediately starts a new cycle ------------------------------------------- One might try not to print anything after a line containing "-end-" including the line itself, with the following command \texttt{/-end-/} \{ d;q \} BUT it doesnt work because the 'q' command will never be executed owing to the behavior of the 'd' delete command ,,, * when putting a ']' character in brackets, dont escape it >> s/\\$\\backslash\\$[[\\^]]*\\$\\backslash\\$]//g NOT s/\\$\\backslash\\$[[\\^\\$\\backslash\\$]]*\\$\\backslash\\$]//g \\#\\#(at least in gnu sed ) SCRIPTS * run a sed script in a file >> sed -f script file >> sed --file=script file \\#\\#(the same) >> cat file | sed -f script \\#\\#(the same) * find out where your sed executable is >> which sed * create an interpreting script with extended regexes (-r) >> \\#!/bin/sed -rf \\#\\#(this is the first script line, try 'which sed') >> ./script \\#\\#(the script can be run with just its name) SED COMMANDS \title{} \author{bumble.sourceforge.net} \maketitle \tableofcontents sed command summary .. d, delete the pattern space, start the next cycle .. p, print the pattern space .. q 4, quit the script with exit code '4' .. e 'ls', execute the shell 'ls' command and put in the pattern space .. e, execute the shell command in the pattern space .. a\\$\\backslash\\$, append text, each line ending in '\\$\\backslash\\$' except the last .. i\\$\\backslash\\$, insert text, each line ending in '\\$\\backslash\\$' except the last .. D, delete text in the pattern space up to the first newline. .. N, add a newline and the next line of input to the pattern space. .. P, print out the portion of the pattern space up to the first newline. .. h, replace the hold space with the pattern space. .. H, append a newline and the pattern space to the hold space. .. g, replace the pattern space with the hold space. .. G, append a newline and the hold space to pattern space. .. x, Exchange the contents of the hold and pattern spaces. .., \title{} \author{bumble.sourceforge.net} \maketitle \tableofcontents modifiers .. \\$\\backslash\\$L, Turn the replacement to lowercase until a \\$\\backslash\\$U or \\$\\backslash\\$E is found, .. \\$\\backslash\\$l, Turn the next character to lowercase, .. \\$\\backslash\\$U, Turn the replacement to uppercase until a \\$\\backslash\\$L or \\$\\backslash\\$E is found, .. \\$\\backslash\\$u, Turn the next character to uppercase, .. \\$\\backslash\\$E, Stop case conversion started by \\$\\backslash\\$L or \\$\\backslash\\$U. .., * add 2 lines of text at the end of the file -------------------------------------------- \\$ a\\$\\backslash\\$ added text\\$\\backslash\\$ last line ,,, MULTIPLE COMMANDS * either use the -e switch or semi-colons ';' for multiple commands >> sed -e 's/a/A/' -e 's/b/B/' > sed 's/a/A/; s/b/B/' > sed 's/frog/toad/g' < oldfile > newfile * This does NOT work, the file 'textfile' get truncated >> sed 's/frog/toad/g' < textfile > textfile \\#\\#(use the -i switch instead) * change yes to no in all files ending in '.txt' and back up to .bak >> sed -i.bak "s/yes/no/g" *.txt \\#\\#(gnused 4) COMMAND LINE SWITCHES * print only the lines in a file which contain the word 'sky' >> sed -n 's/sky/\\&/p' > sed 's/tree//g' f1 f2 f3 * run the sed script 'script' >> sed -f script > \\#!/bin/sed -rf \\#\\#(this is the first line of the script, try 'which sed') >> ./script \\#\\#(the script can be run with just its name) * create an interpreting script which doesnt print the input lines >> \\#\\#(!/bin/sed -nf \\#\\#(this may not work in some versions of sed) SUBSTITION SWITCHES * delete only the second match of the word 'big' on each line >> sed 's/big//2' > sed 's/big//g' > sed 's/cat//2g' > sed 's/.//80' > sed '100q' test INSERTING TEXT .... * insert 2 lines of text before the first line of the file ---------------------------------------------------------- 1i\\$\\backslash\\$ \\$\\backslash\\$ \\$\\backslash\\$ ,,, WRITING TO A FILE * write all lines which contain the word 'sky' to the file 'words' >> sed -n 's/sky/\\&/w words' < file \\#\\#(only one space between w and 'words') READING FROM A FILE Maybe the 'r' and 'e' commands cannot be used within a \{ \} brace block? * replace a line in a text file with the contents of another file >> cat index.html | sed '/aaa/r file.txt' | less * replace a block of text in a file with another file >> cat index.html | sed '/