Administering The Linux Operating System

Table of Contents

last revision
27 October 2011, 6:33pm
book quality
0.1
,

quote: ",..." ()

This book is designed to provide "recipes" for administering a Linux operating system. The emphasis shall be on using the command line (the bash shell) to carry out tasks. This book will treat subjects such as the administration of servers and services, hardware configuration

www: http://bumble.sf.net/books/linux/linux-book.txt
another booklet about using linux from the command line, for oriented towards basic usage rather than adminstration
www: http://bumble.sf.net/books/
a series of booklets about linux software
www: http://www.shell-fu.org/
lots of advanced command line recipes.

Bug Tracking ‹↑›

trac, nicely polished html display

Managing Installed Software ‹↑›

Viewing Installed Software ‹↑›

list your largest installed packages (on debian/ubuntu)

 dpigs

list your largest installed packages.

 wajig large

show all packages which are currently installed

 dpkg -l | less

generate a list of installed packages on debian-based systems

 dpkg --get-selections > LIST_FILE

show just the names and descriptions of packages installed

 dpkg -l | awk '{$1=""; $2=$2 ","; $3=""; print }' | less

show all files which were installed as part of 'package'

 dpkg -L package

Analysing Installed Software ‹↑›

display the total number of packages available

 apt-cache stats

Find the dates your debian/ubuntu packages were installed.

 ls /var/lib/dpkg/info/*.list -lht |less

List your largest installed packages.

 dpkg --get-selections | cut -f1 | while read pkg; do dpkg -L $pkg | xargs -I'{}' bash -c 'if [ ! -d "{}" ]; then echo "{}"; fi' | tr '\n' '\000' | du -c --files0-from - | tail -1 | sed "s/total/$pkg/"; done

show installed but unused linux headers, image, or modules

 dpkg -l 'linux-*' | sed '/^ii/!d;/'"$(uname -r | sed "s/\(.*\)-\([^0-9]\+\)/\1/")"'/d;s/^[^ ]* [^ ]* \([^ ]*\).*/\1/;/[0-9]/!d'

Find running binary executables that were not installed using dpkg

 cat /var/lib/dpkg/info/*.list > /tmp/listin ; ls /proc/*/exe |xargs -l readlink | grep -xvFf /tmp/listin; rm /tmp/listin

Automatic Start Up ‹↑›

Services can be configured to automatically start when the computer starts. This may be convenient or else dangerous, depending.

stop the apache2 service from beginning a computer start up

 sudo chkconfig apache2 off

Software Installation ‹↑›

www: install
software This term refers to the process of transferring and arranging files on your computer so that the computer is capable of executing these files in order to carry out a particular task.
www: package
a package is a way of combining and organising all the files which make up a program or piece of software so that the program is easy to install on a linux computer. Packages are good because they remove the need for you to compile, maintain or worry about the installed software.
www: repository
Is an internet location which contains software which has already been packaged for easy installation on your computer.
www: cache
update update the information about available software which is stored on your computer.
www: download
This refers to the process of transfering a file from the internet onto the local computer.
www: program
The word program is more or less synonymous with the term 'a piece of software' and refers to a file or set of files which a computer is able to execute in order to carry out a particular task (such as editing a text file, or navigating the web)
www: application
synonymous with 'program' or 'piece of software'
www: compile
Compiling software is the process of converting a series of text files containing instructions (and which are readable and understandable by human beings) into files which the computer can execute (run). In order to compile a program you need another program, which is called a 'compiler' which is the program which is responsible for converting the source code text files into executable program files (which are readable only by a computer).
www: compiler
A program which converts source code text files into executable program files. An example of a compiler is the 'gcc' program
www: source
code The text files which are readable by human beings which can be converted into programs or software with the use of a 'compiler' program.
www: programming
language All software is written in a programming language, which is a language which has a syntax which is much simpler than normal human languages (such as Greek) and which can be converted into files which are readable and executable by a computer.
www: open
source software This is software for which the source code text files are freely available for anyone who wants them (usually via the Internet). Closed source software are programs for which only the binary executable files are publicly available , and since these files are only readable by a computer this prevents human beings from inspecting or modifying this type of software. The Linux operating system is itself an example of open source software, since the source code is freely available to anybody who wants to have it or modify it (and who has the time to try to understand it).
www: free
software a contentious and ambiguous term which causes fruitless debates.
In the context of computers, software is also known as 'programs', 'applications' or 'executables'. Software allows you to carry out a particular task on your computer.

The simple installation of new software on a Linux system is achieved via 'packages'. Thousands of packages of free and open-source software are available. On a Debian-style linux distribution (such as Ubuntu) these package files have a '.deb' extension, where as on a redhat-style Linux distribution (such as Fedora), these files have a '.rpm' file-name extension

download a file to install and then check the md5sum

 md5sum filename    compare this with the given value

software installation jargon

Searching For Available Software ‹↑›

www: freshmeat
a list of new linux software and updates to existing software
www: linux.softpedia.org
another software indexing site
check which versions of a program or package are available
 apt-cache policy programname

search for a program to install which has something to do with 'web'

 apt-cache search web

show all programs whose names do not begin with 'lib'

 apt-cache search '.*' | grep -v '^lib' | sort | less

The names of 'code libraries' often begin with the letters 'lib' and normally they are automatically installed when you install some piece of linux software with 'apt-get'

convert a software search for 'splice' to a LaTeX table

 apt-cache search splice | sed 's/ - / \& /; s/$/ \\\\/' | less

Latex can be used for printing documents using the 'pdf' or postscript format. See http://bumble.sf.net/books/latex/ for more detailed information about latex.

choose a software bundle to install

 tasksel

find a file's package or list a package's contents.

 dlocate [ package | string ]

Installing From The Web ‹↑›

update the repository cache

 sudo apt-get update

Add repositories to /etc/apt/sources.list

Install a LAMP server in a Debian based distribution

 sudo tasksel install lamp-server

install the 'scribus' desktop publishing software on a debian system

 sudo apt-get install scribus

install a particular version of a program / package

 apt-get install program=X.Y.Z-n

limit the apt-get download speed (to make it share nicely with others)

 sudo apt-get -o Acquire::http::Dl-Limit=25 install <package>

Installing From Files ‹↑›

Another way to install software is to obtain package files (from the net, from a cd or usb-key) transfer them to you computer and then install them using dpkg. Since the 'apt' tool uses dpkg to actually carry out the installation of the files which it obtains from the net, this is equivalent to using 'apt-get' except that you have to manually download or obtain the files yourself rather than having apt-get do it for you.

install the package in the 'miscfiles-1.1.7.deb' file

 dpkg -i miscfiles-1.1.7.deb

Installing Source Code ‹↑›

The whole point of the linux system, is that the source code for software is available to you if you want to look at it or modify it.

get the source code for a program or package

 sudo apt-get source program

the files get put in the current folder uncomment the 'deb-src' line in 'sources.list'

put a 'source line' in /etc/apt/sources.list

 deb-src http://http.us.debian.org/debian etch main contrib
 sudo apt-get update  to update the package lists

Uninstalling And Removing Programs ‹↑›

Anything that you do, you need to be able to undo. This is a basic principle of computing. One of the great advantages of using packaged software, is that it is easy to get rid of the programs installed.

uninstall a program

 sudo apt-get remove --purge program

purge all packages marked with 'rc'

 sudo dpkg --purge `dpkg -l | awk '/^r/{print $2}'`

Analysing Software Packages ‹↑›

search ubuntu packages to find which package contains the file

 apt-file find bin/programname

find out which package a file belongs to

 dpkg -S /path/to/file

show some ubuntu "easter eggs"

 apt-get moo

Converting Package Formats ‹↑›

Convert a debian package to an rpm (redhat/fedora) package

 alien -r -c file.deb

Debian Packages ‹↑›

Dpkg may be used to install '.deb' files which are already present on the local machine, where as apt-get is capable of downloading the package from a repository, as well as resolving dependencies of a package on other packages (so that the user doesn't have to worry about it)

Check the apt security keys

 apt-key list

update a program providing a functionality on debian

 update-alternatives --config java

check which version of the program is installed on your Debian

 aptitude show $PROGRAM | grep Vers

show the details for a particular program/ package

 apt-cache show programname

upgrade your Debian system to the most recent release

 apt-get update
 apt-get dist-upgrade

List installed deb packages by size

 dpkg-query -Wf '${Installed-Size}\t${Package}\n' | sort -n

Remove today's installed packages

 grep "install " /var/log/dpkg.log | awk '{print $4}' | xargs apt-get -y

Repeatedly purge orphaned packages on Debian-like Linuxes

 while [ $(deborphan | wc -l) -gt 0 ]; do dpkg --purge $(deborphan); done

Does a full update and cleaning in one line

 sudo apt-get update && sudo apt-get upgrade && sudo apt-get autoclean && sudo apt-get autoremove

Obtaining Source Code ‹↑›

The first step to installing an open source software project is to obtain the source code. If this fills you with dread and perplexity, remember that you almost never (these days) actually have to do this, since you can install just about any piece of linux software from the software repository for you version (distribution) of linux, with one smooth command such as 'sudo apt-get install prog' where "prog" is the software which you want to use.

However, to enter into the spirit and enjoyability of the open source world, you can download the source code, compile it and install it all by yourself, if you are so inclined.

There are various ways to obtain the source code for an open source software application. Usually the code will be stored in a 'version control system' which is designed to keep track of changes to the code and allow several developers at once to work on the project.

download a git source code tree

 git clone git://tasktools.org/task.git task.git

use subversion

 svn ...

use cvs to download a source code tree

 cvs ...

'Cvs' is an older version control system

use wget/curl to get the source code tar ball

 wget ....

download the source code in the form of a 'tar ball' into /usr/local/src

 cd /usr/local/src; wget something.tar.gz

Compiling And Installing Programs ‹↑›

If a 'package' is not available for the Linux distribution which you have on your computer, then you may have to download and install the source code and any libraries which it depends on. This is also the case when you want the absolutely latest cutting-edge version of a piece of open-source software.

download the source code in the form of a 'tar ball' into /usr/local/src

 cd /usr/local/src; wget something.tar.gz

If you dont want the source code, download it to the /tmp folder

unpack the downloaded source code

 tar xvzf prog.n.n.tar.gz
 tar jvzf prog.n.n.tar.bz2   ?? 

look in the unpacked folder to see if there is a 'configure' script

 cd prog.n.n; ls

make sure that a compiler (gcc for example) is installed on the computer

 sudo apt-get install build-essentials

If there is a file 'makeFile' then it may not be necessary to run 'configure'. You may need to use

configure, compile and install a program configure the software for your version (distribution) of linux

 ./configure

compile the source code into executeable programs

 make

if you see something like the following then you have a problem!!

   s2d_main.c:21:17: error: png.h: No such file or directory
   make: *** [s2d_main.o] Error 1

The message above means that you have an unfulfilled dependency. These dependencies are usually library of code which the current application needs in order to run. The names of the libraries often start with 'lib'.

Try reading the 'README.txt' file to see if that gives you any hints about what you need to install.

 vim README.txt

If the readme file mentions a dependency then search for its package.

search for libjpeg related packages

 apt-cache search libjpeg

If a program has many dependencies, and you know what they are, you could try installing those dependencies via the package management tool (such as apt-get or yum).

install the 'libjpeg' development files

 apt-get install libjpeg-dev

copy the compiled executables to their proper locations on the system

 sudo make install

find out what library dependencies a given executable has

 ldd programname

get rid of some files

 make clean; make dist-clean ... untested

Trouble Shooting ‹↑›

If you see the message "cant find foo.h" but the library 'libfoo' is installed, then try installing the library 'libfoo-dev'

install a development library

 sudo apt-get install libfoo-dev

If you see 'cant locate -lfoo' try installing 'libfoo'

Example Source Installations ‹↑›

This section provides a few 'worked examples' of installing software using tar.gz files containing the source code for the application.

obtain, compile and install the 'task' organiser application

  % cd /usr/local/src
  % wget http://www.taskwarrior.org/download/task-1.9.0.tar.gz
  % ls
  % gunzip task-1.9.0.tar.gz
  % tar xf task-1.9.0.tar
  % cd task-1.9.0
  % ./configure
  % make
  % make install  

Repositories ‹↑›

add a ppa repository for the chrome browser in ubuntu 10.4

 sudo add-apt-repository ppa:chromium-daily/ppa

the list of repositories on a debian-type system is at

 /etc/apt/sources.list

add a ppa repository for ubuntu

 deb http://ppa.launchpad.net/chromium-daily/ppa/ubuntu karmic main
 deb-src http://ppa.launchpad.net/chromium-daily/ppa/ubuntu karmic main
 sudo apt-get update
 sudo apt-get install chromium-browser

Making Debian Packages ‹↑›

www: http://tldp.org/HOWTO/html_single/Debian-Binary-Package-Building-HOWTO/
Instructions about how to make a debian (binary) package that can be installed with the 'apt-get' command.
list all files which a debian package will install
 dpkg-deb -c var/cache/apt/archives/somepackage.deb

show information about a debian package

 dpkg-deb -I var/cache/apt/archives/somepackage.deb

list all the files within a debian package file

 ar tv somepackage.deb

show information about the format of the 'control' file of a debian package

 man 5 deb-control the control file determines dependencies etc

find problems in a debian package

 lintian somepackage.deb

Redhat Style Software Packages ‹↑›

download and install the software package in one step

 rpm -ivh 'http://www.website.com/path/to/desired_software_package.rpm'

Find out what package some command belongs to (on RPM systems)

 rpm -qif `which more`

tools for managing software
apt - the debian system
dpkg - install or query package files on the local computer
aptitude - slightly more 'sophisticated' than apt-get
synaptic - a graphical program similar to 'add/remove programs' on Ms
dselect -
tasksel - a graphical way to install whole software 'suites'
yum - the red/hat fedora package tool
wajig

Account And User Management ‹↑›

tools
yum

Scheduling ‹↑›

do something in 1 hours time

 sleep 1h; dosomething

This is not very good because if the computer reboots the task will not be executed

carry out a task at 3pm (either today or tomorrow if after 3pm)

 echo "task -opt -opt" | at 15:00

This command will still work if the computer reboots

carry out a task at 5am in 2 days time

 echo "task -a -b" | at 5am + 2 days

carry out a task in 1 hours time

 echo "task" | at noew + 1 hour

see what tasks are scheduled

 atq

remove a task which was scheduled to run

 atrm

perform a series of tasks at midnight tonight

 at midnight < commands.txt

Wine ‹↑›

A way of running windows progs

www: www.winehg.com
wine official site
install wine on debian
 sudo apt-get install wine

tools
ac - from the gnu acct package login accounting

Emulation ‹↑›

www: virtualbox
allows windows to be run on top of linux in an emulation Thus allowing windows progs to be used.

File Systems ‹↑›

www: fuse
use space file systems. That is file systems which dont have to be compiled with the kernel in order to work. Allows for example remote file system to be mounted as if local, eg SSHFS, EncFS for encryption, NTFS-3G access to ntfs partitions
www: ext2
A linux file system without journalling
www: ext3
A linux file system with journalling
www: FAT
a microsoft file system used widely on usb keys because it is not journalled and can be read by most operating systems, including MS, Linux and Mac OSX without any extra software installed
www: NTFS
Another microsoft one
www: ReiserFS
www: XFS

Organisation ‹↑›

using
new application menu is created
download an exe double click
wine installs the prog in .wine/drive_c (c:)

Task ‹↑›

www: http://taskwarrior.org
The home page for the 'task' command line organiser
Task is a very 'unixy' application for organising things which you have to do.

install the 'task' tool on ubuntu version > 10.4

sudo add-apt-repository ppa:ultrafredde/ppa sudo apt-get update sudo apt-get install task ,,,

show the version and set up a new configuration file if none exists

 task version

add a new task with description 'Prepare tax'

 task add Prepare tax

add a new task 'Pay bills' with a high priority

 task add priority:H Pay bills

display existing tasks

 task ls

display more information about all the tasks

 task list

mark the second task as having been done

 task 2 done

delete the second task

 task 2 delete

another way to delete the second task

 task 2 rm

put task 1 in the project 'Wedding'

 task 1 project:Wedding

display only those tasks in the 'Wedding' project

 task ls project:Wedding
 task ls project:We     the same usually

show only tasks with 'tax' in the description

 task ls tax

set the priority of task 2 to 'medium' (H,M,L are allowed)

 task 2 priority:M
 task 2 pri:M      the same

create a new task 'Do tax' with high priority in project money

 task add project:money priority:H Do tax
 task add pro:money pri:H Do tax

add the due date 25 may 2008 for task 3

 task 3 due:5/25/2010

tools
task - a command line personal organiser

add a new task 'pay rent' due on thursday recuring weekly until 31/aug

 task add pay rent due:thursday recur:weekly until:8/31/2008

a task which recurs every fortnight

 task add Pay rent due:fri recur:fortnight
 task add Pay rent due:fri recur:2w

Imap Servers ‹↑›

establish a connection with an imap server via 'telnet'

 telnet imap.server.name 143

show the capabilities of an imap server tel:>> a capability

logout of a telnet session on an imap server tel:>> z logout

Input And Output ‹↑›

Use /dev/full to test language I/O-failsafety

 perl -e 'print 1, 2, 3' > /dev/full

Virtual Machines ‹↑›

Detect if we are running on a VMware virtual machine

 dmidecode | awk '/VMware Virtual Platform/ {print $3,$4,$5}'

Log Files ‹↑›

An important part of administering linux is viewing, analysing and maintaining the 'log' files which record many different types of activity on a linux machine. Log files are kept in '/var/log'

Analyse an Apache access log for the most common IP addresses

 tail -10000 access_log | awk '{print $1}' | sort | uniq -c | sort -n | tail

see old logins using rotated login files

 ( last ; ls -t /var/log/wtmp.* | while read line ; do ( rm /tmp/wtmp-junk ; zcat $line 2>/dev/null || bzcat $line ) > /tmp/junk-wtmp ; last -f /tmp/junk-wtmp ; done ) | less

The 'rm' stuff is supposed to be guarding against 'symlink' attack

Log File Rotation ‹↑›

The process of compressing ('gzipping') log files and moving them to a different name is called 'rotation' of the log files. The logrotate utility uses a simple configuration format to determine how often files are rotated and how many 'back issues' are kept.

view the manual page for how log files rotate

 man logrotate

view the default settings for how often log files are rotated

 less /etc/logrotate.conf

Some applications override the settings in 'logrotate.conf'

view which applications override the default settings

  ls /etc/logrotate.d

DOCUMENT-NOTES:

due dates for task
task 3 due:tomorrow - task 3 due tomorrow
task 1 due:23rd - task 1 due 23rd of this month
task 1 due:eom - task 1 at the end of the month
task 1 due:friday - task 1 next friday
task 1 due:fri - the same
task 1 du:fri - the same again
task 1 due:mon - task 1 next monday