Unix Seminar Notes - Intermediate


About this document.

This document covers intermediate Unix user commands and information.

This document was first written by Alex Batko in 1999; it was updated over the years, and most recently in October 2006.

This document is licensed under the GNU Free Documentation License: http://www.gnu.org/copyleft/fdl.html

This document along with others is available at the following URL: http://socsinfo.cs.mcgill.ca/wiki/Unix_Seminars

Please refer all complaints and comments to "help AT cs.mcgill.ca".



Who is giving this seminar ?

Seminars are given by system staff of the School of Computer Science (SOCS), of McGill Univeristy, as a service offered at the start of each fall and winter term. Apart from systems and network administration, SOCS system staff also provide a daily user support service called the SOCS Help Desk.

For information about upcoming seminars, please consult: http://socsinfo.cs.mcgill.ca/wiki/Unix_Seminars


The SOCS Help Desk.

The SOCS Help Desk is the user support centre for answers to computing questions of students, faculty, staff, and anyone else having a valid account at the School of Computer Science at McGill University.

The SOCS Help Desk is physically located at the McConnell Engineering Building, in room 209N. For help, call 398-7087 or send an email to help@cs.mcgill.ca. For more information about the SOCS Help Desk, please consult: http://socsinfo.cs.mcgill.ca/wiki/Help_Desk

When sending mail to the Help Desk make sure to complete the subject field. Write one to four key words that are relavent to your email. Write short messages that get to the point. In most cases you should include the name(s) of the host(s), or computer(s), involved in your particular problem.

For information about issues specific to SOCS, such as user accounts, the network, and services, please consult: http://socsinfo.cs.mcgill.ca/


Viewing directory contents.

When no argument is passed to ``ls'', the contents of the current directory is listed; when a ordinary file is passed to ``ls'' as an argument, the filename is repeated (along with other associated information, if it is requested by one or more options); when a directory is given as an argument, its contents are listed. The output of ``ls'' is sorted alphabetically and files usually appear before directories, though the latter is system specific.

Dot-files.

Dot-files are files and directories whose names begin with a period (``.''). The only difference between non-dot-files and dot-files is that dot-files are not displayed by the command ``ls'' unless the ``-a'' option or the ``-A'' option is used. Because dot-files are not shown by ``ls'' by default, they are also known as hidden files.

% ls -a         - List the contents of the current directory including
                  files and directories that begin with a dot ``.''.

Dot-files are usually configuration files of programs, and are considered "uninteresting", and therefore they don't need to be displayed each time the command ``ls'' is exectued.

Note that because of the nature of the Unix filesystem, ``ls -a'' will list the entries ``.'' (working directory) and ``..'' (parent directory) as members of every directory, without exception. That is because those directories are indeed found in every directory in the filesystem. This fact should cause you to question what this means for the root directory. The root directory is a special directory because rules apply to it that that do not apply to other "ordinary" directories. Amongst other rules, the ``..'' entry in the root directory still exists, but instead of pointing to a parent directory it contains a null value, and hence has no effect.

% ls -A         - Lists the contents of the current directory including
		  files and directories that begin with a dot ``.'',
                  with the exception of the working directory ``.'' and
                  the parent directory ``..''.

File attributes.

File attributes are properties associated with files, such as the mode (which is the file type, and permissions), the number of links to the file, the owner, the group, the size in bytes, the time of most recent modification, and the file name.

For completeness, it must be mentioned that everything in the filesystem is actually a file, including ordinary files, directories, and other special files such as links, device files, and named pipes; so in fact, all these entities have file attributes.

% ls -l         - List the contents of the pwd in long format, printing for
		  each entity the mode (file type and permissions), number of
		  links to the file, owner name, group name, size in bytes,
                  timestamp, and filename.

The following are a number of common invocations of ``ls'':

% ls -la        - List the contents of the pwd in long format, including
                  dot-files.
% ls -ltr       - List the contents of the pwd in long format, sorted by time
                  stamp (most recent modification time), and reversed (such
		  that the most recently modified files appear at the bottom).
% ls -ld        - List the contents of the pwd in long format, but if an
                  argument is a directory, list only its name (not its
                  contents).
% ls -lF        - List the contents of the pwd in long format, and mark each
                  non-regular file with symbol to indicate whether it is a
                  directory, executable, symbolic link, etc.

Having shown a command that takes two options, it's necessary to point out that option order has no meaning. To illustrate this point, note that the following invocations have the same effect:

% ls -la
% ls -al
% ls -l -a
% ls -a -l

The following is an example of the output of ``ls -laF'':

drwx--x--x  47 abatko   12345       3072 Sep 30 09:58 .
drwxr-xr-x  34 root     other       1024 Jul  2 08:30 ..
drwx------   3 abatko   12345        512 Jul 11 14:51 bin/
drwx--x--x  21 abatko   12345        512 Jul 30  2002 courses/
drwx------   4 abatko   12345       5632 Sep 30 09:58 mail/
drwx--x--x  34 abatko   12345       1024 Sep 26 08:23 public_html/
-rw-------   1 abatko   12345        732 Sep 24 15:16 reminders

The file attributes "owner", "group", and "mode" are described next. The file attribute for the "number of links" is described in the advanced-level material.


File access permissions (owner, group, mode).

Owner & group.

Unix file access is based on the concept of "users" and "groups". Every user on a system has a unique account with a unique login name and associated user identification number (UID). Likewise, every user on a system has a unique group identification number (GID).

It is possible, and sometimes convenient for different users to be able to share files that they are not owners of; for this purpose there exist "groups". If a user belongs to a group, they may have "read", "write", and/or "executable" access to particular files associated with that group.

Every user belongs to a primary group; by default at SOCS, primary GIDs are the same as each user's UID.

Every file belongs to one user and one group. A file is "owned" by the user who created it; its group is the user's primary group.

% id U          - Print UIDs, GIDs, and group membership of user U.
% groups U      - Print the groups user U is in.

% chgrp G F...  - Change the group ownership of file(s) F..., to group G.
% chgrp -R G D...
                - Recursively change the group ownership of directories D...,
                  and their contents, to group G.

% chown O F...  - Change the ownership of file(s) F, to owner O.
% chown -R O D...
                - Recursively change the ownership of directories D...,
                  and their contents, to owner O.
% chown U:G F...
                - If the username (or UID) U is followed by a colon and a
                  group (or GID) G; U is made the owner, and G is made the
                  group of file(s) F....

Note that if you need to change both a file's owner and group, change the group first, otherwise you won't have permission to change the group after you aren't the owner.

Mode.

The filesystem assigns a 10 bit "mode" to every file to govern how it can be accessed. The first bit indicates what type of file it is, while the remaining nine bits (which themselves are called "the mode") indicate the access privilages. The mode grants or denies permission to read, write, and/or execute a file. The mode grants permission separately to the owner of a file, to users belonging to group that the file is associated with, and to all other users (aka "the world").

With respect to non-directory files.

"Read" permission allow a user to see the content of a file. "Write" permission allows a user to make modifications to a file, as well as to delete it. "Execute" permission allows the user to execute a file as if it were a command, that is, to run it.

With respect to directories.

For directories, "execute" permission allow a user to enter a directory. When a directory has "write" permission and "execute" permission it is possible to add a file to a directory. To delete an empty directory it is enough to have "write" permission; to delete a non-empty directory, it is necessary to have "read", "write", and "execute" permissions. To see the contents of a directory it is necessary to have "read" and "execute" permissions.

The following first line is the output of the command ``ls -l index.html'':

-rw-r--r--   1 abatko   12345        7970 Sep  6 15:47 index.html

-
|---
| | ---
| |  | ---
| |  |  |
| |  |  +--------> Other permissions
| |  +-----------> Group permissions
| +--------------> User  permissions
+----------------> File type:    - = file
                                 d = directory
                                 l = link

From the output of the command above, the leading dash ``-'' tells us that the file ``index.html'' is an ordinary file, as opposed to being a directory, link, or some other special file. The owner of this file is user "abatko", and the group owner is the group "12345". Looking at just the permissions part of the mode, "rw-r--r--" tell us that the user has read and write permissions on this file, while the group "12345", as well as all other users, have only read permission.

Note that the mode also contains three bits that perform special tasks: the set-user-id bit, the set-group-id bit, and the stick bit; you can read more about file attributes by doing ``man chmod''.

Only the owner of a file, or the superuser may change a file's mode.

% chmod M F...  - Change the access permissions of file(s) F..., to mode M.
                  The mode M can be specified in symbolic mode according to
                  the format [ugo][+-=][rwx] or using the octal format.

Octal format.

In octal format each permission type is associated with a number, as shown in the table below. The numbers add up to a total of 7, which means that the maximum permissions, "rwx" is equal to "7". It is possible to sum any combination of the three types of permissions, for instance, to give "read and write" permission, the number would be "4+2=6".

type      octal   binary
------------------------
Read    =   4   =  100
Write   =   2   =  010
Execute =   1   =  001
           ---
            7

Below a user's home directory the most frequent permissions are 700 (for directories) and 600 (for non-directory files); below a user's website directory, the most frequent permissions are 711 (for directories) and 644 (for non-directory files). The reasoning is that the user (owner) should be able to "read, write, and execute" directories, and "read and write" files, while the group and world should be able to do the bare minimum, that is, to either have no access, or to have just "read" access to files, or just "execute" access to directories.

A user's home directory should have the mode set to "rwx--x--x", or "711". 711 gives the user (owner) full access, whereas the members of the group and the world are only able to execute the directory.

Some people like think of permissions in binary format (as in the table below) because the "1" digits nicely correspond to the positions in the "rwx" sequence. To illustrate this, consider "read and write" access; it would be "rw-", which requires a "1" on the left, and a "1" in the middle, and those are "100" (read) and "010" (write). So following binary addition, "100 + 010 = 110", which is 6 (read and write).

type      binary
----------------
Read    =  100
Write   =  010
Execute =  001

Default permissions.

It is possible to force the initial file mode of a newly created file. This can be implemented for security reasons. For instance it is not advisable for newly created files to be world readable, since the creator could forget to adjust the permissions. The umask utility sets the file mode creation mask of a given shell's execution environment.

% umask                 - Get the file mode creation mask of the current shell.
% umask M               - Set the file mode creation mask of the current shell
                          to mask M.  This mask affects the initial value of
                          the file permission bits of subsequently created
                          files.

The mask operand may be specified using the octal number representation. The mask is three octal digits corresponding to user, group, and other, respectively. For example, umask 022 removes read permission for (g)roup and (o)ther, changing directories normally created with mode 777 to 755. Files are normally created with mode 666 minus the umask; therefore they will result in the mode 644. When ``umask'' is executed with no arguments, it may return a two digit number, this is because the leading zero was omitted; therfore, 22 actully means 022.

Permanently setting the umask.

Setting the umask at the command line will take effect for the duration of the shell session. To avoid having to manually set the umask for each shell session it is possible to write the command in a personal shell configuration file (aka shell startup file) which is executed by the shell at startup (before it gives a user the prompt).

To learn how to set the umask permanently, continue reading - the information is explained below.


Personal webpages.

Users are permitted to have personal webpages at SOCS. Personal webpages are accessible via the address http://www.cs.mcgill.ca/~username. The following recipe outlines the creation of a very simple web page. By preciely carrying out these commands, you will create a publicly accessible webpage. The symbol ``~'' denotes the user's home directory.

% cd            - Change current directory to your home directory.
                  [This step is optional given that all the following steps
                  are relative to the home directory].

% chmod 711 ~
                - Change the mode of your home directory to 711.

% mkdir ~/public_html
                - Make a directory called public_html.

% chmod 711 ~/public_html
                - Change the mode of directory public_html to 711.

% xemacs ~/public_html/index.html
                - Make a file called index.html within the
                  directory public_html, having the following content:
                  <html><body>Hello, world!</body></html>
                  Note that you need not use xmacs; use any text editor you are
                  familiar with.

% chmod 644 ~/public_html/index.html
                - Change the mode of file index.html to 644.

For more information, please consult http://socsinfo.cs.mcgill.ca/wiki/Personal_Webpage


Environment variables.

Every shell session is associated with its own environment which is, in part, a list of name-value pairs called environment variables. Note that variables are named entities that store values. Some environment variables are used by the shell and other programs as sources of information needed in part to accomplish what they do.

% env           - Writes the current environment variables, with one name=value
                  pair per line.

PATH.

In the list of environment variables you will notice a variable called ``PATH''. This is perhaps the most important environment variable as it is a list of directories (aka search path) in which the shell looks for commands.

When a command is entered, the shell checks if it contains slashes. If the command contains slashes, the shell executes the named command. If the command does not contain slashes, the shell attempts to locate the command and it executes the first occurence found. The shell first checks whether a shell function by that name exists, then it scans the list of builtin shell commands. Finally the elements of the PATH variable are scanned for an executable file matching the command entered. As you can see, a command can reside in so many places that without the help of the shell, and especially the PATH variable, it would be very burdensome to enter commands.

% echo $PATH    - Returns the value of the environment variable PATH.  Note
                  that to indicate to the shell that a variable follows, the
                  variable name must be preceeded by the symbol ``$''.

The following is an example of the return value of ``echo $PATH''.

/sbin:/usr/sbin:/usr/local/sbin:/bin:/usr/bin:/usr/local/bin

``.'''' in PATH.

Note that when a command is not prefaced by its location (the path to where it resides), it's not certain from which directory it will be executed. This bit of uncertainty can pose some risk to your account, though if ``.'' is listed in PATH, the risk is greater.

Imagine that a malicious person has gained access to the system, and has placed a trojan horse program (a program that does something other than what it's disguised as) in ``/tmp'', a temporary word-writable directory found on all systems, with the hopes that eventually someone will execute it. If ``.'' is in your PATH, and one day you ``cd'' into ``/tmp'', you may unknowingly execute that command. Imagine that the trojan horse program was named ``les'', and one day, meaning to type ``less'', you accidentally omitted the second ``s''; in this case the trojan horse program would be executed, because there is no ``les'' command in the search path, other than in ``.''.

Based on this explanation, you may realise that the worst position in PATH to put ``.'' is at the front. The reason is that at that point the trojan horse program does not have to be a miss-spelled command. The trojan horse program could be named ``less'', and it will be executed because the shell will always execute the first occurence in the search path.

./foo vs foo.

The reason some users place ``.'' in PATH is because they want to be able to execute their own programs from their working directores. Indeed there are two ways to execute a program from a working directory: for reasons mentioned above, the wrong way is to put ``.'' in PATH; the proper way is to preceed the program name by ``./'' on the command line.

% ./foo         - The proper way to execute the program foo from the current
                  working directory.

% foo           - The wrong way to execute the program foo from the current
                  working directory, because it relies on ``.'' being in PATH.

``bin'''' in PATH.

If you write your own commands or programs that you want to have available for execution it is best to create the directory ``~/bin'' and put those commands or programs there. For convenience sake you should append ~/bin to your PATH. Modifying environment variables is discussed below.

which and whereis.

The following commands will display what will actually be executed when a given command is entered, as well as where things related to a given command are located.

% which C       - Displays the pathname or alias that would be executed if
                  argument C were given as a command.

% whereis C     - Locate the binary, source, and manual page files for
                  command C.

Shells.

Dozens of shells have been written for Unix systems, and a handful of them have become widely accepted, among those are: Bourne shell (sh), GNU Bourne-Again Shell (bash), Berkeley Unix C shell (csh), and tcsh.

A shell normally has a set of associated files called startup files that the shell sources (ie. it reads and executes each line of the file) during different instances of its execution (for instance, startup, and logout). There are two types of startup files - ones for system-wide settings, and ones for personal settings. The startup files containing system-wide configurations allow the system administrator to configure certain features for all users of a given shell, for instance, setting default environment variables and aliases. The personal startup files are intended for users to configure the shell to their liking, including modifying environment variables, setting aliases, creating shell functions, setting the way the prompt looks, etc. Modifying startup files is discussed below; however it must be mentioned that despite shells having a lot in common, they also have a lot of differences (with the most apparently obvious differences being syntactic).

Ways shells can be run.

Some shells can be run in a variety of ways: interactive login shell, interactive non-login shell, non-interactive login shell, non-interactive non-login shell.

Interactive shell.

An interactive shell session is one during which input is accepted from the keyboard; that is, it is a shell which presents a person with a command prompt and waits for input.

Non-interactive shell.

A non-interactive shell session is one in which input is accepted from a file of commands.

Login shell.

A login shell is the shell that a user has logged into (ie. it runs when a user first logs in - it is the first shell seen). A login shell is distinguished from a non-login shell (see below) by the first character of argument zero (``$0) of a login shell being ``-. A login shell can also be started if the shell was invoked with the ``--login'' option.

Non-login shell.

A non-login shell is one that is started by a means other than logging in. For example, if a user logs in, the first shell is a login shell; next if that user enters the command ``bash'', a new bash shell will be launched as a non-login shell (because logging in was not the method by which it was launched).

sh.

The Bourne shell is the oldest of the current Unix shells, thus it is a bit primitive and lacks job control features. Despite this, most Unix users consider the Bourne shell superior for shell programming (aka shell scripting). Csh and tcsh are considered "utterly inadequate" for shell programming. Shell programming is discussed in the advanced-level material.

Shell startup files.

/etc/profile    - System-wide settings and actions.

~/.profile      - User-specific settings and actions.

bash.

GNU Bourne-Again SHell is an enhanced and compatible version of sh, developed by the Free Software Foundation. Bash has features from sh, ksh, and csh. Some improvements offered by bash include: interactive command-line editing, and unlimited size command history.

Shell startup files.

/etc/profile    - System-wide settings and actions, for login shells only.

~/.bash_profile - User-specific settings and actions, for login shells only.

~/.bashrc       - User-specific settings and actions, for all non-login shells.

tcsh.

Tcsh is an enhanced and compatible version of csh. Notable enhancements over csh include: an enhanced history mechanism; a command-line editor, which supports GNU Emacs or vi-style key bindings; programmable, interactive word completion and listing; and spelling correction of filenames, commands, and variables.

Shell startup files.

/etc/csh.cshrc  - System-wide settings and actions.  Note that it's possible
                  that the file may actually be different depending on
                  compile-time options.

~/.cshrc        - User-specific settings and actions executed by login and
                  non-login shells; though if ~/.tcshrc exists, it will be
                  sourced instead.

~/.login        - User-specific settings and actions executed by login shells
		  after ~/.cshrc or ~/.tcshrc.  Please note that the shell
                  may be compiled to read ~/.login before ~/.tcshrc instead of
                  after: see the version shell variable to check.

Determine login shell.

% echo $0       - Returns the value of the name of the shell or shell script.
                  This variable is set at shell initialization.

Change login shell.

The file /etc/shells contains valid login shells.

Change shell temporarily.

To change your shell temporarily, enter the name or absolute path of the shell you would like to use.

Change shell permanently.

To change your shell permanently, you must ssh to one of the servers, and use the passwd command as follows:

% passwd -r nis -e U    - Change the user login shell of user U.

Modifying shell features.

Temporary changes.

Entering any command at the command line to modify the shell's environment and features is temporary - the change lasts as long as the shell session - so the next time the shell is started those settings will be nonexistent.

Permanent changes.

To retain any setting, such as a modified environment variable, an alias, a shell function, the prompt's appearance, etc., it is neccessary to write the those personalizations in your shell's startup file. In other words, to make a shell customization permanent, it is necessary to write the customization in the personal shell-startup file.

As an example, if for instance you want to set the umask, and your shell is tcsh, the place to set your umask is in ~/.cshrc because the login program sources ~/.cshrc before carrying out commands on remote systems. If your shell is sh, then set the umask in ~/.profile, and if your shell is bash, then set the umask in ~/.bash_profile. Make sure to place the umask command before the non-interactive-shell check (if any).

After having modified the shell startup file, for the settings to take effect, it is necessary to do one of two things. The simplest is to source the file without logging out. The other way is to terminate the shell session and start another one.

% source F      - Read and execute commands from file F in the current shell
                  environment.

% . F           - (bash) Same as ``source F'', though in bash the built-in
                  command ``.'' seems to be preferred over ``source''.

Setting environment variables.

bash.

% export        - With no arguments, the values of all variables are printed.

% export N=V    - Set the environment variable named N to value V.

% export PATH=$PATH:~/bin
                - Append ``~/bin'' to PATH.

tcsh.

% setenv        - With no arguments, the values of all variables are print.

% setenv N V    -  Set the environment variable named N to value V.

% setenv PATH $PATH:~/bin
                - Append ``~/bin'' to PATH.

Setting command aliases.

Aliases are shorthand names for commands. An alias is a name that is assumed temporarily, that is, until it is removed or the shell session ends. To retain an alias, such that it is set on all future login sessions, the alias command must be added to the personal shell startup file.

% alias         - (bash) Prints the list of current aliases, one per line.
% aliases       - (tcsh) Prints the list of current aliases, one per line.

% alias B=A     - (bash) Create an alias B for command A.
% alias B A     - (tcsh) Create an alias B for command A.

% unalias B     - Unalias the alias B.

The following are examples of useful aliases (written for the bash shell). It is easy to translate bash aliases to tcsh aliases by replacing the equal sign (``='') with a space (`` '').

% alias rm='rm -i'
% alias mv='mv -i'
% alias cp='cp -i'

% alias ..='cd ..'
% alias l='ls -l'
% alias j='jobs -l'

Window system.

Note: Most of the following text is taken from related entries at http://www.wikipedia.org

A graphical user interface (or GUI, often pronounced "goo-ee") is a method of interacting with a computer that uses graphical images and "widgets" in addition to text. GUIs generally consist of graphical widgets such as windows, menus, buttons, and icons, and employ both a pointing device and a keyboard.

A window system is a system that provides functionality for drawing and moving windows on the screen and also provides a mouse cursor for the computer's graphical user interface. The most common window system for Unix systems is the X Window System (aka X); and the most widely used implementation of it is "XFree86", which is free (open source) software.

Display manager.

A display manager is a program used to

A "session" is defined by the lifetime of a particular process; in the traditional character-based terminal world, a session is defined by the lifetime of the user's login shell. In the graphical display context, a session is defined by the lifetime an arbitrary session manager.

The default display manager for X is called "XDM", while the program is named ``xdm''. As part of its operation, xdm runs a script called ``Xsession'', which looks for the user's ``.xsession'' file to issue commands that determine the user's X session (for instance, what desktop environment or window manager to launch). Xsession implements a default session if no user-specific session exists.

~/.xsession     - User-specific settings and actions, executed at the start
                  of an X session.

Desktop environments and window managers.

A [graphical] desktop environment is a complete GUI solution that extends the X Window System. Examples of popular desktop environments are:

Desktop environments are complete GUI solutions to the point that they come with their own replacements for XDM. Though if XDM is the default display manager, it is still possible to use GNOME and KDE. See below for how to set your desktop environment.

Similar to a desktop environment in principle is a window manager, though it is not a "complete GUI solution" - it is a program for controlling the placement and appearance of application windows under X. Examples of popular window managers are:

See below for how to set your window manager.

Setting your desktop environment or window manager.

When XDM is the default display manager, it is necessary to edit the file ``~/.xsession'' by inserting a command that launches the desired desktop environment or window manager. The commands that launch these desktop environments and window managers are written in brackets in the two lists above.


I/O redirection operators.

The standard I/O facility provides some simple defaults for managing Input/Ouput. There are three default I/O streams: standard input (stdin), standard ouput (stdout), and standard error (stderr), respectively corresponding to file descriptor 0, 1, and 2.

Stdout consists of all "normal" ouput from programs, while stderr consists of error messages generated by programs. Stdin is normally read from the keyboard while stdout and stderr are sent to the terminal. The point is that these streams can be redirected using special redirection operators.

While standard I/O is a basic feature of Unix, the syntax used to redirect standard I/O depends on the shell being used.

The following describes common I/O redirections:

% P > F         - Redirect stdout of program P into the file F.
% P >> F        - Redirect stdout of program P appending it to file F.
% P >! F        - Redirect stdout of program P into the file F, overwriting
                  its contents (if any).

% P &> F        - Redirect stdout and stderr of P into the file F.
% P 2> F        - Redirect stderr of P into the file F.

% P < F         - Read stdin of P from the file F.
% P << c        - Read stdin of P from the keyboard until condition c.

The following are examples of redirecting stdout:

% ./a.out > foo.txt
                - Run the binary a.out and send its output to the file foo.txt.
% ./a.out >! foo.txt
                - Run the binary a.out and overwrite an existing version of
                  the file foo.txt.
% ./a.out >> foo.txt
                - Run the binary a.out and append its output a possibly
                  existing version of the file foo.txt.

Redirecting more than one file descriptor at a time is show by example below:

% ./a.out < iF.txt > oF.txt
                - Run the binary a.out, using as input, the input file iF.txt,
                  and redirect the output to the output file oF.txt,
                  appending to it if it already exists.
% ./a.out < iF.txt >! oF.txt
                - Run the binary a.out, using as input, the input file iF.txt,
                  and redirect the output to the output file oF.txt,
                  overwriting it if it already exists.
% ./a.out < iF.txt >> oF.txt
                - Run the binary a.out, using as input, the input file iF.txt,
                  and appending the output to the output file oF.txt.

It is also possible to redirect stdout of one program into the input of another, by a method called piping. This is accomplished by placing the pipe character (``|'') between two commands as follows:

% P1 | P2       - Pipe the stdout of P1 into the stdin of P2.
                  The output of P1 is streamed into P2, and is not buffered
                  until the completion of P1.

The following are examples of piping:

% sort F | uniq
                - Alphabetically sort the contents of file F, pipe the output
                  into the input of the command ``uniq'' so that only unique
                  lines are printed.

% P | less      - Execute some program P, and "pipe" its output to the input of
		  the program called ``less''.  This is commonly done when the
                  output of some program P is longer than the terminal window.

% P1 2>&1 | P2  - Duplicate stderr of P1 to its stdout and pipe the single
                  stdout stream to P2.  If P2 was the command ``less'', this
                  would be convenient for reading all the output of P1
		  (including stderr) if it scrolls beyond the height of your
                  terminal window).

How to find files.

% find P... E   - Search for files matching the expression E, in paths P.

% find . -name '*foo*' -print
                - Search for files in a directory hierarchy, for a base file
                  name that matches a shell pattern.
% find /u[0-9]*/[u]grad/ -name core -type f | xargs -i file {} \
    | grep 'core file' | awk -F: '{print $1}' | xargs -i ls -l {}

% find /u[0-9]*/[u]grad/ -name core -type f | xargs -i file {} \
    | grep 'core file' | awk -F: '{print $1}' | xargs -i rm -f {}

% find . ! \( -path '*oracle8.0.5*' -or -path '*developer*' \
    -or -path '*matlab*' \) -name '*.html' -exec grep about.html {} \;

% find . ! -path '*oracle8.0.5*' ! -path '*developer*' \
    ! -path '*matlab*' -name '*.html' -exec grep about.html {} \;

% find . ! -path '*oracle8.0.5*' -name '*.html' -type f -exec grep -iH
    'www-staff' {} \;

% find . ! -path '*oracle8.0.5*' -name '*.html' -type f -exec grep -il
    'www-staff' {} \; | xargs -n 1 perl -pi -e 's/www-staff/staff/'

How to get more information about a command ``foo''''.

% foo --help    - ``--help'' is a parameter built into most commands.
% foo -h        - ``-h'' is a parameter built into most commands.
% foo -?        - ``-?'' is a parameter built into most commands.

% whatis foo    - Display a one-line summary about a keyword.

% apropos foo   - List manpages (to locate commands) by keyword lookup.
                  (This is the same ``man -k K'' command).

% info foo      - A hypertext system for browsing documentation.

% locate foo    - Lists files in databases that match a pattern.

% locate '*foo*'
                - Lists files in databases that match a pattern.
                  ("locate" assumes that the command updatedb has been ran
                  by the system administrator).

% locate '*foo*' | less
                - Same as above, but the result, which is often big, is sent
                  into the ``less'' text pager.  Press ``q'' to quit it.

Check directories /usr/doc/foo and /usr/lib/foo. Do a ``man'' on info(1), apropos(1), locate(1), find(1), and undocumented(2).


Job control (user process management).

% ps auxww      - Report process status of all users on the system, including
                  associated user names, excluding associated controlling
                  terminal and using wide ouput.

% jobs -l       - List jobs (subprocesses) currently running, and
                  their associated job number and process IDs (PIDs).

% bg J          - Place job J in the background, as if it had been started
                  with &.  If job J is not specified, the shell's notion of
                  the current job is used.
% fg J          - Place job J in the foreground, and make it the current job.
                  If job J is not present, the shell's notion of the current
                  job is used.
% kill J        - Kill job J; where J is specified with a job number, or the
                  PID.
% [CTRL-Z]      - Suspend the current job, placing it in the background.

% %n            - Bring up the active job having job number n.

Networking and Communications (remote opertations).

News.

Usnet is a collection of thousands of computers worldwide that exchange files called news articles. There are thousands of interactive discussion groups, known as electronic bulletin boards that talk about literally everything. The news server is ``news.RISQ.QC.ca'' and SOCS courses are listed under ``mcgill.socs.courses.cs*''

FTP.

File Transfer Program enables the transfer of files between networked computers with TCP/IP. For information about FTP at SOCS, please consult: http://socsinfo.cs.mcgill.ca/wiki/FTP

SSH/SCP.

For information about SSH at SOCS, please consult: http://socsinfo.cs.mcgill.ca/wiki/SSH

% scp           - Secure copy.  Uses ssh authentication and security to copy
                  files remotely.

% scp U@H:F U2@H2:F2
                - Copy file F from host H (as user U) to file F2 on host H2,
                  (as user U2).

% scp index.html joe@mimi:~/public_html/.

Who and w.

% who           - Show who is currently logged on the system.
% w             - Display information about users currently logged on the
                  system.

Write.

% write U       - Send a message to user U.

Talk.

% talk U        - Talk to user U.  Talk is a visual communication program which
                  copies lines from your terminal to that of another user.
                  Press Ctrl-C to terminate the session.

Finger.

% finger U@H    - Display information about user U on host H, including
                  real name, terminal name, idle time, login time, etc.

E-Mail.

% mail U        - Send an email to user U from the command line.  User U can
                  be specified by a login name, or by an email address.
                  Mail will prompt you for the 'subject'.  Everything typed
                  on subsequent lines will compose the body of the message.
                  The message is sent when a new line contains a single
                  character ".".

Intermediate commands and options.

Intermediate commands are ones that a beginner is not concerned with, as not knowing them does not inhibit getting work done.

% head F        - Display the first few lines (default is 10) of file F.
% head -N F     - Display the first N lines of file F.

% tail F        - Display the last few lines (default is 10) of file F.
% tail -N F     - Display the last N lines of file F.
% tail -f F     - Display the growth of input-file F.  This command may be
                  used to monitor the growth of a file that is being written
                  by some other process.
% tail -r F     - Print file F in reverse order.

% tac F         - Print file F in reverse order.

% wc F          - Print the number of bytes, words, and lines in file F.

% diff F1 F2    - Display line-by-line differences between pairs of text files
                  (file F1 and file F2).
% cmp F1 F2     - Compare file F1 and file F2.  Print the byte and line
                  numbers at which the difference occurred.

% sort F        - Sort lines of file F, and print the result to the standard
                  output.
% uniq F        - Display unique lines of sorted file F.

% tty           - Print the filename of the terminal connected to standard
                  input.

% uname -a      - Print system information, including machine (hardware) type,
                  hostname, operating system release, name and version.
                  Solaris and Linux versions are overlapping, but somewhat
                  different.

% cat           - Writes standard input to standard output.  The output can be
                  redirected to a file.
% cat -         - Same as cat.
% cat F         - View the contents of file F.  (Write the contents of file F
                  to the standard output.)
% cat -n F      - Display the contents of file F, preceding each line output
                  with its line number.
% cat F...      - Write the contents of each given file to standard output.
% cat F1 F2 F3 > F4
                - Concatenate files F1, F2, and F3, and redirect the
                  concatenation to file F4.  Without the redirection the
                  concatenation would be printed on the standard output.

% gzip F        - Compress the file F, and append to it the extension gz.
% gzip -d F.gz
                - Expand, or restore the compressed file F to it's original
                  form.
% gunzip F.gz   - Same as gzip -d.
% zcat F.gz     - Same as gunzip.

% tar -cvf F.tar D
                - Create an archive of directory D in file F.tar.
% tar -cvf F.tar F...
                - Create an archive of files F... in file F.tar.
% tar -cvvf F.tar D
                - Same as tar -cvf, except more verbose.

% tar -xvf F.tar
                - Extract the archived file F.tar.
% tar -xvvf F.tar
                - Same as tar -xvf, except more verbose.

% tar -zxvf F.tar.gz
                - Extract the tarred and gzipped file F.tar.gz.

% tar -tzf F.tar.gz
                - List the contents of the zipped archive F.tar.gz, without
                  extracting.

% tar -tIf F.tar.bz2
                - List the contents of the bz2 tarball.
% tar -xvIf F.tar.bz2
                - Extract the tarred and bz2 zipped archive.


% du D          - Summarize disk space usage of directory D, and its
                  subdirectories.  The space is measured in 1k blocks.


% find D -name P
                - Recursively find file(s) having the pattern P, starting in
                  directory D.  Pattern P can simply be a string, or a regular
                  expression.
% find D -type T
                - Recursively find file(s) being of type T.  Check the "find"
                  manpage to see the acceptable types T.  If the expression
                  contains the option -print the result of the search for
                  which the expression is true. (The -print option may be used
                  with any "find" expression).

% psnup -4 F1.ps F2.ps
                - Puts four (4) logical pages of input Post Script file F1.ps
                  onto each physical sheet of paper (output Post Script
                  file F2.ps).

% psnup -p letter -c -l -4 -m .1in F1.ps F2.ps

% history       - Display the list of all commands that were executed
                  (aka command history) during the time of the current login
                  session.
% !$            - Run the last command entered that began with the partial
                  string S.
% !!            - Execute the most recently executed command.
% !N            - Execute the Nth command in the history list.
% C !$          - Execute command C with the parameter that was the first
                  parameter to the most recently executed command.

% cal           - Display a calendar.

% clear         - Clear the terminal screen.  It is easier to clear the screen
                  using the key combination CTRL-L.

% mkdir -p D    - Make the directory D, including all mentioned parent
                  directories, if they don't yet exist.  For example, if D
                  is ``code/perl/tests'', then mkdir would create the directory
		  ``code'' (if it didn't already exist), within which it would
                  create the directory ``perl'' (if it didn't already exist),
                  within which it would create the directory ``tests'' (if it
                  didn't already exist).

Common file types.

Very common file types are Post Script (having extension .ps), Portable Document Format (having extension .pdf), DeVice Independent (having extension .dvi), and plain text, aka ASCII. There are several conversion tools for converting between these file types. One would want to convert particular files to the Post Script format in order to print a given file to a printer. There are several programs and methods of viewing these file formats as well.

% ghostview F.ps    - View PS file F.ps.
% acroread F.pdf    - View PDF file F.pdf.
% evince F.pdf      - View PDF file F.pdf.
% a2ps F.txt F.ps
                - Convert plain text (ascii) file F.txt, to a PS format file
                  called F.ps.

% a2ps -nH -nu -nL F.txt F.ps
                - Convert plain text (ascii) file F.txt, to a PS format file
                  called F.ps, without page headers, without filename at
                  bottom of page, and without login ID at top of page.

% dvips F.dvi -o F.ps
                - Convert DVI format file F.dvi, to a PS format file
                  called F.ps.

% pdf2ps F.pdf F.ps
                - Convert PDF format file F.pdf, to a PS format file
                  called F.ps.

Mounting filesystems.

Normally, only the superuser can mount file systems; however when the file /etc/fstab contains the "user" option on a line, then anybody can mount the corresponding system.

The following are a few lines extracted from an /etc/fstab file on a host named troy:

/dev/fd0        /disks/dosa     msdos           user,noauto,nosuid      0 0
/dev/fd0        /disks/win95a   vfat            user,noauto,nosuid      0 0
/dev/fd0        /disks/floppy   minix           user,noauto,nosuid      0 0
/dev/fd0        /disks/ext2     ext2            user,noauto,nosuid      0 0
/dev/sda4       /disks/zipwin95 vfat            user,noauto,nosuid,sync,rw 0 0
/dev/sda4       /disks/ziplinux ext2            user,noauto,nosuid,sync,rw 0 0
/dev/sda4       /disks/zipdos   msdos           user,noauto,nosuid,sync,rw 0 0
/dev/cdrom      /disks/cdrom    iso9660         user,noauto,nosuid      0 0

Because these lines contain the option "user", they can be mounted by anybody. For instance, if we wanted to mount the linux file system ext2 found on a zip disk on troy, we would type the following command:

% mount /disks/ziplinux
                - Mount the ext2 file system on /dev/sda4.

% mount Dev     - Mount a file system; attach the file system found on some
                  device to the big file tree.
% mount Dir     - Mount a file system; attach the file system found on some
                  device to the big file tree.

Before ejecting the zip disk, it must be unmounted. The command that accomplishes this task is 'umount'.

% umount /disks/zip
                - Unmount

% umount Dev    - Unmount a file system; detach the file system associated
                  with device Dev, by giving the special device Dev on which
                  it lives.
% umount Dir    - Unmount a file system; detach the file system associated
                  with directory Dir, by giving the directory Dir where it has
                  been mounted.

% df            - List all currently mounted filesystems and their disk space
                  usage.
% mount         - List all currently mounted filesystems with their types, and
                  options.  This information is retrieved from the file
                  /etc/mtab.

DOS floopy disk access.

A utility called mtools provides access to DOS disks in Unix. The mtools commands and corresponding command line parameters are described on the mtools(1) manpage. The following is a quick summary of usefull commands.

% mcd D         - Change the mtools working directory on the MS-DOS disk.

% mcopy F1 F2   - Copy MS-DOS file F1 to file F2 in the current mtools
                  directory.

% mcopy F... D  - Copy the file(s) F... to the mtools directory D.
% mcopy F       - Copy file MS-DOS file F into the current directory.

% mdel F...     - Delete an MS-DOS file(s) F.
% mdeltree D...
                - Remove a directory and all the files and subdirectories it
                  contains.

% mdir          - Display the current mtools directory content.
% mdir D        - Display the the content of the MS-DOS directory D.

% mmd D         - Make an MS-DOS directory D.

% mmount D A    - Mount an MS-DOS disk on drive D, with mount arguments A.

% mmove F1 F2   - Move or rename an MS-DOS file F1 to file F2.

% mrd D...      - Removes an MS-DOS directory D.

% mren F1 F2    - Renames MS-DOS file F1 to file F2.

% mtype F       - Display the specified MS-DOS file F on the screen.

% mformat       - Format (add a minimal MS-DOS filesystem (boot sector, FAT,
                  and root directory) to a floppy disk.  You may create a
                  Linux second extended file system with the command mke2fs.


This document was generated using AFT v5.0792