| CONTENTS |
A shell script need be no more than a command line saved in a file. For example, let's assume that you'd like a compact list of all the users who are currently logged in on the system.
A command like this might do the trick:
% who | cut -c1-8 | sort -u | pr -l1 -8 -w78 -t
A list of logged-in users should come out in columns, looking something like this:
abraham appleton biscuit charlie charlott fizzie howard howie hstern jerry kosmo linda ocshner peterson root ross sutton yuppie
We used four Unix commands joined with pipes:
who (Section 2.8) gives a list of all users.
cut -c1-8 (Section 21.14) outputs columns 1-8 of the who output the usernames.
sort -u (Section 22.6) puts names in order and takes out names of users who are logged on more than once.
pr -l1 -8 -w78 -t (Section 21.15, Section 45.6) takes the list of usernames, one per line, and makes it into 8 columns on 78-character-wide lines. (The -l1 is the lowercase letter L followed by the digit 1.)
If you wanted to do this frequently, wouldn't it be better if all you had to do was type something like:
% loggedin
to get the same result? Here's how:
Start a text editor on a new file named loggedin.
If your system supports the special #! notation (Section 36.2) (and it probably does), the first line of the script file should be:
#!/bin/sh
Otherwise, leave the first line blank. (When the first line of a script is blank, most shells will start a Bourne shell to read it. Section 36.2 has more information.)
I think that the second line of a shell script should always be a comment to explain what the script does. (Use more than one line, if you want.) A comment starts with a hash mark (#); all characters after it on the line are ignored. Oh, and try to make sure there's a bit of whitespace between the comment character and the actual comment; that's a pet peeve of mine:
# loggedin - list logged-in users, once per user, in 8 columns
Put this on the third line, just like you did on the command line:
who | cut -c1-8 | sort -u | pr -l1 -8 -w78 -t
Save the file and leave the editor. You've just written a shell script.
Next, you need to make the shell script executable. The chmod (Section 50.5) (change mode) command is used to change permissions on a file. The plus sign followed by an x (+x) makes the file executable:
% chmod +x loggedin
If your login shell (Section 3.4) is csh or tcsh, you'll need to reset its command search table. To do that, type:
rehash Section 27.6
% rehash
Finally, try the script. Just type its name and it should run:
% loggedin
If that doesn't run, your current directory may not be in your shell's command search path (Section 35.6, Section 35.7). In that case, try this:
% ./loggedin
If it still doesn't work, and you started the first line of your script with #!, be sure that the Bourne shell's pathname on that line (like /bin/sh) is correct. Another common error is to swap the # and !, so check that, too. You should get an error like this, if that is the problem, although the script may itself run as well, depending on your system:
!#/bin/sh: No such file or directory
If you want to run the script from somewhere other than the current directory, or if you want other programs and scripts you write to be able to use it, you need to put it in a directory that's in your search path and/or change your search path (Section 27.6). If you're the only person who plans to use the script, you should put it in your personal bin directory (Section 7.4). Otherwise, you might ask your system administrator if there's a systemwide directory for local commands.
JP
One of the great things about Unix is that it's made up of individual utilities, "building blocks" like cat and grep, that you run from a shell prompt. Using pipes, redirection, filters, and so on, you can combine those utilities to do an incredible number of things. Shell programming lets you take the same commands you'd type at a shell prompt and put them into a file you can run by just typing its name. You can make new programs that combine Unix programs (and other shell scripts) in your own way to do exactly what you need. If you don't like the way a program works, you can write a shell script to do just what you want.
Because many Unix users use the shell every day, they don't need to learn a whole new language for programming, just some tips and techniques. In fact, this chapter covers a lot of programming techniques that you'll want to use even when you aren't programming. For example, loops and tests are handy on the command line.
(This series of articles does assume that you've written programs in some language before or are generally familiar with programming concepts. If you haven't and aren't, you might start with a more comprehensive shell programming book.)
Summary BoxUnix has plenty of other scripting languages Perl, Python, and Tcl/Tk are some of the best known. So when should you write a script with the shell and when shouldn't you? That's a personal choice; as you learn more languages and their strengths and weaknesses, you're better able to choose the best one for a situation. My rule of thumb is something like this. I write a shell script if:
On the other hand, maybe your script needs lots of pipes ( | ) (Section 1.5) or temporary files, or you have out-of-band data that you have to keep passing in to each Unix utility (maybe because you can't shoehorn multiple types of data into a single chain of pipelines between utilities). In that case, you'll be happier with a scripting language that doesn't depend on Unix utilities and pipes. Some of the topics you need to learn about as a beginning shell programmer have already been covered in other chapters. Here are the articles you'll probably want to read, in an order that makes sense if you're looking for something of a tutorial:
Then, once you've had your refresher, come on back and read the following articles:
|
This chapter discusses only Bourne shell programming. We don't cover many features from more advanced Bourne-type shells, like bash and zsh, because those can make your shell scripts nonportable; we stick to concepts that should work almost anywhere. Also, in most cases, the C shell isn't great for shell programming.
A note about command versions: unfortunately, the same commands on different versions of Unix can have different options. Some Bourne shells are a little different from others. For instance, some test (Section 35.26) commands have a -x option to test for an executable file; others don't. Some echo commands use a -n option to mean "no newline at the end of this string"; others have you put \c at the end of the string. And so on. Where there are differences, these articles generally use the commands in original Berkeley Unix from the 1980s. If a command doesn't seem to work on your system, check its online manual page or the sh manual page.
JP
Many Unix utilities, including the shell, need information about you and what you're doing in order to do a reasonable job.
What kinds of information? Well, to start with, a lot of programs (particularly editors) need to know what kind of terminal you're using. The shell needs to know where any commands you want to use are likely to be found. Lots of Unix programs (like mail programs) include a command to start an editor as a subprocess; they like to know your favorite editor. And so on.
Of course, one could always write programs that made you put all this information on the command line. For example, you might have to type commands like:
% mail -editor vi -term aardvark48 -favoritecolor blue_no_red
But your favorite editor probably doesn't change every day. (Nor will your favorite color.) The terminal you use may change frequently, but it certainly won't change from the time you log in until the time you log out. And you certainly wouldn't want to type something like this whenever you want to send mail.
Rather than forcing you to type this information with every command, Unix uses environment variables to store information you'd rather not worry about. For example, the TERM (Section 5.2) environment variable tells programs what kind of terminal you're using. Any programs that care about your terminal type know (or ought to know) that they can read this variable, find out your terminal type, and act accordingly.
Similarly, the directories that store the commands you want to execute are listed in the PATH (Section 35.6) variable. When you type a command, your shell looks through each directory in your PATH variable to find that command. Presumably, Unix wouldn't need a PATH variable if all commands were located in the same directory, but you'll soon be writing your own commands (if you aren't already), and storing them in your own "private" command directories (Section 7.4), and you'll need to tell the shell how to find them (Section 27.6).
Environment variables are managed by your shell. The difference between environment variables and regular shell variables (Section 35.9) is that a shell variable is local to a particular instance of the shell (such as a shell script), while environment variables are "inherited" by any program you start, including another shell (Section 24.4). That is, the new process gets its own copy of these variables, which it can read, modify, and pass on in turn to its own children. In fact, every Unix process (not just the shell) passes its environment variables to its child processes.
You can set environment variables with a command like this:
% setenv NAME value C-type shells $ NAME=value ; export NAME all Bourne-type shells $ export NAME=value newer Bourne-type shells
There's nothing particularly special about the NAME; you can create environment variables with any names you want. Of course, these don't necessarily do anything for you; variables like PATH and TERM are important because lots of programs have "agreed" (Section 35.5) that these names are important. But if you want to create an environment variable that holds the name of your lover, that's your business:
% setenv LOVER Judy
If you're so inclined, you could write a program called valentine that reads the LOVER environment variable and generates an appropriate message. If you like short-term relationships or tend to forget names, this might even be convenient!
By convention, the names of environment variables use all uppercase letters. There's nothing to enforce this convention if you're making your own names, you can use any capitalization you please. But there's no advantage to violating the convention, either. The environment variables used by standard Unix programs all have uppercase names. Making shell variable names lowercase so it's easy to tell the difference is helpful.
If you want the C shell to forget that an environment variable ever existed, use the command unsetenv NAME. The tcsh understands filename wildcard (Section 1.13)-type expressions for instance, unsetenv VAR* would unset all environment variables whose names start with VAR. Most Bourne-type shells, but not all, have a similar command, unset NAME, but it doesn't understand wildcards like the tcsh version. The bash version accepts multiple names as arguments, however, and can also unset functions with the -f option.
Go to http://examples.oreilly.com/upt3 for more information on: printenv, env
If you want to list all of your environment variables, use printenv or env. The printenv command also lets you ask about a particular variable. Here's a typical report:
% printenv EDITOR EDITOR=/usr/local/bin/emacs % printenv HOME=/home/los/mikel SHELL=/bin/csh TERM=sun USER=mikel PATH=/usr/local/bin:/usr/ucb:/bin:/usr/bin:.:/home/los/mikel/bin LOGNAME=mikel PWD=/home/los/mikel/power/articles PRINTER=ps EDITOR=/usr/local/bin/emacs
The set (Section 35.9) command provides a similar listing of shell variables and functions (in newer Bourne-like shells such as bash).
You can also use the echo command to show the value of a particular variable by preceding the variable name with a dollar sign (which tells the shell to substitute the value of the variable):
% echo $TERM xterm
Or and this is particularly useful when you want a shell or environment variable's value interpolated into a line you can surround the variable name with curly brackets:
% echo ${TERM} vt100 % echo ${TERM}-like vt100-like
ML
No, this is not about the psychology of computing. It's just a reminder of one important point.
In the environment variable overview (Section 35.3) we said that each process gets its own copy of its parent's environment variables. We chose those words carefully, and if you think about them, you won't make one common mistake.
Sooner or later, almost everyone writes a shell script that gathers some information, sets a few environment variables, and quits. The writer then wonders why there's no trace of the "new" environment variables to be found. The problem is simple. A Unix process (Section 24.3) cannot change its parent's environment; a Unix process gets its own copy of the parent's environment, and any changes it makes it keeps to itself. A process can make changes and pass them to its children, but there's no way of going in reverse.
(You can't teach an old dog new tricks.)
[This is important in window systems, too. Environment variables set in one window (more exactly, in one process) probably won't affect any process running in any other window. To affect all windows, set the environment variable before you start the window system. For instance, if you log in and then type startx from a shell prompt to start X, you can set environment variables from that prompt or from that shell's setup files (Section 3.3). JP]
ML
We've said that environment variables are used to store information that you'd rather not worry about, and that there are a number of standard environment variables that many Unix programs use. These are often called "predefined" environment variables not because their values are predefined, but because their names and uses are predefined. Here are some important ones:
contains your command search path (Section 27.6). This is a list of directories in which the shell looks to find commands. It's usually set in one of your shell setup files (Section 3.3).
can be loaded with the name of your favorite editor. They're usually set in one of your shell setup files. Some programs distinguish between EDITOR (usually set to a line editor (Section 20.1) such as ed) and VISUAL (set to a full-screen editor like vi). Many people don't follow that convention; they set both to the same editor. (The Korn shell checks VISUAL and EDITOR, in that order, to determine your command editing mode (Section 30.14).)
can be loaded with the name of your default printer. This is quite useful at a site with many printers you don't need to tell lpr or lp (Section 45.2) which printer to use. (PRINTER works on systems that print with lpr , and LPDEST is for lp.) This variable is usually set in one of your shell setup files.
may contain the absolute pathname of your current directory. It's set automatically by the cd command in some Unix shells. PWD may be fooled by cding through symbolic links.
contains the absolute pathname of your home directory. It's set automatically when you log in.
contains the absolute pathname of your login shell. It's set automatically whenever you log in.
contains your username. It's set automatically when you log in.
contains the name of your terminal type in the termcap or terminfo database. It's usually set in a shell setup file. On Darwin, in the Terminal program, the TERM_PROGRAM variable is also set.
is an environment variable that can be loaded with the complete termcap database entry for the terminal you are using. This may make some programs start up more quickly, but it's not necessary. It's set (under some conditions) by the tset command, which is usually run in your shell setup file.
contains the name of an initialization file to be executed whenever a new Korn shell is started. (See Section 3.3.) Korn shell only.
contains the name of an initialization file to be executed whenever a new bash shell is started. (See Section 3.3.) bash only. Often set to .bashrc by default.
can be set to the name of your favorite page-by-page screen display program like less (Section 12.3) or more. (Programs like man (Section 2.1) use PAGER to determine which paging program to use if their output is longer than a single screen.)
contains the primary prompt (i.e., interactive command prompt) for Bourne-type shells. You also can set it in a particular shell, as a shell variable with the same name, but it won't be passed to subshells automatically. (The C shell doesn't store the prompt in an environment variable. It uses a shell variable called prompt because the .cshrc file (Section 3.3) is read to set up each instance of the shell. See Section 4.4.)
contains the secondary prompt (used within compound commands like while and for) for Bourne shells. Some Bourne-type shells also use PS3 and PS4. As with PS1, these don't have to be stored in the environment.
if your man (Section 2.1) command supports it, is a colon-separated list of directories to search for manual pages.
contains the time zone. This is the name of a file in the zoneinfo directory that provides time zone information for your locality. It is read by commands such as date.
is used by the X Window System (Section 1.22) to identify the display server (keyboard and screen handling program) that will be used for input and output by X applications. It may be set by ssh when you log into a remote system, as well.
lets you choose a setup filename for the Readline library instead of the default $HOME/.inputrc.
lists the colors used by the color ls command (Section 8.6).
Because Bourne-type shells don't make as strict a distinction between environment variables and shell variables as the C shell does, we've included a few things here that might not be on other people's lists.
But we haven't even tried to include everything. Here are two good ways to see what's there. One is to look at the end of a command's manual page (Section 2.1) in the ENVIRONMENT section (if there is one). Another is to list your current environment variables (with env or printenv (Section 35.3)) and make some guesses from the names and corresponding values.
We may have implied that environment variables are relatively constant (like your favorite editor). That's not true. For example, in a windowing environment, the current length of your window might be kept in an environment variable. That can change as often as you resize your window. What is true (fortunately) is exactly what we've said: environment variables store information that you'd rather not have to worry about.
ML, JP, and SJC
Of all the environment variables, the PATH and TERM variables are the most important. The others are often great conveniences, but PATH and TERM can make your life miserable if they get screwed up.
The PATH variable is just a list of directories separated by colon (:) characters. The shell searches through these directories in order whenever it needs to find a command. So, if you want to execute commands in /bin, /usr/bin, /usr/local, the current directory, and your personal bin directory, you would put a line like the one below in your .login file. An empty entry (: as the first or last character, or :: in the middle) means "the current directory."
$HOME/bin Section 7.4
setenv PATH /bin:/usr/bin:/usr/local::$HOME/bin
Section 27.6 explains more about setting the path.
The most common problem with PATH is that, somehow, it gets deleted. This usually happens if you try to change PATH and do so incorrectly. When PATH is deleted, your shell can find only its built-in commands (Section 1.9) and commands for which you give the complete pathname. Here's a demonstration:
% setenv PATH Set PATH to null accidentally % ls ls: Command not found.
Needless to say, this can be very frustrating especially if you can't figure out what's going on. There are a couple of easy fixes. The easiest is just to log out and log back in again. (logout is a built-in C shell command, so you won't have trouble finding it. If you get an error message like "Not login shell," try exit instead.) Another fix is to read (Section 35.29) whichever initialization file defined your PATH variable, usually .login for C shell users or .profile or .bash_profile for Bourne or bash shell users, respectively:
% source ~/.login $ . $HOME/.profile bash$ . $HOME/.bash_profile
This will almost certainly give you some of your path back; the problem is that a lot of initialization files merely add a few "private" directories to a system-wide default path. In this case, just execute the system-wide initialization files first (if your system has them). Their pathnames vary:
+$ source /etc/profile $ source /etc/profile.d/*.sh $ source ~/.login bash$ source ~/.bash_profile
Your best bet, if you're unfamiliar with the quirks of your system and how it sets up your shell, is to simply log out and log back in again. Some newer Linux systems, for example, use /etc/profile for bash setup, inheriting the hardwired defaults set by the login command, and then go on to read shell-specific files (often in /etc/profile.d, for example). tcsh and csh are configured using the /etc/csh.login, /etc/csh.cshrc, and other files in /etc/profile.d in similar fashion to bash. This allows package managers to install package specific initialization without modifying (and potentially corrupting) the system's default initialization.
The other common PATH problem is that users sometimes can't find the commands they want. This happens most often when someone writes a new shell script with the same name as a standard Unix command say, true. He or she tries to execute it and can't; in fact, all that happens is:
% true %
After staring at the script for a long time, the user sometimes gets the right idea: the script is fine; it's the path that's wrong. The PATH variable will look something like this:
% printenv PATH /bin:/usr/local/bin:/usr/bin:/sbin::/home/schampeo/bin
The shell searches the PATH in order; therefore, it finds the system's standard true command before seeing the new one. The new command never gets a chance. You could fix this problem by putting the current directory and $HOME/bin at the head of the search path, in which case commands in the current directory and your private bin directory will override the standard commands. However, that's not recommended; it's a well-known security hole.
So what is recommended? Nothing much, except that if you write shell scripts or other programs, give them names that are different from the standard Unix utilities (Section 35.27). If you really need an overlapping name, you can use a relative pathname (Section 1.16) to specify "the program called true in the current directory":
% ./true
You can search your PATH for a command with which (Section 2.6), findcmd, and whereiz. Section 35.7 explains the pair of path variables in csh and zsh.
ML and SJC
For csh and zsh, it's slightly incorrect to say that PATH contains the search list for commands. It's a bit more complicated. The PATH environment variable is used to set the path shell variable; that is, whenever you use setenv PATH (Section 35.6) in csh or export PATH in zsh, the shell modifies path accordingly. For example:
setenv PATH /bin:/usr/bin:/usr/local::$HOME/bin csh export PATH=/bin:/usr/bin:/usr/local::$HOME/bin zsh
In PATH, an empty entry (::) stands for the current directory. The shells' path shell variable (Section 35.9) is the actual search list. Its syntax is slightly different; the list of directories is enclosed in parentheses ([XREF: UPT-ART-0508]), and the directories are separated by spaces. For example:
set path=(/bin /usr/bin /usr/local . ~/bin) csh path=(/bin /usr/bin /usr/local . ~/bin) zsh
If you set the path shell variable, the shell will automatically set the PATH environment variable. You don't need to set both. Many people set the shell variable instead of the environment variable.
ML
The most important environment variable for X Window System clients is DISPLAY. When a user logs in at an X terminal, the DISPLAY environment variable in each xterm window is set to her X terminal's hostname followed by :0.0.
ruby:joan % echo $DISPLAY ncd15.ora.com:0.0
When the same user logs in at the console of the workstation named sapphire that's running X, the DISPLAY environment variable is defined as just :0.0:[1]
sapphire:joan % echo $DISPLAY :0.0
The DISPLAY environment variable is used by all X clients to determine what X server to display on. Since any X client can connect to any X server that allows it, all X clients need to know what display to connect to upon startup. If DISPLAY is not properly set, the client cannot execute:
sapphire:joan % setenv DISPLAY foo:0 sapphire:joan % xterm xterm Xt error: Can't open display:
You can override the value of DISPLAY by using the -display command-line option. For example:
sapphire:joan % xterm -display sapphire:0.0 &
The first part of the display name (up to and including the colon) identifies the type of connection to use and the host that the server is running on. The second part (in most cases, the string 0.0) identifies a server number and an optional screen number. In most cases, the server and screen numbers will both be 0. You can omit the screen number name if the default (screen 0) is correct.
Note that we used both :0.0 and sapphire:0.0 to access the local console display of the workstation named sapphire. Although both these names will work, they imply different ways of connecting to the X server.
The : character without an initial hostname specifies that the client should connect using UNIX domain sockets (IPC).
Since processes can communicate via IPC only if they are running on the same host, you can use a leading colon or the unix keyword in a display name only if both the client and server are running on the same host that is, for local clients displaying to the local console display of a workstation.
Using the hostname followed by a colon (e.g., sapphire:) specifies that the client should connect using Internet domain sockets (TCP/IP). You can use TCP/IP connections for displaying clients on any X server on the TCP/IP network, as long as the client has permission to access that server.
Note that like all other environment variables set in your shell environment, the DISPLAY environment variable will propagate (Section 35.3) to all processes you start from that shell.
When you run clients from remote machines, some additional problems with the DISPLAY environment variable need to be addressed. See Section 6.10 for more information on running remote clients.
LM and EP
Shell variables are really just the "general case" of environment variables (Section 35.3). If you're a programmer, remember that a Unix shell really runs an interpreted programming language. Shell variables belong to the shell; you can set them, print them, and work with them much as you can in a C program (or a FORTRAN program or a BASIC program). If you're not a programmer, just remember that shell variables are pigeonholes that store information for you or your shell to use.
If you've read the articles on environment variables, you realize that we defined them in exactly the same way. How are shell variables different from environment variables? Whenever you start a new shell or a Unix program, it inherits all of its parent's environment variables. However, it does not inherit any shell variables; it starts with a clean slate (except, possibly, variables in some shell setup files (Section 3.3)). If you're a programmer, you can think of environment variables as "global" variables, while shell variables are "local" variables. By convention, shell variables have lowercase names.
Just as some programs use certain environment variables, the shell expects to use certain shell variables. For example, the C shell uses the history (Section 30.1) variable to determine how many of your previous commands to remember; if the noclobber (Section 43.6) variable is defined, the C shell prevents you from damaging files by making mistakes with standard output. Most users insert code into their .cshrc or .tcshrc (Section 3.3) files to define these important variables appropriately. Alternatively, they split them up into context-specific files and then read them into their environment (Section 35.29) as needed.
To set a shell variable, use one of these commands:
% set name = value C shell $ name = value other shells
As a special case, if you omit value, the shell variable is set to a "null" value. For example, the following commands are valid:
% set name C shell $ name = other shells
This is important: giving a variable a null value is not the same as deleting the value. Some programs look at variables to see whether or not they exist; they don't care what the actual value is, and an empty value is as good as anything else.
Most newer shells but not the original C and Bourne shells let you prevent accidental changes in a variable by marking it read-only after you've stored its value:
% set -r name tcsh $ readonly name other shells
(In zsh, you can mark a variable read-only as you initialize it: readonly name=value.) If you want to make the shell forget that a variable ever existed, use the unset command. Note that, in general, you can't unset a read-only variable! Also, older Bourne shells don't have a command like unset:
% unset name C shell $ unset name others except old Bourne shell
If you want to list all of your environment variables, use the command printenv or env (Section 35.3).[2] If you want to list all of your Bourne or C shell variables, just type set. Here's a typical report in the C shell:
% set argv ( ) cwd /home/los/mikel/power/articles history 40 home /home/los/mikel noclobber path (/home/los/mikel/bin /usr/local/bin /usr/ucb /bin /usr/bin .) prompt los% shell /bin/csh status 0 term sun user mikel
If you want to print the value of an individual variable, give the command:
% echo "$ variablename "
(While the example above gives a C shell prompt, this command works in all Unix shells.) The quotes aren't necessary for something as simple as an echo statement, but if you want the value captured, for example, so that you can apply it to another variable, they are recommended.
Whenever you need the value of a shell variable not just with echo you need to put a dollar sign ($) in front of the name. Don't use the dollar sign when you're assigning a new value to a shell variable. You can also stick curly braces ({}) around the name if you want to (e.g., ${name}); when you're writing shell programs, this can often make your code much clearer. Curly braces are mostly used when you need to separate the variable name from what comes after it.
But that's getting us out of the range of interactive variable use and into shell programming (Section 35.2).
ML and SJC
Each time you type a command line at a shell prompt, you can see what happens and decide what command to run next. But a shell script needs to make decisions like that itself. A case statement helps the script make decisions. A case statement compares a string (usually taken from a shell or environment variable (Section 35.9, Section 35.3)) to one or more patterns. The patterns can be simple strings (words, digits, etc.) or they can be case wildcard expressions (Section 35.11). When the case statement finds a pattern that matches the string, it executes one or more commands.
Here's an example that tests your TERM (Section 5.2) environment variable. If you're using a vt100 or tk4023 terminal, it runs a command to send some characters to your terminal. If you aren't on either of those, it prints an error and quits:
exit Section 35.16
+case "$TERM" in vt100 echo 'ea[w' | tr 'eaw' '\033\001\027' ;; tk4023) echo "*[p23" ;; *) # Not a VT100 or tk4023. Print error message: echo "progname: quitting: you aren't on a VT100 or tk4023." 1>&2 exit ;; esac
Here are more details about how this works. The statement compares the string between the words case and in to the strings at the left-hand edge of the lines ending with a ) (right parenthesis) character. If it matches the first case (in this example, if it's the vt100), the command up to the ;; is executed. The ;; means "jump to the esac" (esac is "case" spelled backwards). You can put as many commands as you want before each ;;, but put each command on a separate line (or separate commands on a line with semicolons (Section 28.16)).
If the first pattern doesn't match, the shell tries the next case here, tk4023. As above, a match runs the command and jumps to the esac. No match? The next pattern is the wildcard *. It matches any answer other than vt100 or tk4023 (such as xterm or an empty string).
You can use as many patterns as you want to. The first one that matches is used. It's okay if none of them match. The style doesn't matter much. Pick one that's readable and be consistent.
JP
A case statement (Section 35.10) is good at string pattern matching. Its "wildcard" pattern-matching metacharacters work like the filename wildcards (Section 1.13) in the shell, with a few twists. Here are some examples:
Matches a string with exactly one character like a, 3, !, and so on.
Matches a string with one or more characters (a nonempty string).
Matches y, Y or yes, YES, YeS, etc. The | means "or."
Matches a file pathname, like /xxx/yyy/somedir/file2, that starts with a slash, contains at least one more slash, and ends with a digit.
Matches the pattern What now?. The quotes (Section 27.12) tell the shell to treat the string literally: not to break it at the space and not to treat the ? as a wildcard.
Matches the contents of the msgs variable. The double quotes let the shell substitute the variable's value; the quotes also protect spaces and other special characters from the shell. For example, if msgs contains first next, this would match the same string, first next.
To clarify: in bash, for example, the case statement uses the same pathname expansion rules it uses elsewhere in the shell, to determine how to expand the value. In other shells, such as ksh, there are minor differences (such as a relaxation of special treatment for . and / characters). See the manual page for your shell if you have any questions or concerns about what rules your shell will follow.
JP and SJC
When a Unix process (command) runs, it can return a numeric status value to the parent process that called (started) it. The status can tell the calling process whether the command succeeded or failed. Many (but not all) Unix commands return a status of zero if everything was okay and nonzero (1, 2, etc.) if something went wrong. A few commands, such as grep and diff, return a different nonzero status for different kinds of problems; see your online manual pages (or just experiment!) to find out.
The Bourne shell puts the exit status of the previous command in the question mark (?) variable. You can get its value by preceding it with a dollar sign ($), just like any other shell variable. For example, when cp copies a file, it sets the status to 0. If something goes wrong, cp sets the status to 1:
$ cp afile /tmp $ echo $? 0 $ cp afiel /tmp cp: afiel: No such file or directory $ echo $? 1
In the C shell, use the status variable instead (tcsh supports both):
% cp afiel /tmp cp: afiel: No such file or directory % echo $status 1 tcsh> cp afiel /tmp cp: afiel: No such file or directory tcsh> echo $status 1
Of course, you usually don't have to display the exit status in this way, because there are several ways (Section 35.13, Section 35.14, Section 35.15) to use the exit status of one command as a condition of further execution.
Go to http://examples.oreilly.com/upt3 for more information on: true, false
Two simple Unix utilities do nothing but return an exit status. true returns a status of 0 (zero); false returns 1 (one). There are GNU versions on the web site and no, they don't have any amazing extra features. ;-)
bash and zsh have a handy way to reverse the status of a command line: put an exclamation point (!) before it. Let's look at a simple example (of course, you'd use ! with something besides true or false):
bash$ true bash$ echo $? 0 bash$ ! true bash$ echo $? 1 bash$ false bash$ echo $? 1 bash$ ! false bash$ echo $? 0
tcsh and zsh have a handy feature for work with exit statuses. If you set the tcsh shell variable printexitvalue or the zsh shell option PRINT_EXIT_VALUE , the shell will print the exit status of any program that doesn't return zero. For example:
zsh$ setopt printexitvalue zsh$ grep '<title>' 0001.sgm <title>Too Many Arguments for the Command Line</title> zsh$ grep '<title>' 0000.sgm grep: 0000.sgm: No such file or directory zsh: exit 2 grep <title> 0000.sgm zsh$ grep '<ttle>' 0001.sgm zsh: exit 1 grep <ttle> 0001.sgm tcsh% set printexitvalue tcsh% true tcsh% false Exit 1
You can't test the exit status of a background job in the Bourne shell unless you use the wait command to wait for it (in effect, to bring the job out of the background). Pipelines, however, return the exit status of the last program in the pipeline.
JP
If you are going to write a shell script of any complexity at all, you need some way to write "conditional expressions." Conditional expressions are nothing more than statements that have a value of "true" or "false", such as "Am I wearing pants today?" or "Is it before 5 p.m.?" or "Does the file indata exist?" or "Is the value of $aardvark greater than 60?"
The Unix shell is a complete programming language. Therefore, it allows you to write "if" statements with conditional expressions just like C, BASIC, Pascal, or any other language. Conditional expressions can also be used in several other situations, but most obviously, they're the basis for any sort of if statement. Here's the syntax of an if statement for the Bourne shell:
+if conditional then # do this if conditional returns a zero ("true") status one-or-more-commands else # do this if conditional returns non-zero ("false") status one-or-more-commands fi
Depending on how many different ways the command might exit, and thus the varying values its exit status may have, you may want to use either a case statement or elif (for testing multiple conditionals in a single if/else block.)
You can omit the else and the block of code following it. However, you can't omit the then or the fi. If you want to omit the then (i.e., if you want to do something special when condition is false, but nothing when it is true), write the statement like this:
if conditional then : # do nothing else # do this if conditional returns non-zero ("false") status one-or-more-commands fi
Note that this uses a special null command, a colon (:) (Section 36.6). There's another, more useful way of expressing the inverse of a condition (do something if conditional is not "true"), the || operator (Section 35.14) (two vertical bars). You can use this to write an if-type statement without the if!
Don't forget the fi terminating the statement. This is a surprisingly common source of bugs (at least for me).
Another common debugging problem: the manual pages that discuss this material imply that you can smash the if, then, and else onto one line. Well, it's true, but it's not always easy. Do yourself a favor: write your if statements exactly like the one above. You'll rarely be disappointed, and you may even start writing programs that work correctly the first time.
Here's a real-life example, a shell script named bkedit that makes a backup copy of a file before editing it. If cp returns a zero status, the script edits the file; otherwise, it prints a message. (The $1 is replaced with the first filename from the command line see Section 35.20.)
#!/bin/sh if cp "$1" "$1.bak" then vi "$1" else echo "bkedit quitting: can't make backup?" 1>&2 fi
You can try typing that shell script in and running it. Or just type in the lines (starting with the if) on a terminal running the Bourne shell; use a real filename instead of $1.
The if statement is often used with a command named test (Section 35.26). The test command does a test and returns an exit status of 0 or 1.
ML, JP, and SJC
The shells let you test for success right on the command line. This gives you a very efficient way to write quick and comprehensible shell scripts.
I'm referring to the || and && operators and in particular, the || operator. comm1 || comm2 is typically explained as "execute the command on the right if the command on the left failed." I prefer to explain it as an "either-or" construct: "execute either comm1 or comm2." While this isn't really precise, let's see what it means in context:[3]
cat filea fileb > filec || exit
This means "either cat the files or exit." If you can't cat the files (if cat returns an exit status of 1), you exit (Section 24.4). If you can cat the files, you don't exit. You execute the left side or the right side.
I'm stretching normal terminology a bit here, but I think it's necessary to clarify the purpose of ||. By the way, we could give the poor user an error message before flaming out (which, by the way, is a way to write an "inverse if (Section 35.13)):
cat filea fileb > filec || { echo sorry, no dice 1>&2 exit 1 }
Similarly, comm1 && comm2 means "execute comm1 AND comm2," or execute comm2 if comm1 succeeds. (But if you can't execute the first, don't do any.) This might be helpful if you want to print a temporary file and delete it immediately.
lpr file && rm file
If lpr fails for some reason, you want to leave the file around. Again, I want to stress how to read this: print the file and delete it. (Implicitly: if you don't print it, don't delete it.)
ML
The Bourne shell has two kinds of loops that run a command and test its exit status. An until loop will continue until the command returns a zero status. A while loop will continue while the command returns a zero status.
The until loop runs a command repeatedly until it succeeds. That is, if the command returns a nonzero status, the shell executes the body of the loop and then runs the loop control command again. The shell keeps running the command until it returns a zero status, as shown in the following example:
% cat sysmgr #!/bin/sh until who | grep "^barb " do sleep 60 done echo The system manager just logged on. % sysmgr & [1] 2345 ...time passes... barb ttyp7 Jul 15 09:30 The system manager just logged on.
The loop runs who (Section 2.8) and pipes that output to grep (Section 13.1), which searches for any line starting with barb and a space. (The space makes sure that usernames like barbara don't match.) If grep returns a nonzero status (no lines matched), the shell waits 60 seconds. Then the loop repeats, and the script tries the who | grep command again. It keeps doing this until grep returns a zero status then the loop is broken and control goes past the done line. The echo command prints a message and the script quits. (I ran this script in the background so I could do something else while I waited for Barb.)
This is also a useful way to get someone with whom you share a machine to turn on their cell phone: just set a loop to wait until they login and then send them a write message (in case they don't always check their email, like a few nocturnal system administrators I know).
[A Bourne shell until loop is not identical to the until construction in most programming languages, because the condition is evaluated at the top of the loop. Virtually all languages with an until loop evaluate the condition at the bottom. ML]
Go to http://examples.oreilly.com/upt3 for more information on: catsaway
The while loop is the opposite of the until loop. A while loop runs a command and loops until the command fails (returns a nonzero status). The catsaway program below uses a while loop to watch the who output for the system manager to log off. It's the opposite of the sysmgr script.
/dev/null Section 43.12
% cat catsaway #!/bin/sh while who | grep "^barb " > /dev/null do sleep 60 done echo "The cat's away..." % catsaway & [1] 4567 ...time passes... The cat's away...
JP
Most standard Unix toolbox commands return a status (Section 35.12). Your shell script should, too. This section shows how to set the right exit status for both normal exits and error exits.
To end a shell script and set its exit status, use the exit command. Give exit the exit status that your script should have. If it has no explicit status, it will exit with the status of the last command run.
Go to http://examples.oreilly.com/upt3 for more information on: bkedit
Here's an example, a rewrite of the bkedit script from Section 35.13. If the script can make a backup copy, the editor is run and the script returns the exit status from vi (usually 0). If something goes wrong with the copy, the script prints an error and returns an exit status of 1. Here's the script:
#!/bin/sh if cp "$1" "$1.bak" then vi "$1" exit # Use status from vi else echo "bkedit quitting: can't make backup?" 1>&2 exit 1 fi
Here's what happens if I run it without a filename:
$ bkedit cp: usage: cp fn1 fn2 or cp fn1 [fn2...] dir bkedit quitting: can't make backup?
And here's what's left in the exit status variable:
$ echo $? 1
JP
If you're running a shell script and you press your interrupt key (Section 5.8) (like CTRL-c), the shell quits right away. That can be a problem if you use temporary files in your script, because the sudden exit might leave the temporary files there. The trap command lets you tell the shell what to do before it exits. A trap can be used for a normal exit, too. See Table 35-1.
|
Signal number |
Signal name |
Explanation |
|---|---|---|
|
0 |
EXIT |
exit command |
|
1 |
HUP |
When session disconnected |
|
2 |
INT |
Interrupt often CTRL-c |
|
3 |
QUIT |
Quit often CTRL-\ |
|
9 |
KILL |
Kill, often used as a way to stop an errant program (it cannot be caught, so don't bother to trap it) |
|
15 |
TERM |
From kill command |
Here's a script named zmore that uses a temporary file named /tmp/zmore$$ in a system temporary-file directory. The shell will replace $$ with its process ID number (Section 24.3). Because no other process will have the same ID number, that file should have a unique name. The script uncompresses (Section 15.6) the file named on its command line, then starts the more file viewer.[4] The script uses traps, so it will clean up the temporary files, even if the user presses CTRL-c. The script also sets a default exit status of 1 that's reset to 0 if more quits on its own (without an interrupt). If you are on a Linux system, you may find that gzcat is simply named zcat.
exit Section 35.16
#!/bin/sh # zmore - UNCOMPRESS FILE, DISPLAY WITH more # Usage: zmore file stat=1 # DEFAULT EXIT STATUS; RESET TO 0 BEFORE NORMAL EXIT temp=/tmp/zmore$$ trap 'rm -f $temp; exit $stat' 0 trap 'echo "`basename $0`: Ouch! Quitting early." 1>&2' 1 2 15 case $# in 1) gzcat "$1" >$temp more $temp stat=0 ;; *) echo "Usage: `basename $0` filename" 1>&2 ;; esac
There are two traps in the script:
The first trap, ending with the number 0, is executed for all shell exits normal or interrupted. It runs the command line between the single quotes. In this example, there are two commands separated with a semicolon (;) (Section 28.16). The first command removes the temporary file (using the -f option (Section 14.10), so rm won't give an error message if the file doesn't exist yet). The second command exits with the value stored in the stat shell variable. Look ahead at the rest of the script $stat will always be 1 unless the more command quit on its own, in which case stat will be reset to 0. Therefore, this shell script will always return the right exit status if it's interrupted before it finishes, it'll return 1; otherwise, 0.[5]
The second trap has the numbers 1 2 15 at the end. These are signal numbers that correspond to different kinds of interrupts. On newer shells, you can use signal names instead of the numbers. There's a short list in Table 35-1. For a list of all signals, type kill -l (lowercase "L") or see your online signal(3) or signal(2) manual page. Alternatively, look for a file named /usr/include/signal.h or /usr/include/linux/signal.h (which itself just includes /usr/include/asm/signal.h, which is where the constants themselves are defined).
This trap is done on an abnormal exit (like CTRL-c). It prints a message, but it could run any list of commands.
Shell scripts don't always have two traps. Look at the nom (Section 33.8) script for an example.
I usually don't trap signal 3 (QUIT) in scripts that I use myself. That gives me an easy way to abort the script without springing the trap (removing temporary files, etc.). In scripts for general use, though, I usually do trap it.
Also, notice that the echo commands in the script have 1>&2 (Section 36.16) at the end. This is the standard way to make error messages. In this particular script, that doesn't matter much because the script is used interactively. But it's a good habit to get into for all of your scripts.
If your trap runs a series of commands, it's probably neater to call a shell function (Section 29.11) than a list of commands:
trap funcname 1 2 15
JP and SJC
The Bourne shell read command reads a line of one or more words from the keyboard (or standard input)[6] and stores the words in one or more shell variables. This is usually what you use to read an answer from the keyboard. For example:
echo -n "Type the filename: " read filename
Here is how the read command works:
If you give the name of one shell variable, read stores everything from the line into that variable:
read varname
If you name more than one variable, the first word typed goes into the first variable, the second word into the second variable, and so on. All leftover words go into the last variable. For example, with these commands:
echo -n "Enter first and last name: " read fn ln
if a user types John Smith, the word John would be available from $fn and Smith would be in $ln. If the user types Jane de Boes, then Jane would be in $fn and the two words de Boes are in $ln.
Some Bourne shells have a built-in function named line that reads a line from standard input and writes it to standard output. Use it with command substitutions (Section 28.14):
value=`line`
Go to http://examples.oreilly.com/upt3 for more information on: grabchars
The grabchars program lets you read from the keyboard without needing to press RETURN.
JP
Although most scripts for most languages can execute directly (Section 36.3) without needing the Bourne shell, it's common to "wrap" other scripts in a shell script to take advantage of the shell's strengths. For instance, sed can't accept arbitrary text on its command line, only commands and filenames. So you can let the shell handle the command line (Section 35.20) and pass information to sed via shell variables, command substitution, and so on. Simply use correct quoting (Section 27.12) to pass information from the shell into the "wrapped" sed script:
#!/bin/sh # seder - cd to directory in first command-line argument ($1), # read all files and substitute $2 with $3, write result to stdout cd "$1" || exit sed "s/$2/$3/g" *
In SunExpert magazine, in his article on awk (January, 1991), Peter Collinson suggests a stylization similar to this for awk programs in shell scripts (Section 35.2):
#!/bin/sh awkprog=' /foo/{print $3} /bar/{print $4}' awk "$awkprog" $*
He argues that this is more intelligible in long pipelines because it separates the program from the command. For example:
grep foo $input | sed .... | awk "$awkprog" - | ...
Not everyone is thrilled by the "advantages" of writing awk this way, but it's true that there are disadvantages to writing awk the standard way.
Here's an even more complex variation:
<<\ Section 27.16
#!/bin/sh temp=/tmp/awk.prog.$$ cat > $temp <<\END /foo/{print $3} /bar/{print $4} END awk -f $temp $1 rm -f $temp
This version makes it a bit easier to create complex programs dynamically. The final awk command becomes the equivalent of a shell eval (Section 27.8); it executes something that has been built up at runtime. The first strategy (program in shell variable) could also be massaged to work this way.
As another example, a program that I used once was really just one long pipeline, about 200 lines long. Huge awk scripts and sed scripts intervened in the middle. As a result, it was almost completely unintelligible. But if you start each program with a comment block and end it with a pipe, the result can be fairly easy to read. It's more direct than using big shell variables or temporary files, especially if there are several scripts.
# # READ THE FILE AND DO XXX WITH awk: # awk ' ...the indented awk program ... ... ... ' | # # SORT BY THE FIRST FIELD, THEN BY XXX: # sort +0n -1 +3r | # # MASSAGE THE LINES WITH sed AND XXX: # sed ' ...
Multiline pipes like that one are uglier in the C shell because each line has to end with a backslash (\) (Section 27.13). Section 27.12 and Section 27.13 have more about quoting.
ML and JP
To write flexible shell scripts, you usually want to give them command-line arguments. As you've seen in other articles (Section 35.16, Section 35.17), $1 holds the first command-line argument. The Bourne shell can give you arguments through the ninth, $9. The Korn shell and some other newer Bourne-type shells understand ${10} for the tenth argument, and so on.
If you've been reading this series (Section 35.2) of articles in order, you saw the zmore (Section 35.17) script that accepted just one command-line argument. If you put "$@" in a script, the shell will replace that string with a quoted (Section 27.12) set of the script's command-line arguments. Then you can pass as many arguments as you want, including pathnames with unusual characters (Section 14.11):
% zmore report memo "savearts/What's next?"
The third argument has a perfectly legal filename; we see more and more of them on our system especially filesystems that are networked to computers like the Macintosh, or on systems that use windowing systems to run graphical programs such as FrameMaker, where spaces and other "special" characters in filenames are common. Double-quoting all arguments through the script helps to be sure that the script can handle these unusual but legal pathnames.
In this case, we want the arguments to be passed to the GNU zcat command. Let's change the zmore script to read:
zcat "$@" >$temp
When the shell runs the script with the arguments shown above, the command line will become:
zcat "report" "memo" "savearts/What's next?" >/tmp/zmore12345
|
A for loop (Section 35.21) can step through all command-line arguments, one by one. You can also use a while loop (Section 35.15) that tests $# (see below) and removes the arguments one by one with the shift command (Section 35.22). The getopt and getopts (Section 35.24) commands handle arguments in a more standard way.
The $# parameter counts the number of command-line arguments. For instance, if there are three arguments, $# will contain 3. This is usually used for error-checking (as in the zmore script in Section 35.17) with case (Section 35.10) or test (Section 35.26).
JP
Sometimes you want a script that will step through the command-line arguments one by one. (The "$@" parameter (Section 35.20) gives you all of them at once.) The Bourne shell for loop can do this. The for loop looks like this:
for arg in list do ...handle $arg... done
If you omit the in list, the loop steps through the command-line arguments. It puts the first command-line argument in arg (or whatever else you choose to call the shell variable (Section 35.9)), then executes the commands from do to done. Then it puts the next command-line argument in arg, does the loop, and so on, ending the loop after handling all the arguments.
For an example of a for loop, let's hack on the zmore (Section 35.17) script.
case Section 35.11
#!/bin/sh # zmore - Uncompress file(s), display with "more" # Usage: zmore [more options] file [...files] stat=1 # Default exit status; reset to 0 before normal exit temp=/tmp/zmore$$ trap 'rm -f $temp; exit $stat' 0 trap 'echo "`basename $0`: Ouch! Quitting early..." 1>&2' 1 2 15 files= switches= for arg do case "$arg" in -*) switches="$switches $arg" ;; *) files="$files $arg" ;; esac done case "$files" in "") echo "Usage: `basename $0` [more options] file [files]" 1>&2 ;; *) for file in $files do zcat "$file" | more $switches done stat=0 ;; esac
We added a for loop to get and check each command-line argument. For example, let's say that a user typed the following:
% zmore -s afile ../bfile
The first pass through the for loop, $arg is -s. Because the argument starts with a minus sign (-), the case treats it as an option. Now the switches variable is replaced by its previous contents (an empty string), a space, and -s. Control goes to the esac and the loop repeats with the next argument.
The next argument, afile, doesn't look like an option. So now the files variable will contain a space and afile.
The loop starts over once more with ../bfile in $arg. Again, this looks like a file, so now $files has afile ../bfile. Because ../bfile was the last argument, the loop ends; $switches has the options and $files has all the other arguments.
Next, we added another for loop. This one has the word in followed by $files, so the loop steps through the contents of $files. The loop runs zcat on each file, piping it to more with any switches you gave.
Note that $switches isn't quoted (Section 27.12). This way, if $switches is empty, the shell won't pass an empty argument to more. Also, if $switches has more than one switch, the shell will break the switches into separate arguments at the spaces and pass them individually to more.
You can use a for loop with any space-separated (actually, IFS (Section 36.23)-separated) list of words not just filenames. You don't have to use a shell variable as the list; you can use command substitution (Section 28.14) (backquotes) or shell wildcards (Section 33.2), or just "hardcode" the list of words:
lpr Section 45.2
for person in Joe Leslie Edie Allan do echo "Dear $person," | cat - form_letter | lpr done
The getopt and getopts (Section 35.24) commands handle command-line arguments in a more standard way than for loops.
JP
A for loop (Section 35.21) is great if you want to handle all of the command-line arguments to a script, one by one. But, as is often the case, some arguments are options that have their own arguments. For example, in the command grep -f filename, filename is an argument to -f; the option and its argument need to be processed together. One good way to handle this is with a combination of while (Section 35.15), test (Section 35.26), case (Section 35.10), and shift. Here's the basic construct:
while [ $# -gt 0 ] do case "$1" in -a) options="$options $1";; ... -f) options="$options $1" argfile="$2" shift ;; *) files="$files $1";; esac shift done
The trick is this: shift removes an argument from the script's argument list, shifting all the others over by one ($1 disappears, $2 becomes $1, $3 becomes $2, and so on). To handle an option with its own argument, do another shift. The while loop uses test (Section 35.26) to check that $# the number of arguments is greater than zero and keeps going until this is no longer true, which only happens when they have all been used up.
Meanwhile, all the case has to do is to test $1 against the desired option strings. In the simple example shown above, we simply assume that anything beginning with a minus sign is an option, which we (presumably) want to pass on to some program that is being invoked by the script. So all we do is build up a shell variable that will eventually contain all the options. It would be quite possible to do something else instead, perhaps setting other shell variables or executing commands.
We assume that anything without a minus sign is a file. This last case could be written more robustly with a test to be sure the argument is a file. Here's an example of a simple script that uses this construct to pass an option and some files to pr and from there to a program that converts text to PostScript and on to the print spooler (or you could convert SGML or XML files to PDF, whatever):
while [ $# -ne 0 ] do case $1 in +*) pages="$1" ;; *) if [ -f "$1" ]; then files="$files $1" else echo "$0: file $1 not found" 1>&2 fi;; esac shift done pr $pages $files | psprint | lpr
This approach is perhaps obsolete if you have getopts (Section 35.24) (it's built into bash, for instance), since getopts lets you recognize option strings like -abc as being equivalent to -a -b -c, but I still find it handy. [In this example, it's essential. The pr option +page-list starts with a plus sign. getopt and getopts don't support those old-style options. JP]
TOR and SJC
Normally a for loop (Section 35.21) iterates until it has processed all its word arguments. while and until loops (Section 35.15) iterate until the loop control command returns a certain status. But sometimes for instance, if there's an error you want a loop to immediately terminate or jump to the next iteration. That's where you use break and continue, respectively.
break terminates the loop and takes control to the line after done. continue skips the rest of the commands in the loop body and starts the next iteration. Here's an example of both. An outer loop is stepping through a list of directories. If we can't cd to one of them, we'll abort the loop with break. The inner loop steps through all entries in the directory. If one of the entries isn't a file or isn't readable, we skip it and try the next one.
'...' Section 28.14, || Section 35.14, * Section 1.13, test Section 35.26
for dir in `find $HOME/projdir -type d -print` do cd "$dir" || break echo "Processing $dir" for file in * do test -f "$file" -a -r "$file" || continue ...process $dir/$file... done done
With nested loops (like the file loop above), which loop is broken or continued? It's the loop being processed at that time. So the continue here restarts the inner (file) loop. The break terminates the outer (directory) loop, which means the inner loop is also terminated. Note also that the -print argument to find is often redundant in the absence of another expression, depending on your version of find.
Here we've used break and continue within for loops, after the shell's || operator. But you can use them anywhere within the body of any loop in an if statement within a while loop, for instance.
JP
Most shell scripts need to handle command-line arguments options, filenames, and so on. Section 35.20, Section 35.21, and Section 35.22 show how to parse command lines with any Bourne shell. Those methods have two problems. You can't combine arguments with a single dash, e.g., -abc instead of -a -b -c. You also can't specify arguments to options without a space in between, e.g., -barg in addition to -b arg.[7]
Your Bourne shell may have a built-in command named getopts.[8] getopts lets you deal with multiple complex options without these constraints. To find out whether your shell has getopts, see your online sh or getopts(1) manual page.
getopt takes two or more arguments. The first is a string that can contain letters and colons (:). Each letter names a valid option; if a letter is followed by a colon, the option requires an argument. The second and following arguments are the original command-line options; you'll usually give "$@" (Section 35.20) to pass all the arguments to getopt.
getopt picks each option off the command line, checks to see if the option is valid, and writes the correct option to its standard output. If an option has an argument, getopt writes the argument after its option. When getopt finds the first nonoption argument (the first argument that doesn't start with a - character), it outputs two dashes (--) and the rest of the arguments. If getopt finds an invalid option, or an option that should have an argument but doesn't, it prints an error message and returns a nonzero status (Section 35.12).
Go to http://examples.oreilly.com/upt3 for more information on: opttest
Your script can use a loop to parse the getopt output. Here's an example script named opttest that shows how getopt works:
|| Section 35.14, : Section 36.6
#!/bin/sh set -- `getopt "ab:" "$@"` || { echo "Usage: `basename $0` [-a] [-b name] [files]" 1>&2 exit 1 } echo "Before loop, command line has: $*" aflag=0 name=NONE while : do case "$1" in -a) aflag=1 ;; -b) shift; name="$1" ;; --) break ;; esac shift done shift # REMOVE THE TRAILING -- echo "aflag=$aflag / name=$name / Files are $*"
The script has two legal options. The -a option sets the variable named aflag to 1. The -b option takes a single argument; the argument is stored in the variable named name. Any other arguments are filenames.
The script starts by running getopt inside backquotes (Section 28.14) and using the set (Section 35.25) command to replace the command-line arguments with the getopt output. The first argument to set, -- (two dashes) (Section 35.25), is important: it makes sure that set passes the script's options to getopt instead of treating them as options to the shell itself. An echo command shows the output of getopt. Then the loop parses the getopt output, setting shell variables as it goes. When the loop finds the -- argument from getopt, it quits and leaves the remaining filenames (if any) in the command-line arguments. A second echo shows what's in the shell variables and on the command line after the loop. Here are a few examples:
% opttest Before loop, command line has: -- aflag=0 / name=NONE / Files are % opttest -b file1 -a file2 file3 Before loop, command line has: -b file1 -a -- file2 file3 aflag=1 / name=file1 / Files are file2 file3 % opttest -q -b file1 getopt: illegal option -- q Usage: opttest [-a] [-b name] [files] % opttest -bfile1 Before loop, command line has: -b file1 -- aflag=0 / name=file1 / Files are % opttest -ab getopt: option requires an argument -- b Usage: opttest [-a] [-b name] [files]
Some old Bourne shells have problems with an empty "$@" parameter (Section 37.5). If the opttest script doesn't work with an empty command line, as in the first example above, you can change the "$@" in the script to ${1+"$@"}. If you find you're still having some trouble running the script, particularly with bash, try setting the GETOPT_COMPATIBLE environment variable, which sets GNU getopt to emulate the older, less featureful version. Also be sure to read the GNU getopt(1) manual page, as it details the support for POSIX-style long options (which let you do things like pass --longoptions to programs such as GNU getopt.)
The advantages of getopt are that it minimizes extra code necessary to process options and fully supports the standard Unix option syntax (as specified in intro of the User's Manual).
JP and BR
[Most of this article, except IFS and --, also applies to the C shell. JP]
You can pass options and arguments to a shell as you start it, as in:
sh -v file1 file2
and also when a script is invoked with #!. The set command lets you set command-line parameters, including most[9] shell options, after you've started the shell. This simple idea has more uses than you might realize.
The Bourne shell command line can have options like -e (exit if any command returns nonzero status). It can also have other arguments; these are passed to shell scripts. You can set new command-line parameters while you're typing interactive commands (at a shell prompt) or in a shell script.
To reset the command-line parameters, just type set followed by the new parameters. For example, to ask the shell to show expanded versions of command lines after you type them, set the -v (verbose) option (Section 27.15):
$ set -v $ mail $group1 < message mail andy ellen heather steve wilma < message $ mail $group2 < message mail<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>< message $ set +v
On many Bourne shells, typing set +v cancels the v option. On other (mostly early) shells, there's no + feature. To work around that problem, you could start a subshell (Section 24.4) with sh -v, run the commands there, then exit the subshell to cancel the verbose option.
Go to http://examples.oreilly.com/upt3 for more information on: users
You can put filenames or any other strings in the command-line parameters interactively or from a shell script. That's handy for storing and parsing the output of a Unix command with backquotes (Section 28.14). For example, you can get a list of all logged-in users from the parameters $1, $2, and so on. Use users (or rusers to find all the logged in users on the local network) if your system has it. Otherwise, use who (Section 2.8) and cut (Section 21.14) to strip off everything but the usernames:
for Section 35.21
$ set `users` $ set `who | cut -c1-8` $ for u > do > ...do something with each user ($u)... > done
You can save the original parameters in another variable and reset them later:
oldparms="$*" set something new ...use new settings... set $oldparms
Be sure to watch your quoting (as the next section explains).
If the first parameter you set starts with a dash, like -e, the shell will treat it as its own option instead of as a string to put into the command-line parameters. To avoid this, use -- (two dashes) as the first argument to set. In this example, $1 gets -e, and the filenames expanded from the wildcard pattern go into $2, $3, etc.:
set -- -e file*
If you type set by itself with no arguments, it will show a list of all currently set shell variables. In newer Bourne-type shells, it also shows shell functions (Section 29.11) and other shell settings.
This can cause you grief if you accidentally don't pass arguments to set in the middle of a script, and screenfuls of variables spew onto the user's screen. For example, your script runs set 'users' from a cron (Section 25.2) job, in the middle of the night when no one is logged on. The users command returns an empty string, so set gets no arguments, so it outputs a long list of junk.
The standard workaround for this problem is to always use a dummy first parameter typically, a single x when you're setting parameters. Then use shift (Section 35.22) to shift away the x, leaving the other parameters (possibly none). For example:
set x `users` shift
Because the shell parses and scans the new parameters before it stores them, wildcards (Section 33.2) and other special characters (Section 27.17) will be interpreted, so watch your quoting (Section 27.12). You can take advantage of this to parse lines of text into pieces that aren't separated with the usual spaces and TABs for instance, a line from a database with colon-separated fields by setting the IFS (Section 36.23) variable before the set command.
If you want to save any special quoting on the original command line, be careful: the quoting will be lost unless you're clever. For example, if $1 used to be John Smith, it will be split after it's restored: $1 will have John and $2 will be Smith. A better solution might be to use a subshell (Section 43.7) for the part of the script where you need to reset the command-line parameters:
# reset command-line parameters during subshell only: (set some new parameters ...do something with new parameters... ) # original parameters aren't affected from here on...
One last note: set won't set $0, the name of the script file.
JP
Unix has a command called test that does a lot of useful tests. For instance, test can check whether a file is writable before your script tries to write to it. It can treat the string in a shell variable as a number and do comparisons ("Is that number less than 1000?"). You can combine tests, too ("If the file exists and it's readable and the message number is more than 500..."). Some versions of test have more tests than others. For a complete list, read your shell's manual page (if your shell has test built in (Section 1.9)) or the online test(1) manual page.
The test command returns a zero status (Section 35.12) if the test was true and a nonzero status otherwise, so people usually use test with if , while, or until. Here's a way your program could check to see if the user has a readable file named .signature in the home directory:
$HOME Section 35.5, $myname Section 35.28
if test -r $HOME/.signature then ...Do whatever... else
echo "$myname: Can't read your '.signature'. Quitting." 1>&2 exit 1 fi
The test command also lets you test for something that isn't true. Add an exclamation point (!) before the condition you're testing. For example, the following test is true if the .signature file is not readable:
if test ! -r $HOME/.signature then echo "$myname: Can't read your '.signature'. Quitting." 1>&2 exit 1 fi
Unix also has a version of test (a link to the same program, actually) named [. Yes, that's a left bracket. You can use it interchangeably with the test command with one exception: there has to be a matching right bracket (]) at the end of the test. The second example above could be rewritten this way:
if [ ! -r $HOME/.signature ] then echo "$myname: Can't read your '.signature'. Quitting." 1>&2 exit 1 fi
Be sure to leave space between the brackets and other text. There are a couple of other common gotchas caused by empty arguments; Section 37.3 and Section 37.4 have workarounds.
JP
When you write a new program or shell script, you'll probably want to be sure that its name doesn't conflict with any other commands on the system. For instance, you might wonder whether there's a command named tscan. You can check by typing one of the commands in the following example. If you get output (besides an error) from one of them, there's probably already a command with the same name. (The type command works on ksh, bash, and many Bourne shells; I've shown it with a dollar sign ($) prompt.)
which Section 2.6, whereis Section 2.3, alias Section 29.2
% man 1 tscan No manual entry for tscan in section 1. % which tscan no tscan in . /xxx/ehuser/bin /usr/bin/X11 /usr/local/bin ... % whereis tscan tscan: % alias tscan % % whatis tscan tscan: $ type tscan tscan not found
JP
A Unix program should use its name as the first word in error messages it prints. That's important when the program is running in the background or as part of a pipeline you need to know which program has the problem:
someprog: quitting: can't read file xxxxxx
It's tempting to use just the program name in the echo commands:
echo "someprog: quitting: can't read file $file" 1>&2
If you ever change the program name, however, it's easy to forget to fix the messages. A better way is to store the program name in a shell variable at the top of the script file and use the variable in all messages:
myname=someprog ... echo "$myname: quitting: can't read file $file" 1>&2
Even better, use the $0 parameter. The shell automatically puts the script's name there. But $0 can have the absolute pathname of the script, such as /xxx/yyy/bin/someprog. The basename (Section 36.13) program fixes this: basename strips off the head of a pathname everything but the filename.
For example, if $0 is /u/ehuser/bin/sendit:
myname="`basename $0`"
would put sendit into the myname shell variable.
Just as you can make links (Section 10.3) to give Unix files several names, you can use links to give your program several names (Section 36.8). For instance, see the script named ll, lf, lg (...and so on). Use $0 to get the current name of the program.
JP
As Section 35.4 explains, Unix programs can never, ever modify the environment of their parents. A program can only modify the environment that later will be passed to its children. This is a common mistake that many new Unix users make: they try to write a program that changes a directory (or does something else involving an environment variable) and attempt to figure out why it doesn't work. You can't do this. If you write a program that executes the cd command, that cd will be effective within your program but when the program finishes, you'll be back in your original (parent) shell.
One workaround is to "source" the shell script file (for csh and bash) or run it as a "dot" script (sh, ksh, and bash, too). For example, if the file named change-my-directory contains:
cd /somewhere/else
you could use the following commands to change the current directory of the current shell:
% source change-my-directory $ . change-my-directory
The source and . commands read a script file into the current shell instead of starting a child shell. These commands work only for shell script files (files containing command lines as you'd type them at a shell prompt). You can't use source or . to read a binary (directly executable) file into the shell.
If your shell doesn't have shell functions (Section 29.11), you can simulate them (Section 29.14) with the . command. It acts a lot like a subroutine or function in a programming language.
ML and JP
So far, we have discussed some shell function basics (Section 29.11), using examples such as the mx( ) function that uses sed and dig to find out what host accepts mail for a given address. In that example, we simply made a set of complex functionality available as a quick and easy (and short) call from the command line. But you can also define functions and use them within shell scripts, or you can use the . and source commands to include those functions from an external file (Section 35.29).
We've also discussed using functions to automate repetitive tasks (Section 29.11), such as calculating factorials.
For now, let's demonstrate both of these approaches specifically with respect to defining a function to automate a repetitive task and then sharing the function with other shell scripts. Using the mx( ) function we defined earlier, let's put it into its own file, mx.sh, and store it in our personal shell function library directory (in this case, $HOME/lib):
$ cat > ~/lib/mx.sh function mx( ) { # Look up mail exchanger for host(s) for host do echo "==== $host ====" dig "$host" mx in | sed -n '/^;; ANSWER SECTION:/,/^$/{ s/^[^;].* //p }' done } ^D $ more !$ function mx( ) { # Look up mail exchanger for host(s) for host do echo "==== $host ====" dig "$host" mx in | sed -n '/^;; ANSWER SECTION:/,/^$/{ s/^[^;].* //p }' done } $
Now the file ~/lib/mx.sh contains a function named mx( ) fair enough, but let's say we want to be able to pass a list of hosts (determined dynamically on a regular basis, say, from spam-fighting tools that find open SMTP proxies) to a shell script, and have that shell script email a form letter to the postmaster address at that host. We will call the shell script proxynotify, and call it as follows:
$ proxynotify < proxyList
proxylist contains a list of hosts, one per line, in the com domain. We want to iterate over them and mail the postmaster for the domain, rather than mailing directly to an address such as postmaster@[IP], on the assumption that maybe the top-level postmaster can fix what may be an unmonitored relay. Just to verify that some other system isn't responsible for delivering the mail, we will check using the mx( ) shell function. We've also included a quickie shell function named ip( ) that returns the IP address for a given hostname. As you can see, we use a local variable for the IP address, and we use the -z test for zero length of a string. We also check whether the file is readable, check the script arguments, and use a variety of other tricks.
#!/bin/sh # proxynotify demo # get our function . $HOME/lib/mx.sh function ip( ) { for host do local ip=`dig in host $host |\ grep $host |\ grep "TABATAB" |\ awk '{print $5}'` echo $ip done } if [ -z "$1" ] then echo "Usage: $0 [file]" exit 1 elif [ -r "$1" ] then echo "found a file and it is readable" else echo "file $1 not readable or does not exist" exit 1 fi for domain in `cat "$1"` do echo "processing $domain" themx=`mx $domain` echo "MX for $domain is '$themx'" if [ ! -z $themx ] then cat formletter | mail -s "proxy" postmaster@$themx else echo "couldn't find MX for $domain," echo "mailing direct-to-IP instead." theip=`ip $domain` if [ ! -z $theip ]; then cat formletter | mail -s "proxy" postmaster@$theip else echo "giving up, can't find anyone to notify" echo "$domain" >> /tmp/proxybadlist.$$ return 1 fi fi done mail -s "bad proxies" <</tmp/proxybadlist.$$ rm /tmp/proxybadlist.$$
SJC
[1] Before X11 Release 5, the DISPLAY variable might appear as unix:0.0.
[2] printenv and env are external (Section 1.9) commands; they work with any shell.
[3] Others refer to it as a "short-circuit" operator.
[4] The script could run gzcat $1 | more directly, but some versions of more can't back up when reading from a pipe. You may prefer to use less, at any rate.
[5] It's important to use single quotes rather than double quotes around the trap. That way, the value of $stat won't be interpreted until the trap is actually executed when the script exits.
[6] Some early versions of read don't handle < redirection (Section 43.1); they can only read from the terminal.
[7] Although most Unix commands allow this, it is actually contrary to the Command Syntax Standard Rules in intro of the User's Manual. Check your shell's manual pages for whether it supports getopts.
[8] Both bash and ksh have it. getopts replaces the old command getopt; it is better integrated into the shell's syntax and runs more efficiently. C programmers will recognize getopts as very similar to the standard library routine getopt(3).
[9] Some options for some shells can be set only from the command line as the shell is invoked. Check the shell's manual page.
| CONTENTS |


Welcome to
VTM Table Topics
Please choose your name.
Start whenever you're ready.
Your Name:
Send Name


P.graph(Cap,...)
P
[ struct ] - Report object created by the report.new
function.
Cap
[ char | cellstr | @auto
] - Title, or cell array with title and subtitle, displayed at the top of the graph; @auto
means that the first comment from the first child series
object will be used for the title.
'axesOptions='
[ cell | empty ] - (Inheritable) Options executed by calling set
on the axes handle before running 'postProcess='
.
'dateTick='
[ numeric | Inf
] - (Inheritable) Date tick spacing.
'grid='
[ @auto
| true
| false
] - (Inheritable) Display grid lines; if @auto
, 'grid='
is true
unless a right-hand-side axis is plotted.
'legend='
[ false
| true
] - (Inheritable) Add legend to the graph.
'legendLocation='
[ char | 'best'
| 'bottom'
] - (Inheritable) Location of the legend box; see help on legend
for values available.
'postProcess='
[ char | empty ] - (Inheritable) String with Matlab commands executed after the graph has been drawn and styled; see Description.
'preProcess='
[ char | empty ] - (Inheritable) String with Matlab commands executed before the graph has been drawn and styled; see Description.
'range='
[ numeric | Inf
] - (Inheritable) Graph range.
'rhsAxesOptions='
[ cell | empty ] - (Inheritable) Options executed by calling set
on the RHS axes handle before running 'postProcess='
.
'style='
[ struct | empty ] - (Inheritable) Apply this style structure to the graph and its children; see help on qstyle
.
'tight='
[ @auto
| true
| false
] - (Inheritable) Set the y-axis limits to the minimum and maximum of displayed data; if @auto
, 'tight='
is true
unless a right-hand-side axis is plotted.
'xLabel='
[ char | empty ] - Label the x-axis.
'yLabel='
[ char | empty ] - Label the y-axis.
'zeroLine='
[ true
| false
| cell ] - (Inheritable) Add a horizontal zero line if zero is included on the y-axis; specify zeroline options in a cell array.
See dat2str
for details on date format options.
'dateFormat='
[ char | cellstr | 'YYYYFP'
] - Date format string, or array of format strings (possibly different for each date).
'freqLetters='
[ char | 'YHQBMW'
] - Six letters used to represent the six possible frequencies of IRIS dates, in this order: yearly, half-yearly, quarterly, bi-monthly, monthly, and weekly (such as the 'Q'
in '2010Q1'
).
'months='
[ cellstr | {'January',...,'December'}
] - Twelve strings representing the names of the twelve months.
'standinMonth='
[ numeric | 'last'
| 1
] - Month that will represent a lower-than-monthly-frequency date if the month is part of the date format string.
See help on generic options in report objects.
The options 'preProcess='
and 'postProcess='
give you additional flexibility in customising the graphics style of the axes object. The values assigned to these options are expected to be strings with an executable Matlab command, or commands separated with semi-colons (as if typed on one line in the command window). The command can refer to the following variables:
H
- a handle to the currently processed axes object.L
- a handle to the corresponding legend object; if no legend object exists for the axes H
, L
will be NaN
.Create a one-page report with a chart in on the LHS and the legend moved to the RHS. Use the function grfun.movetosubplot
in the option 'postProcess='
, referring to L
(handle to the legend object associated with the respective axes object) to move the legend around.
% Create random data series.
A = tseries(1:10,@rand);
B = tseries(1:10,@rand);
% Open a new report.
x = report.new();
% Open a new figure in the report with a 1-by-2 layout.
x.figure('My Figure','subplot=',[1,2]);
% The graph will be placed in the LHS space.
% Use `grfun.movetosubplot` to move the legend to the RHS space.
x.graph('My Graph','legend=',true, ...
'postProcess=','grfun.movetosubplot(L,1,2,2)');
x.series('Series A',A);
x.series('Series B',B);
x.publish('test.pdf');
open test.pdf;


Next: Sharpsign X, Previous: Sharpsign B, Up: Sharpsign
#Orational reads rational in octal (radix 8). For example,
#o37/15 ≡ 31/13 #o777 ≡ 511 #o105 ≡ 69 ;105_8
The consequences are undefined if the token immediately following the #O does not have the syntax of an octal (i.e., radix 8) rational.


The primary function to run a simulation.
simulate
Simulate Spread of Gene for Altruism
Default and custom functions to govern mating behavior.
randomMatches
Random Mating
sexualSelection
Mating via Sexual Selection
Default and custom functions to govern warner behavior under attack.
warnRelatives
Warning Relatives When Under Attack
warnRelativesGraph
Warning Relatives When Under Attack (Graph Version)
warnWarners
Warning Other Warners


Home
About
Products
Blog
Contact
Contact US :
FIRSTNAME:
LASTNAME:
EMAIL:
GENDER:
MALE
FEMALE
OTHER
You can write the feedback about the product in the box given:


Description: For each element in the set, get the first element that matches the selector by testing the element itself and traversing up through its ancestors in the DOM tree.
selector: Selector A string containing a selector expression to match elements against.
selector: Selector A string containing a selector expression to match elements against.
context: Element A DOM element within which a matching element may be found. If no context is passed in then the context of the jQuery set will be used instead.
jQuery object: jQuery A jQuery object to match elements against.
element: Element An element to match elements against.
Given a jQuery object that represents a set of DOM elements, the .closest()
method searches through these elements and their ancestors in the DOM tree and constructs a new jQuery object from the matching elements. The .parents()
and .closest()
methods are similar in that they both traverse up the DOM tree. The differences between the two, though subtle, are significant:
.closest()
|
.parents()
|
|---|---|
| Begins with the current element | Begins with the parent element |
| Travels up the DOM tree until it finds a match for the supplied selector | Travels up the DOM tree to the document's root element, adding each ancestor element to a temporary collection; it then filters that collection based on a selector if one is supplied |
| The returned jQuery object contains zero or one element for each element in the original set | The returned jQuery object contains zero or more elements for each element in the original set |

I

II

A

B

1

2

3

C

III

Suppose we perform a search for

elements starting at item A:
$('li.item-a').closest('ul')
.css('background-color', 'red');
This will change the color of the level-2

, since it is the first encountered when traveling up the DOM tree.
Suppose we search for an

element instead:
$('li.item-a').closest('li')
.css('background-color', 'red');
This will change the color of list item A. The .closest()
method begins its search with the element itself before progressing up the DOM tree, and stops when item A matches the selector.
We can pass in a DOM element as the context within which to search for the closest element.
var listItemII = document.getElementById('ii');
$('li.item-a').closest('ul', listItemII)
.css('background-color', 'red');
$('li.item-a').closest('#one', listItemII)
.css('background-color', 'green');
This will change the color of the level-2

, because it is both the first

ancestor of list item A and a descendant of list item II. It will not change the color of the level-1

, however, because it is not a descendant of list item II.
<!DOCTYPE html>

Click me!

You can also Click me!

<!DOCTYPE html>

Click me!

You can also Click me!

Description: Gets an array of all the elements and selectors matched against the current element up through the DOM tree.
selectors: Array An array or string containing a selector expression to match elements against (can also be a jQuery object).
context: Element A DOM element within which a matching element may be found. If no context is passed in then the context of the jQuery set will be used instead.
This signature (only!) is deprecated as of jQuery 1.7. This method is primarily meant to be used internally or by plugin authors.
<!DOCTYPE html>


1.5. Comparing Records: Key-Value Pairs (WORK IN PROGRESS)¶
If we want to sort some things, we have to be able to compare them, to decide which one is bigger. How do we compare two things? If all that we wanted to sort or search for was simple integer values, this would not be an interesting question. We can just use standard comparison operators like “<” or “>”. Even if we wanted to store strings, most programming languages give us built-in functions for comparing strings alphabetically. But we do not usually want to store just integers or strings in a data structure. Usually we want to store records, where a record is made up of multiple values, such as a name, an address, and a phone number. In that case, how can we “compare” records to decide which one is “smaller”? We cannot just use “<” to compare the records! Nearly always in this situation, we actually are interested in sorting the records based on the values of one particular field used to represent the record, which itself is something simple like an integer. This field is referred to as the key for the record.
Likewise, if we want to search for a given record in a database, how should we describe what we are looking for? A database record could simply be a number, or it could be quite complicated, such as a payroll record with many fields of varying types. We do not want to describe what we are looking for by detailing and matching the entire contents of the record. If we knew everything about the record already, we probably would not need to look for it. Instead, we typically define what record we want in terms of a key value. For example, if searching for payroll records, we might wish to search for the record that matches a particular ID number. In this example the ID number is the search key.
1.5.1. Comparables¶
To implement sorting or searching, we require that keys be comparable. At a minimum, we must be able to take two keys and reliably determine whether they are equal or not. That is enough to enable a sequential search through a database of records and find one that matches a given key. However, we typically would like for the keys to define a total order, which means that we can always tell which of two keys is greater than the other. Using key types with total orderings gives the database implementor the opportunity to organize a collection of records in a way that makes searching more efficient. An example is storing the records in sorted order in an array, which permits a binary search. Fortunately, in practice most fields of most records consist of simple data types with natural total orders. For example, integers, floats, doubles, and character strings all are totally ordered.
But if we want to write a general purpose sorting or searching
function, we need a general way to get the key for the record.
We could insist that every record have a particular method called
.key()
.
That seems like a good name for it!
Some languages like Java and C++ have special infrastructure for
supporting this (such as the Comparable
interface in Java,
which has the .compareTo()
method for defining the exact process
by which two objects are compared).
But many languages like Processing and JavaScript do not.
But what if the programmer had already used that method name for another purpose? An even bigger problem is, what if the programmer wants to sort the record now using one field as the key, and later using another field? Or search sometimes on one key, and at other times on another? The problem is that the “keyness” of a given field is not an inherent property within the record, but rather depends on the context. So, you cannot always count on being able to use your favorite method name (or even the comparable interface) to extract the desired key value.
1.5.2. Comparators¶
Another, more general approach is to supply a function or class—called a comparator—whose job is to extract the key from the record. A comparator function can be passed in as a parameter, such as in a call to a sorting function. In this case, the comparator function would be invoked on two records whenever they need to be compared. In this way, different comparator functions can be passed in to handle different record types or different fields within a record. In Java (with generics) or C++ (with templates), a comparator class can be a parameter for another class definition. For example, a BST could take a comparator class as a generics parameter in Java. This comparator class would be responsible for dealing with the comparison of two records.
Unfortunately, while flexible and able to handle nearly all situations, there are a few situations for which it is not possible to write a key extraction method. In that case, a comparator will not work. 1
1.5.3. Implementing Key-Value Search¶
One good general-purpose solution is to explicitly store key-value pairs in the data structure. For example, if we want to sort a bunch of records, we can store them in an array where every array entry contains both a key value for the record and a pointer to the record itself. This might seem like a lot of extra space required, but remember that we can then store pointers to the records in another array with another field as the key for another purpose. The records themselves do not need to be duplicated. A simple class for representing key-value pairs is shown here.
// KVPair class definition
public class KVPair<K extends Comparable<K>, E> implements Comparable<KVPair<K, E>> {
K theKey;
E theVal;
KVPair(K k, E v) {
theKey = k;
theVal = v;
}
// Compare KVPairs
public int compareTo(KVPair<K,E> it) {
return theKey.compareTo(it.key());
}
// Compare against a key
public int compareTo(K it) {
return theKey.compareTo(it);
}
public K key() {
return theKey;
}
public E value() {
return theVal;
}
public String toString() {
String s = "(";
if (theKey != null) { s += theKey.toString(); }
else { s += "null"; }
s += ", ";
if (theVal != null) { s += theVal.toString(); }
else { s += "null"; }
s += ")";
return s;
}
}
The main places where we will need to be concerned with comparing
records and extracting keys is for various dictionary
implementations and sorting algorithms.
To keep them clear and simple, visualizations for sorting algorithms
will usually show them as operating on integer values stored in an
array.
But almost never do people really want to sort an array of integers.
But to be useful, a real sorting algorithm typically has to
deal with the fact that it is sorting a collection of records.
A general-purpose sorting routine meant to operate on multiple record
types would have to be written in a way to deal with the generic
comparison problem.
To illustrate, here is an example of
Insertion Sort implemented to
work on an array that stores records that support the Comparable
interface.
Note that since KVPair
is implemented to implement the
Comparable
interface, an array of KVPair
could be used by this
sort function.
static <T extends Comparable<T>> void inssort(T[] A) {
for (int i=1; i < A.length; i++) { // Insert i'th record
for (int j=i; j > 0 && A[j].compareTo(A[j-1]) < 0; j--) {
swap(A, j, j-1);
}
}
}
def inssort(A):
for i in range(len(A)): # Insert i'th record
j = i
while j > 0 and A[j] < A[j-1]:
swap(A, j, j-1)
j -= 1
Here are some review questions to test your knowledge from this module.
- 1
One example of a situation where it is not possible to write a function that extracts a key from a record is when we have a collection of records that describe books in a library. One of the fields for such a record might be a list of subject keywords, where the typical record stores a few keywords. Our dictionary might be implemented as a list of records sorted by keyword. If a book contains three keywords, it would appear three times on the list, once for each associated keyword. However, given the record, there is no simple way to determine which keyword on the keyword list triggered this appearance of the record. Thus, we cannot write a function that extracts the key from such a record.


Using the Brother GT-361, I was able to make a T-shirt. To make the design for the t-shirt, I used Adobe Photoshop to cut out a picture of Tom Brady, I then copied the picture to a blank page in Corel Draw and typed above. I saved the final design as a jpg and downloaded it onto a flash drive. Next, I grabbed a blank t-shirt and heated it so it would be flat. I then put the shirt into theBrotherGT-361 and plug my flashdrive in. My t-shirt came out with my design on it. After I heated ita gain to dry the ink.
T-Shirt Design File
Sublimation Printing
For my dye sublimation project, I made a dog bowl. First, I went to The Conde Dye Trans website and picked out the dog bowl that Mr. Gerber then ordered. To design my bowl, I went used Corel Draw and Adobe photoshop. First, I had to down load the dog bowl template. On my dog bowl, I wanted pictures of my dog, so I cut out their faces from pictures in photoshop and copied them to Corel Draw. I had to fill two of the templates with pictures because one only went half way around the bowl. When I finished my designs, I save them and put them on a flash drive. I walked it over to the Sawgrass 800 printer and printed out the two designs.  I needed to heat it, so I set the oven to about 400 degrees and heated it for 15 minutes. When it was finished, my design, that was on the paper, transferred to the bowl. After letting it cool, I was able to take the bowl home and feed my dogs with it.
Bowl Design File
Conde Dye Trans Website
Home