Académique Documents
Professionnel Documents
Culture Documents
Preface
1.Introduction
2.Strings
3.Lists
4.Pattern matching
5.Arrays
6.Numbers and Arithmetic
7.Control structure
8.Procedures
9.File commands
10.eval: building Tcl commands with Tcl and feeding them back to Tcl
11.exec: building Unix commands with Tcl and feeding them to Unix
12.Further Study
Preface
part of Tcl for Web Nerds by Hal Abelson, Philip Greenspun, and Lydia Sandon
The last thing that the world needs is another tutorial book on the Tcl programming language. So here's ours...
When we sat down to plan a class at MIT on Web service design and implementation, our first thought was to give the
students our favorite commercial Tcl book: Brent Welch's Practical Programming in Tcl and Tk. At 630 pages, nobody can
accuse Prentice-Hall of shortchanging the reader on information. Sadly, however, for the Web nerd most of this information is
useless. Two-thirds of the book is devoted to Tk, a toolkit for constructing graphical user interfaces. On the Internet, the user
interface is a Web browser. If you're programming a Web service, all you care about is using Tcl to produce some HTML that gets
sent to the client.
Another problem with using Welch's book for our class is that he is unable to assume that his readers are primarily interested
in building Web applications. So his examples have to be extremely generic.
A final and crushing problem with the book is that it isn't available on the Web. Even if our students can afford to pony up
$40 or $50 for a 630-page book, they probably can't afford the physical therapy and orthopedic surgery they'd require after
lugging it around from class to class.
What about Tcl for Web Nerds then? We hope that a professional programmer or MIT student can breeze through it in one
evening. By the end of the evening, that person should have learned Tcl, learned a little something about the Web, and not have
been bored.
It is available on the Web at a permanent URL. If you don't like it, the authors will happily refund your purchase price. :-)
Introduction
part of Tcl for Web Nerds by Hal Abelson, Philip Greenspun, and Lydia Sandon
If you're an MIT student reading this tutorial, you have a special advantage in learning Tcl, because you've read Structure and
Interpretation of Computer Programs (Abelson and Sussman 1996; MIT Press). We're not going to try to redeliver the
knowledge in SICP packaged up in Tcl syntax. However, we are going to try to show how some of the big ideas from SICP can be
accomplished even in the comparatively impoverished Tcl language.
One of the strengths of Lisp, the programming language used to illustrate the principles in SICP, is that it has very simple
syntax:
e.g.,
will set ten-factorial to the result of evaluating the factorial procedure with an argument of 10, then compute a graduate
student's checking account balance, then print out some career advice to a computer science graduate student.
These examples are written in the Scheme dialect of Lisp, which is distributed from
http://www-swiss.ai.mit.edu/scheme-home.html.
How would you accomplish the same goals in Tcl? Tcl syntax is
This is like Lisp, except that procedures invoked at top level need not be surrounded by delimeters. Procedures invoked from
within other procedure calls must be surrounded by [] instead of ():
Tcl variables can't include dashes; you have to use underscore instead. Note that doing arithmetic with Tcl requires escaping into a
world of infix syntax with the expr procedure.
Arguments are evaluated in sequence. The resulting values are then passed to procedure_name, which is assumed to be a system-
or user-defined procedure. Tcl assumes that you're mostly dealing with strings and therefore stands some of the conventions of
standard programming languages on their heads. For example, you might think that set foo bar would result in Tcl complaining
about an undefined variable (bar). But actually what happens is that Tcl sets the variable foo to the character string "bar":
> tclsh
% set foo bar
bar
% set foo
bar
%
The first command line illustrates that the set command returns the new value that was set (in this case, the character string
"bar"), which is the result printed by the interpreter. The second command line uses the set command again, but this time with only
one argument. This actually gets the value of the variable foo. Notice that you don't have to declare variables before using them.
By analogy with Scheme, you'd think that we could just type the command line $foo and have Tcl return and print the value.
This won't work in Tcl, however, which assumes that every command line invokes a procedure. This is why we need to explicity
use set or puts.
Does this mean that you never need to use string quotes when you've got string literals in your program? No. In Tcl, the
double quote is a grouping mechanism. If your string literal contains any spaces, which would otherwise be interpreted as argument
separators, you need to group the tokens with double quotes:
In the first command above, the Tcl interpreter saw that we were attempting to call set with seven arguments. In the second
command, we grouped all the words in our string literal with the double quotes and therefore the Tcl interpreter saw only two
arguments to set. Note a stylistic point here: multi-word variable names are all-lowercase with underscores separating the words.
This makes our Tcl code very compatible with relational database management systems where underscore is a legal character in a
column name.
In this example, we invoked the Unix command "tclsh" to start the Tcl interpreter from the Unix shell. Later on we'll see how
to use Tcl in other ways:
writing a program file and evaluating it
embedding Tcl commands in Web pages to create dynamic pages
extending the behavior of the Web server with Tcl programs
For now, let's stick with typing interactively at the shell. You can keep evaluating Tcl commands at the % prompt until you exit the
Tcl shell by evaluating exit.
To indicate a literal string that contains a space, you can wrap the string in double quotes. Quoting like this does not prevent the
interpreter from evaluating procedure calls and variables inside the strings:
The interpreter looks for dollar signs and square brackets within quoted strings. This is known as variable interpolation What if
you need to include a dollar sign or a square bracket? One approach is to escape with backslash:
If we don't need Tcl to evaluate variables and procedure calls inside a string, we can use braces for grouping rather than
double quotes:
Throughout the rest of this book you'll see hundreds of examples of braces being used as a grouping character for Tcl code. For
example, when defining a procedure or using control structure commands, conditional code is grouped using braces.
The good news is that Tcl does not suffer from cancer of the semicolon. The bad news is that any Tcl procedure call or command
must be on one line from the interpreter's point of view. Suppose that you want to split up
If you want to have newlines within the double quotes, that's just fine:
set a_very_long_variable_name
"a very long value of some sort...
with a few embedded newlines
makes for rather bad poetry"
Tcl would interpret this as two separate commands, the first a call to set to find the existing value of
a_very_long_variable_name and the second a call to the procedure named "a very long value...":
If you want to continue a Tcl command on a second line, it is possible to use the backslash to escape the newline that would
otherwise terminate the command:
% set a_very_long_variable_name \
"a very long value of some sort...
with a few embedded newlines
makes for rather bad poetry"
a very long value of some sort...
with a few embedded newlines
makes for rather bad poetry
%
Note that this looks good as code but the end-result is probably not what you'd want. The second and third lines of our poem
contain seven spaces at the beginning of each line. You probably want to do something like this:
For completeness we note that semicolon will allow you to put multiple Tcl commands on one line:
The great case-sensitivity winter descended upon humankind in 1970 with the Unix operating system.
% set MyAge 36
36
% set YearsToExpectedDeath [expr 80-$Myage]
can't read "Myage": no such variable
%
Variables and procedure names in Tcl are case-sensitive. We consider it very bad programming style to depend on this, though.
For example, you shouldn't simultaneously use the variables Username and username and rely on the computer to keep them
separate; the computer will succeed but humans maintaining the program in the future will fail. So use lowercase all the time with
underscores to separate words!
Procedures
One of the keys to making a large software system reliable and maintainable is procedural abstraction. The idea is to take a
complex operation and encapsulate it into a function that other programmers can call without worrying about how it works.
Here's the factorial procedure in Tcl:
At first glance, you might think that you've had to learn some new syntax here. In fact, the Tcl procedure-creation procedure is
called like any other. The three arguments to proc are procedure_name arglist body. The creation command is able to extend
over several lines not because the interpreter recognizes something special about proc but because we've used braces to group
blocks of code. Similarly the if statement within the procedure uses braces to group its arguments so that they are all on one line
as far as the interpreter is concerned.
As the example illustrates, we can use the standard base-case-plus-recursion programming style in Tcl. Our factorial
procedure checks to see if the number is 0 (the base case). If so, it returns 1. Otherwise, it computes factorial of the number minus
1 and returns the result multiplied by the number. The # character signals a comment.
We'll have more to say about procedures later on. For now, examine the following example:
% factorial 145
0
One of the reasons that commercial programmers work so hard and accomplish so little is that they use languages that can't even
do arithmetic! Tcl's feeble brain is overwhelmed by the idea of an integer larger than one machine word (32 bits in this case). You'll
have to go back to Scheme if you want to find Bill G's predicted worth:
(factorial 145)
;Value:
80479260574719919448490292577980627710999743900750061634474528104711541237364652141085048187983964922743929823029
89150198131082216516636595724416094085569177391493159059928114118666357860755246018358156427933025042432000000000
00000000000000000000000000
which tests whether the variable foobar is defined. The info command can also tell a running program what local variables are
defined (info local), what procedures are defined (info procs), what arguments a procedure takes (info args
procedure_name), and what the source code for a procedure is (info body procedure_name).
One useful aspect of reflection extends the behavior of set desribed above: a Tcl procedure can examine the calling stack
and set variables in its caller's environment. For example, in the ArsDigita Community System toolkit, there's a procedure called
set_variables_after_query that looks at the last SQL query executed and sets local variables to the values retrieved from the
database:
set_variables_after_query
Note that this example uses procedures defined by AOLserver, e.g., ns_db to talk to the database and ns_write to send bytes
back to the Web client. The prefixing of procedure names by a package name is a good convention that you should follow in your
own programming. In this case, the "ns_" prefix alerts fellow programmers to the fact that you are calling API calls for the
"NaviServer". NaviServer? That's what AOLserver was called before being purchased by America Online. The fact that the API
prefixes were not changed illustrates another good programming practice: don't change things gratuitiously.
In most cases, rather than typing Tcl expressions at the interpreter you'll be using Tcl to generate dynamic Web pages. AOLserver
provides two mechanisms for doing this:
1..tcl pages are Tcl programs that are executed by the webserver (in this case AOLServer). They typically generate
character strings that are sent to the client browser with ns_write.
2..adp pages are like ordinary HTML pages, but they contain escapes to Tcl commands that are evaluated by the server.
Which option you choose depends largely on whether static HTML or Tcl constitutes the majority of the page you are
creating. If you're printing fixed character strings with Tcl, you'll need to "quote" any embedded quotes by preceding them with
backslash. E.g., to print to a Web page the string <a href="goodies.html"> you would use the Tcl command
So if the majority of your Tcl expressions are simply printing strings, you'll find it more convenient to avoid the hassles of quoting
by
using .adp pages rather than complete .tcl programs.
Why the use of ns_write instead of puts? The puts command writes to a program's standard output stream. If you were
writing a Tcl CGI script, standard output would be linked through the Web server to the Web client and you could indeed use
puts. This idea doesn't make sense with AOLserver, where all the Tcl scripts are running inside the Web server process. If the
Web server is serving 100 clients simultaneously, to which client should bytes written with puts be sent?
Anyway, if you want to have some fun one night, try redefining puts to forward its argument to ns_write and see what
happens.
If in reading this introduction, you've come to realize that "Hey, Tcl is just like Lisp, but without a brain, and with syntax on
steroids", you might wonder why Lisp isn't a more popular scripting language than Tcl. Lisp hasn't been a complete failure, by the
way; it is used as an extension language by users of some popular programs, notably AutoCAD. But Tcl has been much more
successful. It has been compiled into hundreds of larger programs, including AOLserver, which is why we wrote this book.
As a software developer, you're unlikely to get rich. So you might as well try to get through your life in such a way that you
make a difference to the world. Tcl illustrates one way:
make something that is simple enough for almost everyone to understand
give away your source code
explain how to weave your source code in with other systems
In the case of AOLserver, for example, Jim Davidson and Doug McKee had only a few months to build the whole server from
scratch. They wanted users to be able to write small programs that ran inside the server process. They therefore needed a safe
interpreted language so that a programming error by one user didn't crash the server and bring down all the Web services for an
organization.
Tcl was available. Tcl was easy to download and designed to fit inside larger application programs. But the Tcl interpreter as
distributed had one terrible bug: it wasn't thread safe, i.e., you couldn't have two copies of the Tcl interpreter running inside the
same program at the same time. Doug and Jim had to read through the Tcl source code and modify it to be thread safe. So it was
critically important for them that Tcl was open-source and simple enough so as to not require months or years of study to
understand the whole system.
Compare this to Lisp. Some of the best and brightest computer scientists raised money to build commercial Lisp
implementations that they then went out and hawked in an indifferent and confused marketplace. They succeeded only in breaking
their hearts and their investors' wallets. A handful of academics produced free open-source implementations, notably CMU
Common Lisp (see http://www.cons.org/cmucl/) and various versions of Scheme (see
http://www-swiss.ai.mit.edu/scheme-home.html; Scheme 48 is the closest to Tcl in spirit). But these multi-megabyte monsters
weren't designed to fit neatly into someone else's program. Nor was there any document explaining how to do it.
Lisp developers have the satisfaction of knowing that they got it right 30 years before anyone else. But that's about all they
have to show for 40 years of hard work and hundreds of millions of dollars in government and private funding. These days, most
former Lisp programmers are stuck using Unix and Microsoft programming environments and, not only do they have to put up with
these inferior environments, but they're saddled with the mournful knowledge that these environments are inferior.
part of Tcl for Web Nerds by Hal Abelson, Philip Greenspun, and Lydia Sandon
If your program receives data from a Web client, it comes in as a string. If your program sends an HTML page back to a Web
client, it goes out as a string. This puts the string data type at the heart of Web page development:
If you're processing data from the user, typically entered into an HTML form, you'll be using a rich variety of built-in string-handling
procedures. Suppose that a user is registering at your site with the form variables first_names, last_name, email,
password. Here's how we might build up a list of exceptions (using the Tcl lappend command, described in the chapter on lists):
If there aren't any exceptions, we have to get these data ready for insertion into the database:
The simplest way to look for a substring within a string is with the string first command. Some users of photo.net complained
that they didn't like seeing classified ads that were simply pointers to the eBay auction site. Here's a simplified snippet from
http://software.arsdigita.com/www/gc/place-ad-3.tcl:
Both implementations will catch any capitalization variant of "eBAY". Both implementations will miss "e-bay" but it doesn't matter
because if the poster of the ad includes a link with a URL, the hyperlink will contain "ebay". What about false positives? If you visit
www.m-w.com and search for "*ebay*" you'll find that both implementations might bite someone selling rhododendrons or a
water-powered mill. That's why the toolkit code checks a "DisalloweBay" parameter, set by the publisher, before declaring this an
exception.
If you're just trying to find a substring, you can use either string first or regexp. If you're trying to do something more
subtle, you'll need regexp (described more fully in the chapter "Pattern Matching"):
In the ArsDigita Community System, we have a page that shows a user's complete history with a Web service, e.g.,
http://photo.net/shared/community-member.tcl?user_id=23069 shows all of the postings by Philip Greenspun. If a comment on a
static page is short, we want to show the entire message. If not, we want to show just the first 1000 characters.
In http://software.arsdigita.com/www/shared/community-member.tcl, we find the following use of the string range
command:
We can never figure out how to use format without either copying an earlier fragment of pattern or referring to the man page
(http://www.scriptics.com/man/tcl7.5/TclCmd/format.n.html). However, here are some examples for you to copy:
The Tcl command scan performs the reverse operation, i.e., parses an input string according to a pattern and stuffs values as it
finds them into variables:
Notice that the number returned by scan is a count of how many conversions it was able to perform successfully. If you really want
to use scan, you'll need to visit the man page: http://www.scriptics.com/man/tcl7.5/TclCmd/scan.n.html. For an idea of how useful
this is for Web development, consider that the entire 250,000-line ArsDigita Community System does not contain a single use of
the scan command.
List Operations
part of Tcl for Web Nerds by Hal Abelson, Philip Greenspun, and Lydia Sandon
A Tcl list holds a sequence of elements, each of which can be a number, a string, or another list. Let's look at the commands for
constructing a list:
At this point, the variable user_preferences is a three-element list. We can pull individual items out with lindex:
% lindex $user_preferences 0
hiking
% lindex $user_preferences 1
biking
% lindex $user_preferences 2
whale watching
% lindex $user_preferences 3
% lindex $user_preferences 5
Indexing is 0-based and lindex will return the empty string rather than an error if you supply an out-of-range index.
When producing a page for a user, we'd be more likely to be interested in searching the list. The command lsearch returns
the index of the list element matching a query argument or -1 if unsuccessful:
Suppose that User A marries User B. You want to combine their preferences into a household_preferences variable using the
concat command:
The Tcl shell's output is giving you an ugly insight into the internal representation of lists as strings, with elements being separated
by
spaces and grouped with braces. There is no reason to rely on how Tcl represents lists or even think about it. Practice data
abstraction by using Tcl lists as your underlying storage mechanism but define constructor and accessor procedures that the rest of
your source code invokes. You won't find a huge amount of this being done in Web development because the really important data
structures tend to be RDBMS tables, but here's an example of how it might work, taken from
http://photo.net/philg/careers/four-random-people.tcl. We're building a list of lists. Each sublist contains all the information on a
single
historical figure. Method A is quick and dirty:
set einstein [list "A. Einstein" "Patent Office Clerk" "Formulated Theory of Relativity."]
set mill [list "John Stuart Mill" "English Youth" "Was able to read Greek and Latin at age 3."]
% set einstein [define_person "A. Einstein" "Patent Office Clerk" "Formulated Theory of Relativity."]
{A. Einstein} {Patent Office Clerk} {Formulated Theory of Relativity.}
% set einsteins_title [person_title $einstein]
Patent Office Clerk
Data abstraction will make building and maintaining a large system much easier. As noted above, however, the stateless nature of
HTTP means that any information you want kept around from page to page must be kept by the RDBMS. SQL already forces you
to refer to data by table name and column name rather than positionally.
Suppose we have a file called addressees.txt with information about people, one person to a line. Suppose each of these lines
contains, among other information, an email address which we assume we can recognize by the presence of an at-sign (@). The
following program extracts all the email addresses and joins them together, separated by commas and spaces, to form a string
called spam_address that we can use as the Bcc: field of an email message, to spam them all:
close $addressees_stream
lindex list i
Returns the ith element from list; starts at index 0.
http://www.scriptics.com/man/tcl7.5/TclCmd/lindex.n.html
llength list
Returns the number of elements in list.
http://www.scriptics.com/man/tcl7.5/TclCmd/llength.n.html
lrange list i j
Returns the ith through jth elements from list.
http://www.scriptics.com/man/tcl7.5/TclCmd/lrange.n.html
http://www.scriptics.com/man/tcl7.5/TclCmd/lappend.n.html
http://www.scriptics.com/man/tcl7.5/TclCmd/linsert.n.html
http://www.scriptics.com/man/tcl7.5/TclCmd/lreplace.n.html
http://www.scriptics.com/man/tcl7.5/TclCmd/lsearch.n.html
http://www.scriptics.com/man/tcl7.5/TclCmd/lsort.n.html
http://www.scriptics.com/man/tcl7.5/TclCmd/concat.n.html
http://www.scriptics.com/man/tcl7.5/TclCmd/join.n.html
http://www.scriptics.com/man/tcl7.5/TclCmd/split.n.html
Pattern matching
part of Tcl for Web Nerds by Hal Abelson, Philip Greenspun, and Lydia Sandon
Pattern matching is important across a wide variety of Web programming tasks but most notably when looking for exceptions in
user-entered data and when trying to parse information out of non-cooperating Web sites.
Tcl's pattern matching facilities test whether a given string matches a specified pattern. Patterns are described using a syntax
known as regular expressions. For example, the pattern expression consisting of a single period matches any character. The
pattern a..a matches any four-character string whose first and last characters are both a.
The regexp command takes a pattern, a string, and an optional match variable. It tests whether the string matches the
pattern, returns 1 if there is a match and zero otherwise, and sets the match variable to the part of the string that matched the
pattern:
Patterns can also contain subpatterns (delimited by parentheses) and denote repetition. A star denotes zero or more occurrences of
a pattern, so a(.*)a matches any string of at least two characters that begins and ends with the character a. Whatever has
matched the subpattern between the a's will get put into the first subvariable:
What you want is a non-greedy regexp, a standard feature of Perl and an option in Tcl 8.1 and later versions (see
http://www.scriptics.com/services/support/howto/regexp81.html).
Lisp systems in the 1970s included elegant ways of returning all possibilities when there were multiple matches for an
expression.
Java libraries, Perl, and Tcl demonstrate the progress of the field of computer science by ignoring these superior systems of
decades past.
A common problem in Web development is pulling information out of cookies that come from the client. The cookie spec at
http://home.netscape.com/newsref/std/cookie_spec.html mandates that multiple cookies be separated by semicolons. So you look
for "the cookie name that you've been using" followed by an equals sign and them slurp up anything that follows that isn't a
semicolon. Here is how the ArsDigita Community System looks for the value of the last_visit cookie:
Note the square brackets inside the regexp. The Tcl interpreter isn't trying to call a procedure because the entire regexp has been
grouped with braces rather than double quotes. Square brackets denote a range of acceptable characters:
[A-Z] would match any uppercase character
[ABC] would match any of first three characters in the alphabet (uppercase only)
[^ABC] would match any character other than the first three uppercase characters in the alphabet, i.e., the ^
reverses the sense of the brackets
The plus sign after the [^;] says "one or more characters that meets the preceding spec", i.e., "one or more characters that isn't a
semicolon". It is distinguished from * in that there must be at least one character for a match.
If successful, the regexp command above will set the match variable with the complete matching string, starting from
"last_visit=". Our code doesn't make any use of this variable but only looks at the subvar last_visit that would also have been
set.
Pages that use this cookie expect an integer and this code failed in one case where a user edited his cookies file and
corrupted it so that his browser was sending several thousands bytes of garbage after the "last_visit=". A better approach might
have been to limit the match to digits:
More generally regexp allows multiple pattern variables. The pattern variables after the first are set to the substrings that matched
the subpatterns. Here is an example of matching a credit card expiration date entered by a user:
Full Syntax
The most general form of regexp includes optional flags as well as multiple match variables:
It's common in Web programming to create strings by substitution. Tcl's regsub command performs substitution based on a
pattern:
matches the pattern against the data. If the match succeeds, the variable named var is set to data, with various parts modified, as
specified by replacements. If the match fails, var is simply set to data. The value returned by regsub is the number of
replacements performed.
The flag -all specifies that every occurrence of the pattern should be replaced. Otherwise only the first occurrence is
replaced. Other flags include -nocase and -- as with regexp
Here's an example from the banner ideas module of the ArsDigita Community System (see
http://photo.net/doc/bannerideas.html). The goal is that each banner idea contain a linked thumbnail image. To facilitate cutting and
pasting of the image html, we don't require that the publisher include uniform subtags within the IMG. However, we use regexp to
clean up:
In the example above, <replacements> specified the literal characters ''. Other replacement directives include:
& inserts the string that matched the pattern
The backslashed numbers \1 through \9 inserts the strings that matched the corresponding sub-patterns in the
pattern.
Here's another web example, which parses HTML, and replaces the comments (delineated in HTML by <!-- and -->) by the
comment text, enclosed in parentheses.
String match
Tcl provides an alternative matching mechanism that is simpler for users to understand than regular expressions. The Tcl command
string match uses "GLOB-style" matching. Here is the syntax:
It returns 1 if there is a match and 0 otherwise. The only pattern elements permitted here are ?, which matches any single character;
*, which matches any sequence; and [], which delimits a set of characters or a range. This differs from regexp in that the pattern
must match the entire string supplied:
Array Operations
part of Tcl for Web Nerds by Hal Abelson, Philip Greenspun, and Lydia Sandon
Tcl arrays are actually hash tables and have nothing in common with the data structures called arrays in other programming
languages . A Tcl array provides a rapid answer to the question "is there a value associated with this key". Here is a rat-simple
example:
% set numeric_day(Sunday) 0
0
% set numeric_day(Monday) 1
1
% set numeric_day(Tuesday) 2
2
% # pull one value out of the hash table
% set numeric_day(Monday)
1
% # let's ask Tcl what keys are defined in the hash table
% array names numeric_day
Monday Sunday Tuesday
% # let's see if there are values for Sunday and Wednesday
% info exists numeric_day(Sunday)
1
% info exists numeric_day(Wednesday)
0
You don't have to declare to Tcl that you're going to treat a particular variable as an array; just start setting variables with the form
"variable_name(key)".
Here's a procedure that computes Fibonacci numbers in linear time by storing intermediate values in an array called fibvals. It
uses the for loop, which we'll see again in the section on control structure.
If your index contains spaces, it will confuse the Tcl parser . For example, imagine an array called snappy_response that contains
appropriate responses to various insults, which are used as the indices to the array. Suppose you want to store a response for
"Have you gained weight?". You can't feed this to Tcl as
This first time this procedure is called with a particular argument, the Tcl statement is evaluated (using Tcl's built-in eval
command). The result of that evaluation is then stored in the array variable generic_cache with a key consisting of the full Tcl
statement. The next time memoize is called with the same argument, the info exists generic_cache($tcl_statement) will
evaluate to true and the value will be returned from the cache.
Here's how a piece of code might look before:
If someone notices that (1) page_with_top_10_popular_items requires sweeping the database and takes 30 seconds to
execute, and (2) the result doesn't change more than once or twice a day, the natural conclusion is memoization:
Our actual toollkit contains Memoize and Memoize_for_Awhile, the latter of which takes an argument of after how many
seconds
the information in the cache should be considered stale and reevaluated.
Typically on the Web the last thing that you'd want is an in-memory database. If the server crashes or the user gets bounced to
another machine by a load-balancer, you don't want critical data to be trapped inside a Web server's virtual memory. However,
there is one situation where you would want an in-memory database: to store information about the server itself.
In the ArsDigita Community System, an attempt is made to document every externally-called procedure. We want to build
up a documentation database that grows as procedures are defined on a running server. The fundamental mechanism is to define
procedures using our own procedure, proc_doc. This takes a documentation string as an extra argument, calls proc to actually
define the procedure, then records in a Tcl array variable the file from which the procedure definition was read and the doc string:
Full Documentation
Tcl provides support for iterating through the indices, and for coverting lists to arrays. These are documented in
http://www.scriptics.com/man/tcl7.5/TclCmd/array.n.html
ns_set instead
If you're using AOLserver and need to associate keys with values, you might be better off using the ns_set data structure. The
advantages of ns_set over Tcl arrays are the following:
you can easily pass ns_sets from procedure to procedure
you can easily pass ns_sets from C code to Tcl and vice versa
A disadvantage of ns_sets is that they require O[n] time to look up the value of key, compared to O[1} time for the Tcl array,
which, as noted above, is actually a hash table. If you are managing thousands of keys, this might become significant. Otherwise,
program using whichever data structure seems more natural.
See the Tcl Developer's guide at www.aolserver.com for documentation of the ns_set facility.
Numbers
part of Tcl for Web Nerds by Hal Abelson, Philip Greenspun, and Lydia Sandon
Arithmetic is not done directly by the Tcl interpreter. It is done by calling the C library using the expr command on arthmetic
expressions. The detailed parsing rules for arithmetic expressions depend on the particular Unix implementation, but they are more
or less like in C.
Here are some examples:
% expr sin(.5)+cos(.9)
1.10103550687
Reference
Here are the numeric functions included in Tcl. (Details may vary depending on your Unix implementation of expr.)
abs(x)
asin(x)
acos(x)
atan(x)
atan2(y,x)
atan2 returns the angle theta of the polar coordinates returned when (x,y) is
converted to (r, theta).
ceil(x)
cos(x)
cosh(x)
double(x)
returns x as a double or floating point.
exp(x)
returns e^x
floor(x)
fmod(x,y)
returns the floating point remainder of x/y.
hypot(x,y)
returns the square root of the sum of x squared plus y squared, the length of the
line from (0,0) to (x,y).
int(x)
truncates x to an integer.
log(x)
returns the natural log of x.
log10(x)
returns log base 10 of x.
pow(x,y)
returns x to the y power.
round(x)
sin(x)
sinh(x)
sqrt(x)
tan(x)
tanh(x)
Control Structure
part of Tcl for Web Nerds by Hal Abelson, Philip Greenspun, and Lydia Sandon
Control structures let you say "run this fragment of code if X is true" or "do this a few times" or "do this until something is no longer
true". The available control structures in Tcl may be grouped into the following categories:
conditional
looping (iteration)
error-handling
miscellaneous (non-local exit)
Note that the words "then" and "else" are optional, as is the entire else clause. The most basic if statement looks like this:
if {condition} {
body
}
In the ArsDigita Community System, we always leave out the "then", but if we include an else or elseif clause, we put in those
optional words. Consistency is the hobgoblin of little minds...
if {condition} {
body
} elseif {other_condition} {
alternate_body
} else {
another_body
}
Note how the curly braces and keywords are artfully positioned so that the entire if statement is on one line as far as the
interpreter is concerned, i.e., all the newlines are grouped within curly braces. An easy way to break your program is to rewrite the
above statement as follows:
if {condition} {
body
}
elseif {other_condition} {
alternate_body
} else {
another_body
}
The Tcl interpreter will think that the if statement has ended after the first body and will next try to evaluate "elseif" as a procedure.
Let's look at an example from http://software.arsdigita.com/www/register/user-login.tcl. At this point in the ArsDigita
Community System flow, a user has already typed his or her email address.
if {$selection == ""} {
# Oracle didn't find a row; this email addres is not in the database
# redirect this person to the new user registration page
ns_returnredirect "user-new.tcl?[export_url_vars return_url email]"
return
}
The same page provides an example both of nested if and if then else:
Notice that the conventional programming style in Tcl is to call if for effect rather than value. It would work just as well to write
the inner if in a more Lisp-y style:
This works because if returns the value of the last expression evaluated. However, being correct and being comprehensible to the
community of Tcl programmers are different things. It is best to write code adhering to indentation and other stylistic conventions.
You don't want to be the only person in the world capable of maintaining a service that has to be up 24x7.
If http://software.arsdigita.com/www/register/user-login.tcl finds a user in the database, it uses a switch on the user's state to
decide what to do next:
switch $user_state {
"authorized" { # just move on }
"banned" {
ns_returnredirect "banned-user.tcl?user_id=$user_id"
return
}
"deleted" {
ns_returnredirect "deleted-user.tcl?user_id=$user_id"
return
}
"need_email_verification_and_admin_approv" {
ns_returnredirect "awaiting-email-verification.tcl?user_id=$user_id"
return
}
"need_admin_approv" {
ns_returnredirect "awaiting-approval.tcl?user_id=$user_id"
return
}
"need_email_verification" {
ns_returnredirect "awaiting-email-verification.tcl?user_id=$user_id"
return
}
"rejected" {
ns_returnredirect "awaiting-approval.tcl?user_id=$user_id"
return
}
default {
ns_log Warning "Problem with registration state machine on user-login.tcl"
ad_return_error "Problem with login" "There was a problem authenticating the account: $user_id. Most likely, the database
contains users with no user_state."
return
}
}
In this case, we're using the standard switch behavior of matching strings exactly. We're also provide a "default" keyword at the
end that indicates some code to run if nothing else matched.
It is possible to use more sophisticated patterns in switch. Here's a fragment that sends different email depending on the
pattern of the address:
The third behavior for switch is invoked using the "-regexp" flag. See the pattern matching chapter for more on how these patterns
work.
More: http://www.scriptics.com/man/tcl7.5/TclCmd/switch.n.html
Looping commands while, foreach, and for
while { conditional_statement } {
loop_body_statements
}
The conditional statement is evaluated; if it is true, the loop body statement is executed, and then the conditional statement is
reevaluated and the process repeats. If the conditional statement is ever false, the interpreter does not execute the loop body
statements, and continues to the next line after the conditional.
Here is a while statement used to display the last name, first name of each MIT nerd using a Web service. The conditional is
the result of calling AOLserver's ns_db getrow API procedure. This procedure returns 1 if it can fetch the next row from the
SQL cursor, 0 if there aren't any more rows to fetch.
Here's an example from http://software.arsdigita.com/www/monitor.tcl, a page that displays current server activity:
# ask AOLserver to return a list of lists, one for each current connection
set connections [ns_server active]
We use this control structure in the winner picking admin page of the ArsDigita Comunity System's contest module:
http://software.arsdigita.com/www/admin/contest/pick-winners.tcl. The input to this page specifies a time period, a contest, and how
many winners are to be picked. Here the result of executing the for loop is a list of N elements, where N is the number of desired
winners:
If a Tcl command throws an error in a CGI script or an AOLserver API page, by default the user will be presented with an error
page. If you don't want that to happen, fix your bugs! Sometimes it isn't possible to fix your bugs. For example, the ns_httpget
API procedure fetches a Web page from the wider Internet. Under certain network-dependent conditions, it may throw an error. If
you don't want your users to be exposed to that as an error, put in a catch:
catch returns 1 if script threw an error, 0 otherwise. If you supply the second argument (variable_name), catch will set that
variable to the result of executing script, whether or not the script threw an error.
Our classic example always involves ns_httpget. Here's one from http://www.webho.com/WealthClock:
Sending email is another time that a Web server has to go outside its carefully controlled world and might experience an error. Here
is the entire http://software.arsdigita.com/tcl/ad-monitor.tcl, which implements a central facility for other sections in the ArsDigita
Community System. The idea is that programmers can put in "email the administrator if broken" instructions on pages that won't
result in a nightmare for the administrator if the page is getting hit every few seconds.
# the overall goal here is that the ad_host_administrator gets
# notified if something is horribly wrong, but not more than once
# every 15 minutes
Make sure that you don't overuse catch. The last thing that you want is a page failing silently. Genuine errors should always be
brought to a user's attention and ideally to the site administrator's. Users should not think that a server has done something on their
behalf when in fact the task was not accomplished.
More: http://www.scriptics.com/man/tcl7.5/TclCmd/catch.n.html
When inside a looping command, it is sometimes desirable to get the command to stop looping or to stop executing the current
iteration but to continue on the next one. The break command is used to permanently escape the loop; the continue command is
used to escape the current iteration of the loop but to start again at the next iteration. The syntax for each consists only of the
appropriate word written on a line by itself within a loop.
We often use the break command when we want to limit the number of rows to display from the database. Here's an
example from the photo.net neighbor-to-neighbor system. By default, we only want to show a "reasonable" number of postings on
one page:
set selection [ns_db select $db ... big SQL query ... ]
More: http://www.scriptics.com/man/tcl7.5/TclCmd/break.n.html
The return command has been shown before. It quits the proc it's in and returns the supplied value. Remember that any
procedure lines after return aren't executed. Too many times we've seen code of the following form:
proc a_new_programmers_proc {} {
set db [ns_db gethandle]
# do a bunch of stuff with the database
return $result
# release the database handle
ns_db releasehandle $db
}
The most interesting thing that you can do with return is write procedures that force their callers to return as well. Here's an example
from http://software.arsdigita.com/tcl/ad-security.tcl:
proc ad_maybe_redirect_for_registration {} {
if { [ad_verify_and_get_user_id] != 0 } {
# user is in fact logged in, return happiness
return
} else {
ns_returnredirect "/register/index.tcl?return_url=[ns_urlencode [ns_conn url]$url_args]"
# blow out of 2 levels
return -code return
}
}
ad_maybe_redirect_for_registration
# the code below will never get executed if the user isn't registered
# ... update the database or whatever ...
More: http://www.scriptics.com/man/tcl7.5/TclCmd/return.n.html
The error command returns from a proc and and raises an error that, if not caught by a catch statement, will result in the user
seeing a server error page. The first argument to error is displayed in the debugging backtrace:
proc divide {x y} {
if {$y == 0} {
error "Can't divide by zero."
} else {
return [expr $x / $y]
}
}
More: http://www.scriptics.com/man/tcl7.5/TclCmd/error.n.html
Procedures
part of Tcl for Web Nerds by Hal Abelson, Philip Greenspun, and Lydia Sandon
This creates a procedure with the name "name." Tcl has a global environment for procedure names, i.e., there can be only one
procedure called "foobar" in a Tcl system.
The next part of the syntax is the set of arguments, delimited by a set of curly braces. Each argument value is then mapped
into the procedure body, which is also delimited by curly braces. As before, each statement of the procedure body can be
separated by a semi-colon or a newline (or both). Here's an example, taken from the calendar widget component of the ArsDigita
Community System:
Of course in the actual ACS, we encourage programmers to use proc_doc instead, which defines the procedure but also records
a doc string, as described in array variable chapter. Here's what the definition would look like with proc_doc:
More: http://www.scriptics.com/man/tcl7.5/TclCmd/proc.n.html
There are three possible scopes for a variable in an AOLserver Tcl script:
local to a procedure (the default)
shared among all procedures executing in one thread (global)
shared among all threads on a server and persistent from one connection to another (ns_share)
To use a variable locally, you need not declare it. To instruct the Tcl interpreter to read and set a variable in the global
environment, you must call global every place that the variable is used. For example, when side graphics have been displayed on
a page, ad_footer needs to know so that it can insert a <BR CLEAR=RIGHT> tag.
One of the strangest and most difficult to use features of Tcl is the ability to read and write variables up the calling stack with
uplevel and upvar.
Optional arguments
Here is an example of a procedure that has one required and one optional argument:
If a page calls ad_header with one argument, it gets the standard appropriate HTML header. If a page supplies the extra optional
argument, that information gets written into the HEAD. Otherwise, the default value of empty string is used for the second
argument.
In addition, Tcl can also provide for a variable number of arguments at the end, using a special last argument called args in any
procedure definition. After all of the other (previous) arguments are bound to names, the rest of the arguments are shoved into a list
called args which can then be accessed inside the procedure.
You could imagine that between optional arguments and extra ones, the interpreter might get confused. It doesn't, because it
assumes that you aren't using extra args at the end without binding all of the optional ones in the middle; that is, it stuffs argument
values into the argument names in strict order without regard to options, extras, etc.
Rename
In case this wasn't flexible enough, Tcl lets you rename procedures using a proc called rename old_name new_name. If you
wanted to keep the old proc but still take advantage of the great abstraction you've used throughout your site (i.e. not change all of
the command calls when you change their bodies), you can rename the old proc and create a new one with the old one's name.
You can also use rename with the empty string for the second argument. This results in the proc being deleted.
More: http://www.scriptics.com/man/tcl7.5/TclCmd/rename.n.html
File Operations
part of Tcl for Web Nerds by Hal Abelson, Philip Greenspun, and Lydia Sandon
Tcl has a built-in interface for dealing with Unix files. The commands themselves are relatively straightforward, so we'll just explain
them in a reference list below.
Reference
ln -s billg-wealth.tcl temp.tcl
file lstat temp.tcl temp ==> (array holding stat info)
ln -s file.txt file1.txt
file readlink file1.txt ==> file.txt
More: http://www.scriptics.com/man/tcl7.5/TclCmd/file.n.html
Input/Output Commands
More: http://www.scriptics.com/man/tcl7.5/TclCmd/open.n.html
More: http://www.scriptics.com/man/tcl7.5/TclCmd/puts.n.html
More: http://www.scriptics.com/man/tcl7.5/TclCmd/gets.n.html
More: http://www.scriptics.com/man/tcl7.5/TclCmd/read.n.html
tell streamname
Return the "seek offset." (See below for seek.)
More: http://www.scriptics.com/man/tcl7.5/TclCmd/tell.n.html
More: http://www.scriptics.com/man/tcl7.5/TclCmd/seek.n.html
eof stream
Returns 1 if you have reached the end of the stream; 0 otherwise.
if {[eof $this_stream]} {
break
}
More: http://www.scriptics.com/man/tcl7.5/TclCmd/eof.n.html
flush streamname
Write buffers of a stream
flush $this_stream
More: http://www.scriptics.com/man/tcl7.5/TclCmd/flush.n.html
close streamname
Close the stream.
close $this_stream
More: http://www.scriptics.com/man/tcl7.5/TclCmd/close.n.html
Eval
part of Tcl for Web Nerds by Hal Abelson, Philip Greenspun, and Lydia Sandon
The interpreter can be called explicitly to complete extra rounds of substitutions or simply to interpret an additional time, using the
subst and eval instructions respectively. The eval command takes a string argument which is a command, as follows:
% eval $cmd
Hello, World!
The subst command does the single round of substitution ordinarily completed by the interpreter, but without invoking any
command. It takes one argument, the string to be substituted into.
While curly braces normally prevent internal substitution, they are not respected by the subst command. In order to prevent the
single round of substitution, the backslash must be used before special characters like a dollar sign or square brackets.
More: http://www.scriptics.com/man/tcl7.5/TclCmd/eval.n.html
Exec
part of Tcl for Web Nerds by Hal Abelson, Philip Greenspun, and Lydia Sandon
You can invoke other Unix programs from within a Tcl script. For a Web developer this is either a tremendous convenience or a
dangerous security hole.
Let's cover the convenience part first. If we want to have a Web page that displays how long the server has been up and the
current load average, the exec command is helpful:
% exec /usr/bin/uptime
4:04pm up 29 days, 4:32, 3 users, load average: 0.61, 0.66, 0.63
To make an AOLserver Tcl page that returns this output, you need only wrap the exec in an API call:
The photo sharing system at http://photo.net/photodb/ stores user-uploaded content in the Unix file system and hence makes
extensive use of Unix commands to create directories, build thumnails of uploaded images, etc. Here are some examples of how the
photodb system uses exec:
If you do need to use exec, at least make sure that your Web server is running as an unprivileged user with limited authority
to execute Unix programs. Depending on your publishing requirements and choice of Web server, it may be possible to run the
Web server in a chroot() environment (this is very easy with AOLserver 3.0). This changes the root directory as far as the Web
server is concerned. Thus a Tcl program running within the Web server will not be able to even look at files or programs elsewhere
on the computer.
If you decide to run chrooted, you will have to copy any programs that you actually do need to exec so that they are
underneath the Web server's root directory.
More: http://www.scriptics.com/man/tcl7.5/TclCmd/exec.n.html
This is the last of the main Tcl topics. For more information, check out aolserver.com and the books referenced in this book.
Further Study
part of Tcl for Web Nerds by Hal Abelson, Philip Greenspun, and Lydia Sandon
Here are our suggestions for improving your knowledge and skills, organized by goal.
Tcl for Web Nerds only covers a small part of Tcl/Tk and public domain Tcl libraries and extensions.
If you want to learn how to build graphical user interface (GUI) applications with the Tk (toolkit), you're best off reading a
book on Tcl/Tk, such as Brent Welch's Practical Programming in Tcl and Tk. Remember, you've just finished a book about Tcl
the language and haven't learn anything about Tk because we expect that all the user interface will be handled by a Web browser.
http://www.scriptics.com/resource/doc/books/ contains a comprehensive and up-to-date list of the traditional Tcl/Tk books.
Scriptics.com is probably the most likely long-term source of information about significant Tcl developments.
It would be nice if modesty prevented us from recommending our own books, such as Philip and Alex's Guide to Web Publishing
and SQL for Web Nerds, both available as hyperlinks from http://photo.net/wtr/, which also contains links to Web standards.
Additional books for Web developers are reviewed at http://photo.net/wtr/bookshelf.html.
Sadly, your path to intellectual glory as a computer scientist will not be smoothed by your knowledge of Tcl. Modesty surely would
prevent Hal from recommending Structure and Interpretation of Computer Programs (Abelson and Sussman; MIT Press) but it is
philg who is typing this page and he feels comfortable telling anyone to start there.
Index
part of Tcl for Web Nerds by Hal Abelson, Philip Greenspun, and Lydia Sandon
abs
acos
append
asin
atan
atan2
break
catch
ceil
close
concat
continue
cos
cosh
double
error
eof
exec
exit
exp
expr
eval
file atime
file dirname
file executable
file exists
file extension
file isdirectory
file isfile
file lstat
file mtime
file owned
file readable
file readlink
file rootname
file size
file stat
file tail
file type
file writable
floor
flush
fmod
for
foreach
format
gets
hypot
global
if
incr -- not listed in TCLfWN; here's the description at scriptics
info args
info body
info exists
info local
info procs
int
join
lindex
linsert
list
llength
log
log10
lrange
lreplace
lsearch
lsort
open
pow
proc
puts
read
regexp
regsub
rename
return
round
scan
seek
set
sin
sinh
split
sqrt
string compare
string first
string last
string match
string range
string tolower
string toupper
string trim
string trimleft
string trimright
string wordend
string wordstart
subst
switch
tan
tanh
tell
uplevel -- not listed in TCLfWN; description at Scriptics:
http://www.scriptics.com/man/tcl7.5/TclCmd/uplevel.n.html
upvar
while