The Real History of the GUI

By Mike Tuck
August 13th 2001
"There is no reason anyone would want a computer in their home." -- Ken Olson,
President, Chairman and Founder of Digital Equipment Corp., 1977
The story is part of the computer world’s mythology. December 1979, an ordinary
afternoon: young computer whiz and entrepreneur Steve Jobs leads a band of his
homeys into the rarefied enclave of Xerox’s Palo Alto Research Center (PARC). Jobs
and friends tour the plant with wide-eyed admiration, doing their best Norman
Rockwell gee-whiz kids-in-the-dugout impression (“Golly, Dr. Kay, can we have your
autograph, huh?”), while behind guileless eyes and black-framed glasses, mental
notes are being taken and schematics memorized.
Jobs leads his friends out of the building, waves bye-bye to the nice lab geeks inside,
and dashes back to his shabby warehouse, where he and cohort Steve Wozniak stuff
every idea and process they can remember from the Xerox tour into their new
product, the Macintosh. Xerox is befuddled, Microsoft’s Bill Gates is enraged, and
Apple gets the jump on everyone with a new dance craze, the GUI. “Do the GUI”
sweeps the computer world and everybody else scrambles to get on the gravy train.
Gates takes Jobs’ thievery one step beyond Jobs’ own and brings out Apple-clone
Windows, Microsoft does a pas de deux with the local judiciary to dodge an Apple
lawsuit, Windows takes over the world, and Apple is relegated to cult status among
the renegade hackers and Mac addicts of the computer industry.
Nice story to read your kids to sleep with. It has everything: drama, criminal
behavior, ruthless rivalry between former associates, everything except sex (which
the stereotyped computer geeks are unfamiliar with, anyway). Hell, it would even
make a good David Allen Coe drinking song. The only problem with it is that it isn’t
true.

The real history of the Graphical User Interface is more complex and interwoven than
the simplistic “It Takes a Thief” conception.
"So we went to Atari and said, 'Hey, we've got this amazing thing, even built with
some of your parts, and what do you think about funding us? Or we'll give it to you.
We just want to do it. Pay our salary, we'll come work for you.' And they said, 'No.' So
then we went to Hewlett-Packard, and they said, 'Hey, we don't need you. You
haven't got through college yet.'" -- Apple Computer Inc. founder Steve Jobs on
attempts to get Atari and HP interested in his and Steve Wozniak's personal
computer
From Small Seeds...
Apple was founded by Steve Jobs and Steve Wozniak in Jobs’s garage in 1976. Jobs
and Wozniak met at Hewlett-Packard and began their collaborative careers by
building (Wozniak) and selling (Jobs) “blue boxes” illegal devices that scammed free
phone calls from Ma Bell. Both shared an interest in the “primitive” computers of the
time and enjoyed cobbling together electronic goodies with solder and breadboards.
Eventually they decided to start a company and build computers that wouldn’t take
up an entire basement, didn’t need supercooling, and didn’t require platoons of guys
in jumpsuits to take care of them. In other words, they envisioned building personal
computers for the masses. Of course, neither Jobs nor Wozniak were the first to think
of personal desktop-sized computers (common wisdom gives that honor to the MITS
“Altair,” a 1975 kit-based creation running Microsoft’s BASIC OS and based on Intel’s
8080 chip), but that’s another story. They put their heads together and decided to
call their company Apple.
In March 1976, Wozniak built the first Apple, the Apple I. It was a cobbled-together
curiosity made of circuit boards and LED displays stuffed into a wooden box, but it
stirred enough interest in the computing community to inspire Jobs and Wozniak to
found Apple on April Fools’ Day, 1976 to sell their little beasties. Jobs sold his VW
minibus, and Wozniak his HP scientific calculator, to finance the startup. They only
managed to sell about 200 of the Apple I’s, so the fledgling company – now
consisting of Wozniak, Jobs, and a few friends/employees – used the money they
managed to raise from Apple I sales to start work on the Apple II (Wozniak has
reputedly said that a large part of his desire to build the Apple II was due to
“Breakout,” a classic video game he had designed for Atari. Wozniak wanted to
program it for a PC). In 1977 the Apple II debuted, featuring a sleek plastic case (as
opposed to the “orange crates” that houses the Apple I’s), game paddles, and color
graphics on the video display. Being descendants of Ugh, people were fascinated by
the bright colors and the flickering images, and the Apple II began to move off the
shelves.
Jobs realized that he had started something that could mushroom into a serious
business concern, and he laid on more employees, more workspace, and buckled
down to the task of meeting the sudden consumer demand for his goodies. When
Apple added the inboard floppy disk in 1977 (abandoning the slow and clumsy tape
storage facility), the II‘s sales really took off, and Apple was suddenly at the crest of
a wave of interest in personal computing. Never mind that many novices bought an
Apple II without a clear idea of what to do with it… the mere concept of the average
Joe being able to own and operate a “personal computer” was catching people’s
imaginations.
WILL SOMEONE PLEASE TELL ME WHAT A PERSONAL COMPUTER CAN DO? -- 1982
Apple Computer ad
Which brings us to Jobs’ infamous trek to Xerox’s PARC facility. Actually, we need to
look further back in time to set the stage for Jobs’ visit.
The 40s - GUI Forefathers: Bush and Engelbart
Let’s back up to 1945 (!) and a visionary named Vannevar Bush. Bush, a scientist
and futurist, went public with his ideas of the “memex,” a computing device that
would use what we’d call hyperlink technology to bring information to every user’s
fingertips.
Bush’s ideas sparked some visionary thinking in a scientist named Douglas
Engelbart. As early as 1962, while Jobs and Wozniak were still drinking Ovaltine and
watching Saturday morning cartoons in their jammies, Engelbart was creating
several items of interest to the personal computing crowd that would follow. He
invented the first “mouse,” which he called an “X-Y Position Indicator,” a little gizmo
housed in a wooden box on wheels that moved around the desktop and took the
cursor with it on the display. Engelbart saw the mouse as being an integral part of a
“graphical windowed interface,” and invented what he called "a windowed GUI" that
fascinated co-workers but wasn’t considered useful outside the lab. In 1968
Engelbart created NLS (oNLine System), a hypermedia groupware system that used
the mouse, the windowed GUI, hypermedia with object addressing and linking, and
even an early version of video teleconferencing to wow its audience, a group of
technicians, engineers, and scientific types at Stanford University.
However, Engelbart was not the only visionary in the history of GUI. In 1963 a grad
student at MIT, Ivan Sutherland, submitted as his thesis a program called
“Sketchpad,” which directly manipulated objects on a CRT screen using a light pen.
"Sketchpad pioneered the concepts of graphical computing, including memory
structures to store objects, rubber-banding of lines, the ability to zoom in and out on
the display, and the ability to make perfect lines, corners, and joints. This was the
first GUI (Graphical User Interface) long before the term was coined." –- from a Sun
Microsystems biography of Ivan Sutherland
The idea of direct manipulation of objects on a screen is integral to the concept of a
graphic interface. In fact, the idea of a GUI derives from cognitive psychology, the
study of how the brain deals with communication. The idea is that the brain works
much more efficiently with graphical icons and displays rather than with words –
words add an extra layer of interpretation to the communication process. Imagine if
all the road signs you saw were uniform white rectangles, with only the words
themselves to differentiate the different commands, warnings, and informational
displays. When the “Stop” signs hardly look different from the “Resume Highway
Speed” signs, the processing of the signs’ messages becomes a slower and more
difficult process, and you’d have even more wrecks than you have now.
Combine this with Alan Kay’s concept of “biological computing,” where computer
components function like organic “cells,” either independently or in concert
whenever appropriate, and you have an idea of the thinking behind both modern
computing, and the GUI.
The 70s - SmallTalk and Xerox
"The best way to predict the future is to invent it." -- informal PARC slogan
The underground buzz stayed underground, but Engelbart’s and Sutherland’s
creations were not lost on the creative fellows at Xerox’s PARC facility. PARC was
(and is), at least in some respects, a computing “think tank,” where brilliant and
brilliantly erratic minds cranked out ideas and tried, with varying success, to
implement them on the workbench.
In the early 70s, as part of a (sadly abortive) project called “Dynabook” that
envisioned notebook-sized, hyperlinked computers, Alan Kay and others developed
an interactive object-oriented programming language called Smalltalk. Kay had
previously worked with a team at the University of Utah that developed a
programming system called Flex. This was a design for a flexible simulation and
graphics-oriented personal computer, with many ideas derived from the Norwegian-
developed Simula programming language, another programming language called
LISP, and Sutherland’s Sketchpad. Kay also borrowed ideas from a highly graphical
language called Logo, which was designed to teach programming to children.
Smalltalk featured a graphical user interface (GUI) that looked suspiciously similar to
later iterations from both Apple and Microsoft.
Smalltalk didn’t stop with an innovation in user interface: it featured a multi-platform
virtual machine years before the folks at Sun came up with Oak/Java, object
orientation, overlapping “windows,” and the first instance of bit-blt or "bit-blitting,"
the last two contributed by Dan Ingalls (the object-oriented language featured in ST
actually showed up in the Simula-67 program in the late 1960s; “bit-blitting,” or bit
block transfer, is, in simplistic terms, the protocol by which objects on a screen can
be manipulated). A lot of observers feel that ST’s clean, easy-to-use interface has yet
to be surpassed even today. The first program to be written under Smalltalk was
Pygmalion, which is most notable for its demonstration that computer programming
could be graphically based and not restricted to text. The idea of using icons to stand
for data was reflected in Pygmalion.
The first real-life, usable GUI appeared in Xerox’s Alto computer, which debuted in
1974 and was envisioned as a smaller, much more portable replacement for the
mainframes of the time. The Alto, which didn’t have a GUI as you and I are used to
using, but instead featured graphically driven applications, was about the size of a
Volkswagen (well, not quite, but the thing was big) and certainly not useful for the
average user, even though it started its life showing an image of Sesame Street’s
“Cookie Monster.” The Alto featured a bit-mapping display, which was essential for
displaying graphics and WYSIWYG printing. Kay, David Canfield Smith, Bill Verplank,
and others also developed iconic representations for various programs for the Alto,
most noticeably the drawing program “Markup,” the text editor “Bravo,” and the
painting program “Superpaint.”
In 1981, the design and concepts which gave birth to the Alto led to the development
and production of the much more streamlined, and more usable Xerox Star – the first
true GUI-driven PC. According to Bruce Horn, an ex-Xerox employee who wound up
working for Apple, the software architecture for Smalltalk and the Star were much
more sophisticated than the Mac or Windows equivalents. While the Apple machines
incorporated much of Xerox’s brainstorms, many of the most innovative and
sophisticated ideas never made it into the Apples, mostly due to Apple’s insistence
on keeping costs down. The Star featured the first “computer desktop,” as well as
overlapping, resizable windows, and the sophisticated PARC mouse, a gee-whiz
gizmo that ran with no moving parts and used laser beams and a metal grid to track
the cursor’s movement (though employees found that the mouse worked just as well
on Levis as it did on the metal grid). The interface was known as WIMP – Windows,
Icons, Menus, and Pointers. PARC’s consensus was that once these ideas were
implemented on a wide scale, computing efficiency would increase dramatically.
1979 - Apple Visits PARC
Jef Raskin, a project manager with Apple, first told Jobs and Wozniak about the
research being done at PARC. It’s a mistake to envision this scene as taking place in
some deserted parking garage, with Raskin hiding in the shadows and doing his best
Deep Throat impersonation. A closer scenario is that Raskin wanted to work more
directly on a GUI, and dropped a bug in Jobs’ ear about the neato keeno work being
done at PARC. Jobs was reluctant to go at first, but eventually Raskin, who wrote his
master’s thesis on a WYSIWYG graphical interface back in 1967 and was seeing some
of his ideas brought to fruition by the folks in PARC, piqued his interest.
At any rate, Jobs, who was first told by Raskin about the fun going on at PARC in
1976, decided that he wanted to bring a team of Appleniks into PARC and see what
was causing such a buzz – but again, the idea of Jobs coming in like a kid touring
Epcot with a tape recorder hidden under his shirt is mistaken. Apple negotiated a
deal with Xerox; in return for a block of Apple stock, Xerox allowed Jobs and his team
to tour PARC in December 1979, take notes, and implement some of the ideas and
concepts being bounced around at PARC in their own creations. I’m not sure how
Xerox felt about Apple subsequently hiring half – perhaps the better half – of PARC’s
staff away from them, but the process was relatively above-board; no night
kidnappings or bribes under the table at Jack In the Box. Xerox allowed Apple to use
their ideas in their machines. As Wozniak says on his Website, “Steve Jobs made the
case to Xerox PARC execs directly that they had great technology but that Apple
knew how to make it affordable enough to change the world. This was very open. In
the end, Xerox got a large block of Apple stock for sharing the technology. That’s not
stealing outright.”
"The reason why Jobs got the reputation of being so brilliant in human-centered
computing is because he neglected to tell anyone at PARC that his perceptive
questions about GUIs and so on were drawn from his discussions of such things with
Raskin at Apple a month or two earlier. He masterfully made it appear as though he
was encountering bitmapped GUIs for the first time in his life instead of having
discussed them with someone who had visited PARC himself." -- Neil Franklin
At any rate, Jobs and the Apple guys came back from their PARC tour with stars in
their eyes. They were entranced with the idea of a “windowing GUI” and loved the
flexibility and power of Smalltalk. They had a new vision, and were determined to
unleash it on the computing world ASAP. Development immediately began on the
Apple “Lisa.”