Académique Documents
Professionnel Documents
Culture Documents
TO
COMPUTER
GRAPHIC S
Introduction
September 4, 2008
Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
(1/2)
Computer graphics generally means creation, storage and manipulation of models and images Such models come from diverse and expanding set of fields including physical, mathematical, artistic, biological, and even conceptual (abstract) structures
Frame from animation by William Latham, shown at SIGGRAPH 1992. Latham uses rules that govern patterns of natural forms to create his artwork.
September 4, 2008
Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
(2/2)
William Fetter coined term computer graphics in 1960 to describe new design methods he was pursuing at Boeing Created a series of widely reproduced images on pen plotter exploring cockpit design, using 3D model of human body.
Perhaps the best way to define computer graphics is to find out what it is not. It is not a machine. It is not a computer, nor a group of computer programs. It is not the know-how of a graphic designer, a programmer, a writer, a motion picture specialist, or a reproduction specialist. Computer graphics is all these a consciously managed and documented technology directed toward communicating information accurately and descriptively. Computer Graphics, by William A. Fetter, 1966
September 4, 2008
Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
First truly interactive graphics system, Sketchpad, pioneered at MIT by Ivan Sutherland for his 1963 Ph.D. thesis
Sketchpad in 1963. Note use of a CRT monitor, light pen and function-key panel.
Andries van Dam September 4, 2008 Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
Card punching (left). IBM 704 (right) took up a whole room and was capable of about 4,000 arithmetic operations/second. Cool facts: Whirlwind, built in early 50s at MIT, cost $4.5 million and could perform 40,000 additions/second. Mac 512K, list price $3,195 in 1984, could do 500,000. Today, commodity PCs perform approximately two or three billion operations/second.
Andries van Dam September 4, 2008 Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
Today, still use batch mode for final production-quality video and film (special effects fx), where one frame of a 24 fps movie may take 8-24 hours to render on fastest PC!
Render farm
September 4, 2008
Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
Environmental (R)evolution
Graphics has been key to technology growth in evolution of computing environments:
graphical user interfaces (GUIs) visual computing, e.g., desktop publishing, scientific visualization, information visualization
(1/6)
Apple iPhoneTM
Radeon series)
Leapfrog PadTM
September 4, 2008
Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
Environmental (R)evolution
Character Displays (1960s now)
(2/6)
Display: text plus alphamosaic pseudo-graphics Object and command specification: command-line typing Control over appearance: coding for text formatting (.p = paragraph, .i 5 = indent 5) Application control: single task
September 4, 2008
Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
Environmental (R)evolution
Vector (Calligraphic, Line Drawing) Displays (1963 1980s)
(3/6)
Display: line drawings and stroke text; 2D and 3D transformation hardware Object and command specification: command-line typing, function keys, menus Control over appearance: pseudo-WYSIWYG Application control: single or multitasked, distributed computing pioneered at Brown via mainframe host <-> minicomputer satellite
Term vector graphics survives as scalable vector graphics library from Adobe and W3C shapes as transformable objects rather than just bitmaps
September 4, 2008 Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
Environmental (R)evolution
(4/6)
2D bitmap raster displays for PCs and workstations (1972 at Xerox PARC - now)
Display: windows, icons, legible text, flat earth graphics
Note: late 60s saw first use of raster graphics, especially for flight simulators
Object and command specification: minimal typing via WIMP (Windows, Icons, Menus, Pointer) GUI: pointand-click selection of menu items and objects, widgets and direct manipulation (e.g., drag and drop), messy desktop metaphor Control over appearance: WYSIWYG (which is really WYSIAYG, What You See Is All You Get) Application control: multi-tasking, networked clientserver computation and window management (even X terminals)
September 4, 2008
Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
Environmental (R)evolution
(5/6)
September 4, 2008
Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
Environmental (R)evolution
(6/6)
Classical time-sharing is dead, 1:n g n:1 PCs and Workstations merging in distributed heterogeneous computer networks (e.g., LANs, WANs, Internet and clusters) But file-, print- and compute-servers and network are still shared Client/server computing, component software technologies are dominant paradigms NCs (Network Computers), thin clients attached to powerful servers reprise dumb terminals and provide central control havent really taken off
September 4, 2008
Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
Versus
Original Macintosh
Date Price CPU Memory
Storage
x 1250000
Monitor
Devices
GUI
September 4, 2008
Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
Multimedia: text and graphics synchronized with sound and video Hypermedia: multimedia with hypertextual links (also called Interactive Multimedia) Discredited term: Digital Convergence, merging of digital television and distributed computing, consumer electronics: set-top computers (e.g., for Interactive TV, Video-On-Demand) The Internet and Internet appliances Embedded computing (information appliances, Personal Digital Assistants) Ubiquitous/pervasive/ invisible/nomadic computing, active badges a la Xerox PARC, with hundreds of devices per person (appliances, clothing, even body parts) ; seamless computing is the dream
September 4, 2008
Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
CAVE
Andries van Dam September 4, 2008 Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
September 4, 2008
Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
September 4, 2008
Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
September 4, 2008
Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
Cave Painting
INTRODUCTION
TO
COMPUTER
GRAPHIC S
September 4, 2008
Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
Video or optics superimposes computergenerated data on real world (e.g., Columbias MARS, led by our Ph.D., Steve Feiner)
September 4, 2008
Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
special
tablet PCs and electronic whiteboards present new opportunities for pen-centric, gesture-based computing Multi-touch (NYUs Jeff Han, MS Surface, Apples iPhone)
Social interfaces (Microsofts Bob and Clippie bombed, other avatars may not) Agents for indirect control
September 4, 2008 Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
CPU
Server: Dual-Core Intel Itanium 2, Dual-Core Intel Xeon, Dual-Core AMD Opteron, Sun UltraSPARC T1 Desktop: Intel Core 2 Duo, AMD Athlon64 X2, IBM G5, Mac ProTM Quad/8-Core
Commodity cards have taken over the mainstream market (nVidia GeForce, ATI Radeon) nVidia (formerly Ageia) PhysX PPU (Physics Processing Unit) Offloads physics processing from the processor and GPU AIseek Intia Processor Accelerated path finding, sensory simulation and terrain analysis
September 4, 2008 Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
ATI Radeon HD 4800 and nVidia GeForce GTX 280 are the top graphics cards in 2008 (approx. $200 - $300) Significant advances in commodity graphics chips every 6 months, outrunning CPU chip advances
AMD Athlon 64 X2 processors have 243 million transistors Radeon HD 4800 chip has over 900 million!
GPUs are now arguably more powerful than CPUs and programmable CPU & OS losing their hegemony as the GPU co-processor tail wags CPU dog! GPUs sometimes used instead of CPUs in clusters for high-performance computing (e.g. bioinformatics algorithms) Workstation market (and vendors) bowing to commoditization (e.g., Suns losing client market)
September 4, 2008 Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
Application Distinctions
Two basic paradigms
Sample-based graphics (left): discrete samples are used to describe visual information
pixels can be created by digitizing images, using a samplebased painting program, etc. often some aspect of the physical world is sampled for visualization, e.g., temperature across the US example programs: Adobe Photoshop, GIMP , Adobe AfterEffects (it came out of CS123/CS224!)
Geometry-based graphics (right): (also called scalable vector graphics or object-oriented graphics) geometrical model is created, along with various appearance attributes, and is then sampled for visualization (rendering a.k.a image synthesis);
often some aspect of physical world is visually simulated, or synthesized examples of 2D apps: Adobe Illustrator, Adobe Freehand (formerly by Macromedia), Corel CorelDRAW examples of 3D apps: Autodesks AutoCAD2009, Autodesks (formerly Alias|Wavefronts) Maya, Autodesks (formerly Discreets) 3D Studio Max
September 4, 2008
Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
Sampled-based Graphics
(1/2)
1 pixel Mathematical pixel grid LCD display NB: Cant resolve adjacent pixels on CRT
Pixels are point locations with associated sample values, usually of light intensities/colors, transparency, and other control information. They are not little circles or little squares
September 4, 2008
Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
Sampled-based Graphics
(2/2)
Samples created directly in paint-type program, or as sampling of continuous (analog) visual materials. E.g., photograph can be sampled (light intensity/color measured at regular intervals) with many devices including:
flatbed and drum scanners digital still and motion (video) cameras add-on boards such as frame grabbers
Sample values can also be input numerically (e.g., with numbers from computed dataset) Once an image is defined as pixel-array, it can be manipulated
Image editing: changes made by user, such as cutting and pasting sections, brush-type tools, and processing selected areas Image processing: algorithmic operations that are performed on image (or pre-selected portion of image) without user intervention. Includes blurring, sharpening, edge-detection, color balancing, rotating, and warping. Preprocessing step in computer vision
September 4, 2008
Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
Sampling an Image
Lets do some sampling of CIT building
3D scene
A color value is measured at every grid point and used to color corresponding grid square
0 = white, 5 = gray, 10 = black
Note: this poor sampling and image reconstruction method creates blocky image
September 4, 2008
Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
Pixel information from one image can be copied and pasted into another, replacing or combining with previously stored pixels
September 4, 2008
Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
But recently, strong interest in image-based rendering to fake 3D scenes and arbitrary camera positions. New images constructed by interpolation, composition, warping and other operations.
September 4, 2008
Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
The news: digitally enhanced images are more and more common. Usually just sharpening, color balancing, but sometimes much more. To Photoshop something in:
Time Magazines O.J. Simpson cover (below left) Reuters doctored Beirut bombing photo (below right; note cloned smoke) National Geographics Pyramid cover University of Wisconsin application Faked NE blackout satellite photo
September 4, 2008
Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
The belief in very strong connection between photorealistic images, still or moving, and reality is being severed
no way to tell if news photos are real photographs photographic evidence no longer considered proof in court of law without clear provenance of the image future of the family photo album?
Andries van Dam September 4, 2008 Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
CS123 emphasizes geometry-based graphics but we do two assignments on sample-based graphics to learn simple image processing.
image processing in general includes image transformation: used for feature detection, pattern recognition, machine/computer vision, and most recently, for image-based rendering new field of "digital photography" (a.ka. computational photography, fauxtography) brings us many neat tricks with images, some now even done in camera
September 4, 2008
Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
Geometry-Based Graphics
Geometry-based graphics applications store mathematical descriptions, or models, of geometric elements (lines, polygons, polyhedrons) and associated attributes (e.g., color, material properties). Elements are primitive geometric shapes, primitives for short Images created as pixel arrays (via sampling of geometry) for viewing, but not stored as part of model. Images of many different views are generated from same model Users cannot usually work directly with individual pixels in geometry-based programs; as user manipulates geometric elements, program resamples and redisplays elements Increasingly rendering combines geometric and sample-based graphics, both as performance hack and to increase quality of final product
September 4, 2008
Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
Modeling is coping with complexity Our focus: modeling and viewing simple everyday objects Consider this:
Through 3D computer graphics, first time in human history we have abstract, easily changeable 3D forms. This has revolutionized working process of many fields science, engineering, industrial design, architecture, commerce, entertainment, etc. This has profound implications for visual thinking and visual literacy
September 4, 2008
Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
Lecture Topics
We manipulated primitive shapes with geometric transformations (translation, rotation, scale). These transformations are essential for model organization, process of composing complex objects from simpler components. Hierarchical models and same geometric transformations are also essential for animation Once objects geometry is established, must be viewed on screen: map from 3D to 2D for viewing and from 2D to 3D for 2D input devices (e.g., the mouse or pen/stylus) While mapping from 3D to 2D, object (surface) material properties and lighting effects are used in rendering ones constructions. This rendering process is also called image synthesis
September 4, 2008
Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
Head
Shaft
Point
composition
decomposition
September 4, 2008
Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
Nail Head Body (cylinder) Shaft Point (cylinder) (cone) tree diagram
root node
leaf nodes
Such diagrams are part of 3D program interfaces (e.g., 3D Studio MAX, Maya) As pointer data structure to be rendered, it is called a scenegraph
September 4, 2008
Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
Translate and Rotate Primitives in their own modeling coordinate system Composition in world (root) coordinate system
Primitives created in decomposition process must be assembled to create final object. Done with affine transformations, T, R, S (as in above example). Other composition operators will be briefly discussed later (e.g., Constructive Solid Geometry (CSG) with Boolean operators).
Andries van Dam September 4, 2008 Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
x, y Cartesian grid
Integer Grid
NB:Often, screen coordinates are measured top to bottom, based on raster scan
(0,0)
September 4, 2008
Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
Example 3D Primitives
Polyline Polyhedron
Sphere
Patch
September 4, 2008
Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
Application model
Application program
Graphics System
September 4, 2008
Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
Graphics Library
Examples: OpenGL, DirectX, Windows Presentation Foundation Primitives Attributes
color line style material properties for 3D
September 4, 2008
Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
2D Hardware/Algorithms Outline
Display Hardware: Raster scan vs. random scan
Color Table
indirect specification of (pseudo) color color correction, simple types of animation
INTRODUCTION
TO
COMPUTER
GRAPHIC S
Ideal Drawing
Vector Drawing
Raster
INTRODUCTION
TO
COMPUTER
GRAPHIC S
Vector Architecture
Vector display works with display list/file stored in refresh buffer Display controller draws all vectors at < 60 Hz (often at variable rate); flicker is a problem
September 4, 2008 Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
2D Raster Architecture
Raster display stores bitmap/pixmap in refresh buffer, also known as bitmap, frame buffer; can be in separate hardware (VRAM) or in CPUs main memory (DRAM) Video controller draws all scan-lines at consistent > 60 Hz; separates update rate of the frame buffer and refresh rate of the CRT
September 4, 2008
Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
Drawing Lines
For horizontal, vertical and diagonal lines all pixels lie on ideal line: special case For lines at arbitrary angle, pick pixels closest to ideal line (Bresenhams midpoint scan conversion algorithm) For thick lines, use multiple pixels in each column or fill a rotated rectangle
September 4, 2008
Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
2. 3.
September 4, 2008
Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
circle outline
filled ellipse
September 4, 2008
Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
Thickness Attribute
Thick circle drawing by tracing rectangular pen
Treat such primitives as regions and fill them, e.g., approximate them as sequences of connected quadrilaterals
September 4, 2008
Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
Drawing Characters
(1/2)
selectively write pixels of refresh buffer corresponding to true bits of character bitmap
Transparent mode: dont write 0s BitBlt with OR (explained later) Opaque mode: write background color for 0s Descenders & proportional spacing easily accommodated
Base line
September 4, 2008
Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
Drawing Characters
(2/2)
Outline fonts: defined in terms of (mathematical) drawing primitives (lines, arcs, splines) and thus scalable, but more CPU intensive (e.g. Adobe PostScript, Microsoft TrueType)
Font design (typography is highly skilled specialty, involving graphical and algorithmic design)
September 4, 2008
Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
Digital intensity value Digital to Analog Conversion (DAC) analog signal to drive electron beam Black & White (or any 2 colors, depending on monitor phosphor color) Original Mac resolution was 512x384 pixels, now from 640x480 up to 2560x1600
n-Bit Display
September 4, 2008
Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
Any specific 2n colors may be inadequate (n typically 16-24 in low-end systems) Look-up table allows 2n colors out of 224 colors to be used in one image, some other 2n in another image
224 = approx. 16.7 million, exceeds eyes ability to discriminate (somewhere between 7-10 million)
INTRODUCTION
TO
COMPUTER
GRAPHIC S
Pixel value is indexed to color look up table (CLUT) where color is stored. Here we use only 12 bits (4bits per color) for clarity typically, 24 bits are used CLUT look up done at video rates, overlapped with fetch and DAC! In 24-bit true color systems, 3 x 8 bits for R, G, B; each color has its own 8-bit CLUT (0-255) CLUT allows variety of effects
pseudo coloring (LandSat images, stress diagrams, thermograms) fast image changes: change table rather than stored image multiple images: select or composite/blend animation hack
September 4, 2008
Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
BitBlt/RasterOp
(1/3)
Logically operate on each pixel in rectangular source and destination regions in same or different pixmaps to achieve dynamics, e.g., to move/scroll windows on screen RasterOp (Source, Destination) g Destination In some implementations either S or D may be masked, and need not be same size
pixmap
September 4, 2008
Introduction #/59
INTRODUCTION
TO
COMPUTER
GRAPHIC S
BitBlt/RasterOp
(2/3)
AND (S,D): S can mask out pixels in D OR (S,D): S is non-destructively added to D; used for painting, transparent and kerned characters (where characters extend beyond their boxes); not as useful in n-bit systems
Lets say you want to add some game sprites to background 1. Mask out with black
AND
OR
INTRODUCTION
TO
COMPUTER
GRAPHIC S
BitBlt/RasterOp
(3/3)
Replace (S,D): S destructively replaces D, i.e., is deleted and copied on top of D (also called Move); used for making opaque characters, icons, scroll Copy (S,D) as above, but S is not deleted XOR (S,D) S selectively inverts D; used in 1-bit systems for rubber-banding/dragging, cheap cursors:
Note: effects in color systems for all but replace may be weird
September 4, 2008
Introduction #/59