Académique Documents
Professionnel Documents
Culture Documents
Images In Java
Introduction
The AWT and how to put a simple image onto the screen Layout managers The Jimage Class,
At the end of this lecture you should have an idea of how to display a JPEG image on the screen, and how to load it into the Jimage class to carry out further image processing.
Agenda
AWT Images Image Producers and Consumers Jimage class Pixel Representations JPEG files
Overview
AWT abstract windows toolkit, supported by JavaSoft Operating system independent layer for windowing in Java Fiendishly obscure Designed around requirements of images being streamed off the web
Connections
Simple image display program to show how to display a JPEG file Pipeline model of image production Jimages act as image consumers Jimages allow arithmetic on image Jimages provide output to AWT images and JPEG
import java.awt.*; import java.awt.image.*; import java.util.*; class JPEGshow extends Frame { ... static public void main(String[] args) { if (args.length == 1) new JPEGshow(args[0]); else System.err.println("usage: java JPEGshow <image file>"); } } This is a standard Java Program class with a public static void main method
JPEGshow(String filename) { super("JPEG show Example"); add( new ImageCanvas(getToolkit().getImage(filename) ), BorderLayout.CENTER); setSize(700, 540); show(); }
The toolkit
Each frame has associated with it a toolkit object the provides an interface to OS specific operations.
CreateImage CreateMenu CreateLabel CreateMenuBar . etc
Constructor class ImageCanvas extends Component { the image Image image; ImageCanvas(Image image) {this.image = image;} public void paint(Graphics g) { g.drawImage(image, 0, 0, this);} }
just stores
Paint is called whenever a component must be shown, the Graphics object does the actual drawing, it has to be passed in because it is what knows about physically drawing on the screen
Image Class
Pipeline flow model of image processing Images are just tokens linking producers and consumers
ImageProducer
Image
ImageConsumer
ImageProducer Methods
addConsumer(ImageConsumer ic) This method is used to register an ImageConsumer with the ImageProducer for access to the image data during a later reconstruction of the Image. removeConsumer(ImageConsumer ic) This method removes the given ImageConsumer object from the list of consumers currently registered to receive image data. startProduction(ImageConsumer ic) This method starts an immediate reconstruction of the image data
ImageConsumer methods
void setDimensions(int width, int height) The dimensions of the source image are reported using the setDimensions method call. Void setPixels(int x, int y, int w, int h,
ColorModel model, byte[] pixels, int off, int scansize)
The pixels of the image are delivered using one or more calls to the setPixels method.
ImageProducer
Image
ImageConsumer
Image.getSource
Images contain a pointer to their producer which holds the actual data for the image. This can be recovered using the getSource method. This allows a consumer to get at the pixel data of an image by adding itself to the producer and starting production
Summary
AWT is operating system independent Streaming image model Images as tokens Producer - consumer pipeline See chapters 6 of textbook
Standard AWT images are just tokens for data streams. A BufferedImage actually contains the data.
BufferedImage
Raster
Colour model
JPEGCodec class
This class has factory methods to create JPEG encoders and decoders:
createJPEGDecoder(InputStream s) createJPEGEncoder(OutputStream d)
Read a BufferedImage
getRGB
alpha Bit 31
red
green
blue Bit 0
Writing pixels
This can be done with the setRGB method. This takes x, and y co-ordinates and a pixel encoded as a 32 bit integer im . setRGB(2, 5, 255);
Would set pixel 2,5 to 255 = bright blue.
You can create a sub area within a buffered image using the public BufferedImage getSubimage(
int int int int x, y, w, h);
Method of BufferedImage
Library of image processing classes developed in the department Available for student practicals Algebraic rather than stream oriented Interfaces to MMX hardware under windows
Algebraic orientation
By this we mean the it is structured around algebraic expressions whose values are images Thus if A and B are images and is some operator then
AB
is also an image
Jimage operators
Arithmetic I+J Universal plus(Universal) I-J Universal minus(Universal) IJ Universal times(Universal) IJ Universal divide(Universal) IUniversal abs() Filtering Jimage convolve(double[] k) convolve with symmetrical separable kernel. public abstract Jimage convolve(double[][] kernel)with non separable kernel
Scaling
This method allows the depth as well as area of an image to be altered if it is reduced the planes are aggregated if increased they are interpolated.
More operations
Data access int rgbpixel(int x,int y) Converts the plane information into a pixel in the direct color model of java. public abstract int upixel(int x, int y, int plane) - returns unsigned integer pixel public abstract float fpixel(int x, int y, int plane) Returns the pixel in the range -1 to +1.
Data Access
Outputs the image to a jpeg file public boolean getImage(java.lang.String fileName) Initialise the Jimage from the specified file. The file must be jpeg or gif.
Jimage implementations
An example program
class Jimageshow extends Frame { Create Jimage Jimageshow(String filename) { with byte pixels super("Jimage show Example"); Jimage raw=new ByteImage(100,200,3); if (raw.getImage(filename)){ Jimage cooked = (Jimage)raw.times(0.3); 0.3 Multiply by add(new ImageCanvas(cooked.getAWTImage()), BorderLayout.CENTER); setSize(700, 540); show(); } }
Pixel Representations
When dealing with displays it is conventional to assume that pixels are bytes holding numbers in the range 0 to 255. 0 Is assumed to be black 1 Is assumed to be white or maximum brightness of any given colour. For multicolour displays with 3 colour components, the convention is to have 3 fields of range 0..255 to hold the colour information.
For multicolour displays with 3 colour components, the convention is to have 3 fields of range 0..255 to hold the colour information. The AWT does this with the class Color. public Color(int rgb)
Creates a color with the specified RGB value, where the red component is in bits 16-23 of the argument, the green component is in bits 8-15 of the argument, and the blue component is in bits 0-7. The value zero indicates no contribution from the primary color component. A Jimage returns this format with int rgbpixel().
The byte data type in Java does not take on the values 0..255. Instead it takes on the values -128 to 127. There are no unsigned bytes in Java. This creates a problem for the representation of pixels in Jimages. The solution adopted is to adopt the following representation -128 = black 0 = mid grey 127 = white
If byte pixels are signed then so must other representations be. The solution adopted is to adopt the following representation for floats -1 = black 0 = mid grey 1 = white
unsigned
min value -1 maxval medianval m 0 range r 255 255 0 127 127.5 255
bytes
-128 2047 -0.5 4095
shorts float
-2048 1 -0.5 2
As shown in table a pixel prin representation r is converted to a pixel ps in representation s by the operation:
ps = ms+(rs(pr-mrrr
Signed pixels seem at first to be counter-intuitive but they have numerous advantages.
A value of 0 or mid grey can be viewed as the most likely value that a pixel takes on in the absence of other information. If you do arithmetic on images, in particular subtract one image from another, then negative values of pixels naturally arise. Signed pixels allow straightforward implementation of contrast adjustments. For instance multiplying an image by 0.5 halves the contrast in the image.
Signed pixels allow straightforward implementation of contrast adjustments. For instance multiplying an image by 0.5 halves the contrast in the image.
0.5 1
0.5
0.5
0.25
-0.25
-0.5
-1
Finalcontrast range
Image Multiplication
Image Addition
Image subtraction
What Is Convolution
Convolution takes a kernel of coefficients and multiplies each pixel in a neighbourhood by the corresponding coefficient, and then sums the result x y p[I+x, j+y]*k[x,y] Will give the convolved pixel at position i, j
1 D convolution
A 1 D convolution takes a one dimensional array as a kernel and applies it first in the X and then in Y dimension. This can often be performed faster than a 2d convolution
marble =
double[] k=
Note sum of coefficients =1
{0.1,0.1,0.2,0.2,0.2,0.1,0.1};
marble.convolve(k)=
marble =
double[] k= Note sum of coefficients =1 number terms is odd
{-0.3,1.6,-0.3}
marble.convolve(k)=
Convolution in Java2D
Java 2D provides a standard library for convolution of buffered images This uses the class Kernel and ConvolveOp
Kernels in JAVA 2D
float[] blur={ 0.0f, 0.1f, 0.0f, 0.1f, 0.6f, 0.1f, 0.0f, 0.1f, 0.1f}; Kernel k= new Kernel(3,3, blur); im = new ConvolveOp(K).filter(im,null); This will blur the image im by applying the 3 by 3 kernel blur to it.
Importance of speed
Image may contain a million pixels, Arithmetic may be required on each one Important to optimise operations or they are very time consuming May need to use assembler kernels May need to use special purpose instructions
Intel and other CPU manufacturers have been adding to the instruction sets of their computers new extensions that handle multi-media data. The aim is to allow operations to proceed on multiple pixels each clock cycle
MMX 2
8 General Registers
eax ebx ecx edx esp ebp esi edi
32 bit
64 bit
MMX 3
8 multimedia registers
mm0 mm1 mm2 mm3 mm4 mm5 mm6 mm7
32 bit
64 bit
MMX 4 motivation
Existing operating systems must still work unchanged Applications not using MMX run unchanged No new state added to the CPU Hence, shared use of the FP registers, since these are already supported by exising OSs
Problem of overflows
A problem with limited precision arithmetic is that overflows frequently occur. This can give rise to meaningless results: consider 200+175 = 375 but in 8 bit binary 11001000 +10101111 =101110111
Leading 1 is discarded
Using saturation
12: 00401043 00401046 00401048 0040104A 0040104D 0040104F 00401051 00401053 00401056 00401059 0040105C 0040105F 00401062 00401065
mov xor mov mov xor mov add mov mov add mov mov add mov
j=(int)(*p1++)+(int)(*p2++); ecx,dword ptr [ebp-4] edx,edx dl,byte ptr [ecx] eax,dword ptr [ebp-8] ecx,ecx cl,byte ptr [eax] edx,ecx dword ptr [ebp-14h],edx edx,dword ptr [ebp-8] edx,1 dword ptr [ebp-8],edx eax,dword ptr [ebp-4] eax,1 dword ptr [ebp-4],eax
Expansion 2
13: 14: 00401068 0040106F 00401071 00401078 0040107A 0040107D 00401080 00401083 00401086 15: 00401088 0040108B 0040108E
*p3 = (unsignedchar)(j>255?255:j); cmp dword ptr [ebp-14h],0FFh jle main+6Ah (0040107a) mov dword ptr [ebp-18h],0FFh jmp main+70h (00401080) mov ecx,dword ptr [ebp-14h] mov dword ptr [ebp-18h],ecx mov edx,dword ptr [ebp-0Ch] mov al,byte ptr [ebp-18h] mov byte ptr [edx],al p3++; mov ecx,dword ptr [ebp-0Ch] add ecx,1 mov dword ptr [ebp-0Ch],ecx
Speed Gain
On image of 256x256 pixels Old C code executes 26*256*256 instructions = 1,703,936 instructions Optimised mmx code executes 6*256*32 instructions = 49,152 Note that no compiler currently will give the optimised code. It has to be hand assembled.
Intel Provide an image porcessing library that can be downloaded from their web site. It provides efficient access to the MMX hardware. It provides frequently used Image Processing Operations. It requires a set of DLLs in your path to run
At the core of IPL is the ability to write to a single API and get the best possible results for any Intel processor. The libraries have as many as six processorspecific branches for each function and six sets of carefully written assembly code, but only one entry point to each function.
Use of Intel IPL complex and requires C I have provided 2 java classes that call the IPL. IntelBImage and IntelFImage. These are documented in the Jimage web pages. They inherit from ByteImage and FloatImage To use them the Intel IPL must have been installed on your machine and be on the path. If you are forced to use Unix machines the libraries will not be available to you.
Feedback
What did you not understand here? What would you like more information on?