Serialization

Why Object Serialization is Inappropriate
for Providing Persistence in Java

Huw Evans
Department of Computing Science
The University of Glasgow
Glasgow, G12 8RZ, UK
huw@dcs.gla.ac.uk
http://www.dcs.gla.ac.uk/huw
Abstract
This paper describes why Object-Serialization is not appropriate for providing persistence in Java. With numerous
code examples, Object-Serialization is shown to be easy to work with initially which seduces the developer into
relying on it for persistence within more complex applications. The advanced use of object-serialization requires
significant work from the programmer, something that is not apparent at first. The use of object-serialization together
with static and transient fields and within multi-threaded programs is discussed together with the big inhale
problem: the need to read in the entire object graph before processing over it can commence. The complexity
of using object-serialization within a distributed environment, when evolving classes and when using specialised
classloaders is also discussed. The paper compares the performance of serializing and deserializing a byte array and
binary tree of the same data size to and from an NFS mounted disk and two kinds of local disk. Alternative solutions
to object-persistence in Java are presented at the end of the paper.
1 Introduction
Since 1995, Java has become a very popular programming language, receiving a great deal of attention both in
academia and industry. One package that must be provided by all implementations of Java is the java.io package
[GJS96, pg 113]. When used in a certain way, this package can be used to provide a form of object persistence so
that a specified subset of the state of a program can outlive the lifetime of the process that created it. This is achieved
by serializing an object graph into a file on disk via an in-memory buffer. Serializing a graph of objects is the process
of flattening that graph into a form suitable to be written to disk or passed across a network. The reverse operation,
deserialization, is the act of taking the data from disk or the network and reconstituting a graph of the same shape.
Javas serialization technology can be used for other purposes such as passing object graphs by value in remote
method invocations. This paper only discusses the use of serialization for object persistence.
Serialization and deserialization are provided by the java.io package via the java.io.ObjectOutputStream
and java.io.ObjectInputStream classes. Figure 1, overleaf, contains the code1 to serialize a single object into a
file.
Line 6 opens a new file object to which the serialized object can be written. Line 7 uses this object when creating
a new ObjectOutputStream. Line 8 initialises the object we wish to serialize, line 9 serializes that object into the file
and line 10 closes the file.
In figure 2 (line 6) we first create an object connected to the underlying file. This is used to create an ObjectInputStream object (line 7) which will perform the deserialization of the objects in the file. Line 8 defines a String
object to hold the result of deserialization. Line 9 calls the readObject method on the ObjectInputStream object which
Parts
of this technical report have appeared in Java Report for October 2000 www.javareport.com.
handling code has been removed from the code fragments in this paper for brevity. They are included only when necessary for the
discussion.
1 Exception
import java.io.*;
2
3
4
5
6
7
public class Test

{
public static void main(String argv[])
{
FileOutputStream file = new FileOutputStream("./test");
ObjectOutputStream out = new ObjectOutputStream(file);
8
9
10
11
12
String s = new String("Will be serialized");

out.writeObject(s);
file.close();
}
}
Figure 1: Serializing an Object to a File
import java.io.*;
2
3
4
5
6
7
public class Deserialize

{
{
FileInputStream file = new FileInputStream("./test");
ObjectInputStream in = new ObjectInputStream(file);
String s = null;
s = (String) in.readObject();
10
11
12
13
System.out.println(s);
file.close();
}
}
Figure 2: Deserializing the Object from the File
deserializes the object in the file, creating a copy of it in memory. Line 10 outputs this String to show it has the same
contents as the serialized String. Line 11 closes the file.
The above code is very concise and easy to write. The programmer only needs three objects to perform object
serialization and deserialization to and from a file. The same code can also be used to make an entire object graph
persistent as the serialization mechanism works by transitive reachability. If the object passed to the writeObject
method contains references to two other objects, for example, the passed object and the two objects reachable from
it will be serialized. If either of these objects contain references themselves, the referenced objects will also be
serialized. This kind of example makes it look very easy to gain simple and cheap (in terms of programmer time)
object persistence.
A programmer may make use of this mechanism as a quick solution to a simple problem that needs object persistence. The problem has been solved quickly and easily; therefore, it is quite likely the code will kept by the
programmer. However, if this is the case, the programmer has been seduced by the apparent simplicity of the object
serialization system. The implications of relying on object serialization for object persistence are huge and the example above is so simplistic that it hides all the pitfalls that become apparent when using the mechanism on a larger
scale.
The rest of this paper argues that Javas object serialization mechanism should not be used to provide objectpersistence within production level code.
1.1
Document Structure
This paper is divided into thirteen main sections. Section 2 discusses in more detail how the serialization and deserialization mechanisms work and how the programmer uses them. Section 3 describes the problem of serializing
and deserializing a file as a single action. Section 4 shows why the copying semantics inherent in the serialization
mechanism can pose problems when using certain data structures, e.g. java.util.Hashtable. Section 5 discusses the
problem of having to reassign static variables. When a graph is sent to disk, a classs static variables are not written
and this state has to be recreated when a deserialization is performed. Section 6 describes why references marked
transient need to be treated in a similar way to statics, as transient variables are also not written by the serialization
mechanism. Section 7 describes the use of serialPersistentFields, an alternative mechanism for specifying which
fields in a class are passed to the serialization mechanism. Section 8 shows that using the serialization mechanism
in a concurrent environment can be problematic. For example, if one thread is used to perform the serialization,
another thread can change the contents of the graph being written, causing inconsistencies at the application-level.
Section 9 discusses the problems that can be raised when using serialization in a distributed environment. Serializing
a graph on the server can cause delay at clients waiting to perform remote method invocations. Section 10 builds
on section 9 to describe the problems that can occur when placing references to remotely invokable objects and the
remotely invokable objects themselves into persistent stores. When a client-side store that contains a reference to
a remotely invokable object is deserialized, the reference to the remote object is automatically re-established. This
will work if the remote object is listening on the port expected, but errors can occur if the remote object is listening
on an alternative port, because, for example, the server crashed and was restarted. Section 11 indicates how using
an application-specific Java classloader can cause problems. Java classes are not serialized to the file, and when the
graph is later deserialized, the required classes need to be accessible by the classloader that was used to load the deserialization code. If the required classes cannot be found, errors will occur. Section 12 describes how class evolution
can be performed and how this impacts instances already on disk when reading them within the context of an evolved,
related class. Section 13 describes some alternative object persistence technologies that do not rely on serialization.
Section14 concludes this paper.
The code fragments presented in this paper have all been tested with Suns Java 2 and the discussion of the
serialization system is made with respect to that version of the JDK and language definition. Java Object Serialization
shall be referred to as JOS for the rest of this paper.
2 Serialization and Deserialization

Sections 2.1 and 2.2 describe in more detail the use of serialization and deserialization as defined in Java version 1.2.
2.1
Serialization
Section 2.1 describes how a graph of objects is serialized into a file. Firstly, the model used by the serialization
mechanism is presented, followed by all the classes that are necessary to make use of the mechanism. This section
ends with a discussion of how to perform specialised serialization of a graph.
2.1.1 Serialization Model

Serialization in Java is based on reachability and the programmer has to identify, at compile time, those instances that
may be made persistent. Reachability means that all objects reachable from a given object referred to in the rest
of this paper as the root of persistence will be serialized into the file. For example, consider the two objectt graphs
in figure 3 that reside in the same virtual machine. If the object o of type O is passed to the serialization mechanism,
all the objects reachable from it will also be serialized. Therefore, objects o, p, q, r, s and t will be serialized.
As neither objects u nor v are reachable from the o rooted graph, they will not be written to the file.
The serialization mechanism handles cyclic graphs. As each object is visited, a note of it is taken. Should that
object be visited again, it is because there is a cycle in the graph, e.g. between p and s in figure 3. On visiting an
object for the second time, the mechanism knows the object has already been serialized and so only has to put enough
information into the serialized form so that the cycle can be rebuilt when the data is deserialized.
o:O
p:P
r:R
u:U
q:Q
s:S
v:V
t:T
Figure 3: Two Object Graphs

The programmer indicates that an instance can be made persistent by having its class implement the
java.io.Serializable interface. For example, for object o to be serializable, the definition of class O would need
to be that in figure 4.
public class O implements java.io.Serializable
{
// Class definition
}
Figure 4: Making a Classs Objects Serializable

The java.io.Serializable interface does not define any methods. It is used to instruct the serialization mechanism that an instance of a class can be serialized. This approach is not orthogonal to type as only those classes that
implement java.io.Serializable can be serialized. If a class does not implement this interface, and an attempt is
made to serialize an instance of it, a run-time exception will be thrown and the serialization of the entire graph will
be aborted.
It does not make sense to serialize instances of some classes, for example, objects that represent communication
sockets. In this case, the class (and all its supertypes) will not implement java.io.Serializable. Although this
gives the programmer the choice of whether instances of a class can be serialized or not, it is expressed at the sourcelevel and so can only be changed by re-compiling the class. Other persistence mechanisms, such as some of those
discussed in section 13, allow this decision to be made more flexibly at run-time.
By default, all non-transient and non-static fields of a class that implements java.io.Serializable will be
serialized. The programmer can override this default in one of two ways: by defining readObject and writeObject
methods on the class (section 6); or by defining a special field, serialPersistentFields, in the class (section 7).
As an alternative, use of another interface, java.io.Externalizable, gives the programmer complete control
over the format and contents of the stream an object is written to. To make use of this flexibility, the programmer is
required to define the format of the stream and manage how data is read and written. The programmer must provide
code to ensure an object and its super-types are handled correctly. Classes that implement the Externalizable interface
must provide a public no-argument constructor. This is because when an Externalizable object is reconstructed, an
instance is created using that constructor and then the readExternal method is called. The rest of this paper does
not discuss the use of the Externalizable interface for providing ad-hoc persistence. Its use is highly applicationdependent and it requires more effort to use than the Serializable interface. However, everything that applies to the
use of the Serializable interface also applies to the Externalizable interface.
2.1.2 Classes Required for Serialization

To perform a serialization, the programmer must use the java.io package and make use of
java.io.ObjectOutputStream to serialize the object graph to an underlying stream. To provide object persistence,
the destination for the serialized graph will be a file, which will be referred to as a store in the rest of this paper.
Object graphs are serialized by calling the writeObject method; in addition there are methods for writing primitive
data types, such as, writeInt and writeFloat. There are other methods that allow the programmer to control access
to the underlying stream, such as drain and close. For a complete description of the java.io package, see [PC98].
2.2
Deserialization
This section describes object deserialization: the act of reconstituting a graph of objects from its serialized form. This
section describes the classes necessary to perform deserialization and gives a detailed example of how it is performed
together with a discussion of the run-time requirements and its implications.
2.2.1 Classes Required for Deserialization

To perform deserialization the programmer needs to use the java.io package and make use of the
java.io.ObjectInputStream to deserialize the data in the serialized stream and rebuild the object graph. Objects
are read by calling the readObject method on java.io.ObjectInputStream once it has been connected to the
underlying store. Primitive data types can be read using readInt and readFloat, for example, and there are other
methods that allow the programmer to control access to the stream, such as skipBytes (skips a nominated number of
bytes in the stream) and resolveClass (allows the programmer to load classes from another source). For a complete
description of the java.io package and the ObjectInputStream class, see [PC98].
2.2.2 Performing a Deserialization

When deserializing a stream of objects, the programmer must: ensure that all the classes of the objects in the graph
are available to the virtual machine; and, initialise any static and transient fields. The versions of the classes must also
match, but this discussion is left until section 12 on the evolution of a store.
Given the graph in figure 3, the code in figure 5 will deserialize it into a graph of the same shape. The exception
handling code has been left in as this is pertinent to the discussion of deserialization.
1
ObjectInputStream in = null;
2
3
4
5
6
7
try {
in = new ObjectInputStream(new FileInputStream("./store"));
} catch(java.io.IOException ioe) {
ioe.printStackTrace();
System.exit(1);
}
O o = null;
9
10
11
12
13
14
15
16
17
try {
o = (O) in.readObject();
System.exit(1);
} catch(java.lang.ClassNotFoundException cnfe) {
cnfe.printStackTrace();
System.exit(2);
}
18
System.out.println(o);
Figure 5: Deserializing a Stream of Objects
Lines 1 to 7 define the ObjectInputStream object that will be used to read the serialized form of the data. The
ObjectInputStream is connected to the underlying file (called ./store) by passing a new FileInputStream to the
constructor of the ObjectInputStream at line 3.
Line 8 defines an object of the correct type to receive the root of the graph. Line 10 reads from the store and
deserializes the graph given on the lefthand side in figure 3, assigning the root to the variable o. Line 18 then prints
out the object to confirm it is the object originally written.
2.3
Comment
Requiring the programmer to specify whether a class implements java.io.Serializable burdens them with the
problem of deciding whether its instances should be capable of being made persistent. Inevitably, the programmer is
going to have to change code when the persistence of previously transient objects is required. This assumes that the
source code or a bytecode rewriting tool is available and that changing either code to add java.io.Serializable
does not change some other aspect of the application. For example, having a class implement
java.io.Serialization allows instances of it to leave the virtual machine that created them2 and this may not be
realised when introducing Serializable for persistence. This point is further discussed in section 10.2.3 within the
section on combining this form of persistence with distribution.
3 The Big Inhale and Exhale Problem

When a program writes an object to a store, the entire stream of bytes is serialized, all the defined writeObject
methods are called, and the stream of bytes is written to the file. This procedure is performed in its entirety and only
then does the writeObject method of ObjectOutputStream return. Deserializing the store has a similar effect on
the program. The stream of bytes in the file are reconstituted into a graph with the same pre-serialization shape and
the root object is passed back to the program. This method does not return until the entire file has been deserialized or
an error is encountered. This need to wait on the reading and writing of the store is known as the big inhale (exhale)
problem3 .
This model of reading from and writing to the store can have a significant impact on the process doing it. The
serialization and deserialization methods are synchronous; they only return once the entire graph has been moved to
or from disk.
In the case of deserializing a store, a process must wait for the entire graph to be read. This is because the
program may make use of an object whose state will be initialised from the store. For example, a multi-threaded
program may start a thread that relies on a store-resident object. This thread will have to contain code to check if
the deserialization has been completed, before it may proceed. It is possible for the thread that accesses the graph
to wait on the deserialization thread so that it only gains access once the graph is in place. However, this requires
the programmer to use Javas wait and notify methods, thus building in additional complexity into the program;
complexity which is required because of the use of object-serialization-based persistence.
If the programmer were to allow access to objects and their methods as soon as they had been read from the store,
programs may unexpectedly crash as one object that has been read may refer to another object that has yet to be read.
Therefore, the successfully read object may not contain a reference to the unread object, possibly causing the program
to fail with a java.lang.NullPointerException.
4 Copying Semantics
The JOS mechanism is based on the copying of objects to and from a stream. In certain circumstances, taking a copy
of an object can break code. This section describes this problem with the aid of the code in figures 6 and 7.
Figure 6 defines two classes, Key and Value. An instance of each will be put into the hash table that is created in
the main method in figure 7.
In main we create a new Hashtable object called table. Single key and value objects, k and v respectively, are
created and v is added to the table, using k as the key. The value stored under k is then retrieved using table.get(k)
and displayed on standard output. As Value overrides the toString method, the string I am a Value is displayed.
The table object is then passed to the serialize method, which serializes the object into a file ./test and closes
it. deserialize() opens ./test and reads the root object from the file, passing it back. The code in main casts the
object to a Hashtable, assigning it to new table. The new table contains the same information as table, as can be
2 The
3 This
objects may leave via a socket or by using RMI, for example.

terminology is due to Malcolm Atkinson.
package copying;
package copying;
public class Key implements

java.io.Serializable
{
public String toString()
{
return "I am a Key";
}
}
public class Value implements

{
{
return "I am a Value";
}
}
Figure 6: Key and Value Classes

seen by comparing the output when printing the table to standard output in figure 8. The method new table.get()
is called, passing in the original key (k). However, this time, null is printed instead of I am a Value.
This is because the implementation of Hashtable uses the memory location of the key object in part of its
hashing algorithm. When the new hash table is created from the stream, the copies of the key and value objects
brought from the stream are at different memory locations to the original key (k). The readObject method defined
on java.util.Hashtable calls the classes put method to associate the key with the value read from the store.
The deserialized key and value are at different memory locations and thus their hash code values, as returned by
hashCode() defined on java.lang.Object, are different. As the original key, k, is at a different memory location
to the new key and value, k cannot be used in a lookup on v. The lookup using it fails; get returns null.
Given the above definition of Key, there is no way of retrieving v from new table using get with k. The
programmer has to use some other means, e.g. calling keys to return an enumeration of all the keys in the table
and iterating to find the one required.
An alternative solution requires the programmer to ensure that Key is immune to being moved around in memory
so that it will always work when passed to get. To do this, the programmer must override the equals and hashCode
methods as shown in figure 9.
This new implementation of Key holds its own hash code as part of its state, initialising it in its constructor.
hashCode returns this value and equals returns true if the hash code values for the current object and the argument
are equal and false otherwise.
Running Test with the new definition of Key generates the output in figure 10.
This time, the value has been successfully retrieved. This is because the hashcode value is part of the state that is
serialized. Thus, when the object is brought from the stream and is put into the hash table, hashCode will return the
same value used before serialization. When get is called on the new table using k, the value will be found because
the object has been placed into the hash table using the hash code in the object, which is immune to the object being
moved around in memory. The equals method must also be overridden to ensure that the object found at this location
is indeed the object that is expected as more than one object may hash to the same location.
4.1
Comment
This problem has happened because the serialization mechanism copies objects, and the implementation of Hashtable
assumes that keys and values remain in the same memory location. This assumption is false when we consider the
hash table in a persistent environment.
The solution adopted above requires the software engineer to build information into their class that allows instances to be used across multiple program executions. The hashCode and equals methods have to be overloaded to
ensure this information is returned so that two objects can be distinguished. This can add considerable complexity to
the code and requires the programmer consider the identity of certain objects, when the default equals method (as
used by Key in figure 6) should suffice.
5 Static References
The serialization mechanism does not serialize static or transient fields. If a field is marked as static, the
default behaviour is to write a default value, e.g. 0 for int and null for object references, when the object containing
those fields is written. This is because the static field is associated with the class and not an object of that class. The
serialization mechanism does not wite the class to the store and so static fields are not preserved. The meaning given
package copying;
import java.io.*;
import java.util.Hashtable;
public class Test
{
public static void serialize(Object root)
{
ObjectOutputStream out = null;
FileOutputStream out_file = null;
public static Object deserialize()

{
FileInputStream in_file = null;
try {
out_file = new FileOutputStream("./test");
out = new ObjectOutputStream(out_file);
System.exit(1);
}
try {
in_file = new FileInputStream("./test");
in = new ObjectInputStream(in_file);
System.exit(1);
}
try {
out.writeObject(root);
out.close();
System.exit(2);
}
Object o = null;
try {
o = in.readObject();
in.close();
System.exit(2);
System.exit(3);
}
return o;
}
{
Hashtable table = new Hashtable();
Key k = new Key();
Value v = new Value();
table.put(k, v);
System.out.println("Value: " + table.get(k));
System.out.println("Writing table: " + table);
serialize(table);
Hashtable new_table = (Hashtable) deserialize();
System.out.println("Reading table: " + new_table);
System.out.println("Value: " + new_table.get(k));
}
}
Figure 7: Serialization and Deserialization of Hashtable
kona:huw% java copying.Test

Value: I am a Value
Writing table: {I am a Key=I am a Value}
Reading table: {I am a Key=I am a Value}
Value: null
Figure 8: Output from Running the Code Fragment in Figure 7
package copying;
public class Key implements java.io.Serializable
{
int hashcode = 0;
public Key()
{
hashcode = System.identityHashCode(this);
}
{
return "I am a Key";
}
public int hashCode()
{
return hashcode;
}
public boolean equals(Object o)
{
return this.hashcode == ((Key) o).hashcode;
}
}
Figure 9: A Definition of Key Suitable for Object Serialization
kona:huw% java copying.Test

Value: I am a Value
Writing table: {I am a Key=I am a Value}
Reading table: {I am a Key=I am a Value}
Value: I am a Value
Figure 10: Output from Running the Code Fragment in Figure 7 using the Definition of Key from Figure 9
package singleton;
public class Singleton
{
private static Singleton s = null;
private Singleton()
{
}
public static void initialise()
{
if(s == null)
s = new Singleton();
}
public static Singleton getReference()
{
return s;
}
}
Figure 11: Singleton Class Definition
to transient has been interpreted differently by different groups [NEED]. However, the interpretation used by JOS is
that fields marked in this way are not written to the store.
Consider the following class, Singleton (figure 11) that implements4 the Singleton pattern[GHJV96]. A singleton class is one such that only one instance can exist within a virtual machine.
The class contains a private static reference to the single instance, and the constructor is private so that it cannot
be called from code outside this class. It is initialised through the initialise method which tests the instance for
null and only allocates the object if it is. The method getReference returns a copy of the reference to the Singleton
instance.
If a programmer were to make use of the singleton reference in a persistent context, they could do so as depicted
in figure 12.
Singleton.initialise();
Singleton s_ton = Singleton.getReference();
store.writeObject(s_ton);
Figure 12: Writing the Singleton Reference to a Store
The Singleton is initialised via the static method initialise and a reference to it is passed to the reference s ton
in line 2. This object is then passed to the serialization mechanism in line 3 and written to the store. If this process is
terminated and re-started, the Singleton object can be read from the store and reassigned to s ton as in figure 13.
However, because the serialization and deserialization mechanism does not handle static references, it is possible
to call Singleton.initialise again and have two Singleton objects in the same process, breaking the required
semantics. This is possible because when the program deserializes the object from the store the Singleton class is
found and its static references are initialised. This causes the s reference inside the Singleton class to be initialised to
null. A subsequent call to initialise will then reinitialise s with a reference to another object.
4 The class implements the singleton pattern as described in the May98 edition of JavaWorld
http://www.javaworld.com/javaworld/javatips/jw-javatip52.html.
10
Singleton s_read = null;

s_read = (Singleton) store.readObject();
Figure 13: Reading the Singleton Reference from the Store
One possible solution to this problem is to explicitly control how the Singleton class serializes and deserializes
its state. This can be done by providing readObject and writeObject methods. However, this solution is complex
for programmers as they are now aware of how the graph is being moved to and from disk, reducing the abstraction
and increasing the code complexity which is not directly related to the providing a Singleton object.
6 Transient References
If the programmer wants to ensure that a reference is not followed during a serialization then they can mark the
reference in the class as transient. In [GJS96, pg 147] the authors describe that marking a variable as transient
indicates that it will not be part of the persistent state of an object, although the specification does not give details
of the system services that would make an objects state persistent. In JDK 1.2, transient is defined as being not
serializable. If an object reference is marked as transient, null is serialized so that on deserialization, the object
reference is initialised with null. On deserialization, primitive types that are marked transient are initialised to
default values, e.g. integer values are initialised to 0 and single characters to the Unicode value nu00005.
For example, if the programmer marked the variable r in figure 14 as transient and did not provide readObject
or writeObject methods, then non of the objects reachable from r would be serialized6 . The same effect can be
achieved by providing readObject and writeObject methods and not passing r to the serialization mechanism
(figure 14).
public class P
{
R r;
S s;
private void readObject(java.io.ObjectInputStream stream)
throws IOException, ClassNotFoundException
{
s = (S) stream.readObject();
}
private void writeObject(java.io.ObjectOutputStream stream)
throws IOException
{
stream.writeObject(s);
}
}
Figure 14: Using the readObject and writeObject Methods

The disadvantages with this approach are related to those for statics as they are both handled in similar ways by
the serialization mechanism. The programmer has to make the decision about transience at compile-time, which can
be useful as for some objects it does not make sense to serialize them, e.g. file objects or objects representing socket
connections. However, changing the status of a class to serializable requires the code to be recompiled. Therefore,
serializing any instance of class P will not cause the r object to be serialized. This all or nothing approach may be too
limiting for some applications and it may be preferable to make this decision at run-time. An example of why you
may want to decide transience at run-time is given in section 10 on Distribution and Persistence.
5 [JOS97]
has a complete list.

If the objects that r refers to are reachable via another reference that is not marked transient, then they would be serialized. This adds to the
complexity of an application as it is not obvious, from an inspection of the source code, if an instance will be serialized or not.
6
11
In addition, all the objects referred to from r are not serialized. If part of this sub-graph needs serializing,
a different non-transient reference is required to point to the root of the sub-graph. As transient references are
not serialized, any object state that the program relies on needs to be defined when the file is deserialized. As
the programmer has marked a field transient, they have to ensure that any state is reinitialised when the store is
deserialized. Such state can be initialized within the readObject method, for example, to re-open a file or socket
connection.
For all these reasons, use of the transient keyword requires careful thinking.
7 The serialPersistentFields Field

By default, all non-transient and non-static fields that are defined within a class that implements java.io.Serializable
are serialized. Section 6 has shown that this default behaviour can be changed by implementing the readObject and
writeObject methods.
JOS provides an alternative mechanism for overriding the default serialization behaviour. A field called
serialPersistentFields is defined within the serializable class. This field contains the name and type of all the
fields that should be passed to the serialization mechanism; an example of such a class is given in figure 15.
package fields;
package fields;
import java.io.ObjectStreamField;
public class A implements
{
private B b;
private C c;
private final static
ObjectStreamField[] serialPersistentFields = {
new ObjectStreamField("b", B.class),
};
public A()
{
b = new B();
c = new C();
}
public class B implements

{
{
return "I am a B";
}
}
package fields;

{
return "I am an A\n " +
b.toString() + "\n " +
((c != null) ? c.toString() : "null");
}
}
public class C implements

{
{
return "I am a C";
}
}
Figure 15: Using serialPersistentFields to Specify the Fields for Serialization

All the code in figure 15 is in the package fields. Classes B and C are very simple, they both implement
java.io.Serializable and override the toString method, returning a string indicating their type.
Class A also implements java.io.Serializable and holds two references, one to an instance of B, called b,
and the other to an instance of C, called c. The constructor for A creates and assigns the two objects. As toString
method a string indicating the type of A and calling the toString methods on b and c. The call on c is checked first
to see if c is non-null. If it is, toString is called, otherwise, null is output.
To define which fields within the class should be passed to the serialization mechanism serialPersistentFields
must be defined in a particular way. An array of ObjectStreamFields is defined which contains instances of
12
ObjectStreamField describing the field to be serialized. The above description says that the field called b of type
B.class should be serialized. As field c is not mentioned, it will not be serialized.
Serializing an instance of A into a file and deserializing from it gives the output in figure 16 when the instance
variables of A are printed.
Before serialization:
I am an A
I am a B
I am a C
After serialization:
I am an A
I am a B
null
Figure 16: Serializing and Object of Class A into a File

Before serialization, the A instance refers to an instance of B and C. After deserialization, the new instance of A
only refers to a B, as the reference to C is null. This shows that the reference called c in A was not followed during
serialization because only B is mentioned in the serialPersistentFields.
7.1
Comment
This mechanism is an alternative to using the transient keyword. It provides the same semantics, but the programmer specifies what is to be serialized, whereas transient specifies what is not to be serialized.
serialPersistentFields may also be changed at run-time, whereas transient can only be changed at compiletime. This allows the programmer to defer the decision of transience to run-time, rather than having to make it
statically at compile-time. However, the system has a number of implications.
7.1.1 Exact Type is Required

The code for class A, in figure 15, specifies that the field called b of type B should be passed to the serialization
mechanism. The use of serialPersistentFields requires that the programmer specify the exact type of the field
they want to serialize. If the programmer defined a subtype of B, called B Subtype and defined A as in figure 17, the
serialization would fail, raising a java.io.InvalidClassException. This is because the type given as an argument
to the ObjectStreamField must be exactly the same type as used in the definition of the field. In our example,
this is not true: field b is defined to be of type B (the super-type) but it has an instance of B Subtype (the sub-type)
assigned to it in the constructor for A. Even though this code is legal at the Java language-level, the code in the
ObjectOutputStream performs a run-time test to check whether the types used are exactly the same.
The solution to this is to specify the super-type in the ObjectStreamField constructor, but continue to assign
the sub-type to field b in the constructor. The sub-type will then be successfully serialized.
7.1.2 Limited Array Flexibility

The serialPersistentFields array must be defined private final static. However, an array that is defined
final can have its elements changed, final only applies to the reference to the array, not to the elements. Therefore,
the programmer has some control over the contents of the array whcih can be changed at run-time. Consider the new
definition of A in figure 18.
In this definition we define serialPersistentFields slightly differently, by using an explicit array allocation
of length one and providing a static block of code to initialise the 0th element so that field b is serialized. A method
is also provided, changeSerialized, that lets the programmer specify that a new ObjectStreamField object is
placed at the specified array index using the given field name and class.
This class could then be used as shown in figure 19. An object of type A is created and printed out to show the
graph before serialization. The changeSerialized method is then called on a, and field c is specified as the field
to serialize. Object a is then serialized into the file and immediately deserialized into a new, which in turn is then
printed out to show the new graph.
Running this example gives the output in figure 20.
13
package fields;
public class A implements java.io.Serializable
{
private B b;
private C c;
ObjectStreamField[] serialPersistentFields = {
new ObjectStreamField("b", B.class)
};
public A()
{
b = new B_Subtype();
c = new C();
}
}
Figure 17: Specifying a Subtype to be Serialized

package fields;
public class A implements java.io.Serializable
{
private B b;
private C c;
ObjectStreamField[] serialPersistentFields = new ObjectStreamField[1];
static {
serialPersistentFields[0] = new ObjectStreamField("b", B.class);
};
public void changeSerialized(int idx, String fieldname, Class cls)
{
serialPersistentFields[idx] = new ObjectStreamField(fieldname, cls);
}
public A()
{
b = new B();
c = new C();
}
{
return "I am an A\n " + ((b != null) ? b.toString() : "null") +
"\n " + ((c != null) ? c.toString() : "null");
}
}
Figure 18: Updating serialPersistentFields at Run-time

14

{
A a = new A();
System.out.println("Before serialization: \n" + a);
a.changeSerialized(0, "c", C.class);
serialize(a);
A a_new = (A) deserialize();
System.out.println("After serialization: \n" + a_new);
}
Figure 19: Using Mutatable serialPersistentFields

kona:huw% java fields.Test
I am an A
I am a B
I am a C
I am an A
null
I am a C
Figure 20: Running After Changing serialPersistentFields

This time, c has been serialized and b has been treated as a transient reference. There are three limitations of
this approach: the array cannot contain any null references; mutating the array before deserialization can cause
information to be lost; and changing the array during a serialization causes an exception to be thrown.
No null References
If serialPersistentFields contains any null references, the serialization mechanism aborts, throwing a
NullPointerException. In our example where we want to change between only two object references, this is not
a problem. However, if A had more than two fields, being able to leave elements of the array null and have them
skipped over would very convenient. As this is not possible, the run-time flexibility of this mechanism is severely
limited.
Changing serialPersistentFields Before Deserialization

serialPersistentFields is also used at deserialization time: only those fields that are mentioned in the array are
assigned. Changing the contents of the static array at run-time, just before deserialization can result in fields that are
present in the deserialization stream not being assigned to fields in the corresponding class.
Consider the code in figures 18 and 19. Class A is defined to write field b, but this is changed to field c at run-time.
The underlying stream, therefore, contains information on classes A and C and serialized data from their two instances.
However, if we re-write the main method in figure 19 to call changeSerialized before deserialization, as in figure
21, then the deserialized graph will loose information.
The code in figure 21 states that field c should be serialized, but field b should be assigned to when the stream is
deserialized. This results in A being assigned two null references, one to field a and one to field b. This is because
the serialization only writes field c into the stream. Then, at deserialization, the stream is read and only b is assigned
to. As b was not written into the stream, null is assigned to field b and field c is not being assigned to because the
array only mentions b, so field c is assigned null. The result of running this example is given in figure 22.
15

{
A a = new A();
System.out.println("Before serialization: \n" + a);
a.changeSerialized(0, "c", C.class);
serialize(a);
a.changeSerialized(0, "b", B.class);
A a_new = (A) deserialize();
System.out.println("After serialization: \n" + a_new);
}
Figure 21: Changing serialPersistentFields Before Deserialization

I am an A
I am a B
I am a C
I am an A
null
null
Figure 22: The Result of Changing serialPersistentFields between Serialization and Deserialization
16
7.1.3 Changing serialPersistentFields During Deserialization

If serialPersistentFields in A is changed7 during a serialization, the mechanism will throw a
java.io.InvalidClassException and abort the serialization. Consider the code in figure 23 which is an amended
version of the serialize method. Assume the definition of A is that in figure 15 ie. it contains two references, B and
C, but only the B field will be serialized.
public static void serialize(A root)
{
ObjectOutputStream out = null;
FileOutputStream out_file = null;
try {
out_file = new FileOutputStream("./test");
out = new ObjectOutputStream(out_file);
System.exit(1);
}
try {
out.writeObject(root);
root.changeSerialized(0, "c", C.class);
out.writeObject(new A());
out.close();
out_file.close();
System.exit(2);
}
}
Figure 23: Serializing static Variables

The code follows the usual procedure of creating a file for the data to be written to (test) and passing this
reference (out file) to the constructor of an ObjectOutputStream, getting back an object called out. The methods
on out are then used to write the object (root) passed as an argument to serialize. The write takes place in the lower
try catch block. In it the root variable is written and then changeSerialized is called to change the contents of
the static serialPersistentField array. This time the field c is specified to be the one that should be written
from now on. However, when the second writeObject method is called, a java.io.InvalidClassException is
thrown as in figure 24.
During the serialization of the graph into a stream, the object-serialization mechanism appears to be checking that
the values specified in serialPersistentField do not indicate any different types. The exception is thrown even if
the programmer amends the call to changeSerialized, indicating field b should be serialized, ie. no change to what
the array specifies is made, other than assigning the same information with a new ObjectStreamField.
This limits the flexibility and power of this mechanism. Once assigned, the programmer cannot change how the
graph of objects is serialized from a given class. The programmer can only alter this between distinct serializations.
The error message given as part of the exception is also misleading. The error above says fields.A does not contain
a field c, which it clearly does. This would be very confusing for the programmer as the error seems to indicate a
problem elsewhere, e.g. that a different definition of A was possibly used.
7.1.4 Overriding Transient

If a field is correctly specified in serialPersistentFields, the field it indicates is serialized. This is true, even if
the field is marked as transient or static. Thus, this mechanism can be used to override the meaning of these
keywords which could be very confusing for programmers.
7 Thanks to
Tony Printezis who pointed out this behaviour.
17

I am an A
I am a B
I am a C
java.io.InvalidClassException: fields.A; Nonexistent field c
at java.io.ObjectOutputStream.outputClassFields(Compiled Code)
at java.io.ObjectOutputStream.defaultWriteObject(Compiled Code)
at java.io.ObjectOutputStream.outputObject(Compiled Code)
at java.io.ObjectOutputStream.writeObject(Compiled Code)
at fields.Test.serialize(Compiled Code)
at fields.Test.main(Compiled Code)
Figure 24: Exception Thrown when Changing serialPersistentFields During Serialization

The serialization mechanism serializes the objects it finds in the serialPersistentFields array. If the class of
the object being serialized mentions that it should serialize a static variable, one copy of that variable is written to
the store for every object of that type that is serialized. This means that the semantics of static have been broken.
This can be seen with reference to the revised definitions of A and B in figure 25.
Class A now contains three references, b1 and b2 of type B and c of type C. The serialPersistentFields in A
states that both b1 and b2 should be serialized. Class B contains an integer x per instance and the static integer y
which is associated with the class. Class Bs serialPersistentFields states that both x and y should be serialized.
When an instance of A is serialized, b1 is serialized and then b2 is. B states that x and y should be serialized. This
means that for both b1 and b2, the static integer y will be written. This can be seen by examining the output from
running the Unix command od(1) on the file containing the serialized objects.
b1s x field is three octal characters in from the right hand side of the line starting with 120. b1s y field is two
characters in from the start of the next line down. The x and y written for b2 are on the 140 line, the fifth value in
from the right and the value at the very end of that line.
It is possible to obtain the same behaviour by using readObject and writeObject methods and explicitly writing
the static variable to the stream. This may be seen as more appropriate as readObject and writeObject are designed
for this purpose, whereas the programmer may not immediately notice that a reference to a field defined as transient
has been declared in the serialPersistentFields array.
7.2
Summary
serialPersistentFields is an alternative to the transient keyword. The programmer specifies what is to be

serialized, rather than stating what is to be omitted, as with transient. The system requires that the programmer
specifies the exact type to the serialization mechanism which is somewhat counter-intuitive, although it does allow
sub-types to be implicitly serialized. Even though the individual elements of the array are mutable, the implementation
of this mechanism severely limits its run-time flexibility. The implementation checks the array for null values and, if
found, a NullPointerException is thrown, aborting the serialization. The contents of serialPersistentFields
are inspected at deserialization time and if this specifies a different graph from that serialized, information can be
lost.
8 Concurrency
Consider a program consisting of two threads running at equal priorities: one thread is devoted to periodically serializing the object graph given in figure 27, while the other thread mutates its contents.
Assume each class in the object graph is a subclass of NameAddress where NameAddress defines four strings,
name, addr1, addr2 and postcode as well as an update method that takes four string parameters and assigns them
to the relevant fields (figure 28).
Also assume that each class in figure 27 overrides the update method to call update on its superclass, thus setting
its state, and then calling update on objects it references. For example, type Q is defined in figure 29.
A call to qs update method causes update to be called on t. Every type also contains an overloaded toString
method that returns a string representation of the objects fields, together with a string representation of the objects
that are reachable from it. Each string returned starts with the type of the object on which the method has been called.
18
package fields;
package fields;
public class A implements

{
private B b1;
private B b2;
private C c;
public class B implements

{
int x = 5;
static int y = 6;
private final static ObjectStreamField[]

serialPersistentFields = {
new ObjectStreamField("b1", B.class),
new ObjectStreamField("b2", B.class)
};
private final static ObjectStreamField[]

serialPersistentFields = {
new ObjectStreamField("x", Integer.TYPE),
new ObjectStreamField("y", Integer.TYPE)
};
public void changeSerialized(int idx,

String fieldname,
Class cls)
{
{
serialPersistentFields[idx] =
new ObjectStreamField(fieldname, cls);
}

{
return "I am a B";
}
}
public A()
{
b1 = new B();
b2 = new B();
c = new C();
}
}
Figure 25: Serializing static Variables
kona:huw% od -c test
0000000 254 355 \0 005
s
r \0 \b
f
i
e
l
0000020 350 267
6 237
e
@ 002 \0 002
L
0000040
t \0 \n
L
f
i
e
l
d
s
/
B
0000060
b
2
q \0
\0 001
x
p
s
r \0
0000100
l
d
s
.
B 017 305
K 177 311 312 203
0000120
I \0 001
x
I \0 001
y
x
p \0 \0
0000140 \0 006
s
q \0
\0 003 \0 \0 \0 005
0000160
d
s
\0 002
;
L
\b
f
V 002
\0 005
\0 \0
.
A
b
1
\0 002
i
e
\0 002
\0 \0
\0 006
Figure 26: Output of Serializing static Variables
19
o:O
p:P
r:R
q:Q
s:S
t:T
Figure 27: Object Graph Concurrently Mutated and Written to Store
public class NameAddress implements java.io.Serializable

{
String name, addr1, addr2, postcode;
public void update(String n, String a1, String a2, String p)
{
name = n;
addr1 = a1;
addr2 = a2;
postcode = p;
}
}
Figure 28: NameAddress
public class Q extends NameAddress

{
T t = new T();
public void update(String n, String a1, String a2, String p)
{
super.update(n, a1, a2, p);
t.update(n, a1, a2, p);
}
{
return "\n Q(Name: " + name + " Addr1: " + addr1 + " Addr2: " + addr2 +
" Postcode: " + postcode + t.toString() + ")";
}
Figure 29: Class Q, part of the graph being serialized
20
8.1
Store Write Thread
In figure 30, the store write thread serializes the graph and writes it to disk.
FileOutputStream
file;
ObjectOutputStream out;
boolean
more = true;
while(more)
{
try {
file = new FileOutputStream("./test");
out = new ObjectOutputStream(file);
out.writeObject(o);
System.exit(2);
}
System.out.print(".");
try {
Thread.yield();
} catch(Exception e) {
e.printStackTrace();
}
}
Figure 30: Store Write Thread

This thread creates a new file called test and connects an ObjectOutputStream to it. The graph is then written
to disk using the call out.writeObject(o). A . is written to standard output to indicate the write has taken
place and the thread then yields. Yielding allows the graph mutation thread to execute8
8.2
Graph Mutation Thread
The graph mutation thread changes the four strings in each object by calling update on the root object o, passing in
the same string, generated from a monotonically increasing integer (figure 31).
boolean more = true;
int
i
= 0;
String i_s = null;
while(more)
{
i_s = String.valueOf(++i);
o.update(i_s, i_s, i_s, i_s);
System.out.print("|");
}
Figure 31: Graph Mutation Thread

Object o has references to objects p and q. When update on o is called, it first calls update on its superclass,
8 There
may be other runnable system threads waiting as well. However, the graph mutation thread will eventually run.
21
setting its own fields, and then calls update on p and then q. Each object in turn calls update for itself, via its
superclass and then on any reachable objects.
In addition, object q also calls Thread.yield. This is to ensure there is concurrent activity between the mutation
and store write thread to simulate the activity of library code. Consider the call to Thread.yield to be in-between the
call to super.update and t.update so the graph is written to disk while an update has only partly been performed
on it. In code from third parties it would not be possible to know at which point a thread yielded to another, for
example, code may explicitly call Thread.yield or it may implicitly give up control, via a call to Thread.sleep or
by performing I/O activity.
We define a consistent graph to be one that contains the same value in each corresponding field of every object in
the graph. For example, if i s above contains the string 9, a consistent graph is one that, for each node o to t,
every field in each object contains the string 9.
8.3
Running the Example
When the two threads are run we see a number of .| characters written to standard output. This indicates that the two
threads are running, and that they are yielding to each other, ensuring the graph is mutated during a write to disk. The
program is stopped after a while9 and another program is run over the contents of the store to retrieve the last data
successfully written.
8.4
Deserializing the Store
To deserialize the store we run the code given in figure 32. We create an in stream, define an object o to hold the
result of the deserialization, deserialize the graph with a call to in.readObject, assigning it to o, and then print the
contents of the graph.
FileInputStream file = null;
try {
file = new FileInputStream("./test");
in = new ObjectInputStream(file);
System.exit(1);
}
O o = null;
try {
o = (O) in.readObject();
System.exit(2);
System.exit(3);
}
System.out.println(o);
Figure 32: Code to Deserialize the Store

Printing the graph causes the overloaded toString methods to be called. Given the above example, a typical
output on deserializing is given in figure 33.
9 The code for this is not shown. At some point more is set to false in each thread, the while loop terminates, and the run method completes. The
thread associated with the static main method waits for the two threads to terminate and then terminates itself.
22
O(Name: 21 Addr1: 21 Addr2: 21 Postcode: 21

P(Name: 21 Addr1: 21 Addr2: 21 Postcode: 21
S(Name: 21 Addr1: 21 Addr2: 21 Postcode: 21))
Q(Name: 21 Addr1: 21 Addr2: 21 Postcode: 21
T(Name: 20 Addr1: 20 Addr2: 20 Postcode: 20)))
Figure 33: Output After Deserializing the Store

As the graph is mutated at the same time as it is being written, the contents of the store are inconsistent. While
the graph was being written out, i in some parts of the graph had the value 21, while in others, it contained the value
from the previous mutation, ie. 20 (figure 33). This is contrary to our definition of a consistent graph, given in section
8.2, that requires the same value to be present in every field of each object in the whole graph.
8.5
Solving the Inconsistency Problem
In the general case, when integrating multi-threaded code from a number of suppliers, there is no way to tell when
a particular thread will run, relative to the thread that writes the graph of objects to disk. It is possible, as the above
example demonstrates, that the graph may be written while it is being manipulated and in certain circumstances this
can lead to inconsistencies.
In this particular example, as we have control over all the code, we could remove the explicit call to yield as
described above. However, this is not possible in the general case. In order to solve this in a generally applicable way,
we need to lock the graph so that only graph mutation or a write to the store can happen at any one time.
In Java, this can be solved by synchronizing on the root object, o, before calling o.update and
out.writeObject(o) as in figure 34.
// Store Mutation Thread
// Store Write Thread
synchronized(o) {
o.update(i_s, i_s, i_s, i_s);
}
synchronized(o) {
out.writeObject(o);
}
Figure 34: Synchronizing on the Root Object to Solve the Inconsistency Problem
The o object above used in the two threads is a reference to the same object, the root of the graph in figure 27.
This code ensures that only the update method or the call to out.writeObject can be called at any one time. If the
update method is being executed, and the store write thread calls synchronized(o), it will block, until the mutation
thread leaves the synchronized block, ie. the call to o.update completes. This ensures the contents of the serialized
graph are consistent and therefore meets the definition given in section 8.2.
8.6
Discussion
There are several problems with the above solution. Firstly, the entire graph must be locked. If the graph is large,
locking a large number of objects may not be acceptable. In addition to locking the entire graph, any code that
changes the state of any objects in the graph must also respect the lock. In our example, it is trivial to ensure that the
graph is locked as mutation only originates from o.update, and so locking o at this level ensures no other changes
will be made to the graph. However, in a large program, changes to the graph will take place at numerous places in
the code, will be performed by different threads and arbitrary objects within the graph will be changed. Ensuring that
the appropriate lock has been acquired would be very difficult and maintaining this code would be time-consuming
and costly.
Locking the entire graph decreases the amount of concurrent processing the application can perform. If the
application has to wait periodically on a lock to ensure the graph is written consistently, graph processing the
progress of the application will be significantly reduced as serializing a large graph can be time-consuming (see
Appendix A). In the Java thread model ([GJS96, chptr 17]) it is possible to alter the priority of a thread, favouring the
graph mutation thread, by assigning it a higher priority. However, this increases the overall complexity of the program
and the programmer has to manage multiple threads running at different priorities, which can be complicated in itself.
If the serialization thread is run at a lower priority, serialization will be performed less often. Therefore, the data will
23
be written to disk less often, thus increasing the likelihood that the state of a computation will be lost on the event of
a failure and would need to be performed again on deserializing the file.
Rather than locking the entire graph, locking on a per object basis may be acceptable in certain circumstances, e.g.
when consistency within a single object is preferred over the consistency of the whole graph. However, this requires
the use of readObject and writeObject methods to be defined on each type that will form part of the graph so
that these methods may acquire a lock on themselves, via this, using the synchronized keyword. This significantly
increases the complexity of the code (deadlocks are also more likely to occur) and requires the programmer to identify
the graphs transitive closure. This solution will only work if the code serializing the graph respects the per-object
locks that have been acquired by readObject and writeObject. The Object Serialization system of Suns Java 2
does not acquire any locks on application-level Java objects during the serialization and so programmers would have
to write code to do this themselves.
9 Distribution
Distributed programs are partly characterised by their ability to support genuine process concurrency. This is possible
because these programs make use of more than one host and therefore have access to more than one processor. A
collection of distributed objects in Java communicate by sending messages across a network. These messages take a
finite time to propagate from one host to another and the receiving object must be listening and ready to accept them.
In addition, distributed systems are subject to partial failure. The receiving process may have failed and will no longer
be listening, causing an exception to be raised in the client.
9.1
Serialization and Distribution
Consider a multi-threaded server process consisting of two threads, running at the same priority: one thread is responsible for serializing a store to disk as in the previous section; and the other thread, the main thread, creates a remotely
invokable object that offers a single method for distributed invocation, called invoke, which outputs a string at the
server. Another process, the RMI registry holds named references to remotely invokable objects, is listening on a
well known port number. The registry is used by the third process, the client, to obtain a copy of a reference to the
remotely invokable object. The client then calls invoke once per second in an infinite loop.
The server is started, the remotely invokable object is created and its reference is placed into the registry so the
client can obtain a copy of it; then the serialization thread is started. The client process is started, it obtains a copy of
the reference to the remote object from the registry, enters its infinite loop and starts to call invoke.
Whenever the server process is serializes a graph to a store, the request from the client is blocked. This behaviour
is due to the equal priority of the two threads in the server and the implementation of JOS and the RMI mechanism. It
will only be serviced when the serialization has completed. If the serialization is sufficiently complex and takes a long
time, the semantics of the client code may be affected. For example, the client may not be able to tolerate the delay
imposed by the server-side serialization. Once again, it is possible to lower the priority of theserialization thread, but
this requires the programmer to use thread priorities. Although the remotely invokable object is associated with its
own thread, this information is hidden from the programmer and, therefore, the priority of the remotely invokable
objects thread cannot be altered.
9.2
Serializing Remotely Invokable Objects
It is possible to serialize a remotely invokable object into a store which can provide a remotely invokable object with
some resilience against failures. For example, assume that the server serializes the remotely invokable object to disk
and then crashes. When the server is re-started, it retrieves its last known state from the store and re-registers the
remotely invokable object reference with the registry. When the server process crashes, the client side will receive a
java.rmi.RemoteException when it next uses the remote reference. If, when this happens, the client periodically
attempts to get another copy of the reference to the remote object, it will eventually retrieve a copy of the value last
placed into the registry when the server process restarted. The server has to re-register a new reference because the
server object is likely to be listening on a different port from before, even though the object, at the Java language
level, is logically the same.
When rebooting such a system it is necessary to read the entire store before the remotely invokable object becomes
available. If the store is very complex, it may be quite a while before a client can make access the remote object. The
programmer has to be aware that this behaviour is likely and needs to add code to handle it. For example, the client
could attempt to get a copy of the reference from the registry, use it, and, if it fails, pause and try again some time
later. However, the client side is affected by the behaviour of the server. This could have a knock-on effect throughout
24
the whole system. If a client has to periodically retry a server that is reinitialising itself from a store, the client may
not be able to deal with incoming calls to its own remotely invokable objects. Behaviour such as this increases the
complexity of the code at the client side, whereas it is usually preferable to have the complexity inside the server.
All the problems described above are also applicable to three-tier systems. However, the problems there are worse
as a pair of client-server interactions are performed within such systems.
10 Distribution and Persistence

This section describes the problems that arise when combining distribution and persistence via serialization in Java.
Section 10.1 describes why combining these models can lead to unexpected results; section 10.2 discusses the problem
of using specialised readObject and writeObject methods in a distributed system with object-serialization-based
persistence and section 10.3 concludes with a description of the problems of using persistence at both the server and
client side.
10.1
Distribution Model vs Persistence Model
Combining distribution and persistence is a hard problem to solve as the two mechanisms can work against each
other when used in conjunction. It is possible for references to build up between processes running over JOS stores
as a client store can contain a reference to a remote object (see section 10.3.2). If such references are allowed to
build up, they increase store inter-dependencies, reduce their autonomy, making it harder to develop and evolve them.
Furthermore, distributed computation in Java requires that method parameters and results are either passed by copy
or as references to remote objects. If passed by copy, then, for the duration of a remote method, two copies of the
parameters are present in the system, one at the client and one in the server. If a reference to a remote object is
passed, this does not cause consistency problems, but rather performance problems as the network has to be traversed
for each method invocation to the remote object passed as a reference. Problems can also arise if the values passed
between client and server become part of the persistent state on either side. In the case of the transient server, a
value may be passed to the server, where it is used, a result is generated and sent back to the client, with the server
discarding its copy and the client retaining it. However, if the server retains the value passed and makes it part of the
persistent state, consistency problems can occur as replicating information in this way may break the semantics of the
application. Stores can also become very large and if a parameter is passed to a remote object that is close to the root
of persistence, a large part of a store can be passed across the network. This can raise performance issues as well as
semantic issues such as consistency and sharing.
These problems exist whenever distribution and persistence are combined and are not peculiar to using the store
mechanisms provided by Java object serialization. For a more detailed description of the problems encountered when
combining distribution and persistence, see [SA97, Spe97, Spe99]. The rest of this section describes some problems
that are specific to using object serialization in a distributed system.
10.2
Specialised Serialization and Distribution
Java allows a programmer to control how an object is serialized by providing readObject and writeObject methods
for the class of the object. However, these same methods are called whether the object is being serialized to pass it
across the network or serialized to write it into a file. In certain circumstances, an object may need to be serialized
differently if it is being passed to another virtual machine rather than if it is being written to disk. However, Java
requires the programmer to use the same mechanism for both.
For example, consider the class in figure 35 that is involved in a distributed application that uses serialization
based persistence. The class represents an employee in a company and it consists of: an id, used to uniquely identify
that person; the persons job title; their salary; and a number of appraisal reports, detailing the employees history
with the company.
When serializing an object of this type all the information should be saved and this can be accomplished using
the default behaviour. However, the appraisal history of the employee is deemed sensitive and only of local interest
and should not, therefore, be transferred outside the virtual machine that contains the object. We assume that storing
the appraisals on a local disk is secure.
It is not enough to make the appHistory field transient as this will stop the information from being serialized
to disk. There are many partial solutions to this problem, each with widely varying implications.
25
public class Employee implements java.io.Serializable

{
Id
empId;
JobTitle
jobTitle;
Salary
salary;
Appraisals appHistory;
}
Figure 35: Employee Record
10.2.1 Partial Solution One
It is possible to create two subtypes of Employee, DiskEmployee and NetworkEmployee, and provide specialised
readObject and writeObject methods for each class. The methods for NetworkEmployee would ensure that
appHistory was not serialized and on deserialization null could be assigned. Such a solution is highly artificial as
two classes have to be introduced that are not directly related to the application problem at hand.
If a DiskEmployee object needed to leave the virtual machine, a new NetworkEmployee object would have to
be created and some fields from the DiskEmployee object copied. To copy DiskEmployee easily would require
it to implement java.lang.Cloneable to tell the virtual machine that clone may be called on its instances. The
DiskEmployee could then be copied and passed to the NetworkEmployee constructor, which could extract the fields
necessary, assuming those methods have been defined on DiskEmployee. Defining Cloneable on DiskEmployee
may also require Cloneable to be defined on the classes that DiskEmployee refers to, in our case Id, JobTitle and
Salary.
For every DiskEmployee object we want to send from the virtual machine, we have to create a NetworkEmployee
which consumes more memory and processing time to copy the objects to initialise it.
An optimisation to reduce the number of classes would be to use just Employee types in the application and only
copy to NetworkEmployee types when the object leaves a virtual machine. Employee would be used to serialize all of
its data to disk. This is only an optimisation and it still suffers from the same problems as the first solution. However,
given that the appraisal information must not leave the virtual machine, some form of object copying is required and
this solution appears to be a reasonable approach.
10.2.2 Partial Solution Two

One other solution that retains only Employee objects is to make the programmer indicate, at run-time, whether the
object is to be moved to disk or copied across the network. Employee objects could have a boolean field added to
them, called local which would be tested in writeObject and, if true, all the fields, including appHistory, would
be written to disk. If false, appHistory would not be written.
However, this approach has its problems. It is possible for an Employee object whose local field is false to be
passed out of the virtual machine. This is because Employee defines java.io.Serializable and the JOS system
does not allow a programmer to distinguish whether an object is being serialized for writing to a store or to be passed
across a network. This is the first partial solution is superior, as the use of two different types allows the programmer
to make this choice.
10.2.3 Discussion
This problem arises because the serialization system gives the programmer one mechanism for dealing with objects
in two different contexts. As this section has shown, distribution and persistence can require an object to be dealt
with differently and Java forces the programmer to code around it. A more flexible solution would be to provide the
programmer with two pairs of methods, one used for serialization to disk, and one for network-based serialization.
This would require the programmer to understand two more methods but it would allow them to distinguish between
the two different cases. The stream used to contain the serialized data would then have to be associated with one of
two modes which the prrogrammer would have to set before using them. This ties the serialized stream to only one
use, or else the programmer would have to be very careful when re-using a stream in the other context to ensure the
correct pair of methods is called.
26
10.3
Server and Client-Side Persistence
Further problems can arise if references to remotely invokable objects become persistent on both the client and server
sides.
10.3.1 Server-Side Persistence

If a remotely invokable object on the server side is made persistent, and if the process that contains it terminates and
restarts and the object is read from the store, then the object is likely to be listening on a different port. This means that
the client will have to rebind to it using the registry in order to be able to interact with it. This is a consequence of the
definition of remote object identity which is partly based on the port number on which the remote object is listening.
As this is likely to change from one server process to another, a rebind is necessary to re-establish communication
between the client and server.
10.3.2 Client-Side Persistence

A reference to a remotely invokable object held at a client can be successfully written into a file-based store. However,
when the object is read in again, Javas serialization and distribution mechanisms try to automatically rebind the
variable to the remote object. This will only work if the remote object is currently available and is listening on the
same port as it was when the reference at the client was last successfully used. For example, if the server has been
rebooted, in between the client writing to the store and reading from it, then the remote object will be listening on a
different port and the rebind will not be possible. This failure to rebind is presented to the program as an exception
and it causes the deserialization of the entire store to be aborted, unless it is caught. If it is caught, for example, within
a readObject method, the reference to the remote object can be marked as null. Then the rest of the deserialization
will proceed. However, when the remote reference is used, code needs to be called once to ensure it is bound to
the server, or a check needs to be performed each time it may be dereferenced to see if it is null, rebinding it as
necessary.
To solve this problem a readObject, writeObject pair of methods can be added to the class which contains the
client side reference. For example, consider the class in figure 36.
import java.io.ObjectOutputStream;
import java.io.ObjectInputStream;
import java.io.Serializable;
public class Wrapper implements Serializable
{
remobject obj = null;
private void writeObject(ObjectOutputStream stream) throws IOException
{
stream.writeObject(obj);
}
private void readObject(ObjectInputStream stream) throws IOException,
ClassNotFoundException
{
try {
obj = (rem_object) stream.readObject();
} catch(java.rmi.ConnectException ce) {
System.err.println("Error connecting to remote side");
}
}
}
Figure 36: Handling a Rebind Failure
27
The class Wrapper contains the reference to the remote object of type rem object. When this container object
is read from the store, its readObject method is called by the serialization mechanism. This code attempts to read the
reference to the remote object from the stream and assign it to the field called obj. If the server is listening on the
same port, the read will be successful and the client will have been successfully bound to the server. However, if the
read is not successful as the server is not available as expected, a java.rmi.ConnectException will be thrown. The
programmer can then arrange for the field to be assigned at a later stage and the rest of the deserialization can take
place.
The serialization and distribution mechanisms in Java have forced the programmer to handle this situation using
the readObject and writeObject methods because, if an error is encountered and it is not handled, the entire
deserialization is aborted. Another solution would be to only try the rebind when the reference to the remote object
is used. When the reference is read from the store at the client side there may be very good reasons why the server
is not available, for example, it may have been taken down for maintenance. It is only when the client tries to use
the reference that dealing with the failure is important. This would allow the failure code to be placed as close as
possible to the use of the reference. In the solution above, the failure handling code is artificially grouped with the
deserialization code.
11 Classloaders
Some applications require the use of their own classloaders to retrieve classes in a specialized way that is not supported
by the default Java classloader. Java classes are typically stored as files on disk and they are loaded lazily by the virtual
machine on first usage by the program.
However, in some circumstances, the required classes may reside in remote location or they may require additional
processing, such as decrypting. The default classloader cannot be used in these circumstances and an application
specific classloader is required.
This section describes how the use of an application-specific classloader can interfere with deserialization. The
delegation model of class loading is not used in these examples, however, what is discussed below is applicable to
that model.
Consider again the Singleton example given in figure 13. It reads a graph from a store, rooted at an object of type
Singleton.
Singleton s = null;
s = (Singleton) store.readObject();
Figure 37: Reading a Store Rooted at a Singleton Object
When the store.readObject line of code is executed, the object graph is deserialized and the root object is
passed back to the program. The object is cast from the Object return type defined by readObject, to the type
Singleton. If this test is successful, the root of the graph is assigned to the variable s. However, for the test
to succeed, not only does the cast have to be performed correctly, but class Singleton has to be available to the
classloader. If the class cannot be found, the serialization is aborted with a java.lang.ClassNotFoundException.
During deserialization the class may not be available. This is because the class may have been loaded from a
different virtual machine when the object was first created and written into the store, or the class may have moved
location on a locally available disk to a directory not accessible via the CLASSPATH. If the process is terminated,
restarted and the store is read, the new virtual machine needs to be able to find the class when the object is read
from the store. The class needs to be found again because only information about a class is written into the store,
the classes themselves are not written. It must be possible for the virtual machine to find the class when the cast is
performed. If the class is only available remotely, the programmer must use a classloader capable of loading it, either
an application specific classloader, or one already provided such as the URLClassLoader, RMIClassLoader or the
AppletClassLoader.
In addition to these requirements, the class that contains the code in figure 37 must be loaded by the application specific classloader. If the code was loaded by the default classloader, when the virtual machine attempts
the cast, the default classloader will be used to try to find Singleton. The default classloader will not be able to
find it and a ClassNotFoundException will be generated. Unfortunately, it is not sufficient to explicitly catch the
ClassNotFoundException when readObject fails, find the class using an application-specific classloader and then
try to read from the store again. This is because the virtual machine uses the classloader that was used to load the
class that contains the cast code, and in this case that classloader was the default classloader. This is the reason the
28
class that contains the code in figure 37 must be read in using the application-specific classloader. If this is the case,
when the cast to Singleton is performed, the application-specific classloader will be used to find the Singleton
class, it will succeed and the cast will succeed.
Therefore, to combine this type of persistence with the ability to download classes from remote virtual machines
(or to find them locally, should their location change) requires an application-specific classloader. This assumes that
objects instantiated from the remotely loaded classes are written to the store. If, when the store is deserialized, all
classes can be found locally by the default classloader, the use of an application specific classloader is not necessary.
12 Evolution and Persistence

The object serialization mechanism defines limited support for the evolution of classes. This section describes the
system with the aid of the example introduced below.
The evolution specification [NEED] defines the notion of a contract, established by the original class, that the
evolved class must respect if the serialization mechanism is to be able to consider the two classes compatible. The
goals defined for the versioning system require that bidirectional communication between different versions of a class
is possible. This means that a class should be able to understand a stream written by an older version, and that a class
can write a stream that an older version of the class can read.
Incompatible changes are defined quite generally to be those changes for which the guarantee of interoperability cannot be maintained. Interoperability occurs when the contract established by the original class is respected
by the evolved class and that bidirectional communication between the different versions is possible. One kind of
incompatible change is the removal of a field which is discussed further in section 12.1.7.
12.1
Evolution Example
Consider the code in figure 38. We define three classes, Point, Point3D and ColourPoint3D. Class Point captures
a single point in a two dimensional space, using two integers x and y. Class Point3D extends Point adding support
for the third dimension using integer z. ColourPoint3D extends Point3D and defines a colour for the point, using a
String colour. Each class defines two constructors, a no-arg constructor and one that initialises all of its state. Each
class also overloads the toString method to output the objects state.
Note that x and y are defined as protected in Point as the author of this class anticipated sub-types being
defined, but that the author of Point3D defined z to be private and no methods are provided to set or get it.
We have two other classes (Serialize and Deserialize, which are not shown) that are used to serialize
and deserialize an instance of ColourPoint3D to and from a file called test. Class Serialize creates an instance of ColourPoint3D with the values x=5, y=66, z=34, colour="Black" and serializes it to test. Class
Deserialize opens test and connects an ObjectInputStream to it. This object is used to retrieve the contents of
test and assign it to an object of type ColourPoint3D. However, in between writing test and reading from it, we
will evolve Point, Point3D and ColourPoint3D.
12.1.1 Example Evolutions

The evolution of the three classes will be performed incrementally as specified below. They represent three common
changes performed to object-oriented code: adding a field to a class; changing the implementation of one aspect of a
class; and altering the class hierarchy by removing a class.
1. Adding a field date of type Date to ColourPoint3D to capture when the ColourPoint3D object is created;
2. Changing the type of colour in ColourPoint3D from String to long;
3. Refactoring the class hierarchy, exchanging the position of Point3D and ColourPoint3D.
12.1.2 Adding a Field

Once test has been written with an instance of ColourPoint3D as defined in figure 38, we update ColourPoint3D
to add the Date field as in figure 39.
12.1.3 First Attempt at Running Deserialize

This completes the change we want to make to ColourPoint3D. If we now run Deserialize, the instance of
ColourPoint3D read from the file will be the previous version as defined in figure 38. The output from running
the code can be seen in figure 40.
29
package evolution;
package evolution;
public class Point implements

{
protected int x;
protected int y;
public class Point3D extends

Point
{
private int z;
public Point()
{
x = 0;
y = 0;
}
public Point3D()
{
z = 0;
}
public Point(int x, int y)

{
this.x = x;
this.y = y;
}
public Point3D(int x, int y, int z)

{
super(x, y);
this.z = z;
}

{
return "x: " + x + " y: " + y;
}

{
return super.toString() + " z: " + z;
}
package evolution;
public class ColourPoint3D extends Point3D
{
String colour;
public ColourPoint3D()
{
colour = "Not Defined";
}
public ColourPoint3D(int x, int y, int z, String colour)
{
super(x, y, z);
this.colour = colour;
}
{
return super.toString() + " Colour: " + colour;
}
}
Figure 38: Point, Point3D and ColourPoint3D Classes
30
package evolution;
import java.util.Date;
import java.io.IOException;
{
String colour;
Date date;
{
date = new Date();
}
{
super(x, y, z);
date = new Date();
}
{
return super.toString() +
" Colour: " + colour + " Date: " + date;
}
}
Figure 39: Evolved ColourPoint3D Class
kona:huw% java evolution.Deserialize

java.io.InvalidClassException: evolution.ColourPoint3D; Local
class not compatible: stream classdesc serialVersionUID=-1421736715104280864
local class serialVersionUID=-5252289778578619139
at java.io.ObjectStreamClass.validateLocalClass(Compiled Code)
at java.io.ObjectStreamClass.setClass(Compiled Code)
at java.io.ObjectInputStream.inputClassDescriptor(Compiled Code)
at java.io.ObjectInputStream.readObject(Compiled Code)
at java.io.ObjectInputStream.inputObject(Compiled Code)
at evolution.Deserialize.main(Compiled Code)
Figure 40: Pre and Post-Evolution Classes are Incompatible
31
An incompatibility in the stream has been encountered. The object serialization mechanism has noticed that
the class of the instance in the stream is different from the evolved class that we are now trying to use. This
incompatibility is detected because the serialization mechanism calculates a hash value for the two classes. The
serialVersionUID of ColourPoint3D when the stream was written was -1421736715104280864 and the evolved
classs is -5252289778578619139.
12.1.4 Defining Class Compatibility Across Versions

To ensure that the object-serialization mechanism considers the evolved class to be compatible with the instance read
from the stream, we have to add one more line of code to the new definition of ColourPoint3D. This is required by
the object serialization mechanism to support this kind of class evolution [JOS97].
Given the evolved class in figure 39 we define its serialVersionUID to be that of the class used to write the
instance in the stream. This is done using a tool called serialver which is run on the original version of the
ColourPoint3D class. Running the tool generates this output:
evolution.ColourPoint3D:
static final long serialVersionUID = -1421736715104280864L;
This value is the hash of the original ColourPoint3D class. The number is the same as in the exception in figure 40
for the instance in the stream (stream classdesc above). To complete our definition of the evolved ColourPoint3D
class we add this line of code to the class, giving the completed class shown in figure 41.
package evolution;
{
String colour;
Date date;
{
date = new Date();
}
{
super(x, y, z);
date = new Date();
}
{
return super.toString() +
" Colour: " + colour + " Date: " + date;
}
}
Figure 41: Compatible Evolved ColourPoint3D Class
Security of the Stream

If the programmer erroneously delcares two classes to be compatible with this mechanism when they are not, the
deserialization cannot be performed and an InvalidClassException will be thrown (figure 42).
32
java.io.InvalidClassException: evolution.ColourPoint3D; Local

class not compatible: stream classdesc
serialVersionUID=-1421736715104280864 local class
serialVersionUID=-1421736715104280834
Figure 42: Exception Thrown when Classes are Incompatible

The exception describes the serialVersionUID of the object in the stream in the exception above. This information could be used, in conjunction with the reflection mechanism, to generate a class that spoofs the original class,
thus gaining access to the information in the stream.
In addition, by allowing the state of objects to be written to a file, it is possible to gain access to the private state
of an object. This could be done by reading from the file a byte at a time, interpreting the codes as the file is read.
12.1.5 Second Attempt at Running Deserialize

Once we have recompiled our new definition of ColourPoint3D, Deserialize can be run again. The output is given
in figure 43.
x: 5 y: 66 z: 34 Colour: Black Date: null
Figure 43: Output from a Successful Deserialization after Evolution
12.1.6 Comment
The contents of the serialized object have been successfully read and assigned to an object whose class is different to
that used when the object was first written. Therefore, Java object serialization provides some support for transferring
the state of objects between different versions of the same class. However, the solution adopted has a number of
implications.
Date is null
The Date value output in figure 43 is null. This is because the serialized object was instantiated from the original
definition of ColourPoint3D which did not define a Date field. Therefore, the object serialization mechanism cannot
assign anything meaningful to this field, so the object reference default of null is used.
However, the date field was added to this object to capture information on when the ColourPoint3D object
was created. In our evolved system, when the pre-evolution object is read from the file, the ColourPoint3D object
is created. We would like to be able to represent this fact in our object. This is not possible as neither of the two
constructors are executed. The object is created by the object-serialization mechanism and its state is initialised from
the contents of the stream. To solve this problem, the programmer could add a pair of readObject and writeObject
methods and create a new date object in the readObject method. This does, however, increase the complexity of
the code. Another solution would be to add a setDate method on the evolved ColourPoint3D class and call this for
every ColourPoint3D object read from the file. This, however, requires the entire graph of objects to be traversed,
looking for objects of the right type to call the method.
Multiple Ancestor Versioning Not Supported

If the programmers wants to use the a pre-evolution instances within the context of an evolved class, the programmer
has to explicitly state, in the code of the evolved class, that the evolved class is compatible with the older class. This
requires the programmer to be able to generate the serialVersionUID by using the serialver tool.
In addition to this, the use of serialVersionUID restricts the programmer to expressing evolution in terms of one
compatible class. For example, a file may contain a serialized form of an object of type T. An evolved type, T2, may
be defined and the relationship between T2 and T can be expressed. An instance of T2 may be written to another file,
so that there are instances of T in one file and instances of T2 in another file. A third evolution may then be required
that is an evolution of T and T2, resulting in T3. This requirement to define one evolved type in terms of a number
of others may arise because T2 may not have captured all the changes to T. Rather than change T2, we can create T3
33
which captures the changes embodied in T2 plus the changes that T2 did not capture over T. It may not be possible,
or desirable, to express the evolution of T3 solely in terms of T2. Given the use of a single serialVersionUID, this
kind of multiple ancestor versioning cannot be expressed. This is further enforced because serialVersionUID is
defined as final and so cannot be redefined in a sub-type, effectively restricting its meaning to exactly the type it is
defined in.
12.1.7 Compatible and Incompatible Changes and Implicit Compatibility

The versioning system described in [JOS97, section 5.6] gives a list of compatible and incompatible changes that
may be made to classes. The approach taken requires the reader to interpret the contents of the stream, extracting the
data it needs and initialising any new object state that is not represented in the stream. In terms of calculating class
compatibility, the stream is the only source of information about the original class.
One kind of incompatible change is the removal of a field. For example, if int z in Point3D should be changed
to a float we cannot simply redefine it as float z as the deserialization will fail when trying to assign an int from
the stream to a float in a class. One solution is to keep the int z, having it initialised by the object-serialization
mechanism and then have another variable, float z2, that is really used, assigning z to z2 at some convenient time,
e.g. within the readObject method. However, this adds code to the definition of the evolved class that is really
only concerned with the evolution from the original class to the new one. This should be separated out, making
the evolutionary step a separate one, so that the new class only needs to contain the fields for its definition. This
separation is addressed in more detail in section 12.4.
Implicit Compatibility
The Java Object Serialization Specification [JOS97] in section 5.1 claims that the versioning mechanism provides
automatic handling of classes that evolve by adding fields and classes to the inheritance hierarchy and that the
serialization mechanism will handle versioning without class-specific methods to be implemented for each version.
This assumes that the serialvVersionUID has been added to the evolved class.
This means for those changes that are defined as compatible the programmer does not need to provide classspecific code. The serialization mechanism implicitly deserializes an object from the stream, initialising a compatible
version with the retrieved state. This is appropriate because the programmer does not need to provide any code to
perform the initialisation of the new object. However, this is inadequate for the same reason as the programmer has
little control over how the initialisation is performed. Section 12.4 proposes a different mechanism for performing
the update of objects which is motivated by the change described in the next section.
12.1.8 Changing the Type of a Field

The second kind of change we want to examine is to change the type of the colour field in our evolved ColourPoint3D
from String to long.
We compile this to ensure this version of ColourPoint3D will be used when deserializing the store and we re-run
Deserialize, obtaining the output in figure 45.
Unfortunately, the evolution we want to perform causes a ClassCastException to be thrown as the type of the
object in the stream is a String and the object serialization mechanism attempts to implicitly assign it to the evolved
field which is of type long.
12.1.9 Solution
One possible solution is to provide a pair of readObject and writeObject methods to handle the conversion between
the two types, as shown in figure 46.
When the stream is deserialized and the original version of ColourPoint3D is found, the stream object is passed
to the readObject method defined on the latest version of ColourPoint3D given in figure 46. The first line in this
method then reads an object from the stream, casting it to what we know it to be, a String. The hashCode method is
then called on the object to generate a long value that we have decided will be the new representation for this colour.
In addition, we also create a new Date object, assigning it to date.
This solution works well. However, [JOS97][section 5.6.2] states that if the version of a class reading the stream
defines a readObject method, readObejct should first call stream.defaultReadObject. Therefore, the code
above is flawed as it does not conform to the specification. If we do call stream.defaultReadObject the code
breaks and we are back where we started, as the String value will automatically be assigned to the long, throwing a
ClassCastException.
34
package evolution;
{
long colour;
Date date;
{
colour = 0;
date = new Date();
}
{
super(x, y, z);
this.colour = colour.hashCode();
date = new Date();
}
{
return super.toString() + " Colour: " + colour + " Date: " + date;
}
}
Figure 44: Second Evolution of ColourPoint3D Class

Exception in thread "main" java.lang.ClassCastException:
Assigning instance of class java.lang.String to field evolution.ColourPoint3D#colour
at java.io.ObjectInputStream.inputClassFields(Compiled Code)
at java.io.ObjectInputStream.defaultReadObject(Compiled Code)
at java.io.ObjectInputStream.inputObject(Compiled Code)
at evolution.Deserialize.main(Compiled Code)
Figure 45: Output from Failed Deserialization after Second Evolution Step
35
package evolution;
{
long colour;
Date date;
{
colour = 0;
date = new Date();
}
{
super(x, y, z);
date = new Date();
}
{
return super.toString() + " Colour: " + colour + " Date: " + date;
}
private void writeObject(java.io.ObjectOutputStream stream) throws IOException
{
stream.writeLong(colour);
stream.writeObject(date);
}
private void readObject(java.io.ObjectInputStream stream) throws IOException,
{
colour = ((String) stream.readObject()).hashCode();
date = new Date();
}
}
Figure 46: Explicit Conversion from String to long During Deserialization
36
An alternative implementation for readObject is possible by using the class

java.io.ObjectInputSteam.GetField which reads all the fields from the stream and then makes them available
to the programmer by specifying their field name to retrieve them.
private void readObject(java.io.ObjectInputStream stream) throws IOException,
{
GetField fields = stream.readFields();
String colour_string = (String) fields.get("colour", null);
colour = colour_string.hashCode();
date = new Date();
}
Figure 47: Retrieving Objects from the Stream by Field Name

This allows the stream to be read field by field, without throwing a ClassCastException. This method does not
use stream.defaultReadObject and is, therefore, contrary to the specification. If stream.defaultReadObject is
used, a ClassCastException will be raised and the above code could then be used to catch the exception and deal
with it. In this case, using stream.defaultReadObject creates more work.
12.2
Comment
In this section we wanted to change the type of colour from String to long. This approach caused a
ClassCastException to be thrown. One possible solution was to provide readObject and writeObject methods to
specialise reading of the stream. However, such a solution confuses two object-serialization issues. The readObject
and writeObject methods are used to customise copying of the state of an object to and from the serialized stream.
In figure 46, readObject is being used to specialise reading from the stream for the purposes of evolution. The
implementation of the readObject method is tied to the contents of the original stream, in effect, restricting it to the
version of the original class. The writeObject method writes the contents of the evolved class to the stream, which
is the normal use for this method.
A problem arises when we want to provide a readObject method so that the evolved ColourPoint3D can read an
instance of itself from a stream in a specialised way, which may be unrelated to the question of evolution. However,
if we want to be able to provide the above functionality as well, we have to combine the code in the readObject
method. This requires code to distinguish between two streams, which is not easy as the hash code describing the
class and its compatibility is not available via a method call.
As the readObject and writeObject methods are defined on the evolved class the programmer is forced to
combine code in this way. A better solution would be to separate the two concerns into two classes: the evolved class
and a special class used to perform the translation between the original and new class, generating an instance of the
new, evolved class. This approach is described in more detail in section 12.4.
12.3
Refactoring the Class Hierarchy
The third kind of change we examine is to modify the class hierarchy. During the development of trees of objectoriented classes, it is common to move a class within the hierarchy. In our example, we swap the positions of
ColourPoint3D and Point3D, so that the functionality for colour is introduced before support for three dimensions.
This means that ColourPoint3D extends Point and Point3D extends ColourPoint3D10 . The modified code is
shown in figure 48.
The main changes are to the constructors Point3D and ColourPoint3D, as they call different super-types; also
Point3D now defines a serialVersionUID to ensure compatibility with the previous definition of Point3D given in
section 12.1. Point has not been changed.
Deserialize reads the stream containing the serialized Point state, casting the result to a ColourPoint3D
object. The current definition of ColourPoint3D as shown in figure 48 is a direct descendant of Point, so it does not
define a z field. Therefore, when the stream is deserialized and the ColourPoint3D is created, it only contains, x, y,
colour and date values (figure 49). Thus, the evolution has caused information to be lost.
10 This
class names are retained for purposes of explanation. However, in reality, these would be changed also to reflect the new semantics.
37
package evolution;
package evolution;
import
import
import
import
import
public class Point3D extends ColourPoint3D

{
private int z;
java.util.Date;
java.io.IOException;
java.io.ObjectInputStream;
java.io.ObjectOutputStream;
java.io.ObjectInputStream.GetField;
public class ColourPoint3D extends Point

{
long colour;
Date date;
static final long serialVersionUID =
-1421736715104280864L;
static final long serialVersionUID =

-1725384684763124259L;
public Point3D()
{
this.z = 0;
}
{
colour = 0;
date = new Date();
}
public Point3D(int x, int y, int z, String colour)

{
super(x, y, colour);
this.z = z;
}
public ColourPoint3D(int x, int y, String colour)

{
super(x, y);
}

{
return super.toString() + " z: " + z;
}

{
return super.toString() + " Colour: " +
colour + " Date: " + date;
}
private void writeObject(ObjectOutputStream stream)

throws IOException
{
stream.writeLong(colour);
stream.writeObject(date);
}
private void readObject(ObjectInputStream stream)
throws IOException,
{
String colour_string = (String)
fields.get("colour", null);
colour = colour_string.hashCode();
date = new Date();
}
}
Figure 48: Class Definition after Refactoring the Class Hierarchy
38

x: 5 y: 66 Colour: 64266207 Date: Sat Jun 26 17:53:44 GMT+01:00 1999
Figure 49: Deserializing After Refactoring the Class Hierarchy

12.3.1 Comment
Such a change has affected the code substantially. The class hierarchy that we are deserializing into now only consists
of two classes and, therefore, information has been lost. We cannot read from the stream, creating an instance of the
evolved Point3D as this is not compatible with the contents of the stream. We can only read from the stream in terms
of the evolved ColourPoint3D which now has less information. To access all the information in the stream, special
classes should be written that just perform the evolution of the stream. This means that the class would deserialize
the stream into a compatible ColourPoint3D, create an instance of the new most concrete type i.e. Point3D and
either use that or serialize that instance into a new file. The old serialized form would then become redundant. Such
a methodology of keeping only two versions of a class is encouraged for users of the object-serialization mechanism
because of the strict nature of compatible changes as defined in the object serialization specification [JOS97, section
5] and the inherently linear nature of using serialVersionUID.
Other changes that might be performed to the hierarchy include changing which class implements
java.io.Serializable. For example, if Point is evolved and it should no longer be capable of being written to
disk, it will no longer implement java.io.Serializable. However, it may be required that ColourPoint3D and
Point3D still be capable of being serialized. If this change is performed, all the sub-types of Point that need to be
serialized must be identified and have their source code changed to add implements java.io.Serializable to
them. The source code then needs to be recompiled. Identifying the sub-types could be non-trivial for a large code
base and this solution assumes the source code is available.
12.4
Constructor Based System
This section briefly outlines an alternative design for combining the support for evolution with the basic objectserialization mechanism. The code presented in this system is valid Java, however, it will not run as the necessary
support from the underlying serialization mechanism does not exist and GetField would need to be re-implemented.
This section has shown that the support for versioning in the object-serialization mechanism is closely coupled
with the normal serialization and deserialization mechanism. This is mainly because readObject and writeObject
are used for expressing conversion of the state between differing versions of the same type and for expressing nondefault object serialization and deserialization. This design proposes separating these two concerns. readObject
and writeObject are only to be used for specialising how an object is serialized and deserialized at that version.
Support for translating between two versions of the same type is handled by a different object that is registered with
the ObjectInputStream object. When an object instantiated from the old type is encountered in the stream, the
stream is passed to the conversion routine. This method then: gains access to all the fields of the old class; converts
them; and then generates an instance of the new type, returning this as a result to the object-serialization mechanism,
which makes this new object part of the deserialized graph of objects.
12.4.1 Deserializing via a Different Type

This design consists of three main ideas: specifying an interface to define the API for translating objects when reading
from the stream; defining a class to encapsulate the translation code; and registering with the input stream an object
that will perform the conversion.
12.4.2 Translation Interface

As the object that will perform the translation is called from the object-serialization mechanism, it should have a well
known signature. This signature is expressed in this interface:
The read method is passed the stream by the object serialization mechanism. The implementation of it extracts
the state of the original object and uses it to create an instance of the new class. This is then returned and entered into
the graph of deserialized objects.
39
package evolution;
public interface JOS_ConvertToCallback
{
public Object read(ObjectInputStream stream) throws IOException, ClassNotFoundException;
}
Figure 50: Convert To Callback

12.4.3 Conversion Object
The interface above is used by the programmer to provide an object that the programmer has defined to translate from
the implementation of the old object to the new one. For our ColourPoint3D object, this is given in figure 51.
package evolution;
import java.io.ObjectInputStream.GetField;
public class ColourPoint3D_Convert implements JOS_ConvertToCallback
{
public Object read(ObjectInputStream stream) throws IOException, ClassNotFoundException
{
int x = fields.get("x", 0);
int y = fields.get("y", 0);
int z = fields.get("z", 0);
String colour_string = (String) fields.get("colour", null);
return new ColourPoint3D(x, y, z, colour_string);
}
}
Figure 51: ColourPoint3D Conversion Object

ColourPoint3D Convert implements the JOS ConverToCallback. In the implementation of read all the fields
of the object are extracted, any conversions are performed and an object, instantiated from the new type is returned.
This is somewhat different to the current object-serialization system. In the current system, each class in the
hierarchy, if it wants to handle the stream manipulation itself, defines its own readObject, writeObject pair. As
the stream is deserialized, calling readObject proceeds from the the least-specific super-type (in our example this is
Point) to the most-specific (ColourPoint3D). The current read point in the stream is advanced as each readObject
method reads the part of the stream relevant to the state of its class.
The above solution assumes that the conversion object can read all the state from the stream and initialise an
instance of the most-specific type. Such a solution requires there to be some way of initialising the state of the
object and this will lead to specialised constructors and more set methods. However, this is a cleaner, more explicit
approach that the current system.
In the above solution it is only possible to have one definition of ColourPoint3D active in the code. This is
because Javas type identity is based on only the name of the type and the class loader that loaded it. It would be
more convenient if we could pass in the instance of the original ColourPoint3D to read. The method could then
be implemented purely in terms of two objects, rather than a stream and an object. This would be more natural for
the programmer. In addition, it would also be possible to build something similar to the proposal being discussed
40
here if Java defined a more powerful meta-level architecture, rather than the currently define reflection mechanism.
A system more akin to that supported by CLOS [KdRB91] would provide access to the necessary internals of objectserialization at the Java-level to facilitate such an approach.
12.4.4 Registering the Conversion Object

To ensure that a conversion object is called when an instance of the original class is seen in the stream, the conversion
object is registered with the input stream in figure 5211.
package evolution;
public class Deserialize
{
{
file = new FileInputStream("./test");
in = new ObjectInputStream(file);
ColourPoint3D_Convert convert = new ColourPoint3D_Convert();
in.register("evolution.ColourPoint3D", ColourPoint3D.getSerialVersionUID(), convert);
ColourPoint3D c_point = null;
c_point = (ColourPoint3D) in.readObject();
in.close();
}
}
Figure 52: Registering the Conversion Object

The in.register call says that if, during the deserialization of the stream, an object of type
evolution.ColourPoint3D is encountered that hashes to the value returned from
ColourPoint3D.getSerialVersionUID(), the stream should be passed to the read method defined on the object
convert. A similar mechanism can be defined for writing as, in some circumstances, it may be easier to convert the
objects as they are written to the stream and then just read them with evolved code.
12.5
Evolution Summary
This section has presented the support for evolution between versions of compatible classes in some detail (sections
12.1 to 12.1.5). Some limitations of using the system have been presented: the use of serialVersionUID is inherently linear, multiple ancestor versioning cannot be expressed and it encourages the programmer to keep only two
versions of any one class (sections 12.1.6 and 12.3.1); the system supports implicit change, which requires less code
from the programmer if the two versions of a classs fields are compatible; however, such automatic assignment
takes away from the programmer a lot of control and as such makes it difficult to access the serialization and deserialization mechanism to specialise it (section 12.1.7); the readObject and writeObject methods are used to describe
both the specialised manipulation of an objects state to and from the stream as well as to convert the state of an
object from the stream to the current object definition, if the change cannot be handled automatically because it is not
compatible (section 12.2); some changes to the class hierarchy can result in information loss that can be difficult to
recover after an evolution as the evolved types essentially provide a restricted view onto the underlying pre-evolution
stream (section 12.3.1. an outline of a design for an alternative versioning system was proposed that separated the
issues of using readObject and writeObject for the specialised manipulation of object state to and from the stream
from the manipulation of the contents of the stream for evolution between versions (section 12.4).
Using the object serialization systems versioning mechanism would be easier if additional tool support was
available. If a tool was able to tell the programmer which changes lead to incompatibilities and which do not, moving
between versions of the same class would be much easier.
11 The
exceptions for the code in figure 52 have not been included.
41
The support for versioning of classes in Suns Java 2 makes the transition of state from an old class definition
to a new definition easy if the evolution can be expressed in terms of additions to the old class. If the evolution is
more complicated, such as removing fields, more work is required from the programmer and more sophisticated tool
support would be appropriate.
13 Alternative Persistence Technology

There are many products that provide object persistence for Java. The purpose of this section is not to give an
exhaustive survey, but to present some alternative approaches, give a brief overview and supply references to them.
Solutions to Java object persistence are grouped into four broad categories: orthogonal persistence; objectoriented databases; relational databases; and distribution-based solutions. These solutions are much more comprehensive and powerful than object-serialization. This section should not be read as a comparison of object serialization
with these other technologies.
13.1
Orthogonal Persistence
An object-oriented orthogonally persistent programming language [AM95] seamlessly integrates the language with
the technology for saving and restoring objects and their code to and from stable storage. Such languages are referred
to as orthogonally persistent as any object, regardless of type, may be made persistent; the issues of typing and
persistence are orthogonal to each other. This means that the programmer does not have to make a decision about
which types are candidates for persistence and which are not.
One implementation of an orthogonally persistent Java is the PJama project [ADJ+96, PAD+97], a collaborative
project between Glasgow University and Sun Microsystems12.
In this system the programmers classes do not inherit from a Persistence class and they do not indicate their
suitability for persistence by implementing an interface such as java.io.Serializable. Rather, an object is identified to the persistence technology by making it reachable from a named root of persistence, and approach known
as persistence by reachability. This is something that can be decided at run-time, on an object by object basis, rather
than at compile time on a per class basis that affects all instances of a type. The object by object, run-time approach
is inherently more flexible than using java.io.Serializable.
The object graph is written to the store either when the programmer calls the org.opj.PJStore.stabilizeAll method
or when the program exits with a successful status, System.exit(0). Object serialization requires the programmer to
explicitly write the graph of objects to a file, so gaining the same effect as PJamas write on successful exit requires
the programmer to coordinate the termination of their program. In a multi-threaded environment, this can be quite
complex and is prone to error.
When starting a PJama-based Java program, objects are brought in from the store when they are first referenced.
In order to load an object, its class must also be available and this information is held in the store in the form of class
objects together with the associated bytecode. This is different to object serialization when only information about
the class is written to disk. PJamas approach has an implication for evolving classes: if the programmer changes
the class, there are now two versions: one that is resident in the store and the new version which lies outside it. If
the programmer wants to use the new version, the store must be evolved, the old class replaced and the state of all
instances converted to work with the new version. Such a tool exists for PJama, it is called opjsubst and is described
in [Dmi98].
PJama defines a version of RMI that supports distributed programming within an orthogonally persistent system.
In a system that supports persistence by reachability, if an object is reachable from a persistent root, it should be
made persistent. In the client/server model supported by RMI, any process can simultaneously be a server and a
client. Supporting persistence by reachability in a system of clients and servers, with objects remotely reachable from
other address spaces, raises a number of issues. For example, if part of the clients persistent state is passed as the
parameters of a remote method invocation and they become reachable from a persistent root in the server, then two
copies exist, one in the client and one in the server. This may be intentional, however, there may be some requirement
at the application-level to keep the replicated state of these two groups consistent. In addition, the graph of persistent
objects is likely to be quite deep and consist of many objects of widely differing sizes. If an object near the top of the
root of persistence is passed as a copied parameter to a remote method, a significant part of the persistent store will
have to be passed across the network with determental effects on performance. These issues and others are discussed
in the work on distribution support in PJama, called PJRMI [SA97, Spe97, Spe99].
12 http://www.dcs.gla.ac.uk/pjama/ and http://www.sunlabs.com/research/forest/.
42
13.2
Object-Oriented Databases
13.2.1 GemStone/J
Gemstone/J [Gem98] is an enterprise-level Java application server that is intended for building and deploying large,
mission-critical Java applications. It is based on a three-tier client/server architecture together with support for the
World Wide Web.
The programming model offered by GemStone/J [BOS+ 98] is similar to that for PJama. Objects are identified to
the persistence mechanism via reachability from a named root of persistence. However, the main difference between
PJama and GemStone/J is the environment in which they are expected to work. GemStone/J is intended to function as
the middle tier of a three-tier architecture made up of web browsers and Java/Corba clients as the first tier, GemStone/J
in the middle with RDBMS and mainframes as the third tier. GemStone/J presents an object-oriented view to the first
tier and provides the necessary integration to support legacy systems in the third tier. PJama, on the other hand, is
a research project conducting an investigation into providing efficient support for orthogonal persistence in Java. As
such there is less focus on integration with legacy systems and multiple-users.
13.2.2 O2
O2 [BDK92] is similar in many ways to GemStone/J. O2 is an object database system that enables developers to
build industrial-strength database applications within an open environment. The main difference between O2 and
Gemstone/J is that at the centre of O2 there is a database engine that that can be queried using OQL, the object query
language [CBB+ 97]. A programmer may also use C++ and Java to communicate with the database. O2 also supports
persistent roots which can be used in a similar manner to PJama and GemStone/Js.
13.3
Relational Databases
One way to connect a Java program to a relational database is to use a Java-based product that implements the Java
Database Connectivity (JDBC) specification [WH98]. The paradigm for communicating with a relational database
using the JDBC is to: load the necessary JDBC driver for the particular database; connect to it, getting back a
Connection object; use this object to generate a Statement object, which has an SQL statement embedded within it
as a String; send this statement to the database; and receive a ResultSet object in return; and then use this object
to extract the result of the query.
In this approach the data is stored in a relational database that is outside the Java virtual machine. The relational
tables that are retrieved are presented to the programming in terms of objects and the JDBC defines many classes to
conveniently access the rows and columns in results. The client side of the architecture is inherently transient and the
programmer is responsible for mapping their data from the object-oriented model supported by Java into one that is
suitable for a relational database. Such a mapping can be expensive to develop and maintain and so the advantages
of an orthogonal model as promoted by PJama are lost. However, the JDBC does give convenient access to relational
legacy data from a Java program.
13.4
Distribution-based Solutions
Our last category allows Java programmers to store their objects in distributed, stable storage. This section divides
these systems into two groups: commercially supported systems and research systems.
13.4.1 Commercially Supported Systems

PSE
ObjectStores PSE Pro [Abe97] is a database management system targeted at embedded devices and mobile systems.
To gain access to objects in the database the programmer: opens the database; starts a transaction; obtains a reference
to a persistent object by using a database root, an external reference or by navigating to the required object; and
then uses the reference to the object. PSE requires the programmer to run a post-processor on the applications
Java class files. This ensures that the database understands that the objects are capable of being made persistent.
The programmer has to tell the post-processor which classes will be put into the object store (persistence-capable),
which classes handle those objects (persistence-aware), and which classes are present but are not involved with the
stored objects. Pre-processing can require some standard Java classes to be recompiled in order to be post-processed
correctly.
43
JavaSpaces
JavaSpaces [Wal98] is a product from Sun Microsystems that supports concurrent, distributed, persistent programming using a model very similar to that of Linda [Gel92]. A JavaSpace is a Java virtual machine that provides a
persistent storage area for application objects from remote clients. Distribution is handled by RMI and persistence
is provided using object-serialization. The main operations on the JavaSpace are to: write objects; read a copy,
leaving the original intact in the space; and to take an object from the space, removing the original. The focus of
JavaSpaces is on providing a platform to make designing and implementing distributed computing systems easier
than it currently is, rather than on providing a sophisticated persistence solution.
JavaBeans
JavaBeans is Sun Microsystems component architecture for Java [Ham97]. A JavaBean is a Java class that: supports
introspection, so that other beans and code can analyze how it works; customization, so that the beans appearance
and behaviour can be changed; generate and receive events, in order to communicate with other beans; provides a set
of properties that help to describe the bean to other code; and provides support for persistence so that any state may
be saved and retrieved later. The default mechanism for persistence in JavaBeans is provided by object-serialization.
Indeed, as specified in [Ham97][section 5.1], all beans must support serialization or externalization.
CORBA Persistent Object Service

CORBA [Obj95] is the Common Object Request Broker Architecture and its focus is on distributed computation
between objects executing in heterogeneous languages hosted on different hardware running disparate operating systems. The main component of a CORBA architecture is the object request broker, or ORB, that acts as a third party,
putting clients in touch with servers. One of the CORBA Services defined for version 2.0 of the CORBA specification is the Persistent Object Service Specification [Obj97, Chapter 5]. The persistence specification defines a common
interface to the mechanisms for storing and managing the persistent state of objects. The specification divides the
applications object into two parts, the non-persistent, dynamic state and the persistent state. A Persistent Data Service (PDS) is defined that actually implements the mechanism for making data persistent. A protocol within the PDS
defines the way data is moved in and out of the object and how the interface to the underlying Data Store is handled.
The Data Store is the lowest-level interface that is concerned with interacting with storage devices. In this system,
clients control the persistence of an object explicitly. Persistent Identifiers (PIDs) are used by client programs to
describe the location of an objects persistent data.
In [KPT96], the authors present the lessons learnt from implementing the Persistent Object Service. The authors
identify two main problems with designing and implementing the persistence service: firstly, the OMG intentionally
leaves the functional core of the persistence service unspecified and, secondly, the OMG encourages reuse of other
object services, within the context of the persistence server, without being specific enough. The authors conclude that
the direct reuse of the other object services is impossible given the current object persistence specification and that
enhancements are require to the other specifications, e.g. the Compound Externalization Service.
13.4.2 Research Systems

Thor
Thor [LCSA99] is a transactional persistent object store that provides access to objects over a wide-area, large-scale,
distributed environment. Thor provides a universe of persistent objects, so the existence of objects is not tied to the
running of particular programs. The universe has a persistent root and all objects reachable from it are persistent.
Persistent objects always reside within Thor. An application communicates with Thor via an API to retrieve the
persistent root, to commit and abort transactions, and to retrieve handles to Thor objects. Once a client has a handle
to a Thor object, only the methods of that object may be invoked. A thin veneer is provided at the client to allow
applications to be written in a number of programming languages, such as Tcl/TK, C++ and Java. Object handles
may travel beyond the veneer boundary, into the application program. Objects in Thor are copied from the Thor
server to the Thor client on the client machine. As a result, a lot of attention has been paid to the caching strategy,
concurrency control and management of moving modified objects to disk.
The focus of Thor is different from that of PJama. Although both systems are based on persistence by reachability,
Thor focuses more on the coherent copying of objects from server to client machine and back again. PJama, however,
focuses on the issues of providing orthogonal persistence and, within PJRMI, extending this model to a distributed
system. Thus, in Thor, object references will tend to point from a Thor client to other objects within the same client,
whereas object references in PJama may identify any remote object within a group of virtual machines, as objects are
not copied to the client.
44
PerDis
PerDis [FSB+98] is a persistent distributed store intended for large-scale object sharing. Its architecture is very
similar to that of Thor and PerDis also defines persistence in terms of reachability. A machine in a PerDis system
is made up of: an application; an API onto the PerDis user-level library (ULL); the user-level library itself; and a
PerDis daemon which interacts with the ULL. The PerDis daemon is responsible for logging and moving objects to
and from disk and the ULL deals with application-level memory mapping and the management of clusters of objects,
locks and transactions. A cluster is an abstraction for the physical grouping of logically-related objects. Programs
allocate an object in a specific cluster, clusters have names and attributes and they are persistent. The cluster is the
user-visible unit of naming (based on URLs), storage and security. PerDis focuses on object caching and coherency
and the management of its clusters with transactions. As such, the comment made above on PJama and Thor also
applies to PerDis.
14 Conclusions
This paper has shown, with numerous supporting examples, that using Javas object-serialization mechanism to provide object persistence is inappropriate. The system appears simple on the surface (section 1) but there are many
implications from relying on it as a persistence technology. The programmer must state the types that are candidates
for persistence at compile time, whereas making this decision at run-time, on a per-object basis, is more appropriate
(section 2). The serialization mechanism suffers from the big-inhale problem where the whole graph must be read
before it can be used (section 3); loading objects on demand is more efficient, reducing delay in starting an application. The serialization mechanism creates copies of objects that it writes and reads. This can break some code
that makes assumptions about the hash code of an object (section 4), requiring fundamental methods such as hashCode and equals to be overridden. Static values are not written to the store by default, requiring the programmer
to manage this explicitly, using readObject and writeObject methods (section 5). The fields of a class that are
marked transient are not followed by the serialization mechanism, thus the programmer has the same problem as
with statics (section 6). Another mechanism (serialPersistentFields) for specifying which fields of a class are
to be serialized was given in section 7. This mechanism uses an array with a well-known signature for specifying
the fields that will be serialized. Although this allows the choice of which objects will be serialized to be deferred to
run-time, its implementation severely limits its usefulness. Section 8 showed that concurrent activity over the graph
while it is being serialized can introduce inconsistencies at the application level. Problems with remote invocations
to server processes while the processes were using object serialization for persistence were discussed in section 9 and
it was shown that this can affect client code. Section 10 showed that the methods readObject and writeObject
are being used for two purposes: specialised serialization for persistence and specialised serialization for distribution.
This expands to three purposes if we consider using these methods for type translation during evolution as well. This
forces the programmer to add code for more than one purpose to these methods, making their implementation and
maintenance very difficult. Section 11 highlighted a problem that required the use of a specialised classloader if an
instance of a remotely loaded class was written to the store. Section 12 discussed the versioning support available in
object serialization and it was shown that moving state between instances in a stream and instances of new versions
of the same type was implicit and automatic. While this is convenient for the programmer, the migration of state is
implicit and without extra code is not under their control. The implications of using the versioning system of Java
object serialization were also discussed in this section. Section 13 showed that there many other solutions to the problem of providing object persistence for Java and that they have widely varying degrees of sophistication and impact
on an application.
15 Acknowledgements
The author would like to thank Susan Spence for interesting discussions about the implementation and use of the
object-serialization mechanism, for suggesting discussing the copying semantics in section 4 and for reading a draft
of this paper. The author also thanks John Hagemeister and Tony Printezis for commenting on an earlier draft.
45
References
[Abe97]
Steven T. Abell. Using Java With PSE. Netscape Communication Corporation, 1997.
[ADJ+96] M. P. Atkinson, L. Daynes, M. J. Jordan, T. Printezis, and S. Spence. An orthogonally persistent Java.
ACM SIGMOD Record, 25(4):6875, December 1996.
[AM95]
M. P. Atkinson and R. Morrison. Orthogonally persistent object systems. VLDB Journal, 4(3):319401,
1995.
[BDK92]
F. Bancilhorn, G. Delobel, and P. Kanellakis, editors. Building an Object-Oriented Database System The Story of O2. Morgan Kaufmann, San Mateo, 1992.
[BOS+ 98] Bob Bretl, Allen Otis, Marc San Soucie, Bruce Schuchardt, and R Venkatesh. Persistent Java Objects
in 3 tier Architectures. In Malcolm Atkinson and Mick Jordan, editors, The Third Persistence and Java
Workshop, Tiburon, California, September 1st to 3rd 1998.
[CBB+ 97] R. G. G. Cattell, D. Barry, D. Bartels, M. Berler, J. Eastman, S. Gamerman, D. Jordan, A. Springer,
H. Strickland, and D. Wade. The Object Database Standard: ODMG 2.0. Morgan Kaufmann Publishers,
Los Altos (CA), USA, 1997.
[Dmi98]
Misha Dmitriev. The First Experience of Class Evolution Support in PJama. In Malcolm Atkinson and
Mick Jordan, editors, The Third Persistence and Java Workshop, Tiburon, California, September 1st to
3rd 1998.
[FSB+98] Paulo Ferreira, Marc Shapiro, Xavier Blondel, Olivier Fambon, Joao Garcia, Sytse Kloosterman, Nicolas
Richer, Marcus Roberts, Fadi Sandakly, George Coulouris, Jean Dollimore, Paulo Guedes, Daniel Hagimont, and Sacha Krakowiak. PerdiS: Design, implementation, and use of a PERsistent DIstributed store.
Technical Report RR-3525, Inria, Institut National de Recherche en Informatique et en Automatique,
1998.
[Gel92]
D. Gelernter. Current research on linda. In Jean Pierre Banatre and Daniel Le Metayer, editors, Proceedings of Research Directions in HighLevel Parallel Programming Languages, volume 574 of LNCS,
pages 7476, Berlin, Germany, June 1992. Springer.
[Gem98]
GemStone Systems, Inc. GemStone/J Programming Guide, 1.1 edition, March 1998.
[GHJV96] E. Gamma, R. Helm, R. Johnson, and J. Vlissides. Design Patterns: Elements of Reusable Objectoriented Software. Addison Wesley, Reading, 1996.
[GJS96]
James Gosling, Bill Joy, and Guy L. Steele. The Java Language Specification. The Java Series. AddisonWesley, Reading, MA, USA, 1996.
[Ham97]
Graham Hamilton. JavaBeans API Specification. Sun Microsystems, v1.01 edition, July 1997.
[JOS97]
Java Object Serialization Specification. Sun Microsystems Technical Specification, Revision 1.2, JDK
1.1 FCS, February 10, 1997.
[KdRB91] Gregor Kiczales, Jim des Rivi`eres, and Daniel G. Bobrow. The Art of the Metaobject Protocol. MIT
Press, 1991.
[KPT96]
Jan Kleindienst, Frantisek Plas il, and Petr Tuma. Lessons learned from implementing the CORBA persistent object service. In Proceedings of the Conference on Object-Oriented Programming Systems,
Languages, and Applications, volume 31, 10 of ACM SIGPLAN Notices, pages 150167, New York,
October 610 1996. ACM Press.
[LCSA99] Barbara Liskov, Miguel Castro, Liuba Shrira, and Atul Adya. Providing Persistent Objects in Distributed
Systems. In Rachid Guerraoui, editor, Proceedings of the European Conference on Object-Oriented Programming (ECOOP 99), volume 1628 of LNCS, pages 230257, Lisbon, Portugal, jun 1999. Springer.
[NEED]
NEEDED. needed. NEEDED, NEEDED NEEDED.
[Obj95]
Object Management Group. The Common Object Request Broker: Architecture and Specification, July
1995. Version 2.0.
[Obj97]
Object Management Group. CORBAServices, December 1997. Version 2.0.
[PAD+97] Tony Printezis, Malcolm P. Atkinson, Laurent Dayn`es, Susan Spence, and Pete Bailey. The design of a
new persistent object store for PJama. In Proceedings of the Second International Workshop on Persistence and Java (PJW2), Half Moon Bay, CA, USA, August 1997.
46
[PC98]
Doug Kramer Patrick Chan, Rosanna Lee. The Java Class Libraries, Second Edition: Volume 1: java.io,
java.lang, java.math, java.net, java.text.java.util. The Java Series. Addison-Wesley, Reading, MA, USA,
1998.
[SA97]
S. Spence and M. Atkinson. A Scalable Model of Distribution Promoting Autonomy of and Cooperation
Between PJava Object Stores. In Proceedings of the Thirtieth Hawaii International Conference on System
Sciences, Hawaii, USA, January 1997.
[Spe97]
S. Spence. Distribution Support for PJama. http://www.perdis.esprit.ec.org/events/java-wkshp-971020/

PerDiS Esprit Project Workshop on Persistence and Distribution in Java, INESC, Lisbon, Portugal, October 1997.
[Spe99]
S. Spence. PJRMI: Remote Method Invocation for a Persistent System. In Proceedings of the International Symposium on Distributed Objects and Applications (DOA99), Edinburgh, Scotland, September
1999. IEEE Press.
[Wal98]
J. Waldo. Javaspace specification - 1.0. Technical report, Sun Microsystems, July 1998.
[WH98]
Seth White and Mark Hapner. JDBC 2.0 API Specification. Sun Microsystems, 1.0 edition, May 1998.
47
Disk Read Cost Experiment
In order to calculate the cost of performing a serialization and deserialization of an object graph a three part experiment
was devised. To calculate the cost of performing a graph serialization, the experiment consists of two parts, each
writing one kind of data to the disk. The first kind is an n-ary tree initialised as a binary tree consisting of between
1000 and 14,000 elements, increasing in units of 1000. The node (TestNode) in the tree is an object consisting of
a single field, of one byte. No methods are defined. This node is wrapped inside another node, an MNode, which is
actually inserted into the tree. This node consists of a string key, so the node can be found, and an array to the nchildren the node refers to. To ensure the key is unique it is generated from a monotonically increasing integer which is
reset for each tree size. A new BTree was created for each of the 14 serializations and the current time in milliseconds
was taken immediately before and after the serialization of the graph using System.currentTimeMillis. The sizes
of the 14 stores were recorded and used in the second part of the experiment.
The second part of the experiment serialized a single byte array to disk. This was performed 14 times, once for
each of the sizes gained from the first experiment. This experiment gives a feel for the base cost of moving the number
of bytes to disk that are being written in the first experiment. Comparison of the two figures then shows how much
overhead there is in serializing the more complex binary tree object graph. The sizes of the stores are 69,662 bytes
for the 1000 node binary tree, increasing to 1,024,766 bytes for the 14,000 element tree. An increase of 1000 nodes
increases the store size by approximately 73KB.
The third part deserialized the 14 stores. The current time in milliseconds was captured before and after the
deserialization was performed.
A.1
Experimental Setup
All the experiments were performed on a lightly loaded Sparc Ultra-4 (called mars) running Solaris 2.5.1 with 1.5Gb
of main memory. Each experiment was conducted several times, once in a different configuration of disk and use of
just-in-time (JIT) compilation. The Java virtual machine was run with all values defaulted.
Three disk configurations were used, one NFS-based and two local. The NFS disk was served from a machine
(called bathurst) running Digital UNIX V3.2C, with 196Mb of main memory. mars is connected to the departmental
gigabit network with 100Mb/s network card. bathurst is connected to the same network via a 10Mb/s network card.
The two disks local to mars were tmp, the machines swap space, and extra, a directory on a local, non-swap disk.
As /tmp is the swap space for the machine, the disk is memory resident, thus no actual writes to a disk will be
performed. Timing the write for the NFS mounted disks give times very similar to that for /tmp. These writes are
actually writes to the local cache on the machine that initiates the writing of the data.
The experiments were run using version 1.2 of Suns Java JDK and each experiment was run once with and
without just-in-time compilation selected. Each experiment was conducted ten times and an average of the elapsed
times was taken.
A.2
Results
Section A.2.1 compares the amount of time taken to serialize the binary tree to an NFS mounted disk and the equivalent amount of raw bytes from a byte array. Section A.2.2 describes the difference between using /tmp, /extra and
the NFS mounted disk for reading and writing the binary tree and the raw byte array.
A.2.1
Writing a Tree and a Byte Array
The time to serialize a graph of objects to disk is dominated by the manipulation of the graph. The overhead of
moving the data to and from the disk is, in comparison, very small. This is illustrated on figure 53 which shows, using
a logscale, the time to write to the NFS disk.
The top curve represents the amount of time taken to serialize the BTrees of varying sizes. The bottom curve is
the time taken to serialize the same number of bytes to disk when they are stored as a simple byte array. Serializing
the data to disk is very fast. For example, it takes approximately 50ms to save 1Mb of data to disk. By contrast it
takes approximately 5000ms to save the same number of bytes from a binary tree consisting of 14,000 elements. The
anomalous first value for the lower curve is because Java is using unoptimised bytecodes. This is not noticeable in
the upper curve as the time to optimise the bytecodes is dominated by the time to process the tree.
A.2.2
Performance Using the Different Disk Configurations
This section divides in two the results of using the three different disk configurations. First, figures for reading the
raw byte array data and the binary tree from disk are given, then figures for writing the data are given.
48
Disk Write to NFS Disk (jit on)

10000
Tree Write
Raw Write
Elapsed Time (ms)

(logscale)
1000
100
10
1
0
100
200
300
400
500
600
700
Number Bytes Written (x 1000)
800
900
1000
1100
Figure 53: Comparison of Serialization and Data Transfer Rates

Reading
Figure 54 shows results for reading the file that contains the binary tree. For larger store sizes, NFS starts to become
more efficient. For example, for the largest store of 1,024,766 bytes, NFS reads the data 2.9s faster than reading from
/tmp and 4.2s faster than reading from /extra. This faster read can be attributed to the use of caching with NFS.
Once the NFS start up cost has been paid, the data is a read from the memory of the local machine (assuming it is
cached there). If it is not in the local cache, the data may be in the remote cache on the file server, and it is quicker to
read from the memory of the remote machine across the network, than it is to read from the local disk.
Disk Read Comparison for Tree over NFS, /tmp and /extra (jit on)
25000
NFS Tree Read
/tmp Tree Read
/extra Tree Read
Elapsed Time (ms)
20000
15000
10000
5000
0
0
100
200
300
400
500
600
700
Number Bytes Read (x 1000)
800
900
1000
1100
Figure 54: Reading the Binary Tree from the Three Disks
Figure 55 shows a similar result to figure 54, namely that using NFS is quicker for larger files. The first initial
read is expensive as unoptimised Java bytecodes are being used.
Figure 56 shows figures for reading the binary tree data from the three kinds of disk with and without JIT selected.
In this graph there is no clear separation of curves based on using JIT (cf. figure 57). The dominant effect is whether
an NFS mounted disk is being used as the two lower curves are for reads from the NFS disk when JIT is on and JIT
is off. The use of JIT together with NFS gives the quickest read.
49
Disk Read Comparison for Raw Data over NFS, /tmp and /extra (jit on)
60
NFS Raw Read
/tmp Raw Read
/extra Raw Read
55
50
Elapsed Time (ms)
45
40
35
30
25
20
15
10
5
0
100
200
300
400
500
600
700
Number Bytes Read (x 1000)
800
900
1000
1100
Figure 55: Reading the Raw Byte Array from the Three Disks
Disk Read Comparison of using JIT for Tree over NFS, /tmp and /extra ((jit on))
25000
NFS Tree READ (jit on)
/tmp Tree READ (jit on)
/extra Tree READ (jit on)
NFS Tree READ (jit off)
/tmp Tree READ (jit off)
/extra Tree READ (jit off)
Elapsed Time (ms)
20000
15000
10000
5000
0
0
100
200
300
400
500
600
700
800
900
1000
1100
Figure 56: Reading the Tree from the Three Disks with and without JIT
50
Writing
Figure 57 shows the elapsed times for writing the binary tree to the serialized stores of various sizes. Here, no disk
configuration is superior.
Disk Write Comparison for Tree over NFS, /tmp and /extra (jit on)
5500
NFS Tree Write
/tmp Tree Write
/extra Tree Write
5000
4500
Elapsed Time (ms)
4000
3500
3000
2500
2000
1500
1000
500
0
0
100
200
300
400
500
600
700
800
900
1000
1100
Figure 57: Writing the Binary Tree to the Three Disks

Figure 58 shows the cost of writing the raw byte array to the three kinds of disk. The first data value is because of
unoptimised Java bytecodes. Writing to /extra is significantly more expensive as the write is to a disk, whereas the
other two, the data is only copied into an area of memory.
For approximately 1Mb of data, writing to /extra takes 180ms and for 147K (the second data point, after the
bytecodes are optimised), the write takes 40ms.
Disk Write Comparison for Raw Data over NFS, /tmp and /extra (jit on)
160
NFS Raw Write
/tmp Raw Write
/extra Raw Write
140
Elapsed Time (ms)
120
100
80
60
40
20
0
0
100
200
300
400
500
600
700
800
900
1000
1100
Figure 58: Writing the Raw Byte Array to the Three Disks
Figure 59 shows performance figures for writing the binary tree data to the three kinds of disk with and without
JIT selected. When writing the data there is a clear advantage to having just-in-time compilation selected. For the
1024KB store, the write using JIT is approximately twice as fast as that without it.
A.3
Optimising the Tree
The experiments above were originally conducted with Suns Java JDK 1.1.6 with no just-in-time compilation installed. Without using JIT technology at this version of the JDK, the time spent serializing the String key value is
51
Disk Write Comparison of using JIT for Tree over NFS, /tmp and /extra (jit on)
10000
NFS Tree Write (jit on)
/tmp Tree Write (jit on)
/extra Tree Write (jit on)
NFS Tree Write (jit off)
/tmp Tree Write (jit off)
/extra Tree Write (jit off)
9000
8000
Elapsed Time (ms)
7000
6000
5000
4000
3000
2000
1000
0
0
100
200
300
400
500
600
700
800
900
1000
1100
Figure 59: Writing the Tree to the Three Disks with and without JIT
significant for larger trees. For example, 4.3s can be saved when serializing the largest store. This section describes
this optimization.
During the course of the serialization experiment a further experiment was conducted to calculate the elapsed
time of running the BTree tests when the String based key was made transient. For large trees the length of the String
becomes longer and there are more of them. Therefore, the curve for the BTree starts to diverge from the optimised
case (figure 60) as the BTree is forcing the keys to be serialized. This shows that it is costly to serialize Strings in
Java. For example, for the 10,000 to 14,000 element trees, the difference in store size and time to serialize them is
given in table 1.
Figure 60 also contains the results for the third experiment, the elapsed time to deserialize each of the 14 stores.
As can be seen, deserializing a store is significantly quicker than serializing it. When serializing a graph a number of
tests have to be made that are not applicable when deserialization is performed. For example, serialization requires
that cycles are dealt with correctly to ensure infinite recursion does not occur. During deserializing, each node can
simply be read from disk and the pre-serialization shape reassembled; a test for infinite recursion is not necessary.
For large stores the difference is quite substantial. For the largest, consisting of 1,024,706 bytes, the time to serialize
is 1.81 minutes (109328ms), whereas the time to deserialize is 39.26s (39269 ms).
The figures for deserializing the BTree are improved by the small number of classes it is made up from. The
BTree is implemented using only four classes: BTree implements the BTree functionality; MNode handles the n
children a node may have; Entry objects contain the particular user level node; and TestMNode is the node used by
the experiment containing the single byte. Therefore, when the store is deserialized the virtual machine has to load in
only four classes in order to read in the entire graph. For a more complex store, made up of numerous object types,
the virtual machine would have to temporarily suspend deserialization to load the classes the store is made up of, thus
increasing the overall read time. The class only has to be located and read the first time an object of that type is found
in the store, subsequent objects of the same type can just be read in.
The serialization and deserialization systems behave in quite different ways as the size of the store increases. The
deserialization mechanism is approximately linear, whereas serialization more accurately fits a gentle squaring effect.
If the store size were to increase in size significantly over the 1Mb size, the time to serialize the store would become
very expensive, tens to hundreds of seconds.
No. Elements
(thousands)
10
11
12
13
14
Unoptimised Store
Size (bytes)
739,114
810,887
888,327
957,882
1,024,706
Optimised Store
Bytes Saved
68921
76921
84921
92921
100921
Time Saved
(ms)
3280.6
4192.2
4232.2
3901.5
4374.0
Table 1: Savings Made when BTree Key is Transient
52
120000
Unoptimised BTree
Optimised BTree
Read Unoptimised BTree
100000
Serialization time (ms)
80000
60000
40000
20000
0
2
10
12
Run Number
Figure 60: Serialization Times for Optimised and Unoptimised BTree

At the end of the previous section the time taken to deserialize the various unoptimised BTree stores is given. For
the largest store the time taken is approximately 86s. Not only does this have an impact on the code, it also affects
the user. As such there is such a long delay in starting the program they may think that something has gone wrong.
This can be solved by giving the user feedback on the fact the store is being read and the likely time it will take to be
read in. However, once again, this increases the complexity of the application.
A.4
Conclusions
The experiments above allow us to draw four conclusions: the absolute amount of time to read and write a store is
large; reading a store is much slower than writing a store; and if an application is likely to exhibit more reading than
writing, an NFS mounted disk should be used; the use of JIT technology significantly increases the speed of using
Java object serialization.
A.4.1
Absolute Time
Using NFS, it takes 1.30s to read the smallest binary tree store of 69KB and 18.04s to read the largest (1024KB).
An increase of 1000 nodes (or 73KB), increases the read time by approximately 1.28s. Writing the smallest binary
tree store takes 0.03s, writing the largest takes 4.94s and an increase of 1000 nodes increases the write time by
approximately 0.03s.
For the raw byte array, the absolute times are much smaller. For example, writing the 1024KB store to disk using
NFS takes 48ms, and reading it takes 38ms.
Therefore, the time to move data to and from disk for Java object serialization is dominated by the manipulation
of the graph.
A.4.2
Reading is Much Slower than Writing
Reading the largest binary tree store from the NFS disk takes 18.04s, whereas writing the same amount of data to the
same disk takes only 4.94s. Reading is likely to be slower as more objects have to be created as the file is deserialized
and the graph is constructed. For stores that have a large number of different types, this value may be made worse by
the dynamic loading of the class file for the newly read object. Each time a new type is encountered, the class file for
it must be loaded, this could slow down the deserialization of the graph quite considerably.
A.4.3
Use NFS if Reads Outweigh Writes
If the application exhibits more reads than writes of the serialized data, using an NFS mounted disk will improve the
performance of the application.
53
A.4.4
JIT Technology
Comparing the results in section A.2 with those in section A.3 we can see that the use of JIT technology and the
move from Suns Java JDK 1.1.6 to 1.2 significantly increases the speed of using object serialization. For the largest
store in the no-JIT experiment using Java JDK 1.1.6 over NFS the time to write the 1024KB store is 1.81 minutes
(109328ms), and the time to read it is 39.26s (39269ms). This can be contrasted with a time of 4.9s to write the same
amount of data from an equivalent tree using Java JDK 1.2 using JIT and 18.04s to read the same amount of data.
Thus, without JIT, write is significantly slower than reading, a situation that is reversed when using JIT.
54

Serialization

Transféré par

Informations du document

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Serialization

Transféré par

Droits d'auteur :

Formats disponibles

Why Object Serialization is Inappropriate

for Providing Persistence in Java

public class Test

String s = new String("Will be serialized");

Figure 1: Serializing an Object to a File

public class Deserialize

Figure 2: Deserializing the Object from the File

2 Serialization and Deserialization

2.1.1 Serialization Model

Figure 3: Two Object Graphs

Figure 4: Making a Classs Objects Serializable

2.1.2 Classes Required for Serialization

2.2.1 Classes Required for Deserialization

2.2.2 Performing a Deserialization

Figure 5: Deserializing a Stream of Objects

3 The Big Inhale and Exhale Problem

objects may leave via a socket or by using RMI, for example.

public class Key implements

public class Value implements

Figure 6: Key and Value Classes

public static Object deserialize()

Figure 7: Serialization and Deserialization of Hashtable

kona:huw% java copying.Test

Figure 8: Output from Running the Code Fragment in Figure 7

Figure 9: A Definition of Key Suitable for Object Serialization

kona:huw% java copying.Test

Singleton s_ton = Singleton.getReference();

Singleton s_read = null;

Figure 14: Using the readObject and writeObject Methods

has a complete list.

7 The serialPersistentFields Field

public class B implements

public String toString()

public class C implements

Figure 15: Using serialPersistentFields to Specify the Fields for Serialization

Figure 16: Serializing and Object of Class A into a File

7.1.1 Exact Type is Required

7.1.2 Limited Array Flexibility

Figure 17: Specifying a Subtype to be Serialized

Figure 18: Updating serialPersistentFields at Run-time

public static void main(String argv[])

Figure 19: Using Mutatable serialPersistentFields

Figure 20: Running After Changing serialPersistentFields

Changing serialPersistentFields Before Deserialization

public static void main(String argv[])

Figure 21: Changing serialPersistentFields Before Deserialization

kona:huw% java fields.Test

7.1.3 Changing serialPersistentFields During Deserialization

Figure 23: Serializing static Variables

7.1.4 Overriding Transient

Tony Printezis who pointed out this behaviour.

kona:huw% java fields.Test

Figure 24: Exception Thrown when Changing serialPersistentFields During Serialization

serialPersistentFields is an alternative to the transient keyword. The programmer specifies what is to be

public class A implements

public class B implements

private final static ObjectStreamField[]

private final static ObjectStreamField[]

public void changeSerialized(int idx,

public String toString()

Figure 25: Serializing static Variables

Figure 26: Output of Serializing static Variables

Figure 27: Object Graph Concurrently Mutated and Written to Store

public class NameAddress implements java.io.Serializable