Académique Documents
Professionnel Documents
Culture Documents
Abstract Computational reection is gaining interest in practical applications as witnessed by the use of reection in the JAVA programming environment and recent work on reective middleware. Reective systems offer many different reection programming interfaces, the so-called MetaObject Protocols (MOPs). Their design is subject to a number of constraints relating to, among others, expressive power, efciency and security properties. Since these constraints are different from one application to another, we should be able to easily provide specially-tailored MOPs. In this paper, we present a generic reication technique based on program transformation. It enables the selective reication of arbitrary parts of object-oriented metacircular interpreters. The program transformation can be applied to different interpreter denitions. Each resulting reective implementation provides a different MOP directly derived from the original interpreter denition. Keywords: reection, OO languages, program transformation, language implementation
1 Introduction
Computational reection, that is, the possibility of a software system to inspect and modify itself at runtime, is gaining interest in practical applications: modern software frequently requires strong adaptability conditions to be met in order to t a heterogenous and evolving computing environment. Reection allows, for instance, host services to be determined dynamically and enables the modication of interaction protocols at runtime. Concretely, the JAVA programming environment [java] relies heavily on the use of reection for the implementation of the JAVA B EANS component model and its remote method invocation mechanism. Furthermore, adaptability is a prime requirement of middleware systems and several groups are therefore doing research on reective middleware [coi99][bc00]. Reective systems offer many different reection programming interfaces, the so-called MetaObject Protocols (MOPs)1 . The design of such a MOP is subject to a number of constraints relating to, among others, expressive power, efciency and security properties. For instance, using reection
Extended version. c 2001 Kluwer Academic Publishers. Higher-Order and Symbolic Computation, 14(1), 2001, to appear. 1 We use the term MOP in the sense of Kiczales et al. [kic91] (page 1): Metaobject protocols are interfaces to the language that give users the ability to incrementally modify the languages behavior and implementation, as well as the ability to write programs within the language.
for debugging purposes may require the MOP to provide access to the execution stack. However, because of security concerns stack access must frequently be restricted: in JAVA , for example, it is not allowed to modify the (untyped) stack because security properties essentially rely on type information. Since these constraints are different from one application to another, we should be able to provide a specially-tailored MOP for a particular set of constraints. Moreover, the constraints may change during the overall software life cycle. Hence, the development of such specially-tailored MOPs should be a lightweight process. Traditional approaches to the development of MOPs do not meet this goal instead each of them only provides a specic MOP which can hardly be modied (see the discussion of related work in Section 9). Consider, for instance, a single-processor application which is to be distributed. In this case, distinct tasks have to be performed on the message sending side and the receiving side: for example, on the sender side local calls are replaced by remote ones (instead of relying on proxies) and on the receiver side incoming messages can be synchronized. Many existing MOPs do not allow the behavior of message senders to be modied. Hence, such a distribution strategy cannot be implemented using reection in these systems. Some systems (see, for instance C ODA [aff95]) provide access to senders right from the start. Therefore, they can introduce an overhead for local applications. In this paper, we present a reication mechanism for object-oriented interpreters based on program transformation techniques. We use a generic transformation which can be applied at compile time to any class of a non-reective interpreter denition. This mechanism can be used to transform different subsets of a metacircular interpreter in order to generate increasingly reective interpreters. It can also be applied to different interpreter denitions in order to automatically get different reective interpreters. Each resulting reective implementation provides a different MOP directly derived from the original interpreter denition. The paper is structured as follows: in Section 2, we briey introduce Smiths seminal reective towers upon which our work is based and we sketch the architecture of our transformational system. Section 3 provides an overview of a metacircular interpreter for JAVA. Our generic reication technique is formally dened and its application to the non-reective interpreter is exemplied in Section 4. Section 5 is devoted to reective programming: it details our reication technique at work by presenting several applications. Section 6 complements Section 4 by presenting a few technicalities postponed for the sake of readability. Section 7 discusses the correctness of the transformation and sketches a formal correctness proof. Section 8 illustrates how a rened denition of the non-reective interpreter produces a more expressive reective interpreter. Section 9 discusses related work. Finally, Section 10 concludes and discusses future work. Code occuring in the paper refers to a freely available prototype implementation, called M ETA J [metaj], which enables execution of the reective programming examples we present and provides a platform for experimentation with our technique.
Interpreter Interpreter Interpreter Program Interpreter Program Interpreter Interpreter Program Interpreter Interpreter Program
where is a reication operator. The application of the reication operator to an expression yields an accessible representation of the value denoted by the expression. In this example, the expression Pair denotes the corresponding Class object, say c, in the interpreters memory (see Figure 7). Pair returns an instance (say i), i.e. an object of type Instance, in the interpreters memory which represents c and which can be inspected and modied. The default superclass of Pair is replaced by Color by assigning the eld extendsLink. From now on, newly instantiated pairs contain color information. It is crucial to our approach that the reied representation i is based on the denition of c. This
Runtime System
ExpAssign.java Class.java Instance.java ...
program transformation
generation of
reflective interpreter
Reflective Interpreter
Reflective_Prog.java
Parser
Java.jjt Java2ExpVisitor.java ...
Runtime System
ExpAssign.java Class.java BaseClass.java Instance.java ...
Class.java
Figure 2: System architecture is achieved by the system architecture shown in Figure 2. Our non-reective JAVA interpreter (represented by the box at the top) takes a non-reective program Prog.java as input. This program is parsed into a syntax tree and evaluated. According to the required reective capabilities, the language designer2 transforms a subset of the classes of the non-reective interpreter. Basically, this transformation generates two classes for each original class. In our example, the le Class.java, which represents classes in the non-reective interpreter, becomes BaseClass.java and a different version of Class.java in the reective one. The reective interpreter relies on the non-reective interpreter in order to build levels of the reective tower. This is the core issue of our approach: the tower levels shown in Figure 1 are effectively built at runtime on the basis of the (verbatim) denition of the non-reective interpreter as in Smiths model. This is why the original denition of Class.java is an input of the reective interpreter in Figure 2. So, the behavior of the reective interpreter is derived from the non-reective one. Furthermore, our approach is selective and complete because the transformation is applicable to any class of the non-reective interpreter denition. M ETA J implements one version of this system architecture. Its parser has been implemented by means of JAVACC and JJT REE (versions 0.8pre2 and 0.3pre6, respectively). M ETA J itself is operational with the JDK versions 1.1.6 and 1.2.
class ExpId extends Exp { private String id; ExpId(String id) { this.id = id; } Data eval(Environment localE) { return localE.lookup(this.id); } }
Figure 4: Class ExpAssign types or loop constructs; all of these could be integrated and reied similarly.) JAVA programs are represented as abstract syntax trees the nodes of which denote JAVAs syntactic constructs and are implemented by corresponding classes. For example, variables, assignment statement, method call, and class instantiation expressions are respectively encoded by the classes ExpId, ExpAssign, ExpMethod and ExpNew. All of these classes dene an evaluation method Data eval(Environment localE) that takes the values of local variables in localE and returns the value of the expression (wrapped in a Data object). In particular, ExpId (see Figure 3) holds the name of a variable and its evaluation method yields the value currently associated to the variable in the local environment. An ExpAssign node (see Figure 4) stores the two subexpressions of an assignment. Its evaluation method evaluates the location of the right-hand side expression, followed by the value represented by the left-hand side expression and nally performs the assignment. ExpMethod (see Figure 5) represents a method call with a receiver expression (exp), a method name (methodId) and its argument expressions (args). Method call evaluation proceeds by evaluating the receiver, constructing an environment from the argument values, looking up the method denition and applying it. ExpNew (see Figure 6) encodes the class name (classId) and constructor argument expressions. Its evaluation fetches the class denition from the global environment, instantiates it and possibly calls the constructor. As suggested before, the interpreter denes a few other classes to provide a runtime system and implement an operational semantics. For example, the class Class (see Figure 7) represents classes by a reference to a superclass (extendsLink), a list of elds (dataList) and a list of methods (methodList). It provides methods for instantiating the class (instantiate()), accessing the list of methods including those in super classes (methodList()), etc. Methods are represented by 5
class ExpMethod extends Exp { private Exp exp; // receiver private String methodId; // method name private ExpList args; // arguments ExpMethod(Exp exp, String methodId, ExpList args) { this.exp = exp; this.methodId = methodId; this.args = args; } Data eval(Environment localE) { // evaluate the lhs (receiver) Instance i = (Instance) this.exp.eval(localE).read(); // evaluate the arguments to get a new local environment Environment argsE = Environment.Empty; this.args.eval(localE, argsE); // lookup and apply method Method m = i.lookupMethod(this.methodId); return m.apply(argsE, i); } }
class ExpNew extends Exp { private String classId; // class name private ExpList args; // constructor arguments ExpNew(String classId, ExpList args) { this.classId = classId; this.args = args; } Data eval(Environment localE) { // get the Class and create an Instance Class aClass = (Class) (Main.globalE.lookup(this.classId).read()); Instance i = aClass.instantiate(); // call non default constructor if it exists if (i.getInstanceLink().methodList().member(this.classId).booleanValue()) { Environment argsE = Environment.Empty; this.args.eval(localE,argsE); // lookup and apply method Method m = i.lookupMethod(this.classId); m.apply(argsE, i); } return new Data(i); } }
class Class { Class extendsLink; // superclass DataList dataList; // field list MethodList methodList; Class(Class eL, DataList dL, MethodList mL) { this.extendsLink = eL; this.dataList = dL; this.methodList = mL; } Class getExtendsLink() { return this.extendsLink; } // implementation of Javas new operator Instance instantiate() { ... } // compute complete method list (incl. superclasses) MethodList methodList() { ... } ... }
class Method { private StringList args; // parameter names private Exp body; // method body Method(StringList args, Exp body) { this.args = args; this.body = body; } Data apply(Environment argsE, Instance i) { // name each argument argsE.zipWith(this.args); argsE.add("this", new Data(i)); // eval the body definition of the method return this.body.eval(argsE); } }
the class Method (see Figure 8) by means of a list of argument names (args) and a body expression (body). Its method apply() binds argument names to values including this and evaluates the body. Other classes include Instance (contains a reference instanceLink to its class and a list of eld values; provides eld lookup and method lookup), MethodList, Data (implements mutable memory cells such as elds), DataList, Environment (maps identiers to values), etc. The architecture of the interpreter follows the standard design for object-oriented interpreters as presented in Gamma et al. [ghjv95] by the Interpreter design pattern. Instantiating this design pattern, the following correspondences hold: their Client is our interpreters main() method, their methods Interpret(Context) is our eval(Environment). The reication technique described in this paper is applicable to other interpreters having such an architecture. Note that these interpreters may implement many different runtime systems.
The dispatch technique is close to the bridge and state patterns introduced in Gamma et al. [ghjv95].
a) before Pair
b) after Pair
BaseClass
dataList = fst, snd methodList = Pair() extendLink = Object
Instance
different representations
Class
dispatch object representation = isReified = false
Class
dispatch object representation = isReified = true
Pair
Pair
Pair
instance of X
Figure 9: Before and after reication of the class Pair Obviously, the two paths provide different interfaces. Consider, for example, the problem of keeping track of the number of Pair instances using a static eld countInstances: this eld could be accessed either by Pair.countInstances or by ( ( Pair).staticDataList).lookup ("countInstances")4. In the last expression, the outermost reication operation is necessary in order to call lookup() on a data list object (cf. the fourth item below). In order to conclude this overview, we briey mention other important properties of our reication scheme:
Since reection provides objects representing internal structure for use in user-level programs, every reication operation returns an Instance (e.g. the one in Figure 9b). This implies that reication of reied entities requires that Instances are reiable.
Exp yields an accessible representation of the value denoted by Exp (i.e. an object in the interpreters memory, such as Class, Instance, Method) 5 .
References from dispatch objects to their active representations cannot be accessed by user programs. Only a call to the reication operator may modify these references. This ensures that the tower structure cannot be messed up by user programs.
The scope of the reication process is limited to individual objects in the interpreters memory. For example, the reication of a class does not reify its list of methods methodList nor its superclass. So, three categories of objects coexist at runtime: reied objects, non-reied (but
M ETA J does not allow static elds but could be extended easily to deal with such examples. is a strict operator. A syntax extension would be necessary to reify the expression (e.g. the AST representing 1+4) rather than the value denoted by the expression (e.g. the integer 5).
5
arg ) { body }
Type method (Type arg , ..., Type arg ) { body } ... Type method (Type arg , ..., Type arg ) { body } }
Figure 10: Original class denition reiable) ones and non-reiable ones. If a program accesses an object o through a reied one, the use of o is restricted exactly as in the non-reective case. Pair.extendsLink, for example, references a Class representing the superclass of Pair. Therefore, the only valid operations on this reference are new ( Pair.extendsLink)() 6 as well as accesses to static elds and members of this class. If the structure or behavior of the superclass is to be changed, it must be reied rst. This implies that accesses to non-reiable objects through reied ones are safe.
10
class BaseName { Type field ; ... Type field ; Name referent; BaseName(Type arg , ..., Type body this.referent = referent; }
Type method (Type arg , ..., Type body [ this.referent / this(~.) ] } ...
arg
) {
Type method (Type arg , ..., Type arg ) { body [ this.referent / this(~.) ] } }
Figure 11: Generated base class elds or methods in the base class it should denote the dispatch object 7 . In the transformation, this is implemented by substituting this(~.) (matching the keyword this followed by anything but a dot) by this.referent. The generated dispatch class, shown in Figure 12, has two elds: representation that points to either the base representation or the reied representation, and a boolean eld isReified that discriminates the active representation. Its constructor creates a base representation for the object. The methods method have the same signature as their original version. When the base representation is active (i.e. isReified is false), the method call is delegated to the base representation. When the reied representation is active (i.e. isReified is true), the method call is interpreted: the corresponding call expression is parsed (Parser.java2Exp()), a local environment is built (argsE.add()) from the method arguments and the eld representation of the dispatch object and the method call is evaluated (eval()). Note that for the sake of clarity, this code is intentionally naive. The actual implemented version could be optimized: for example, the call to the parser could be replaced by the corresponding syntax tree. The method reify() builds a reied representation of the base representation by evaluating a new-expression. The corresponding class is cloned in order to build a new tower level. So, every reied object has its own copy of a Class. This way, the behavior of each reied object can be specialized independently. If sharing is required the application programmer can achieve it by explicitly manipulating references. Finally, the reied representation is installed as the current representation and a reference to it is returned. A series of experiments led us to this sharing strategy. A previous version of the transformation did not clone the class. This sharing led to cycling dependency relationships and reective overlap after reication: in particular, reication of the class Class introduced
7
This is a typical problem of wrapper-based techniques that introduce two different identities for an object.
11
class Name { Object representation; boolean isReified; Name(Type arg , ..., Type arg ) { this.isReified = false; this.representation = new BaseName(arg , ..., arg , this); } Type method (Type arg , ..., Type arg ) { if (this.isReified) { Exp exp = Parser.java2Exp( "reifiedRep.method (arg ,...,arg )"); Environment argsE = Environment.Empty; argsE.add("reifiedRep", this.representation); argsE.add("arg ", arg ); ... argsE.add("arg ", arg ); Data result = exp.eval(argsE); return result.read(); } else return (BaseName) this.representation.method (arg , ... , arg ); } ... Instance reify() { if (!this.isReified) { Class aClass = Main.globalE.lookup("Name").clone(); Exp exp = Parser.java2Exp("new aClass(baseRep_field ,..., baseRep_field , Environment argsE = Environment.Empty; argsE.add("baseRep_field ", this.representation.field ); ... argsE.add("baseRep_field ", this.representation.field ); argsE.add("aClass", aClass); this.isReified = true; this.representation = exp.eval(argsE).read(); } return (Instance)this.representation; } }
12
non-termination. Alternatively, we experimented with one copy of each class per level but in this case the reication (without modication) of an object could already change its behavior. This generic reication technique is based on only two assumptions: 1. Each syntactic construct is represented by an appropriate expression during interpreter execution. We assume that all of these expressions can be evaluated using the method eval(argsE) where argsE contains the current environment, i.e. the values of the free variables in the current expression. 2. We assume that the textual denitions of all reiable classes have been parsed at interpreter creation time and that they are stored as Class objects in the global environment Main.globalE. These objects have to be cloneable. This way, reify() creates an extra interpreter layer based on the actual interpreter denition. Note that these simple assumptions and the formal denition enable the transformation to be performed automatically. In Java, the operator new returns an object (i.e. an Instance). Therefore, in order to let the user build other runtime entities than Instances, such as Classes and Methods, we provide a family of deication8 operators, one for each of these entities. These operators are the inverse of the generic reication operator. For example, in the reective program (where Class denotes the deication operator for classes): ( Pair).extendsLink = Class (new Class(...)); the right-hand side expression returns a Class dispatch object in front of the Instance created by new. Note that the deication operators while functionally inverting the reication operation do not change the representation of an object back to its unreied structure (e.g. to a BaseClass in the case of classes). The dispatch objects engender the structure of the reective tower; their implementation is not accessible to the user. In particular, the reication operator and the deication operators encapsulate the elds representation and isReified of dispatch objects as well as the eld referent from the base class. So, user programs cannot arbitrarily change the tower structure. However, the user or a type system to be developed should avoid the creation of meaningless structures, such as Class (new Method(...)).
We prefer the term deication [iyl95] to the equivalent terms reection [wf88] and absorption [meu98].
13
a) before pair
b) after pair
BaseInstance
dataList = dataList, instanceLink instanceLink = Instance
BaseInstance
dataList = fst, snd instanceLink = Pair
Instance
dispatch object representation = isReified = false
Instance
dispatch object representation = isReified = false
Instance
dispatch object representation = isReified = true
pair
pair
pair
instance of X
class Instance { public Class instanceLink; // ref. to Class public DataList dataList; // field list Instance(Class instanceLink, DataList dataList) { this.instanceLink = instanceLink; this.dataList = dataList; } // field access Data lookupData(String name) { return this.dataList.lookup(name); } ... }
14
class BaseInstance { Class instanceLink; DataList dataList; Instance referent; BaseInstance (Class instanceLink, DataList dataList, Instance referent) { this.instanceLink = instanceLink; this.dataList = dataList; this.referent = referent; } Data lookupData(String name) { return this.dataList.lookup(name); } ... }
Figure 15: Class BaseInstance Figure 13a). Once pair has been reied (see Figure 13b), it is represented by an Instance which points to a BaseInstance (say ). Note that in contrast to the reication of classes shown in Figure 9, the reied representation of an instance is reiable (because it is an instance itself; hence, the second dispatching Instance in Figure 13b). Since the reication is based on the actual denition of the original Instance, the dataList of contains the three elds instanceLink, dataList (itself containing fst and snd) and referent. The denition of the method lookupData() in the dispatch object calls the method lookupData()of as long as pair is not reied. Once it is reied, the denition of lookupData() of Instance is interpreted. In order to prove the feasibility of our approach, we applied this reication technique to different classes dening object-oriented features of our JAVA interpreter resulting in the prototype M ETA J. The imperative features of the non-reective interpreter can be tackled analogously. This way we could, for example, redene the sequentialization operator ; in order to count the number of execution steps in a given method (say m). One way to achieve this is by reication of occurrences of ExpS in reied m and dynamically changing their classes by a class performing proling within the eval() method. Another solution would be to replace ExpS nodes in reied m by nodes including proling.
5 Reective Programming
In this section, we express several classic examples of reective programming in our framework. These detailed examples of our reective interpreter at work should help the readers understanding of the systems working. The examples highlight an important feature of our design: since our reication scheme relies on the original interpreter denition, the meta-object protocol of the corresponding reective interpreter (i.e. the interface of a reective system) is quite easy to apprehend. It consists of a few classes which are reiable in M ETA J, the reication operator and the deication operators . In Figure 17 the class Pair is dened, and in main() a new instance pair is created. In the interpreter, the object pair is represented by an Instance (see Figure 13a). Our generic reication method provides access to a representation of this Instance which we name metaPair (denoted by pair in Figure 13b). The most basic use of reection in object-oriented languages consists in
15
class Instance { Object representation; boolean isReified; Instance(Class instanceLink, DataList dataList) { this.isReified = false; this.representation = new BaseInstance(instanceLink, dataList, this); } Data lookupData(String name) { if (this.isReified) { // interpret lookup method call Exp exp = Parser.java2Exp("reifiedRep.lookupData(name)"); // pass already evaluated values Environment argsE = Environment.Empty; argsE.add("name", name); argsE.add("reifiedRep", this.representation); Data result = exp.eval(argsE); // unpack result return (Data)result.read(); } else return ((BaseInstance)this.representation).lookupData(name); } ... Data reify() { if (!this.isReified) { // copy the base class BaseInstance Class aClass = Main.globalE.lookup("Instance").clone(); // create and initialize new representation Exp exp = Parser.java2Exp("new aClass(baseRep_instanceLink, baseRep_dataList)"); Environment argsE = Environment.Empty; argsE.add("baseRep_instanceLink", this.representation.instanceLink); argsE.add("baseRep_dataList", this.representation.dataList); argsE.add("aClass", aClass); this.isReified = true; this.representation = exp.eval(argsE).read(); } return new Data(this.representation); } }
16
class Pair { String fst; String snd; Pair(String fst, String snd) { this.fst = fst; this.snd = snd; } } class PrintablePair extends Pair { String toString() { return "(" + this.fst + "," + this.snd + ")"; } } class InstanceWithTrace extends Instance { Method lookupMethod(String name) { // trace method-called System.out.println("method called: " + name); return this.instanceLink.methodList().lookup(name); } } class Main { void main() { Pair pair = new Pair("1", "2"); // 1 - invariance under reification Instance metaPair = pair; System.out.println("pair.fst: " + pair.fst); // 2 - introspection: test existence of a super class Class metaClass = Pair; if (metaClass.getExtendsLink() == null) System.out.println("Class Pair has no superclass"); // 3 - intercession: dynamic class change metaPair.instanceLink = PrintablePair; System.out.println("pair.toString(): " + pair.toString()); // 4 - intercession: change method-call semantics Instance metaMetaPair = metaPair; metaMetaPair.setInstanceLink(InstanceWithTrace); System.out.println("pair.toString(): " + pair.toString()); // 5 - instance and class deification System.out.println(( InstancemetaPair).fst); reify(PrintablePair).extendsLink = Class metaClass; System.out.println("pair.toString(): " + pair.toString()); } }
17
reifying an object: changing the internal representation without modifying its behavior (see Example 1). Another simple use is introspection. Let us consider the problem of testing the existence of a super class of a given class. In Example 2, the class Pair (represented by a Class in the interpreter) is reied which enables its method getExtendsLink() to be called. In M ETA J, reective programming is not limited to introspection, but the internal state of the interpreter can also be modied (aka intercession). The third example in main() shows how the behavior of an instance can be modied by changing its class dynamically. Imagine that we would like to print pairs using a method called toString(). We dene a class PrintablePair which extends the original class Pair and implements a method toString(). A pair can then be made printable by dynamically changing its class from Pair to PrintablePair (remember that the eld instanceLink of Instance holds the class of the represented instance, see Figure 15). Afterwards the object pair understands the method toString(). The fourth example deals with method call tracing for debugging purposes. The class Instance of the interpreter denes the method Method lookupMethod(String name) that returns the effective method to be called within the inheritance hierarchy. In our interpreter each lookupMethod() is followed by an apply(). Thus, method call tracing can be introduced by dening a class InstanceWithTrace which specializes the class Instance of the interpreter such that its method lookupMethod() prints the name of its parameter. In order to install the tracing of method calls of the instance pair, its standard behavior dened in the interpreter by the class Instance (note that this class can be accessed because the interpreter denition is an integral part of the reective system built on top of the reective interpreter) is replaced by InstanceWithTrace. Reication of pair provides access to an Instance whose eld instanceLink denotes the class Pair. A sequence of two reication operations on pair provides access to an Instance whose instanceLink denotes the class Instance. This link can then be set to the class InstanceWithTrace. A method call of the object pair then prints the name of the method. Therefore, "toString" is printed by our third example. Finally, note that our tower-based reection scheme makes it easy to trace the tracing code if required because any number of levels may be created by a sequence of calls to . The fth (rather articial) example illustrates deication by deifying metaPair and metaClass in order to create an instance and a class at the base level. After deication of the reied representation metaPair we show that base-level operations can be performed on the resulting object. In the case of class deication, we restore the original class of pair. More advanced examples that illustrate our approach rely on the capacity to reify arbitrary parts of the underlying interpreter. As discussed in Section 4.3, the reication of ExpS allows the behavior of the sequence operator ; to be changed. This way, we could, for instance, stop program execution at every statement for debugging purposes or handle numeric overow exceptions by re-executing the current statement block with higher-precision data representations. Furthermore, reication of the control stack would allow Javas try/catch-mecanism for exception handling to be extended by a retry variant.
18
class ExpId extends Exp { // same fields and constructor as in Data eval_original(Environment localE) { // same definition as eval in } Data eval(Environment localE) { if (!localE.member("#meta_level").booleanValue()) return this.eval_original(localE); else { if (this.id.equals("this")) return new Data(((Instance) localE.lookup("this").read()).referent); else return eval_original(localE); } }
19
First, in the reective interpreter a reied object is represented by a dispatch object and a reied representation. So, basically a reied object has two different identities. With our technique, this is bound to the representation rather than the dispatch object by parsing the expression "reifiedRep. method (arg ,\dots,arg )" in the dispatch object (see Figure 12). However, if a statement return this is to be interpreted, this should denote the dispatch object. Otherwise, userlevel programs could expose the reied representations. The interpreter class ExpId is in charge of identier evaluation (including this) and has therefore to be modied to account for this behavior. In Figure 18 the method eval() distinguishes two cases by means of the environment-tag #meta_level.9 First, interpretation has been initiated by the interpreters entry point and nonreective evaluation is necessary. Second, interpretation has been initiated by a dispatch object and reective interpretation is required. In the rst case eval_original() is called: this method has the same denition as eval() in the non-reective interpreter. In the second case if the identier is this, the dispatch object of the current representation is returned. Remember that the eld referent points back from the base representation to the dispatch object, the same mechanism is used to link the reied representation to the dispatch object. This eld must be set by the methods reify(), so the class Instance has to provide such a eld 10 . Second, remember that the scope of reication is limited to a single object in the interpreter memory. This means interpretation involves reied and non-reied objects. For example, the reication of an Instance does not reify neither its eld list dataList nor its class denoted by instanceLink. In particular, once an Instance has been reied, the interpretation of its method lookupData (repeated from Figure 14): Data lookupData(String name){return this.dataList.lookup(name);} requires this.dataList to be interpreted and the call lookup(name) to be delegated because this.dataList denotes a non-reiable object. In abstract terms, a dispatch object introduces an interpretation layer (a call to eval()) and this layer has to be eliminated when the scope of the current (reied) object is left. This scheme is implemented in ExpMethod.eval() (see Figure 19). Because of these two problems, the methods ExpData.eval() and ExpNew.eval() have to be modied similarly. This means that our reication scheme cannot be applied to the four classes ExpId, ExpMethod, ExpData, ExpNew 11. However, our method provides much expressive power: these restrictions x the relationship between certain syntactic constructs and the runtime system, but the runtime mechanisms themselves can still be modied as exemplied in Section 5. In order to weaken this restriction, we designed and implemented a variant 12 of our reication scheme that does not require ExpId and ExpData to be modied. Unfortunately, this advantage comes at a price: the eld referent can be exposed and modied by reication in this case.
The dispatch objects insert this tag into the local environment. For the sake of simplicity, the code shown in Figures 12 and 16 does not mention the eld referent. 11 The restriction that all parts of a reective system cannot be reied seem to be inherent to reection [wf88]. 12 This variant is also bundled in the M ETA J distribution.
20
Semantics of reective programming systems is a complex research domain. Almost all of the existing body of research work in this domain is about reection in functional programming languages [wf88][dm88][mul92][mf93]. Even in this context, foundational problems still exist. For example, it seems impossible to give a clean semantics which avoids introducing non-reiable components [wf88] and logics of programming languages must be considerably weakened in order to obtain a consistent theory of reication [mul92]. One of the very few formal studies of reection in a non-functional setting has been done by Malenfant et al. [mdc96]. This work deals with reection in prototypebased languages and focuses on the lookup() ; apply() MOP formalized by means of rewriting systems. This approach is thus too restricted to serve as a basis for our correctness concerns. In general, semantic accounts of imperative languages are more difcult to dene than in the functional case. In particular, the transposition of the results obtained in the functional case to our approach requires further work. We anticipate that this should be simpler in a transformational setting such as ours than for arbitrary reective imperative systems. In order to prove the correctness of our scheme, the basic property to satisfy would be equivalence between a non-reective interpreter and a reective interpreter generated by applying our transformation to , i.e.
"!#"$%&(' )02143 )021
Since the transformation 5 is operating on individual classes, this property can be tackled by establishing an equivalence between an arbitrary class (say c) of the non-reective interpreter and its transformed counterpart. Essentially, the transformation introduces an extra interpretation layer into the evaluation of the methods of c. Programs and their interpretations introduced by transformation satisfy the property $7 $78 39 $7@ $A8 ", 6 )).read() java2Exp(" ").eval(Environment.Empty.add("6 6 6 This property can be proven by induction on the structure of the AST representation of . (Note that the formulation of this property is intentionally simplistic and should be parameterized with contextual information, such as a global environment and a store.) It can be applied to the dispatch classes (see Figure 12) to fold interpreting code into delegating code. When the then-branches of dispatching methods are rewritten using the property from left to right, the then-branches equal the corresponding else-branches. Henceforth, the conditionals become useless and the dispatch objects become simple indirections that can be suppressed. In the case of the method reify(), the rewriting leads to the expression new Name(...) that creates a copy of the non-reied representation. Finally, we strongly believe our transformation is type-safe (although we did not formally prove this): every well-typed interpreter is transformed into a well-typed reective interpreter. Obviously, wrongly-typed user programs may crash the non-reective interpreter. In the same way, some reective programs may crash the reective interpreter, for instance by confusing reective levels or trying to access a eld which has been previously suppressed using intercession. Specialized type systems and static analysis methods for safe reective programming should be developed.
class Instance { ... // add two new methods Data send(Msg msg) { return msg.to.receive(msg); } Data receive(Msg msg) { return msg.to.lookupMethod(msg.methodId) .apply(msg.argsE, msg.to); } } class ExpMethod extends Exp { ... Data eval(Environment localE) { // as before evaluate receiver and arguments: o, argsE ... // new code: determine sender, build and send message Instance self = (Instance)(localE.lookup("this").read()); Msg msg = new Msg(self, o, this.methodId, argsE); return self.send(msg); } }
class InstanceWithSenderTrace extends Instance { Data send(Msg msg) { System.out.println("method called " + msg.methodId + " by " + msg.from); return super.send(msg); } }
22
In the original interpreter, ExpMethod.eval() evaluates a method call by implementing the composition lookupMethod();apply(). So, the behavior of the receiver of a method call can be modied easily by changing the denition of lookupMethod() (as illustrated by trace insertion in the Section 5). However, a modication concerning the sender of the method call (see C ODA [aff95] for a motivation of making the sender explicit in the context of distributed programming) is much more difcult to implement. Such a change would require the modication of all instances of ExpMethod in the abstract syntax tree, i.e. all occurrences of the operator .. Indeed, we have to check whether the object this in such contexts has a non-standard behavior. A solution to this problem is to modify the non-reective interpreter, such that its reective version provides a MOP enabling explicit access to the sender in a method call. Intuitively, we split message sending in two parts: the sender side and the receiver side. First, we introduce a new class Msg which is a four-tuple. For each method call, it contains the sender from, the receiver to, the method name methodId and the corresponding argument values argsE. Then, two methods dealing with messages are added to the denition of Instance in the original interpreter: send() and receive() (see Figure 20). Finally, ExpMethod.eval() is redened such that it creates and sends a message to the receiver. This new version of the non-reective interpreter is made reective by applying our program transformation. Then, the user can, for example, introduce tracing for message senders (see Figure 21), the same way traces have been introduced in the previous section. This example highlights three advantages of our approach: MOPs are precisely dened, application programmers are provided with the minimal MOPs tailored to their needs and language designers can extend MOPs at compile time without anticipation of these changes.
9 Related work
A comparison between reective systems is inherently difcult because of the wide variety and the conceptual complexity of reective models and implementations. For example, the detailed denition of the CLOS MOP requires a book [kic91] and a thorough comparison between CLOS and S MALLTALK already lls a book chapter [coi93]. Consequently, we restrict our comparison to the three basic properties our reection model obeys (the rst and second characterizing Smith-like approaches, the third being fundamental to our goal of the construction of specially-tailored MOPs): 1. (tower) There is a potentially innite tower of reective interpreters. 2. (interpreter) The interpreter at level
3. (selectivity & completeness) Any part of the runtime system and almost all of the syntax tree (see Section 6) of an interpreter at level can be reied and has an accessible representation at level .
First, most reective systems are based on some notion of reective towers and provide a potentially innite number of levels. A notable exception to this are O PEN -C++ [chi95] and I GUANA [gc96] whose MOPs only provide one metalevel. Second, our approach is semantics-based following Smiths seminal work on reective 3-L ISP [smi84] for functional languages. This is also the case for the prototype-based languages 3-KRS [mae87] and AGORA [meu98]. The other object-oriented approaches to reection (including O BJ VL ISP [coi87], S MALLTALK [bri89] [riv96], C LASSTALK [bri89], CLOS [kic91], MetaXa [gol97]) 23
are not semantics-based (in the sense of the second property cited above) because they do not feed higher-level interpreters with the code of lower-level interpreters. Instead, different levels are represented by appropriate pointer structures. This proceeding allows more efcient implementations but has no semantic foundation. Moreover, these reective languages are monolithic entities while our modular approach consists of three simple parts: a non-reective interpreter, the operator and the operators . Third, our approach enables language designers to precisely select which mechanisms of the language are reective. With the exception of I GUANA and O PEN -C++, all the reective systems cited above do not have this characteristic. Finally, note that our approach shares a general notion of completeness with 3-L ISP, 3-KRS and AGORA: the programming model is dened by the interpreter and almost all of its features can be made reiable (up and down are primitives in 3-L ISP and cannot be reied, for instance). Asai et al. [amy96] also starts from such a complete model but this interesting approach to reection in functional languages restricts reiable entities in order to allow optimization by partial evaluation. In contrast, the remaining reective systems described above do not base reection on features of an underlying interpreter but implement an ad hoc MOP. The notion of completeness therefore does not make sense for them.
used to generate specially-tailored reective semantics from a non-reective one. Finally, we rmly believe that our reication technique can also be applied to (parts of) applications instead of an interpreter in order to make them reective (preliminary results can be found in a related paper by the authors [ds00]). Acknowledgements. We thank the anonymous referees for their numerous constructive comments and the editor Olivier Danvy. The work reported here has also beneted from remarks by Kris de Volder, Shigeru Chiba and Jan Vitek. It has been improved through many discussions with our colleagues Noury Bouraqadi, Mathias Braux and Thomas Ledoux.
References
[aff95] J. McAffer. Meta-Level Programming with C ODA. Proceedings of ECOOP, LNCS 952, Springer Verlag, pp. 190-214, 1995.
[amy96] K. Asai, S. Matsuoka, A. Yonezawa. Duplication and Partial Evaluation For a Better Understanding of Reective Languages. Lisp and Symbolic Computation, 9(2/3), pp. 203241, 1996. [bc00] [bn00] G. Blair, R. Campbell (chairs). Workshop on Reective Middleware, 2000. http://www.comp.lancs.ac.uk/computing/RM2000/ M. Braux, J. Noy. Towards Partial Evaluating Reection in JAVA. Proceedings of Workshop on Partial Evaluation and Semantics-Based Program Manipulation, ACM Press, pp. 2-11, 2000. J.P. Briot, P. Cointe. Programming with Explicit Metaclasses in S MALLTALK. Proceedings of OOPSLA, ACM SIGPLAN Notices, 24(10), pp. 419-431, 1989. S. Chiba. A Metaobject Protocol for C++. Proceedings of OOPSLA, ACM SIGPLAN Notices, 30(10), pp. 285-299, 1995. P. Cointe. Metaclasses are First Class Objects: the O BJ VL ISP Model. Proceedings of OOPSLA, ACM SIGPLAN Notices, 22(12), pp. 156-162, 1987. P. Cointe. CLOS and S MALLTALK: a comparison. In Object-Oriented Programming: The CLOS perspectives?, A. Ppcke (ed.), MIT Press, ch. 9, pp. 215-274, 1993. P. Cointe (ed.). Proceedings of Reection99, LNCS 1616, Springer Verlag, 1999.
[dm88] O. Danvy, K. Malmkjr. Intensions and Extensions in a Reective Tower. Proceedings of the ACM Conference on Lisp and Functional Programming, pp. 327341, 1988. [ds00] R. Douence, M. Sdholt. On the lightweight and selective introduction of reective capabilities in applications. International Workshop on Reection and Metalevel Architectures at ECOOP, 2000. ftp://ftp.disi.unige.it/person/CazzolaW/EWRMA/sudholt.ps.gz
25
[gol83] A. Goldberg, D. Robson. S MALLTALK 80, the Language and its Implementation. AddisonWesley, 1983. [gol97] M. Golm. Design and Implementation of a Meta Architecture for Java. Masters Thesis. Universitt Erlangen, 1997. [gc96] [iyl95] B. Gowing, V. Cahill. Meta-Object Protocols for C++: The I GUANA Approach. Informal Proceedings of Reection96, pp. 137-152, 1996. J.-I. Itoh, Y. Yokote, R. Lea. Using Meta-Objects to Support Optimisation in the Apertos Operating System. Proceedings of the USENIX Conference on Object-Oriented Technologies (COOTS), USENIX Association, pp. 147-158, 1995. JAVA home page. Sun Microsystems, Inc. http://java.sun.com G. Kiczales, J. des Rivires, D. Bobrow. The Art of the Metaobject Protocol. MIT Press, 1991.
[java] [kic91]
[mae87] P. Maes. Concepts and Experiments in Computational Reection. Proceedings of OOPSLA, ACM SIGPLAN Notices, 22(12), pp 147-155, 1987. [mdc96] J. Malenfant, C. Dony, P. Cointe. A Semantics of Introspection in a Reective PrototypeBased Language. Lisp and Symbolic Computation, 9(2/3), pp. 153-180, 1996. [mf93] A. Mendhekar, D. P. Friedman. Towards a Theory of Reective Programming Languages. Informal Proceedings of the Third Workshop on Reection and Metalevel Architectures in Object-Oriented Programming at OOPSLA, 1993. M ETA J home page: http://www.emn.fr/sudholt/research/metaj.
[metaj]
[meu98] W. De Meuter. AGORA: The Story of the Simplest MOP in the World. In Prototype-based Programming, J. Noble et al. (ed.), Springer Verlag, 1998. [mul92] R. Muller. M-LISP: A Representation-Independant Dialect of LISP with Reduction Semantics. ACM TOPLAS, 14(4), pp. 589-616, 1992. [riv96] F. Rivard. S MALLTALK: a Reective Language. Informal Proceedings of Reection96, pp. 21-38, 1996.
[smi84] B.C. Smith. Reection and Semantics in L ISP. Proceedings of POPL, ACM Press, pp. 23-35, 1984. [wf88] M. Wand, D. P. Friedman. The Mystery of the Tower Revealed: A Non-Reective Description of the Reective Tower. Lisp and Symbolic Computation, 1(1), pp. 11-38, 1988.
26