RESTful Platform Service API Design Principles and Service Contract

Yonghong Chen, Liming Zhu, Xiwei Xu

NICTA, Australia Technology Park, Eveleigh, Australia School of Computer Science and Engineering, University of New South Wales, Australia {Yonghong.Chen, Liming.Zhu, Xiwei.Xu}@nicta.com.au description languages such as WSDL. By mutually agreeing on a set of rigidly defined service interfaces, the internal implementation on both client and server sides can evolve independently. However, the interface itself is brittle to change. In contrast, REST architecture style allows service contract to be negotiated and delivered at runtime aiming to achieve higher level of evolvability. Current approaches of designing RESTful platform API [5, 6] are usually divided into several steps: (1) resource abstraction and identification; (2) resource representation and media types design; (3) HATEOAS design to connect resources and convey application state. These steps provide flexible ways of expressing a business concept syntactically in its API. However these steps do not answer a more fundamental question of what business concepts and to what level of details is required to be understood by the clients at semantic level of the current API in order to complete the business operation and consume the service. For example, in Twitter, a tweet is a short text message identified by a numeric number as unique id. However, initially, developers were not clear how many bits it is defined as and assumed it was 32 bits. Later, twitter changed the way id was generated in 2010 and settled on 64 bits due to its underlying database system and performance trade-offs [7]. But it caused problems with Javascript clients which failed to parse numbers more than 53 bits. More important questions than how to solve this particular id problem at the syntactic level are: is it necessary for clients to understand this semantic concept of tweet id to consume the services provided by Twitter platform? Or clients can treat the semantic concept of a Tweet id as a black box? What are the tradeoffs in terms of platform API between these two approaches and what is the impact on overall evolvability? To overcome this shortcoming, we use domain and task ontologies to identify and document semantic concepts as semantic service contract. Unlike many approaches, as summarized in [8], which use web service oriented ontologies such as OWL-S and WSMO, and aim to align the meaning of semantic concepts across software agents to achieve reusability for service discovery and composition purposes, our unique use of lightweight ontology is to accurately define the level of understanding required from service consumers point of view to maximize overall evolvability. In addition, we deliberately separate the concerns of evolvability into two loosely coupled layers. One layer focuses on evolvability from an

Abstract Evolvability is the ability for a system to change to meet business requirement with minimal effort. It is particularly important and challenging for a system such as a platform with public APIs servicing a large number of external applications and encouraging independent third party enhancements from its digital ecosystem community. In this paper, we present a novel approach of designing platform service APIs systematically in order to achieve maximum evolvability. The API design process combines lightweight ontology, REST principles and selected Component-Based Software Engineering (CBSE) principles in a unique way. This approach allows facets of evolvability to be separated into two independent layers of concerns (internal and external viewpoints respectively). We applied our approach to the Twitter API evolution. The result shows significant improvements in the areas of adding new features (e.g. retweet), contract leaking prevention (e.g. tweet id) and usability of the APIs. Keywords-Component; REST; Platform service API; Evolvability; Service contract

I. INTRODUCTION The emergence of web services gives rise to service ecosystems [1], where service platforms are exposed to external parties and their community of users to provide solutions for the market niches in order to respond to increasing competition and economy pressure [2]. This requires platform APIs to evolve rapidly in order to work with third party service components and support new business requirements especially in a very dynamic, highly diversified and uncontrolled market environment. For example, Twitter is a popular service platform claimed to have 660k developers and 900k applications [3]. Changes to the API could have wide impact and long term ripple effect to many external applications and web services. Despite the effort of preventative measurements and planning, changes are often forced to rollback due to the extreme difficulty of managing such ultra large size digital community outside its enterprise boundary. There are two different classes of web services used in platform APIs identified by W3C [4]: (1) RESTcompliance web services, where the primary of the services is to manipulate web resources via a uniformed set of stateless operations; (2) arbitrary web services, which may expose any arbitrary set of operations. In the arbitrary web services, the interoperability of machine-tomachine interaction is typically based on interface

internal viewpoint and aims to improve modularity, internal reusability and extensibility. The second layer focuses on evolvability from an external viewpoint and aims to improve the usability and interoperability to allow platform services to be easily consumed and enhanced by external developers while reducing the number of brittle changes. II. ARCHITECTURE DESIGN PROCESS This design process is divided into three steps as illustrated in Fig. 1. Firstly, we use domain and task ontologies to take long-term business requirements and evolution models into consideration. Our approach has no constraint on the selection of ontology languages except it should be lightweight and easy to use for external developers. Thus general purposed format such as RDF is used here for simplicity. Secondly, ontology models are also applied to guide the design of a component layer to improve evolvability from an internal viewpoint and focus on modularity, internal reusability and extensibility. Thirdly, REST service layer is built upon the component layer by mapping components and their associated interfaces into resources and enforcing REST uniformed interface constraints. The mapping also determines the design time service contract at the syntactic level. Therefore, the overall design time service contract is separated into two parts in semantic and syntactic levels respectively. To illustrate our approach, we use the REST API of Twitter as an example throughout this paper and demonstrate how our approach can be applied to systematically improve existing Twitter API. A. Semantic Contract Design Applying ontology to software development and component model design is quite common [9]. Our approach uses existing ontology approaches except the focus shifts towards defining the level of understanding of domain concepts intended for external clients. Domain concepts that are not exposed must be either removed or labeled with a scoping attribute to indicate it is only for internal development. We use both domain and task ontologies in this approach as semantic service contract. a) Domain ontology The domain concepts exposed must only include stable domain concepts and attributes that are unlikely to change and necessary to complete business operations. Repeatable structural relationship patterns among domain concepts are generalized to improve reusability. Detailed domain concepts and attributes will be communicated to client at runtime using component reflection and REST message selfdescriptiveness.

In our Twitter platform example, the relationships between timelines and statuses are generalized into collections sorted by timestamp. A more generalized collection relationship without the sorting constraint can be widely applied to many domain objects such as favorites, lists, friends, followers, users and etc. Detailed attributes such as user profile background color should be acquired through the reflection interface of the user component and pre-rendered as bgcolor attribute in xhtml media type to conform to REST self-descriptiveness. b) Task ontology For simplicity we roughly follows [9] to divide task ontology into 1) Control ontology which is the standard vocabulary of control such as sequential, iteration and etc. 2) Goal ontology which essentially abstracts tasks into goals and sub-goals and defines the structural relationships among them. Goals have associated roles that map to domain ontology concepts. In addition, goals are generalized into a very small set of operations in order to facilitate REST uniformed interface design in the later stage. Generalizing task ontology usually involves the trade-off between usability and reusability as the more generalized goals the better reusability but lesser usability. This approach favors reusability and alleviates the tradeoff because we can apply the reflection design pattern to the component model to discover the context at runtime and use REST message self-descriptiveness to communicate detailed operational context to users. In the Twitter platform, users can publish short text messages called tweets. Tweets are then organized in various ways and presented as resources, such as timelines, favorites and lists, to facilitate easy discovery by its potential interested audiences. Applying our ontology approach, we generalize them into a small set of domain concepts such as timeline, collection and item, where collection is a container for items and timeline is a collection sorted by creation time. We deliberately omit tweet id from the domain ontology so that tweet is treated as a black box domain concept. Retweet is a popular service which allows a user to relay a tweet he likes to his followers. Then we represent the task of Retweet original message as a goal retweet with two associated roles current user and tweet. The goal is generalized as add item to collection with two roles tweet and retweet collection to maximize reusability. Both roles can now be generalized as item and collection respectively. Similarly, we avoided referencing to item id when defining roles in our task ontology. Retweet is now a simple task requires no further breaking down of sub-goals unlike the original Twitter API. B. Component Model Design The component model design in our approach is much stricter comparing to general distributed component models. It must satisfy the following extra requirements: a) Use the reflection design pattern to allow enquiring of component type, instance id, supported interfaces, public attributes, sub-components, method metadata, binding information and etc. Service layer is able to discover component capability and execution context from

Figure 1 architecture design process

a small set of standard reflection interfaces, then translate meta-information into dynamic service contract at runtime. b) Standardize interfaces and common relationship/interaction patterns. In the Twitter platform example, we identified collection and sorted collection, such as via timestamp, as repeated structural and interaction patterns in both domain and task ontologies. Standard interfaces are to be designed in the component model to support these patterns. c) Guarantee full connectedness of components. HATEOAS constraint of REST architecture style requires all services to be discoverable from initial URL. It is one of the design challenges in existing REST approaches to make sure all resources are correctly inter-connected. Resource connectedness testing frameworks were proposed [7] to mitigate the risk of leaving behind inaccessible resources. In our approach, a root component is required to serve as the starting point to discover the references to other components in the system. The full connectedness conformance can be verified by analyzing design models such as UML components and class diagrams without the need for a testing framework at runtime. Full connectedness of components will ensure full connectedness of REST resources in the service layer. To satisfy these requirements in our Twitter example, we define IComponent interface to support querying metadata, ICollection and ISortedCollection interfaces to support collection-item relationship and a series of components to satisfy connectedness such as: Root>Timelines->PublicTimeline->Status->(Tweet + Author). C. Service Layer Design This step focuses on deriving design time service contract at syntactic level and supporting REST design constraints in details. Design decisions in this step are relatively localized and cheap to change. Most decisions involved in this step are only external view focused. a) Identify components to be exposed externally Components correspondent to domain concepts defined in semantic contract are exposed externally by mapping into a group of resources. Each resource group is assigned with a base URL. Within the component, metadata, value objects and interfaces are mapped as sub-resources under the resource group. In addition, exposed component should have no exposed interface methods that contain parameters outside of defined domain ontology, such as the tweet id in our example. b) Normalize interfaces Component interfaces often contain large numbers of arbitrary methods and need to be normalized into a small set of operations such as CRUD that center on value objects in order to be easily mapped to a small common set of uniformed operations. During the process of normalization, additional value objects could be identified and mapped as sub resources under the same resource group. For example, IComponent interface can be normalized to center on a newly defined meta-object containing a list of public attributes and interface descriptions.

c) Select/Design media types As common relationship/interaction patterns were already identified in the component model, media types with nature support of designed patterns can be selected to further improve interoperability. For example, we identified collectionitem relationship in our ontologies and realized it with ICollection interface. Atom media types defined in the AtomPub protocol is naturally a good candidate to express such relationship. d) Mapping normalized interfaces to resources Since interfaces are normalized, the mapping between value objects to resources is relatively straightforward. e) Mapping non-normalized interfaces to resources Some interfaces are difficult to be normalized such as searching queries. Parameters are required to be supplied in the form of URL templates or in the payload of POST method to invoke the methods. If the parameters are expected to be supplied by a machine, the ontology is required to be revisited to make sure all parameters required are pre-defined in the domain ontology to prevent any unintentional contract leak. However, if parameters are only expected to be manually entered by human, it is not required for them to be included in domain ontology. In our Twitter example, normalizing ICollection to center on collection and item resources is straightforward. We also show two implementations of retweet as examples of mapping normalized and un-normalized interfaces respectively in Fig. 2. For the normalized interface, we use app:collection tag from the AtomPub protocol in retweet resource representation to improve interoperability and advertise tweets can be posted. For unnormalized interface, tweet id is pre-encoded into URL as part of dynamic contract and communicated to client directly. In both cases, tweet id remains transparent to client. In the original Twitter API, the retweet operation is defined as POST statuses/id and retweet collection resource is defined as statues/retweet_by_me. These resource definitions are ad-hoc and there is no discovering mechanism apart from reading developer documentation. The mapping rules of how interface and composition patterns are translated into platform API serves as the design time service contract at syntactic level are naturally reused across multiple components and resources. The runtime platform API from clients perspective is the result of applying the static mapping rules to the current runtime configuration of components in the system.

Figure 2. Sample APIs

Component model design in the previous step plays a critical role of reducing design complexity of service layer design and connecting designs of semantic contract and syntactic contract. Firstly, semantic contract is codified in the component layer which in turn is mapped into service layer via syntactic contract. Thus, changes to semantic contract are consistently applied across the entire system via the conformity of component models and interfaces. Secondly, the run time platform API is generated automatically via reconfiguring, extending and composing components on the fly and its consistency is also enforced via the conformity of component model. Thirdly, the loose coupling between component and service layer design allows great flexibility and enables early design decisions of component models to be changed with minimal impact of the system. These benefits are clearly illustrated in our Twitter example as: a) tweet id is hidden away by removing references to item id from a rather small set of standard component interfaces; b) this design decision was consistently applied across all operations such as deleting or updating tweets; c) this design decision was consistently applied across the entire system regardless the current component configuration, for example, it does not matter if tweet is a sub component of a timeline or a favorite list. III. RELATED WORK To the best of our knowledge, currently there is no research specifically uses ontology to define the boundary of semantic understanding between Platform APIs and clients or use explicit underlying component models to support REST service API design. There are approaches use component model to support both REST and SOAP such as WebComposition [10]. However, it employs a dedicated service module to act as a gateway to map the interactions between external web services and existing components.. This simplifies the design but the service module itself becomes a central point of tight coupling because it needs to be aware of all other modules and inter-relationships among them in order to inject links to connect resources across components. This would severely hinder its ability to scale or evolve especially in the uncontrolled open ecosystem. In addition, it does not enforce the conformity of uniformed interface constraints. On the other hand, CBSE is often applied to traditional non-REST service API design. Interestingly, well designed SOA service APIs sometimes naturally exhibits REST-like uniformed interface, such as the case study presented in [11]. The same observation is also noted in [12]. Our approach exploits this natural bond between CBSE and REST and further improves evolvability by systematically mapping components to REST APIs and enforcing REST uniformed interface constraints. IV. DISCUSSION & CONCLUSION Twitter REST API is a typical example of commercial REST-style APIs. It exhibits some levels of conformance to some REST principles such as resource abstraction and uniformed operation. However, HATEOAS and message

self-descriptiveness are the most commonly ignored constraints. Its service discovery heavily relies on out-ofband documentation, thus, creating brittleness of the API. In this paper, we use Twitter API as an example and improved the evolvability of original API in many folds: a) contract leak, such as illustrated with tweet id, is prevented by defining semantic contract to clearly communicate the boundary of business information required to be understood by client; b) improved usability of resource definition. Resources are mapped from components and preserved original semantic relationships among metadata domain object and interfaces. This prevents apparent adhoc definition of resources such as GET statuses/ retweeted_by_me in the original API; c) enforced uniformed interfaces. Resources are well connected and message self-descriptiveness is well supported. New features can be easily introduced and discovered by the client without completely relying on out-of-band documentation. The retweet feature in our example advertises itself without the need for extra documentation. However, Twitter is a data centric platform which does not involve dividing complex goals into sub goals in the task ontology design step. We plan to expand our work to apply this approach to an operation centric platform to better evaluate its validity. REFERENCES
[1] A. Barros, M. Dumas, and P. Bruza, "The Move to Web Service Ecosystems," BPTrends, vol. 3, pp. 1-9, 2005. [2] J. Bean, SOA and Web services interface design: Morgan Kaufmann, Elsevier. 30 Corporate Drive, Suite 400, Burlington, MA 01803, USA, 2010. [3] Twitter REST API. Available: https://dev.twitter.com/ docs/api [4] D. Booth, H. Haas, F. McCabe, E. Newcomer, M. Champion, C. Ferris, and D. Orchard. (2004). Web Services Architecture. Available: http://www.w3.org/TR/2004/NOTE-ws-arch-20040211/ [5] L. Richardson and S. Ruby, RESTful Web Services: O'Reilly Media, 2007. [6] J. Webber, S. Paras, I. Robinson, and M. Fowler, REST in Practice, First Edit ed. Newgen, North America: OReilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472., 2010. [7] Announcing Snowflake. Available: http://engineering.twitter.com/ 2010/06/announcing-snowflake.html [8] M. Lanthaler, M. Granitzer, and C. Gtl, "Semantic Web Services: State of the Art," in Proceedings of the IADIS International Conference on Internet Technologies Society ITS 2010, 2010, pp. 107-114. [9] C. Josephson and J. Josephson, "Ontology of Tasks and Methods," in Proceedings of the 11th Knowledge Acquisition Modeling and Management Workshop, 1998, pp. 1-25. [10] H. W. Gellersen, R. Wicke, and M. Gaedke, "WebComposition: an object-oriented support system for the Web engineering lifecycle," Computer Networks and ISDN Systems, vol. 29, pp. 1429-1437, 1997. [11] A. Brown, S. Johnston, and K. Kelly, "Using service-oriented architecture and component-based development to build web service applications," Rational Software Corporation, 2002. [12] S. Vinoski, "REST Eye for the SOA Guy," in IEEE Internet Computing, 2007, pp. 82-84.