Académique Documents
Professionnel Documents
Culture Documents
Outline
Introduction to Grid Architecture Basic Components of a Grid WSRF and OGSA: Creating Grid Enabled Web Services
Web services WSRF OGSA
Fault Tolerance
Software
How to effectively connect and harness the geographically dislocated computer power? Answer: meta-computing
GIIS
DUROC
GRIS
GRAM
GRIS
GRAM
GRIS
GRAM
Globus provides a uniform API for high-level functions like discovery and job submission but does not ease building the application across platforms.
resource sharing can be used to divide the workloads not easy to integrate with applications
9
10
Non-Computational Resources
Resource (Grid) is any element of the networked infrastructure that is made available for usage with standardized Grid protocols.
storage system with standardized interfaces network sensors, telescopes, industrial infra and other specialized instruments application software where confidentiality of data is managed
Each resource has a feature set that makes it unique. Resource interfaces differs in characteristics that is important for interoperability.
11
12
Virtualization
While the hardware capabilities are constant, the application demand fluctuates overtime. service pool : adopt an organized approach to pooling the resources. Virtualization: applications contact the services to submit their processing request but do not need to know the execution. Benefits of virtualization
better classification and presentation of capabilities effective utilization of common resources thinner and better maintainable application layer common access to rare resources
13
Virtualization Organization
On the management layer, the entities are constrained by membership to various categories A VO is a set of Grid entities that are related to each other by some level of trust United for a common goal rather than membership One grid could host several VOs, an entity may join several host and VOs can overlap
14
Trust Relationships
For the VO model to work, parties need a trust relationship
the alternative: every user needs to register at every resource we need to provide a sign-on for the user that works across VOs
Policy Authority
Policy Authority
Sub-Domain A1
Sub-Domain B1
Domain A
Task
Domain B
AuthZ Service Federation GSI Server X
Server Y
Proxy certificates (GSI) allows other host to process the services of an original host.
17
Interface
Interface
18
19
20
Web Service
Web Applications
Web is designed for application to human interactions.
Information sharing: a distributed content library. Enabled B2C e-commerce. Non-automated B2B interactions.
22
A web service is a piece of software that is made available on the Internet and utilizes a standardized XML messaging system. In other words a web service is a remote procedure call over the Internet using XML messages. Web services standards are being defined within the W3C (World Wide Web Consortium) and other standard bodies
23
Service requester
Service interface: WSDL file (XML)
Service provider
SOAP provides a means of messaging between a service provider and a service requester. SOAP is a simple enveloping mechanism for XML payloads that defines an RPC convention. Benefit: loosely coupling components by document oriented communication 24
WS Standards 2: UDDI
How can I discover business partners with compatible web service solutions? How do let other business know about my exposed web services? Web services are great, after you find out about them, but the discovery process is difficult Information system for Web services: UDDI - Universal Description, Discovery and Integration
25
The WS vision
1.
SW companies, standards bodies, and programmers populate the registry with descriptions of different types of services Marketplaces, search engines, and business apps query the registry to discover services at other companies
4.
2.
5.
Businesses populate the registry with descriptions of the services they support
3.
UBR assigns a programmatically unique identifier to each service and business registration
Business uses this data to facilitate easier integration with each other over the Web
26
SOA implies the next step of evolution of distributed architecture but will break the security model of enterprises. Web services are static and cannot provide developers to design on-demand systems. What is needed? Service-base utility computing paradigm Technology needed: cross-organizational virtualization of the resources
27
28
29
30
A Grid Services
Describe Web service interfaces that implement useful behaviors
primitives: resources, state, security services: program execution, data movement, data access,
31
Internet standards
Custom solutions
Globus Toolkit
Defacto standard Single implementation
1990
1995
2000
2005
2010 33
34
OGSI to WSRF
Open Grid Service Infrastructure (OGSI) was published by the Global Grid Forum (GGF) in June 2003
intended to provide an infrastructure layer for the Open Grid Services Architecture (OGSA) extending Web services to accommodate grid computing resources that are both transient and stateful
IBM realized that OGSI was too far from standard Web services OGSI was replaced by WS Resource Framework which is more appropriate WSRF retains essentially all of OGSI concepts
35
WS-Resource Framework
Researchers from Globus group and IBM, in 2004, proposed a standard called WS-Resource Framework (WSRF) WSDL (Web Service Definition Language) is the language on creating interfaces WS-Resource is a resource associated with a Web service WSRF aim is to define the foundation of virtualized service layer and its relation to the resource layer Standardized layer for virtualized services will ease and open the communication of services
36
37
WSRF Specification
Defines fundamental interfaces (using extended WSDL) and behaviors that define a Grid Service
A unifying framework for interoperability & establishment of total system properties
Defines basic patterns of interaction, which can be combined with each other and with custom patterns in a myriad of ways
38
WS-Resource Framework & WS-Notification* (Resource identity, lifetime, inspection, subscription, ) Web services (WSDL, SOAP, WS-Security, WS-ReliableMessaging, )
39
A WSRF Illustration
Client
Introspection: What port types? What policy? What state? Lifetime management Explicit destruction Soft-state lifetime Data access Other standard interfaces: factory, notification, collections Service
data element
GridService (required)
handle resolution
Grid Service Reference
Implementation
Hosting environment/runtime (C, J2EE, .NET, )
40
Related specifications
WS-Notification WS-Addressing
41
The Physiology of the Grid: An Open Grid Services Architecture for Distributed Systems Integration, Foster, Kesselman, Nick, Tuecke, 2002
OGSA Structure
A standard substrate:
Standard interfaces and behaviors that address key distributed system issues A refactoring and extension of the Globus Toolkit protocol suite
GridFTP
(GRAM)
(MDS)
C Common Libraries
Non-WS Components
Security
Data Mgmt
Execution Mgmt
Info Services
Common Runtime
MDS3
Java WS Core
GridFTP
Replica Location
(GRAM)
(MDS)
Non-WS Components
Security
Data Mgmt
Execution Mgmt
Info Services
Common Runtime
Trigger
Index
GridFTP
Replica Location Data Mgmt
Pre-WS Pre-WS Grid Resource Monitoring Alloc. & Mgmt & Discovery
Non-WS Components
www.globus.org
Execution Mgmt Info Services
Security Service
Data Replication Credential Mgmt Delegation
Community Authorization
Authentication Authorization
Replica Location
Data Access & Integration
WebMDS
Python Runtime
C Runtime Java Runtime Common Runtime
Trigger
GridFTP
Data Mgmt
Index
Info Services
Security
Execution Mgmt
48
User Proxies
Minimize exposure of users private key A temporary, X.509 proxy credential for use by our computations
We call this a user proxy certificate Allows process to act on behalf of user User-signed user proxy cert stored in local file Created via grid-proxy-init command
Delegation
Remote creation of a user proxy Results in a new private key and X.509 proxy certificate, signed by the original key Allows remote process to act on behalf of the user Avoids sending passwords or private keys across the network
User who wants access to resource asks CAS for a capability credential
Restricted proxy of the CAS user cred., checked by resource
Execution Management
Data Replication Credential Mgmt Delegation
Community Authorization
Authentication Authorization
Replica Location
Data Access & Integration
WebMDS
Python Runtime
C Runtime Java Runtime Common Runtime
Trigger
GridFTP
Data Mgmt
Index
Info Services
Security
Execution Mgmt
53
I. Foster, Globus Toolkit Version 4: Software for Service-Oriented Systems, LNCS 3779, 2-13, 2005
Client GRAM
Can negotiate access to clusters, creation of virtual machines, establishment of virtual networks,
54
Community Scheduler
Individual users
Require service Have application goals
Community schedulers
Broker service Aggregate scheduling
Individual resources
Provide service to clients Have policy autonomy
Data Management
Data Replication Credential Mgmt Delegation
Community Authorization
Authentication Authorization
Replica Location
Data Access & Integration
WebMDS
Python Runtime
C Runtime Java Runtime Common Runtime
Trigger
GridFTP
Data Mgmt
Index
Info Services
Security
Execution Mgmt
56
57
GridFTP in GT4
100% Globus code No licensing issues Stable, extensible IPv6 Support XIO for different transports Striping multi-Gb/sec wide area transport
27 Gbit/s on 30 Gbit/s link
Bandwidth Vs Striping 20000 18000 16000 14000 12000 10000 8000 6000 4000 2000 0 0 10 20 30 40 50 60 70 Degree of Striping # Stream = 1 # Stream = 8 # Stream = 2 # Stream = 16 # Stream = 4 # Stream = 32
Pluggable
Front-end: e.g., future WS control channel Back-end: e.g., HPSS, cluster file systems Transfer: e.g., UDP, NetBLT transport
Bandwidth (Mbps)
Disk-to-disk on TeraGrid
58
RFT Service
GridFTP Server
Master DSI Protocol Interpreter Data Channel Data Channel
GridFTP Server
Protocol Interpreter Master DSI
IPC Link
IPC Receiver Slave DSI Data Channel Data Channel
IPC Link
Slave DSI IPC Receiver 59
Index
Index
60
Information Service
Data Replication Credential Mgmt Delegation
Community Authorization
Authentication Authorization
Replica Location
Data Access & Integration
WebMDS
Python Runtime
C Runtime Java Runtime Common Runtime
Trigger
GridFTP
Data Mgmt
Index
Info Services
Security
Execution Mgmt
61
Indexes can be registered to other indexes Indexes at top of slide contains information about all 6 hours
MDS Components
Collective services aggregate information from multiple resources Index service maintains the set of registered Grid Resources Trigger service contains the list of rules to perform actions Component-specific information Queue status information for GRAM GT4 WSRF core provides underlying layer
GT4 Container
MDSIndex
adapter
GT4 Cont.
Custom protocols for non-WSRF entities
MDSIndex
GRAM
65
66
MathService
MathService is a service processing simple mathematical operation using a server Client sends a request to process the values and operations to the server Allow users to perform the following operations:
Addition Subtraction
67
RMI Program
68
2.
3.
69
Interface
import java.rmi.*; public interface Math extends Remote{ public void add(int a) throws RemoteException; public void subtract(int a) throws RemoteException; public int getValueRP() throws RemoteException; }
java.rmi.* is a library for RMI implementations interface Math, not class Math, to declare the code as interface and extend to Remote for RMI procedure Both add and subtract are procedures (void) getValueRP returns the current value
70
MathImpl Procedures
class MathImp name of the server that will handle the
object
UnicastRemoteObject - for non-replicated remote
object whose references are valid only while the server process is alive procedures public void add(int a) add the value of a to current value public void subtract(int a) - subtracts the value of a to current value public int getValueRP() getting the value
72
remote object
MathImpl Mathserver = new MathImpl() creating an
Naming Service
Directory that associates names to remote objects (bind)
Naming
X Y
Remote Object C
74
Client code
import java.rmi.*; import java.rmi.server.*; public class MathClient{ public static void main(String[] args) { System.setSecurityManager(new RMISecurityManager()); try { System.out.println("Security Manager loaded"); String url = "rmi://nodea:1099/mathserver"; Math remoteObject = (Math)Naming.lookup(url); remoteObject.add(10); remoteObject.add(5); System.out.println("Current value: " + remoteObject.getValueRP()); remoteObject.subtract(5); System.out.println("Current value: " + remoteObject.getValueRP()); } catch (RemoteException exc){ System.out.println("Error in lookup: "+exc.toString()); } catch (java.net.MalformedURLException exc){ System.out.println("Malformed URL: "+exc.toString()); } catch (java.rmi.NotBoundException exc){ System.out.println("Not bound: "+exc.toString()); } 75 }}
Client Procedures
Naming.lookup(url) searches the rmi server with the remote object method and passes to remoteObject "rmi://nodea:1099/mathserver the name of the
server
remoteObject.add() and remoteObject.subtract()
the interface for the remote method of the object remoteObject.getValueRP - get the current value of the object
76
Naming Service
Client use Naming Service to find a particular Server object (lookup)
Naming
Object Client lookup(Y) X Y Remote Object Server
Z Host titi.epfl.ch
77
78
79
80
Define the service's interface. This is done with WSDL Implement the service. This is done with Java. Define the deployment parameters. This is done with WSDD and JNDI Compile everything and generate a GAR file. This is done with Ant Deploy service. This is also done with a GT4 tool
81
</xsd:schema> </types>
83
84
87
this.propSet = new SimpleResourcePropertySet( MathQNames.RESOURCE_PROPERTIES); try { ResourceProperty valueRP = new ReflectionResourceProperty( MathQNames.RP_VALUE, "Value", this); this.propSet.add(valueRP); setValue(0); ResourceProperty lastOpRP = new ReflectionResourceProperty( MathQNames.RP_LASTOP, "LastOp", this); this.propSet.add(lastOpRP); setLastOp("NONE"); } catch (Exception e) { throw new RuntimeException(e.getMessage()); } }
88
89
91
92
93
94
The Client
EndpointReferenceType object representing the endpoint reference of this service. an endpoint reference is used to address a particular WS-Resource Obtain a reference to the service's portType
MathServiceAddressingLocator locator = new MathServiceAddressingLocator(); try { String serviceURI = args[0]; EndpointReferenceType endpoint = new EndpointReferenceType(); endpoint.setAddress(new Address(serviceURI)); MathPortType math = locator.getMathPortTypePort(endpoint); math = locator.getMathPortTypePort(endpoint); math.add(10); math.add(5); System.out.println("Current value: + math.getValueRP(new GetValueRP())); math.subtract(5); System.out.println("Current value: "+ math.getValueRP(new GetValueRP())); } catch (Exception e) { e.printStackTrace(); 95 }
96
Exercises
Build the codes for RMI and WSRF that performs multiplication and division using the same implementation codes from the previous lesson.
Modify the current code to add the 2 new methods Create a client that will invoke these methods
97