Unit-II GRID MONITORING

1.Grid Monitoring Architecture

(GMA)
2.An Overview of Grid
Monitoring Systems -Grid ICE
3.JAMM
4.MDS
5.Network Weather Service
6.R-GMA
7.Other Monitoring Systems Ganglia
8.GridMon

Grid Monitoring
Architecture
(GMA)

The diversity of components and their large

Introduction
number of users render them vulnerable to

faults, failure and excessive loads

Grid monitoring is a critical facet for providing
a robust, reliable and efficient environment
The goal is to measure and publish the state
of all resources software, hardware, and
networks at a particular point in time
Monitoring data can be used in

Fault detection
Recovery
Performance forecast
Grid Computing

The Challenges of Grid

Monitoring

No single point of observation

No central point of monitoring information
Diverse Hardware and Software Systems
Different policies and decision making
mechanisms
Network monitoring is very important
Larger monitoring data sets
Security

Grid Computing

Characteristics for Grid

Monitoring

Scalable
Dynamic
Robust
Flexible
Should be integrated with other Grid
Technologies and middleware (security
infrastructure, resource brokers,
schedulers, ...)

Grid Computing

Grid Monitoring
Architecture
(GMA)

The GMA consists of three types of

components

Directory Service: supports information publication and

discovery
Producer: makes performance data available
(performance event source)
Consumer:Consum
receives performance data (performance
Lookup Location
event sink) er

Subscription to
event

Data Transfer

Producer
6

Directory
Service

Store Location

Grid Computing

Consumer

Consumer is a program that receives monitoring

data (events) from one or more producers
Different types of consumer

The archiving consumer: aggregates and stores

monitoring data for later retrieval/analysis.
Real-time consumer: collects monitoring data in real
time.
Overview consumers: collects events from several
sources. And uses the combined information for
decision making.
Job monitoring consumers: can be used to trigger an
action based on an event from a job.
Grid Computing

Consumer

Consumer steps
1.
2.
3.
4.
5.
6.

7.
8.

9.

Locate events: Consumers search a schema repository for a new event type.
The schema repository can be a part of the GMA Directory Service.
Locate producers: Consumers search the Directory Service to find a suitable
producer.
Initiate a query: Consumers request event(s) from a producer, which are
delivered as part of the reply.
Initiate a subscription: Consumers can subscribe to a producer for certain kinds
of events they are interested in. Consumers request event(s) from a producer
Initiate an unsubscribe: Consumers terminate a subscription to a producer.
Register: Consumers can add/remove/update one of more entries in the
Directory Service that describe events that the consumer will accept from
producers.
Accept query: Consumers can also accept a query request from a producer. The
query will also contain the response.
Accept subscribe: Consumers accept a subscribe request from a producer. The
producer will be notified automatically once there are requests from the
consumers.
Accept unsubscribe: Consumers accept an unsubscribe request from a producer.
If this succeeds, no more events will be accepted for this subscription.

Consumers that initiate the flow of events should support steps 2-5
Consumers that allow a producer to initiate the flow of events should
support steps 6-8
8

Grid Computing

Directory Service

Directory Service provides information about

producers or consumers.
When producers and consumers publish their
existence, they must provide event types they
produce or consume.
The publication information allows producers
and consumers to discovery the types of
available events, the characteristics of that
data, and sources or sinks of data.
Directory Service is not responsible for the
storage of event data; only information about
which event instances can be provided.
9

Grid Computing

Directory Service

Functions supported by the Directory Service

Authorize a search: Establish the identity of a

consumer that wants to undertake a search.
Authorize a modification: Establish the identity of a
consumer that wishes to modify entries.
Add: Add a record to the directory.
Update: Change the state of a record in the directory
Remove: Remove a record from the directory.
Search: Perform a search for a producer or consumer
of a particular type, possibly with fixed values for
some of the event elements.

There can be more than one directory services.

10

Grid Computing

Grid Monitoring
Architecture
(GMA)
Extended Grid Monitoring Architecture with multiple

Directory Service
Consumer

Consumer

Consumer

Event Directory Service Gateway

Event
Event Directory
Directory
Service
Service

Producer

Event
Event Directory
Directory
Service
Service

Produce
r

Produce
r

Event
Event Directory
Directory
Service
Service

Produce
r

Produce
r

Grid Resources
11

Grid Computing

Producer

A producer is a software component that

sends monitoring data (events) to a consumer
Producers can deliver events in a stream or as
a single response per request.
Producers are also used to provide access
control to the event depending on policies,
varying frequencies of measurement and
ranges of performance detail.

12

Grid Computing

Producer

Producer steps
1.
2.
3.
4.
5.
6.
7.
8.
9.

Locate event: Search the Event Directory Service for the description of an event.
Locate consumer: Search the Event Directory Service for a consumer.
Register: Add/remove/update one of more entries in the Event Directory Service
describing events that the producer will accept from the consumer.
Accept query: Accept a query request from a consumer. One or more event(s) are
returned in the reply.
Accept subscribe: Accept a subscribe request from a consumer. Further details about
the event stream are returned in the reply
Accept unsubscribe: Accept an unsubscribe request from the consumer. If this
succeeds, no more events will be sent for this subscription.
Initiate query: Send a single set of event(s) to a consumer as part of a query request.
Initiate subscribe: Request to send events to consumers, which are delivered in a
stream. Further details about the event stream are returned in the reply.
Initiate unsubscribe: Terminate a subscription to a consumer. In this succeeds, no more
data will be sent for this subscription.

Producers that wish to handle new event types dynamically

should support the step 1
Producers that allows consumers to initiate the flow of events
should support steps 2-6
Producers that initiate the flow of events should support steps 7-9

13

Grid Computing

Producer

Optional producer tasks

Event caching allows consumers to request

historical data from a particular sensor for
prediction algorithm
Event filtering can be applied to sent only if data
value crosses a certain threshold

14

CPU utilization is > 50%

1, 10, 60-minute average CPU usage.

Grid Computing

Intermediary

The compound
producer/consumer is a
single component that
implements both
producer and consumer
interfaces
Forward, broadcast,
filter, or cache the
performance events
Lessen the load on
producers of event data
that is of interest to
many consumers
15

Consume
r
Producer
Interface

Monitoring
Service
X
Consumer
Interface

Produce
r

Produce
r

Grid Computing

Monitoring data

Time-related data

16

Time-stamped dynamic data may be provided by

a counter related to the sampling rate. Data
includes performance event and status
monitoring.
Time-stamped asynchronous data indicate when
an event happens (alerts and checkpoints)

Non-time-related data includes static

information such as OS type and version,
hardware characteristics or the update time of
monitoring information
Grid Computing

Monitoring data

Information flow data

Direct producer-consumer flow does not need a

central component. Three interactions are
described by GMA document:
1.
2.
3.

17

Publish/subscribe
Query/response
Notification

Indirect data distribution via a centralized

repository. This is useful for static information.

Grid Computing

Summary

Monitoring is critical for providing a robust, highperformance Grid environment

A basic monitoring has the following components

Producers(sensors) that generate monitoring data (events)

Consumers that consume events
One or more directory services for registration and discovery of
sensors/events/consumers

A monitoring system should have

18

GMA compliance
Caching capability
Scalable
Resources monitored include network resources, host resources
and jobs
Resource performance forecasting
Resource performance analysis
Various presentation views for resource monitoring
Directory service for events subscription and notification
Grid Computing