Dynamic request allocation and scheduling for context aware applications subject to a percentile response time SLA in a distributed

cloud Keerthana Boloor , Rada Chirkova , Tiia Salo and Yannis Viniotis
Department

of Electrical and Computer Engineering Department of Computer Science North Carolina State University IBM Software Group Research Triangle Park

Cloudcom 2010, Indianapolis, Indiana, USA

1 / 17

Agenda

Agenda

Problem description Dynamic request allocation and scheduling scheme Comparison with static allocation and FIFO/Weighted Round Robin scheduling scheme Conclusion

Cloudcom 2010, Indianapolis, Indiana, USA

2 / 17

Problem description

Problem description

More web applications are designed to be context aware. Most context aware applications are built on SOA principles. Cloud computing systems - the most preferred platform for deployment. Service Level Agreements (SLA) - terms of service and pricing model. What is this presentation about?
Cloudcom 2010, Indianapolis, Indiana, USA

3 / 17

Problem description

Geographically distributed cloud computing system

Geographically distributed cloud computing system

Data center hosting K context-aware applications

Data center hosting K context-aware applications

Clients

Data center hosting K context-aware applications

Data center hosting K context-aware applications

Cloudcom 2010, Indianapolis, Indiana, USA

4 / 17

Problem description

Context aware applications

SOA based context aware application

2. Client request allocated to and scheduled at end-server

Gateway

3. Load required service-endpoint

4. Load required contextdata

Contextaware SOA applications End servers

Contextdata stores

DATA CENTER

Internet 1. Client request with context-id Updates to contexts at contextdata stores

Cloudcom 2010, Indianapolis, Indiana, USA

5 / 17

Problem description

Model of an end-server

An end-server serving multiple user classes

Class 1

Class 2

Server j at data center i

Class K

Each context aware application services multiple classes of users Each user class is guaranteed different quality of service based on economic considerations SLA species different service levels and service charges for the different user classes
Cloudcom 2010, Indianapolis, Indiana, USA

6 / 17

Problem description

Percentile Service Level Agreements

Percentile Service Level Agreements

Profit

100

Conformance(%)

X % - the fraction of requests of a particular user class which need to have a response time less than r seconds $P - The prot charged by the cloud, if the percentile of requests that have response time less than r seconds is greater than or equal to X %
Cloudcom 2010, Indianapolis, Indiana, USA

7 / 17

Problem description

Problem statement

Problem statement Allocate and schedule service requests locally at the end-servers so as to globally:

max
1j K

protj

(1)

where protj is the prot charged for conformance of the requests from users of class j .

Cloudcom 2010, Indianapolis, Indiana, USA

8 / 17

Problem description

Problem statement

Problem statement Allocate and schedule service requests locally at the end-servers so as to globally:

max
1j K

protj

(1)

where protj is the prot charged for conformance of the requests from users of class j . This problem is NP-hard!!

Cloudcom 2010, Indianapolis, Indiana, USA

8 / 17

Solution

Management scheme description

Heuristic-based data-oriented request management scheme

Periodic allocation and adaptation at each datacenter.

Allocation phase

Allocation phase

Allocation phase

Allocation phase

Allocation phase

Adaptation phase

Adaptation phase

Adaptation phase

subinterval Adaptation Adaptation phase phase

Observation interval (T)

00:00

06:00

Cloudcom 2010, Indianapolis, Indiana, USA

9 / 17

Solution

Management scheme description

Heuristic-based data-oriented request management scheme

Periodic allocation and adaptation at each datacenter.

Allocation phase

Allocation phase

Allocation phase

Allocation phase

Allocation phase

Adaptation phase

Adaptation phase

Adaptation phase

subinterval Adaptation Adaptation phase phase

Observation interval (T)

00:00 Adaptation phase

06:00

Datacenters exchange conformance levels. Allocation phase Rank-based request allocation and gi-FIFO scheduling. Aim at increasing global prot.
Cloudcom 2010, Indianapolis, Indiana, USA

9 / 17

Solution

Rank-based allocation and gi-FIFO scheduling

Rank-based allocation and gi-FIFO scheduling

Prot-score calculation
Prot: pk
Profitscore assigned to each arriving request of class 1 ($)

Required global conformance: ck Current global conformance: cck If cck < ck Prot-score = pk /(ck cck ) Else Prot-score = 0

Class 1 SLA Profit of 2000$ on conformance of 75% 2000

1500

1000

500

0 0 10 20 30 40 50 60 70 Current conformance of class 1 (%) 80 90 100

Cloudcom 2010, Indianapolis, Indiana, USA

10 / 17

Solution

Rank-based allocation and gi-FIFO scheduling

Rank-based request allocation

1

Query hash-based lookup table ([context-id,machine-id] or [service-id,machine-id])

Rank-based compatibility test

1

The arriving request is assigned a rank based on its prot-score and deadline. Does the arriving request meet its deadline? - Machine compatible!!!

Compatible machine not found? - Choose least loaded closest to context DB

Cloudcom 2010, Indianapolis, Indiana, USA

11 / 17

Solution

Rank-based allocation and gi-FIFO scheduling

gi-FIFO scheduling
Choose the request of user class with the highest current prot-score

Choose one with maximum waiting time but which results in a response time less than or equal to r
If no such request exists, choose the request with higher waiting time resulting in a response time greater than r

gi-FIFO has been proven to be the most suitable for percentile SLAs for a single server serving multiple classes.

Cloudcom 2010, Indianapolis, Indiana, USA

12 / 17

Evaluation

Evaluation
Dynamic scheme vs static schemes
11000 10000 9000 8000 Profit incurred ($) 7000 6000 5000 4000 3000 2000 1000 0 5 10 15 20 25 30 Request rate 35 40 45 50 Dynamic rank based allocation with giFIFO scheduling Static allocation with WRR scheduling Static allocation with FIFO scheduling

Cloudcom 2010, Indianapolis, Indiana, USA

13 / 17

Evaluation

Dynamic rank based allocation vs static allocation scheme

11000 10000 9000 8000 Profit incurred ($) 7000 6000 5000 4000 3000 2000 1000 0 0 50 Request rate 100 150 Static allocation with giFIFO scheduling Dynamic rank based allocation with giFIFO scheduling

Cloudcom 2010, Indianapolis, Indiana, USA

14 / 17

Evaluation

Variation in subinterval length

Variation in context update interval

18000 16000 14000 12000 10000 8000 6000 4000 2000 0 Uniform distribution of classes, stringent SLA Uniform distribution of classes, relaxed SLA Nonuniform distribution of classes, stringent SLA Nonuniform distribution of classes, relaxed SLA

18000 16000 14000 12000 10000 8000 6000 4000 2000 0 Low contextdata load times High contextdata load times Medium Contextdata load times

Profit obtained ($)

Profit obtained($)

50

100

150

200 250 300 Subinterval period

350

400

450

500

20

40

60

80 100 120 140 Contextdata update interval

160

180

200

Cloudcom 2010, Indianapolis, Indiana, USA

15 / 17

Conclusion

Conclusion

Identied the need for dynamic request scheduling and allocation for context aware applications in a distributed cloud.

Proposed a novel rank-based request allocation and gi-FIFO scheduling scheme for managing percentile SLAs with an aim to maximize prot obtained by the cloud.

Cloudcom 2010, Indianapolis, Indiana, USA

16 / 17

Questions??

Cloudcom 2010, Indianapolis, Indiana, USA

17 / 17