Parallel Job Submission In Grid Environment

Using
Parallel Particle Swarm Optimization

Dr. G. Sudha Sadhasivam

Asst. Professor
D. Komagal Meenakshi (07MW05) Dept. of CSE.
PSG College Of Technology. PSG College Of Technology.
 Outline
• Scheduling in Grid.
• Problem Statement
• Need For Job Grouping in Scheduling
• Previous Work Done in Job Grouping
• Proposed System
• Trust Based Filtering of jobs
• Particle Swarm Optimization
• Parallel PSO
• Model for PPSO
• Dynamic jobs
• Results
• Conclusion and Future work
• Bibliography
 Scheduling in Grid.
• Grid computing is a high performance computing
environment to solve large scale computational demands.

• Task scheduling is a fundamental issue in achieving

high performance in grid computing systems.

• Reason: Large numbers of tasks are computed on the

geographically distributed resources, a reasonable
scheduling algorithm must be adopted order to minimize
job completion time with uniform load distribution.
 Need

An unorganized deployment of grid applications with a

large amount of fine-grain jobs
Leads to
communication overhead dominate the overall processing
time
Results
Low computation-communication ratio.
 Need For Job Grouping in Scheduling
• Efficient job grouping-based scheduling system is required.

• A Grid Scheduler should

Reduce the total transmission of user jobs to/from the
resources.
Reduce the overhead processing time of the jobs at the
resources.
 Job Grouping
Jobs of an application [ fine grained ]

Dynamically
assemble

job groups [ coarse grained ]

Transmit

Grid resources
 Previous Work Done in Job Grouping

• Comparison of Scheduling algorithms with and without job

grouping.

• In the context of DAG scheduling, grouping of jobs into clusters

to reduce inter-job communication.

• Job Grouping strategy, adaptive to run time environment

• Job Grouping with PSO.

 Proposed System
• A novel job grouping method using Parallel PSO
• To reduce the communication overhead.
• Enhance the speed of completion of processes.
• Improve resource utilization.
• Improve parallel efficiency.
• Uses PPSO to select the resources to minimize the make span.
• Trust level and dynamism of jobs is considered

• Tool Used - Gridsim-4.2-beta.

The Project aims at …
• Job Grouping based on trust Using PPSO
• Parallel Job Submission
• Enhancing Computation-communication Ratio
• Reducing The Overall Processing Time Of Jobs Using
Parallelization
• Improving Resource Utilization In The Grid Environment.
• Trust based job filtering
• Dynamic job submission
 1. Job Grouping
Filtered Jobs of an application [ fine grain] based on Trust

Dynamically assemble
Using PPSO

job groups [ coarse grained ]

Transmit to

In Parallel

Grid resources Grid resources Grid resources

 Job Grouping
User Input
Total number of jobs
Average MI rate of job Grid Resource File
Granularity time
MI deviation Percentage Grid resource 0
Overhead processing time Grid resource 1
Grid Resource
Grid resource N
Trust level

(1) (2)
Filter jobs
(7)
based on Gridlets
trust Grid resources’ characteristics
(3)

Gridlet MI Resource MIPS Granularity time

(4)
Gridlet Scheduler
Total MIPS

(5)

Grid resource 0 Grid resource 1 Grid resource 2

…..
Gridlet group 0 Gridlet group 1 Gridlet group 2

(6)
Gridlet groups Resource IDs

In parallel
 2. Trust Based Filtering of jobs
• The Grid Information Service GIS gives the information
about all the trust level of the resources .

• The user submits the jobs with different trust values.

• From this, the jobs that have trust values greater than
the resource's trust value are filtered out.

• Trust aware resource management and scheduling offer

Quality of Service at application layer in grid
environment.
 3. Particle Swarm Optimization
• If large numbers of tasks are computed on the
geographically distributed resources, a reasonable
scheduling approach must be adopted in order to
get the minimum completion time.

• Task scheduling is a NP-Complete problem

• Heuristic optimization algorithm can be used to

solve NP-complete problems.
• Particle Swarm Optimization (PSO) is an evolutionary
optimization technique inspired by nature.

• It simulates the process of a swarm of birds preying.

• Its global searching ability can be used for neural

network training, control system analysis and design,
structural optimization.

• It also has fewer algorithm parameters than genetic

algorithm.

• PSO algorithm works well on most global optimal

problems.
 PSO Concept
• A swarm intelligence based algorithm finds a solution to
an optimization problem in a search space.

• Proposed solution exists in the form of a fitness function.

• The swarm is typically modeled by particles in

multidimensional space that have a position and a
velocity.

• A Particle is a candidate solution in the population and

represents a task.
• Particles fly through hyperspace .

• An iterative process to improve candidate solutions is set in motion.

The particles iteratively evaluate the fitness of the candidate
solutions.

• Particles posses two essential reasoning capabilities

– Memory of their own best position and
– knowledge of the global best of the swarm.

• As the swarm iterates, the fitness of the global best solution

improves.

• All particles being influenced by the global best eventually approach

the global best. This phenomenon is called 'convergence'.
 PSO Algorithm
• Initialize parameters

• Initialize population randomly

• Initialize each particle position vector and velocity vector

Do {
• Update each particle’s velocity and position;
• Find a permutation according to the updated each particle’s position;
• Evaluate each particle and update the personal best and the global best;
• Apply the local search;
• } While (!Stop criterion)
 Parallel PSO
Recent advances in computer and network technologies led to parallel optimization
algorithms.

Parallel PSO (parallel implementation of stochastic optimization alg)

 Parallel PSO design
Intialize

# of particles

f(x) f(x) f(x)

# of iterations

Check Convergence

Update
 Model for PPSO

Master

RECEIVE INDIVIDUAL
SEND GLOBAL VALUE
VALUE

Slave Slave Slave

 Gridlet Grouping
Trust based
filtered Gridlet Resource list
list

Scheduler

Call PPSO to assign Gridlet To Resources

Create new grouped Gridlet

With length= Total length

Assign to resources
 4. Dynamic jobs

• Dynamic submission of jobs is considered.

• User can submit jobs when other jobs are being
processed.
• The unused MIPS rating of the resources can be
utilized in a efficient way such that grouping is
done by considering the unused MIPS as total
MIPS and the jobs are processed.
• Then Parallel Submission of grouped Gridlets to
resources is done
Simulation Time for Job Grouping
using PSO vs. Parallel PSO
150
Simulation Time

140
130
PPSO
120
PSO
110
100
90
20 40 60 80
No of Gridlets
 Total number of processed gridlets for
different granularity time and resources

100
No of Gridlets completed in gran

80
10
60 20
time

30
40 40
50
20

0
R1 R1-R2 R1-R3 R1-R4 R1-R5
Resources
Load at resources during job grouping
with PPSO
1000
900
800
700 50gridlets
600 60gridlets
load

500 70gridlets
400 80gridlets
300 90gridlets
200
100
0
R1 R2 R3 R4 R5
Resources
Difference in submission time of gridlets
with PSO and PPSO
• Add load balancing feature graph here
 Conclusion
• The proposed framework using PPSO has less simulation
time compared to job scheduling framework using PSO as
the simulation time is reduced.
• Resource selection based on PPSO is used to generate an
optimal schedule so as to complete the tasks in a minimum
time than PSO as well as utilizing the resources in an
efficient way.
• Simulated results demonstrates load balanced resource
selection.
• Simulation results demonstrate that PPSO algorithm can get
better effect for a large scale optimization problem.
 Future Work
• Future work would involve developing a more
comprehensive job grouping-based scheduling system that
takes into account QoS (Quality of Service) requirements of
each user job before performing the grouping method.
• Resource utilization can be done according to the capacity
of the resource.
THANK YOU