Message Passing Architecture

Introduction
The architecture is used to communicate Each PE has its own local memory and
communicates with other PEs using message. data among a set of processors without the need for a global memory.

Message Passing Systems

104
MESSAGE PASSING ARCHITECTURE

P 1

M1

P2

M2

Pn

Mn

Link 1

Link 2

Link n

Interconnection Network
Figure 5.1 Message passing systems.

networks have received considerable attention over the years. As shown in Chapter 2, two important factors must be considered in designing message passing interconnection networks: link bandwidth and the network latency. The link bandwidth is dened as the number of bits that can be transmitted per unit of time (bits/s). Network latency is dened as the time to complete a message transfer through the network.

Message Passing Systems

104
MESSAGE PASSING ARCHITECTURE

P 1

M1

P2

M2

Pn

Mn

Link 1

Link 2

Link n

Interconnection Network
Figure 5.1 Message passing systems.

networks have received considerable attention over the years. As shown in Chapter 2, two important factors must be considered in designing message passing interconnecexternal channel tion networks: link bandwidth and the network latency. The link bandwidth is dened as the number of bits that can be transmitted per unit of time (bits/s). Network latency is dened as the time to complete a message transfer through the network.

MP Network

Normally, a static-type network

hypercubes nearest-neighbor 2D & 3D mesh link bandwidth - the number of bits that can be transmitted per unit of times (bits/s) network latency - the time to complete a message transfer through the network

Two important factors must be considered:

Executing Program

The program is divided into concurrent processes; each is executed on a separate processor. If the no. of processes is larger than the number of processors, then more than one process will have to be executed on a processor in a time-shared fashion.
104
MESSAGE PASSING ARCHITECTURE

P 1

M1

P2

M2

Pn

Mn

Link 1

Link 2

Link n

Interconnection Network
Figure 5.1 Message passing systems.

networks have received considerable attention over the years. As shown in Chapter 2, two important factors must be considered in designing message passing interconnection networks: link bandwidth and the network latency. The link bandwidth is dened as the number of bits that can be transmitted per unit of time (bits/s). Network latency

Process Communication
Processes running on a given processor
use what is called internal channels to exchange messages among themselves. use the external channels to exchange messages.

Processes running on different processors

Data Exchanged
Data exchanged among processors cannot
be shared; it is rather copied (using send/ receive messages).

An important advantage of this form of

data exchange is the elimination of the need for synchronization constructs, such as semaphores, which results in performance improvement.

More advantages
A message passing scheme offers exibility
in accommodating a large number of processors in addition to being readily scalable. process, each at a given time.

A given node can execute more than one

5.2

ROUTING IN MESSAGE PASSING NETWORKS

105

Outside World

P 1 P2 m1 m4 P3 m2 P4 m3

Figure 5.2

An example of a message passing system.

Three types of granularity can be distinguished. These are: 1. Coarse granularity: Each process holds a large number of sequential instruc-

Process Granularity
Its a parameter which describes the size of
a process in a message passing system. computation time Process Granularity = communication time

Granularity Types

There are 3 types of granularity:

Coarse - each process holds a large number of sequential instructions and takes a substantial amount of time to execute. Medium - a middle ground where communication overhead is reduced Fine - each process contains a few sequential instructions

Use mostly medium or coarse granularity

Routing in MP Networks
Involves the identication of a set of

permissible paths that may be used by a message to reach its destination, and a function () that selects one path from the set of permissible paths.

Routing Techniques

Adaptive

the path taken by the message depends on network conditions for a given source and destination pair determines the path using only the source and destination regardless of the network conditions - simple but bandwidth inefcient

Deterministic (oblivious)

Routing for Broadcasting & Multicasting

Communication operations: unicast and
collective

Unicast - communicate to only a single

destination

Collective - several routing operations

are dened, broadcast & multicast are the most widely used.

Broadcast one-to-all operation distribute data during computation of a distributed memory program

Multicast one-to-many operation uses in large-scale multiprocessors, inc. parallel search algorithm

routing algorithms must be deadlock-free little time and short path is desirable

of these resources in a strict monotonic order. This restricted way for using network resources prevents the occurrence of circular wait, and hence prevents the occurrence of deadlock. The channel dependency graph (CDG) is a technique used to develop a deadlock-free routing algorithm. A CDG is a directed graph D G(C, E ), where the vertex set C consists of all the unidirectional channels in the network and the set of edges E includes all the pairs of connected channels,
000

110

111
011

001 101 110 100 101 111

010 011 100 001 101 111 011 110

100

100 010

101 011

111 110

010

001 010

000

001

Figure 5.3

Hypercube broadcast tree-based communication.

Routing Potential Problems

Deadlock

When two messages each hold the resources required by the other in order to move, both message will be blocked.

Livelock A situation in which a message keeps going around the network and never reaches its destination.

Starvation A node is said to suffer from starvation if it has a message to inject into the network but is never allowed to do so.

Message Passing vs Shared Memory

Shared memory systems may be easier to program, but the difcult to scale up to a large number of processors. If scalability to larger and larger systems was to continue, systems had to use message passing techniques. However, a number of problems associated with message passing systems: communication overhead and difculty of programming.