Académique Documents
Professionnel Documents
Culture Documents
3.6.1
WebSphere MQ Interoperability
3.6.1
WebSphere MQ Interoperability
Note
Before using this information and the product it supports, read the information in Notices on page 23.
Contents
WebSphere MQ Interoperability
. . . . 1
Basic MQ architecture . . . . . . . . . . . 1
Message workflow . . . . . . . . . . . . 2
HTTP to MQ . . . . . . . . . . . . . 2
MQ to HTTP . . . . . . . . . . . . . 3
Routing . . . . . . . . . . . . . . . . 5
Back side Request routing . . . . . . . . . 5
Back Side Destinations . . . . . . . . . . 5
MQ URL . . . . . . . . . . . . . . 6
Front side Reply routing . . . . . . . . . 7
MQ Headers . . . . . . . . . . . . . . 8
MQ API support . . . . . . . . . . . . . 11
Authentication and authorization . . . . . . . 12
Error handling . . . . . . . . . . . . . 13
.
.
.
.
.
13
13
14
15
15
16
. 17
. 21
. 21
Notices . . . . . . . . . . . . . . 23
Trademarks .
. 23
iii
iv
WebSphere MQ Interoperability
The IBM WebSphere DataPower XML Integration Appliance XI50 (DataPower
device) can exchange messages with WebSphere MQ systems by acting as an MQ
client node. This capability allows the DataPower device to bridge disparate
messaging and transport protocols, such as HTTP or TIBCO EMS, to WebSphere
MQ. Messages originating within or outside of an MQ messaging bus can flow
easily to and from another MQ messaging bus or other messaging system, such as
HTTP or TIBCO EMS. It is the Multi-Protocol Gateway service running on the
DataPower device that makes this possible.
In addition to the messaging system bridging, the Multi-Protocol Gateway service
running on the DataPower device can also apply the full range of transformation,
security, authorization, routing, logging and customization services available to the
messages flowing through the DataPower device to and from the WebSphere MQ
system. In every case, the DataPower device behaves as an intermediary in the
message flow. The device does not store or hold messages outside the context of a
single transaction, as an MQ Queue Manager might do.
A Multi-Protocol Gateway service, and the other configuration objects employed on
the DataPower device to implement interoperability with WebSphere MQ
messaging systems, offer a great deal of flexibility for tuning the optimal
interconnection. This paper discusses these configuration options in the context of
an enterprise architecture.
Basic MQ architecture
Figure 1 is an illustration of the basic architecture created when a DataPower
device is used to connect an HTTP-based messaging system (typical of a Web
Services architecture) to a WebSphere MQ-based messaging bus inside the
enterprise. This illustration includes the primary configuration objects created on
the DataPower device as well as the configuration of the MQ Queue Manager to
which the DataPower device connects and exchanges messages.
Server Host
Queue
Channel
HTTP
DataPower
Reply
Request
Web
Services
Front
Side
Handler
Multi-Protocol
Gateway
Q Mgr
Object
Processing
Policy
Q Mgr
Group
Error
Application
Server
MQ Queue
Manager
The Front Side Handler object implements HTTP transport connectivity on the
client, or front, side of the device. On the backside, the Multi-Protocol Gateway
employs MQ-based URLs to determine the MQ queue to which requests are
forwarded, and also from which replies are pulled.
Copyright IBM Corp. 2006,2007
Server Host
Queue
Channel
Reply
DataPower
Request
Error
MQ Queue
Manager
Front
Side
Handler
Multi-Protocol
Gateway
HTTP
Application
Server
Processing
Policy
Q Mgr
Object
Q Mgr
Group
Here, the Front Side Handler polls a particular MQ queue for request messages,
and places replies from the back end services on another MQ queue. The Front
Side Queue Manager object might optionally place messages in an error queue on
the MQ Queue Manager. On the backside, a standard HTTP URL is used to
determine the destination to which requests are forwarded, and from which
answers are received in accordance with the HTTP specification.
In both of these architectures, the DataPower device acts as an MQ client only. The
DataPower device does not act as a Queue Manager. The DataPower device does
implement some transactionality, as discussed in Units of Work on page 15.
Message workflow
HTTP to MQ
As illustrated in Figure 1 on page 1 (HTTP to MQ), messages flow to and from the
DataPower device, and work is performed by the DataPower device, in the
following sequence.
1. The HTTP client sends an HTTP-based request (typically an HTTP Post
containing a SOAP XML document, but might contain binary data) to the
DataPower device. An HTTP Front Side Protocol Handler listens on an
assigned port for incoming requests.
2. The Front Side Handler passes the message to the Multi-Protocol Gateway
service object. The Multi-Protocol Gateway then applies any and all relevant
Processing Policy actions on the message.
3. The Multi-Protocol Gateway can dynamically determine the appropriate
destination to which to route the message, or can route all messages statically
to a particular destination. In either case, in this architecture, the destination is
a particular queue managed by a particular MQ Queue Manager. The
DataPower MQ Queue Manager object contains the necessary information to
establish a connection to the MQ Queue Manager.
4. The message is placed on the destination queue with MQPUT.
If the network connection to a Queue Manager fails for any reason other than
MQ Error Code 2009 (connection broken), the DataPower device will abort the
transaction and start error handling (such as run a configured Error Rule) and
also start the automatic retry mechanism (as determined by the Automatic
Retry property of the Queue Manager object). The DataPower device will
automatically retry placing a message on a queue once, in response to a 2009
MQ error code.
If the remote Queue Manager becomes unavailable, the DataPower device can
optionally attempt to place the message on an alternate, or failover, remote
Queue Manager, determined by a Queue Manager Group.
In general, backside protocol level error handling is controlled by the Process
HTTP Errors property of the Multi-Protocol Gateway. If the property is On, the
Gateway will use the configured Response Rule to process any message that
may be received with the error. Note that errors in MQ often contain no
message, which will cause the Gateway to run any configured Error Rule. If the
Process HTTP Errors property is set to Off, the Gateway will automatically
return an HTTP 500 response to the caller unless an Error Rule is configured.
5. The DataPower device polls the Reply-To queue specified in the Destination
URL to find a correlated response message. The Multi-Protocol Gateway
examines the Correlation ID value in the MQ header of messages on the
Reply-to queue; when this ID is the same as the Message ID assigned to the
request, the Multi-Protocol Gateway takes the message as the response. Note
that the Reply-to queue specified in the MQMD header of the message should
agree with the Reply-to queue specified in the Destination URL when the
Multi-Protocol Gateway employs a static back end configuration. If it does not,
the Gateway will not be able to find the response message.
If such a message is found, the Multi-Protocol Gateway might again apply any
of the configured Processing Policy actions to the response and returns the
reply to the requesting HTTP client. This can include error responses from the
back end application server. If no response is found within the Timeout set in
the Destination URL, the Multi-Protocol Gateway handles the error in the same
fashion as a back end error. Note that the Timeout must be set in the
Destination URL used to contact the MQ back end queue or the Gateway will
wait forever polling for a response message. This will cause the front side
HTTP connection to time out.
The Multi-Protocol Gateway can also be configured to retrieve and process any
message found on the Reply-to queue, rather than only the correlated message.
This can be done by using the Setvar processing action, or by using the
set-variable() extension function. Using either method, set the variable
var://context/INPUT/_extension/header/X-MQMD-Get to <MQMD/>.
The URL used to open the connection to the back end Request queue can omit
designating a Reply-to queue. In this case, the Multi-Protocol Gateway will not
wait for a correlated response message or any error message, terminating the
transaction immediately after putting the message on the back side queue. No
response is sent to the front side HTTP client.
MQ to HTTP
As illustrated in Figure 2 on page 2 (MQ to HTTP), messages flow to and from the
DataPower device, and work is performed by the DataPower device, in the
following sequence.
1. An MQ Front Side Protocol Handler polls the configured Request queue,
managed by the referenced MQ Queue Manager, for incoming requests. All
messages found on the queue are copied from the queue.
WebSphere MQ Interoperability
The GET can be controlled with MQ GET options that are available for large
segmented messages through the Front Side Protocol Handler. Table 1 list the
possible values.
Table 1. Available MQ Get options
Option String
Hex
Decimal
MQGMO_ACCEPT_TRUNCATED_MSG
0x40
64
MQGMO_LOGICAL_ORDER
0x8000
32768
MQGMO_COMPLETE_MSG
0x10000
65536
MQGMO_ALL_MSGS_AVAILABLE
0x20000
131072
MQGMO_ALL_SEGMENTS_AVAILABLE
0x40000
262144
The Front Side Protocol Handler accepts the decimal value for the desired
option.
To implement redundancy, the Multi-Protocol Gateway can be configured to
use more than one Front Side Protocol Handler polling a range of queues. It is
then up to the MQ system, independent of the DataPower device, to route
messages to an active queue. If the message is a Request, the Front Side
Handler will save the MessageID, ReplyToQueue and ReplyToQueueManager
fields of the message for later processing of a reply message. See item 4 below
for more information.
By default, all request messages are immediately removed from the Request
queue. It is possible to leave the request message on the Request queue until
processing by the Multi-Protocol Gateway is complete with no errors by setting
the Units of Work property of the MQ Queue Manager object in use by the
Front Side Protocol Handler to 1. Refer to Units of Work on page 15 for more
information.
2. The Front Side Handler passes the message to the Multi-Protocol Gateway
service object. The Multi-Protocol Gateway then applies any and all relevant
Processing Policy actions on the message. The Multi-Protocol Gateway can
dynamically determine the appropriate destination to which to route the
message, or can route all messages statically to a particular destination. In this
scenario, the back end employs HTTP. The Multi-Protocol Gateway opens an
HTTP connection and typically posts the request.
If the Gateway encounters a network error or is otherwise unable to establish a
connection to the back end server, by default the Gateway will run any
matching Error rule in the Processing policy. This result is then returned to the
Front Side Protocol Handler for delivery. See #4 below.
3. The Multi-Protocol Gateway gets the response from the back end application
server. In this scenario, the HTTP protocol requires a response (which might
contain an error message) or the Multi-Protocol Gateway will register an error.
When the back end HTTP server returns a good response, the Multi-Protocol
Gateway applies any applicable Processing Policy actions to the response. Any
processing errors will typically terminate the transaction unless error handling
has been built into the Policy. When the back end application returns a
response with a protocol error, such as HTTP 500, by default the Gateway will
process this response using the same Response Rules it uses for a good
response. The result of any such processing is then forwarded to the PUT
queue of the Front Side Protocol Handler.
When the back end returns an error (such as HTTP 500), the Gateway can run
an Error rule, rather than a standard Response rule. To do this, set the Process
HTTP Errors property of the Gateway to Off. The Gateway places the result of
the Error rule on the Put queue of the Front Side Protocol Handler unless the
Queue Manager's Units of Work property has been set to 1, in which case the
Error rule runs but no response message is delivered. See Refer to Units of
Work on page 15 for more information.
4. The response message can be placed on the Reply-to queue specified in the
Front Side Protocol Handler. The Front Side Handler sets the MQMD CorrelID
to the saved MsgID of the original request. If the original request message
specifies a Reply-to queue, the Front Side Protocol Handler will place the
response on the queue specified in the message, rather than the Put queue
specified in the Front Side Protocol Handler. Unlike HTTP, the MQ protocol
does not require a response. However, if the Response Rule has changed the
ReplyToQ and the ReplyToQMgr in the MQMD header, those will be honored.
See the Routing section for more detail.
Routing
A Multi-Protocol Gateway can route messages to and from MQ queues just as
messages are routed to and from HTTP destinations. The Multi-Protocol Gateway
can use static or dynamically determined routes.
message, could ignore the responses, or could send a response message that does
nothing more than acknowledge receipt of the original request.
All destination determinations can employ an MQ URL to express the back end
destination.
MQ URL
The DataPower device employs an MQ URL to express a destination queue to
which messages are put. An MQ URL is similar to the following example:
dpmq://QueueManager/URI?RequestQueue=PUTQ;ReplyQueue=GETQ;
Sync=true;Model=true;Timeout=1000;PMO=2048
QueueManager
The name of an existing MQ Queue Manager object. You can optionally
use the name of a Queue Manager Group to implement failover.
URI
RequestQueue
The name of a queue managed by the QueueManager selected. Requests are
placed on this queue.
ReplyQueue
The name of a queue managed by the QueueManager selected. The
Multi-Protocol Gateway polls this queue for responses.
Sync
Model
Timeout
Sets a timeout, in milliseconds, for the transaction.
PMO
The PMO tag allows the user to set the MQPMO.Options field on the MQPUT
call used to place the Request message on the back side request queue. The
values defined by the MQ API are:
/* Put-Message Options */
#define MQPMO_SYNCPOINT 0x00000002
#define MQPMO_NO_SYNCPOINT 0x00000004
#define MQPMO_NEW_MSG_ID 0x00000040
#define MQPMO_NEW_CORREL_ID 0x00000080
#define MQPMO_LOGICAL_ORDER 0x00008000
#define MQPMO_NO_CONTEXT 0x00004000
#define MQPMO_DEFAULT_CONTEXT 0x00000020
#define MQPMO_PASS_IDENTITY_CONTEXT 0x00000100
#define MQPMO_PASS_ALL_CONTEXT 0x00000200
#define MQPMO_SET_IDENTITY_CONTEXT 0x00000400
#define MQPMO_SET_ALL_CONTEXT 0x00000800
#define MQPMO_ALTERNATE_USER_AUTHORITY 0x00001000
#define MQPMO_FAIL_IF_QUIESCING 0x00002000
#define MQPMO_NONE 0x00000000
chanName
The name of the channel used to access the MQ Queue Manager
qmName
The name of the target MQ Queue Manager. You can optionally use the
name of a Queue Manager Group to implement failover.
uName
maxChannels
An integer within the range 2 to 5000 that specifies the maximum number
of open channels allowed between the appliance and MQ Queue Manager
timeout
An integer specifying the number of seconds that inactive channels will be
cached (remain open) the dynamic channels will be cached until they
timeout after all channels have timed out, the dynamically-defined
Queue Manager is deleted from the appliance
requestQueueName
Identifies the request queue
replyQueueName
Identifies the reply queue
Value
Direction
Front
Name
ReplyToQ
Value
QUEUE5
WebSphere MQ Interoperability
Value
Direction
Front
Name
ReplyToQM
Value
WASQM
You can also determine the routing of messages to front side MQ queues using the
MQMD header contained in the message to be delivered by the Front Side Protocol
Handler. The MQMD header is covered in detail in MQ Headers. You can set the
Reply-to queue and Queue Manager in this header. Note that if you use this
method, you should store a copy of the MQMD header (and other desired
MQ-related headers) in a context variable during the request processing phase,
using a stylesheet executed in a Transform action as part of a Request rule. You can
then retrieve the original request MQMD header and alter it or reinstate it as
needed in the response processing. See MQ Headers below for more information.
MQ Headers
Release 3.6.0.0 of the DataPower firmware introduces the ability to determine the
full set of MQMD header values. Here is an example MQMD request header
nodeset:
<MQMD>
<StrucId>MD </StrucId>
<Version>1</Version>
<Report>0</Report>
<MsgType>8</MsgType>
<Expiry>-1</Expiry>
<Feedback>0</Feedback>
<Encoding>546</Encoding>
<CodedCharSetId>437</CodedCharSetId>
<Format>MQSTR</Format>
<Priority>0</Priority>
<Persistence>0</Persistence>
<MsgId>414d51205741535f6970315f7365727667b65c450a920020</MsgId>
<CorrelId>000000000000000000000000000000000000000000000000</CorrelId>
<BackoutCount>0</BackoutCount>
<ReplyToQ>QUEUE5</ReplyToQ>
<ReplyToQMgr>WAS_ip1_server1</ReplyToQMgr>
<UserIdentifier>mqm</UserIdentifier>
<AccountingToken>1601000000000000000000b</AccountingToken>
<ApplIdentityData> </ApplIdentityData>
<PutApplType>11</PutApplType>
<PutApplName>les\IBM\MQ-test\rfhutilc.exe</PutApplName>
<PutDate>20061121</PutDate>
<PutTime>21452170</PutTime>
<ApplOriginData> </ApplOriginData>
<GroupId>000000000000000000000000000000000000000000000000</GroupId>
<MsgSeqNumber>1</MsgSeqNumber>
<Offset>0</Offset>
<MsgFlags>0</MsgFlags>
<OriginalLength>-1</OriginalLength>
</MQMD>
The device processes MQMD headers as follows as the message moves from the
request (front) side to the application (back) side, handling a request message.
1. The Front Side Protocol Handler parses the MQMD of the message into an
XML nodeset and stores it in the variable var://context/INPUT/_extension/
header/MQMD.
2. The Handler saves the MessageID, ReplyToQ and ReplyToQMgr header values
for later use in processing the response message.
3. The Handler can be configured to strip out unwanted headers, such as JMS,
CICS or IMS headers.
4. The Gateway can be configured to suppress the existing MQMD header in the
message. When this is done, the Gateway constructs a new header using the
default values for the backside connection.
5. The Gateway can be configured to inject a new MQMD header, replacing any
existing MQMD header.
6. The Gateway can use a custom stylesheet during policy processing to create or
alter an MQMD header. Examples are shown below.
The device processes MQMD headers as follows as the message moves from the
application (back) side to the request (front) side, handling a response message.
1. The Gateway can be configured to suppress or inject the header as detailed
above.
2. If the message is in response to a request, the device provides the CorrelID
corresponding to the MessageID provided no response rule alters the MQMD
header. The saved ReplyToQ and ReplyToQMgr information corresponding to
the request message are also provided for Front Side Reply routing.
You can discover which supported MQ-related headers are contained in a message
by reading the var://service/header-manifest variable. This returns a nodeset
containing the names of all supported headers in the message. You can then obtain
the values set for each header by using a dp:get-request-header() or
dp:get-response-header() function.
You can change or set the MQMD header by using the extension functions
dp:set-request-header() or dp:set-response-header().
You must create an MQMD header containing all desired elements whenever you
want to change or add just one. So, for example, if you wanted to change the
UserIdentifier header value sent to the back end queue, you would first need to
save the entire MQMD header nodeset, change the desired value and then rewrite
the MQMD header using the saved values.
Here is an example stylesheet that retrieves the MQMD header contained in a
client request to the front side of a Multi-Protocol Gateway. This stylesheet then
stores the key elements of the header in DataPower variables.
WebSphere MQ Interoperability
Here is an example stylesheet that uses the values taken from the request MQMD
header to construct a response header the DataPower device can use to route the
server reply to the front side client.
<?xml version="1.0" encoding="utf-8"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0"
xmlns:dp="http://www.datapower.com/extensions"
extension-element-prefixes="dp"
exclude-result-prefixes="dp">
<xsl:output method="xml"/>
<xsl:template match="/">
<!-- make sure these values are blank or they will override MQMD -->
<dp:set-response-header name="ReplyToQ" value=" "/>
<dp:set-response-header name="ReplyToQM" value=" "/>
<!-- create a variable to hold the new MQMD -->
<xsl:variable name="newMQMDStr">
<MQMD>
<CorrelId>
<xsl:value-of select="dp:variable(var://context/MYMQMD/MsgId)"/>
</CorrelId>
<ReplyToQ>
<xsl:value-of select="dp:variable(var://context/MYMQMD/ReplyToQ)"/>
</ReplyToQ>
<ReplyToQMgr>
<xsl:value-of select="dp:variable(var://context/MYMQMD/ReplyToQMgr)"/>
</ReplyToQMgr>
<Format>MQSTR</Format>
</MQMD>
</xsl:variable>
<!-- serialize the nodeset for transport -->
<xsl:variable name="mqmdStr">
<dp:serialize select="$newMQMDStr"/>
</xsl:variable>
<!-- set the MQMD of the response with new MQMD -->
<dp:set-response-header name="MQMD" value="substring-after($mqmdStr, ?>)"/>
<xsl:message>
<xsl:value-of select="concat (MQMD with modified Response ReplyToQ and
10
ReplyToQMgr : , dp:response-header(MQMD))"/>
</xsl:message>
</xsl:template>
</xsl:stylesheet>
Setting the complete MQMD header, using variables containing the Request
MQMD header, provides assured control of message routing. You can also create
the complete MQMD header sent to a back end application server's queue using
these functions.
MQ API support
Release 3.6.0.0 introduced support for more of the full MQ Client API through the
use of extension variables. For the user who is very familiar with the MQ
architecture and implementation details, this new capability might provide a
needed flexibility.
In general, these options should only be used by an MQ expert who has a specific
desire to set a certain option in an MQ call. These options only work for
connections created using the dynamic mq:// URI format explained in MQ URL
on page 6. The options are set using the same values as detailed in the MQ API
(available at http://publib.boulder.ibm.com/infocenter/wmqv6/v6r0/index.jsp).
For example, suppose the customer needs to send a message to a distribution list
(a distribution list is a list of local queues on a single Queue Manager).
This is done by defining an array of MQOR structures. Each structure needs the
ObjectName field set to one of the destination queues. The MQOD structure then
points to this list and sets a field indicating the length of the list. This only works
with version 2 of the MQOD structure. The MQOD structure is finally the input to
the MQPUT call. The requirement can be implemented on an DataPower service
by doing the following :
1. Create a Processing Policy with a rule matching the desired criteria.
2. Employ a Set Variable action that sets the var://context/INPUT/_extension/
header/MQOD variable to:
<MQOD><Version>2</Version><RecsPresent>3</RecsPresent></MQOD>
11
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
12
Error handling
A Multi-Protocol Gateway can employ all of the standard error handling
capabilities available for HTTP traffic. This includes Processing Policy Error rules,
the On-Error action, automatic fault generation when a request or response
message does not pass validation checks, and custom error message generation.
The Multi-Protocol Gateway property Process HTTP Errors affects how the
Gateway handles MQ error conditions. By default, this property is set to On,
causing the Gateway to pass any faults and errors generated by the back end MQ
system to the Gateway Processing Policy, as if no error had occurred. You must
then create a Response Rule in the Processing Policy capable of handling the error.
In this way, you can successfully complete transactions even though an error
occurred. If you do not create a Response rule to handle this condition, the device
will generate an automatic error message. To cause the Gateway to automatically
generate an error message in all error cases, set Process HTTP Errors to Off. You
may then optionally configure an Error Rule in the Processing Policy of the
Gateway to handle the error condition.
The device provides a number of MQ-specific event (error) codes that can be read
and used for error processing.
Select Administration from the navigation bar and under the Debug heading, click
View List of Event Codes.
These error codes can be determined by reading the X-dp-responsecode header. For
example, dp:get-response-header('X-dp-responsecode').
13
v The Proxy could employ an MQ Front Side Protocol Handler to get SOAP
request messages from an MQ queue, and optionally place any response on a
corresponding reply queue.
v The Proxy could employ a statically-defined backend that uses MQ queues to
put requests and get responses. As most WSDL files indicate an HTTP endpoint,
you might need to manually change the Proxy type property to static backend
from the typical default of static-from-wsdl. You will then need to specify the
backend URL using the MQ URL syntax.
v The Proxy could employ a Results (or Results Async) action in the Processing
Policy that sends messages to an MQ queue.
MQ and JMS
It is possible to transport JMS messages over an MQ system. In such a case, some
architectures allow for the selection of MQ messages based on values contained in
the JMS message headers.
The DataPower device views JMS messages over MQ as illustrated in Figure 3.
MQ Headers
MQ Body
JSM Headers
JMS Body
Until release 3.6.0.7, the DataPower device did not natively recognize JMS headers
as header values in the MQ message. In release 3.6.0.7, the DataPower device
recognizes and parses JMS headers in MQ messages.
It is possible to use the DataPower device to read and select messages based on
JMS header values regardless of release. However, the methods will differ.
For all releases, it is possible to select MQ messages based on JMS header values
by using a Binary Transform operation on the MQ message body to extract the
JMS header values into an XML nodeset. This extracted nodeset can then be
examined for the desired attributes.
For release 3.6.0.7 and later, it is possible to examine the nodeset contained in the
var://service/header-manifest variable, which contains the parsed JMS headers.
This includes headers in the formats
v MQMD
v MQRFH2
v MQRFH
v MQIIH
v MQCIH
14
v Others...
The MQMD.Format field value would be MQRFH2. A JMS header follows after the
standard MQMD header structure. The JMS header consists of the MQRFH2 structure
followed by a number of short XML documents. The system now adds these XML
documents to the header manifest as "X-MQRFH2-Data-xxxx" headers.
For example:
Table 4. Header Manifest
Header
Value
MQMD
<MQMD>...</MQMD>
MQRFH2
<MQRFH2>...</MQRFH2>
X-MQRFH2-Data-0
<mcd><Msd>jms_text</Msd></mcd>
X-MQRFH2-Data-1
<jms><Dst>queue:///Q2</
Dst><Tms>1164748514102</Tms></jms>
Units of Work
The MQ protocol includes the concept of Units of Work - that is, the ability to
remove messages from a queue only when the application that removed the
message has successfully processed the message. This concept also includes the
ability to roll back transactions, much like a database system. This section will
describe how MQ Units of Work function in general, and then describe how Units
of Work functions on the DataPower device. Finally, this section includes an
examination of several common usage patterns.
WebSphere MQ Interoperability
15
MQPUT operations. Because these operations take place on the same connection,
the queues are all managed by the same Queue Manager.
Here is a summary discussion of Units of Work:
1. A Unit of Work begins when an MQ Client specifies SYNCPOINT=True on the first
MQGET or MQPUT. The Unit of Work is committed using MQCMIT, or rolled
back using MQBACK, on the same connection.
2. When an MQ Client MQGETs a message from a queue within a Unit of Work,
that message remains on the queue but is not available to other clients.
3. The MQ Queue Manager permanently deletes the message from the queue
when the client issues an MQCMIT on the same connection. If the client issues
an MQBACK, the Queue Manager restores the message by making it available
to be retrieved by any client.
4. When a client MQPUTs a message on a queue within a Unit of Work, the
Queue Manager makes that message available to other applications only when
the client issues an MQCMIT on the same connection. If the client detects an
error, it can issue an MQBACK. The Queue Manager restores the queue by
removing the message that was put on the queue by that Unit of Work.
5. If the application failed before issuing MQCMIT/MQBACK, or the network
failed, the Unit of Work is rolled back by the Queue Manager. If the application
MQDISC before issuing MQCMIT/MQBACK, the Queue Manager will commit
the Unit of Work.
6. Clients can determine the number of times the same message has been
retrieved from a queue by examining the BackoutCount field in the MQMD
protocol header. This field contains the number of times the message has been
previously requested and subsequently backed out.
16
5.
6.
7.
8.
GMO of the MQGET for the reply. If the DataPower MQ Queue Manager object
handling the transmission of the request to the back end server has the Units
of Work property set to 1, and the MQPUT is issued with SYNCPOINT=true, the
MQPUT of the request message to the back end queue is immediately followed
by an MQCOMIT upon success. It is important to note that the back end
MQPUT typically goes to a different Queue Manager than the front, and even if
it is to the same Queue Manager as the front it would use a different
connection. Therefore, the Unit of Work for the back end connection is not the
same as the Unit of Work for the front side connection.
The Multi-Protocol Gateway retrieves a response from the back end server
using an MQGET issued to the queue specified by the ReplyToQueue of the
destination URL). Note that although the MQGET for reply message can be
issued with SYNCPOINT=true using GMO, there is no corresponding logic to
later issue an MQCMIT or MQBACK and hence should be avoided. Any
Response Rule in the Processing Policy matching the message executes,
possibly altering the message or the message headers.
When the Processing Policy completes, MQ Front Side Handler is signaled and
any response data from the backside is MQPUT to a front queue using one of
the following methods: 1. the Reply-to-Q in the MQMD of the reply message
(which may be altered during response processing; 2. the Reply-to-Q in MQMD
of the initial request message; and 3. the statically configured Put Queue in MQ
Front Side Handler which uses the same connection as the Get Queue of the
MQ Front Side Handler.
The MQ Front Side Handler issues an MQCOMIT on the same connection to
the Queue Manager handling the queue from which the Handler originally
retrieved the message when the Handler successfully places the response on the
correct Reply-to queue. Note that if the statically configured Put Queue is used
in #6 above, the MQCMIT will apply to both the initial MQGET on the Get
Queue and the MQPUT on the Put Queue. Any error that aborts the Processing
Policy processing or causes an Error rule to run will result in a MQBACK
issued instead of a MQCMIT.
When a message remains on a request (GET) queue after a failure, the MQ
Front Side Handler will again retrieve the same message. It is possible that this
loop could continue indefinitely (although many Queue Managers detect
messages left on a queue for a long time and remove them). It is possible to
configure the MQ Queue Manager object used by the MQ Front Side Handler
on the DataPower device to handle this case automatically. Set the Automatic
Backout flag to On and then set the BackoutThreshold and
BackoutQueueName properties of the Queue Manager to the appropriate
values. The MQ Front Side Handler using the Queue Manager tracks the
MQMD.BackoutCount header value of the MQGET of the request (GET) queue. If
the MQMD.BackoutCount reaches the configured backout threshold, the MQ Front
Side Handler will move (MQPUT) the message on the queue identified by the
BackoutQueueName property (typically the dead letter queue) and issue
MQCMIT to the request queue to break the loop.
17
to send a message from one MQ queue to another (fire) and make no attempt to
track the results or response (forget). The MQ messaging system can easily support
this pattern because MQ is asynchronous. A response to a request is not required,
unlike HTTP. Often, sites also require that this asynchronous (fire-and-forget)
pattern provide assured delivery of messages, with no loss of messages and no
duplication of messages during operation.
Many sites require transaction-style (synchronous) delivery, in which each request
must result in a response. The MQ system does provide some mechanisms to
support this delivery pattern, which the DataPower device uses (notably the
correlation of requests to responses using the Message ID of the request set to the
Correlation ID of the response).
This section discusses each of these two common delivery patterns in some detail.
These patterns apply to scenarios in which the DataPower device retrieves
messages from an MQ queue using an MQ Front Side Handler, and delivers
messages to a destination MQ queue on the back end. The following illustrates this
architecture:
Server Host
Server Host
Queue
Queue
Channel
Channel
Reply
DataPower
Reply
Request
Request
Error
Error
MQ Queue
Manager A
MQ Queue
Manager B
18
that this happens, the original front side request message will be restored by the
Queue Manager automatically (due to a lost connection) and processed by the
DataPower device again upon its recovery, resulting in duplicate delivery.
For guaranteed asynchronous delivery with no duplicates, both Multi-Protocol
Gateways can be configured to use loopback processing, thus omitting the back
side altogether. Instead, the Gateway uses the Front Side Handler to place
(MQPUT) the message on the Front Side Handler's PUT queue after all Processing
Policy actions for the request have completed. This is done by setting the
write-only service variable var://service/mpgw/skip-backside to 1(on) in the
MultiStep Policy. The configured Put Queue in the MQ Front Side Handler must
be used as it is opened on the same connection as the MQ Front Side Handler Get
Queue. The MQCMIT issued by the Gateway then commits both the MQGET and
MQPUT messages atomically.
This delivery pattern requires that the same MQ Queue Manager manages both the
queue containing request messages (typically the front side) and the queue
monitored by the back end server (typically the back side). This delivery pattern
also requires that the same MQ Queue Manager manages both the queue
containing back end response messages and the reply queue monitored by the
requesting application. The two Queue Managers (one for each direction) may be
different.
All the policy contexts and variables for each direction are separate and not
available to each other. It is not, for example, possible to correlate a request to the
back end server with the response using Message IDs and Correlation IDs.
Synchronous Delivery
MQ does provide some features to support synchronous (request/response
transaction type) delivery patterns. Requests can be correlated with responses, and
dynamic routing can be used to facilitate workflows. The DataPower device
provides such support and other additional features such as failover across
multiple queues and Queue Managers. Note that the use of these additional
features does conflict with the ability to use Units of Work to guard against
message duplication.
For a synchronous delivery pattern, a single Multi-Protocol Gateway processes
both the request and reply messages of the transaction. Here are the important
configuration parameters:
v The Units of Work property of the MQ Queue Manager object on the
DataPower device used by the Front Side Handler should be set to 1, enabling
Units of Work.
v The Process HTTP Errors property of the Gateway should be set to Off so that
back side errors will cause the front side to issue an MQBACK to the Queue
Manager.
v The Gateway's Processing Policy should include an error rule to handle failures
encountered during the transaction.
Using the reference diagram below, the following are the possible failure situations:
#1
#4
<--------- GET
PUT
GET
<--------- PUT
Request
Request --------->
Reply
--------->
Reply
#2
#3
WebSphere MQ Interoperability
19
20
Note that the use of two separate Multi-Protocol Gateways for a synchronous
delivery pattern is not recommended. The two separate Gateways cannot share
transactional data, such as Message ID and Correlation ID.
Front Side
The DataPower device will only commit the GET of a message from the Front Side
Protocol Handler's GET queue when the entire transaction completes successfully.
This does not change when the protocol used to connect to the back end
application server is not MQ.
As with an MQ back end, errors can be handled by controlling the Process HTTP
Errors and Response Type properties of the Gateway, plus any Timeout parameters
used in the back end URL.
Back Side
The MQ Units of Work affects only the placement (PUT) of a message on the
request queue of the back side server, and is not in any way affected by the
operations of a different protocol on the front side of the Gateway. The operation
of the front side may be affected by the methods used to handle errors
encountered when using Units of Work on the back side MQ connection. Errors
can be handled by controlling the Process HTTP Errors and Response Type
properties of the Gateway, plus any Timeout parameters used in the back end URL.
WebSphere MQ Interoperability
21
22
Notices
This information was developed for products and services offered in the U.S.A.
IBM may not offer the products, services, or features discussed in this document in
other countries. Consult your local IBM representative for information about the
products and services currently available in your area. Any reference to an IBM
product, program, or service is not intended to state or imply that only that IBM
product, program, or service may be used. Any functionally equivalent product,
program, or service that does not infringe any IBM intellectual property right may
be used instead. However, it is the users responsibility to evaluate and verify the
operation of any non-IBM product, program, or service.
IBM may have patents or pending patent applications covering subject matter
described in this document. The furnishing of this document does not grant you
any license to these patents. You can send license inquiries, in writing, to:
IBM Director of Licensing
IBM Corporation
North Castle Drive
Armonk, NY 10504-1785
U.S.A.
The following paragraph does not apply to the United Kingdom or any other
country where such provisions are inconsistent with local law: INTERNATIONAL
BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION AS IS
WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED,
INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR
PURPOSE. Some states do not allow disclaimer of express or implied warranties in
certain transactions, therefore, this statement may not apply to you.
This information could include technical inaccuracies or typographical errors.
Changes are periodically made to the information herein; these changes will be
incorporated in new editions of the publication. IBM may make improvements or
changes in the product(s) or the program(s) described in this publication at any
time without notice.
Trademarks
IBM, IBM logo, DataPower, DataPower device, WebSphere, and z/OS are
registered trademarks of the International Business Machines Corporation in the
United States or other countries.
Microsoft and Windows are trademarks of Microsoft Corporation in the United
States, other countries, or both.
Java and all Java-based trademarks and logos are trademarks or registered
trademarks of Sun Microsystems, Inc. in the United States and other countries.
Other company, product, and service names may be trademarks or service marks
of others.
23
24
Printed in USA