Enterprise Integration

Erik Doernenburg ThoughtWorks, Inc. edoernen@thoughtworks.com

EMEA 2

Agenda
Challenges in enterprise application development Design patterns Enterprise Integration patterns Interop technologies

Conclusion & Resources

EMEA 3
Copyright ThoughtWorks, Inc. 2005

The challenge
Enterprises depend on a growing number of IT systems The systems must provide an integrated solution for the enterprise The systems must interoperate with each other

Architectural trends and waves of technologies Changing business needs and requirements

EMEA 4
Copyright ThoughtWorks, Inc. 2005

Finance Operator Application

fax/telex app email mail web

New System

document recognition

Existing Systems

ERP

ref. data

Workflow items are stored on Tibco RV/CM queue Tibco Integration Manager organises workflow handles imports from document recognition handles interaction with existing ERP systems Reference data imported using linked servers
EMEA 5
Copyright ThoughtWorks, Inc. 2005

Portfolio Management Tool

New application
Excel front-end

Front-end uses Office 2003 code-behind technology communicates with server using WebSphere MQ queues Message format defined with XSD schema Application uses existing services accesses analytics library with COM bridge accesses stored deals through deal manager service accesses service bus through messaging abstraction library
EMEA

application logic

technology adaptor

analytics library

deal manager

messaging library

Existing

Existing

Existing

Copyright ThoughtWorks, Inc. 2005

Design Patterns

EMEA 7

Why design patterns?

Code reuse remains difficult

but Knowledge reuse can be very valuable Patterns encapsulate knowledge of successful designs

EMEA 8
Copyright ThoughtWorks, Inc. 2005

Using technology successfully

Syntax Basic language mechanism A necessity but not a guarantee Constructs Classes, Interfaces, Inheritance, Polymorphism, etc. Helpful but not sufficient for good design Principles Separation of concerns, Dependency Injection, etc. Steer us towards a better solution

Patterns Model-View-Controller, Observer, Decorator Concrete design guidance

EMEA 9
Copyright ThoughtWorks, Inc. 2005

Patterns
Mind sized chunk of information (Ward Cunningham) Shows a good solution to a common problem within a specific context Observed from actual experience

Has a distinct name

Not copy-paste ThoughtWorks collaborates with Microsoft to document design patterns for the .NET platform
EMEA 10
Copyright ThoughtWorks, Inc. 2005

Enterprise Integration Patterns

Message Construction
Endpoint

Message Routing

Message Transformation
Endpoint

Message

Application A

Channel

Router

Translator

Application B

Messaging Endpoints

Messaging Channels

Monitoring

System Management

Gregor Hohpe defined a visual pattern language describing message-based enterprise integration solutions
Pattern language comprises 65 patterns in 6 categories
EMEA 12
Copyright ThoughtWorks, Inc. 2005

Pattern: Request-Response

Consumer

Request

Provider

Request Channel

Reply Channel

Response

Service Consumer and Provider (similar to RPC) Channels are unidirectional Two asynchronous point-to-point channels Separate request and response messages
EMEA 14
Copyright ThoughtWorks, Inc. 2005

Multiple consumers
Consumer 1
Requests Request Channel Requests

Provider

Reply Channel 1 Reply Channel 2

?
Responses

Consumer 2

Each consumer has its own reply queue But how does the provider send the response? Could send to all consumers (very inefficient) Hard code (violates principle of context-free service)
EMEA 15
Copyright ThoughtWorks, Inc. 2005

Pattern: Return Address

Consumer 1
Reply Channel 1 Reply Channel 2

Provider

Request Channel

Reply Channel 1 Reply Channel 2

Responses

Consumer 2

Consumer specifies Return Address (the reply channel) in the request message Service provider sends response message to specified channel
EMEA 16
Copyright ThoughtWorks, Inc. 2005

Load-balanced service providers

Consumer
Request Channel

Provider 1

Provider 2
Reply Channel

Request message can be handled by multiple providers Point-to-point channel supports competing services Only one service receives each request message But what if the response messages are out of order?
EMEA 17
Copyright ThoughtWorks, Inc. 2005

Pattern: Correlation Identifier

Consumer
1 2 Message Identifier Request Channel

Provider 1
1

Provider 2
2 1 Reply Channel 2 1 Correlation Identifier

Consumer assigns a unique identifier to each message Identifier can be an arbitrary ID, a GUID, a business key Provider copies the ID to the response message Consumer can match request and response
EMEA 18
Copyright ThoughtWorks, Inc. 2005

Multiple specialised providers

Order Messages

Widget Inv.

Order Entry

?
Gadget Inv.

Each provider can only handle a specific type of message Route the request to the "appropriate" provider. But how? Do not want to burden sender with decision Letting providers "pick out" messages requires coordination EMEA
20
Copyright ThoughtWorks, Inc. 2005

Pattern: Content-Based Router

Order Messages

Widget Inv. Gadget Inv.

Order Entry
Content Based Router

Insert a content-based router Routers forward incoming messages to different channels Message content not changed Mostly stateless, but can be stateful, e.g. de-duper
EMEA 21
Copyright ThoughtWorks, Inc. 2005

Composite messages

Order Message

Widget Inv.

Order Entry

?
Gadget Inv.

How can we process a message that contains multiple elements?

EMEA 22
Copyright ThoughtWorks, Inc. 2005

Pattern: Splitter & Router

Order Item 1

Order Message

Order Items

Widget Inv. Gadget Inv.

Order Item 2

Order Entry
Splitter Router

Use a splitter to break out the composite message into a series of individual messages Then use a router to route the individual messages as before Note that two patterns are composed
EMEA 23
Copyright ThoughtWorks, Inc. 2005

Producing a single response

Order Item 1

Response 1

Widget Inv.

Confirmed Order

?
Gadget Inv.
Order Item 2 Response 2

Billing

How to combine the results of individual but related messages? Messages can be out-of-order, delayed Multiple conversations can be intermixed
EMEA 24
Copyright ThoughtWorks, Inc. 2005

Pattern: Aggregator
Order Item 1

Response 1

Widget Inv. Gadget Inv.

Order Item 2 Response 2

Confirmed Order

Billing
Aggregator

Use a stateful filter, an Aggregator Collects and stores messages until a complete set has been received (completeness condition) Publishes a single message created from the individual messages (aggregation algorithm)
EMEA 25
Copyright ThoughtWorks, Inc. 2005

Communicating with multiple parties

Vendor A Vendor B
Request For Quote Quotes

?
Aggregator

Vendor C

Best Quote

How to send a message to a dynamic set of recipients? And return a single response message?

EMEA 26
Copyright ThoughtWorks, Inc. 2005

Pattern: Scatter-Gather
Pub-Sub channel

Vendor A Vendor B

Quotes

Request For Quote

Vendor C

Best Quote

Aggregator

Send message to a pub-sub channel Interested recipients subscribe to a "topic" Aggregator collects individual response messages may not wait for all quotes, only returns one quote
EMEA 27
Copyright ThoughtWorks, Inc. 2005

Complex composition
Pub-Sub channel

Vendor A Vendor B

Quotes

Splitter

Quote request for each item

Vendor C

Aggregator

Best Quote for each item

Aggregator

Receive an order message Use splitter to create one message per item Send to scatter/gather which returns "best quote" message Aggregate to create quoted order message
EMEA 28
Copyright ThoughtWorks, Inc. 2005

Interop technologies

EMEA 29

Major interop technologies

Resource based
DB files

RPC style / distributed objects

RMI-IIOP (CORBA) Remoting (DCOM) in-process bridge WS

Message based / document-oriented

MOM WS WS-*

EMEA 30
Copyright ThoughtWorks, Inc. 2005

Interop technology characteristics

platform neutral J2EE server .NET server in-proc

DB/files

MOM

WS-*

WS

RMI-IIOP (CORBA)

Remoting (DCOM)

bridge

point-to-point routable

transient messages
durable message

clustering
server lifetimemanagement

EMEA 31
Copyright ThoughtWorks, Inc. 2005

Messaging
Channels are separate from applications Removes location dependencies Channels are asynchronous & reliable Removes temporal dependencies Data is exchanged in self-contained messages Removes data format dependencies Loosely coupled integration enables independent variation
EMEA 32
Copyright ThoughtWorks, Inc. 2005

Why Web Services?

Web services provide all benefits of messaging solution loose-coupling service oriented reliable communication Why web services? composable protocol vendor neutral suitable for access through Internet

EMEA 33
Copyright ThoughtWorks, Inc. 2005

Web Services development

Web Services are expected to become the default Messaging solution in the future WS-* standards are evolving rapidly, most important WS-Security WS-Policy WS-Addressing WS-I defines profiles, sample applications and testing tools Profiles are guidelines for using WS specifications Use Basic profile 1.1 to build interoperable solutions Further research on WS is being carried out
EMEA 34
Copyright ThoughtWorks, Inc. 2005

Conclusion
Enterprise systems are becoming more complex

We have patterns to help us with the design

We have technologies to implement our designs

Building for interoperability and integration is key

EMEA 36
Copyright ThoughtWorks, Inc. 2005

Recommended books
Enterprise Integration Patterns Gregor Hohpe, Bobby Woolf Addison-Wesley, 2004 Integration Patterns Microsoft Patterns & Practices 2004 Patterns of Enterprise Application Architecture Martin Fowler Addison-Wesley, 2003 Enterprise Solution Patterns using Microsoft .NET Microsoft Patterns & Practices 2003 Enterprise SOA Dirk Krafzig, Karl Banke, Dirk Slama Prentice Hall, 2004
EMEA 37
Copyright ThoughtWorks, Inc. 2005

Please complete your session feedback form

THANK YOU