PRAGMATIC BUSINESS VALUE OF USING
SERVICE ORIENTATED ARCHITECTURE
AND IMPLEMENTING AN ENTERPRISE
SERVICE BUS
black
marble
the strategic IT asset for your organisation
Robert Hogg
FBCS, CEng
Senior Architect
Black Marble LTD
black
marble
Agenda
•
•
•
•
•
What is SOA?
Connecting Systems
Demo
Governance
Getting the Return
black
marble
black
marble
Key Business System Drivers
Businesses need:
• Flexibility
•
•
•
•
Agile Systems
Rapid Development Cycle
Extensibility
Collaboration – local/national/global
• Rigidity
•
•
•
•
black
marble
Reliability
Concrete
Guarantees and Risk Management
Compliance with governance
SOA Terminology
Two Definitions of SOA
• Service Oriented Architecture: a software design
approach in which key functions are built as reusable
components which implement industry standards for
interoperable communications.
• Service Oriented Architecture: enables loose coupling,
interoperability, management of change, and operation of
business services in a governable environment. Business
Services operating in a well run SOA can be composed
into business process that align IT with the business.
black
marble
State of SOA
• Perceptions
– SOA = IT centralization/standardization
– “SOA is dead” (Anne Thomas Manes, Burton Group; January 2009)
– Cloud computing will replace SOA
• Reality
–
–
–
–
SOA does not solve all enterprise IT issues
SOA is complex, and difficult
Core value of SOA is agility, but needs vary
SOA demands change, but organizational resistance is
often the biggest impediment
– Need to focus on architecture, not technology
black
marble
SOA Manifesto
Service orientation is a paradigm that frames what you do.
Service-oriented architecture (SOA) is a type of architecture
that results from applying service orientation.
Business value over technical strategy
Strategic goals over project-specific benefits
Intrinsic interoperability over custom integration
Shared services over specific-purpose implementations
Flexibility over optimization
Evolutionary refinement over pursuit of initial perfection
.
black
marble
The Benefits of SOA
• Align Business
and IT
• Increase Agility
• Reduce Costs
BUSINESS
BUSINESS
R&D
Focus on
document
Focus
exchange
on
Processes
service
Business
Business
model
model
Service model
Technology
model
Soft boundaries
(process
and services)
Technology
model IIS
Siebel
Outward
Inward
businesstechnology
driven
driven
viewview
black
marble
Requirements
Business
and solutions
siloed by IT
closely aligned
TECHNOLOGY
TECHNOLOGY
Services
Tandem
AS400
Solutions
Focus
impose
on
constraints
contract
Applications
Source: Dr Arvindra Sehmi
Common Myths about SOA
Myths
Facts
1. SOA is a technology
1. SOA is a design philosophy independent
of any product technology or industry
trend.
2. SOAs require web services
2. SOAs may be realized via web services
but using web services will not necessarily
result in a SOA.
3. SOA is new and revolutionary.
3. EDI, CORBA, and DCOM were
conceptual examples of SOA.
4. SOA ensures the alignment of IT and
business
4. SOA is not a methodology
5. A SOA Reference Architecture reduces
implementation risk
5. SOAs are like snowflakes, not two are
the same.
6. SOA requires a complete technology
and business process overhaul
black
marble
7. We need
to build a SOA
6. SOA should be incremental and built on
your current investments.
7. SOA is a means, not an end.
Common Myths about SOA #2
Myths
Facts
SOA begins at the
firewall
SOA begins at the
firewall
SOA Aligns the
Business With IT
People Align
Business With IT
SOA Governance
Fixes Everything
Governance needs to
span all of IT,
not just SOA
black
marble
CONNECTING SYSTEMS
black
marble
the strategic IT asset for your organisation
Connecting Application is Not Enough
Point-to-Point Integration Inhibits Agility
Java
Appl.
Service
• Point-to-Point
–
–
–
–
Complex Interface
Redundant Logic
Doesn’t Scale
Lacks Visibility
• Business Impact
– Delays Response
to changing
business needs
black
marble
.Net
Appl.
Service
SAP
Service
Oracle
Service
JD
Edwards
Service
MS
CRM
Service
CICS
Service
AS/400
Service
“End Point/Service Oriented Spaghetti”
Even a few dozens of services pose quite a plumbing challenge!
Guess how many Service Providers and
Consumers
black are in this Spaghetti?
marble
15 Service Providers
22 Service Consumers
Introduction to ESB
“A Web-services-capable infrastructure that
supports intelligently directed communication
and mediated relationships among loosely
coupled and decoupled biz components.”
- Gartner Group
black
marble
Enterprise Service Bus (ESB)
BizTalk was positioned as a Hub and spoke
Static Schema
Static Receive
Port made and
Most decisions
are
Hard Coded Map
Name
locked in at Design
time or at
Deployment not at Run time!
Any change is a re-development
or a system re-configuration!
We’ve allowed
the easy to use
Static Send Port
CRM
tools to define our architecture
Static Schema
black
marble
Now we’re saying it can be an Enterprise Service Bus?
We need to agree on what an Enterprise
Service Bus is and what it does.
•
•
•
•
•
Message Broker
Message Transformation
Message Validation
Adaption
Message-Oriented
Middleware
Service Orchestration
black
marble
ESB Core
• Configurable, reusable architecture
• Addresses common concerns generically - Routing,
Transformation, Exception Handling, Monitoring etc.
• Enforces Consistency and Unification
• Leverages modern Standards (WS-*), still supports
Legacy
• Central point for Service Monitoring and Administration
• Increases agility (through added Abstraction, Loose
coupling)
• Faster implementation times (through Configuration
vs. Coding)
black
marble
Agility thru Service Composition
Enterprise Service Bus
Service
Consumer
Service
Consumer
Service
Consumer
Location & Version
Transparency
Transport Protocol
Conversion
Data
Format Transformation
Invocation &
Orchestration
Error Handling
& Repair
Service
Provider
Service
Provider
black
marble
Message Interactions
Support
Service
Provider
Relevant Pattern Implementation
ESB Guidance
ESB Usage Patterns
Message
Routing
Message
Transformation
Message
Router
Perimeter
Service Router
Content
Enricher
Message
Broker
Content-Based
Router
Data Model
Transformation
Data Format
Transformation
Scatter-Gather
Recipient
List
Routing
Slip
Repair and
Resubmit
black
marble
?
Architectural
Design Patterns
Metadata
Centralization
Policy
Centralization
Rules
Centralization
Event-Driven
Messaging
Gateway
Reply
?
Forward
VETO/VETRO
Protocol
Bridge
Legacy
Wrapper
ESB Toolkit - New Abstraction Layer on Top of BizTalk
Declarative, Meta-data, Policy and Configuration –Driven.
1.
2.
3.
4.
Transform my message
Determine which endpoint I need
Route my message
Route the response to a second
service
5. Return the final result to me
Transform my message
Routing
Transform Service
Process
Orchestration
On Ramp
Service Consumers
Off Ramp
Protocol
Adaptation
Pub/Sub Service
Resolve a service end
point address for me
black
marble
End Point
Resolution
Service Providers
Microsoft ESB Guidance Architecture
Web Services
On-ramps
BizTalk
black
marble
Exception
Itinerary Services
Resolvers
Adapters
Management Portal
Off-ramps
BizTalk ESB Toolkit Architecture
ESB Management Portal
Core Web Services
Transformation Web
Service
Resolver Web Service
Provisioning Framework
ESB Toolkit Core
Itinerary Services
Exception Web Service
Operations Web Service
UDDI Web Service
Transform
Service
Generic SOAP
Receive
Pipeline
Generic WCF
Receive
Pipeline
Generic JMS
Receive
Pipeline
Custom
Receive
Custom
Pipeline
Custom
Service
Resolver-Adapter Provider Framework
Adapter Providers(…)
Exception Management Framework
Exception
Logger
On-Ramps
Alerts
Route
Service
Resolvers (…)
BizTalk Receive Ports
Reports
Fault
Processor
UDDI 3.0
Exception
Handler
BAM
Business Rules Engine
Exception
Management
Store
BizTalk Send Ports
Off-Ramps
Pipeline
Generic SOAP
Send
Pipeline
Generic WCF
Send
Pipeline
Generic JMS
Send
Custom
Pipeline
Custom
Send
Transformation Engine
Orchestration Engine
black
marble Itinerary
Store
BizTalk Pub/Sub
Engine
The Core ESB Toolkit Components
Itinerary
Services:
Message Processor
Itinerary
Mediation Policy
Resolvers:
Context Finder
black
marble
Adapter
Providers:
Adapter Properties
Itinerary Concept
• This is the heart of ESBG
– Allows us to implement the ESB pattern
as opposed to a Hub/Spoke pattern
• It provides the runtime flexibility that
BizTalk doesn’t have by default
black
marble
Itinerary Concept
Logical overview
Itinerary
Message body
Generic
OnRamps
Itinerary Mechanism
Itinerary Processing
Route
Route
Transfor
m
Messaging Base ESB
Services
Orchestration Based ESB
Services
MessageBox
black
marble
Transf
orm
Itinerary Cache
Three Ways to Assign
Itineraries
Advanced
Service
Client
Adaptive
Service
Client
 Service
Proxy
black
marble
Client sends an itinerary in a WCF/SOAP Header request
Client resolves itinerary via resolver service and then
sends it in a WCF/SOAP Header request
Itinerary is resolved on the server’s ESB
On-Ramp receiving port via configurable resolver
Itinerary Based Routing & Processing
The “heart” of the ESB Toolkit
• Light-weight service
sequencing &
composition
– Invokes itinerary
(internal) and external
services
– Dynamic service context
resolution at runtime
• Maps internal service
invocation to BizTalk
service containers
– pipelines in ports and
orchestrations.
black
marble
BizTalk Receive Port
On Ramp
Receive
Pipeline
Pub/Sub Engine
BizTalk Orchestration
Routing Service
ItineraryService1
Resolver
Resolver1
Resolver1
Adapter Provider
Pub/Sub Engine
BizTalk Dynamic Send Port
Pipeline
Off Ramp
Send
Itineraries Deployment
Design Time
Test/Run Time
Message
OnRamp
Itinerary
Itinerary
Selector
XML File
Itinerary
Repository
black
marble
BizTalk Deployment
Tools
Itinerary &
Message
Itinerary
Repository
ESB
Processing
Resolvers
For runtime flexibility services are not
hardcoded to specific endpoints or maps
Service end-point metadata can be
determined at runtime
Resolvers are called by services to set
an artifact (map , itinerary) or endpoint
metadata
black
marble
ESB Toolkit Resolver
• Can resolve:
–
–
–
–
–
Message delivery endpoints
Maps for transformations
Endpoint configuration
Custom service metadata
Itinerary to be associated with a message
• Endpoint/Map resolution with:
•
•
•
•
UDDI 2.0/3.0
XPATH
Static
Business Rules Engine (BRE)
• Itinerary resolution with:
• Itinerary (BRE)
• Itinerary Static
black
marble
Resolver Mechanism
Rules Engine
Service Registry
Static
SQL
Database
UDDI
BRE
XPath
WSMEX
A resolver configuration
string contained within the
itinerary is provided to the
resolver manager
ESB
Services
Resolver Manager
Resolver Mechanism
black
marble
The resolver mechanism is able
to load different resolvers
based on where the runtime
data is stored
ESB Resolvers
ESB Resolvers
Endpoint Resolution
Artifact Resolution
UDDI 2.0
Static
Static
BRE
XPATH
BRE
Itinerary - BRE
Itinerary - Static
UDDI 3.0
ESB Guidance V1
ESB Toolkit V2
black
marble
ESB Adapter Provider
• The following Adapter Providers supplied:
– WCF-BasicHTTP
– WCF-WsHTTP
– WCF-Custom
• WCF-SQL
–
–
–
–
FTP
WebSphere MQ
File
SMTP
• Custom
– any BizTalk adapter (that support dynamic ports) can be
exposed as an adapter provider
black
marble
ESB Adapter Providers
ESB Adapter Providers
FTP
WCF-BasicHTTP
WCF-Custom
WebSphere MQ
WCF-WsHTTP
SMTP
WCF-SQL
File
Custom
ESB Guidance V1
ESB Toolkit V2
black
marble
ESB Exception Management Portal
black
marble
DEMO
black
marble
GOVERNANCE
black
marble
SOA Governance Architecture
IIS
Web Service
On-ramp
ESB Core Services
Management
Point
Publish/Subscribe
Infrastructure
Off-ramp
Virtualize
d Service
Governance Server
Security Enforcement
Service Registry
Service Metrics Collection
SLA Enforcement
Governance Portal
black
marble
Service Configuration
Metrics Reporting
Metrics Reporting
Service Lifecycle Management
Physical
Service
ESB Governance Integration
• Integrated with governance solutions from
AmberPoint and SOA Software
• Specific functionality varies by governance
provider, but typically include:
– Service metrics
– Policy-driven security
– Policy-driven service-level agreement
– Alerting
black
marble
Defining Solutions As A Set Of Capabilities
Service Consumer
black
marble
1.
2.
3.
4.
Transform my message
Determine which endpoint I need
Route my message
Route the response to a second
service
5. Return the final result to me
WSDL Schema
Routing
Mapping Service
Process Orchestration
Protocol
Adaptation
Pub/Sub Service
End Point Resolution
WSDL Schema
GETTING THE RETURN
black
marble
the strategic IT asset for your organisation
Business Value
• Broaden availability of Information and
Capability
• Exception based processing
• Shorten business process cycles
• Regulatory compliance
• Alignment of systems with the business
black
marble
Limitations
• Not all Capabilities are modular
• Issues over Transfer of Control and Ownership
• Modelled process do not implement all activities, e.g.
Human intervention
• Difference between managing documents and
business processes
black
marble
Road to failure about for
Implementing SOA
• Typically a one or more of:
– Lack of Comprehension of Business needs.
– IT first Design.
– Poor Management.
– Poor implementation.
– No Need for SOA
black
marble
Key Lessons
• Critical to have a single shared vision by business and IT
• Enterprise logging and monitoring critical for finding
and resolving problems
• Apply Solid change management
–
Test Change Management before initial rollout
• Governance, Governance, Governance
• Consider an Agile rollout but do it well.
• After Rollout keep talking to the business and keep the
value of the investment visible by showing agility.
black
marble
Enterprise Service Bus - Summary
What it gives you…
Loosely coupled messaging environment
Policy driven message level routing and mediation
Itinerary based processing
Why you want it…
Higher levels of reuse
Lower operational costs
Dynamic business changes
Business and exception analytics
Business requirements in the drivers seat
black
marble
Where Next?
• Online content
– http://www.msdn.com/esb
– http://www.codeplex.com/esb
– http://www.soa-manifesto.org/
black
marble
How to Contact Us
01274 841305
www.blackmarble.com
[email protected]
@blackmarble
black
marble
APPENDIX
black
marble
the strategic IT asset for your organisation
Endpoint Metadata
Version Policy
Schema
Metadata
black
marble
UDDI
Repository
Endpoint A
Intelligent Routing
Endpoint C
black
marble
UDDI
Repository
Endpoint B
Endpoint A
Transformation
Endpoint C
black
marble
UDDI
Repository
Endpoint B
Endpoint A
Orchestration
Endpoint C
black
marble
UDDI
Repository
Endpoint D
Endpoint B
Endpoint A
Adaption
.NET Application
black
marble
Mainframe
Application
Web Service
Endpoint
Database
J2EE Application
Vertical Market
Application
Message Exchange Patterns
Synchronous
Service
black
marble
Publishing/Subscribe
Service
Service Consumer
Asynchronous
Service