How to build ESP (Event Stream Processing)
and BAM (Business Activity Monitoring) into
your Application
Pat Bonser
Senior Manager, Product Readiness
Agenda
Interactive Introduction to Apama Scenario Creation
Definitions
• What is ESP (Event Stream Processing)?
• What is BAM (Business Activity Monitoring)?
Apama platform
• Architecture & components
Building an example application
• Revenue Assurance
Conclusions
How to build ESP and BAM into your Application
2
© 2006 Progress Software Corporation
Event Stream Processing
A New Computing Physics
%
%
%
%
%
%
$
$
&
& '
'
(
(
!
How to build ESP and BAM into your Application
3
"
#
© 2006 Progress Software Corporation
ESP & BAM
A New Computing Physics
)
)
*
*
-.
.
.
.
+
+,
,
+
*
,
+
*
,+
+,
,
/
/ 0
0.
.
%
%
.
.
!
How to build ESP and BAM into your Application
4
"
#
© 2006 Progress Software Corporation
Example Event Driven Applications
Monitor, Analyze, and Act on Business Conditions in Real Time
Application
Customer
Service
Event Examples
Customer requests,
CRM records
Event-Driven Analysis & Action
Alert me if an open customer request with priority
> 3 did not get any reply event for at least 2 hours
Alert me if an open customer request was
reassigned more than 3 times
When (Warehouse 1 capacity < 50%) and
(Warehouse 2 capacity < 30%) and
(Warehouse 3 capacity < 25%)
Alert the head of operations with the average
capacity in each of the warehouses
Supply Chain
Inventory system,
warehouse deliveries,
warehouse restock
RFID/Supply
Chain
RFID reader detecting
When pallet is loaded onto the wrong truck alert
EPC codes, goods
warehouse manager
delivery schedule
Automated
Trading
Stock tick & quote
data
Within any 20 second window, when HP rises by
more than 2%, but IBM doesn’t, buy IBM
Smartcard ID read
When the rider travels more than 5 times within
zone 1, charge at single-ride rate for subsequent
trips
Transportation
How to build ESP and BAM into your Application
5
© 2006 Progress Software Corporation
What do events look like?
Events are typed messages describing a change
within a system
• Example:
– NewInvoice(customer, product, amount)
– StockTick(symbol, price, volume)
– TruckLocation(truck, driver, cargo, x,y,z)
Events are analogous to a new entry in a database
table or a Sonic message
Detecting patterns in individual events or a
correlation of several events over time can indicate
an opportunity or threat to the business
How to build ESP and BAM into your Application
6
© 2006 Progress Software Corporation
Business Benefits of ESP
Monitor, Analyze, ACT
• The ability to take an action immediately when a business
scenario is detected
– Enables your application to respond instantly to opportunity or threat
It’s easy to add powerful capabilities to your existing application
• You already have the events flowing in your system – you just
need to send them into Apama
Add real-time rules to your application without continually changing
your code
• You don’t have to go on extending your application
• Once the events are flowing into Apama, simply add or modify a
rule in Apama
• Rules can be defined by business or technical users
– Codeless and code based development
How to build ESP and BAM into your Application
7
© 2006 Progress Software Corporation
Agenda
Interactive Introduction to Apama Scenario Creation
Definitions
• What is ESP (Event Stream Processing)?
• What is BAM (Business Activity Monitoring)?
Apama platform
• Architecture & components
Building an example application
• Revenue Assurance
Conclusions
How to build ESP and BAM into your Application
8
© 2006 Progress Software Corporation
Apama Product Suite
Developer Studio
Dashboards
Dashboard
Studio
Scenario
Modeler
Event
Manager
Research Studio
Apama
IDE
Event
Store
Integration Adapter Framework
How to build ESP and BAM into your Application
9
© 2006 Progress Software Corporation
Inside the Event Manager
Feedback
Event
Registration
Events
Event
Input
Queue
Multidimensional
event
matcher
Composite
event
sequencer
MonitorScript
Virtual Machine
Java
Virtual Machine
Match
Notification
Event
Output Events
Queue
C/C++
Scenarios
How to build ESP and BAM into your Application
10 © 2006 Progress Software Corporation
Monitorscript
event SMS-Request {
string user;
string service;
location loc;
}
monitor RingtoneServiceMonitor {
SMS-Request request;
action onload {
on all SMS-Request (service=“Ringtone"):request {
on SMS-Request (user = request.user) within(60.0) {
}
}
}
}
emit UpdateBilling(request.user, “Double Discount”);
How to build ESP and BAM into your Application
11 © 2006 Progress Software Corporation
JMON: Monitors in Java
import com.apama.jmon.*;
public class BillRequestService implements Monitor,
MatchListener {
public void onLoad() {
EventExpression eventexpr =
new EventExpression(“SMSRequest(service=\“Billing\”)”);
eventexpr.addMatchListener(this);
}
public void match(MatchEvent event) {
System.out.println(“Billing Information Requested”);
}
}
How to build ESP and BAM into your Application
12 © 2006 Progress Software Corporation
Apama IDE
How to build ESP and BAM into your Application
13 © 2006 Progress Software Corporation
The Apama Scenario Modeler
Enables Graphical Construction of ESP Scenarios
Express time-based realtime rules with a high level
development tool
Each scenario, or group of rules, represents a
“pattern” which can be adjusted by business
users to specify conditions to monitor, analyze
and act on.
Intuitive
visual user
interface
designed for
business
analysts
“SmartBlocks”
encapsulate prepackaged
modules made
available to nonprogrammers.
How to build ESP and BAM into your Application
14 © 2006 Progress Software Corporation
Apama Dashboard Studio
Create Customized Dashboards for Your Real-Time Business Processes
Select from a palette of
graphical objects. Each
object can be laid out in a
graphical dashboard and
bound to Apama events
•
Real-time variables and
analytics can be
visualized using graphs,
charts, tables etc.
•
Enables event-driven
Apama logic to be
visualized in real-time
•
Deployment options from
standalone dashboard to
thin client portal
Users can customize the look and
feel of all widgets, and specify
which Apama event scenarios to
visualize.
How to build ESP and BAM into your Application
15 © 2006 Progress Software Corporation
Event Store - Capture
1.
2
*
•
Event Store captures
all events in time
series order
•
Event Store is also
able to capture
derived events, e.g.
Moving Average,
from Event Manager
•
Event Store can
capture all output
from Event Manager
as an audit trail
Real-Time Event Processing
1.
EventStore
Historical Event Processing
How to build ESP and BAM into your Application
16 © 2006 Progress Software Corporation
Event Store – Replay & Backtesting
1.
3
)
1.
2
*
4
5
%
%/
•
Enables “pre-flight
testing” of Apama
scenarios with historical
data sequences
•
Output of runs can be
captured in Event Store
and charted
•
Also enables “digital
forensics” – using
recorded information to
tune future performance
Real-Time Event Processing
1.
EventStore
Historical Event Processing
How to build ESP and BAM into your Application
17 © 2006 Progress Software Corporation
Integration Adapter Framework
Event
Manager
Purpose of an adapter
• Receive an event stream or
create an event stream
– Intercept transactions
– Use database triggers
– Subscribe to Sonic
messages
Apama
Events
Mapping
Mapping
Normalization
Normalization
Transport
Integration
Transport
Integration
• Normalize to Apama event
format for efficient analysis Publish/
subscribe
& response
• Convert Apama events
Sonic Bus
back to actionable messages
API
OpenEdge
Native Sonic
Messages
How to build ESP and BAM into your Application
18 © 2006 Progress Software Corporation
Agenda
Interactive Introduction to Apama Scenario Creation
Definitions
• What is ESP (Event Stream Processing)?
• What is BAM (Business Activity Monitoring)?
Apama platform
• Architecture & components
Building an example application
• Revenue Assurance
Conclusions
How to build ESP and BAM into your Application
19 © 2006 Progress Software Corporation
Building an Example Application
Case Study: Revenue Assurance
• Ensuring that all service requests are both fulfilled and
billed correctly and within a timely manner
• Monitor via complex KPIs and present an graphical
overview of the status
• Alert relevant staff with specific problems
• Undertake Autonomous Actions: modify the Service
QoS settings to resolve problems
Same underlying principles are valid for tracking & billing
in Supply Chain, ERP, Logistics etc.
How to build ESP and BAM into your Application
20 © 2006 Progress Software Corporation
A (very) simplified revenue chain
1
2
3
5
4
Process
a) User requests a service (1 + 2)
b) Service is fulfilled (4 + 5)
c) Request is Billed (3)
Problem
• Any message (1-5) may be lost, corrupt or delayed
• Depends on the throughput and the QoS
Parameters (Complex-KPIs) that we calculate:
• Revenue leakage: requests that are not billed
• Overcharging: requests that are billed but not fulfilled
How to build ESP and BAM into your Application
21 © 2006 Progress Software Corporation
Our revenue assurance demo package
Dashboard
#*
#
#+
Scenario
$
,
$
#
!
#
$
# %&
&
"
,- "
"
'' "(
! )
Smart blocks
!
1
"" "
2
3
5
How to build ESP and BAM into your Application
4
22 © 2006 Progress Software Corporation
Demonstration
How to build ESP and BAM into your Application
23 © 2006 Progress Software Corporation
Agenda
Interactive Introduction to Apama Scenario Creation
Definitions
• What is ESP (Event Stream Processing)?
• What is BAM (Business Activity Monitoring)?
Apama platform
• Architecture & components
Building an example application
• Revenue Assurance
Conclusions
How to build ESP and BAM into your Application
24 © 2006 Progress Software Corporation
Conclusions
ESP & BAM enables
• Rules to be defined external to the application
to monitor, alert & respond to key business
opportunities & threats
• Vizualization of key business events on real-time
dashboards
In many cases events are already flowing through
existing applications
Apama enables
• Easy connection to existing applications
• Graphical ESP rules definition for rapid development
• Graphical dashboard creation
How to build ESP and BAM into your Application
25 © 2006 Progress Software Corporation
Questions?
How to build ESP and BAM into your Application
26 © 2006 Progress Software Corporation
Thank you for
your time
How to build ESP and BAM into your Application
27 © 2006 Progress Software Corporation
How to build ESP and BAM into your Application
28 © 2006 Progress Software Corporation