Analysera strömmande data med Event Stream Processing
SAS Forum
Börje Edlund
Chief Architect
[email protected]
Twitter @BorjeEdlund
Linkedin: Börje Edlund
Copyright © 2015, SAS Institute Inc. All rights reserved.
SAS® EVENT
STREAM
PROCESSING
ENGINE
C op yr i g h t © 2 0 1 3 , S A S I n s t i t u t e I n c . A l l r i g h t s r es er v e d .
• Behovet av Event Stream Processing
• Användningsexempel
• Översikt SAS® Event Stream Processing
Engine och Streaming Analytics
• Demonstration
BIG DATA
FRÅN STORE IT -> SCORE IT TILL SCORE IT -> STORE IT
INTRESSET SKIFTAR TILL STREAMING ANALYTICS
FÖR ATT AGERA och FATTA BESLUT I TID
Internet – Sociala Media
Kundinteraktioner
Banktransaktioner
Telecommunikation
Sensorer
Intelligenta enheter «Internet of Things »
….och mycket mer
C op yr i g h t © 2 0 1 5 , S A S I n s t i t u t e I n c . A l l r i g h t s r es er v e d .
ANVÄNDNINGSOMRÅDEN
Exempel på områden för ESP
Tre korta kundexempel
Copyright © 2015, SAS Institute Inc. All rights reserved.
EXEMPEL PÅ OMRÅDEN
ECOMMERCE OPTIMISATION
•
•
•
Clickstream Analysis
Optimize user experience
Real time marketing and
advertising
FRAUD DETECTION
•
•
•
•
CONNECTED DEVICES (IoT)
•
•
•
•
Real time sensors survey
RT anomalies detection
RT triggering and decision
Predictive maintenance
TELECOMUNICATIONS
•
•
•
•
DECISION MANAGEMENT
•
•
•
Complement SAS EDM
RT decisions on event streams
Deploy additive and incremental
models
Copyright © 2015, SAS Institute Inc. All rights reserved.
Real time transaction analysis
User behavior detection
Customer profile correlation
RT alerts and case management
CDRs analysis
Real Time Marketing
Fraud detection
IT systems survey
CAPITAL MARKETS
•
•
•
Complement SAS HP Risk
Reduce time from trading to
reporting
Continuous calculations
“Connected Truck”
FORTSÄTT RULLA!
MÅL
Förutsäga underhållsbehov för enskilda lastbilar innan fel
uppstår
• Proaktivt serva lastbilar vid lämplig tidpunkt
• Erbjuda kunder ny typ av nivå på service plan
Process:
• Data från 60+ sensors/truck.
• Integrerat med produkt detaljer,
garanti händelser, och andra data
för “context”.
• Analytiska modeller för att
förutsäga sannolikheten för
specifikt fel inom 30 dagar.
Resultat
•
Modeller förutsäger sannolika fel 30 dagar före
med 90% tillförlitlighet
•
Förbättrad insikt i vad som orsakar fel leder
till ökat produktivitet
Image credit: Mike, https://www.flickr.com/people/pmiaki/
Copyright © 2015, SAS Institute Inc. All rights reserved.
UNDVIKA AVBROTT I KRITISKA KOMPONENTER
Olje &
Gasbolag
UTMANING
•
Monitorera Elektroniska Sänkbara Pumpars (ESP)
effektivitet och riggar djuphavsborrning
•
En trasig pump är $2M/dag; en dags förlorad produktivitet
är $20M föresenad intäkt
Effekter
Över 2.1 Millioner sensorer
3 miljard rader data per minut
•
Millisekundsnabb automatiserad upptäckt av spektrala
prestanda och sensorers datakvalitet
monitoredade för potentiella
fel (temperature, vibration, ..)
•
Analys i dataströmmen för att effektivt undvika problem
•
Löser fler händelsedrivna problem och snabbare än förut
prediktiva modeller för att
detektera mönster som
indikerar fel
Copyright © 2013, SAS Institute Inc. All rights reserved.
Europeiskt
Telebolag
BESLUT OM ERBJUDANDE INOM DELAR AV EN SEKUND
När hastigheten spelar roll
Så är tid pengar
Events coming in
20,000 requests/sec
Snabbt urval av intressanta
händelser
För sen reaktion och
reaktionen tappar sitt
värde!
SAS® REAL TIME
DECISION MANAGER
10x Förbättring av svar på
erbjudande
Latency
100-250 ms
C op yr i g h t © 2 0 1 2 , S A S I n s t i t u t e I n c . A l l r i g h t s r es er v e d .
ESP
Image: https://www.flickr.com/photos/spielbrick/
Relevant events
15 requests/sec
Latency
5-15 ms
SAS® EVENT STREAM PROCESSING
ENGINE OCH STREAMING ANALYTICS
Hur fungerar det?
Copyright © 2015, SAS Institute Inc. All rights reserved.
VAD KARAKTERISERAR SAS® EVENT STREAM PROCESSING
ENTERPRISE CLASS
PERFORMANCE
SAS® Event Stream Processing
provides seamless and flexible
integration with existing IT
architecture components in addition
to high availability and guaranteed
delivery
SAS® Event Stream Processing can
process huge volumes of streaming
data flowing at very high rates
(Millions of events/sec) with very low
latency (<1 millisec)
FLEXIBILITY
SAS® Event Stream Processing
provides flexible modeling
interfaces, including the SAS®
Event Stream Processing Studio
to enable new analysis &
processing models to be quickly
developed and modified
STREAMING ANALYTICS
Copyright © 2015, SAS Institute Inc. All rights reserved.
SAS® Event Stream Processing
leverages top of class SAS Analytics
capabilities including Text Analytics
and Data Quality to provides the
most accurate and efficient pattern
detection
Event Stream Processing (ESP) vs. relational Database Management System (RDBMS)
EVENTS
C op yr i g h t © 2 0 1 2 , S A S I n s t i t u t e I n c . A l l r i g h t s r es er v e d .
ESP
RDBMS
ESPs store the queries
and continuously
stream data
through the queries
Databases store
the data
and periodically
run queries
against the stored data
INCREMENTAL
RESULTS
QUERIES
RESULTS
Sas event stream
processing
LOGICAL MODELING FLOW
Processes
Data Stores
LASR Server
Streaming Events
SAS-generated
Insights
Enrichment Analytic
Data
Rules
Business
Models
Copyright © 2015, SAS Institute Inc. All rights reserved.
Event Stream Processing Studio
Copyright © 2015, SAS Institute Inc. All rights reserved.
ESP STREAM VIEWER
Out-of-the-box event streams
visualization for
ESP model designers
Display output event streams
HTML5 Web interface
Table and Graphs
Streaming and update modes
Manual publishing of events
Copyright © 2015, SAS Institute Inc. All rights reserved.
Broker
Surveillance
POST-TRADE ORDER PRACTICE & COMPLIANCE
ALERTING
Event Stream
Processing Server
Event Stream
Sources/Publishers
Trades
Market Feed
Trades
Trades of
Large Size
(filter)
Broker Alerts: Broker aggregates for each alert type and total.
Front-running: broker buys securities for his own account before
buying the same securities for his customer, then sells when the
price rises; or broker sells securities out of his personal accounts
prior to selling the same securities for his clients.
Frontrunning
Patterns
Restricted Sale: sales of securities that have ownership restrictions.
(procedural)
Brokers
of Interest
Trades of
Interest
Restricted
Sales
(join)
(join)
FrontRunning
Alert
Case Management GUI
Restricted
Sale Alert
Brokers
Restricted
Securities
Open/Close
Mark Alert
Marking
Open/Close
Patterns
(procedural)
Restricted Securities
Broker
Alerts
Venues
Trades
Venue Trade Windows
Broker Alerts
(aggregate)
Marking the Close: attempting to influence the closing price of a
security by executing purchases at the close of normal trading hours.
Marking the Open: attempting to influence the opening price of a
security by making trades at the opening of normal trading hours.
Copyright © 2015, SAS Institute Inc. All rights reserved.
SAS® EVENT STREAM PROCESSING
ENGINE OCH STREAMING ANALYTICS
Kort Demonstration
Copyright © 2015, SAS Institute Inc. All rights reserved.
Analysera strömmande data med Event Stream Processing
Några länkar:
The connected truck
SAS Forum
https://m.youtube.com/watch?v=SkU8dmrWIPE
SAS Event Stream processing demonstration
https://m.youtube.com/watch?v=3xW-hUDsRIQ
Börje Edlund
Chief Architect
[email protected]
Twitter @BorjeEdlund
Linkedin: Börje Edlund
Copyright © 2015, SAS Institute Inc. All rights reserved.