Analysera strömmande data med Event Stream Processing
Analysera strömmande data med Event Stream Processing SAS Forum Börje Edlund Chief Architect [email protected] Twitter @BorjeEdlund Linkedin: Börje Edlund Copyright © 2015, SAS Institute Inc. All rights reserved. SAS® EVENT STREAM PROCESSING ENGINE C op yr i g h t © 2 0 1 3 , S A S I n s t i t u t e I n c . A l l r i g h t s r es er v e d . • Behovet av Event Stream Processing • Användningsexempel • Översikt SAS® Event Stream Processing Engine och Streaming Analytics • Demonstration BIG DATA FRÅN STORE IT -> SCORE IT TILL SCORE IT -> STORE IT INTRESSET SKIFTAR TILL STREAMING ANALYTICS FÖR ATT AGERA och FATTA BESLUT I TID Internet – Sociala Media Kundinteraktioner Banktransaktioner Telecommunikation Sensorer Intelligenta enheter «Internet of Things » ….och mycket mer C op yr i g h t © 2 0 1 5 , S A S I n s t i t u t e I n c . A l l r i g h t s r es er v e d . ANVÄNDNINGSOMRÅDEN Exempel på områden för ESP Tre korta kundexempel Copyright © 2015, SAS Institute Inc. All rights reserved. EXEMPEL PÅ OMRÅDEN ECOMMERCE OPTIMISATION • • • Clickstream Analysis Optimize user experience Real time marketing and advertising FRAUD DETECTION • • • • CONNECTED DEVICES (IoT) • • • • Real time sensors survey RT anomalies detection RT triggering and decision Predictive maintenance TELECOMUNICATIONS • • • • DECISION MANAGEMENT • • • Complement SAS EDM RT decisions on event streams Deploy additive and incremental models Copyright © 2015, SAS Institute Inc. All rights reserved. Real time transaction analysis User behavior detection Customer profile correlation RT alerts and case management CDRs analysis Real Time Marketing Fraud detection IT systems survey CAPITAL MARKETS • • • Complement SAS HP Risk Reduce time from trading to reporting Continuous calculations “Connected Truck” FORTSÄTT RULLA! MÅL Förutsäga underhållsbehov för enskilda lastbilar innan fel uppstår • Proaktivt serva lastbilar vid lämplig tidpunkt • Erbjuda kunder ny typ av nivå på service plan Process: • Data från 60+ sensors/truck. • Integrerat med produkt detaljer, garanti händelser, och andra data för “context”. • Analytiska modeller för att förutsäga sannolikheten för specifikt fel inom 30 dagar. Resultat • Modeller förutsäger sannolika fel 30 dagar före med 90% tillförlitlighet • Förbättrad insikt i vad som orsakar fel leder till ökat produktivitet Image credit: Mike, https://www.flickr.com/people/pmiaki/ Copyright © 2015, SAS Institute Inc. All rights reserved. UNDVIKA AVBROTT I KRITISKA KOMPONENTER Olje & Gasbolag UTMANING • Monitorera Elektroniska Sänkbara Pumpars (ESP) effektivitet och riggar djuphavsborrning • En trasig pump är $2M/dag; en dags förlorad produktivitet är $20M föresenad intäkt Effekter Över 2.1 Millioner sensorer 3 miljard rader data per minut • Millisekundsnabb automatiserad upptäckt av spektrala prestanda och sensorers datakvalitet monitoredade för potentiella fel (temperature, vibration, ..) • Analys i dataströmmen för att effektivt undvika problem • Löser fler händelsedrivna problem och snabbare än förut prediktiva modeller för att detektera mönster som indikerar fel Copyright © 2013, SAS Institute Inc. All rights reserved. Europeiskt Telebolag BESLUT OM ERBJUDANDE INOM DELAR AV EN SEKUND När hastigheten spelar roll Så är tid pengar Events coming in 20,000 requests/sec Snabbt urval av intressanta händelser För sen reaktion och reaktionen tappar sitt värde! SAS® REAL TIME DECISION MANAGER 10x Förbättring av svar på erbjudande Latency 100-250 ms C op yr i g h t © 2 0 1 2 , S A S I n s t i t u t e I n c . A l l r i g h t s r es er v e d . ESP Image: https://www.flickr.com/photos/spielbrick/ Relevant events 15 requests/sec Latency 5-15 ms SAS® EVENT STREAM PROCESSING ENGINE OCH STREAMING ANALYTICS Hur fungerar det? Copyright © 2015, SAS Institute Inc. All rights reserved. VAD KARAKTERISERAR SAS® EVENT STREAM PROCESSING ENTERPRISE CLASS PERFORMANCE SAS® Event Stream Processing provides seamless and flexible integration with existing IT architecture components in addition to high availability and guaranteed delivery SAS® Event Stream Processing can process huge volumes of streaming data flowing at very high rates (Millions of events/sec) with very low latency (<1 millisec) FLEXIBILITY SAS® Event Stream Processing provides flexible modeling interfaces, including the SAS® Event Stream Processing Studio to enable new analysis & processing models to be quickly developed and modified STREAMING ANALYTICS Copyright © 2015, SAS Institute Inc. All rights reserved. SAS® Event Stream Processing leverages top of class SAS Analytics capabilities including Text Analytics and Data Quality to provides the most accurate and efficient pattern detection Event Stream Processing (ESP) vs. relational Database Management System (RDBMS) EVENTS C op yr i g h t © 2 0 1 2 , S A S I n s t i t u t e I n c . A l l r i g h t s r es er v e d . ESP RDBMS ESPs store the queries and continuously stream data through the queries Databases store the data and periodically run queries against the stored data INCREMENTAL RESULTS QUERIES RESULTS Sas event stream processing LOGICAL MODELING FLOW Processes Data Stores LASR Server Streaming Events SAS-generated Insights Enrichment Analytic Data Rules Business Models Copyright © 2015, SAS Institute Inc. All rights reserved. Event Stream Processing Studio Copyright © 2015, SAS Institute Inc. All rights reserved. ESP STREAM VIEWER Out-of-the-box event streams visualization for ESP model designers Display output event streams HTML5 Web interface Table and Graphs Streaming and update modes Manual publishing of events Copyright © 2015, SAS Institute Inc. All rights reserved. Broker Surveillance POST-TRADE ORDER PRACTICE & COMPLIANCE ALERTING Event Stream Processing Server Event Stream Sources/Publishers Trades Market Feed Trades Trades of Large Size (filter) Broker Alerts: Broker aggregates for each alert type and total. Front-running: broker buys securities for his own account before buying the same securities for his customer, then sells when the price rises; or broker sells securities out of his personal accounts prior to selling the same securities for his clients. Frontrunning Patterns Restricted Sale: sales of securities that have ownership restrictions. (procedural) Brokers of Interest Trades of Interest Restricted Sales (join) (join) FrontRunning Alert Case Management GUI Restricted Sale Alert Brokers Restricted Securities Open/Close Mark Alert Marking Open/Close Patterns (procedural) Restricted Securities Broker Alerts Venues Trades Venue Trade Windows Broker Alerts (aggregate) Marking the Close: attempting to influence the closing price of a security by executing purchases at the close of normal trading hours. Marking the Open: attempting to influence the opening price of a security by making trades at the opening of normal trading hours. Copyright © 2015, SAS Institute Inc. All rights reserved. SAS® EVENT STREAM PROCESSING ENGINE OCH STREAMING ANALYTICS Kort Demonstration Copyright © 2015, SAS Institute Inc. All rights reserved. Analysera strömmande data med Event Stream Processing Några länkar: The connected truck SAS Forum https://m.youtube.com/watch?v=SkU8dmrWIPE SAS Event Stream processing demonstration https://m.youtube.com/watch?v=3xW-hUDsRIQ Börje Edlund Chief Architect [email protected] Twitter @BorjeEdlund Linkedin: Börje Edlund Copyright © 2015, SAS Institute Inc. All rights reserved.
© Copyright 2024