1 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. 2 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Key Patterns Converge: Data Integration and Data Virtualization for SOA (CON8514) Alex Kotopoulis and Ayush Ganeriwal Data Integration Product Management 3 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 of the corporate presentation template Agenda Oracle Data Integration Solutions Overview Combining SOA and Oracle Data Integrator Data Virtualization with Oracle Data Services Q&A 4 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 of the corporate presentation template Oracle Data Integration Overview 5 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 of the corporate presentation template Oracle Data Integration Solutions Thousands of customers including Fortune 500 companies, representing Financial Services, Communications, Healthcare, Public Sector, Retail, Utilities industries • 4 of top 5 largest commercial banks • 3 of top 3 busiest ATM networks • 5 of top 5 financial data services companies • 5 of top 5 telecommunications providers • 3 of top 5 largest food & drug stores 6 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 of the corporate presentation template Oracle Data Integration 12c Delivering Real-Time Integration for Big Data and Cloud Cloud Oracle Data Integrator Apps Oracle GoldenGate Database Big Data 7 Oracle Enterprise Data Quality Oracle Data Service Integrator Copyright © 2013, Oracle and/or its affiliates. All rights reserved. • Real-time replication; optimized for Database 12c and Oracle Exadata • End-to-end integrated with simplified deployment • Unified tooling for both structured data sources and Hadoop / NoSQL • Flexible deployment on-premise or in the Cloud for heterogeneous systems • Expanded support for 3rd party systems and Oracle Applications in real-time data integration and continuous availability solutions Insert Information Protection Policy Classification from Slide 12 of the corporate presentation template Oracle Data Integrator Bulk Data Processing and Data Transformation Cloud Apps Oracle Data Integrator High Performance E-LT Declarative Design Database Extensible Knowledge Modules Certified for leading technologies to deliver fast time to value High-performance, low cost of ownership E-LT architecture Lightweight deployment Big Data CEP Data Services Capable of Integrating Structured & Unstructured Data 8 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 of the corporate presentation template Flexible, easy to enrich functionality Oracle GoldenGate Real-time Data Integration Cloud Oracle GoldenGate Apps Log-based Change Data Capture and Delivery Active-Active Replication Database Big Data Heterogeneous Source and Targets Reliability and Transaction Integrity Java Integration, Real-time Events 9 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 of the corporate presentation template • High-performance, low-impact real-time data integration and replication • Timely data for improved business insight • Continuous availability for 24/7 operation Oracle Enterprise Data Quality Data Quality for Customer and Product Data Cloud Oracle Enterprise Data Quality Profile, Explore, Audit Apps Parse, Standardize, Cleanse Database Big Data Match, Merge, Enrich Governance Case Management 10 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 of the corporate presentation template • Improves data accuracy, usability and ‘fitness for purpose • Unified interface for ease-of-use, lower TCO • Depth of capability in multiple data domains reduces project risk Oracle Data Services Integrator Data Federation and Virtualization Cloud Oracle Data Services Integrator Federated Data Services Apps Data Virtualization Database Big Data Integrates with Oracle SOA Bidirectional Reads & Writes Integrates with Oracle Business Analytics 11 Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 of the corporate presentation template • Easily develop data services and ad-hoc queries without infrastructure changes • Open and standards based—SOA, Web Services, Java, Service Data Object Oracle Data Integration Combining SOA and Oracle Data Integrator 12 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Data Integration Services Client Applications: BPEL, OSB, Custom OGG Data Replication Services WSDL ODI ELT Services Agent WSDL Data Access Services Data Access MoveTransformed & Replicate Data in Real Time Transform Bulk Data JMS Oracle Data Warehouse Other DB WSDL WSDL ODSI Data Virtualization Services OEDQ Data Quality Services Validate Virtualize Access to and Cleanse Data distributed sources Application File Web Service Real-time Data Warehousing with ODI, OGG and OEDQ ODI Agent OEDQ Transform Validation & Cleansing Kick off OEDQ process Bulk for load snapshotofof Waits availability Production DB into incremental data staging Production Schema EMP Transform and load data into Data Warehouse DEPT EMP DEPT Initial Load Staging Schema ODI generates and kicks of OGG process to replicate incremental changes DBMS Logs OGG Trail Pump DIM FACT Transform & Load DIM DIM DW Schema TCP/IP Extract DIM Trail Replicat OGG ODI for Transforming Large Payloads Oracle BPEL PM or ESB 1 Order DB 2 <yxz> <zyx> Reference 3 Reference 8 7 Reference ODI Invoke Data Integrator 4 Product Suppliers Shared Metadata Repository 5 Transform 6 Shared Container/JVM Shared SOA Server 1. Large File arrives, detected by BPEL File Adapter 2. Execution Starts (BPEL/ ESB) – and a step for transforming a large document payload occurs 3. Pass XML payload, by reference, to ODI 4. ODI loads payload 5. ODI transforms payload 6. ODI sends payload wherever instructed 7. ODI notifies BPEL/ESB that job is complete 8. Core BPEL/ESB processing completes 15 ODI Loads a BI Warehouse (with Error Hospital Human Workflow) 1 Oracle BPEL PM or ESB Event 1. 2 An event occurs which prompts a data load 2. Execution starts within BPEL/ ESB process 3. Instruction to start ODI 4. ODI extracts & loads data a) Some errors w/in batch b) Notification to BPEL Human Workflow engine c) Repair & approvals process d) Repairs are committed e) ODI looks for repaired data when “Recycle” option is on 5. ODI transforms the data 6. ODI confirms job complete 7. BPEL/ESB process continues any additional process execution steps c Human Workflow 7 d 3 Instruction 6 Confirmation ODI Invoke Shared Metadata Repository b Data Integrator e Transform Shared Container/JVM Shared SOA Server a DB Server Any DB 5 4 Transform Records A DW Server Business Intelligence Records B 16 ODI for SOA-based ERP Integration 1. 2. 3. A business process for Migration (initial bulk data load) or Replication (ongoing synch) is invoked a) b) 1 2 5. BPEL/ESB begins processing ERP business transactions BPEL/ESB leverage same Unique IDs for canonical XML UI XML 5 App Instruction c ODI updates Unique ID for existing objects ODI confirms job c) App ERP App 2 Oracle BPEL PM or ESB XML ODI creates Unique ID for new ERP data objects 4. 6. UI BPEL/ESB sends instruction to ODI ODI performs E-LT Messaging and bulk data may leverage the same unique object IDs – thereby ensuring uniform ERP data objects (eg: PK123 = Pkxyz) ERP App 1 Confirmation 4 ODI Invoke Shared Canonical ID XREF Lookups Data Integrator Data 3 Shared Container/JVM DDL a Data Transform b DDL All SOA and ETL jobs keep business data aligned with Unique IDs 17 ODI High Availability Performance and Reliability through Weblogic and Coherence Clients WebLogic Cluster ODI Repositories on Oracle RAC WLS1 ODI Public Web Services ODI Agent Repository Explorer ODI Studio Designer WLS2 Proxy / Load Balancer ODI Agent Operator WLS3 Topology ODI Agent Coherence ODI Scheduler Sync ODI SDK APIs ODI Master Repository ODI Work ODI Work Repository ODI Work Repository Repository Security 18 XML File processing <customer id=‘4433’ name=‘Paul’> <order id=‘1’> <item id=‘101’ count=‘5’/> <item id=‘102’ count=‘3’/> <item id=‘103’ count=‘3’/> </order> <order id=‘2’> <item id=‘201’ count=‘5’/> <item id=‘202’ count=‘3’/> </order> </customer XML Schema customer order ODI XML JDBC Driver PK FK 1 1 2 1 PK ID Name 1 4433 Paul item PK FK id count 1 1 101 5 2 1 102 3 3 1 103 3 4 2 201 5 5 2 202 3 Complex File processing Native Format Builder UI Staging Tables customer order C|4433|Paul O|1 I|101|5 I|102|3 I|103|3 O|2 I|201|5 I|202|3 I|203|3 nXSD Schema S_OPTY ODI Complex File JDBC Driver PK FK 1 1 2 1 PK ID Name 1 4433 Paul item PK FK id count 1 1 101 5 2 1 102 3 3 1 103 3 4 2 201 5 5 2 202 3 Oracle Data Integration Data Virtualization with Oracle Services Integrator 21 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Different Data Access Techniques • • • Consolidation: Move data from distributed sources to central hub; consuming applications query hub database Data Virtualization / Federation: Query and update distributed sources in-place and provide combined result Hybrid Approach: Federation and Consolidation are combined based on best approach for individual use cases Consumer Consumer Consumer ODSI Consolidated Hub ODSI ODI/OGG Source A Source B Consolidation only Consolidated Hub ODI/OGG Source A Source B Data Virtualization only Source A Source B Source C Hybrid Consolidation + Data Virtualization When to Use Which Integration Technology Data Virtualization Consolidation • Data cannot be replicated (security, ownership) • Performance is priority • Consolidation is not efficient for access pattern • Data needs to be offloaded from original source • Real-time access is required and replication is not efficient • Replication is used for redundancy • Transactional updates • Need to change model frequently 23 Data Virtualization Service Bus / Business Process • Combine & normalize data access services (Create, Read, Update, Delete) • Orchestrate services to business processes • Service Routing / Mediation Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Data Virtualization Use Cases Data Services Layer – Provide a clean data abstraction for SOA and application developers – Shield consumers from changes in data topology Augmenting Business Intelligence – Extend access to sources outside the data warehouse Rapid Prototyping of Data Integration – Quick creation of an interface to integrated data without building a consolidated store Cloud Data Integration – Combine onsite and offsite data resources in a unified interface 24 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Data Virtualization: Oracle Data Service Integrator Product Architecture Java Web Svc JDBC .NET Fast Track Transport Metadata Monitoring/Auditing ODSI SDK Admin Console ODSI Studio IDE OSB Security Cache Query Engine Fast Track Transport OSB Database 25 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Web Svc XML/Flat File API / Apps OER ODSI Core Strengths • Native SOA Data Integration – Native XML type system allows processing and passing of hierarchical & SOA data. – Based on open standards (XML, WS, JDBC/SQL, XQuery) – Tight integration in Oracle SOA: Fast-path with OSB, Governance with OER • Normalization and Reuse through Data Services – Multi-layer approach to normalizing and transforming source data • Highly Scalable, Industrial-Strength Performance – – Linear scalability through WLS Clusters Performance through caching, streaming and query optimization • Fine-Grained Security and Auditing – Row and column-based data security • Comprehensive Update Capabilities – – 26 Automated CRUD operations on data services Distributed XA transactions through WLS Transaction Manager Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Data Service Design & Modeling Layered Approach ODSI Development Environment 28 ODSI Logical Data Service 29 ODSI Parameterized Data Service 30 ODSI Query Plan 31 ODSI Test Execution 32 ODSI Update Mapping 33 ODSI Administration Console 34 Oracle Data Integration Q&A Questions & Answers 36 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Join the Data Integration Community Twitter twitter.com/ORCLGoldenGate LinkedIn “Oracle Data Integration” Facebook facebook.com/OracleDataIntegration Oracle’s Data Integration blog blogs.oracle.com/dataintegration Data Integration YouTube channel youtube.com/oraclegoldengate Oracle Data Integrator Books http://www.packtpub.com/oracle-data-integrator-11ggetting-started/book http://www.packtpub.com/oracle-data-integrator-11gcookbook/book Oracle.com/goto/dataintegration 37 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle OpenWorld – Key DIS Sessions Oracle Data Integrator Monday Tuesday Wednesday CON8513 Oracle Data Integrator Product Updated and Future Strategy 1:45 PM Moscone West 2024 HOL10130 Introduction to Oracle Data Integrator 12:00 PM Marrriott Marquis Nob Hill AB CON8515 Customers Perspectives: Oracle Data Integrator 1:15 PM Moscone West 2012 HOL10144 Combining Oracle Data Integrator and Oracle GoldenGate 3:45PM Marriott Marquis Nob Hill AB CON8544 Making the Move to Oracle Data Integrator 3:30 PM Moscone West 2012 CON8514 Key Patterns Converge: Data Integration and Data Virtualization for SOA 5:15 PM Moscone West 2024 CON8516 Extracting Strategic Insights from Business Data wit Oracle Data Integrator 5:15 PM Moscone 2012 38 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Thursday Graphic Section Divider 39 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 40 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 41 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
© Copyright 2024