Architecting a Predictive and Proactive Cyber Defense System
Little Book of Security: Cyber Defense Mission Precognition: Architecting a Predictive and Proactive Cyber Defense System A must-read for IT decision-makers and information security professionals Get There Faster Little Book | Cyber Defense 2 Get There Faster Little Book | Cyber Defense Contents Is Proactive Cyber Security Really Achievable?....................................................4 Architecting a Predictive Cyber Defense System................................................ 5 Architecture.............................................................................................................. 7 Design for Scale.......................................................................................................9 Monitoring.............................................................................................................. 10 Correlation............................................................................................................... 10 Prediction................................................................................................................ 12 Response................................................................................................................. 14 Analyses.................................................................................................................. 15 Realizing the Possible.............................................................................................16 Get There Faster 3 Little Book | Cyber Defense Is Proactive Cyber Security Really Achievable? The ever-increasing rise in cyber warfare attacks on our country’s defense, corporate, and financial systems, has given rise to the need for a better approach to cyber defense. The old reactive paradigm will no longer suffice to protect our strategic cyber assets. Attack vectors are consistently becoming more complex and harder to detect. Attacks patterns are moving up the stack from the network layer to the application layer. Our current approach of monitor, detect, and then analyze after the fact is putting us at risk. In effect, we are shutting the barn doors after the horses are already out. Once we have completed analyzing the attack, the damage is done and usually can’t be reversed. We need a new approach to defend against this new threat pattern. In order to prevent catastrophic damage to our infrastructure, we need the ability to detect cyber attacks in real time and stop them before they complete. Traditional defenses such as firewalls and intrusion detection systems are not enough, we need a new generation of tools that can react as fast as, or even faster, than the automated tools used to attack our systems. Many question whether achieving a proactive cyber posture is even possible. The answer is yes! The products and tools necessary to properly protect our mission-critical systems already exist today. We will look at how this new generation of products is making proactive cyber defense a reality. 4 Get There Faster Little Book | Cyber Defense Architecting a Predictive Cyber Defense System This guide proposes an architecture that leverages a new generation of tools, to create a real-time proactive cyber defense system. A system that not only detects attacks as they are happening but even uses predictive analytics to anticipate them and take proactive action to prevent them. A system that learns from previous attack patterns in order to detect and predict similar attacks in the future. We will prescribe an event-driven architecture utilizing a complex event processing (CEP) engine to enable this capability by providing a highly performant, intelligent monitoring and response system to fuse security events from a number of point solutions, detect attack patterns across the security product spectrum and in automated fashion, take proactive measures to thwart or reduce the impact of attacks. We will also discuss how predictive analytics can minimize infrastructure risk by merging real-time events with historical Big Data into actionable intelligence. The key steps in this process are: 1. Monitor real-time security feeds from all available security and relevant social data sources 2. Combine and correlate data feeds and build predictive models using features across data feeds 3. Analyze patterns to predict system, network, and social vulnerabilities that are most attractive to attackers 4. Automate response workflows that proactively prevent threats 5. Store relevant data in Hadoop for both historical analysis (finding previously undiscovered attacks) and real-time investigations using inmemory computing and visual analytics Get There Faster 5 Little Book | Cyber Defense Attack vectors are consistently becoming more complex and harder to Figure 1 - Threat Prevention Steps It’s difficult to build systems that can ingest the amount of data needed to recognize cyber attacks and then process that data within the relevant time windows needed to prevent or mitigate the impact of the attack. The data needed for recognition of the event is unknown at design time and therefore requires a model that can dynamically recognize new threat clusters, alert analysts to label these new clusters, and feed them to an online learning model for future predictions. As shown in Figure 1, the process for proactive threat prevention consists of five main phases: 1. Monitoring: Ingesting large amounts of streaming, real-time security events from multiple systems 2. Correlation: Aggregating, filtering, and transforming data streams into correlated events 6 Get There Faster Little Book | Cyber Defense 3. Prediction: Detecting patterns and anomalies across correlated events to predict attacks 4. Response: Initiating automated and manual workflows to respond to and prevent attacks 5. Analysis: Analyzing historical and real-time data to detect and label new or undetected attacks Each of thee phases is critical to the success of a proactive cyber defense system. Unfortunately, most of today’s cyber security products and applications focus heavily on monitoring and detection at a granular level. They lack the ability to correlate events across other vendors’ products, predict complex attacks in flight, and most importantly, respond to those attacks using dynamic workflows to ensure immediate and thorough coverage of the response protocols. Architecture As the amount of security data associated with applications continues to grow, the ability to store and process that data in a useful timeframe continues to shrink. Big Data technologies let us store and process extremely large amounts of data, but don’t necessarily provide an adaptive way to process and analyze the data in real-time, thus limiting our ability to perform predictive analyses on the data. Big Data technologies such as Hadoop, are batch-oriented, focusing more on running different analyses over the same data set, than ingesting new data and performing the same types of analyses, such as you would want to do in a cyber-crime or fraud detection scenario. What is needed is a technology that allows data in the form of events from multiple sources to be correlated in real-time and actions to be taken on events matching patterns of interest in order to see ahead and take action to avert the pattern end-state. For example, Get There Faster 7 Little Book | Cyber Defense cyber attacks and fraud often follow known patterns. If the pattern can be detected early, actions can be taken to avoid a successful intrusion or a fraudulent transaction. In order to address the way we approach this new era of real-time data collection and predictive analyses we need a new paradigm. One such approach is to utilize an event driven architecture which allows you to treat data feeds as streams, searching for items of interest, rather than continuously storing all the data and repeatedly processing it. 8 Get There Faster Little Book | Cyber Defense Design for Scale The basis for any real-world threat detection system must be a scalable architecture. Cyber attacks are increasingly growing more complex and the data that must be collected continues to multiply. In order to address the ever-growing number of events and the shrinking response window, an event-driven architecture (EDA) must be used to ensure real-time performance. Event Driven Architectures are an approach to handling a large amount of asynchronous events, such as in a stock trading system. They differ from traditional Service Oriented Architectures in the way that they handle requests. Instead of batching data together into requests that are submitted and responded to synchronously. EDA exposes fine-grain data that represent events in an asynchronous manner. As a result, data streams can be filtered for particular events. EDA architectures are usually designed for performance, concentrating on handling thousands to millions of events a second. This type of architecture is well suited for large sensor networks in which millions of events must be sifted through and correlated. Traditionally, rules engines were used for this type of processing, but they become quickly overwhelmed with such high throughput systems. Get There Faster 9 Little Book | Cyber Defense Monitoring The first step in any threat prevention architecture is a pervasive monitoring strategy. Most organizations have robust network monitoring tools, but neglect the entire stack in favor of focusing on the entry points. Monitoring needs to be put in place across all of the platform tiers and at every layer. It is not sufficient to protect only the perimeter when threats are increasingly coming from the inside, as well as higher up the application stack. In addition to network sensors, database logs, application server logs, application logs, iDM logs, directory server logs, OS logs, host-based intrusion detection systems, and a slew of other security sensors must be monitored continuously. Within the government, CDM (continuous diagnostics and mitigation) has become a religion because security experts all agree that most of the attacks taking place today can be stopped just through better monitoring. Correlation When all of the proper security sensors are in place and are being continuously monitored, all of that data must be correlated. Today’s threats are sophisticated and designed to evade single monitoring sensors, such as a network-based intrusion detection systems. Instead, they exploit vulnerabilities at many different layers and in several disparate products. In order to detect these sophisticated attacks, events from all of the different types of security sensors must be aggregated and correlated. Aggregating and correlating the millions of events per second in real time that can occur in a large organization or government agency requires a different way of handling events. In order for you to process this magnitude of events from so many different data streams, a complex event-processing (CEP) engine is required. 10 Get There Faster Little Book | Cyber Defense A complex event processor, correlates events by creating extemely lightweight event listeners that can be tied together and given lifetimes in order to detect higher order events. For instance, a simple correlation rule might look like: “if event A from stream X happens with 5 seconds of event B from stream Y, take action C”. Simple rules like this can then be strung together to look for very sophisticated patterns, such as those that are used in fraud detection by credit card companies, or high frequency trading violations. Get There Faster 11 Little Book | Cyber Defense A good CEP engine, such as Apama, provides an integrated development environment with graphical tools to construct and dynamically deploy event monitors. It also provides a rich set of analysis tools to gain deeper insights into the data streams as well as providing a playback capability that allows you test different scenarios and create monitors that can be tested against real data in an accelerated or decelerated time frame. Using a CEP, events from all of the different security sensors that are being monitored can be aggregated and correlated to provide a more holistic view of the ongoing threat landscapes. Threats that were once hard to detect becuase they spanned multiple sensors, can now be easily recognized, once the corresponding events are correlated. Prediction When all the events from the various security sensors and logs have been aggregated and correlated to detect patterns, these patterns can be recognized and future events can be predicted. For instance, consider a multi-phase attack that starts with a phishing attack, then downloads a file onto a target’s host, scans the file system for certain document types and then uploads those documents to another compromised machine, where the attacker can collect them. Once the system correlates the events from the email server and the host-based intrusion detection system, it can recognize the pattern and predict that documents will be stolen by being uploaded to one of the attacker’s machines. It is this ability to recognize patterns and predict future events that is core to an adaptive security strategy. 12 Get There Faster Little Book | Cyber Defense To be able to continuously detect patterns and make predictions on trained data sets, but those data sets must be continuously updated. In order to process in real-time, an In-memory computing solution such as Software AG’s Terracotta platform, must be used. In-memory computing allows the millions of streaming events to be maintained and processed in memory, in order to allow for predictions to happen in real-time. It also speeds up the process of building new classification trainings sets to recognize new features and re-train and/or learn online with new data sets. Training sets for the prediction algorithms must be built by analysts that can identify and label attack patterns in the correlated event sets. They can also be built from a visual analytics platform like Presto, using current data available in-memory along with historical data stored in a Big Data store such as Hadoop. Response Now that the capability exists to predict future stages of an attack, steps Get There Faster 13 Little Book | Cyber Defense must be taken to proactively prevent theft or corruption of the data by responding with preventative measures. To respond in real-time to known and new, learned attacks, requires the ability to dynamically create and deploy response workflows. To accomplish this, a business processing engine (BPE) is needed. A BPE, like webMethods, provides dynamic workflow creation that is easy to use and powerful enough to integrate all of the governance requirements associated with a formal threat response capability. Security analysts can interact with BPE processes using task management and collaboration capabilities. The various business and IT stakeholders can define and change rules that drive the threat response processes “on the fly” without any development work. Processes are orchestrated, resulting in transparent, efficient and adaptive processes for active threat prevention and end-to-end visibility. 14 Get There Faster Little Book | Cyber Defense In addition to manual portions of the workflow, many automated responses can be configured as well, such as the ability to lock down ports and user accounts. In our earlier example, once the threat has been recognized and the future stages predicted, the BPE could respond by shutting off outbound traffic from the compromised host and then alerting a security response team so that they can go in and quickly remove the attacker’s tools from the machine, before any data is stolen. All of the steps in the process can be managed, monitored and can be tracked to ensure a proper auditing of the attack and the response. Analyses Any security incident starts and ends with analyses. Security analysts need the tools to be able to weed through piles of data to dissect an attack and understand its impact. Large organizations employ groups of security forensic experts to analyze gigabytes of data in log files and reports, to comprehend the extent of damage of a successful attack. Get There Faster 15 Little Book | Cyber Defense Therefore, the platform must provide visual analytics support that enables analysts to merge, filter, sort and run complex queries on multiple static and streaming data sources. Through visual analytics, analysts are empowered to easily combine different data sources, from log files to vendor provided web services, and not only analyze the data, but apply a variety of visualizations to it and then share those visualizations with other analysts and stakeholders. Presto is a visual analytics platform built for analysts that need to aggregate and visualize data from multiple data sources in real-time. It leverages its Real-Time Analytics Query Language (RAQL), a SQL-like query language that allows access to static and streaming data sources to provide drag and drop analytics to analysts without the need for programming. Realizing the Possible Architecting a predictive cyber defense system entails several steps and requires a variety of different products and technologies. Such a system should provide a single solution to monitor and manage threat investigations as well as proactive threat defense in the form of automated response workflows, in order to prevent attacks in process. The first step in the process is to ensure there is proper monitoring in place at all critical points throughout the IT landscape. All of the cross vendor/product monitoring data must be aggregated and correlated to provide a common operating picture from a security perspective. The cyber defense system must have the ability to predict events and our straw man architecture does so using industry standards such as R and JavaML. It has the ability to detect new threat scenarios by allowing security analysts to visually analyze and label new threats and then create subsequent response workflows to prevent completion of those scenarios 16 Get There Faster Little Book | Cyber Defense This is an open architecture that can be tailored to an agency’s specific requirements. It can be rapidly implemented through use of GUI-driven tools and Intuitive graphical interfaces and visualizations. It also prescribes a case management approach to threat response, leveraging a fully audited workflow for governance compliance and better risk awareness. Our proactive threat response architecture relies on technologies and tools that are readily available today. Leveraging these tools, it is possible to implement a predictive cyber security defense system that doesn’t just enhance existing cyber defense capabilities, but extends them to provide proactive defense by predicting an attack in progress and taking action to stop it before any real damage is done. Get There Faster 17 Find out how to Prove IT First and Prove IT Fast at www.SoftwareAGgov.com ABOUT SOFTWARE AG GOVERNMENT SOLUTIONS Software AG Government Solutions is dedicated to serving the U.S. Federal Government and Aerospace and Defense communities with massive-scale and real-time solutions for integration, business process management and in-memory computing. Our flagship products are webMethods, Terracotta/Enterprise Ehcache, ARIS and Apama. The company’s highly effective “special forces” approach to solving complex IT challenges quickly and efficiently is embraced across our customer base as a prove it first and prove it fast means for minimizing risk associated with IT investments. Headquartered in Reston, VA, Software AG Government Solutions is a wholly owned subsidiary of Software AG USA with more than 30 years of experience with the federal government. Learn more at www.SoftwareAGgov.com © 2015 Software AG Government Solutions Inc. All rights reserved. Software AG and all Software AG products are either trademarks or registered trademarks of Software AG. SAG_Cyber_Defense_18PG_5.5_7.5_LB_Jan15 Get There Faster
© Copyright 2024