The Leverege Platform Technical White Paper The Internet of Things (IoT) is the next evolution of the Internet; providing the opportunity to collect and analyze massive amounts of new information. However, simply having access to more information is not sufficient - businesses need to turn this tidal wave of data into actionable insights that can be used to improve products and increase operational efficiency. The Leverege platform is a comprehensive IoT solution that helps cut through the noise and enables businesses to run smarter and leaner. This Leverege Platform white paper provides insight into how Leverege Reason ™, Leverege Vision ™, and Leverege Curator ™ work synergistically to solve large-scale customer data problems for IoT applications. The Leverege Platform | Leverege LLC ©2015 | www.leverege.com About Leverege Leverege develops software products that enable customers to intelligently manage and visualize large networks of diverse sensors; providing actionable insights and spotting trends that lead to better decision making. Leverege is headquartered in Germantown Maryland, just outside of Washington DC, and was founded in 2014 by successful technology entrepreneurs with deep experience in sensor networking and “big data” system architectures. The mission-critical products built by the Leverege team are used daily by Fortune 500 companies and Governments around the world and span many industries including defense and aerospace, advertising, and sports and entertainment. 1 The Leverege Platform | Leverege LLC ©2015 | www.leverege.com Leverege Architecture Overview The Leverege Platform is a distributed, scalable system designed to collect and analyze data streams, enabling customers to manage and visualize large networks of sensors. Several overarching objectives in the design of platform are: ● Deliver a complete platform that also includes APIs as well as integrated tools for management, monitoring, and deployment. ● Create a platform that easily scales and adapts to customer needs. ● Build support for a diverse data collection environments that may be custom or include M2M and internet messaging standards such as MQTT, CoAP, DDS, HTTP/2 and others. ● Provide a robust system capable of running across a wide range of deployment environments that include on-premise, in a co-location facility, or via a cloud provider. ● Provide an open architecture that allows customers and partners easy access to the data as well as the ability to extend the platform. This architecture allows the platform to support a wide variety of sensor and data collection use cases, such as: ● ● ● ● ● ● ● ● Defense and aerospace Industrial control Asset tracking and inventory management Smart city initiatives such as smart parking and resource metering Vehicle tracking and fleet applications Logistics and remote monitoring Infrastructure protection and perimeter security Smart lighting and energy management 2 The Leverege Platform | Leverege LLC ©2015 | www.leverege.com System Architecture The Leverege Platform is a comprehensive visualization and decision support platform for the Internet of Things (IoT) that is composed of three major component areas: Reason ™, Vision ™, and Curator ™. The following sections introduce the major component areas and provide additional details about each set of components. Leverege Reason ™ Leverege Reason™ is a “big data” message processing system that supports complex event processing (CEP), providing the ability to combine data from multiple sources to detect patterns and quickly identify trends, threats, or important data elements. Reason supports processing data in real time, allowing a user to take immediate action, and batch processing for jobs that evaluate data collected in the past. Reason also enables users to create ad-hoc jobs for answering one-time or infrequent questions. All analysis results are then available for visualization and interaction in Vision™ and are also accumulated in long term storage. Data streams can also pass directly to Vision without any intermediary processing. 3 The Leverege Platform | Leverege LLC ©2015 | www.leverege.com Leverege’s open architecture allows for any information flowing through Reason to be available in near real time for Vision, utilized for real time batch processing jobs, or simply available for reporting. It also enables customers to build their own reporting or visualization tools on top of Reason. Each component in the system is scalable and distributable, allowing for horizontal scaling as needed. Leverege makes it easy to increase or decrease the number of running components in any subsystem to respond to changes in the data volume or processing requirements. This is important as it alleviates issues around resource bottlenecks caused by overutilization and eliminates resource waste due to underutilization. Ingestion and Comprehension The ingestion and comprehension subsystems are the entry point for all data into the system. Each data source is collected through data ingestors, which can be scaled to consume very small or very large data sets. Ingestion and comprehension are split into separate components to facilitate rapid data ingestion. This architecture ensures that there is very little delay in capturing the raw data from the source into the Leverege Platform for processing and interaction. As mentioned previously, the number of ingestors can be quickly increased or decreased to respond appropriately to changes in source data volume. Once the data is consumed by the ingestors, it moves through the pipeline to the comprehension components. Each of these components is designed to understand the ingested data such that it can route the data appropriately and place it onto the proper data queue for downstream processing. Once data is in a data queue, it can be consumed by any number of downstream processes, including real time processing pipelines, batch 4 The Leverege Platform | Leverege LLC ©2015 | www.leverege.com processing, or directly by Vision without any additional data processing. Data is not limited to a single data queue or a single processing path - the comprehension components place data into single or multiple queues depending on the governing business logic and that data can be consumed by both the real time processing and batch processing subsystems as well as by Vision directly. Data queues by default are designed to retain the data for a configurable time period, but they also support the ability to purge the data once it has been read from the queue. All of the data can also be stored in long term storage from any location in the data pipeline. Real Time Processing The real time processing components enable operational intelligence (OI), allowing customers to take immediate action against live data feeds. Real time data processing involves continuous evaluation of data streams, the results of which may be utilized in real time operational dashboards or tactical displays. The Reason real time processing subsystem consumes data from data queues and evaluates the data stream in a processing pipeline. A processing pipeline involves one or more steps, each of which evaluate the data and generate output or transform the data. This includes performing various activities such as: ● ● ● ● Correlating events or data collected from multiple devices Applying statistical noise reduction filters Discerning patterns in the data flow Triggering notifications for geo-fencing, activity monitoring, environmental monitoring, or other event analysis Reason is also able to utilize historical data in combination with the real time data streams to detect patterns in the data on the fly. Each relevant data point in the processing flow may be stored or made directly available to other components in the Leverege ecosystem. Batch Processing Batch processing provides a facility for evaluating very large data sets over an extended period of time in an automated fashion. This type of processing is typically thought of as more related to traditional business intelligence (BI) mechanisms. Each of these processes is broken into jobs that are scheduled to run across a scalable cluster. Every job or job grouping is assigned a schedule and relative priority, which determine when the job runs and how many, or what percentage, of resources are allocated to completing the job. Job priority can be altered on the fly if required to meet a deadline or to free up more resources for other jobs. All jobs can also be cancelled or paused, if needed. 5 The Leverege Platform | Leverege LLC ©2015 | www.leverege.com The batch processing engine also supports the creation of ad-hoc jobs for those times when a customer has a new question to answer but doesn’t require a scheduled job. Each of these ad-hoc jobs can easily be turned into a scheduled job as well. Data Storage The data storage layer is a multi-model datastore used to securely house all forms of customer data together into a single storage system that is both horizontally and vertically scalable. It is the heart of the overall Leverege platform as this layer is used as a repository for all raw and processed data in the system. This architecture allows Leverege to store petabyte sized data sets composed of many different types of data and make those datum available for quick retrieval, while replicating the data across a storage cluster to ensure data availability. Each data type is stored in the most appropriate form as determined by the type, use, and format of the data. Some examples of data that can be stored in the data storage layer include: ● ● ● ● ● Geospatial data that could be used for geo-visualization, geo-fencing, or geo-routing Time series data that indicates energy use or temperature change over time Geospatial and relationship / graph data that may be used for network analysis Sensor messages that can be used to visualize trends or detect threats Video and images files that can be tagged, correlated, and evaluated via object detection algorithms While the list above indicates some of the data storage types and examples of data the storage layer supports, the real power is that these different datasets and technologies can be used together to build comprehensive solutions to complex analysis and visualization problems in an environment that ensures fault tolerance and blazing fast data retrieval. Leverege Vision ™ Leverege Vision™ is a portion of the Leverege platform which customers utilize to gain insight into their sensor networks. Vision provides Command and Control (C2) dashboard functionality giving customers not only a tactical view into the current state of their network, but also an interface to manage and operate the entire system. Additionally, Vision includes a robust API layer that powers the presentation layer and can be used to more deeply integrate with the Leverege platform. 6 The Leverege Platform | Leverege LLC ©2015 | www.leverege.com Presentation Logic A user interface should not be simply utilitarian - it should be human-centric, intuitive, and easy to use. The Vision UI is an extensible web-based framework that allows customers to quickly understand and make decisions on all of the data that is available. It is designed to run 24x7 and can support both passive and active operational modes. Vision allows a customer to visualize their data in a number of ways that include everything from simple tables and detail boxes, to much more complex geospatial and time series views, as well as visualization drill downs. The logic used in data presentation is separate from the data itself, allowing the UI to remain responsive, lightweight, and to easily adapt to changes in the data. This also allows Vision to support a wide variety of device form factors, from very large, room-size tactical command centers to individual desktop displays and all the way down to tablets and smart phones. Each of the screens are built using a configuration driven approach. Vision utilizes display and layout factories such that each object can be defined and then tied together into a complex interface. Display objects can be easily reused from screen to screen and a model-view-controller approach naturally supports the linking of events and objects among display elements. This feature enables users to quickly navigate to the information that they need to find or search for objects of interest across the entire interface without losing situational awareness. Service Layer The Vision service layer serves as the middleware between Vision and Reason. Everything that appears in the Vision UI passes through this layer, which facilitates the strong separation of concerns that allows the UI to remain lightweight and unencumbered with business logic related to serving data. This layer enables data retrieval and collaboration technologies, as well as sending commands back upstream to the devices. Every action taken in the UI is governed by a Role Based Access Control (RBAC) security service, providing the granular controls customers need to ensure that each user has the ability to perform their job while protecting the data from untrained or unauthorized individuals. 7 The Leverege Platform | Leverege LLC ©2015 | www.leverege.com The Vision service layer also serves as a web-based API into the Leverege Platform. Every datapoint or service available to the Vision UI is also available through the Services API. Most of this functionality is provided through REST services, although endpoints that provide streaming data feeds leverage technologies such as websockets or server sent events (SSE). Leverege Curator ™ Every good system requires a powerful management layer and the Leverege Platform is no exception. Curator™ is our platform management solution. Just as Reason and Vision are the C2 platform for IoT, Curator is the control platform for all of the Reason and Vision components and infrastructure. Curator allows customers to view the Leverege system logs and inspect the overall system health in one place. However, it goes several steps beyond simple monitoring and inspection. Curator allows an administrator to provision new resources, define data governance rules, change batch job schedules, and plays a central role in registering new components. It serves as a virtual operating system, resource manager, configuration manager, and monitoring solution all rolled up into one cohesive set of management tools with an intuitive UI that places all of that power at an administrator’s fingertips. 8
© Copyright 2024