Matrix: Achieving Predictable Virtual Machine Performance in the Clouds Ron C. Chiang, Jinho Hwang, H. Howie Huang, Timothy Wood 2014, 11th International Conference on Autonomic Computing Presented by Philip Ehret and Tomas Mertens Context ● IaaS: Infrastructure as a Service ○ allows on-demand creation of virtual machines (VMs) ● VM properties ○ efficiency ○ resource control ○ equivalence ● In practise ○ little performance guarantee ○ cost-efficiency? Problem Statement ● Hard-to-predict VM performance ● Cost-performance tradeoff Goal: deliver desired performance using a VM while minimizing resource cost VM 1 25 % CPU 10 % Memory VM 2 10 % CPU 10 % Memory VM 1 20 % CPU 10 % Memory VM 2 10 % CPU 15 % Memory Proposed solution: Matrix Workload Signatures ● Signatures consist of: - CPU utilization I/O per second Memory size ... ● Representative workloads - video server, web server, file server, ... ● Signatures used for performance estimation of new applications Clustering ● Multiclass Classification based on SVM’s, using probability estimates ○ Input: workload signature ○ Output: probability of belonging to specific app category ● Example: Darwin, an open-source video stream server ○ 67% video server, 22% mcf, 10% soplex, … Clustering Procedure ● Data scaling ● Parameter selection ○ ○ ○ ○ ○ using grid search SVC type (C-SVC vs ν-SVC) kernel function … ⇒ v-SVC with RBF kernel ● Training Performance Modeling ● Relative performance metric: 1 2 ● Two main parts 1. Construct Basic RP models 2. Construct RP model of a new workload (online) Performance Modeling continued ● ● ● ● fi(R): n basic performance models (v-SVR) R = {r1 , … , rm } with 0 ≤ ri ≤ 1 ri : resource configurations performance model of new workload fnew(R) using its gene composition {p1 , … , pn } Automatic Resource Configuration ● Given the RP model of a new workload ○ find the cheapest resource allocation ○ while keeping desired performance Implementation ● Repeatedly check workload composition ○ using xentop ● Rebuilding of probability estimates ● Suggestion of new VM configuration ● Optimization algo in Matlab Evaluation Scenarios ● Single virtual machine case ● Multiple machines case ○ local virtual cluster ○ virtual cluster in public cloud Evaluation Metrics - Three metrics: - Prediction accuracy: - RP-Cost product: - Performance per cost: Evaluation: Single Machine case ● Performance prediction ○ Average prediction accuracy is 90,15% ○ Most prediction accuracies are > 85% ● Resource savings using Matrix ○ 37% CPU ○ 55% memory ○ while achieving a RP of 1,06 Evaluation: Local VC ● Mean performance prediction accuracy of 90,18% ● Very good PPC and RPC Evaluation: Public Cloud VC ● Matrix costs much less than static EC2 settings ● Matrix has better PPC values ⇒ good costperformance efficiency Conclusion ● Matrix ○ uses machine learning and optimization ○ to provide advice about instance types that achieve performance at lowest cost ● Matrix achieves high accuracy ○ even when shifting to public cloud VMs ● Keeps performance, minimizes costs Paper Evaluation ● Positive elements ○ ○ ○ ○ solution to a very actual problem nice application of machine learning techniques compares different IaaS providers results look promising ● Negative elements ○ unclear evaluation section, mixing local/public VCs with physical computers ○ too much noise in the information Discussion Questions? Advice?