Recommendations for Sizing HP Vertica Nodes and Clusters
Recommendations for Sizing HP Vertica Nodes and Clusters HP Vertica Analytic Database HP Big Data Document Release Date: April, 2015 Legal Notices Warranty The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein. The information contained herein is subject to change without notice. Restricted Rights Legend Confidential computer software. Valid license from HP required for possession, use or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor's standard commercial license. Copyright Notice © Copyright 2006 - 2015 Hewlett-Packard Development Company, L.P. Trademark Notices Adobe® is a trademark of Adobe Systems Incorporated. Microsoft® and Windows® are U.S. registered trademarks of Microsoft Corporation. UNIX® is a registered trademark of The Open Group. Contents Overview ....................................................................................................................................................... 4 Optimal Hardware Configurations for HP Vertica ........................................................................................ 4 Sizing Your Cluster ........................................................................................................................................ 4 Server Configuration .................................................................................................................................... 5 For More Information.................................................................................................................................... 5 Overview This hardware planning guide contains recommendations for hardware for your HP Vertica nodes. It also provides information that helps you appropriately size the hardware components to meet the needs of your environment. Optimal Hardware Configurations for HP Vertica The following hardware configurations provide excellent performance for your HP Vertica database. Component Recommendations Processor For optimal performance, Hewlett Packard recommends running the following processors: Two-socket servers with 8- to 14-core CPUs, clocked at or above 2.6 GHz for clusters over 10 TB Single-socket servers with 8 to 12 cores clocked at or above 2.6 GHz for clusters under 10 TB Memory HP Vertica requires a minimum of 8 GB of memory per physical CPU core in each server. However, in high-performance applications, you should run 12–16 GB of memory per physical core. The memory should be at least DDR3-1600 (preferably DDR4-2133), and should be appropriately distributed across all memory channels in the server. Storage HP Vertica requires a minimum read/write speed of 40 MB/s per physical core of the CPU. However, for best performance, you should have 60–80 MB/s per physical core. Each node should have 1–9 TB of storage post RAID. In a production setting, Hewlett Packard recommends RAID 10. RAID 50 can be a viable alternative. Due to the heavy compression/encoding that HP Vertica performs, you do not need to use solid-state drives (SSDs). To satisfy HP Vertica requirements, a RAID array of more, less expensive hard disk drives (HDDs) works just as well as a RAID array of fewer SSDs. Note: If you intend to use RAID 50 for your data partition, keep a spare node in every rack. This allows for manual failover of an HP Vertica node in the case of a drive failure. (Recovering an HP Vertica node is faster than rebuilding a RAID 50.To keep node recovery times at an acceptable rate, never put more than 10 TB compressed data on any node. Network Hewlett Packard recommends 10G networking over 1G networking in almost every situation. Sizing Your Cluster Consider these factors when sizing your cluster: Data volume (compression): First, look at the total raw data volume for the cluster and then apply a reasonable compression number. In most cases, 2:1 compression with high availability is a good start. To calculate compression for your data, try one of these approaches: o Use a previously attained compression number. o Install HP Vertica on an existing system and run Database Designer (DBD). DBD calculates an excellent compression for all columns. o Load several hundred gigabytes to 1 TB of data into the database. After you load the data, you can calculate the compression ratio by running an audit and summing up the used bytes in projection storage. For more information, see Calculating the Database Size in the product documentation. Data growth: Once you have a good idea of the starting compressed data volume, consider: o The rate of data ingest (i.e., the amount of data you load in the database each day) o Your organization’s retention policy, how long you intend to save the data You can estimate the total data volume required for your cluster using the following formula: ingest_rate × retention_period Workload: Develop an understanding of your total workload. The following features are part of every HP Vertica database workload: o Concurrency: The number of concurrently running queries. The higher the concurrency, the more core memory you need. o The amount of data on which the average query will operate o How you manage resources using runtime priority and resource pools. Server Configuration For high availability, Hewlett Packard recommends a minimum of three nodes. If you are getting better than 2:1 compression you may need fewer nodes. The following table lists the recommended server configurations. All configurations are based on raw data volumes and assume 2:1 compression. For configurations where the raw data size exceeds 10 TB, the best practice is to engage HP presales or an HP Vertica technical representative to help provide an exact sizing that best fits your needs. Before purchasing hardware for a production cluster, always run your hardware Bill of Materials (BOM) past the HP Vertica technical representative for review. Note: In a high-concurrency or heavy-workload environment, Hewlett Packard may recommend oversizing your cluster to achieve better performance. Raw Data Size Recommended Server Configuration and Comments Up to 5 TB 3 nodes each with: At least 8 15K RPM spinning disks for the data partition (total 1–2 TB per node or equivalent SSD) A single 8–12 core processor with clock speed or 2.6 GHz or higher 96–128GB of RAM 5-10 TB 6 nodes each with: At least eight 15K rpm spinning disks for the data partition (total 1–2 TB per node or equivalent SSD) A single 8–12 core processor with clock speed or 2.6 GHz or higher 128–256GB of RAM 10-40 TB 3–4 nodes each with: 22 10K rpm drives Dual 12-core processors at 2.6 GHz 256–512GB of RAM 40 TB – 1 PB 4–100 nodes each with: 22 10K rpm drives Dual 12-core processors at 2.6 GHz 256–512GB of RAM Larger than 1 PB In most cases, the same configuration for 40 TB–1 PB will work for raw data sizes greater than 1 PB. But first, contact an HP Vertica technical representative for recommendations on sizing clusters over 1 PB. For More Information In this document, Hewlett Packard engineers have provided you with sizing information for your HP Vertica cluster. The following resources provide more detailed configurations, and they explain how to best deploy your HP Vertica database. To… Go to… Configure the HP DL380 Gen9 24-SFF CTO server as an HP Vertica node https://community.dev.hp.com/t5/Vertica-Wiki/Configuring-the-HPDL380-Gen9-24-SFF-CTO-Server-as-an-HP-Vertica/ta-p/227450 Read the latest HP Vertica product documentation http://www.vertica.com/hp-vertica-documentation/hp-vertica-7-1-xdocumentation/ Learn how customers can deploy HP Vertica in a Cloud environment with the HP Vertica OnDemand documentation http://www.vertica.com/hp-vertica-documentation/hp-verticaondemand-documentation/ Create an HP Vertica cluster on Amazon Web Services (AWS) http://www.vertica.com/resources-for-technology-partner-integrations/
© Copyright 2024