Driving Better Benchmarks
Standard Performance Evaluation Corporation
http://www.spec.org
Overview
•
World-wide non-profit consortium formed in 1988 to establish, maintain and endorse a standardized set
of relevant benchmarks
•
Development of benchmark suites, review and publication of results
•
100 computer hardware and software vendors, educational and research institutions, and government
agencies
•
Large repository of well documented, peer reviewed, benchmark results
•
Understanding of workloads, benchmark code, fair comparisons across different systems
•
Provide guidance on workloads and benchmark development
Philosophy
•
Ensure that the marketplace has a fair and useful set of metrics to differentiate systems
•
Provide standardized code ported to a wide variety of platforms, metrics that are relevant and
comparable across systems, and run rules to ensure representative and repeatable tests
Advantages
•
Costs of benchmarking are kept down since:
•
Benchmarks are applicable to a broad range of platforms
•
A single standard benchmark replaces dozens of single use custom benchmarks
•
Peer review saves time and money over 3rd party audits
•
Benchmarks are retired and replaced every 3-5 years to keep pace with technology
•
New benchmarks can be created whenever there is sufficient interest
Membership Benefits
•
Access to the cutting edge benchmark development
•
Access to all benchmarks of a specific group
•
Participation in development meetings
•
Contribute your ideas - get information on the latest thinking in commercial engineering
•
Real world experience for grad students aiding in dissertation research and job searches
•
Real world experience from grad students aiding companies in recruiting top talent
•
Connect with dozens of leading universities worldwide who are SPEC associates and Research Group
members for research collaboration
•
Participate in ICPE conference with ACM - the premier performance evaluation conference in the world
•
Join SPEC and participate
SPECsip_Infrastructure2011
SPECvirt_sc2013
SPEC SFS 2014
SPEC CPU2006
Open
Systems
Group
SPECpower_ssj2008
Graphics &
Workstation
Performance
Group
SPEC Viewperf 12
SPECjvm2008
SPEC OMP 2012
Server Efficiency Rating Tool
SPEC PTDaemon
SPEC RG Newsletter
High
Performance
Group
Big Data Working Group
DevOps Working Group
Cloud Working Group
Contact: [email protected]
SPECjEnterprise2010
Chauffeur Worklet Development Kit
SPEC WPC
SPEC APC
SPECjms2007
SPEC ACCEL
SPEC MPI 2007
Research
Group
Intrustion Detection Systems
Working Group
Copyright 1995 - 2015 Standard Performance Evaluation Corporation (SPEC). All rights reserved.