International High Performance
Computing
Troels Haugbølle
Centre for Star and Planet Formation
Niels Bohr Institute
PRACE User Forum
Why International HPC ?
•  Large-scale science projects can require resources beyond
what is available in Denmark:
•  Achieving science that requires enormous resources for
a short time [capability computing]
•  Carrying out science programs that necessitates a large
amount of resources [capability & capacity computing]
•  Why should you care ?
Break new ground scientifically, go to the next level
•  Danish membership of PRACE give researchers in
Denmark free access to the largest computers in Europe
•  Even larger resources exist through INCITE and HPCI
DeiC, Middelfart – Oktober 2015
Troels Haugbølle – [email protected]
The HPC Pyramid
Tier-0
International
50.000+ cores
PRACE Tier-1
Computerome
Tier-1: National &
International 10-50.000 cores
Tier-2: Local HPC centers
1.000-10.000 cores
DeiC, Middelfart – Oktober 2015
Local centers
Abacus-2.0
Troels Haugbølle – [email protected]
http://www.prace-ri.eu/prace-in-a-few-words
• 
25 members countries
• 
Hosting members: €400M (30 ø/ch)
• 
Established in 2010
• 
EU & other members: €130M
• 
• 
412 projects awarded
10.700 million core-hours
• 
18 Pflops/s at 6 world-class systems
DeiC, Middelfart – Oktober 2015
Troels Haugbølle – [email protected]
FERMI (IT)
2 PFs, 164 TB
163.840 cores
Blue-Gene/Q
SUPERMUC (DE)
3.2 PFs, 340 TB
155.656 cores
x86 + Infiniband
Curie (FR)
2 PFs, 378 TB
52.992 cores
x86 + Infiniband
DeiC, Middelfart – Oktober 2015
Mare Nostrum (ES)
1 PFs, 100 TB
49.568 cores
x86 + Infiniband
JUQUEEN (DE)
5.9 PFs, 459 TB
458.752 cores
Blue-Gene/Q
HORNET (DE)
3.8 PFs, 493 TB
94.565 cores
Cray XC40
Troels Haugbølle – [email protected]
What does PRACE offer to Danish researchers ?
• 
• 
Access based on scientific excellence
Access is free, once awarded
• 
Three levels:
•  Prepatory access: Used to establish the readiness of codes, comes with
abundant technical support – low threshold
• 
• 
• 
DECI Tier-1 access: One call per year; collection of national Tier-1
machines; based on juste retour for 70% of resources, 30% are “free”, and
can be used by e.g. Danish researchers
Tier-0 access: Two calls per year, based on membership and pure scientific
excellence
PRACE Training and support:
•  PRACE offer courses at all levels by the best international experts
• 
• 
Workshops about scaling of your own applications
PRACE make BEST PRACTICE guides for different HPC architectures
DeiC, Middelfart – Oktober 2015
Troels Haugbølle – [email protected]
DK: ~3% or 320 million core-hours
Access to PRACE resources
by country 2010-2015
DeiC, Middelfart – Oktober 2015
Troels Haugbølle – [email protected]
PRACE by discipline
DeiC, Middelfart – Oktober 2015
Troels Haugbølle – [email protected]
PRACE Tier-0 Call is open!
MareNostrum: 120 million core-hours
FERMI: 390 million core-hours
Typically 10 (MN) or 40 (FERMI) million
core-hours per project
Transition: Larger grants in the future
http://www.prace-ri.eu/prace-project-access
DeiC, Middelfart – Oktober 2015
Troels Haugbølle – [email protected]
Requirements for Tier-0
•  Excellent, exciting, and disruptive science(!)
•  ~10 pages of application; some parts highly technical – but
return is equivalent to ~1 MDKK
•  Open research by industry is encouraged
•  Application readiness, proven scalability past 1000 cores
DeiC, Middelfart – Oktober 2015
Troels Haugbølle – [email protected]
Requirements for Tier-1
•  Excellent and exciting science
•  ~3-4 pages of application containing
•  Scientific objectives; technical innovation potential
•  Profile and performance of code(s), including scalability
•  How requested core-hours was calculated
•  Specific benefits expected from PRACE
•  Typical grant size: 0.5 to 3 million core-hours
•  Call deadline just passed in September (every year)
•  Many machine types: Cray, x86, GPUs, Phi’s, BG/Q, SMPs
•  ~100 Million core-hours offered
DeiC, Middelfart – Oktober 2015
Troels Haugbølle – [email protected]
Access through PRACE Peer Review
Free-of-charge required to publish results at the end of the award period
Open to international projects
Project Access (every 6 months) award period 1 to 3 years
Individual researchers and groups
No restriction on nationality for both researcher and centre
Required to demonstrate technical feasibility of project
Preparatory Access (cut-off date every 3 months)
Optional support from PRACE experts
Prepare proposals for Project Access
[From PRACE standard slides]
Project Access
Open Call
for
Proposals
Technical
Review
Priorisation
Scientific
Right
+
Peer Review to reply
Resource
Allocation
~ 3 Months
~ 2 Months
Technical
experts in
PRACE
systems and
software
Researchers
with expertise
in scientific
field of
proposal
Project
+
Final
Report
Up to 3 years
Access
Committee
Researcher
www.prace-ri.eu/call-announcements/
[From PRACE standard slides]
DeiC
http://www.prace-ri.eu/organization
DeiC, Middelfart – Oktober 2015
Troels Haugbølle – [email protected]
PRACE is too complicated for me!
Core-hours
Application
Scalability
Science case
Additional PRACE activities
•  Preparatory grants:
•  Very easy to obtain
•  Grants to establish scaling, remove bottlenecks
SEND YOUR STUDENTS
•  Lots of technical
support
AND
YOURSELF!
•  ~100.000 core-hours, not for science
•  PRACE Advanced Training centers: (~70 free schools/yr!)
•  BSC (Barcelona), CINECA (Italy), CSC (Finland), EPCC
(Edinburgh), Gauss Centre for Supercomputing (Germany),
Maison de la simulation (Paris)
•  Best of the best teachers http://www.prace-ri.eu/trainings
•  Topics: Parallel programming, viz, scaling, application
specific schools, data analytics, etc etc
•  Train-the-Trainers program for teachers
DeiC, Middelfart – Oktober 2015
Troels Haugbølle – [email protected]
Beyond PRACE
•  Use your network!
Has worked well for us to get access in Germany, Spain,
NASA/Ames, Blue Waters, SDSC, …
•  Go beyond PRACE
Grants of ~100 million core-hours
Deadline spring 2016, 6 Int. projects
https://proposals.doeleadershipcomputing.org
Grants of 10-100 million core-hours
Call for proposals open; deadline November 2
http://www.hpci-office.jp/pages/e_h28_boshu
DeiC, Middelfart – Oktober 2015
Troels Haugbølle – [email protected]
Summary
•  Through international HPC Danish research and industry can
get access to substantial resources at all levels
•  Tier-1: 0.5-3 million core-hours
•  Tier-0: ~10 million core-hours (to increase in next call!)
•  …and beyond: at the ~100 million core-hour level
•  Universally based on scientific excellence
•  PRACE also give access to valuable training resources
•  PRACE is currently in a transition phase; this should be
resolved by 2016
•  Some scientific breakthroughs can only be achieved using
international resources ! Critical for the success of Danish
researchers, and a necessary complement to national HPC
DeiC, Middelfart – Oktober 2015
Troels Haugbølle – [email protected]