Introduction to High-Performance Computing

20 Aug 2012, 08:00 → 31 Aug 2012, 18:00 Europe/Stockholm

KTH main campus

Erwin Laure (PDC - Center for High-Performance Computing), Jesper Oppelstrup (CSC - KTH School of Computer Science and Communication), Michael Hanke (CSC - KTH School of Computer Science and Communication)

PDC- Center for High-Performance Computing and the KTH School of Computer Science and Communication (CSC) are organizing an introductory course in high-performance computing. This is a course within the National Graduate School in Scientific Computing (NGSSC) and is endorsed by the KTH Computational Science and Engineering Centre (KCSE) Graduate School. Both NGSSC and KCSE help sponsor the Summer School. The course is open to KTH masters students as course number DN2258. Other students, both national and international, as well as academic and industrial, are also welcome to apply.

This course will give an introduction to the skills needed to utilize high performance computing resources, including an introduction to HPC programming languages, libraries and tools, modern computer architectures, parallel algorithms, and optimization of serial and parallel programs. Case studies in various scientific disciplines will help illustrate these topics. The course consists of both lectures and guided hands-on lab experience. It carries 7.5 ECTS (European Credit Transfer and Accumulation System), where 1.5 ECTS credits are equivalent to one week's workload of 40 hours. The student receives these credits on successful completion of the post-course project. The number of seats are limited.

The course is suitable for scientist and graduate students with interest in high-performance computing. It requires good experience in programming. The course language is English.

Course registration opens on March 15, and closes June 1, 2012.

The Summer School will be held at the KTH main campus, Stockholm.

Participants will become part of the long tradition of the Summer School. This will be the 17th year it has been given. The course pages for all the previous years are online and may help give you an idea of what the course has been like.

document 2013PaymentAuthorisation15000.pdf 2013PaymentAuthorisation3000.pdf

Monday, 20 August

1 Registration

2 Welcome to PDC and the Summer School

Speakers: Erwin Laure (PDC), Michael Hanke (KTH)

Example Project Report JeanetteSpuhlerMeshSmoothing.pdf

Slides Intro2012.pdf

10:00 Coffee Break

3 Concepts and Algorithms for Scientific Computing

Speaker: Björn Engquist (KTH and the University of Austin)

Slides PDC-engquist-12.pdf

12:00 Individual Lunch

4 HPC Activities at Scania

Speaker: Mattias Chevalier (Scania)

5 Introduction to PDC's Environment

Speaker: Ul Hassan Izhar

Slides pdcintro.pdf

15:00 Coffee Break

6 Lab: Introduction to PDC's Environment

Speaker: Izhar Ul Hassan

17:00 Picnic

Tuesday, 21 August

7 High-Performance Computer Architecture

Speaker: Erik Hagersten (Uppsala University)

Slides 1._Intro_and_pipelines.pdf 2._Caches_and_memory_systems.pdf

10:00 Coffee Break

8 High-Performance Computer Architecture

Speaker: Erik Hagersten (Uppsala University)

Slides 3._Multiprocessors.pdf 4._HW_optimizations.pdf

12:00 Individual Lunch

9 High-Performance Computer Architecture

Speaker: Erik Hagersten (Uppsala University)

Slides 5._Multicores.pdf 6._SW_optimizing.pdf

15:00 Coffee Break

10 Lab: Introduction to PDC's Environment

Speaker: Izhar Ul Hassan

Lab instructions http://www.pdc.kth.se/education/tutorials/summer-school/introduction-to-pdcs-environment

11 PDC Machine Room Tour

Wednesday, 22 August

12 Shared memory programming, OpenMP

Two common ways of speeding up programs are to use processes or threads executing in parallel. The second week of the Summer School presents process-based parallel computing using MPI. In this lecture parallelization using threads will be discussed. Threads can be managed in different ways, e.g. using a low level library, like the POSIX threads library. The lecture deals with OpenMP, which is a more convenient way to use threads. OpenMP is a specification for a set of compiler directives, library routines, and environment variables that can be used to specify shared memory parallelism in Fortran and C/C++ programs. The threaded program can be executed on a shared memory computer (e.g. a multi- core processor). Typically, the iterations of a time consuming loop are shared among a team of threads, each thread working on part of the iterations. The loop is parallelized by writing a directive in the code and a compiler or a pre- processor will generate parallel code. Using short code examples, the lecture covers the most important OpenMP-constructs, how to use them and how not to use them. A few more realistic examples are presented as well.

Speaker: Thomas Ericsson (Chalmers University)

Slides OpenMP.pdf

10:00 Coffee Break

13 Shared memory programming, OpenMP

See preceding description.

Speaker: Thomas Ericsson (Chalmers University)

12:00 Individual Lunch

14 Shared memory programming, OpenMP

See preceding description.

Speaker: Thomas Ericsson (Chalmers University)

15 Lab: Programming Exercises on OpenMP

Speaker: Mr Niclas Jansson (CSC/NA)

Lab instructions http://www.pdc.kth.se/education/tutorials/summer-school/programming-exercises-on-openmp

15:00 Coffee Break

16 Lab: Programming Exercises on OpenMP

Speaker: Niclas Jansson (KTH)

Thursday, 23 August

17 Code optimization

In this lecture techniques for speeding up code on a uniprocessor are presented. The lecture covers basic optimization principles such as using the memory hierarchy in an efficient way. Using short code examples, the lecture covers stride minimization, blocking, loop- fusion, loop- splitting and other approaches. The choice of programming language (Fortran or C), and compiler will be discussed. There are some hints of how to tune Matlab programs. Other topics are vector arithmetic for floating point and elementary functions, virtual memory, files and the LAPACK and BLAS libraries.

Speaker: Thomas Ericsson (Chalmers University)

Slides Code_Opt.pdf

10:00 Coffee Break

18 Code optimization

See preceding description.

Speaker: Thomas Ericsson (Chalmers University)

12:00 Individual Lunch

19 Lab: OpenMP Advanced Project

Speaker: Mr Niclas Jansson (CSC/NA)

Lab instructions http://www.pdc.kth.se/education/tutorials/summer-school/openmp-advanced-project

15:00 Coffee Break

20 Lab: OpenMP Advanced Project

Speaker: Mr Niclas Jansson (CSC/NA)

Friday, 24 August

21 Wrap up: OpenMP Advanced Project

Speaker: Mr Niclas Jansson (CSC/NA)

22 GPU Architectures for Non-Graphics People

David Black-Schaffer (Uppsala University) Today everyone is positioning GPUs for general purpose computing. They claim that you can get 10-100x speedups over conventional CPUs, and sometimes they're even right. However, to get the most out of current- (and next-) generation GPUs, one needs to understand the architectural differences and how they effect your choice of algorithm. In this talk I will cover GPU architecture in comparison to current CPUs, discuss the implications for getting good performance, and introduce OpenCL as a general-purpose programming language for accessing GPUs and CPUs today.

Speaker: David Black-Schaffer (Uppsala University)

Slides

• 1-2012_GPU_Architecture.pptx.pdf
• 2-2012_Introduction_to_OpenCL.pptx.pdf
• 3-2012_OpenCL_Optimization.pptx.pdf
• 4-2012_GPU_Hype.pptx.pdf

10:00 Coffee Break

23 Introduction to OpenCL

Speaker: David Black-Schaffer (Uppsala University)

12:00 Individual Lunch

24 Lab: OpenCL

Speaker: David Black-Schaffer (Uppsala University)

15:00 Coffee Break

25 Lab: OpenCL

Speaker: David Black-Schaffer (Uppsala University)

Lab instructions http://www.pdc.kth.se/education/tutorials/summer-school/gpu-exercises

Monday, 27 August

26 MPI Introduction

Speaker: Erwin Laure (PDC-HPC)

Slides MPI-1.pdf

10:00 Coffee Break

27 MPI Basic Concepts & Point-to-Point Communication

Speaker: Erwin Laure (PDC-HPC)

Slides MPI-2.pdf MPI-3.pdf

12:00 Individual Lunch

28 Lab: MPI Part 1

Speaker: Jonathan Vincent (PDC)

Lab instructions http://www.pdc.kth.se/education/tutorials/summer-school/mpi-exercises/mpi-lab-1-program-structure-and-point-to-point-communication-in-mpi/mpi-lab-1-program-structure-and-point-to-point-communication-in-mpi

15:00 Coffee Break

29 Lab: MPI Part 1

Speaker: Jonathan Vincent (PDC)

Tuesday, 28 August

30 MPI - Collective Communication

Speaker: Erwin Laure (PDC-HPC)

Slides MPI-4.pdf

10:00 Coffee Break

31 MPI - Intermediate MPI

Speaker: Erwin Laure (PDC-HPC)

Slides MPI-5.pdf

32 Performance Engineering

Speaker: Pekka Manninen (CSC, Finland)

Slides Performance_engineering.pdf

12:00 Individual Lunch

33 Performance Engineering

Speaker: Pekka Manninen (CSC, Finland)

34 Lab: Performance Engineering

Speaker: Pekka Manninen (CSC, Finland)

Lab instructions http://www.pdc.kth.se/education/tutorials/summer-school/performance-engineering-lab

15:00 Coffee Break

35 Lab: Performance Engineering

Speaker: Pekka Manninen (CSC, Finland)

36 Project Work

Discuss your project ideas with other students, the course examiner, and tutors.

Speakers: Jesper Oppelstrup (KTH), Michael Hanke (KTH)

Wednesday, 29 August

37 Wrap up: Performance Engineering

Speaker: Pekka Manninen (CSC, Finland)

38 MPI: Advanced Concepts

Speaker: Erwin Laure (PDC-HPC)

Slides MPI-6.pdf

10:00 Coffee Break

39 MPI: Advanced Concepts

Speaker: Erwin Laure (PDC-HPC)

40 Parallel Performance Engineering

Speaker: Pekka Manninen (CSC, Finland)

12:00 Individual Lunch

41 Lab: Parallel Performance Engineering

Speaker: Pekka Manninen (CSC, Finland)

15:00 Coffee Break

42 Lab: MPI Part 2

Speaker: Jonathan Vincent (PDC)

Lab instructions http://www.pdc.kth.se/education/tutorials/summer-school/mpi-exercises/mpi-lab-2-collective-and-non-blocking-communication/mpi-lab-2-collective-and-non-blocking-communication

Thursday, 30 August

43 Interconnection Networks

Interconnect networks have large influence on the performance and efficiency of message-passing and File-I/O operations in cluster systems. The lecture presents concise information about interconnect networks in order to support the understanding of their influence and how to use it in favour of simulation applications.

Speaker: Michael Schliephake (PDC)

10:00 Coffee Break

44 Interconnection Networks

Speaker: Michael Schliephake (PDC)

Slides 20120830_summer_interconnect_(1).pdf

45 Software co-design and the exascale challenge

The next frontier of High Performance Computing is the exascale. By the early part of the 2020s we expect computers to exist that can perform 1018 calculations per second. In order to achieve this goal the systems will be incredibly complex with many millions of processor cores and complex memory and communication hierarchies. Programming such systems will be extremely challenging. This lecture will focus on the use of software co-design to prepare applications for exascale systems. It will explain how we need to use incremental and disruptive approaches to meet the many challenges posed by such systems and discuss some of the key issues we face over the next decade at the frontier of supercomputing.

Speaker: Mark Parsons (EPCC)

Slides PDC_Summer_School_Parsons.pdf

12:00 Individual Lunch

46 Lab: MPI Part 3

Speaker: Jonathan Vincent (PDC)

Lab instructions http://www.pdc.kth.se/education/tutorials/summer-school/mpi-exercises/mpi-lab-3-bandwidth-latency-and-timings/mpi-lab-3-bandwidth-latency-and-timings

15:00 Coffee

47 Lab: MPI Part 3

Speaker: Jonathan Vincent (PDC)

48 Project Work

Speakers: Jesper Oppelstrup (KTH), Michael Hanke (KTH)

18:00 Summing Up Dinner

Friday, 31 August

49 Future Programming Languages

The lecture presents the current limitations of traditional programming models, such as MPI and openMP, and provides motivations to adopt new programming models for parallel computing. The potential of innovative programming models such as the Partitioned Global Address Space (PGAS) languages, and task-based programming models is discussed. Few examples of how to test on Lindgren these innovative programming models are also presented.

Speaker: Stefano Markidis (PDC)

Slides markidisPDC_SummerSchool2012.pdf

10:00 Coffee Break

50 Future Programming Languages

The lecture presents the current limitations of traditional programming models, such as MPI and openMP, and provides motivations to adopt new programming models for parallel computing. The potential of innovative programming models such as the Partitioned Global Address Space (PGAS) languages, and task-based programming models is discussed. Few examples of how to test on Lindgren these innovative programming models are also presented.

Speaker: Stefano Markidis (PDC)

51 Abstractions for processing large data sets

At Google there is often a need to process very large data sets across many machines. Building efficient parallel processing programs is not trivial and to avoid the overhead of each engineer reinventing the wheel, Google has created several programming models to abstract away the complexities of parallelism and have the programmer concentrate on the core of his processing problem instead. I will talk mainly about the MapReduce model, how it works and the conceptual model the programmer has to work with when using it. I will also mention a bit about a more recent framework, Pregel, which is built for processing very large graph models.

Speaker: Jonas Yngvesson (Google)

12:00 Individual Lunch

52 Open Lab and Project Work