Introduction to High-Performance Computing

16 Aug 2010, 08:00 → 27 Aug 2010, 18:00 Europe/Stockholm

E3 (KTH main campus)

Erwin Laure (PDC - Center for High-Performance Computing), Jesper Oppelstrup (CSC - KTH School of Computer Science and Communication), Michael Hanke (CSC - KTH School of Computer Science and Communication)

The Center for High-Performance Computing (PDC) and the KTH School of Computer Science and Communication (CSC) are organizing an introductory course in high-performance computing. This is a course within the National Graduate School in Scientific Computing (NGSSC) and is endorsed by the KTH Computational Science and Engineering Centre (KCSE) Graduate School. Both NGSSC and KCSE help sponsor the Summer School. The course is open to KTH masters students as course number DN2258. Other students, both national and international, as well as academic and industrial, are also welcome to apply.

This course will give an introduction to the skills needed to utilize high performance computing resources, including an introduction to HPC programming languages, libraries and tools, modern computer architectures, parallel algorithms, and optimization of serial and parallel programs. Case studies in various scientific disciplines will help illustrate these topics. The course consists of both lectures and guided hands-on lab experience. It carries 7.5 ECTS (European Credit Transfer and Accumulation System), where 1.5 ECTS credits are equivalent to one week's workload of 40 hours. The student receives these credits on successful completion of the post-course project. The number of seats are limited.

The course is suitable for scientist and graduate students with interest in high-performance computing. It requires good experience in programming. The course language is English.

Course registration opened March 15, 2010; registration closed on June 4, 2010.

The Summer School will be held at the KTH main campus, Stockholm.

Participants will become part of the long tradition of the Summer School. This will be the 15th year it has been given. The course pages for all the previous years are online and may help give you an idea of what the course has been like.

Monday, 16 August

Registration

Welcome to PDC and the Summer School

1 Welcome to PDC and the Summer School

Speakers: Erwin Laure (PDC - Center for High-Performance Computing), Jesper Oppelstrup (CSC - KTH School of Computer Science and Communication)

Slides (EL) Intro2010_el.pdf

10:00 Break

The Current State of Computing

2 Lecture: Massively Parallel Computations: Where Are We Now?

Speaker: Sverker Holmgren (Uppsala University and SNIC)

Slides para_comp_status_report_sh.pdf

Introduction to PDC's Environment: Lecture

3 Lecture: Introduction to PDC's Environment

Speaker: Elisabet Molin (PDC - Center for High-Performance Computing)

Printable slides pdcintro_nooverlays.pdf

Slides pdcintro.pdf

12:00 Lunch

High-Performance Computer Architecture: Lecture

4 High-Performance Computer Architecture

Speaker: Erik Hagersten (Uppsala University)

Slides 1_Intro_and_UP_eh.pdf

15:00 Break

5 High-Performance Computer Architecture

Speaker: Erik Hagersten (Uppsala University)

Slides 2_MP_eh.pdf

17:30 Get-together Picnic

Tuesday, 17 August

09:15 Message of the Day

High-Performance Computer Architecture: Lecture

6 High-Performance Computer Architecture

Speaker: Erik Hagersten (Uppsala University)

Slides 3_MC_eh.pdf

10:00 Break

7 High-Performance Computer Architecture

Speaker: Erik Hagersten (Uppsala University)

Slides 4_Optimizing_for_Speed_eh.pdf

Algorithms and Data Structures

8 Lecture: Algorithms and Data Structures

The goal of the lecture is to present the overarching strategy of the full scientific computing process that include the mathematical formulation of a problem, the numerical algorithm with appropriate data structures and the implementation on modern computer architectures. We will discuss distributed computing for different classes of algorithms. Examples are techniques based on domain decomposition and the effect of causality on parallel algorithms.

Speaker: Björn Engquist (KTH and the University of Austin)

Slides PDC-Engquist.pdf

12:00 Lunch

9 Algorithms and Data Structures

Speaker: Björn Engquist (KTH and the University of Austin)

Introduction to PDC's Environment: Lab

10 Lab: Getting Acquainted with the PDC Environment

Speaker: Elisabet Molin (PDC - Center for High-Performance Computing)

General Instructions http://www.pdc.kth.se/education/tutorials/summer-school/general-instructions

Specific Lab Instructions http://www.pdc.kth.se/education/tutorials/summer-school/introduction-to-pdcs-environment

15:00 Break

11 Lab: Getting Acquainted with the PDC Environment

Speaker: Elisabet Molin (PDC - Center for High-Performance Computing)

Machine Room Tour

Introduction to PDC's Environment: Non-obligatory Lab

Wednesday, 18 August

09:15 Message of the Day

OpenMP: Lecture

12 Shared Memory Parallelism

Speaker: Thomas Ericsson (Chalmers)

Slides openmp_te.pdf

Source http://www.math.chalmers.se/~thomas/PDC/

10:00 Break

13 Shared Memory Parallelism

Speaker: Thomas Ericsson (Chalmers)

12:00 Lunch

14 Shared Memory Parallelism

Speaker: Thomas Ericsson (Chalmers)

OpenMP: Lab (Intro)

15 Lab: Programming exercises on OpenMP

Speaker: Niclas Jansson (KTH)

General Instructions http://www.pdc.kth.se/education/tutorials/summer-school/general-instructions

Specific Lab Instructions http://www.pdc.kth.se/education/tutorials/summer-school/programming-exercises-on-openmp

15:00 Break

16 Lab: Programming exercises on OpenMP

Speaker: Niclas Jansson (KTH)

Non-obligatory Open Lab

Thursday, 19 August

09:15 Message of the Day

Efficient Programming

17 Writing Efficient Programs

Speaker: Thomas Ericsson (Chalmers)

Paper springer_te.pdf

Slides Slides

Source http://www.math.chalmers.se/~thomas/PDC/

10:00 Break

18 Writing Efficient Programs

Speaker: Thomas Ericsson (Chalmers)

12:00 Lunch

OpenMP: Lab (Project)

19 Lab: OpenMP advanced project

Speaker: Niclas Jansson (KTH)

General Instructions http://www.pdc.kth.se/education/tutorials/summer-school/general-instructions

Specific Lab Instructions http://www.pdc.kth.se/education/tutorials/summer-school/openmp-advanced-project

15:00 Break

20 Lab: OpenMP advanced project

Speaker: Niclas Jansson (KTH)

Non-obligatory Open Lab

Friday, 20 August

OpenMP: Wrap up

21 Wrap-up: OpenMP advanced project

Speaker: Niclas Jansson (KTH)

GPU

22 GPU Architectures for Non-Graphics People

Today everyone is positioning GPUs for general purpose computing. They claim that you can get 10-100x speedups over conventional CPUs, and sometimes they're even right. However, to get the most out of current- (and next-) generation GPUs, one needs to understand the architectural differences and how they effect your choice of algorithm. In this talk I will cover GPU architecture in comparison to current CPUs, discuss the implications for getting good performance, and introduce OpenCL as a general-purpose programming language for accessing GPUs and CPUs today.

Speaker: David Black-Schaffer (Uppsala University)

Slides KTH_GPU_Architecture_dbs.pdf

10:00 Break

23 Introduction to OpenCL

See the description for "GPU Architectures for Non-Graphics People computing."

Speaker: David Black-Schaffer (Uppsala University)

Slides KTH_Introduction_to_OpenCL_dbs.pdf

12:00 Lunch

Performance Engineering: Lecture: Serial Performance

24 Lecture: Performance Engineering: Serial Performance

Speaker: Dag Lindbo (KTH)

How to hpc10_papiex_howto.pdf

Slides hpc10_serial_perf_measurement.pdf

Performance Engineering: Lab: Serial Performance

25 Lab: Performance Engineering: Serial Performance

Speaker: Dag Lindbo (KTH)

General Instructions http://www.pdc.kth.se/education/tutorials/summer-school/general-instructions

Specific Lab Instructions http://www.pdc.kth.se/education/tutorials/summer-school/performance-engineering-lab

Wrap up serial_code_eff_wrap_up_dl.pdf

15:00 Break

26 Lab: Performance Engineering: Serial Performance

Speaker: Dag Lindbo (KTH)

MATMUL Competition hpc10_matmul_comp.pdf

Non-obligatory Open Lab

Saturday, 21 August

Project Work

Monday, 23 August

MPI: Lecture: Basics

27 Lecture: MPI Basics

This lecture gives you an overview of MPI functionality and discuss the value of having a standard message passing library. It provides a tailored tour of the basic concepts and mechanisms that will be of most value to you as beginning MPI programmers. This includes gaining a familiarity with some of the most commonly used MPI calls, the way in which MPI is initialized and terminated, the structure of messages, and the communicator concept. At the conclusion of this material, you can expect to be conversant with a basic set of six MPI calls necessary to begin designing and coding your own MPI applications.

Speaker: Lilit Axner (PDC - Center for High-Performance Computing)

More information http://www.pdc.kth.se/publications/talks/mpi/basics

Slides mpi_basics_la.pdf

10:00 Break

MPI: Lecture: Point-to-Point

28 Lecture: MPI Point-to-Point Communications I

In point to point communication, one process sends a message and a second process receives it. This is in contrast to collective communication routines, in which a pattern of communication is established amongst a group of processes. This lecture will cover the different types of send and receive routines available for point to point communication.

Speaker: Lilit Axner (PDC - Center for High-Performance Computing)

More information http://www.pdc.kth.se/publications/talks/mpi/pt2pt-i

Slides mpi_p2p_la.pdf

Project Work: Further Details and Electronic Submission

29 Project Work Requirements and Expectations

Speaker: Jesper Oppelstrup (CSC - KTH School of Computer Science and Communication)

More information http://www.pdc.kth.se/training/2010/SummerSchool/projects.html

Slides hpc10notes.pdf

30 Using the Project Submission System

This is a brief overview of how to use the course project submission system to submit your project report and follow its progress as it get reviewed.

Speaker: Michael Hammill (PDC - Center for High-Performance Computing)

More information http://www.pdc.kth.se/training/2010/SummerSchool/projects.html

Slides using_the_proj_sub_sys.pdf

12:00 Lunch

MPI: Lab: MPI Basics

31 Lab: MPI Basics

Lab exercises to accompany the material on the basics of MPI programming

Speaker: Lilit Axner (PDC - Center for High-Performance Computing)

General Instructions http://www.pdc.kth.se/education/tutorials/summer-school/general-instructions

Specific Lab Instructions http://www.pdc.kth.se/education/historical/publications/talks/mpi/basics-lab/lab.html

15:00 Break

MPI: Lab: Point-to-Point

32 Lab: MPI Point-to-Point Communications I

Lab exercises to accompany the material titled "Point-to- Point Communication I"

Speaker: Lilit Axner (PDC - Center for High-Performance Computing)

General Instructions http://www.pdc.kth.se/education/tutorials/summer-school/general-instructions

Specific Lab Instructions http://www.pdc.kth.se/publications/talks/mpi/pt2pt-i-lab

Project Work

Non-obligatory Open Lab

Tuesday, 24 August

MPI: Wrap up: MPI Basics and Point-to-Point

33 Wrap-up: MPI Basics and Point-to-Point Communications I

During this question and answer period, we will discuss questions that came up from yesterday's lectures and labs on MPI Basics and Point-to-Point Communication.

Speaker: Lilit Axner (PDC - Center for High-Performance Computing)

MPI: Lecture: Point-to-Point II

34 Lecture: MPI Point-to-Point Communications II

MPI Point to Point Communication I introduced many of the routines related to sending a message between two processes, with a focus on programming with blocking and non-blocking routines. This lecture will discuss another important area: communication mode. Choosing a communication mode gives the programmer some control over how the system handles the message and can improve communication performance.

Speaker: Michaela Lechner (PDC - Center for High-Performance Computing)

More information http://www.pdc.kth.se/publications/talks/mpi/pt2pt-ii

Slides p2p2_ml.pdf

10:00 Break

MPI: Lecture: Collective Communication

35 Lecture: MPI Collective Communications

There are certain communication patterns that appear in many different types of applications. Rather than requiring each programmer to code these using point to point communication, MPI provides routines that handle these patterns for you, called collective communication routines. This lecture will survey these routines, the communication pattern established, and their syntax.

Speaker: Michaela Lechner (PDC - Center for High-Performance Computing)

More information http://www.pdc.kth.se/publications/talks/mpi/collective-i

Slides collective_com_ml.pdf

Case Study: Challenges and Progress in Computational Materials Science

36 Challenges and Progress in Computational Materials Science

Speaker: Pavel Korzhavyi (MSE - KTH, Department of Materials Science and Engineering)

Slides Korzhavyi_PDC10.pdf

12:00 Lunch

MPI: Lab: Point-to-Point II

37 Lab: MPI Point-to-Point Communications II

The lab that accompanies the material titled "Point-to-Point Communication II"

Speaker: Michaela Lechner (PDC - Center for High-Performance Computing)

General Instructions http://www.pdc.kth.se/education/tutorials/summer-school/general-instructions

Specific Lab Instructions http://www.pdc.kth.se/publications/talks/mpi/pt2pt-ii-lab

15:00 Break

MPI: Lab: Collective Communications

38 Lab: MPI Collective Communications

The lab that accompanies the material titled "Collective Communication I"

Speaker: Michaela Lechner (PDC - Center for High-Performance Computing)

General Instructions http://www.pdc.kth.se/education/tutorials/summer-school/general-instructions

Specific Lab Instructions http://www.pdc.kth.se/publications/talks/mpi/collective-i-lab

Project Work

Non-obligatory Open Lab

Wednesday, 25 August

MPI: Wrap up: MPI Point-to-Point II and Collective Communication

39 Wrap-up: MPI Point-to-Point Communications II and Collective Communication

During this question and answer period, we will discuss questions that came up from yesterday's lectures and labs on MPI Point-to-Point Communication II and Collective Communication.

Speaker: Michaela Lechner (PDC - Center for High-Performance Computing)

High-Performance Computer Architecture: Lecture

40 Interconnection Networks

The lecture deals with interconnect networks in parallel computers. Starting with a presentation of general principles and widespread network technologies we introduce important criterions for network topologies. A look to routing and switching strategies leads us to a basic idea over the costs of message transports in networks. In the second part of the lecture we will present examples how communication operations may be implemented. They allow to get an idea how the interconnect influences the performance and scalability of parallel applications and how these can be improved.

Speaker: Michael Schliephake (PDC - Center for High-Performance Computing)

Slides 20100825_summer_interconnect.pdf

10:00 Break

41 Interconnection Networks

See previous description for "Interconnection Networks"

Speaker: Michael Schliephake (PDC - Center for High-Performance Computing)

Performance Engineering: Lecture: Parallel Performance

42 Lecture: Performance Engineering: Parallel Performance

Speaker: Elisabet Molin (PDC - Center for High-Performance Computing)

Printable slides mpiperf_noaccumulate.pdf

Slides mpiperf.pdf

12:00 Lunch

Performance Engineering: Lab: Parallel Performance

43 Lab: Performance Engineering: Parallel Performance

Speaker: Elisabet Molin (PDC - Center for High-Performance Computing)

General Instructions http://www.pdc.kth.se/education/tutorials/summer-school/general-instructions

Specific Lab Instructions http://www.pdc.kth.se/education/tutorials/summer-school/mpi-performance-tools

15:00 Break

Case Study: Visualization

44 Case Study in Visualization

Speaker: Anders Ynnerman (Linköpings Universitet)

16:15 Break

Non-obligatory Open Lab

Thursday, 26 August

Performance Engineering: Wrap up

45 Wrap-up: Performance Engineering

Speaker: Elisabet Molin (PDC - Center for High-Performance Computing)

MPI: Lecture: Hybrid (MPI + OpenMP)

46 Lecture: MPI Hybrid (MPI+OpenMP)

When OpenMP is used in conjunction with the Message Passing Interface (MPI), the result can provide a second level of parallelism which has the possibility to gain the greater efficiency on clusters of SMP nodes. In this lecture, we will show how to do such hybird programming by using examples.

Speaker: Olav Vahtras (PDC - Center for High-Performance Computing)

Slides hybrid-mh.pdf

10:00 Break

47 Lecture: MPI Hybrid (MPI+OpenMP)

A continuation of the lecture on MPI + OpenMP (i.e., hybrid) programming

Speaker: Olav Vahtras (PDC - Center for High-Performance Computing)

Slides hybrid-mh.pdf

Case Study: Green Computing

48 Green Computing

Speaker: Lennart Johnsson (PDC - Center for High-Performance Computing, and University of Houston)

Slides green-comp-lj.pdf

12:00 Lunch

MPI: Lab: Hybrid (MPI + OpenMP)

49 Lab: MPI Hybrid (MPI+OpenMP)

In this lab exercise, you will parallelize some simple algorithms using shared memory programming (OpenMP) and distributed memory programming (MPI) simultaneously.

Speaker: Olav Vahtras (PDC - Center for High-Performance Computing)

General Instructions http://www.pdc.kth.se/education/tutorials/summer-school/general-instructions

Specific Lab Instructions http://www.pdc.kth.se/publications/talks/mpi/hybrid-lab/advanced-programming-lab-hybrid-openmp-mpi-programming

15:00 Break

Non-obligatory Open Lab

18:00 Summing up Dinner

Friday, 27 August

MPI: Wrap up: Hybrid (MPI + OpenMP)

50 Wrap up: Hybrid (MPI + OpenMP)

During this question and answer period, we will discuss questions that came up from yesterday's lectures and labs on MPI + OpenMP (i.e., hybrid) programming.

Speaker: Olav Vahtras (PDC - Center for High-Performance Computing)

MPI: Lecture: Virtual Topologies

51 Lecture: MPI Virtual Topologies

A virtual topology is a mechanism for naming the processes in an MPI communicator in a way that fits the communication pattern better. In this lecture we cover the basic concept behind virtual topologies, the main MPI calls used to achieve it, and some examples.

Speaker: Olav Vahtras (PDC - Center for High-Performance Computing)

General Instructions http://www.pdc.kth.se/education/tutorials/summer-school/general-instructions

Optional lab http://www.pdc.kth.se/publications/talks/mpi/lab-virtual-topologies

Slides virt_top_derived_dt-mh.pdf

10:00 Break

52 Lecture: MPI Virtual Topologies

A continuation of the lecture on MPI virtual topologies

Speaker: Olav Vahtras (PDC - Center for High-Performance Computing)

General Instructions http://www.pdc.kth.se/education/tutorials/summer-school/general-instructions

Optional lab http://www.pdc.kth.se/publications/talks/mpi/lab-virtual-topologies

Slides virt_top_derived_dt-mh.pdf

Case Study: Future Computing Environments

53 New Languages for High Performance Computing

This lecture will introduce new programming paradigms for the multi- and many-core area as well as various languages to program for hardware accelerators.

Speaker: Iris Christadler (LRZ - Leibniz-Rechenzentrum)

Slides NewLanguages4HPC_ic.pdf

12:00 Farewell

12:15 Lunch

Non-obligatory Open Lab

15:00 Break

Project Work

15:00 Break