Introduction to High-Performance Computing

19 Aug 2013, 08:00 → 30 Aug 2013, 15:00 Europe/Stockholm

KTH main campus

The PDC- Center for High-Performance Computing and the KTH School of Computer Science and Communication (CSC) welcome you to our summer school introductory course on high-performance computing. This course is part of the Swedish National Graduate School in Scientific Computing (NGSSC) and is endorsed by the KTH Computational Science and Engineering Centre (KCSE) Graduate School, both of whom help to sponsor the summer school.

Interested students and researchers (with academic or industrial backgrounds) from all over the world are invited to apply to attend the course, which will be held at the KTH main campus in Stockholm during late August. (Please note that for KTH Masters students, the summer school is available as course number DN2258.)

This course provides the skills needed to utilize high-performance computing (HPC) resources, and includes an introduction to a range of important topics, such as:

• HPC programming languages, libraries and tools,
• modern computer architectures,
• parallel algorithms, and
• optimizing serial and parallel programs.

Case studies in various scientific disciplines will be used to help illustrate these topics. The course consists of both lectures and guided hands-on lab sessions. Participants who successfully complete the course (including the associated programming project) will be awarded 7.5 ECTS (European Credit Transfer and Accumulation System).

The course is suitable for scientists and graduate students who are interested in high-performance computing. Applicants must be able to communicate in English, and have previous programming experience.

2013 will be the 18th year that the course has been held – participants will become part of the long tradition of the PDC Summer School, which you can read about in the course pages for all the previous years.

Please note that there are a limited number of places available in the summer school, so be sure to register early! Course registration opens on March 15, and closes on June 1, 2013.

document 2013PaymentAuthorisation15000.pdf 2013PaymentAuthorisation3000.pdf

Monday, 19 August

08:30 → 09:15 Registration

09:15 → 10:15 Welcome to PDC and the Summer School

Speaker: Erwin Laure (PDC-HPC)

Slides Intro2013.pdf

10:15 → 10:30 Coffee break

10:30 → 12:15 Concepts and Algorithms for Scientific Computing

Speaker: Björn ENGQUIST

Slides PDC13-engquist.pdf

12:15 → 14:00 Picnic

14:00 → 15:00 Introduction to PDC's Environment

Speaker: Izhar UL HASSAN

Slides pdcintro.pdf

15:00 → 15:15 Coffee break

15:15 → 16:45 Lab: Introduction to PDC's Environment

Speaker: Izhar UL HASSAN

Tuesday, 20 August

09:15 → 10:00 High-Performance Computer Architecture

Speaker: Erik HAGERSTEN

Slides 1._Intro_and_pipelines.pdf 2._Caches_and_memory_systems.pdf

10:00 → 10:15 Coffee break

10:15 → 12:00 High-Performance Computer Architecture

Speaker: Erik HAGERSTEN

Slides 3._Multiprocessors.pdf 4._HW_optimizations.pdf

12:00 → 13:00 Individual lunch

13:00 → 15:00 High-Performance Computer Architecture

Speaker: Erik HAGERSTEN

Slides 5._Multicores.pdf 6._SW_optimizing.pdf

15:00 → 15:15 Coffee break

15:15 → 17:00 Lab: Introduction to PDC's Environment

Speaker: Izhar UL HASSAN

15:15 → 17:00 PDC Machine Room Tour

Wednesday, 21 August

09:15 → 10:00 Shared memory programming, OpenMP

Two common ways of speeding up programs are to use processes or threads executing in parallel. The second week of the Summer School presents process-based parallel computing using MPI. In this lecture parallelization using threads will be discussed. Threads can be managed in different ways, e.g. using a low level library, like the POSIX threads library. The lecture deals with OpenMP, which is a more convenient way to use threads. OpenMP is a specification for a set of compiler directives, library routines, and environment variables that can be used to specify shared memory parallelism in Fortran and C/C++ programs. The threaded program can be executed on a shared memory computer (e.g. a multi- core processor). Typically, the iterations of a time consuming loop are shared among a team of threads, each thread working on part of the iterations. The loop is parallelized by writing a directive in the code and a compiler or a pre- processor will generate parallel code. Using short code examples, the lecture covers the most important OpenMP-constructs, how to use them and how not to use them. A few more realistic examples are presented as well.

Speaker: Thomas ERICSSON

Slides OpenMP_4.pdf

10:00 → 10:15 Coffee break

10:15 → 12:00 Shared memory programming, OpenMP

Speaker: Thomas ERICSSON

12:00 → 13:00 Individual lunch

13:15 → 14:00 Shared memory programming, OpenMP

T

Speaker: Thomas ERICSSON

14:15 → 15:00 Lab: Programming Exercises on OpenMP

Speakers: Niclas JANSSON, Stefano MARKIDIS

15:00 → 15:15 Coffee break

15:15 → 16:00 Lab: Programming Exercises on OpenMP

Speakers: Niclas JANSSON, Stefano MARKIDIS

Thursday, 22 August

09:15 → 10:00 Code optimization

In this lecture techniques for speeding up code on a uniprocessor are presented. The lecture covers basic optimization principles such as using the memory hierarchy in an efficient way. Using short code examples, the lecture covers stride minimization, blocking, loop- fusion, loop- splitting and other approaches. The choice of programming language (Fortran or C), and compiler will be discussed. There are some hints of how to tune Matlab programs. Other topics are vector arithmetic for floating point and elementary functions, virtual memory, files and the LAPACK and BLAS libraries.

Speaker: Thomas ERICSSON

Slides Code_Opt_4.pdf

10:00 → 10:15 Coffee break

10:15 → 12:00 Code optimization

Speaker: Thomas ERICSSON

12:00 → 13:00 Individual lunch

13:15 → 15:00 Lab: OpenMP Advanced Project

Speakers: Niclas JANSSON, Stefano MARKIDIS

15:00 → 15:15 Coffee break

15:15 → 17:00 Lab: OpenMP Advanced Project

Speakers: Niclas JANSSON, Stefano MARKIDIS

Friday, 23 August

08:30 → 09:00 Wrap up: OpenMP Advanced Project

Speakers: Niclas JANSSON, Stefano MARKIDIS

09:15 → 10:00 GPU Architectures for Non-Graphics People

Today everyone is positioning GPUs for general purpose computing. They claim that you can get 10-100x speedups over conventional CPUs, and sometimes they're even right. However, to get the most out of current- (and next-) generation GPUs, one needs to understand the architectural differences and how they effect your choice of algorithm. In this talk I will cover GPU architecture in comparison to current CPUs, discuss the implications for getting good performance, and introduce OpenCL as a general-purpose programming language for accessing GPUs and CPUs today.

Speaker: David BLACK-SCHAFFER

Slides black-schaffer-gpu-architecture.pdf

10:00 → 10:15 Coffee break

10:15 → 11:00 GPUs: The Hype, The Reality, and The Future

We'll take a look at the promise and the results of porting applications to GPUs and try to distinguish the hype from the reality. We'll take a closer look at where GPUs fit into HPC today and in the near future, and try to do a fair comparison of GPU optimized programs, CPU optimized programs, and even take a look at competing accelerators such as the Intel Xeon Phi. The goal of this lecture is to give you some perspective on GPU technology and where it may (or may not) fit into HPC in the next few years.

Speaker: David Black-Schaffer

Slides black-schaffer-gpu-hype.pdf

11:00 → 12:00 Introduction to CUDA

CUDA is a parallel computing platform and programming model for NVIDIA GPU devices that allows to use the capabilities of the hardware efficiently. Lecture and lab give an overview of CUDA, discuss aspects of programming that needs the programmer's attention in order to gain the desired performance, and provide a first hands-on experience.

Speaker: Michael SCHLIEPHAKE

Slides

• 20130823_cuda_lecture01_(1).pdf
• 20130823_cuda_lecture02_(1).pdf
• 20130823_cuda_lecture03.pdf

12:00 → 13:00 Individual lunch

13:15 → 14:15 Lab: CUDA

Speaker: Michael SCHLIEPHAKE

14:15 → 15:00 Introduction to CUDA

Speaker: Michael SCHLIEPHAKE

15:00 → 15:15 Coffee break

15:15 → 16:00 Introduction to CUDA

Speaker: Michael SCHLIEPHAKE

16:00 → 17:00 Lab: CUDA

Speaker: Michael SCHLIEPHAKE

Monday, 26 August

09:15 → 10:00 MPI Introduction

Speaker: Erwin LAURE

Slides MPI-1.pdf

10:00 → 10:15 Coffee break

10:15 → 12:00 MPI Basic Concepts & Point-to-Point Communication

Speaker: Erwin LAURE

Slides MPI-2.pdf MPI-3.pdf

12:00 → 13:00 Individual lunch

13:15 → 15:00 Lab: MPI Part 1

Speaker: Jonathan VINCENT

15:00 → 15:15 Coffee break

15:15 → 17:00 Lab: MPI Part 1

Speaker: Jonathan VINCENT

Tuesday, 27 August

09:15 → 10:00 MPI - Collective Communication

Speaker: Erwin LAURE

Slides MPI-4.pdf

10:00 → 10:15 Coffee break

10:15 → 11:00 MPI - Intermediate MPI

Speaker: Erwin LAURE

Slides MPI-5.pdf

11:15 → 12:00 Performance Engineering

Speaker: Pekka MANNINEN

Slides Performance_engineering.pdf

12:00 → 13:00 Individual lunch

13:15 → 14:00 Performance Engineering

Speaker: Pekka MANNINEN

14:15 → 15:00 Lab: Performance Engineering

Speaker: Pekka MANNINEN

15:00 → 15:15 Coffee break

15:15 → 17:00 Lab: Performance Engineering

Speaker: Pekka MANNINEN

15:15 → 17:00 Project work

Discuss your project ideas with other students, the course examiner, and tutors.

Wednesday, 28 August

08:30 → 09:00 Wrap up: Performance Engineering

Speaker: Pekka MANNINEN

09:15 → 10:00 MPI: Advanced Concepts

Speaker: Erwin LAURE

Slides MPI-6.pdf

10:00 → 10:15 Coffee break

10:30 → 11:15 MPI: Advanced Concepts

Speaker: Erwin LAURE

11:15 → 12:00 Parallel Performance Engineering

Speaker: Pekka MANNINEN

12:00 → 13:00 Individual lunch

13:15 → 15:00 Lab: Parallel Performance Engineering

Speaker: Pekka MANNINEN

15:00 → 15:15 Coffee break

15:15 → 17:00 Lab: MPI Part 2

Speaker: Jonathan VINCENT

Thursday, 29 August

09:15 → 10:00 Interconnection Networks

Interconnect networks have large influence on the performance and efficiency of message-passing and File-I/O operations in cluster systems. The lecture presents concise information about interconnect networks in order to support the understanding of their influence and how to use it in favour of simulation applications.

Speaker: Michael SCHLIEPHAKE

Slides 20130828_summer_interconnect.pdf

10:00 → 10:15 Coffee break

10:15 → 11:00 Interconnection Networks

Speaker: Michael SCHLIEPHAKE

11:15 → 12:00 Case Study: Numerical experiments of turbulent flows on supercomputers

After a short description about the activities of the Swedish e-Science Research Centre (SeRC) a general motivation of studying turbulence is given, including elucidating the large range of scales needed to be resolved for high Reynolds numbers. The infrastructure needed to perform these simulations are discussed, including the simulation codes and large scale computational resources. Examples of such large scale simulations are then given in terms of high Reynolds number turbulent boundary layers on a flat plate, and also such a boundary layer with an obstacle. Finally the possibility to scale up the simulations to flows of direct aeronautical interest is discussed, e.g. when will it be possible to replace typical university wind tunnel experiments with DNS? This begs the question of the possibility of exa-scale computing and questions if the current methods can be used on such e-Infrastructure is posed?

Speaker: Philipp SCHLATTER (KTH)

Slides PDC-turbulence.pdf

12:00 → 13:00 Individual lunch

13:15 → 15:00 Lab: MPI Part 3

Speaker: Jonathan VINCENT

15:00 → 15:15 Coffee break

15:15 → 17:00 Lab: MPI Part 3

Speaker: Jonathan VINCENT

15:15 → 17:00 Project work

18:00 → 21:00 Social dinner

Friday, 30 August

09:15 → 10:00 Future Programming Languages

The lecture presents the current limitations of traditional programming models, such as MPI and openMP, and provides motivations to adopt new programming models for parallel computing. The potential of innovative programming models such as the Partitioned Global Address Space (PGAS) languages, and task-based programming models is discussed. Few examples of how to test on Lindgren these innovative programming models are also presented.

Speaker: Stefano MARKIDIS

Slides markidisPDC_SummerSchool2013.pdf

10:00 → 10:15 Coffee break

10:15 → 11:00 Future Programming Languages

Speaker: Stefano MARKIDIS

11:15 → 12:00 Abstractions for processing large data sets

At Google there is often a need to process very large data sets across many machines. Building efficient parallel processing programs is not trivial and to avoid the overhead of each engineer reinventing the wheel, Google has created several programming models to abstract away the complexities of parallelism and have the programmer concentrate on the core of his processing problem instead. I will talk mainly about the MapReduce model, how it works and the conceptual model the programmer has to work with when using it. I will also mention a bit about a more recent framework, Pregel, which is built for processing very large

Speaker: Jonas YNGVESSON

12:00 → 12:15 Individual lunch

13:00 → 17:00 Open Lab and Project Work