Each module contains 3 ECTS. You choose a total of 10 modules/30 ECTS in the following module categories:
- 12-15 ECTS in technical scientific modules (TSM)
TSM modules teach profile-specific specialist skills and supplement the decentralised specialisation modules. - 9-12 ECTS in fundamental theoretical principles modules (FTP)
FTP modules deal with theoretical fundamentals such as higher mathematics, physics, information theory, chemistry, etc. They will teach more detailed, abstract scientific knowledge and help you to bridge the gap between abstraction and application that is so important for innovation. - 6-9 ECTS in context modules (CM)
CM modules will impart additional skills in areas such as technology management, business administration, communication, project management, patent law, contract law, etc.
In the module description (download pdf) you find the entire language information per module divided into the following categories:
- instruction
- documentation
- examination
The objective of this module is to provide the student with an introduction to parallel computing and algorithms. Students learn to parallelize systems at three different levels: Shared memory systems, distributed memory systems, and heterogeneous shared memory systems. For all three systems, which can be part of a high-performance cluster, students learn the necessary parallelization techniques and a few classical parallel algorithms.
This course includes hands-on work to train students in the application of parallel programming techniques and the programming and analysis of parallel algorithms.
Prerequisites
- Procedural programming (C/C++ knowledge is helpful
- Algorithms and data structures
- Basic notion of complexity analysis and big-O notation
- Basic notions of concurrent programming (Threads)
Learning Objectives
By the end of the course, students will know:
- the most common heterogeneous parallel hardware infrastructures
- different ways to model and efficiently program these architectures
- how to assess the performance of parallel algorithms
- how to efficiently implement parallel algorithms
- how to choose the right parallel algorithm to solve a specific problem on a specific parallel architecture
Contents of Module
Introduction (30%)
- Motivation and high-performance cluster model
- Basic concepts in parallel programming in C++
- Asymptotic analysis of parallel programs
- Performance metrics for parallel systems
- Decomposition and mapping techniques for load balancing
- Scalability of parallel systems
Shared memory systems (20%)
- Architectures of widely used multi-core systems
- Cache coherence in multiprocessor systems
- Parallel programming models (OpenMP)
- Parallel algorithms (numerical, sorting, graphs)
Distributed memory systems (30%)
- Architectures of distributed memory systems
- Communications models and communication costs
- Message passing paradigm (MPI)
- Collective operations and their costs
- Distributed algorithms (numerical, sorting)
Heterogeneous shared memory systems (20%)
- GPU architectures
- SIMD programming models (vectorization, SYCL)
- Matrix-vector and matrix-matrix multiplication
- GPU algorithms (numerical, image processing)
Teaching and Learning Methods
The most important basics are taught in theory lessons in face-to-face classes. In between, students work on the practical exercises at home and exchange information via online channels.
Literature
A. Introduction to Parallel Computing, Zbigniew J. Czech, Cambridge University Press, 2017
B. An Introduction to Parallel Programming, 1st edition, Peter Pacheco , Morgan Kaufmann Publishers Inc, 2011
Download full module description
Back