Lecture notes

Lecture notes

Week 1:

Lectures 1 & 2: Introduction

-- history and reasons for paradigm shifts in supercomputing

-- structure of class, expectations, goals, etc

-- Homework 1

Week 2:

Lecture 3: Primers -- grape, hummingbird, Linux, vi

Lecture 4: Primers -- Fortran (slides from CSC)

OTHER USEFUL MATERIAL:

Linux primers/cheat sheets:

Linux cheat sheet

Another Linux (Bash) cheat sheet

Fortran tutorials:

Fortran tutorial from AMS 129 (Prof. Dongwook Lee;Youngjun Lee):

https://users.soe.ucsc.edu/~ylee/18fall/ams129/chapters/chapter02.html

Another Fortran tutorial to work through (Prof. Pascale Garaud):

This is a nice tutorial to learn from too: https://www.tutorialspoint.com/fortran/index.htm

Week 3:

Lectures 5 & 6: Theory Chapter 1: Parallel machine models and parallel programming models

Week 4:

Lecture 7 : Batch job schedulers -- PBS on grape and SLURM on Hummingbird

Lecture 8: MPI

Week 5:

Lecture 9, 10 : MPI (continued)

Other useful MPI material links (some that I used as sources for mine):

Basic MPI

Blaise Barney's LLNL MPI online tutorial

Bill Gropp's intro to MPI lectures

(and here is a link to more from Bill and others associated withe Argonne National Lab)

Advanced MPI

MPI advanced features from CINECA -- derived datatypes, groups, splitting communicators and processor topologies

Another one from U Tenn similar to the above but does one-sided comm and IO too plus some performance analysis

MPI-IO from TACC

Week 6:

Lecture 11: Theory Chapter 2: Designing parallel algorithms -- PCAM

Lecture 12: PCAM (continued) and PCAM in-class exercises

Week 7:

Tuesday -- no class!

Lecture 13: Theory Chapter 3: A quantitative basis for design

Week 8:

Lecture 14: OpenMP

Here are some other useful links for OpenMP stuff:

F95+OpenMP from Ploytecnica de Madrid

Advanced OpenMP topics (Tasks, Nested, ..) from NASA

A set of OpenMP examples from the OpenMP 4.5 API specification

Week 9:

Lecture 15: Theory Chapter 4: Putting components together

Lecture 16: Part 1: Performance analysis and debugging

Lecture 16: Part 2: Other programming tools -- HPF, CoArray Fortran etc

Here are some other useful links for this stuff:

General profiling

Jumpshot, Vampir, Tau

The rise and fall of HPF

CoArray Fortran 1

CoArray Fortran 2

Week 10:

Dear All,

*** DON'T FORGET TO HELP OUT FUTURE GENERATIONS OF THIS CLASS BY FILLING IN YOUR SET'S (EVALS)!! *** Dedicate some of the class time this week to getting that done!

Week 10 is here. This week we concentrate on getting the final projects finished and fleshing out some of our understanding of parallel computing fundamentals through guest speakers.

FOR TUESDAY:

(a) I will be available during class time for questions, comments, consultation on anything to do with the course, including final projects etc.

(b) Our guest speaker will be this video by Prof. James Demmel, Berkeley on parallel marix multiplication. This will help you understand how complex it is to make things go as fast as library routines do! https://www.youtube.com/watch?v=ECo-5FN5sTI

Lecture 17: Case Study: Parallel Matrix Multiply (online lecture by James Demmel, UCB)

FOR THURSDAY:

(a) I will be available again during class time for questions, comments, consultation on anything to do with the course, including final projects etc.

(b) Pick another video of your choice from the Berkeley CS267 series. You can access the latest versions (Spring 2020) here and last years versions (Spring 2019) here. There are great guest lectures on all kinds of topics ranging from cloud computing to computational biology. Your homework is simply to tell me which one you watched.

(c) Everyone must join the regular Zoom class on Thursday at 12:45pm for final comments and questions.

Any questions, drop me a line!

Nic