CS 554 / CSE 512: Parallel Numerical Algorithms
Fall 2019 (for the latest materials, see the 2021 course webpage)

Wed/Fri 11:00-12:15, 1131 Siebel

Instructor: Edgar Solomonik
4229 Siebel Center

TA: Samah Karim
4336 Siebel Center

Lecture Materials

Slides and notes are based on the Fall 2015 slides by Michael T. Heath

Chapter Notes Section Slides Video (Fall 2017)
Parallel Computing chapter 1 1. Parallel Computing slides lec 1.1 lec 2.1 lec 2.2 lec 3.1
Parallel Thinking chapter 2 2.1 Parallel Algorithm Design slides lec 3.2 lec 4.1
2.2 Parallel Programming slides lec 4.2
2.3 Parallel Performance slides lec 5 lec 6.1
Dense Linear Systems 3.1 Vector and Matrix Products slides lec 6.2 lec 7 lec 8.1
3.2 LU Factorization slides lec 8.2 lec 9 lec 10.1
3.3 Triangular Linear Systems slides lec 10.2
Sparse Linear Systems 4.1 Direct Methods slides lec 11 lec 12
4.2 Tridiagonal and Banded Matrices slides lec 13.1
4.3 Sparse Iterative Methods slides lec 13.2 lec 14 lec 15.1
Eigenvalue Problems 5.1 QR Factorization slides lec 15.2 lec 16.1
5.2 Eigenvalue Computation slides lec 16.2 lec 17.1
Matrix Models 6.1 Fast Fourier Transform slides lec 17.2 lec 18
6.2 Low Rank Approximation slides lec 19 lec 20
6.3 Numerical Optimization slides lec 21
Differential Equations 7.1 Ordinary Differential Equations slides lec 22 (first 15 min)
7.2 Partial Differential Equations slides lec 22 lec 23.1
7.3 Particle Methods slides lec 23.2 lec 24
7.4 Electronic Structure Calculations slides lec 25
7.5 Tensor Analysis slides lec 26

Course Information

Brief Course Description: Numerical algorithms for parallel computers: parallel algorithms in numerical linear algebra (dense and sparse solvers for linear systems and the algebraic eigenvalue problem), numerical handling of ordinary and partial differential equations, and numerical optimization techniques.


Homeworks and some course material will be released through on Piazza. sign-up for CS 554 on Piazza

Office Hours

Samah: 1-2 Wednesday, Edgar: 2-3 Friday


36% project, 22% homework, 18% midterm, 18% final, 6% participation, may be subject to curve


Submit initial proposal by Oct 11, revisions may be requested and will be due Oct 18, poster presentations will be at the end of the semester, probably final week and reports will be due at the end of the semester. Projects related to ongoing investigations or overlapping with other courses are encouraged, so long as they have some component related to this course.

Some student project posters from Fall 2017 are below

Thanks to everyone who volunteered to share their poster!

student project poster 1
student project poster 2
student project poster 3
student project poster 4
student project poster 5
student project poster 6


Midterm and final will be given in class, final is cumulative, but with focus on latter half of the material.


Homeworks will be due at 7 pm (email to swkarim2@illinois.edu and solomon2@illinois.edu, with CS 554 in title), can be submitted by 7 pm the next day with a 15% penalty or a week late with a 40% penalty

Group/Collaborative Work

Students are encouraged to work on homeworks and projects in small groups (2-4), those desiring to work in larger groups should confer with the instructor. All write-ups should be done independently with references to online materials where appropriate, university academic integrity guidelines apply.

Related/Previous Courses

The course is based on the previous versions of CS 554 / CSE 512 taught by Michael Heath, but with various changes to content, some new material, and different communication cost models. Also see James Demmel's course for much web-material on parallel numerical algorithms.

Michael Heath: Parallel Numerical Algorithms, 2015
James Demmel: Applications of Parallel Computers, 2015 (other years available)

For more advanced web-material on the theory of parallel algorithms, also see the following courses.

Satish Rao: Foundations of Parallel and Distributed Systems, 2012
Pavel Tvrdik: Topics in Parallel Computing, 1999
Guy Blelloch: Parallel Algorithms, 2009
E. S.: Communication Cost Analysis of Algorithms, 2016

For further information on parallel programming, see course material for CS 420 and CS 484.