CS 554 / CSE 512: Parallel Numerical Algorithms
Tue/Thu 11:00-12:15, 1103 Siebel

Instructor: Edgar Solomonik
4229 Siebel Center

Lecture Materials

Slides and notes are based on the Fall 2015 slides by Michael T. Heath

Chapter Notes Section Slides Video
Parallel Computing chapter 1 1. Parallel Computing slides lec 1.1 lec 2.1 lec 2.2 lec 3.1
Parallel Thinking chapter 2 2.1 Parallel Algorithm Design slides lec 3.2 lec 4.1
2.2 Parallel Programming slides lec 4.2
2.3 Parallel Performance slides lec 5 lec 6.1
Dense Linear Systems 3.1 Vector and Matrix Products slides lec 6.2 lec 7 lec 8.1
3.2 LU Factorization slides lec 8.2 lec 9 lec 10.1
3.3 Triangular Linear Systems slides lec 10.2
Sparse Linear Systems 4.1 Direct Methods slides lec 11 lec 12
4.2 Tridiagonal and Banded Matrices slides lec 13.1
4.3 Sparse Iterative Methods slides lec 13.2 lec 14 lec 15.1
Eigenvalue Problems 5.1 QR Factorization slides lec 15.2 lec 16.1
5.2 Eigenvalue Computation slides lec 16.2 lec 17.1
Matrix Models 6.1 Fast Fourier Transform slides lec 17.2 lec 18
6.2 Low Rank Approximation slides lec 19 lec 20
6.3 Numerical Optimization slides lec 21
Differential Equations 7.1 Ordinary Differential Equations slides lec 22 (first 15 min)
7.2 Partial Differential Equations slides lec 22 lec 23.1
7.3 Particle Methods slides lec 23.2 lec 24
7.4 Electronic Structure Calculations slides lec 25
7.5 Tensor Analysis slides lec 26

Course Information

Brief Course Description: Numerical algorithms for parallel computers: parallel algorithms in numerical linear algebra (dense and sparse solvers for linear systems and the algebraic eigenvalue problem), numerical handling of ordinary and partial differential equations, and numerical optimization techniques.


Homeworks and some course material will be released through on Piazza. sign-up for CS 554 on Piazza

Office Hours

3:00-4:00 pm Tuesdays and 1:30-2:30 pm Thursdays in SC 4229


30% project, 25% homework, 20% midterm, 20% final, 5% participation, may be subject to curve


Submit initial proposal by Oct 13, revisions may be requested and will be due Oct 20, poster presentations planned for final exam slot (Monday, Dec 18, 9-11 am), reports will be due at the end of the semester. Projects related to ongoing investigations or overlapping with other courses are encouraged, so long as they have some component related to this course.

Some student project posters are below

Thanks to everyone who volunteered to share their poster!

student project poster 1
student project poster 2
student project poster 3
student project poster 4
student project poster 5
student project poster 6


Midterm and final will be given in class, final is cumulative, but with focus on latter half of the material.


Homeworks will be due at 5 pm (email to solomon2@illinois.edu, with CS 554 or CSE 512 in title), can be submitted by 5 pm the next day with a 15% penalty or a week late with a 40% penalty

Group/Collaborative Work

Students are encouraged to work on homeworks and projects in small groups (2-4), those desiring to work in larger groups should confer with the instructor. All write-ups should be done independently with references to online materials where appropriate, university academic integrity guidelines apply.

Related/Previous Courses

The course will be similar to the previous versions of CS 554 / CSE 512 taught by Michael Heath, but the precise material covered and the analytical models will be different. Also see James Demmel's course for much web-material on parallel numerical algorithms.

Michael Heath: Parallel Numerical Algorithms, 2015
James Demmel: Applications of Parallel Computers, 2015 (other years available)

For more advanced web-material on the theory of parallel algorithms, also see the following courses.

Satish Rao: Foundations of Parallel and Distributed Systems, 2012
Pavel Tvrdik: Topics in Parallel Computing, 1999
Guy Blelloch: Parallel Algorithms, 2009
E. S.: Communication Cost Analysis of Algorithms, 2016

For further information on parallel programming, see course material for CS 420 and CS 484.