CS426/CS525 – Parallel Computing

Lecture Contents (Tentative)

Begins

Lecture Contents

Lecture Slides

Sep 18th

Introduction

  • Motivating Parallelism
  • Scope of Parallel Computing

 Set1

Sep 25th

Parallel Programming Platforms

  • Implicit Parallelism: Trends in Microprocessor Architectures
  • Limitations of Memory System Performance

 Set2

Oct 2nd

Parallel Programming Platforms

  • Dichotomy of Parallel Computing Platforms
  • Physical Organization of Parallel Platforms Communication Costs in Parallel Machines
  • Routing Mechanisms for Interconnection Networks

 

 

Oct 9th

Programming Using the Message Passing Paradigm

  • Principles of Message-Passing Programming
  • The Building Blocks: Send and Receive Operations
  • MPI: The Message Passing Interface

Set3

Oct 16th

Basic Communication Operations

  • One-to-All Broadcast and All-to-One Reduction
  • All-to-All Broadcast and Reduction
  • All-Reduce and Prefix-Sum Operations

MPI Links

Set4

Oct 23rd

Basic Communication Operations

  • Scatter and Gather
  • All-to-All Personalized Communication
  • Circular Shift
  • Improving the Speed of Some Communication Operations

Set5  

Project1 out

Oct 30th

Principles of Parallel Algorithm Design

  • Methods for Containing Interaction Overheads
  • Parallel Algorithm Models

Project 1 in

Project 2 out

Nov 6th

Analytical Modeling of Parallel Programs

  • Sources of Overhead in Parallel Programs
  • Performance Metrics for Parallel Systems
  • Effect of Granularity and Data Mapping on Performance
  • Scalability of Parallel Systems
  • Minimum Execution Time and Minimum Cost-Optimal Execution Time

Set6

Set7

Nov 13th

Programming Shared Address Space Platforms

  • Thread Basics
  • Tips for Designing Asynchronous Programs
  • History of OpenMP

Midterm EXAM I – Wed Nov. 15th at 13:30-15:20 during class hours

Project 2 in

Project 3 out

Nov 20th

Multicore programming

  • Compiling and Running OpenMP programs
  • Shared Memory Systems
  • Concepts in OpenMP

Set8 

Nov 27th

Programming Using the Message Passing Paradigm

  • Topologies and Embedding
  • Overlapping Communication with Computation
  • Collective Communication and Computation Operations
  • Groups and Communicators

Project 3 in

Project 4 out

Dec 4th

Parallel Programming Concepts

  • Coverage
  • Granularity
  • Locality

 

 Set9

Dec 11th

GPU programming

  • Hardware Overview
  • Performance
  • Software Environment – Programming Models

Set10

Set11

Dec 18th

Performance Optimizations

-          Midterm EXAM II – Wed Dec. 20th at 13:30-15:20 during class hours

-           20 December – Last Day of Classes

Set12

Dec 25th

Finals Period

-          22 December 2023, Friday -6 January 2023, Saturday

Project 4 in