Skip to content

CS492(D): Diffusion Models and Their Applications

Minhyuk Sung, KAIST, Fall 2024


Teaser

Time & Location

Time: Mon/Wed 1:00 p.m. - 2:15 p.m. (KST)
Location: Zoom / N1 Rm 201.

Zoom Link

Description

Recent breakthroughs in generative AI have amazed people with the unprecedented quality of generated images and videos, as exemplified by SORA, Midjourney, StableDiffusion, and many others. These advancements have been achieved using diffusion models, which have become the new standard technique for generative models. Diffusion models offer numerous advantages, including superior performance in the quality of generated outputs, as well as capabilities in conditional generation, personalization, zero-shot manipulation, and knowledge distillation.

In this course, we will discuss the theoretical foundations and practical applications of diffusion models. While the goal is to cover both theory and practice, the focus will be on gaining hands-on experience by implementing diffusion model techniques in programming assignments and solving real-world problems in the course project. There will be no midterm or final exams.

Course Staff

Instructor: Minhyuk Sung (mhsung@kaist.ac.kr)

Course Assistants:

Prerequisites

This course is designed for students with a fundamental understanding of deep learning and experience using PyTorch.

Grading

  • Programming Assignments: 45%
  • Project: 45%
  • In-Class Participation: 10%

Paper List

Paper List

Useful Resources

Important Dates

ALL ASSIGNMENTS ARE DUE 23:59 KST.

(Subject to Change)

  • 1st Programming Assignment: Due Sep 29 (Sun)
  • 2nd Programming Assignment: Due Oct 9 (Wed)
  • 3rd Programming Assignment: Due Oct 21 (Mon)
  • 4th Programming Assignment: Due Nov 5 (Tue)
  • 5th Programming Assignment: Due Nov 18 (Mon)
  • 6th Programming Assignment: Due Dec 6 (Fri)
  • 7th Programming Assignment: Due Dec 13 (Fri)
  • Project Proposal: Due Oct 19 (Sat)
  • Project Interim Report: Due Nov 9 (Sat)
  • Project Early Reporting Due: Due Nov 22 (Fri)
  • Project Submission: Due Nov 30 (Sat)

Schedule

(Subject to Change)

Week Mon Topic Wed Topic
1 Sep 02 Course Introduction
Slides
Sep 04 Introduction to Generative Models /
GAN / VAE
Slides
Recording
2 Sep 09 DDPM 1
Slides
Recording
Sep 11 DDPM 2
Slides
Recording
Assignment 1 Session
Slides
3 Sep 16 No Class (Chuseok) Sep 18 No Class (Chuseok)
4 Sep 23 DDIM 1
Slides
Recording
Sep 25 DDIM 2 / CFG
Slides
Recording
Assignment 2 Session
Slides
5 Sep 30 CFG / Latent Diffusion /
ControlNet / LoRA
Slides
Recording
Oct 02 No Class (Substitution of Hangul Day)
6 Oct 07 Zero-Shot Applications
Slides
Recording
Assignment 3 Session
Slides
Oct 09
Oct 10 (Thu)
4:00pm KST
Guest Lecture 1
Or Patashnik
Ph.D. Student at Tel-Aviv University
Recording
7 Oct 14 DDIM Inversion / Score Distillation 1
Slides
Recording
Oct 16 Score Distillation 2
Slides
Recording
Assignment 4 Session
Slides
8 Oct 21 No Class (Midterm Week) Oct 23 No Class (Midterm Week)
9 Oct 28 Diffusion Synchronization
Slides
Recording
Oct 30 Assignment 5 Session
Slides
10 Nov 04 Inverse Problems 1
Slides
Recording
Nov 06 Inverse Problems 2
Slides
Recording
Project Orientation Session
11 Nov 11 Probability Flow ODE / DPM-Solver
Slides
Recording
Nov 13 Assignment 6 Session
Slides
12 Nov 18 Flow Matching 1
Slides
Recording
Nov 20 Flow Matching 2
Slides
Recording
Assignment 7 Session
Slides
13 Nov 25 Course Summary
Slides
Recording
Nov 27 Guest Lecture 2
Jiaming Song
Chief Scientist at Luma AI
14 Dec 02 Project Presentations 1 Dec 04 Project Presentations 2
15 Dec 09 No Class (Conference Trip) Dec 11 No Class (Conference Trip)
16 Dev 16 No Class (Final Week) Dec 18 No Class (Final Week)

AI Coding Assistant Tool Policy

You are allowed (and even encouraged) to utilize AI coding assistant tools, such as ChatGPT, Copilot, Codex, and Code Intelligence, for your programming assignments and projects. Utilizing AI coding assistant tools will not be deemed as plagiarism. However, it is still strictly prohibited to directly copy code from the Internet or from someone else. Doing so will lead to a score of zero and a report to the university.