CS492(A): Machine Learning for 3D Data
Minhyuk Sung, KAIST, Spring 2022
Time & Location
Time: Mon/Wed 10:30am - 11:45am (KST)
Location: Online via Zoom
Description
3D Data (both 3D scans captured by depth sensors and 3D models created by designers) are widely used in many applications in computer vision, computer graphics, and robotic, such as autonomous driving, AI-assisted 3D object/scene design, augmented reality, and physical robot interaction. Along with the recent increasing demands on processing and analyzing such 3D data, there has been tremendous progress in developing novel technologies, especially based on deep learning. In this course, we will cover the recent advances in machine learning techniques for 3D data and also discuss the remaining challenges. Most of the course material will be less-than-5-year-old research papers in CVPR/ICCV/ECCV (Vision), SIGGRAPH/SIGGRAPH Asia (Graphics), and NeurIPS/ICML (Machine Learning). The course will be project-oriented (no exam, no paper-and-pencil homework, but easy programming assignment), and it will combine pedagogical lectures and seminar-style reading group presentations (followed by interactive discussions).
Prerequisites
This course is intended for undergraduate/master students who have a basic background in deep learning and experience with PyTorch.
Course Staff
Instructor: Minhyuk Sung (mhsung@kaist.ac.kr)
- Office hours: By appointment (See #announcements channel in Discord.)
Head Course Assistant : Juil Koo (63days@kaist.ac.kr)
Course Assistants
- Seungwoo Yoo (dreamy1534@kaist.ac.kr)
- Shinjeong Kim (aakseen@kaist.ac.kr)
Links
Course Website: https://mhsung.github.io/kaist-cs492a-spring-2022/
Zoom links, recordings, and slides: KLMS
Quizzes, Q&A, and communication: Discord (An invitation will be sent to the registered students by email.)
Programming Assignments: https://github.com/63days/kaist-3dml-assignments
Past Years:
(Spring 2021) https://mhsung.github.io/courses/kaist-cs492h-spring-2021/
Grading
- Project: 50%
- Paper Review Presentation: 30%
- Programming Assignments: 10%
- In-Class Participation: 10%
Course Logistics
(Last Update: Mar 20, 2022)
Important Dates
ALL ASSIGNMENTS ARE DUE 23:59 KST. NO LATE DAYS!
(Subject to Change)
- Lab Session: Mar 17 (Thu) 7pm KST
- Team Sign-Up: Due Mar 21 (Mon)
- 1st Programming Assignment: Due Mar 21 (Mon)
- 2nd Programming Assignment: Due Mar 30 (Wed)
- Project Proposal: Due Mar 30 (Wed)
- Project Pitch Video: Due Apr 11 (Mon)
- Presentation Slides: Due 4 days before your presentation date
- Project 1st Midterm Check-In: Due May 2 (Mon)
- Project 2nd Midterm Check-In: Due May 16 (Mon)
- Project Report/Poster/Code: Due Jun 06 (Mon)
- Project Review: Due Jun 11 (Sat)
- Project Rebuttal: Due Jun 15 (Wed)
- Project Final Decision: Due Jun 17 (Fri)
Paper Lists
KAIST 3D ML Paper List (Spring 2022)
https://github.com/timzhang642/3D-Machine-Learning
Schedule
(Subject to Change)
| Week | Mon | Topic | Wed | Topic |
|---|---|---|---|---|
| 1 | Feb 28 | Introduction Slides |
Mar 02 | 3D Shape Representations 1 Slides Recording |
| 2 | Mar 07 | Point Cloud Processing Slides Recording Paper: PointNet |
Mar 09 | No Class (Presidential Election) |
| 3 | Mar 14 | Point Cloud Generation Slides Recording Papers: PointNet++, DGCNN, Point Set Generation, Point Cloud AE/GAN Team Sign-Up Starts |
Mar 16 | 3D Shape Representations 2 Slides Recording Lab Session |
| 4 | Mar 21 | Implicit Functions Slides Recording Papers: DeepSDF, IM-NET, Occupancy Networks, Deep Meta Functionals, PiFU,DISN,SIREN, Fourier Feature Networks 1st Assignment Due Team Sign-Up Ends |
Mar 23 | Neural Rendering Slides Recording Papers: Neural Volumes, DeepVoxels, NeRF |
| 5 | Mar 28 | Co-Segmentation Slides Recording Papers: Deep Functional Dictionaries, BAE-NET, AdaCoSeg |
Mar 30 | Detection/Segmentation Slides Recording Papers: VoteNet, ImVoteNet, PointGroup 2nd Assignment Due Project Proposal Due |
| 6 | Apr 04 | Guest Lecture 1: Charles Ruizhongtai Qi Staff Research Scientist at Waymo |
Apr 06 | Unsupervised Decomposition Slides Recording Papers: BSP-Net, CvxNet, UCSG-Net, CSG-Stump |
| 7 | Apr 11 | Guest Lecture 2: Kaichun Mo Ph.D. Student at Stanford University Project Pitch Video Due |
Apr 13 | Project Pitches Video |
| 8 | Apr 18 | No Class (Midterm Week) | Apr 20 | No Class (Midterm Week) |
| 9 | Apr 25 | Paper Presentations 1/2 Architectures Point Cloud Transformer Point-GNN |
Apr 27 | Paper Presentations 3/4 Segmentation PointGroup DyCo3D |
| 10 | May 02 | Paper Presentations 5/6 Mesh Generation Pixel2Mesh Text2Mesh Project 1st Midterm Check-In Due |
May 04 | Paper Presentations 7/8 Single-Image-to-3D View Generalization GAN2Shape |
| 11 | May 09 | Paper Presentations 9/10: Equivariance / Implicit Representation 1 SE(3)-Transformers DualSDF |
May 11 | Paper Presentations 11/12: Implicit Representation 2 Deep Implicit Templates SPAGHETTI |
| 12 | May 16 | Paper Presentations 13/14: Implicit Representation 3 MetaSDF VolSDF Project 2nd Midterm Check-In Due |
May 18 | Paper Presentations 15/16: Neural Rendering 1 NeRF++ BARF |
| 13 | May 23 | Paper Presentations 17/18: Neural Rendering 2 NerFACE Instant NGP |
May 25 | Paper Presentations 19/20: Neural Rendering 3 Plenoxels StyleNeRF |
| 14 | May 30 | Course Summary Slides |
Jun 01 | No Class (Local Election) |
| 15 | Jun 06 | No Class (Memorial Day) Project Report/Poster/Code Due |
Jun 08 | Project Poster Presentations (Two Sessions) Project Review Due (Jun 11) |
| 16 | Jun 13 | No Class (Final Week) | Jun 15 | No Class (Final Week) |