Warning: This is the previous version of the class.
Click here for the webpage for Fall 2019.


COMPSCI 682 Neural Networks: A Modern Introduction

Note

  • This is a tentative class outline and is subject to change throughout the semester.
  • Slides will be finalized after each lecture.
Event TypeDateDescriptionCourse Materials
Lecture Tuesday, Sept. 4 Intro to Deep Learning, historical context. [slides]
[python/numpy tutorial]
[jupyter tutorial]
Lecture Thursday, Sept. 6 Image classification and the data-driven approach
k-nearest neighbor
Linear classification
[slides]
[image classification notes]
[linear classification notes]
Optional Discussion Friday, Sept. 7 No discussion section
Lecture Tuesday, Sept. 11 Loss functions [slides]
Lecture Thursday, Sept. 13 Optimization: Stochastic Gradient Descent and Backpropagation [slides]
[optimization notes]
Optional Discussion Friday, Sept. 14 (9:05-9:55am CS140) Slicing and broadcasting in Python [slicing and broadcasting ipynb]
Lecture Tuesday, Sept. 18 Backpropagation & Neural Networks I [slides]
[backprop notes]
[Efficient BackProp] (optional)
related: [1], [2], [3] (optional)
Lecture Thursday, Sept. 20 Neural Networks II
Higher-level representations, image features
Vector, Matrix, and Tensor Derivatives
[slides]
handout 1: Vector, Matrix, and Tensor Derivatives
handout 2: Derivatives, Backpropagation, and Vectorization
Deep Learning [Nature] (optional)
Optional Discussion Friday, Sept. 21 (11:05-12:05 CS142) Vector, Matrix, and Tensor Derivatives [notes]
Lecture Tuesday, Sept. 25 Neural Networks III [slides]
tips/tricks: [1], [2] (optional)
Lecture Thursday, Sept. 27 Training Neural Networks I:
Activation Functions
[slides]
[Neural Nets notes 1]
Optional Discussion Friday, Sept. 21 No discussion section
Lecture Tuesday, Oct. 2 Training Neural Networks II:
weight initialization, batch normalization
[slides]
[Neural Nets notes 2]
[Batch Norm]
Copula Normalization (optional)
Lecture Thursday, Oct. 4 Training Neural Network III:
babysitting the learning process, hyperparameter optimization
[slides]
[Bengio 2012] (optional)
Optional Discussion Friday, Oct. 5 (11:05-12:05 CS142) Google Cloud, PyTorch, Tensorflow tutorials [Google Cloud Tutorial]
[PyTorch Notebook (from Stanford)]
No Class (Monday Schedule) Tuesday, Oct. 9 Monday class schedule will be followed
Lecture Thursday, Oct. 11 Training Neural Network III:
babysitting the learning process, hyperparameter optimization Training Neural Network IV:
model ensembles, dropout
[slides]
[slides]
[Neural Nets notes 3]
LeNet (optional)
Optional Discussion Friday, Oct. 12 (11:05-12:05 CS142) A closer look at the maths inside batch normalization
Lecture Tuesday, Oct. 16 Training Neural Network V:
parameter updates
Convolutional Neural Networks:
convolution layer, pooling layer, fully connected layer
[slides]
[slides]
Lecture Thursday, Oct. 18 Convolutional Neural Networks: (cont.)
convolution layer, pooling layer, fully connected layer
[slides]
Optional Discussion Friday, Oct. 19 Convolutional neural networks [slides]
Lecture Tuesday, Oct. 23 Geust Lecture: Subhransu Maji, Deep Texture Representations
[slides]
Lecture Thursday, Oct. 25 ConvNets for spatial localization, Object detection
[slides]
FCN
Optional Discussion Friday, Oct. 26 No discussion section
Lecture Tuesday, Oct. 30 ConvNets for spatial localization, Object detection (cont.) [slides]
Lecture Thursday, Nov. 1 Recurrent Neural Networks (RNN) [slides]
DL book RNN chapter (optional)
min-char-rnn, char-rnn, neuraltalk2
Optional Discussion Friday, Nov. 2 No discussion section
Lecture Tuesday, Nov. 6 Long Short Term Memory (LSTM)
[slides]
The Unreasonable Effectiveness of RNN (optional)
Understanding LSTM Networks (optional)
Lecture Thursday, Nov. 8 Understanding and visualizing Convolutional Neural Networks
Backprop into image: Visualizations, deep dream
[slides]
[visualization notes]
Optional Discussion Friday, Nov. 9 Midterm Review
Lecture Tuesda, Nov. 13 Creating Adversarial Examples
Generative Models
Generative Adversarial Networks
[slides]
[slides]
Midterm Thursday, Nov. 15 In-class midterm
Optional Discussion Friday, Nov. 16 No discussion section
Lecture Tuesday, Nov. 27 Training ConvNets in practice [slides]
Guest Lecture Thursday, Nov. 29 Geust Lecture: Rajarshi Das, An Introduction to Neural Networks for Natural Language Processing [slides (pdf)], [slides (keynote)]
Optional Discussion Friday, Nov. 30 No discussion section
Lecture Tuesday, Dec. 4 Training ConvNets in practice [slides]
Lecture Thursday, Dec. 6 Training ConvNets in practice [slides]
Optional Discussion Friday, Dec. 7 No discussion section
Presentation (tentative) Tuesday, Dec. 11 Poster presentations
Twi sessions at CS150/151:
8-10am (regular time)
11:15am-2pm