| Event Type | Date | Description | Course Materials |
|---|---|---|---|
| Lecture | Tuesday, Sept. 4 | Intro to Deep Learning, historical context. |
[slides] [python/numpy tutorial] [jupyter tutorial] |
| Lecture | Thursday, Sept. 6 | Image classification and the data-driven approach k-nearest neighbor Linear classification |
[slides] [image classification notes] [linear classification notes] |
| Optional Discussion | Friday, Sept. 7 | No discussion section | |
| Lecture | Tuesday, Sept. 11 | Loss functions |
[slides] |
| Lecture | Thursday, Sept. 13 | Optimization: Stochastic Gradient Descent and Backpropagation |
[slides] [optimization notes] |
| Optional Discussion | Friday, Sept. 14 | (9:05-9:55am CS140) Slicing and broadcasting in Python |
[slicing and broadcasting ipynb] |
| Lecture | Tuesday, Sept. 18 | Backpropagation & Neural Networks I |
[slides] [backprop notes] [Efficient BackProp] (optional) related: [1], [2], [3] (optional) |
| Lecture | Thursday, Sept. 20 |
Neural Networks II Higher-level representations, image features Vector, Matrix, and Tensor Derivatives |
[slides] handout 1: Vector, Matrix, and Tensor Derivatives handout 2: Derivatives, Backpropagation, and Vectorization Deep Learning [Nature] (optional) |
| Optional Discussion | Friday, Sept. 21 | (11:05-12:05 CS142) Vector, Matrix, and Tensor Derivatives | [notes] |
| Lecture | Tuesday, Sept. 25 | Neural Networks III |
[slides] tips/tricks: [1], [2] (optional) |
| Lecture | Thursday, Sept. 27 |
Training Neural Networks I: Activation Functions |
[slides] [Neural Nets notes 1] |
| Optional Discussion | Friday, Sept. 21 | No discussion section | |
| Lecture | Tuesday, Oct. 2 |
Training Neural Networks II: weight initialization, batch normalization |
[slides] [Neural Nets notes 2] [Batch Norm] Copula Normalization (optional) |
| Lecture | Thursday, Oct. 4 |
Training Neural Network III: babysitting the learning process, hyperparameter optimization |
[slides] [Bengio 2012] (optional) |
| Optional Discussion | Friday, Oct. 5 | (11:05-12:05 CS142) Google Cloud, PyTorch, Tensorflow tutorials | [Google Cloud Tutorial] [PyTorch Notebook (from Stanford)] |
| No Class (Monday Schedule) | Tuesday, Oct. 9 | Monday class schedule will be followed | |
| Lecture | Thursday, Oct. 11 |
Training Neural Network III: babysitting the learning process, hyperparameter optimization Training Neural Network IV: model ensembles, dropout |
[slides] [slides] [Neural Nets notes 3] LeNet (optional) |
| Optional Discussion | Friday, Oct. 12 | (11:05-12:05 CS142) A closer look at the maths inside batch normalization | |
| Lecture | Tuesday, Oct. 16 |
Training Neural Network V: parameter updates Convolutional Neural Networks: convolution layer, pooling layer, fully connected layer |
[slides] [slides] |
| Lecture | Thursday, Oct. 18 |
Convolutional Neural Networks: (cont.) convolution layer, pooling layer, fully connected layer |
[slides] |
| Optional Discussion | Friday, Oct. 19 | Convolutional neural networks | [slides] |
| Lecture | Tuesday, Oct. 23 |
Geust Lecture: Subhransu Maji, Deep Texture Representations |
[slides] |
| Lecture | Thursday, Oct. 25 | ConvNets for spatial localization, Object detection |
[slides] FCN |
| Optional Discussion | Friday, Oct. 26 | No discussion section | |
| Lecture | Tuesday, Oct. 30 | ConvNets for spatial localization, Object detection (cont.) |
[slides] |
| Lecture | Thursday, Nov. 1 | Recurrent Neural Networks (RNN) |
[slides] DL book RNN chapter (optional) min-char-rnn, char-rnn, neuraltalk2 |
| Optional Discussion | Friday, Nov. 2 |
No discussion section |
|
| Lecture | Tuesday, Nov. 6 |
Long Short Term Memory (LSTM) |
[slides] The Unreasonable Effectiveness of RNN (optional) Understanding LSTM Networks (optional) |
| Lecture | Thursday, Nov. 8 |
Understanding and visualizing Convolutional Neural Networks Backprop into image: Visualizations, deep dream |
[slides] [visualization notes] |
| Optional Discussion | Friday, Nov. 9 | Midterm Review | |
| Lecture | Tuesda, Nov. 13 |
Creating Adversarial Examples Generative Models Generative Adversarial Networks |
[slides] [slides] |
| Midterm | Thursday, Nov. 15 | In-class midterm | |
| Optional Discussion | Friday, Nov. 16 | No discussion section | |
| Lecture | Tuesday, Nov. 27 | Training ConvNets in practice | [slides] |
| Guest Lecture | Thursday, Nov. 29 | Geust Lecture: Rajarshi Das, An Introduction to Neural Networks for Natural Language Processing | [slides (pdf)], [slides (keynote)] |
| Optional Discussion | Friday, Nov. 30 | No discussion section | |
| Lecture | Tuesday, Dec. 4 | Training ConvNets in practice | [slides] |
| Lecture | Thursday, Dec. 6 | Training ConvNets in practice | [slides] |
| Optional Discussion | Friday, Dec. 7 | No discussion section | |
| Presentation (tentative) | Tuesday, Dec. 11 |
Poster presentations Twi sessions at CS150/151: 8-10am (regular time) 11:15am-2pm |