Welcome to COMSM0045. The unit introduces the students to deep architectures for learning linear and non-linear transformations of big data towards tasks such as classification and regression. The unit paves the path from understanding the fundamentals of convolutional and recurrent neural networks through to training and optimisation as well as evaluation of learnt outcomes. The unit's approach is hands-on, focusing on the 'how-to' while covering the basic theoretical foundations. For further general information, see the syllabus for the unit.
UPDATE - 15/09/2023 Welcome new students for the 23/24 year!Michael Wray (MW) | Unit Director |
Tilo Burghardt (TB) |
Adriano Fragomeni (AF), Ahmad Dar Khalil (AK), Dan Whettam (DW), Jacob Chalk (JC), Kevin Flanagan (KF), Sam Pollard (SP), Saptarshi Sinha (SS), Siddhant Bansal (SB), Zhifan Zhu (ZZ)
Wks | Tues 09:00-11:00 | Friday 09:00-12:00 | Labs |
1 |
26/09/2023 - 9am - Queens BLDG 1.69 Wk1 - LECTURE 1 INTRODUCTION TO THE UNIT intro PDF BASICS OF ARTIFICIAL NEURAL NETWORKS (Queens Building 1.69, in-person) (Introduction, Neural Networks, Perceptron, Cost Functions, Gradient Descent, Delta Rule, Deep Networks) PDF Slides |
29/09/2023 - 9am - Queens BLDG Bill Brown Suite Wk1 - LECTURE 2 TOWARDS TRAINING DEEP FORWARD NETWORKS (Queens Building F.101c (Bill Brown Suite), in-person + lecture recap) (Network Representation, Computational Graphs, Reverse Auto-Differentiation) PDF Slides, Extra Recap Recording Lecture 2 Refresher (first part of video) |
GETTING STARTED: Register Individually on BlueCrystal4 (details see below) RECAP WORKSHEETS: -Convolutions (Homework) -Lab0 - Python (Homework) |
2 |
03/10/2023 - 9am - Queens BLDG 1.69 Wk2- LECTURE 3 BACKPROPAGATION ALGORITHM (Queens Building 1.69, in-person + recorded lecture) (The Backpropagation Algorithm in Full Detail, Activation Functions) PDF Slides Extra Recap Recording (second part of video) Wk2 - LECTURE 4 OPTIMISATION TECHNIQUES (Queens Building 1.69, in-person + recorded lecture) (Stochastic Gradient Descent, Nesterov Momentum, RMSProp, Newton's Method, AdaGrad, Adam, Saddle Points) PDF Slides Extra Recap Recording |
06/10/2023,9am,(Queens BLDG Bill Brown Suite) - PRACTICAL 1 (Slides), Your first fully connected layer gradient descent stochastic gradient descent |
06/10/2023, (Queens BLDG Bill Brown Suite) - 3hrs -BC4 Setup Lab 1 - Training your first Deep Neural Network |
3 |
10/10/2023 - 9am - Queens BLDG 1.69 Wk3 - LECTURE 5 (Queens Building 1.69, in-person + recorded lecture) COST FUNCTIONS, REGULARISATION AND DEPTH (SoftMax, Cross Entropy, L1 and L2 Regularisation, DropOut, DropConnect, Depth Considerations) PDF Slides Extra Recap Recording |
13/10/2023,9am,(Queens BLDG Bill Brown Suite) - PRACTICAL 2 (Slides),
(Video). Your first convolutional connected layer |
13/10/2023, (Queens BLDG Bill Brown Suite) - 3hr Lab 2 - Your First Convolutional Connected Network |
Wk3 - LECTURE 6 (Queens Building 1.69, in-person + recorded lecture) CONVOLUTIONAL NEURAL NETWORKS (Video Pt. 1, Video Pt. 2, Slides) (sharing parameters, conv layers, pooling, CNN architectures) |
|||
4 | - |
20/10/2023,9am,(Queens BLDG Bill Brown Suite) -
PRACTICAL 3 (Slides, Video Recording) Error rate monitoring (training/validation/testing) Batch-based training Learning rate Weight Freezing Batch normalisation Parameter intialisation |
20/10/2023, (Queens BLDG Bill Brown Suite) - 3hr Lab 3 - Hyperparameters |
5 | - |
27/10/2023, 9am, (Queens BLDG Bill Brown Suite) - PRACTICAL 4 Data Augmentation Debugging strategies Dropout (Slides), Video Recording |
27/10/2023, (Queens BLDG Bill Brown Suite) - 3hr Lab Lab 4 - Data Augmentation |
6 | READING WEEK (no activities) | ||
7 |
07/11/2023 - 9am - Queens BLDG 1.69 Wk8 - LECTURE 7 (Queens Building 1.69, in-person + Recorded) RECURRENT and RELATIONAL NEURAL NETWORKS (RNN, bi-directional RNNs, encoder-decoder, Transformers) (Slides, Video) Wk 8 - LECTURE 8 (Queens Building 1.69, in-person + Recorded) GENERATIVE MODELS (GANs, Diffusion Models) (Slides, Video pt. 1, Video pt. 2) |
10/11/2023, 9am, (Queens BLDG Bill Brown Suite) - PRACTICAL 5 Transformers (Slides, Video Recording) |
10/11/2023, (Queens BLDG Bill Brown Suite) - 3hr Lab 5 Transformers |
8 | - | 17/11/2023, 9am, (Queens BLDG Bill Brown Suite) Continuation Lab + First CW Lab |
17/11/2023, (Queens BLDG Bill Brown Suite) - 3hr |
9 | - | 24/11/2023, 10am [2 hours], (Queens BLDG Bill Brown Suite) - CW Support Session | - |
10 | - | 01/12/2023, 10am [2 hours], (Queens BLDG Bill Brown Suite) - CW Support Session | - |
11 | CW Deadline Thurs 07 Dec 13:00 (Blackboard Submission) | ||
12 | - | 15/12/2023, Queens BLDG Bill Brown Suite - Exam Support Session | - |
Coursework specs can be found here.
Please fill in the form noting your chosen group (even if you're working solo) here
The data and provided code can be found here or on BC4 (see notes in coursework specs).2 hours exam in January (in person).
We have now released the exam preparation materials on the unit webpage, you can also find these in the folder here.
This includes a list of exam topics, 3 past papers (1 with answers) and example questions of new content this year.
Please note that you cannot take notes into the exam, but calculators are permitted.
All technical resources will be posted on the COMSM0045 ADL Github organisation. If you find any issues, please kindly raise an issue in the respective repository.
Recommended Reading:
Goodfellow et al (2016). Deep Learning. MIT Press
All students must apply online to register an account on BC4 for this unit. This also applies to students who already have accounts on BC4 for other units (e.g. HPC), in this case you must register again using the instructions below.
Note that it takes up to 48 hours to enable your account on BC4.