Final Exam

The final exam will be held from 6:30 to 9:00 on Sunday, Nov. 23. The exam will be a written exam featuring short answer essay questions. The exam will concentrate on material we have covered since the midterm, although we may ask a few questions on the earlier material.

Notes

We will allow everyone to bring a single page of notes to the exam. You can use this page to write down formulas or other information that would be hard to memorize.

Topics to Review

Here is a list of the most important topics we have covered since the second midterm. You should be prepared to answer "what is this" or "how does this work" type questions about any of the topics listed below.

• Tokenizers
• Embedding vectors
• Self-attention
• Transformers for text encoding
• Transformers for text generation
• How LLMs are trained
• Forward diffusion process
• Forward diffusion equation
• Training diffusion models
• Reverse diffusion process (how to sample)
• Why can’t we just throw data at a neural network until it works?
• True positives and true negatives vs. false positives and false negatives
• Recall (Chirpy system)
• Precision (Dozer system)
• Making an ideal dataset
• Data augmentation (five techniques: 1. Mirroring, 2. Shifting, 3. Scaling, 4. Rotating, 5. Noise)
• Reinforcement learning - learning a policy
• Reinforcement learning - Q learning
• Monte Carlo tree search