This is the homepage of the course CMPUT 365: Introduction to Reinforcement Learning taught by Csaba Szepesvári at the University of Alberta in the Winter semester of 2025.
You should also be on the course slack workspace (check eClass for the invitation link).
Lecture Date/Location: MWF 1:00 to 1:50 pm at T LB-002
Course Material
We are linking the notes and the worksheets here for easy access.
Notes:
Worksheets:
W0,
W1,
W2,
W3,
W4,
W5,
W6,
W7,
W8,
W9,
W10,
W11
FAQ on using Coursera
Error with Quiz Submission
If you get have any issues with submitting quizzes, try clearing the internet cache, closing all browser windows, and re-logging again to Coursera.
Jupyter Notebook Assignment Grading
Jupyter notebook assignments include local tests (included in the notebook), as well as grader tests that is hidden from the learners.
Please make sure your assignment passes all the local tests before submitting. Also, the solutions have to be general (i.e. not hard-coded) in order to pass the grader tests. Local test cases are not comprehensive, and even if you pass all the local tests, you may not get full marks.
Try to make your code general to work robustly for various cases. (e.g. using variable grid_w instead of value 12)
On rare occasion you may face issues submitting jupyter notebook assignments. If the submit button is missing, please make sure you are only working on the notebook on a single device. If the problem still persists, try setting ?forceRefresh=true in your notebook URL (see this reference).
Resources
- Our textbook is Sutton & Barto’s Reinforcement Learning: An Introduction.
- Vlad’s notes for his class `Machine Learning I’. This was CMPUT 267 in the fall of 2024.
- Reinforcement Learning: Foundations – a textbook for advanced undergraduate classes by Shie Mannor, Yishay Mansour, and Aviv Tamar from the fall of 2024.
- David Silver’s course on RL is still awesome. Slightly more advanced than our class.
- My short take on RL in book format: just and only the algorithms and facts about them as cleanly as possible. From 2010. Almost good still.
- The inevitable bandit book that has everything and more than what you wanted to know about bandits and exploration and exploitation. Also coauthored by yours truly. This is from 2020. Chapter 2 is a self-contained introduction to probability theory; you will never need anything more than included there
- Updated slides of RL applications after I got good feedback from the good people on twitter
- Here, Erwan Le Pennec has multiple sets of slides based on the Sutton-Barto 2018 book; a 2018-19, 2021-22, 2022 These are following the book quite closely and are very nicely done.
- Martha’s ML notes (CMPUT 267)
- Math for ML book
- Deep RL course from Berkeley, just linking it for fun
- Coursera RL Glue on Github
- RL Theory: our big brother course here at the UofA
- Atari and discrete world models!? video! Just a random paper that shows that discrete world models are not hopeless?
- Can Computers Think? Seven giant posters in the bottom of this page, just for fun. What do you all think?