Announcements


There are no announcements

Forum

General


Description

Optimal Control and Reinforcement Learning: Theory, Numerical Methods, and Applications
Description
This course concerns multi-stage decision processes in the framework of optimal control theory, dynamic programming and the Bellman equation, where optimal policies are synthesized based on both immediate and long-term rewards. However, the computational requirements of dynamic programming techniques can be prohibitive as the policy/state space is overwhelmingly large, the so-called Bellman's curse of dimensionality". In this course we will overcome this difficulty by means of different techniques for the computation of suboptimal solutions to dynamic programming equations. The lectures will address theoretical, algorithmic, and computational aspects of such techniques.

Semester

Autumn 2020 (Monday, October 5 to Friday, December 11)

Hours

  • Live lecture hours: 10
  • Recorded lecture hours: 0
  • Total advised study hours: 40

Timetable

  • Tue 13:05 - 13:55

Prerequisites

Some general knowledge on Dynamical Systems, Iterative Methods, Optimisation and/or Markov Chains is useful, but not essential.

Syllabus

Syllabus:
1. Dynamical systems and control essentials.
2. Optimization and optimal control: characterization of optimal actions, necessary optimality condtions.
3. Optimal feedback control and the Hamilton-Jacobi-Bellman PDE.
4. Discrete Dynamic Programming: the Bellman Equation, Value and Policy Iteration Methods.
5. Neural Networks: basic architectures, approximation properties, training/optimization.
6. Continuous Optimization: deterministic and stochastic gradient descent, variants.
7. Approximate Dynamic Programming I: Monte Carlo policy evaluation and Approximate Policy Iteration.
8. Approximate Dynamic Programming II: Approximate Value Iteration.
9. An overview of Deep Reinforcement Learning.
10. Case studies: playing Pac-man, Tetris, and the financial market with reinforcement learning.

Lecturer


Dante Kalise
Email Dante.Kalise@nottingham.ac.uk
Phone 01157484095


Bibliography


Introduction to the Mathematical Theory of ControlA. Bressan and B. Piccoli
Neuro-Dynamic ProgrammingDimitri P. Bertsekas and John Tsitsiklis
Reinforcement Learning: An IntroductionR. Sutton and A. Barto
Deep Reinforcement Learning: A Brief Survey, IEEE Signal Processing Magazine 34(6), 2017K. Arulkumaran, M. P. Deisenroth, M. Brundage, A. A. Bharath


Note:

Clicking on the link for a book will take you to the relevant Google Book Search page. You may be able to preview the book there. On the right hand side you will see links to places where you can buy the book. There is also link marked 'Find this book in a library'. This sometimes works well, but not always. (You will need to enter your location, but it will be saved after you do that for the first time.)

Assessment



No assessment information is available yet.

No assignments have been set for this course.

Files



No files have yet been uploaded for this course.

Recorded Lectures


Please log in to view lecture recordings.