There are no announcements




Optimal Control and Reinforcement Learning: Theory, Numerical Methods, and Applications
This course concerns multi-stage decision processes in the framework of optimal control theory, dynamic programming and the Bellman equation, where optimal policies are synthesized based on both immediate and long-term rewards. However, the computational requirements of dynamic programming techniques can be prohibitive as the policy/state space is overwhelmingly large, the so-called Bellman's curse of dimensionality". In this course we will overcome this difficulty by means of different techniques for the computation of suboptimal solutions to dynamic programming equations. The lectures will address theoretical, algorithmic, and computational aspects of such techniques.


Autumn 2020 (Monday, October 5 to Friday, December 11)


  • Live lecture hours: 10
  • Recorded lecture hours: 0
  • Total advised study hours: 40


  • Tue 13:05 - 13:55


Some general knowledge on Dynamical Systems, Iterative Methods, Optimisation and/or Markov Chains is useful, but not essential.


1. Dynamical systems and control essentials.
2. Optimization and optimal control: characterization of optimal actions, necessary optimality condtions.
3. Optimal feedback control and the Hamilton-Jacobi-Bellman PDE.
4. Discrete Dynamic Programming: the Bellman Equation, Value and Policy Iteration Methods.
5. Neural Networks: basic architectures, approximation properties, training/optimization.
6. Continuous Optimization: deterministic and stochastic gradient descent, variants.
7. Approximate Dynamic Programming I: Monte Carlo policy evaluation and Approximate Policy Iteration.
8. Approximate Dynamic Programming II: Approximate Value Iteration.
9. An overview of Deep Reinforcement Learning.
10. Case studies: playing Pac-man, Tetris, and the financial market with reinforcement learning.


Dante Kalise
Phone 01157484095


Introduction to the Mathematical Theory of ControlA. Bressan and B. Piccoli
Neuro-Dynamic ProgrammingDimitri P. Bertsekas and John Tsitsiklis
Reinforcement Learning: An IntroductionR. Sutton and A. Barto
Deep Reinforcement Learning: A Brief Survey, IEEE Signal Processing Magazine 34(6), 2017K. Arulkumaran, M. P. Deisenroth, M. Brundage, A. A. Bharath


Clicking on the link for a book will take you to the relevant Google Book Search page. You may be able to preview the book there. On the right hand side you will see links to places where you can buy the book. There is also link marked 'Find this book in a library'. This sometimes works well, but not always. (You will need to enter your location, but it will be saved after you do that for the first time.)


No assessment information is available yet.

No assignments have been set for this course.


No files have yet been uploaded for this course.

Recorded Lectures

Please log in to view lecture recordings.