MAGIC019: Markov Decision Processes with Applications

Course details

Semester

Autumn 2009
Monday, October 5th to Friday, December 11th

Hours

Live lecture hours
20
Recorded lecture hours
0
Total advised study hours
0

Description

Prerequisites

Knowledge of basic Probability, Markov chains and optimisation methods like linear programming would be helpful. The first 4 lectures will be devoted to the revision, so that all information needed will be briefly provided.

Syllabus

  • Introduction: Revision of Probability.
  • Markov chains: Definitions. Transition probability, diagrams. Classification of states, limiting behaviour, absorbing and ergodic chains.
  • Markov Decision Processes: Finite and infinite horizon, dynamic programming approach. Discounted model and expected average reward. Canonical equations and the linear programming approach. Linear-quadratic regulators. Applications to Reliability, Queues, Inventory, Finance, Epidemiology etc.
  • Other optimal control problems: deterministic models, continuous time Markov chains, diffusion processes.

Lecturer

  • AP

    Dr Alexei Piunovskiy

    University
    University of Liverpool

Bibliography

No bibliography has been specified for this course.

Assessment

Attention needed

Assessment information will be available nearer the time.

Files

Only consortium members have access to these files.

Please log in to view course materials.

Lectures

Please log in to view lecture recordings.