MAGIC019: Markov Decision Processes with Applications

Course details

A specialist MAGIC course

Semester

Autumn 2007
Monday, October 8th to Friday, December 14th

Hours

Live lecture hours
20
Recorded lecture hours
0
Total advised study hours
0

Timetable

Mondays
09:05 - 09:55
Tuesdays
09:05 - 09:55

Announcements

Description

Prerequisites

Knowledge of basic Probability, Markov chains and optimisation methods like linear programming would be helpful. The first 4 lectures will be devoted to the revision, so that all information needed will be briefly provided.

Syllabus

  • Introduction: Revision of Probability.
  • Markov chains: Definitions. Transition probability, diagrams. Classification of states, limiting behaviour, absorbing and ergodic chains.
  • Markov Decision Processes: Finite and infinite horizon, dynamic programming approach. Discounted model and expected average reward. Canonical equations and the linear programming approach. Linear-quadratic regulators. Applications to Reliability, Queues, Inventory, Finance, Epidemiology etc.
  • Other optimal control problems: deterministic models, continuous time Markov chains, diffusion processes.

Lecturer

  • AP

    Dr Alexei Piunovskiy

    University
    University of Liverpool

Bibliography

No bibliography has been specified for this course.

Assessment

Attention needed

Assessment information will be available nearer the time.

Files

Only consortium members have access to these files.

Please log in to view course materials.

Lectures

Please log in to view lecture recordings.