Announcements


There are no announcements

Forum

General


Semester

Autumn 2007 (Monday, October 8 to Friday, December 14)

Timetable

  • Mon 09:05 - 09:55
  • Tue 09:05 - 09:55

Prerequisites

Knowledge of basic Probability, Markov chains and optimisation methods like linear programming would be helpful. The first 4 lectures will be devoted to the revision, so that all information needed will be briefly provided.

Syllabus

  • Introduction: Revision of Probability.
  • Markov chains: Definitions. Transition probability, diagrams. Classification of states, limiting behaviour, absorbing and ergodic chains.
  • Markov Decision Processes: Finite and infinite horizon, dynamic programming approach. Discounted model and expected average reward. Canonical equations and the linear programming approach. Linear-quadratic regulators. Applications to Reliability, Queues, Inventory, Finance, Epidemiology etc.
  • Other optimal control problems: deterministic models, continuous time Markov chains, diffusion processes.

Lecturer


Alexei Piunovskiy
Email piunov@liverpool.ac.uk
Phone (0151) 7944737
Photo of Alexei Piunovskiy


Students


Photo of Karandeep S. Dhingra
Karandeep S. Dhingra
(Liverpool)
Photo of Chris Pearce
Chris Pearce
(Liverpool)
Photo of Yilei Ren
Yilei Ren
(Nottingham)
Photo of Dong-mei Wang
Dong-mei Wang
(Manchester)
Photo of Yi Zhang
Yi Zhang
(Liverpool)


Bibliography


No bibliography has been specified for this course.

Assessment



No assessment information is available yet.

No assignments have been set for this course.

Files


Files marked L are intended to be displayed on the main screen during lectures.

Week(s)File
biblio.pdf
1-2sec1.pdfL
3-4sec2.pdfL
5-6sec3.pdfL
7-8sec4.pdfL
9-10sec5.pdfL


Recorded Lectures


Please log in to view lecture recordings.