Announcements


There are no announcements

Forum

General


This course is part of the MAGIC core.

Description

Markov processes in discrete and continuous time will be presented for elementary and general state spaces and for some jump & so called linear-Markov (semi-Markov) processes in continuous times. Standard stuff will include generators, Dynkin’s formula, ergodicity for finite state spaces, strong Markov, Feller & strong Feller properties. A bit more advanced material will include coupling & recurrence applied to convergence rates and to queueing & reliability systems.

Semester

Autumn 2016 (Monday, October 3 to Friday, December 9)

Timetable

  • Thu 10:05 - 10:55
  • Fri 10:05 - 10:55

Prerequisites

Some basic knowledge about Markov chains is highly desirable.

Syllabus

1. Stochastic processes. Definitions of a Markov process. 2. Examples: Random Walks. Generators. Chapman-Kolmogorov equations. 3. Dynkin’s identity. Stopping times, strong Markov property, Feller & strong Feller processes. 4. Irreducible Markov processes, ergodic theorem for finite state spaces and in general case. 5. Doob-Doeblin's and Markov-Dobrushin's conditions of ergodicity. 6. Positive & null recurrent, polynomially & exponentially recurrent Markov processes. 7. Coupling method, lemma about three random variables, application to convergence rates. 8. Applications to queueing: Erlang telephone systems, stationary regimes, convergence rates. 9. Piecewise-linear Markov processes, extended Erlang formulae & convergence rates. 10. Applications to some reliability theory problems.

Lecturer


Alexander Veretennikov
Email A.Veretennikov@leeds.ac.uk
Phone (0113) 3435183
vcard
Photo of Alexander Veretennikov


Students


Photo of Adam Barker
Adam Barker
(Reading)
Photo of Junbin Chen
Junbin Chen
(Durham)
Photo of Xiaoxuan Ding
Xiaoxuan Ding
(Loughborough)
Photo of Dario Domingo
Dario Domingo
(Leeds)
Photo of Afredo Garbuno Inigo
Afredo Garbuno Inigo
(Liverpool)
Photo of Xin Guo
Xin Guo
(Liverpool)
Photo of Blaine Keetch
Blaine Keetch
(Nottingham)
Photo of Yuija Liu
Yuija Liu
(Loughborough)
Photo of Hugo Lo
Hugo Lo
(Durham)
Photo of Yan Luo
Yan Luo
(Loughborough)
Photo of Xiao Ma
Xiao Ma
(Loughborough)
Photo of James McRedmond
James McRedmond
(Durham)
Photo of Cedric Spire
Cedric Spire
(Leicester)
Photo of Shane Turnbull
Shane Turnbull
(Lancaster)
Photo of Robert West
Robert West
(Leeds)
Photo of Jingsi Xu
Jingsi Xu
(Manchester)


Bibliography


Stochastic Processes and ModelsStirzaker
Markov Processes: Characterization and ConvergenceEthier and Kurtz
A course in the theory of stochastic processesVent︠t︡selʹ and Wentzell
Markov Models & OptimizationDavis


Note:

Clicking on the link for a book will take you to the relevant Google Book Search page. You may be able to preview the book there. On the right hand side you will see links to places where you can buy the book. There is also link marked 'Find this book in a library'. This sometimes works well, but not always. (You will need to enter your location, but it will be saved after you do that for the first time.)

Assessment



The assessment for this course will be via a single take-home paper in January with 2 weeks to complete and submit online. Provisionally, there will be five questions and you will need the equivalent of three questions to pass.

Exam problems

Files:Exam paper
Released: Monday 9 January 2017 (74.9 days ago)
Deadline: Sunday 22 January 2017 (60.9 days ago)


Recorded Lectures


Please log in to view lecture recordings.