MAGIC093: Introduction to Markov processes, with coupling and convergence rates, and applications

Course details

A core MAGIC course

Semester

Autumn 2017
Monday, October 9th to Friday, December 15th

Hours

Live lecture hours
20
Recorded lecture hours
0
Total advised study hours
0

Timetable

Thursdays
10:05 - 10:55
Fridays
10:05 - 10:55

Announcements

Some auxiliary files will be offered. The first two already downloaded are: (1) a preprint from arxiv, lecture notes Ërgodic Markov processes and Poisson equations"; (2) introduction about some basic probability. Not all sections of the preprint will be needed (probably tere will be no Poisson equations in this module), but everything from the introduction ("magic-v0-17a") is assumed to be known.

Description

Markov processes in discrete and continuous time will be presented for elementary and general state spaces and for some jump & so called linear-Markov (semi-Markov) processes in continuous times. Standard stuff will include generators, Dynkin’s formula, ergodicity for finite state spaces, strong Markov, Feller & strong Feller properties. A bit more advanced material will include coupling & recurrence applied to convergence rates and to queueing & reliability systems.

Prerequisites

Some basic knowledge about Markov chains is highly desirable.

Syllabus

1. Stochastic processes. Definitions of a Markov process. 2. Examples: Random Walks. Generators. Chapman-Kolmogorov equations. 3. Dynkin’s identity. Stopping times, strong Markov property, Feller & strong Feller processes. 4. Irreducible Markov processes, ergodic theorem for finite state spaces and in general case. 5. Doob-Doeblin's and Markov-Dobrushin's conditions of ergodicity. 6. Positive & null recurrent, polynomially & exponentially recurrent Markov processes. 7. Coupling method, lemma about three random variables, application to convergence rates. 8. Applications to queueing: Erlang telephone systems, stationary regimes, convergence rates. 9. Piecewise-linear Markov processes, extended Erlang formulae & convergence rates. 10. Applications to some reliability theory problems.

Lecturer

  • AV

    Professor Alexander Veretennikov

    University
    University of Leeds

Bibliography

Follow the link for a book to take you to the relevant Google Book Search page

You may be able to preview the book there and see links to places where you can buy the book. There is also link marked 'Find this book in a library' - this sometimes works well, but not always - you will need to enter your location, but it will be saved after you do that for the first time.

Assessment

The assessment for this course will be released on Monday 8th January 2018 and is due in by Sunday 21st January 2018 at 23:59.

Assessment will consist of four questions. Results are pass or not. For pass you should solve an equivalent of 50%. The marking scheme will be available, but most likely all questions will have the same weights. (Hint: even if you are sure, it is not recommended to stop after having solved two questions!) Exam problems will be offered around 08/01/2018 (to be confirmed) and you will have four weeks for writing your answers, at home and öpen book". Mock exam will be also offered a bit closer to the end of the semester, but likely less than four questions long.

Please note that you are not registered for assessment on this course.

Files

Only consortium members have access to these files.

Please log in to view course materials.

Lectures

Please log in to view lecture recordings.