MAGIC093: Markov Processes

Course details

A core MAGIC course


Autumn 2022
Monday, October 3rd to Friday, December 9th


Live lecture hours
Recorded lecture hours
Total advised study hours


13:05 - 13:55 (UK)
11:05 - 11:55 (UK)


Markov processes in discrete and continuous time will be presented for finite and countable state spaces in discrete time, as well as for some basic processes in continuous time. Standard material will include generators, Dynkinâ's formula, ergodicity, and (strong) Markov and (strong) Feller properties. A bit more advanced material could include coupling and recurrence applied to convergence rates. Some attention will also be paid to the applicability of Markov processes in a variety of fields such as economics, operational research, biology, and physics. Time permitting we will also look at Monte Carlo Markov chain (MCMC) methods as used in, e.g., (Bayesian) statistics. 


Some basic knowledge about Markov chains is highly desirable. 


1. Stochastic processes. Definitions of a Markov process. 2. Examples: Random Walks. Generators. Chapman-Kolmogorov equations. 3. Dynkin's identity. Stopping times, (strong) Markov property, (strong) Feller processes. 4. Irreducible Markov processes, ergodic theorems, conditions of ergodicity. 5. Coupling method and application to convergence rates. 6. Applications and MCMC.


  • JT

    Professor Jacco Thijssen

    University of York


No bibliography has been specified for this course.


The assessment for this course will be released on Monday 9th January 2023 at 00:00 and is due in before Sunday 22nd January 2023 at 23:59.

Assessment for all MAGIC courses is via take-home exam which will be made available at the release date (the start of the exam period).

You will need to upload a PDF file with your own attempted solutions by the due date (the end of the exam period).

If you have kept up-to-date with the course, the expectation is it should take at most 3 hours’ work to attain the pass mark, which is 50%.

Please note that you are not registered for assessment on this course.


Only current consortium members and subscribers have access to these files.

Please log in to view course materials.


Please log in to view lecture recordings.