Bayesian Statistics, Philosophy and Practice - 2024 entry

Since the 1980s, computational advances and novel algorithms have seen Bayesian methods explode in popularity, today underpinning modern techniques in data science and machine learning with applications across science, social science, the humanities and finance.

This module will introduce Bayesian statistics and reasoning. It will develop the philosophical and mathematical ideas of subjective probability theory for decision-making and explore the place subjectivity has in scientific reasoning. It will develop Bayesian methods for data analysis and introduce modern Bayesian simulation, including Markov Chain Monte Carlo and Hamiltonian Monte Carlo. The course balances philosophy, theory, mathematical calculation and analysis of real data ensuring the student is equipped to use Bayesian methods in future jobs aligned to data analysis whilst being ready to study masters and PhD level courses with Bayesian content and to take Bayesian research projects.

This module will cover the Bayesian approach to modelling, data analysis and statistical inference. The module describes the underpinning philosophies behind the Bayesian approach, looking at subjective probability theory, subjectivity in science as well as the notion and handling of prior knowledge, and the theory of decision making under uncertainty. Bayesian modelling and inference is studied in depth, looking at parameter estimation and inference in simple models and then hierarchical models. We explore simulation-based inference in Bayesian analyses and develop important algorithms for Bayesian simulation by Markov Chain Monte Carlo (MCMC) such the Gibbs sampler, Metropolis-Hastings and Hamiltonian Monte Carlo. We introduce decision theory with Bayes as a route to personalised decision making under uncertainty. At M-level, in addition to the above, students are introduced to topics in Bayesian approximation such as Laplace approximation and variational inference via material for self-study.

Pre-requisite: MTH1004 + MTH2006 or equivalent (amounting to a 1^st year introduction to probability and a 2^nd year introduction to likelihood methods of statistical inference and regression analysis).

On successful completion of this module you should be able to:

Module Specific Skills and Knowledge

Discipline Specific Skills and Knowledge

Personal and Key Transferable / Employment Skills and Knowledge

Introduction: Bayesian vs Classical statistics, Nature of probability and uncertainty, Subjectivism.

Bayesian inference: Conjugate models, Prior and Posterior predictive distributions, Posterior summaries and simulation, Objective and subjective priors, Normal approximation, Bernstein Von-mises results Bayesian Hierarchical models, Bayesian regression and logistic regression.

Bayesian Computation: Monte Carlo, Inverse CDF, Rejection Sampling, Importance Sampling, Markov Chain Monte Carlo (MCMC), The Gibbs sampler, Metropolis Hastings, Hamiltonian Monte Carlo.

Bayesian Approximation: (Topics from) MAP estimation, Laplace approximation, Mixture approximations, Variational Inference.

Decision Theory: Bayes’ rule, Decision trees, Utility theory.

*Please refer to reassessment notes for details on deferral vs. Referral reassessment

ELE – College to provide hyperlink to appropriate pages

Web based and electronic resources:

Other resources:

Reading list for this module: