Bayesian statistics, Philosophy and Practice - 2019 entry

Since the 1980s, computational advances and novel algorithms have seen Bayesian methods explode in popularity, today underpinning modern techniques in data analytics, pattern recognition and machine learning as well as numerous inferential procedures used across science, social science and the humanities.

This module will introduce Bayesian statistical inference, describing the differences between it and classical approaches to statistics. It will develop the ideas of subjective probability theory for decision-making and explore the place subjectivity has in scientific reasoning. It will develop Bayesian methods for data analysis and introduce modern Bayesian simulation based techniques for inference. As well as underpinning a philosophical understanding of Bayesian reasoning with theory, we will use software currently used for Bayesian inference in the lab, allowing you to apply techniques discussed in the course to real data.

Pre-requisite: MTH2006 Statistical Modelling and Inference or equivalent

This module will cover the Bayesian approach to modelling, data analysis and statistical inference. The module describes the underpinning philosophies behind the Bayesian approach, looking at subjective probability theory, subjectivity in science as well as the notion and handling of prior knowledge, and the theory of decision making under uncertainty. We then move to Bayesian modelling and inference looking at parameter estimation in simple models and then hierarchical models. Finally, we explore simulation-based inference in Bayesian analyses and develop important algorithms for Bayesian simulation by Markov Chain Monte Carlo (MCMC) such the Gibbs sampler and the Metropolis-Hastings algorithm.

On successful completion of this module you should be able to:

Module Specific Skills and Knowledge

1. Show understanding of the subjective approach to probabilistic reasoning;

2. Demonstrate an awareness of Bayesian approaches to statistical modelling and inference and an ability to apply them in practice;

3. Demonstrate understanding of the value of simulation-based inference and knowledge of techniques such as MCMC and the theories underpinning them;

4. Demonstrate the ability to apply statistical inference in decision-making;

5. Utilise appropriate software and a suitable computer language for Bayesian modelling and inference from data.

Discipline Specific Skills and Knowledge

6. Demonstrate understanding, appreciation of and aptitude in the quantification of uncertainty using advanced mathematical modelling;

Personal and Key Transferable / Employment Skills and Knowledge

7. Show advanced Bayesian data analysis skills and be able to communicate associated reasoning and interpretations effectively in writing;

8. Apply relevant computer software competently;

9. Use learning resources appropriately;

10. Exemplify self-management and time-management skills.

Introduction: Bayesian vs Classical statistics, Nature of probability and uncertainty, Subjectivism.

Decision Theory: Bayes’ rule, Bayes’ risk, Decision trees, Sequential Decision making, Utility.

Bayesian inference: Conjugate models, Prior and Posterior predictive distributions, Posterior summaries and simulation, Objective and subjective priors, Nuisance parameters, Hierarchical models, Bayesian regression.

Bayesian Computation: Monte Carlo, Inverse CDF, Rejection Sampling, Markov Chain Monte Carlo (MCMC), The Gibbs sampler, Metropolis Hastings, Diagnostics.

Basic reading:

ELE: http://vle.exeter.ac.uk/

Web based and Electronic Resources:

Other Resources:

Lindley, D. V. “Making Decisions”

De Groot, M. H. “Optimal Statistical Decisions”.

Sivia, D. S. “Data Analysis, A Bayesian Tutorial”.

Reading list for this module: