August 15: Today's exam and a preliminary solution can be found here.

July 15: The resit exam will be on Friday, August 15, 9:00-13:00 in Trondheim and will be written.

June 2: The grades on the exam should appear on studweb shortly. In general, each point 1a, 1b,… gave a maximum score of 10. Instead of https://i.ntnu.no/wiki/-/wiki/norsk/prosentvurderingsmetoden, 86, 74, 62, 50 and 38 were used as minimum scores for each grade. The resit exam in August will most likely be written if the expected number of students sign up.

May 26: Today's exam and a preliminary solution can be found here.

May 13: In the solution to the August 2024 exam, an error (a missing normalising constant) in the block proposal in problem 3c has been corrected.

April 24: Zoom link to today's oral presentations

March 27: Project 3 is out.

March 12: The zoom-link for tomorrows oral presentations meeting is https://NTNU.zoom.us/j/96209634496?pwd=sNb2kQNY3osaOYtzr3s61pquyEmgYp.1

February 27: All tasks of project 2 are finalized.

February 5: The zoom-link to tomorrows oral presentations meeting is https://NTNU.zoom.us/j/99787396038?pwd=H83WJhLzl6VWwfFWzgyxJWbYv0c2Ic.1

January 17: Project 1 is finalized, see link below and carefully read the guidelines.

November 27: First lecture is on Monday, January 6 at 8:15 in B3.

Lecturer: Jarle Tufto

Teaching assistant: Karina Lilleborge

Lectures: Mondays 8:15-10 in B3 and Tuesdays 10:15-12:00 in G21.\(^*\)

Project supervision: Tuesdays 10:15-12:00 in G21 in weeks without lectures.

EdStem forum (sign up using the link, ntnu email required). Use this when you have questions about any part of the course.

Reference group: NN and NN.

The curriculum is what is covered in the lectures and by the exercises. The curriculum is based on sections (see below) in G.H. Givens and J.A. Hoeting (2013). Computational Statistics, 2nd edition (NTNU access through this link) (GH) and D. Gamerman and H.F. Lopes (2006). Markov chain Monte Carlo - Stochastic Simulation for Bayesian Inference, 2nd edition (sold by Akademika) (GL). You may be able to buy a second-hand physical copy or get hold of a digital pdf version of GL.

Stochastic simulation and an introduction to Bayesian inference. Givens and Hoeting: 1-1.8 (mostly repetition), 6.1, 6.2-6.2.3.2, 6.3.1, 6.4.1. Gamerman and Lopes: 1-1.5.

Week 2.1: Introduction, pseudorandom number generators, inversion sampling. GL: 1.1-1.2-1.3.1, GH: 1 (review)

Week 2.2: Transformation formula for joint densities, ratio of uniforms method, Box-Muller algorithm. GL: 1.3.2, GH: 6-6.2.2

Week 3.2: Simulation via mixtures, the multivariate normal, rejection sampling. GL: 1.4, 1.5.1, GH: 6.2.2, 6.2.3

Week 3.2: Rejection sampling variants, Monte Carlo, importance sampling. GL: 1.5 (all), GH: 6.2.3 (all), 6.3.1, 6.4.1. Alias method.

Week 4.1: Antithetic sampling (GH: 6.4.2). Introduction to Bayesian inference. GL: 1.5 (all), GH: 6.2.3 (all), 6.3.1, 6.4.1

Week 4.2: More Bayes. Summary part 1.

Updated lecture notes

R code examples

Week (4), 5 and 6: Project 1

Bayesian inference and Markov chain Monte Carlo. Maybe some INLA and/or TMB. Givens and Hoeting: 7-7.3.3. Gamerman and Lopes: 2.1, 2.2-2.2.2, 2.3.1, 2.3.3, 2.4, 4.1-4.6, 5.1-5.2, 6.1-6.4.4.

Week 7.1: Hierarchical Bayesian models, review of Markov chain, time reversability, detailed balance equation, Metropolis-Hastings algorithm.

Week 7.2: Tuning of random-walk metropolis algorithm, comparison with independence sampler, numerical issues, M-H with iterative conditioning, Gibbs sampling.

Week 8.1: Gibbs sampling examples, Slice sampling, Metropolis within Gibbs (hybrid sampler), blocking. The correct conjungate prior for \(\boldsymbol{\beta},\tau\) to a linear regression likelihood. and the resulting posterior is now in the updated lectures below (I made an error in the lecture). Note also that the prior on p. 13 is not conjungate to this likelihood.

Week 8.2: Single site vs. block updating example cont. Estimating autocorrelation and effective sample size. More about DAGs. Gibbs sampling for random intercept LMM.

Week 9.1: Laplace approximation in hierarchic latent variable models. Computation of the Laplace approximation in INLA vs. via AD in TMB and RTMB. Automatic differentation (AD). INLA in some more detail. For even more details, see Rue, Martino & Chopin 2009) which also has an interesting "Discussion of the paper" by other authors, discussing for instance alternative MCMC strategies using eq. (3) to construct a one-block proposal.

Week 9.2: RTMB and INLA examples.

Updated lecture notes

R code examples

Project 2

Some comments on the solution of project 2

Bootstrapping and the EM-algorithm. Givens and Hoeting: 4.1, 4.2-4.2.1, 9.1, 9.2-9.2.4, 9.3, 9.3.1, 9.3.2.1-2, 9.5.1, 9.8.

Week 12.1 Non-parametric bootstrap via simulation and via exact method. Parametric bootstrap. Examples. Bootstrap estimation of bias and bias correction. Bootstrap confidence intervals via percentile method. Validity of percentile method.

Week 12.2 Regression and bootstrapping residuals versus bootstrapping pairs. Accelerated bias-corrected percentile method BC\(_a\) with R-implementation. Bootstrap t confidence intervals. Residual bootstrapping for time series models.

Week 13.1 The EM-algorithm. ABO-blood types example.

Week 13.2 Gaussian mixture example. Convergence. Proof that \(f_X(x|\theta^{(t+1)})\ge f_X(x|\theta^{(t)})\).

Updated lecture notes (further update on April 22 to the EM-algorithm bloodtype example)

R code examples

Project 3

Week 18: Tuesday: Summary of the course. More examples from part 1, 2 and 3. A very brief overview of quasi-Monte Carlo (for an in-depth treatment, see Lemieux 2009). Some discussion about importance sampling and harmonic mean estimator of model evidence. R code. There is a video recording in Blackboard but not that the video starts a bit late (at 10:33)

May 26, 15-19 (counts 70% towards final grade)

Previous exams can be found in this folder.

• Will the lectures be recoreded? Yes, the lectures are recorded and will be available in Blackboard (follow the "Panopto" link in the left margin).