Date | Topic | Reading | Notes |
---|---|---|---|
Jan 22 | Introduction; linesearch | Ch. 1-2 of Givens+Hoeting | See Sun and Yuan (2006) for further details on convergence analysis |
Jan 24 | Choosing search directions: Newton, generalized linear models, inexact Newton, quasi-Newton, Fisher scoring, BFGS | See Vandenberghe's notes for some further background | |
Jan 29 | Exploiting special structure to solve Newton linear equations more efficiently: banded, sparse, low-rank (etc.) matrices | ||
Jan 31 | Conjugate gradients | Shewchuk (1994) | |
Feb 5 | Preconditioning. Toeplitz and circulant matrices | See Chan and Ng (1996) on PCG for Toeplitz systems. See HW 1 here; due Feb 19. | |
Feb 7-12 | No class | ||
Feb 14 | Gaussian process regression | See Rasmussen and Williams (2006) for more background on GP regression | |
Feb 19 | Expectation maximization | ||
Feb 21 | Constrained and non-smooth optimization: convex functions; interior point methods | Boyd and Vandenberghe, ch. 3-4 | |
Feb 26 | Linear, quadratic, and semidefinite programs; support vector methods | ||
Feb 28 | LASSO methods | Efron et al (2004), Friedman et al (2010) | |
Mar 5 | Convex duality, KKT conditions | Boyd and Vandenberghe, ch. 5 | |
Mar 7 | A brief tour of some advanced topics: proximal methods, dual decomposition, and convex relaxation | Background: Bach et al (2011), Boyd et al (2011), Luo et al (2010) | |
Mar 12 | Graphical models; dynamic programming | Rabiner tutorial, Jordan (2004) | Background: Wainwright and Jordan (2008), Smith et al (2012) |
Mar 14 | Guest lecture by Arian Maleki on message passing and approximate message passing | notes | |
Mar 19-21 | No class | Spring break | |
Mar 26 | Short project presentations | ||
Mar 28 | Monte Carlo basics. Rejection and importance sampling; adaptive rejection sampling; filter-forward-sample-backward algorithm for HMMs | Ch. 1-7 of Robert and Casella | Background: Devroye (1986), Doucet (2010). |
Apr 2-4 | Metropolis-Hastings. Gibbs sampling: slice sampling, Bayesian lasso, spike-and-slab, hit and run. MCMC diagnostics. Rao-Blackwellization. Adaptive simulated tempering. | Background: Park and Casella (2008), Neal (2003), Papandreou and Yuille (2010), Mohamed et al. (2011), Salakhutdinov (2010) | |
Apr 9,16 | Sequential Monte Carlo | Doucet and Johansen (2011), Pitt and Shephard (1999) | Further reading collected by A. Doucet |
Apr 11 | Hamiltonian Monte Carlo | Neal (2010) |
Guest lecture by Ari Pakman. Further background: Hoffman and Gelman (2012), Pakman and Paninski (2013) |
Apr 18 | Deterministic approximations for posteriors: variational Bayes, expectation propagation | Ormerod and Wand (2010), Sudderth (2002) | Additional reading: Hoffman et al (2013) |
Apr 23 | Stochastic approximation methods | Ch. 4-5 in Spall (2003) | Guest lecture by Lauren Hannah |
Apr 25 | Nonparametric Bayes methods | Guest lecture by Peter Orbanz | |
Apr 29 | No class | Office hours to discuss project | |
May 2 | Integration wrap-up: Gaussian quadrature, RJMCMC, bridge sampling, annealed importance sampling | Ch. 5,8 in Givens+Hoeting | Further reading: Gelman and Meng (1998), Neal (2001) |
May 7,9 | Project presentations | Send me your report as a .pdf by May 13. |