Homework will include a mix of paper and computer problems, and will be assigned in class as we go along; the assignments and due dates will be posted on this webpage. No late homework will be accepted; to compensate for this, we will drop the lowest score.

- Gaussian distributions

- Joint, conditional distributions

- Law of large numbers, central limit theorem

- Estimation

- Bias, variance, covariance

- Maximum likelihood

- Hypothesis testing

- Confidence intervals

The second part of the course will look specifically at the challenges posed by multivariate data. We will do a very brief linear algebra review, but it will be essential to be familiar with the following topics from linear algebra:

- Vectors, matrices

- Linear transformations, bases

- Matrix inverse

- Eigenvalues, eigenvectors

- Quadratic forms

- Determinants

If you haven't taken linear algebra before, don't despair. Some information to help you get started is here.

Date | Topic | Notes |
---|---|---|

W, Jan 18 | Introduction | |

M, Jan 23 | Simple linear regression model; least squares; residuals | Read chapter 1 in book. |

W, Jan 25 | Normal error regression model; maximum likelihood | HW due Feb 1: Exercises 5, 7, 13, 18, 19, 23, 34-36, 40, and 41 from Chapter 1 in the book. (The data sets referred to can be downloaded for free here.) Solutions here. |

M, Jan 30 | Convex optimization: least-squares, least-absolute deviation, least-maximal deviation. Inference in simple normal regression model | Read chapter 2.1-2.6 for this week. |

W, Feb 1 | Proof of Gauss-Markov thm; more on inference in normal regression model | HW due Feb 8: Exercises 1, 4, 13, 50-52, and 54 from Chapter 2 in the book, and problems 1-3 here. Solutions here and some sample contour plot code here. |

M, Feb 6 | Prediction of new observations | |

W, Feb 8 | Analysis of variance (ANOVA); F-test | Read chapter 2.7-2.8. HW due Feb 15: 11, 12, 16, 18, 55-57 from Chapter 2. Solutions here. |

M, Feb 13 | General linear test; coefficient of determination | Read chapter 2.9-2.11. |

W, Feb 15 | Normal correlation model | HW due Feb 22: 53, 59-61, and 66 from Chapter 2, and problems 1-2 here. Solutions here. |

M, Feb 20 | Rank correlation; model diagnostics | Read 3.1-3.7 for this week. |

W, Feb 22 | Goodness of fit | HW due Mar 1: 6, 14, and 19-23 from Chapter 3. Solutions here. |

M, Feb 27 | Remedial measures: weighted least-squares and transformations | Read the rest of Chapter 3 and take a look at Chapter 4 for this week. |

W, Mar 1 | Nonparametric estimation of the regression function; regression through the origin | No more HW due until after the break. |

M, Mar 6 | Midterm review | Bring questions! |

W, Mar 8 | Midterm | |

Mar 13-17 | Spring break |

Date | Topic | Notes |
---|---|---|

M, Mar 20 | Midterm rehash. Linear algebra review: matrix version of simple linear regression. | Read chapter 5. |

W, Mar 22 | Linear algebra review: geometry of quadratic forms, multivariate Gaussians | HW due Mar 29: 17, 20, 24, 26, and 29 from Chapter 5. |

M, Mar 27 | PCA, change of basis, Cochran's theorem and chi-square degrees of freedom | Read chapter 6. (Some more info on PCA and related topics is available here.) |

W, Mar 29 | Multiple linear regression, regression with nonlinear terms | HW due Apr 5: 3, 4, 5, 22, 24, 25 from Chapter 6 and problems 1 and 2 here. Solutions here; code sample here. |

M, Apr 3 | Geometry of normal equations, joint inferences | Read chapter 7. |

W, Apr 5 | More generalized linear tests, standardized variables, and introduction to multicollinearity | HW due Apr 12: 1, 8, 16, 20, 22, 27, 31, and 35 from Chapter 7. Solutions here; code sample here. |

M, Apr 10 | Handling quantitative vs. qualitative predictors | Read chapter 8. |

W, Apr 12 | Model selection: prediction error, cross-validation, BIC | Read chapter 9. HW due Apr 19: 2, 6, 20, 24, and 42 from Chapter 8; 12, 13, and 23 from Chapter 9. Solutions here. |

M, Apr 17 | Outlier detection and handling | Read chapter 10. |

W, Apr 19 | Regularization: ridge regression, robust regression | Read chapter 11. HW due Apr 26: problems 10.12, 10.23, 10.24, 11.21, 11.22, and 14.12 from the book. Solutions here. |

M, Apr 24 | Intro to logistic regression | Read chapter 14. |

W, Apr 26 | More on logistic regression; classification; support vector machines | |

M, May 1 | Last day of class: review | Bring questions! |

W, May 10 | Final exam | During usual class hours, in usual place |