The aim of the first half of the course is for students to master the concepts of probability theory needed to understand two results that are fundamental to statistics, the law of large numbers and the central limit theorem. Along the way, students will also obtain a foundation sufficient for STAT 6501.

The aim of the second portion of the course is for students to master the standard mathematical formulations of the goals of inference and some of the elementary theory for evaluating the statistical methods that acheive these goals. Students will also gain some familiarity with a few of the classical statistical methods. Practical aspects of data analysis, however, will not be covered. Along the way, students will also obtain a foundation sufficient for, for example, STAT 4315, STAT 4220, and STAT 4201.

Old homeworks will be deposited in room 904 in the stat dept building.

No makeup midterm or final will be given.

Homework will be due at the beginning of the following class. No late homework will be accepted.

Students are encouraged to work together on the homework assignments but should write up solutions on their own. Of course, all work on the exams absolutely must be each student's alone.

Solutions to the homework assignments will be posted on Courseworks each week.

Date | Topic | Notes |
---|---|---|

Tu, Sept 4 | Introduction, sample spaces, probability axioms | Read chapter 1 in the book. Due for Th 9/6: Problems 1.4.2, 1.4.4, 1.4.6, 1.5.2, 1.5.4, 1.5.6, 1.5.10, 1.5.12 from the book. Lecture notes for chapter one here (pdf). |

Th, Sept 6 | Combinatorics, Stirling's approximation; conditional probability | Read chapter 2.1-2.3 in the book. Due T 9/11: Problems 1.6.2, 1.6.6, 1.6.8, 1.7.2, 1.7.4, 1.7.6, 1.7.8, 1.7.10, 1.8.2, 1.8.8, 1.8.14, 1.9.4, 1.9.8. |

Tu, Sept 11 | More on conditional probabilities, Bayes rule | Due Th 9/13: Problems 1.12.2, 1.12.10, 2.1.2, 2.1.6, 2.1.8, 2.2.2, 2.2.4, 2.2.6, 2.2.10, 2.3.8, 2.3.10, 2.3.20. Lecture notes for chapter two here (pdf). |

Th, Sept 13 | Independent events. Random variables and distributions; pmf's, pdf's, and cdf's | Read chapter 3 in the book. Due Tu 9/18: Problems 2.2.12, 2.2.14, 2.3.12, 2.3.18, 3.1.2, 3.1.4, 3.1.6. |

Tu, Sept 18 | Multivariate distributions; functions of a random variable; convolution | Due Th 9/20: Problems 3.1.8, 3.2.2, 3.2.4, 3.2.8, 3.2.10, 3.3.2, 3.3.14, 3.4.2, 3.4.4. Lecture notes for chapter three here (pdf). |

Th, Sept 20 | Expectations, variance | Read chapter 4 in the book. Due Tu 9/25: Problems 3.8.2, 3.8.6, 3.8.8, 3.9.2, 3.9.4, 3.9.8, 3.9.16, 3.10.20. Lecture notes for chapter four here. |

Tu, Sept 25 | Moment-generating functions; Covariance and correlation; sample means | Due Th 9/27: Problems 4.1.2, 4.1.12, 4.2.2, 4.2.8, 4.2.10, 4.3.2, 4.3.4, 4.3.6, 4.4.2, 4.4.4, 4.4.10. |

Th, Sept 27 | Inequalities: Markov, Chebyshev, Chernoff, and Jensen; law of large numbers; special discrete distributions | Due for Tu 10/2: Problems 3.5.2, 3.5.4, 3.5.6, 3.6.2, 3.6.4, 3.6.10, 3.7.2, 3.7.8. Lecture notes for inequalities here. |

Tu, Oct 2 | Special continuous distributions; order statistics | Read chapter 5 in the book. Due Th 10/4: Problems 4.5.2, 4.5.6, 4.5.12, 4.6.4, 4.6.8, 4.6.10, 4.7.2, 4.7.6, 4.7.12, 4.8.2, 4.8.8, 4.8.12. Lecture notes for chapter five here. |

Th, Oct 4 | Central limit theorem; convergence in distribution | Due Tu 10/9: Problems 5.2.4, 5.2.6, 5.2.8, 5.3.2, 5.3.6, 5.3.8, 5.4.2, 5.4.8, 5.4.14, 5.5.2, 5.5.6. Notes on CLT here. |

Tu, Oct 9 | Basic simulation theory: Monte Carlo integration, importance sampling | Read chapter 11.1-11.3 in the book. Due Th 10/11: Problems 5.6.2(a)-(d), 5.6.6, 5.6.12, 5.6.14, 5.6.18, 5.7.2, 5.7.4, 5.7.6, 5.7.10. |

Th, Oct 11 | Decision theory | Read chapter 6 in the book, and reread section 4.9. Due Tu 10/16: Problems 5.8.2, 5.8.6, 5.9.6, 5.9.16, 5.9.18, 5.9.22, 5.10.2, 5.10.8, 5.11.4, 11.2.2, 11.2.4. Notes on decision theory here. |

Tu, Oct 16 | Introduction to estimation theory; Bayes estimation; maximum likelihood | Due Th 10/18: Problems 4.9.4, 4.9.6, 4.9.8, 6.2.4, 6.2.6, 6.2.10, 6.3.6, 6.3.10, 6.3.12. Notes on estimation theory here. |

Th, Oct 18 | Midterm review | No HW Due Th 10/18: midterm (covers material in chapters 1-5 and 11). |

Tu, Oct 23 | Midterm exam | No HW due Th 10/25. |

Th, Oct 25 | Bias and variance; more on maximum likelihood | Due Tu 10/30: Problems 6.3.8, 6.4.2, 6.4.4, 6.4.8, 6.5.2, 6.5.6, 6.5.10, 6.6.2, 6.6.4 |

Tu, Oct 30 | Sufficiency; exponential families | Read chapter 7 in the book. Due Th 11/1: Problems 6.6.6, 6.6.12, 6.7.2, 6.7.6, 6.8.4, 6.8.14, 6.9.2, 6.9.8, 6.9.14. |

Th, Nov 1 | More on exponential families: conjugate priors, method of moments. Asymptotic ideas: consistency, asymptotic efficiency | Due Th 11/8: Problems 7.1.2, 7.1.4, 7.1.6, 7.1.8, 7.2.2, 7.2.4, 7.2.10, 7.3.6a, 7.3.8. |

Tu, Nov 6 | No class - University holiday | Remember to vote... |

Th, Nov 8 | Consistency of the MLE; Kullback-Leibler divergence. Asymptotic normality of the MLE; Fisher information; Cramer-Rao bound | Due Tu 11/13: Problems 7.5.2, 7.5.6, 7.5.10, 7.6.2, 7.7.2, 7.7.4, 7.7.6 |

Tu, Nov 13 | Simple hypothesis testing; likelihood ratio tests; Neyman-Pearson lemma | Read chapter 8 in the book. Notes on hypothesis testing here. Due Th 11/15: Problems 7.7.8, 7.7.14, 7.8.2, 7.8.4, 7.8.8, 7.9.6, 7.9.16. |

Th, Nov 15 | Hypothesis testing with compound alternates; uniformly most powerful tests | Due Tu 11/20: Problems 8.1.2, 8.1.4, 8.1.8, 8.1.14, 8.2.4, 8.2.7, 8.3.2, 8.3.6, 8.3.12. |

Tu, Nov 20 | Testing with compound null hypotheses; t-tests, F-tests | Due Tu 11/27: Problems 8.4.4, 8.4.6, 8.4.12, 8.5.2, 8.5.6, 8.5.8, 8.6.2, 8.6.8, 8.7.2, 8.7.8. Read chapter 9 in the book. |

Th, Nov 22 | No class | Happy thanksgiving. |

Tu, Nov 27 | Chi-square tests for goodness of fit, homogeneity, and independence | Due Th 11/29: Problems 9.1.4, 9.1.6, 9.1.8, 9.2.2, 9.2.4, 9.3.4, 9.3.6, 9.4.4. |

Th, Nov 29 | Nonparametrics: Kolmogorov-Smirnov / sign and rank tests. Basics of linear regression. | Due Tu 12/4: Problems 9.6.3, 9.6.4, 9.8.2, 9.8.12, 9.9.4, 9.9.6, 9.9.14 |

Tu, Dec 4 | The EM algorithm; fitting mixture models | Due Th 12/6: E-mail me questions for the review. |

Th, Dec 6 | Review session - last class. | E-mail me questions. See here for some extra course notes to help study. |

Tu, Dec 18 | Final exam | Note change in time: 7:10-10:00 pm (usual place). |