Statistics 4107: Statistical Inference

Fall 2005

This is a master's / advanced undergraduate level course in mathematical statistics.

NOTE: If you're thinking of taking probability (4105, or the equivalent) and this class simultaneously, I strongly recommend you take 4109 instead; 4109 is a double-credit course that covers all the material in 4105 and 4107 in a single semester, with the big advantage that you learn things in the proper sequence.

Time: T, Th 7:40-8:55
Place: Math 520
Instructor: Liam Paninski; Office: 1255 Amsterdam Ave, Rm 1028. Email: liam at stat dot columbia dot edu; be sure to put "4107" at the beginning of the subject line, or I might miss it.
Office Hours: Th 5:30-7:30 (but note that these are subject to change, so check the website before stopping by).
Text: Introduction to Mathematical Statistics, 6th Ed., by Hogg, McKean, and Craig.
TA: Lucy Robinson; Office: 1255 Amsterdam Ave, Rm 1021. Email: lucy at stat dot columbia dot edu.
TA Hours: Th 1-2 in 1255 Amsterdam Ave, Rm 901.
Recitation: Tu 6:10-7:30 in 412 Pupin. (Not required; just for extra help.)
Prerequisite: Stat-IEOR W3000 or Stat W4105, or the equivalent master's-level probability course.

Grading: Grades will be assigned on a curve, using the following percentages: 10% Homework, 20% Quizzes, 30% Midterm, 40% Final.
No makeup midterm or quizzes will be given. (If you miss the midterm, the final will count towards 70% of your grade.)
Homework will be assigned in class as we go along and will be due in class on the Thursday of the week after the assignment. No late homework will be accepted; to compensate for this, we will drop the lowest score. We'll also drop the lowest quiz score (out of about three quizzes).

Final exam: The final will be held on Thursday, Dec. 15, during class hours, 7:40-8:55. Classes end officially on Dec. 12, so we'll have an optional review class on Tuesday, Dec. 13 (again during usual class hours).

Part 1: Probability Review

Course notes available here (pdf, 1.6Mb). Last updated 10/5/05.
Date Topic Notes
Tu, Sept 6 Introduction
Th, Sept 8 Sample spaces, probability functions; conditional probability, Bayes' rule, independence; discrete vs. continuous random variables; transformation formulas Due for Thursday Sept. 15: Exercises 1 and 2 from the lecture notes.
Tu, Sept 13 means, variances; joint, conditional, marginal distributions
Th, Sept 15 correlation, covariance; moment-generating functions, convolution Due for Thursday Sept. 22: Exercises 4, 5, 6, 8, 9, 11, 14, 15 from the lecture notes.
Tu, Sept 20 special distributions: binomial, Gaussian (normal), Poisson, uniform, exponential, etc.
Th, Sept 22 probability inequalities: Markov, Chebysheff, Chernoff, Jensen, Cauchy-Schwarz. Convergence in probability and distribution. Law of large numbers. Due for Thursday Sept. 29: Exercises 17, 18, 21, 23, 24, 25, 26, 27 from the lecture notes.
Tu, Sept 27 Central limit theorem, delta method

Part 2: Decision theory fundamentals

Course notes part 2 available here (pdf, 260Kb). Last updated 10/7/05.

Date Topic Notes
Th, Sept 29 loss functions, expected loss, worst-case vs. average risk; domination + admissibility of decision rules Due for Thursday Oct 6: Exercises 31, 32, 33, 36, 37, 39 from the lecture notes.

Part 3: Estimation theory

Course notes part 3 available here (pdf, 640Kb). Last updated 12/08/05.

Date Topic Notes
Tu, Oct 4 Bayes estimators, bias and variance, bias-variance decomposition, unbiased, minimum-variance estimators
Th, Oct 6 Maximum likelihood estimation Quiz today: on material covered in HW sets 1-3. Due for Thursday Oct 13: Exercises 40, 42, 43, 44, 45, 50 from the lecture notes.
Tu, Oct 11 More on MLE
Th, Oct 13 Sufficient statistics Due for Thursday Oct 20: Exercises 51-54, 57, 58, 60, 61 from the lecture notes.
Tu, Oct 18 Minimal sufficiency; Rao-Blackwell theorem
Th, Oct 20 Exponential families Due for Thursday Oct 27: Exercises TBA from the lecture notes. (Just kidding, take a break.)
Tu, Oct 25 Midterm review Come with questions!
Th, Oct 27 Midterm On material covered in HW sets 1-6 (HW 6 was the one that was due on Oct. 20). Due for Thursday Nov 3: Exercises 62, 64-71 from the lecture notes.
Tu, Nov 1 Midterm post mortem
Th, Nov 3 More on exponential families Due for Thursday Nov 10: Exercises 73, 75-79 from the lecture notes.
Tu, Nov 8 No class (holiday) Vote.
Th, Nov 10 Asymptotics of estimation: consistency, method of moments, confidence intervals Due for Thursday Nov 17: Exercises 83-87 from the lecture notes.
Tu, Nov 15 Consistency of the MLE Quiz today: on sufficiency and exponential families
Th, Nov 17 More MLE asymptotics: Fisher information Due for Thursday Dec 1: Exercises 88-95 from the lecture notes.
Tu, Nov 22 Cramer-Rao bound, asymptotic efficiency Happy Thanksgiving

Part 4: Hypothesis testing:

Course notes part 4 available here (pdf, 220Kb). Last updated 12/06/05.

Date Topic Notes
Tu, Nov 29 Simple hypotheses: ML tests; power, size; Neyman-Pearson lemma
Th, Dec 1 Compound alternate hypotheses; uniformly most powerful tests Due for Thursday Dec 8 (last homework!): Exercises 99-101, 107, 108, and 110-113 from the lecture notes.
Tu, Dec 6 Compound null and alternate; t-tests; likelihood ratio test in compound case
Th, Dec 8 Exam review (optional) Bring questions!
Tu, Dec 13 Exam review (optional) Bring questions!
Th, Dec 15 Exam (not optional) At the usual time and place.

Other topics (time/interest permitting)

  • multiple hypothesis testing
  • complete sufficient statistics
  • robust statistics
  • nonparametric procedures
  • overfitting
  • experimental design