Statistics 4107: Statistical Inference

Fall 2005

This is a master's / advanced undergraduate level course in mathematical statistics.

NOTE: If you're thinking of taking probability (4105, or the equivalent) and this class simultaneously, I strongly recommend you take 4109 instead; 4109 is a double-credit course that covers all the material in 4105 and 4107 in a single semester, with the big advantage that you learn things in the proper sequence.

Time: T, Th 7:40-8:55
Place: Math 520
Instructor: Liam Paninski; Office: 1255 Amsterdam Ave, Rm 1028. Email: liam at stat dot columbia dot edu; be sure to put "4107" at the beginning of the subject line, or I might miss it.
Office Hours: Th 5:30-7:30 (but note that these are subject to change, so check the website before stopping by).
Text: Introduction to Mathematical Statistics, 6th Ed., by Hogg, McKean, and Craig.
TA: Lucy Robinson; Office: 1255 Amsterdam Ave, Rm 1021. Email: lucy at stat dot columbia dot edu.
TA Hours: Th 1-2 in 1255 Amsterdam Ave, Rm 901.
Recitation: Tu 6:10-7:30 in 412 Pupin. (Not required; just for extra help.)
Prerequisite: Stat-IEOR W3000 or Stat W4105, or the equivalent master's-level probability course.

Grading: Grades will be assigned on a curve, using the following percentages: 10% Homework, 20% Quizzes, 30% Midterm, 40% Final.
No makeup midterm or quizzes will be given. (If you miss the midterm, the final will count towards 70% of your grade.)
Homework will be assigned in class as we go along and will be due in class on the Thursday of the week after the assignment. No late homework will be accepted; to compensate for this, we will drop the lowest score. We'll also drop the lowest quiz score (out of about three quizzes).

Final exam: The final will be held on Thursday, Dec. 15, during class hours, 7:40-8:55. Classes end officially on Dec. 12, so we'll have an optional review class on Tuesday, Dec. 13 (again during usual class hours).

Part 1: Probability Review

Course notes available here (pdf, 1.6Mb). Last updated 10/5/05.

Date	Topic	Notes
Tu, Sept 6	Introduction
Th, Sept 8	Sample spaces, probability functions; conditional probability, Bayes' rule, independence; discrete vs. continuous random variables; transformation formulas	Due for Thursday Sept. 15: Exercises 1 and 2 from the lecture notes.
Tu, Sept 13	means, variances; joint, conditional, marginal distributions
Th, Sept 15	correlation, covariance; moment-generating functions, convolution	Due for Thursday Sept. 22: Exercises 4, 5, 6, 8, 9, 11, 14, 15 from the lecture notes.
Tu, Sept 20	special distributions: binomial, Gaussian (normal), Poisson, uniform, exponential, etc.
Th, Sept 22	probability inequalities: Markov, Chebysheff, Chernoff, Jensen, Cauchy-Schwarz. Convergence in probability and distribution. Law of large numbers.	Due for Thursday Sept. 29: Exercises 17, 18, 21, 23, 24, 25, 26, 27 from the lecture notes.
Tu, Sept 27	Central limit theorem, delta method

Part 2: Decision theory fundamentals

Course notes part 2 available here (pdf, 260Kb). Last updated 10/7/05.

Date	Topic	Notes
Th, Sept 29	loss functions, expected loss, worst-case vs. average risk; domination + admissibility of decision rules	Due for Thursday Oct 6: Exercises 31, 32, 33, 36, 37, 39 from the lecture notes.

Part 3: Estimation theory

Course notes part 3 available here (pdf, 640Kb). Last updated 12/08/05.

Date	Topic	Notes
Tu, Oct 4	Bayes estimators, bias and variance, bias-variance decomposition, unbiased, minimum-variance estimators
Th, Oct 6	Maximum likelihood estimation	Quiz today: on material covered in HW sets 1-3. Due for Thursday Oct 13: Exercises 40, 42, 43, 44, 45, 50 from the lecture notes.
Tu, Oct 11	More on MLE
Th, Oct 13	Sufficient statistics	Due for Thursday Oct 20: Exercises 51-54, 57, 58, 60, 61 from the lecture notes.
Tu, Oct 18	Minimal sufficiency; Rao-Blackwell theorem
Th, Oct 20	Exponential families	Due for Thursday Oct 27: Exercises TBA from the lecture notes. (Just kidding, take a break.)
Tu, Oct 25	Midterm review	Come with questions!
Th, Oct 27	Midterm	On material covered in HW sets 1-6 (HW 6 was the one that was due on Oct. 20). Due for Thursday Nov 3: Exercises 62, 64-71 from the lecture notes.
Tu, Nov 1	Midterm post mortem
Th, Nov 3	More on exponential families	Due for Thursday Nov 10: Exercises 73, 75-79 from the lecture notes.
Tu, Nov 8	No class (holiday)	Vote.
Th, Nov 10	Asymptotics of estimation: consistency, method of moments, confidence intervals	Due for Thursday Nov 17: Exercises 83-87 from the lecture notes.
Tu, Nov 15	Consistency of the MLE	Quiz today: on sufficiency and exponential families
Th, Nov 17	More MLE asymptotics: Fisher information	Due for Thursday Dec 1: Exercises 88-95 from the lecture notes.
Tu, Nov 22	Cramer-Rao bound, asymptotic efficiency	Happy Thanksgiving

Part 4: Hypothesis testing:

Course notes part 4 available here (pdf, 220Kb). Last updated 12/06/05.

Date	Topic	Notes
Tu, Nov 29	Simple hypotheses: ML tests; power, size; Neyman-Pearson lemma
Th, Dec 1	Compound alternate hypotheses; uniformly most powerful tests	Due for Thursday Dec 8 (last homework!): Exercises 99-101, 107, 108, and 110-113 from the lecture notes.
Tu, Dec 6	Compound null and alternate; t-tests; likelihood ratio test in compound case
Th, Dec 8	Exam review (optional)	Bring questions!
Tu, Dec 13	Exam review (optional)	Bring questions!
Th, Dec 15	Exam (not optional)	At the usual time and place.