Joe Blitzstein and Xiao-Li Meng write:
An e ffectively designed examination process goes far beyond revealing students’ knowledge or skills. It also serves as a great teaching and learning tool, incentivizing the students to think more deeply and to connect the dots at a higher level. This extends throughout the entire process: pre-exam preparation, the exam itself, and the post-exam period (the aftermath or, more appropriately, afterstat of the exam). As in the publication process, the first submission is essential but still just one piece in the dialogue.
Viewing the entire exam process as an extended dialogue between students and faculty, we discuss ideas for making this dialogue induce more inspiration than perspiration, and thereby making it a memorable deep-learning triumph rather than a wish-to-forget test-taking trauma. We illustrate such a dialogue through a recently introduced course in the Harvard Statistics Department, Stat 399: Problem Solving in Statistics, and two recent Ph.D. qualifying examination problems (with annotated solutions). The problems are examples of “nano-projects”: big picture questions split into bite-sized pieces, fueling contemplation and conversation throughout the entire dialogue.
This is just wonderful and it should be done everwhere, including, I hope, in my own department. I am so tired of arguments about what topics students should learn, long lists of seemingly-important material that appears on a syllabus, is taught in a class, and is never used again, and so forth.
(The exam problems described in the article are a bit on the theoretical side for my taste, but I presume the same ideas would apply to applied statistics as well.)
P.S. I have fond memories of my own Ph.D. qualifying exam, which I took a year before Xiao-Li took his. It was an intense 12-day experience and I learned a huge amount from it.
Like Andrew, I really enjoyed my quals (Edinburgh Uni Cognitivie Science, 1985). We had four qual areas (syntax, semantics, comp ling, psycholing), and the exam spanned two weeks, with two topics/week. For each, we got questions Monday AM and had two or three days to produce answers, which were supposed to be like short papers, not like exam answers.
In the CMU comp ling program, we didn't give exam-based quals, but rather required a bunch of courses, an MS project (not a thesis, because then the university got involved a bit), and for the PhD students, one or two area qualifying papers (no time pressure) that were more like a literature survey than anything else. We tried to get students to write their thesis proposals as soon as possible after the quals.
I was a professor for eight years, teaching two classes/semester, and I never gave out a single exam. I think projects are much better. They were like the "nano-projects" cited by Blitzstein and Meng. After the intro classes, we really focused on teaching people to write real research papers, not exam answers.
Most effective, I think, was the second-year project course for MS and PhD students, which ran all year. If I recall, it was the brainchild of David Evans, but I taught it for a couple of years. It was for MS and PhD students. The students selected a topic at the end of the first year, and were supposed to show up in year 2 with a literature survey and a basic idea of what they wanted to do (most of our students were around for the summer, too). Then we did all sorts of things like have them review each other's papers (students are harsh), assign students to ask questions on talks of other students (they'll be the host for speakers at some point where the audience doesn't ask a question), review "real" papers (hey, might as well get some real work out of them), of others, write proposals, and produce a finished MS-thesis-like project.
Nowadays, I'd probably have them write blog entries, if not tweets, as well as posters, none of which were relevant at the time (we only started posters in comp ling recently). The real goal is to teach/help the students to conceive, execute, and then market their ideas.