Recently in Teaching Category

Guilherme Rocha writes:

Here. Official opening is Monday but youall get to see it earlier.

After reading what Christian wrote about the solutions manual that he and his collaborator wrote for their Bayesian Core book, I'm reminded of my own efforts with the Bayesian Data Analysis solutions. Not long after the first edition of the book came out, nearly fifteen years ago, I wrote up solutions to fifty of the homework problems (out of a total of about 150). People sometimes ask for more solutions, but the difficulty is that, once you have official solutions, you want them to be correct, you want them to be clear, and you want them to illustrate good statistical practice. It's a lot of work. Somehow I was able to write up those fifty, back when I had more time on my hands, but, really, writing up another fifty would almost be the equivalent of writing a (short) book! Originally I thought I could quickly put together a complete or nearly complete set by gathering solutions from students, or people just emailing them in, but I quickly realized that this wouldn't work. I think it would be ok to post scanned-in versions of student solutions, but once I start typing them up, I need them to be cleaner, and that takes work. That's one reason I didn't even try to write a solution set for ARM.

Phil Turk wrote:

Carl Bialik reports on a website called the Book of Odds (really, as Carl points out, these are probabilities, not odds, but that's not such a problem because, at least to me, probabilities are much more understandable than odds anyway). It's pretty cool. I could give some examples here but I encourage you to just go to the site yourself and check it out. One thing I really like is that it gives the source of every number: right on the page it gives the country and date of the information, then you can click to get the details. Awesome.

The only thing that bothers me a little bit about the site is that it is almost too professional. When something's that slick, I worry about whether I can trust them.

In contrast, Nate Silver's website is respected but not particularly attractive. And the NameVoyager is just the coolest thing in the world, and, yes, it's professional and it's commercial--that's fine--but it doesn't have the suspicion-inducing hyper-professionalism of the Book of Odds. Seeing the all-so-appealing photo of the bright-eyed oldsters illustrating the "Will you live to be 100?" item that's currently featured on the site's main page, I just think--this is too slick to be trusted. (In case you're wondering, their data say that a randomly-chosen 90-year-old has only a 1-in-9 chance of living to 100. Actually, they say 1 in 8.85, but you know what I think about extra decimal points.)

In some way I prefer the charmingly and unabashedly commercial OK Cupid site to the Book of Odds, which looks so, so commercial but claims only purely altruistic goals. I just don't know what to think.

Anyway, whatever the true story happens to be, it's great stuff. Fun to browse, and a great teaching tool too, I'd think. Enjoy.

Helen DeWitt links to this British-style rant (much different from my own story), which reminds me that I never erase the board after my classes either. But the board is usually clean for me. Probably because I usually teach morning classes. Once, though, about 15 years ago, I taught in a classroom right after a biology professor, an elderly lady who used several colors of chalk to construct beautiful pictures on the blackboard every lecture. She was very good about erasing as I was coming in. That semester and the one after that, I remembered to erase the board after class, but I've since slipped to the more natural equilibrium.

One more thing. A few years ago I invited Frank Morgan to give a talk at Columbia on how to teach better. One of the little things he discussed was how to use the blackboard. One of his points was that students aren't always paying attention--and, even if they are, they are often lost in thought trying to make sense of something or another. As a result, they won't be following every step of yours on the blackboard.

To put it another way: to the lecturer, the blackboard is a space-time process in which things get written down, erased, modified, and so on. To many a student, the blackboard is a series of snapshots of a spatial process. And it helps if each of these snapshots makes sense on its own terms. To that end, Frank recommends that you write a header at the top of each section of the blackboard so that students can keep track of what is going on.

Colin Gillespie writes:

A couple of weeks ago I did your suggested exercise (from Teaching Statistics: A Bag of Tricks) on 'Guessing the age', with the additional twist that the people were actors/actresses out of CSI. As well as discussing the data in class, I used it for there first R lab, where they generated simple scatterplots, boxplots and histograms.

In case your interested, the main results where:

1. Watching CSI didn't seem to affect your guess

2. Females guessed better than males.

3. The vast majority of guesses where too low (unsurprising for actors), except for the youngest actor.

If you are interested you can find my slides/handouts here.

Cool!

Pour l'année sabbatique, à Sciences Po.

John Sides and Joshua Tucker link to a news article by Jeremy Peters that reports that former New York State governor Eliot Spitzer is teaching a course at the City University of New York for "$98.43 an hour, or about $4,500 for the semester." This comes out to 45 or 46 hours--let's say 3 classroom hours a week for 15 weeks.

I noticed a few interesting things in the article.

1. I think it's ridiculous to consider $4500 for a course to be a rate of $98/hour. Teaching isn't just lecturing. You also have to prepare the classes, meet with students, write exams, and grade homeworks. $98/hour sounds like a lot, but it's based on a low denominator.

(There are exceptions, though. I know of a professor who paid the T.A. $100 per lecture to teach the class when the prof was out of town. It happened several times during the semester.)

2. I thought it was interesting that the commenters identified in the news article seemed to think that $4500 was a high rate of pay. I mean, suppose you teach 8 courses a year at $4500 each. That's $36,000. Hardly Richie Rich territory. This point is made at the very end of the article ("The point is not that Spitzer is paid too much, but rather that most adjuncts are paid too little") but it didn't really come through at first.

3. Sides writes that "This isn't pretty." I don't see what's so bad about Spitzer teaching a class. He knows a lot about politics and would seem to be well qualified to be an adjunct professor. I thought that was the ideal, to have adjuncts who are working professionals who take time off to teach a class.

I received the following email:

Rebecca Weitz-Shapiro points me to this blog by Luis von Ahn, suggesting that college lectures be replaced by high-production-quality Hollywood-style videos. You can click over to see his arguments, but, I gotta say, I don't buy it. When I teach a course, the goal of the classroom time (I don't call it "lectures") is to to get the students engaged in the material and thinking hard, not to sit back and be entertained, which is what you're getting from the Hollywood video.

Lee Wilkinson writes:

Also, someone asked me yesterday about Central Limit Theorem Java applets. I [Lee] looked out there and wasn't too impressed with the ones I saw. They didn't convey the essential aspects of the theorem and they were cluttered with unnecessary detail. So I [Lee] wrote this one.

Looks good to me!

I received the following email:

David Afshartous writes:

I recall you had a post awhile back RE the difficulty kids have excelling in statistics versus mathematics, e.g., there are few statistics prodigies yet many mathematics prodigies. In any event, my 10 year old nephew was on his school math team last year and I helped him with his homework which consisted mainly of previous math competition problems (2xweek via skype video). It seemed like they were developing a bag of tricks and not learning the underlying material behind the problems. As he is on the fence about joining the math team in the fall, I'm thinking about continuing our weekly meetings but teaching him basic statistics/probability instead. As I don't want to turn him off from the subject at an early age, my guess is that I should focus on fun probability problems that he can relate to (e.g., binomial problems related to basketball, or perhaps mix in some intriguing aspects of the history of probability) and then later introduce additional material. I'd like to come up with a plan for the semester and would appreciate any advice you have on what a 10 yr old should be taught in statistics/probability.

My reply:

First off, I envy your nephew. I had zero math education at age 10. No math team, nothing like that. I just considered myself lucky when the teacher let me sit in the library and read books.

I do remember math team from high school, and I agree that much of it was centered around silly tricks. On the other hand, silly math tricks are still math. I don't know that he really needs to learn the underlying principles right away. Maybe what it really takes is the proverbial 10,000 hours of practice. If he's enjoying it, that should be fine.

If you're doing statistics and probability . . . I really have no idea! I personally like a lot of the games in my Bag of Tricks book, so you could start with some of them. A natural area of applications would be board games, if he likes Monopoly or Scrabble or whatever, there are a lot of probabilities to calculate. You could also try getting a little roulette set, if you're not worried about turning him into a gambling addict.

Any other ideas out there?

Xiao-Li wrote an article on his experiences putting together a statistics course for non-statistics students at Harvard. Xiao-Li asked for any comments, so I'm giving some right here:

I think the ideas in the article are excellent.

The challenges of getting students actively involved in statistics learning have motivated me to write a book on teaching statistics, develop a course on training graduate students to teach statistics, and even to offer general advice on the topic.

But I have not put it all together into a successful introductory course the way Xiao-Li has, and so I read his article with interest, seeking tips in how we can do better in our undergraduate teaching.

The only thing I really disagree with is Xiao-Li's description of statisticians as "traffic cops on the information highway." Sure, it sounds good, but often I find my most important role as a statistician is to tell people it's ok to look at their data, it's ok to fit their models and graph their inferences. There's always time to go back and check for statistical significance, but I've found the biggest mistakes are when scientists, fearing the statistician over their shoulder, discard much of their information and don't spend enough time looking at what they have left.

I'm certainly not arguing that simple methods are all we need. (See here for my recent advertisement for fancy modeling). What I'm saying is that I'm happier being an enabler than a police officer. I think I've done more good by saying yes than by saying no.

On the other hand, in Xiao-Li's defense, he's prevented three false discoveries (see bottom of page 206 of his article), whereas I've proved one false theorem. So perhaps we just put different values on our Type 1 and Type 2 errors!

To return to XL's article, on pages 207-208 he tells a story involving a scientist who was stopped just in time before making a big mistake, by discussing the questionable analysis with Policeman Meng, who noticed the problem. I assume we can all agree that the crucial step in this process was that the scientist was (a) worried that something might be wrong and (b) went to a statistician for help. I'd like to believe that many of the readers of this article would've been able to find the problem, but this sort of eagle-eyed criticism is different from what I think of as the most common bit of policing, which is statisticians giving scientists a hard time about technicalities.

Or, to put it another way, I don't mind the statistician as critic, but I don't think we should have the police officer's traditional power to arrest and detain people at will. Except maybe in some extraordinary cases.

To return to undergraduate education: I've taught undergraduate statistics several times at Berkeley and at Columbia. Berkeley had an exciting undergraduate program with about 15 juniors and seniors taking a bunch of topics classes. I have fond memories of my survey sampling and decision analysis classes and also of the department's annual graduation ceremony, which included B.A.'s, M.A.'s, and Ph.D.'s in one big celebration. I've heard that the program has since grown to about 50 students. At Columbia, in contrast, we have something in the neighborhood of 0 statistics majors. It's a feedback loop: few courses, few students, few courses, etc. I think this was the case at Harvard for many many years, although maybe it's changed recently.

My point? The intro courses at Berkeley for non-majors were very well organized, much more so than at Columbia, at least until recently. Perhaps no coincidence. I suspect it's easier to confidently teach statistics to non-majors if you have a good relationship with the select group of undergraduates who are interested enough in statistics to major in it. And, conversely, an excellent suite of introductory statistics classes is a great way to interest students in further study.

Teacher training is also important, as Xiao-Li indicates in the last sentence of his article. At Berkeley there was no formal course in statistics teaching, but most of the Ph.D. students went through the "boot camp" of serving as T.A.'s in large courses under the supervision of experienced lecturers such as Roger Purves; between this direct experience and word-of-mouth guidance from other students in the doctoral program, they quickly learned which way was up. At Columbia we have recently revived our course, The Teaching of Statistics at the University Level, and I hope that this course--and similar efforts at Harvard and other universities--will help move us in the right direction.

In addition, wider awareness of statistical issues outside of academia (for example, at our sister blog) will, I hope, make college students demand statistical thinking in all their classes, whether taught by statisticians or not. It wouldn't be a bad thing for a student in a purely qualitatively-taught history class to consider the role of selection bias in the gathering of historical data (see Part 2 of A Quantitative Tour for more on this sort of thing), just as it isn't a bad thing for a student in a statistics class to think about the social implications of some of the methods we use.

Peter Flom writes:

I am now up for a position which would require teaching some introductory statistics to people studying to work in health care. Mostly, these people will have only a HS diploma, and it may be a fairly old HS diploma (a lot of them are returning to school).

For the interview, though, I am assigned to give a 30 minute talk (no powerpoint or anything, just a white board).

I received the following email:

The American Statistical Association organizes a program in which young researchers can submit writing samples and get comments from statisticians who are more experienced writers. I agreed to participate in this program, as long as the authors were willing to have their articles and my comments posted here.

I'm going to start with my general advice after reading and commenting on the two articles sent to me. I think this advice should be of interest to nearly all the readers of this blog. Then I'll link to the articles and give some detailed comments.

General advice

Both the papers sent to me appear to have strong research results. Now that the research has been done, I'd recommend rewriting both articles from scratch, using the following template:

1. Start with the conclusions. Write a couple pages on what you've found and what you recommend. In writing these conclusions, you should also be writing some of the introduction, in that you'll need to give enough background so that general readers can understand what you're talking about and why they should care. But you want to start with the conclusions, because that will determine what sort of background information you'll need to give.

2. Now step back. What is the principal evidence for your conclusions? Make some graphs and pull out some key numbers that represent your research findings which back up your claims.

3. Back one more step, now. What are the methods and data you used to obtain your research findings.

4. Now go back and write the literature review and the introduction.

5. Moving forward one last time: go to your results and conclusions and give alternative explanations. Why might you be wrong? What are the limits of applicability of your findings? What future research would be appropriate to follow up on these loose ends?

6. Write the abstract. An easy way to start is to take the first sentence from each of the first five paragraphs of the article. This probably won't be quite right, but I bet it will be close to what you need.

7. Give the article to a friend, ask him or her to spend 15 minutes looking at it, then ask what they think your message was, and what evidence you have for it. Your friend should read the article as a potential consumer, not as a critic. You can find typos on your own time, but you need somebody else's eyes to get a sense of the message you're sending.

Tobias Verbeke writes:

I just noticed in your blog post you use Sharon Lohr's book on sampling design and analysis for your course.

Some time ago I made an R package with the datasets and a vignette which reproduces part of the analyses with Thomas Lumley's survey package.

This could be useful.

I wasn't actually so thrilled with how the course went--I last taught it a few years ago--but I thought it might help to share some of my experiences.

1. I used the excellent book by Lohr. And students always like when you follow the book.

2. That said, whenever I deviated from the straight sampling stuff and talked about modeling (for example, forecasting or missing data imputation or just an overview of regression), they loved it. Our students are much more interested in modeling than in sampling.

3. You have to decide ahead of time how much you want them to do with real data on the computer, and how much you want to have them deriving formulas. Either is ok, you just need to figure that out.

4. Stata is the standard software for survey sampling. I use R because that's what I know.

5. Lohr's book, like all books on surveys, is strongest on design, and weakest on analysis of surveys collected by others (survey weights and the like).

6. I assigned the Groves et al book as a supplementary text. It's a great book, but it didn't work so well to teach out of. It's still probably a good idea to assign it, just so students have it as a reference.

Here's a syllabus, a schedule of homework assignments, and some notes.

Greg Mankiw links to an article that illustrates the challenges of interpreting raw numbers causally. This would really be a great example for your introductory statistics or economics classes, because the article, by Robert Book, starts off by identifying a statistical error and then goes on to make a nearly identical error of its own! Fun stuff.

This note by Steve Hsu on the history of the Wranglers (winners of a mathematics competition held each year from 1753-1909 at Cambridge University) reminded me of my experience in the U.S. math olympiad training program in high school. At the time, it seemed clear that we were clearly ordered by ability (with my position somewhere between 15th and 20th out of 24!). In retrospect, I think there are a lot of tricks to solving and writing up solutions to "Olympiad problems," and I didn't know a lot of these tricks.

It was the usual paradox of measurement: I was confusing reliability with validity, as they say in the psychometric literature.

The Howard Wainer story.

On of the fun parts is this story from his days as an assistant professor:

Donna Harrington writes:

I will be teaching a new multilevel models course in the fall and am currently reading your text, /Data Analysis Using Regression and Multilevel/Hierarchical Models/ as I prepare. I am enjoying the book and am considering adopting it for use in the course.

Would you be willing to share the syllabus you have used for your Applied Regression and Multilevel Models course? I am particularly interested in seeing how much of the book you use in a one semester course.

My reply:

I have to admit that, over the years, I've made my syllabuses less and less detailed as I've focused more and more on writing the books. For a multilevel modeling course, I suggested the following:

- chapters 3,4,5: linear and logistic regression
- chapter 7: basics of simulation
- chapter 9: basics of causal inference
- chapters 11-14: multilevel linear and logistic regression (up to and including varying-intercept, varying-slope models)
- chapter 18: all the theory that they'll need.

For a one-semester introductory course, my usual strategy for a one-semester course is to focus chapters 2-10: that is, cover everything except multilevel modeling. Linear regression, logistic, glm, computation, and causal inference. Then for the last part of the course, I can choose among some options, including: intro to multilevel models, sample size and power calculations, and missing data imputation.

P.S. To those of you who haven't had the opportunity to take a course from me: Don't worry about it. I'm better at writing than teaching. Maybe you're better off learning out of one of my books with somebody else actually teaching the class.

The only thing that puzzles me about this article (sent to me by Chris Wiggins) is that at first it's presented as new: "The trend is buried deep in United States census data . . " A couple paragraphs down, the article explains that these patterns were published last year by Lena Edlund and Doug Almond (who presented the results in our quantitative political science seminar). In any case, it's an excellent news article and discusses the issues well. The only thing I'd like to see are some sample sizes, so that students who are given this article to read can compute the standard errors on their own.

Also, I have a couple problems with their graph. First, I'm not a fan of expressing sex ratios as #boys per 100 girls. To me, it's clearer just to give %girls (or %boys) as a straight number: 48.8% or whatever. Second, it's a mistake to make these as bar graphs starting at zero. Here, zero is not a reasonable baseline: it's not like you're really expecting to see zero girl births. I appreciate that they were trying to make a pretty graph, but in this case I'd go with a simple dot plot with +/- 1 standard error bars on the points. Or, better still, a line plot with time on the x-axis (one point for each decade) lines connecting the dots for each ethnic group, and also the vertical lines indicating standard errors.

Line plots are the best, and it's great when you can put time on the x-axis.

David VandenBos writes:

I stumbled upon your blog a few weeks ago . . . However, a good amount of your technical articles go over my head because of my lack of statistics education/training/experience. Do you have any basic reading suggestions for learning applied statistics? My organization captures tons of info and safely tucks it away into databases, but I'm really interested in learning how to get it out and make use of it.

Does anybody have any suggestions? I like my book with Jennifer but maybe there's something more basic to start with? There's also this online book on statistical graphics by Rafe Donahue which is actually fun to read.

P.S. I don't think any of the usual intro stat books would be good here. I think they focus too much on conventional topics and not enough on applied statistics. Not really the fault of these books: they're designed for the undergraduate curriculum, not for practitioners.

I am at a conference which had an excellent poster session yesterday. I realized the session would have been even better if the students with posters had been randomly assigned to stand next to and explain other students' posters. Some of the benefits:

1. The process of reading a poster and learning about its material would be more fun if it was a collaborative effort with the presenter.

2. If you know that someone else will be presenting your poster, you'll be motivated to make the poster more clear.

3. When presenting somebody else's poster, you'll learn the material. As the saying goes, the best way to learn a subject is to teach it.

4. The random assignment will lead to more inderdisciplinary understanding and, ultimately, collaboration.

I think just about all poster sessions should be done this way.

P.S. In reply to comments:

- David writes that my idea "misses the potential benefit to the owner of the poster of geting critical responses to their work." The solution: instead of complete randoimization, randoimize the poster presenteres into pairs, then put pairs next to each other. Student A can explain poster B, student B can explain poster A, and spectators can give their suggestions to the poster preparers.

- Mike writes that "one strong motivation for presenters is the opportunity to stand in front of you (and other members of the evaluation committee) and explain *their* work to you. Personally." Sure, but I don't think it's bad if instead they're explaining somebody else's work. If I were a student, I think I'd enjoy explaining my tellow-students' work to an outsider. The ensuing conversation might even result in some useful new ideas.

- Lawrence suggests that "the logic of your post apply to conference papers, too." Maybe so.

Daniel Becker's Random-Walk graphically demonstrates how different distributions can be generated with physical processes: Normal distribution falls out of a Pachinko machine, and Poisson from a dart-throwing process. He also shows how pseudo random number generators have higher-order correlations within them. Pretty!

[via Infosthetics]

Andrew Grogan-Kaylor writes:

Aleks sent me this. I have nothing to say on the substance here, but the grumpy-old-man quotes are amusing:

A Quantitative Tour of the Social Sciences has just come out. The book is edited by Jeronimo Cortina and myself, and it is intended to give the reader a sense of how research is done in different areas of social science. It is not a book of statistical methods, nor is it that sort of academic book that has a zillion little chapters of things that people submitted because they couldn't get them accepted into journals. Rather, it is a set of in-depth examples and discussions of social science research from a variety of perspectives.

I think the book should be especially useful for courses for graduate students or advanced undergraduates in social science, who typically aren't familiar with the way people think in neighboring fields. For example, a political science student might know a little bit about economics but nothing about psychology. Or a sociology student might not know much about historical data collection. And so forth.

Here's the table of contents:

I. Models and Methods in the Social Sciences (Andrew Gelman)
1. Introduction and overview
2. What's in a number? Definitions of fairness and political representation
3. The allure and limitations of mathematical modeling: Game theory and trench warfare

II. History (Herbert Klein and Charles Stockley)
1. Historical background of quantitative social science
2. Sources of historical data
3. Historical perspectives on international exchange rates
4. Historical data and demography in Europe and the Americas

III. Economics (Richard Clarida and Marta Noguer)
1. Learning from economic data
2. Econometric forecasting and the flow of information
3. Two studies of interest rates and monetary policy

IV. Sociology (Seymour Spilerman and Emanuele Gerratana)
1. Models and theories in sociology
2. Demographic explanations of social disturbances in the 1960s
3. Studying the time series of lynchings in the South
4. Attainment processes in a large organization

V. Political Science (Charles Cameron)
1. What is political science?
2. The politics of Supreme Court nominations: the critical role of the media environment
3. Modeling strategy in congressional hearings

VI. Psychology (E. Tory Higgins, Elke Weber, and Heidi Grant)
1. Formulating and testing theories in psychology
2. Some theories in cognitive and social psychology
3. Signal detection theory and models for tradeoffs in decision making

VII. To Treat or Not to Treat: Causal Inference in the Social Sciences (Jeronimo Cortina)
1. The potential-outcomes model of causation; propensity scores
2. Some statistical tools for causal inference with observational data
3. Migration and Solidarity

The cover is an adaptation of this image that was sent to us from Chris Albon last year after we asked for cover ideas on the blog. Thanks, Chris. You're getting a free copy!

A couple days ago I asked what was it that Brad Paley did to get such active participation in his seminar. I can get active participation, but it takes work: I have to ask students to work in pairs to prepare questions, etc. Brad didn't need such mechanical tricks; the participation came naturally.

So what did he do? Brad writes:

I'm glad my "seminar style" presentation worked for you--sometimes I'm a little worried that people won't follow the digressions, but I think the material holds together well in itself and I try hard to shape/steer the digressions back to the "main topic." BTW: I write "main topic" in quotes because the outline of my talks is typically pretty strategically defined, and I let people steer me towards the things they want to get from me, figuring that what they ask will always stick better in their minds and be a better "gift" from me to them than if I just plowed through some pre-arranged plan. (It's important that the material--really anything I do--is a gift to the audience rather than a plea for attention; and when I realize it's about them, not me, there's less pressure to push my point.)

Programmer/designer W. Bradford Paley spoke yesterday for the data visualization group here at Columbia. He gave an amazing talk, one of the best I've ever seen. One reason I say this is that about half the talk was devoted to an application he built for Wall Street trading--something I just couldn't care less about, it's hard for me to imagine a topic I'm less interested in--and, even so, I liked the talk a lot.

The seminar participants--a mixture of architects, computer scientists, and some other people, including a psychologist and even a statistician--had lively discussion throughout. In fact, there was so much going on, that I'll spread my comments through several blog entries over the next few days.

Right now I want to talk about Paley's speaking style, which was great in so many ways, but what really got to me was how he managed to get so many questions and comments from the audience---so much that people had to ask him to stop taking questions so he could move forward with the material. This was amazing. When I speak, I always struggle to get audience participation. Usually I get a few questions at the end, but not this kind of barrage all the way through.

What can I do to involve the audience more? I've always thought I need more "hooks" but have not been sure how to do it. After seeing Paley's talk, my new idea is to devote more of my talks to process. I typically present results without a lot of detail on how I got there. But maybe it would be better to talk more about what I did. At least, that worked for Paley.

The funny thing is that I love answering questions, and I think I'm good at it. That's one reason I get so frustrated that I don't get more questions when I speak. People typically think my talks are entertaining, informative, and thought provoking--at least, that's what they tell me--but I'm lacking the hooks that draw people in.

P.S. More here on Paley's talk.

Here [link fixed]. I love this stuff.

Antonio Ramos writes:

In most social sciences, maximum likelihood is taught before topics such as multilevel modeling and or Bayesian statistics more generally. However, in the preface of your multilevel book you say that basic statistics and regression classes are sufficient for one to study your book. So may question is: should we learn maximum likelihood first or it is just historical convention, without much pedagogical basis?

My reply: Maximum likelihood is fine; we discuss it in chapter 18, I believe it is, where we discuss Bayesian methods as a generalization of maximum likelihood. All of this is important to learn, but I think you can get started in serious applied statistics (which is what our book is about) without necessarily already knowing it.

Image by George Eastman House via Flickr

Newspapers are dying. Are universities next? The parallels between them are closer than they appear. Both industries are in the business of creating and communicating information. Paradoxically, both are threatened by the way technology has made that easier than ever before.[...]

And it would be a grave mistake to assume that the regulatory walls of accreditation will protect traditional universities forever. Elite institutions like Stanford University and Yale University (which are, luckily for them, in the eternally lucrative sorting and prestige business) are giving away extremely good lectures on the Internet, free. Web sites like Academic Earth are organizing those and thousands more like them into "playlists," which is really just iPodspeak for "curricula." Every year the high schools graduate another three million students who have never known a world that worked any other way.

There are four value propositions of universities:

1. Helping students familiarize themselves with a topic (an informative, entertaining lecture showing why someone should care and up-to-date pointers)
2. Helping students master the applications of a topic (project supervision, in-depth tutorials, apprenticeship)
3. Helping students contribute to a topic and push the envelope (research apprenticeship, collaboration, leadership)
4. Network building among students of similar capability, similar or complementary interests

The first value proposition can be done more effectively with the use of the WWW, but one should be careful: if this content isn't compensated, the compensation will take the form of sometimes hidden advertising (did you notice the image on this post?). The second can partly be done remotely with a lower cost - at places like Columbia, it's often the grad students who tutor, while professors do research. The third requires excellence, dedication and research funding - if there is no funding, the research will become confidential and proprietary. The fourth requires sorting and community organization.

It is always a good idea to rapidly adopt new technology, some institutions and individuals are pushing the edge with either video lectures, or with course materials. We're trying to innovate with blogging.

[Updated after reading thoughtful comments by hal, yolio and Igor Carron. This blog is really a community.]

Jeronimo writes:

I have been using small whiteboards in my research methods class to have the students work in pairs and it has been a huge success.

I asked, "How large are the whiteboards? And why do you use these rather than simply having them work in their notebooks?" and he responded:

The whiteboards are about 8x11. I like the boards because it changes the dynamic of the class. It introduces the sense of doing something different and also they can erase everything and start all over again. And I guess we don't waste a lot of paper.

I'll try it for the next course I teach.

P.S. As Seth might say, how come I have no problem with anecdotal evidence in education--the area in which I actually work--but when it comes to medicine and public health I focus on potential selection biases, insist on randomized trials, etc. In my defense, I'd point out that there has been some education research showing the benefits of working in pairs, peer instruction, and so forth--thus the "whiteboard for each pair of students" idea makes sense. But, then again, medical interventions typically make sense, whether or not they work (recall The Doctor's Dilemma).

I got this bit of spam in the email but it's actually sort of cool, would be an excellent topic for discussion in an intro stat class or a Bayesian class:

MEDIA ALERT: NCAA COLLEGE BASKETBALL TOURNAMENT - MARCH MADNESS NCAA College Basketball Tournament Bracket-Picking Tips. RJ Bell of Pregame.com, the top Las Vegas based sports betting authority, provides a simple blueprint to improve anyone's bracket results.

See here for my thoughts on the surprising stability of the economics curriculum.

I received the following email:

I am working on a paper for a political course which I must discuss "a what if" Pennsylvania transformed into a state of rationality. Everything is the same except that all the citizens, all the candidates for state office, all the state legislators, and all the lobbyists in the state behave rationally in a economic sense. Of these groups, which one is most likely to be the most politically powerful.

I am not sure how to exactly get started and thought I would see if you might have any suggestions or thoughts on the subject.

Sounds like a good assignment to me. I only teach statistics and methods courses, so I never think about this sort of interesting political-science homework problem.

Image by jurvetson via Flickr

Thus, a master's degree actually hurts performance, and seniority was irrelevant as a factor. But master's degree and seniority are the only two factors that will increase a teacher's pay.

Now, Gates is pushing a lot for gathering and analyzing data. So there might be opportunities for those interested in doing research in education to get grants from the Gates foundation.

In the spirit of Bullwinkle, I think that all blog entries should be required to have two titles. . . .

Anyway, Seth linked to this amusing note by Preston McAfee.

P.S. In a comment to my earlier entry, somebody linked to McAfee's free introductory economics textbook. I started reading it, and it seems great so far. Maybe if I'd read a book like this thirty years ago I would've become an economist. Or maybe not, I dunno. It's not like my statistics textbooks were so delightful; I just liked the subject. And I've never read a poli sci textbook in my life.

Just a few thoughts in response to all the comments:

1. Several people point out that it is the publisher, not the author, who decides the cost of a book. That's right. The author has some input, and for almost all of my books, I've talked with the publisher, before signing any contract, about keeping the price down. We insisted that Bayesian Data Analysis sell for $45, Teaching Statistics sold for $40, and ARM sold for $40 as well. I thought that A Quantitative Tour of the Social Sciences was going to sell for around $25 but now I see on Amazon that it's selling for $33; I don't really know what's up with that. And, as a trade book, Red State, Blue State was always going to be reasonably priced--people aren't generally prepared to pay $40 for a book that they don't feel they need for their work--and, as for the Applied Bayesian Modeling book co-edited with Meng, we never tried to keep the price down, and as a result the publisher charged $100 for it.

2. Ragna is irritated that my Teaching Statistics book is selling for $190 at the local bookstore. This is simply a mistake--they seem to have ordered the hardcover rather than the softcover. Teaching Statistics is a great book but I wouldn't pay $190 for it. Annoyingly enough, if you look up the book on Amazon it sends you to the hardcover. But if you look carefully you can find the softcover for $63 ("list price $70"). I don't know how this happened. It was $40 when it came out.

3. Bayesian Data Analysis now costs $60 on Amazon. But, to be fair, it has been well over a decade since the original $45 version came out. I'd like it to still be $45 but I don't have much influence over this. It's a matter of negotiation.

4. I understand that if the book sells for more, the author probably makes more money. Certainly for technical books. I'd guess that if all my books were doubled in price, they'd sell more than half as well as they sell now (and, conversely, if they were halved in price, I doubt they'd sell at anything like twice their current rate). But my books don't make a lot of money for me (and, as for my book with Deb, we donate all the royalties to charity). What the books do do is make money for the publishers. That's fine, but making money for publishers is not one of my major goals in life.

5. I'll have to look into this open source thing. I'm a traditionalist myself and like hard-copy books. I've seen how students work on the computer: they seem to have the ability to only look at one window at a time, and so I think they need the hard copy of the textbook.

6. Some people were surprised that I didn't already know that these books were expensive. Yes, I know that technical books are expensive (hence my struggle to keep my own books under $40), but . . . an intro stat book? These things don't have a lot of content. $150 seems like a lot. If you pay $70 for Jun Liu's book on statistical computing . . . well, you get Jun Liu's book--that's a pretty good deal! But paying twice as much for something generic--that just seems horrible.

7. In answer to the questions of what my book will be like: I'm not sure! Seeing the $150 books makes me want to quickly write a generic book for $10 or free or whatever, just to do my part to destroy the market for the $150 books, but, no, I'm gonna do something new. I'm still struggling to figure out how it should be structured.

8. In answer to Bob's question: It's my impression that the Ivy League colleges get zillions of applicants, so they have no motivation to break the coalition and charge less for tuition. But, for an intro textbook, it would only take one author to change things, right?

9. I believe that many intro stat books, including Dick DeVeaux's and many others, have strengths. I have my own ideas for how to teach intro statistics, but I'm certainly not trying to claim that the current books are pure crap. And if the choice were DeVeaux's book for $100 or a generic book for $40, I'd probably assign DeVeaux's for $100. I think it would be worth the students' $60 to learn from a better book. But what amazes me is that even completely generic books are selling for well over $100.

10. Yes, I agree with everyone on the basic economic argument that it's the profs who assign the texts but the students (or their parents) who pay. Nonetheless . . . how did they even get the chutzpah to charge $150 in the first place?

11. Sometimes I sort of wish that Jennifer and I had self-published ARM or gone with some zero-margin publisher such as Dover, who do publish new books, by the way, including some great kids' activity booklets for something like $1.50 each. Anyway, if the goal is a $40 book, I think I can go with a regular publisher; after all, Cambridge is selling ARM for $40 and will sell Regression and Other Stories for a bit less, I believe.

12. From some of the resources provided by the commenters, it seems as if free textbooks are out there, and so maybe the current $150 texts are just the last bit of profit-taking before the collapse. I'd love to see time series plots of intro textbook prices in various fields.

13. Regarding the issue of homework questions and test banks: This is a real concern, I agree, or at least it should be a concern. In the courses I've seen, instructors don't actually use these test banks, but maybe that just means we're not getting our money's work.

14. I noticed a remark on cost per page. As an author of a couple reasonebly-priced 600+ page books, I'm sympathetic to this argument . . . but, no, I don't think there's 600 pages worth of material in these intro stat books. My impression is that, at some point, a book being heavy makes it that much harder to use.

I received a free copy in the mail of an introductory statistics textbook; I guess the publisher wants me to adopt it for my courses. The book isn't bad, actually it's pretty good: it follows the "Moore and McCabe" format, starting with descriptive statistics (up to correlation and regression), then a bit on data collection, then probability, then statistical inference, and at the end chapters on various more advanced topics.

I showed the book to Yu-Sung and he said: Wow, it's pretty fancy. I bet it costs $150. I didn't believe him, but we checked on Amazon and lo! it really does retail for that much. What the . . . ? I asked around and, indeed, it's commonplace for students to pay well over $100 for introductory textbooks.

Well. I'm planning to write an introductory textbook of my own and I'd like to charge $10 for it. Maybe this isn't possible, but I think $40 should be doable. And why would anybody require their students to pay $150 for a statistics book when something better is available at less than 1/3 the price?

This won't be easy, because I'm planning to write an entirely new kind of intro book, starting from scratch. But why hasn't someone written a more conventional book at a cut-rate price? Or maybe they have, and I just haven't heard about it?

It just mystifies me that, in all these different fields, it's considered acceptable to charge $150 for a textbook. I'd think that all you need is one cartel-breaker in each field and all the prices would come tumbling down. But apparently not. I just don't understand.

P.S. More thoughts here.

Eric Loken writes:

Last week the New York Times published an article on a possible Obama effect on test scores of black test takers. . . . The authors claim that they gave a short academic aptitude type test to black and white test-takers. When they administered the test last summer, they noted a difference between average scores for blacks and whites. However, after (now) President Obama had received his party's nomination and given his acceptance speech, the difference in scores disappeared. The theory is that Obama's rise has had a positive motivating influence on test taking performance.

Eric then gives some background:

The guest teacher will be Prof. Shigeo Hirano of the Dept. of Political Science. If he has any extra readings for you, I'll let you know!

A McKinsey interview (sorry, it's behind a registration wall, but the registration is worth it if you're interested in business topics or "futurism") with Google's chief economist Hal Varian has an interesting quote:

I keep saying the sexy job in the next ten years will be statisticians. People think I'm joking, but who would've guessed that computer engineers would've been the sexy job of the 1990s? The ability to take data--to be able to understand it, to process it, to extract value from it, to visualize it, to communicate it--that's going to be a hugely important skill in the next decades, not only at the professional level but even at the educational level for elementary school kids, for high school kids, for college kids. Because now we really do have essentially free and ubiquitous data. So the complimentary scarce factor is the ability to understand that data and extract value from it.

I think statisticians are part of it, but it's just a part. You also want to be able to visualize the data, communicate the data, and utilize it effectively. But I do think those skills--of being able to access, understand, and communicate the insights you get from data analysis--are going to be extremely important. Managers need to be able to access and understand the data themselves.

I'm sure everyone reading this blog will feel warmer and fuzzier now. :) But the teaching of introductory statistics really has to convey how to:

• capture data relevant to the problems


• visualize, communicate and effectively utilize the data


• access, understand, and communicate the insights of data analysis

Update 5.28.09: Michael E. Driscoll has a better written description of the above points, along with the guidelines.

When we have a grad school applicant who's taken the GRE or TOEFL multiple times, we typically just look at the highest score. It's my impression that pretty much everybody does this, even though basic statistical principles would suggest taking the average. Eric Rasmusen reminded me of this point in the context of the SAT, which apparently has changed its policy to encourage multiple test-taking even more, by allowing students to report only their highest score. Throwing away information--that doesn't sound like a good idea! But, as Rasumusen points out, it might make more money for the organizations that administer the test.

According to the linked news article, students "will have the option of choosing which scores to send to colleges while hiding those they do not want admissions officials to see." My question is: will their score report state whether they've chosen this option? If so, it should be possible to at least try to correct for the bias.

In any case, all this discussion makes me think we should be more careful about just looking at the maximum when our applicants take the GRE multiple times. And then there's the possibility of cheating. . . . I guess the real lesson is that these admissions decisions aren't going to be perfect, and we should think more about how to incorporate this perspective into our admissions process.

I received the following email from a Ph.D. student who wishes to remain anonymous:

I came into operations research with a masters in mechanical engineering therefore my statistical analysis is very unbalanced. I tool stochastic processes I and II (both PhD level courses) but I have no applied background in statistics, now I am doing a lot of number crunching using R but I believe I still lack broad understanding of statistical tools and I don't know enough about analysis (although I believe I have a solid understanding of the basics like mean, divergence, confidence interval, etc)

Given all my embarrassing situation I would appreciate it if you would please, in a blog post, lay out a learning plan for people like me who like to dive deeper into statistical analysis and do it in daily basis but come from weird backgrounds like mechanical engineering.

My reply: Since you're not at Columbia, I can't simply recommend that you take my course. You could read my books, that might help. More seriously, you gotta think about all the great skills you have that many statistics students don't have: you can make yourself useful in a lot of different sorts of projects . . .

Aleks sends in this link to an interactive statistics web course and writes, "It requires registration, but I highly recommend you go through this. You will probably not like the statistics teaching style that goes back to Fisher's 1930s textbook, but it's hard not to appreciate the effort that went into interactive 3D demonstrations of various concepts."

There is something incredibly old-fashioned about what they're teaching, but I agree that the 3-D graphics are extremely impressive. Probably the wave of the future, once it can be combined with a more up-to-date (that is, ARM-style) set of statistical concepts and methods.

This semester I'm teaching my "how to teach" class: The Teaching of Statistics at the University Level. (Stat 6600, or those of you here at Columbia.) I'll post more on that in a bit. Here I want to talk about an idea I had as I was falling asleep last night, of a new course I'd like to teach sometime.

The new course will be called Statistical Communication and it will cover the following topics:

- Graphical presentation (not just of raw data, also visualization of inferences)

- Writing research reports

- Writing computer code that can be used by others

- Working with colleagues (including "consulting" but also research collaborations)

- Email, blogging, hallway conversations, and other informal interactions

I think there was some other aspect of statistical communication that I wanted to include that I can't remember right now. The big idea is that maybe something is to be learned by thinking about all these activities as modes of communicating statistical ideas.

Aleks links to "The Manga Guide to Statistics":

and commenter David Warde-Farley links to the similar-looking "Cartoon Guide to Statistics."

My thoughts

Based on the example shown above, the point of the comic-book format seems to be to allow a punchy, power-point sort of delivery. The picture conveys essentially no content, which would suggest that the entire contents of a 222-page comic book could be presented in a 10-page pamphlet of text. The remaining 212 pages are essentially a reader-friendly trick to get students to turn the pages. It's the printed analogy to a power-point presentation.

So . . . let's take the customers' word for it that these cartoon guides are good. If so, this suggests that the useful content of a typical introductory statistics book can be captured in 10 pages. And, if this is the case, it in turn suggests that textbook writers could do a better job with those other 212 pages. Maybe it would be better to have a 10-page textbook and 212 pages of examples? Presumably a good textbook author could do better than those silly cartoons.

It's a tricky issue. Thinking about my own 600-to-700-page textbooks, it's hard for me to see what I would cut to bring it down to 30 pages. At the same time, the actual material that students learn in the class can probably be written in 30 single-spaced pages, so maybe it would be a good idea to try to pull that out.

(My books aren't directly comparable to these comic books, as I'm covering higher-level material. But the issues of presentation can't be that different.)

Slashdot has a review of "The Manga Guide to Statistics". Here is a snippet:

The story is silly and sets up some humorous examples of how to use statistics. Ramen noodle prices get graphed, Rui looks at grading on a curve and explores why her and a class mate get different grades for identical scores. Cramer's coefficient is used to examine how boys and girls prefer to be asked out. I thought that this was helpful not only because it helps to keep the readers interest but because it also moves the problems from the abstract to more concrete applications.

I haven't seen the book, but I like the tagline: "Statistics with heart-pounding excitement!"

AT writes:

A Facebook friend pointed me to this Gladwell piece discussing how you can('t) predict whether a teacher will be successful, but more importantly, on the range of advancement of a class depending on a teacher's ability.

The claim that a good teacher can make as much as a full year's difference in their students' advancement isn't too surprising to me [AT]. What I want to know is whether there are good studies around that look at the interaction effects between teachers and their students' backgrounds. In particular, I'd say that there are a number of teachers I've had who are very polarizing -- some make their students advance far, some stall them in their paths and make them give up the discipline.

My quick reply: From the work of Jonah Rockoff and others, I am convinced that teacher effects are real and they are large. And school effects are mostly the composition of teacher effects. I'm not sure about how large the interactions are (i.e., if some teachers do better with good students and others do better with poor students). Jennifer and I have talked about estimating such interactions (with a big multilevel model, of course) but I don't know what's up with that.

And, of course, this has no implications for university teaching, where as we know the sole qualification is to publish technical articles in obscure journals.

It's an old, old story but always worth hearing again, this time from Kevin Carey:

Frank Morgan is a wonderful teacher. I took a course from him in college and was impressed by his ability to help students of varying ability levels. (This was MIT so I guess the abilities were all on the high end, but still I think this is a challenge in any group.)

A few years ago I invited Frank to come to give a seminar on teaching to the mathematics and statistics departments at Columbia. One message I got from his talk was that much of teaching success comes from hard work. For example, every semester Frank would put the names and photos of his students on flash cards and memorize who was who.

This sort of thing was impressive to me. Any expert can demonstrate how great he is, but it takes someone very special to convey that anyone could achieve that level of success by just working hard.

That said, hard work is not enough. For example, statistics T.A.'s often spend dozens of hours preparing elaborate handouts for their students; this is almost always (in my impression) a waste of time. Better to adapt to the textbook, I think.

Anyway, I noticed this note by Frank on how he helps students prepare for their senior presentations:

At Williams every senior math major chooses a faculty advisor and gives a 35/40-minute colloquium talk. Since we currently have over fifty senior majors, this keeps us pretty busy, but we think it well worth the effort.

Here is how I like my advisees to prepare, starting a month before the talk and consulting with me every day or two . . .

Every day or two . . . that's impressive!

Will Wilkinson interviewed me for Bloggingheads today, and it was a disaster. I was too relaxed and I treated it as a conversation rather than a formal presentation or interview. As a result, I did too much b.s.-ing and too much conversational yapping, and not enough presentation of our research findings. I also said a bunch of things that are interesting or funny in informal conversation but probably come off as obnoxious or off-the-cuff in an interview that can be viewed interactively.

It's too bad, because my Red State, Blue State presentation is fun and informative, and I think the radio interviews I've done (with lengths ranging from 5 minutes to an hour) have gone well also. The two things that threw me off:
1. I've met Will before and I felt comfortable with him, hence too relaxed. Will was an excellent interviewer and gave me many opportunities to explain things; it wasn't his fault that I spouted off too much.
2. I've already spoken with Will about the book and so it was hard for me to remember to start from scratch--the audience won't necessarily be familiar with it.
3. Seeing my image in front of me while I was talking made me extra-focused on not twitching--always a bad thing. In a face-to-face or telephone interview, I usually forget about the twitching after a minute or so. Trying to suppress it takes a lot of mental effort that would be better used to think about my responses.

It would've been better to have some written talking points in front of me to keep me focused. The funny thing is, I did that for my early radio interviews but as I got more used to the format, I started speaking more off the cuff and it was going fine. This was just an interview too far. I had fun while it was happening, but afterward I realized what had gone wrong.

Anyway, it felt good to get this off my chest.

I graded this week's homeworks (from chapter 12 of ARM). When I write homework problems, I think about what they will be like to do. I don't think about what they will be like to grade. I'll try to write better homework problems in future books.

Bill Harris writes:

When I taught a graduate course at UW last year, I followed this sequence:

- - Student reading assignment
- - Student homework on the reading
- - Lecture and peer instruction on the reading
- - Homework graded and returned

Many reported they'd much prefer something like

- - Student reading assignment
- - Lecture and peer instruction on the reading
- - Student homework on the reading
- - Homework graded and returned

Do you have any pointers to evidence as to which sequence works best? I had been concerned that the latter approach involved students in three sets of work each week:

- - Reading the new material to prepare for class
- - Reviewing the previous week's material to do the homework
- - Reviewing the material from two weeks ago to understand the feedback on the returned homework

but I guess there could be advantages in that. Thoughts?

I'm embarrassed to admit I don't have any thoughts on this at all, but, yes, there must be some research on the topic. Can anybody help here?

I've blogged about this before but it's worth mentioning again as a good teaching example. The site How Many of Me purports to estimate how many people in the U.S. have any particular name. But it can give wrong results; as "the other Craig Newmark" noted, it said there was only one of him, and there are actually at least two.

What the site actually does is to plug in esitmates of the frequency of the first name and the frequency of the last name and assume independence. The results can be wrong.

This could be a great example for teaching probability. Three questions: first, how can you check that the site really is assuming independence; second, how many people does the site assume are in the U.S.; third, how could you do better?

1. How can you check that the site really is assuming independence? We'll check four names and see how many it says there are of each:

Rebecca Schwartz: 171
Rebecca Smith: 6600
Mary Schwartz: 1047
Mary Smith: 40941

Calculate the ratios: 6600/171=39, 40941/147=39. Check.

Actually, to one more digit, the ratios are 38.6 and 39.1. Why the difference? Shouldn't they be exactly the same? Playing around with the last digits reveals that it can't be simple rounding error. Maybe some internal rounding error in the calculations? (Perhaps another good lesson for the class?) Hmm, let me go back and check. Number of Mary Schwartzes: 1047. Check. Number of Mary Smiths? 40491. Uh oh, I'd transposed the digits when copying the number. Now the ratios agree (to within rounding error)

The website is definitely assuming independence. I have no doubt that there are some Mary Schwartzes out there but no way that the frequencies of Marys among Smiths and Schwartzes is exactly identical.

2. How many people does the site assume are in the U.S.? The site says there are 4,024,977 people in the U.S. with the first name Mary, 3,069,846 people in the U.S. with the last name Smith, and 40,491 Mary Smiths. 4024977*3069846/40491 = 305 million. So that's what they're assuming.

3. How could you do better? Phone books are an obvious start. They don't have everybody and there are other sampling difficulties involved (for example, a telephone that's under the name of only one person in the family, leaving the others unlisted) but it would give you some clear information about how large are the discrepancies from indepdence.

And, a bonus:

4. A bad idea (which might be tried by a naive instructor who doesn't get the point): Using this to teach the chi-squared test for statistical independence. This is a bad idea for two reasons: first, the data in HowManyofMe.com are not a sample under statistical independence; they are exactly statistically independent (a/b=c/d) and so a chi-squared test is beside the point. Second, for real data the point is not whether they could be explained by statistical independence--they can't--but how large the discrepancy is. This can be expressed using probabilities or odds ratios or whatever but not by the magnitude or the p-value of a chi-squared test. (If you want to use this example to illustrate chi-squared, this is the point you'd have to make.)

P.S. I've never met the other Andrew Gelman, but I did once meet someone who lives down the street from him (in New Jersey).

Rachel writes that she gave our students (it's a grad class in applied statistics, based on the Gelman and Hill book) what she thought of as a "Taxonomy of Confusion"... types of things they might be confused about and what they should do before asking the T.A.:

1. statistics-related questions that are prerequisite to the course--get an Intro to Stats book, don't ask the T.A. unless you really must.

2. statistics-related questions that are part of the course--read the book, ask a friend, then ask the T.A.

3. you know what you want statistically but you don't know the name of the function--google "R standard deviation" or write the function yourself... if you can't find it, ask a friend then ask the T.A.

4. you know the function's name but you can't figure out how it works: type help(sd), then ask a friend then ask the T.A.

5. you wrote code but you get error messages: DEBUG using tips like, print things out, break into smaller steps (we should do more on this later).

6.you wrote code and it doesn't do what you think it should do but there are no errors: DEBUG (more on this later).

This just seemed hilarious to me. Maybe it was the deadpan tone.

I flew to Denver, saw some people, went to my session and gave my talk, and flew back. The talk was fun, and in preparing it I had some general thoughts on presentations:

- You don't have to try to impress the audience; just explain what you did. (Hal Stern gave me that advice 20 years ago, and it's still good.)

- When writing articles, I always tell people not to include anything that you don't want people to read. For example, don't display a table full of numbers if you're not expecting to convey some information with each number. Anyway, when preparing my talk, I realized that I hadn't been following my own advice! I went back and looked at each slide and removed lots of material that I couldn't really expect people to be looking at.

- You can't optimize to every audience. In my talk, I chose to make the big picture clear, but that meant less detail on our data and our models. Sometimes I've seen the advice to start broad and then "drill down" to some interesting detail, but in practice you still have to make some choices. It's ok to give a detailed, technical talk, but then you have to accept that people won't be getting the big picture. If it's going to be technical, get into it right away so you'll have time to explain things.

- Plan to end 5 minutes early. Put extra stuff you need at the end of the presentation (after the slide you'll end with), then you can use it to answer questions if need be.

After the talk, I rode to the airport in a cab with a statistician who said his dad is a political scientist. Who? Steven Rhoads. That's the guy who wrote "The Economist's View of the World"? Yeah. Wow--I love that book. And then on the flight back, the lady sitting next to me took a look at Red State, Blue State, and said she was going to buy a copy for her son, who's an economics student. That was really cool--she'll either buy the book, which is great, or she was just being polite, which isn't so bad either.

You need one of these before you can do this wonderful demonstration. What's amazing to me is that the entry has 34 comments. I mean, what's there to say about kitchen scales?

Drew Conway links to this site that simulates text for Garfield cartoons using a Markov model. I don't actually think these are funny, but it strikes me that they could be a good demo for a lesson on Markov chains.

Here are my thoughts, to appear in the American Statistician:

1. Introduction
2. Teaching Bayesian statistics to social scientists, including a discussion of what is Bayesian about making graphs to get a better understanding of the deterministic part of a model
3. Other thoughts on teaching statistics to non-statisticians
4. A case study: the sampling distribution of the sample mean
5. Starting an (implicity) Bayesian applied regression course: two weeks of classroom activities
6. How is there time, in a course with class participation, to cover all the material?

1. Introduction

I was trying to draw Bert and Ernie the other day, and it was really difficult. I had pictures of them right next to me, but my drawings were just incredibly crude, more "linguistic" than "visual" in the sense that I was portraying key aspect of Bert and Ernie but in pictures that didn't look anything like them. I knew that drawing was difficult--every once in awhile, I sit for an hour to draw a scene, and it's always a lot of work to get it to look anything like what I'm seeing--but I didn't realize it would be so hard to draw cartoon characters!

This got me to thinking about the students in my statistics classes. When I ask them to sketch a scatterplot of data, or to plot some function, they can never draw a realistic-looking picture. Their density functions don't go to zero in the tails, the scatter in their scatterplots does not match their standard deviations, E(y|x) does not equal their regression line, and so forth. For example, when asked to draw a potential scatterplot of earnings vs. height, they have difficulty with the x-axis (most people are between 60 and 75 inches in height) and having the data consistent with the regression line, while having all earnings be nonnegative. (Yes, it's better to model on the log scale or whatever, but that's not the point of this exercise.)

Anyway, the students just can't make these graphs look right, which has always frustrated me. But my Bert and Ernie experience suggests that I'm thinking of it the wrong way. Maybe they need lots and lots of practice before they can draw realistic functions and scatterplots. They'll certainly need lots of practice to learn Bayesian methods.

Gal Elidan writes:

I am starting as a faculty next year in the statistics department at the Hebrew University, Israel. As it may be interesting to both the computer science and statistical community, I plan to give a course a course next year on Bayesian data analysis. My (still in its early stages) plan is to give a course based on your book along with some relevant topics/applications that have seen light in the computer science community in recent years (e.g. the Chinese restaurant process). I would greatly appreciate it greatly if you could share with me any material that you have used in the past in teaching this course. Since I have little experience estimating work load, I could use help in knowing how many problems you assigned each time.

My reply:

I led a 4-hour workshop on teaching statistics at the Association for Psychological Science meeting yesterday. Here's the powerpoint--I didn't actually get through all of it, because we spent nearly half the time in group discussions.

Here's the book, and here are other thoughts on teaching from the blog.

My favorite statistics demonstration is the one with the bag of candies. I've elaborated upon it since including it in the Teaching Statistics book and I thought these tips might be useful to some of you.

Preparation

Buy 100 candies of different sizes and shapes and put them in a bag (the plastic bag from the store is fine). Get something like 20 large full-sized candy bars, 20 or 30 little things like mini Snickers bars and mini Peppermint Patties. And then 50 or 60 really little things like tiny Tootsie Rolls, lollipops, and individually-wrapped Life Savers. Count and make sure it's exactly 100.

You also need a digital kitchen scale that reads out in grams.

Also bring a sealed envelope inside of which is a note (details below). When you get into the room, unobtrusively put the note somewhere, for example between two books on a shelf or behind a window shade.

Setup

Hold up the back of candy and the scale and write the following on the board:

Each pair of students should:
1. Pull 5 candies out of the bag
2. Weigh the candies
3. Write down the weight
4. Put the candies back in the bag!!
5. Pass the scale and bag to your neighbors
6. Silently multiply the weight of the 5 candies by 20.

(And, as Frank Morgan told me once, remember to read aloud everything you write on the board. Don't write silently.)

The students should work in pairs. Explain that their goal is to estimate the total weight of all the candies in the bag. They can choose their 5 candies using any method--systematic sampling, random sampling, whatever. Whichever pair guesses closest to the true weight. they get the whole bag!

Demonstrate how to zero the scale, give the scale and the bag of candies to a pair of students in the front row, and let them go.

Action

The demo will proceed silently while the rest of the class proceeds. So do whatever you were going to do in class. Take a look to make sure the scale and bag are moving slowly through the room. After about 30 or 40 minutes, it will reach the back and the students will be done.

At this point, ask the pairs, one at a time, to call out their estimates. Write them on the board. They will be numbers like 3080, 2400, 4340, and so forth. Once all the numbers are written, make a crude histogram (for example, bins from 2000-3000 grams, 3000-4000, 4000-5000, etc.). This represents the sampling distribution of the estimates.

Now call up two students from the class (but not from the same pair) to look at all the estimates. Ask them what their best guess is, having seen this information. As the class if they agree with these two students. Now give the bag to the two students in the front of the room and have them weigh it.

Punch line

The weight of all 100 candies will be something like 1658. It's always, always, always lower than all of the individual guesses on the board. Write this true weight as a vertical bar on the histogram that you've drawn. This is a great way to illustrate the concepts of bias and standard error of an estimator.

Now call out to the students who are sitting near where you hid the envelope: "Um, uh, what's that over there . . . is it an envelope??? Really? What's inside? Could you open it up?" A student opens it and reads out what's written on the sheet inside: "Your guesses are all too high!"

Aftermath

Now's the time to talk about sampling. Large candies are easy to see and to grab, while small candies fall through the gaps between the large ones and end up at the bottom of the bag. You can draw analogies to doing a random sample by going to the mall or by sending out an email survey and seeing who responds. Ask, How could you do a random sample. It won't be obvious to the students that the way to do a random sample is to number each of the candies from 1 to 100 and pick numbers at random. Also, as noted above, this is an example you can use later in the semester to illustrate bias and standard error.

P.S. My feeling about describing these demos is the same as what Penn and Teller say about why they show audiences how they do their tricks: it's even cooler when you know how it works.

P.P.S. Remember--it's crucial that the candies in the bag be of varying sizes, with a few big ones and lots of little ones!

Joel Beal, also known in our research group as the New Kid, is the valedictorian. Joel did some excellent work on our red-blue project, although we were too disorganized to make full use of him. We'd give him a project on Monday, then on Tuesday he'd return with a bunch of graphs and ask us for more to do. I guess we could've called him the Original Mitch. Econ undergraduate R.A.'s rule.

Dan sends along this article which reports a study saying that math is more effectively taught using drills instead of story problems. Speaking as a teacher (and without actually reading the report of the study), I'd say this is plausible. After 20 years of teaching, I've come to the conclusion that teaching skills works better than teaching concepts (or, should I say, trying to teach concepts).

I usually don't like spam, but this message I got the other day from Ed Tranham was pretty good:

Here.

Tyler Cowen links to a calculation by Tom Elia that "of Sen. Obama's 711,000 popular-vote lead, 650,000 -- or more than 90% of the total margin -- comes from Sen. Obama's home state of Illinois, with 429,000 of that lead coming from his home base of Cook County." This is interesting, but it's more a comment on how close the (meaningless) total popular vote count is, than a reflection of something funny going on in Cook County.

Put it another way. Suppose Obama's total margin was only 111,000 votes instead of 711,000. Then his 650,000 vote margin in Illinois would represent a whoppin 580% of the total margin, and Cook County would represent 390% of the total margin! But wait, how can a part be 390% of the whole??

What I'm sayin is, the "90%" and "60%" figures are misleading because, when written as "a percent of the total margin," it's natural to quickly envision them as percentages that are bounded by 100%. There is a total margin of victory that the individual state margins sum to, but some margins are positive and some are negative. If the total happens to be near zero, then the individual pieces can appear to be large fractions of the total, even possibly over 100%.

I'm not saying that Tom Elia made any mistakes, just that, in general, ratios can be tricky when the denominator is the sum of positive and negative parts. In this particular case, the margins were large but not quite over 100%, which somehow gives the comparison more punch than it deserves, I think.

Michael Spagat has written this paper criticizing the study of Iraq mortality by Burnham, Lafta, Doocy, and Roberts:

I [Spagat] consider the second Lancet survey of mortality in Iraq published in 2006. I give evidence of ethical violations against the survey’s respondents including endangerment, privacy breaches and shortcomings in obtaining informed consent. Violations to minimal disclosure standards include non-disclosure of the survey’s questionnaire, data-entry form, data matching anonymized interviewer IDs with households and sample design. I present evidence suggesting data fabrication and falsification that falls into nine broad categories: 1) non-disclosure of key information; 2) implausible data on non-response rates and security-related failures to visit selected clusters; 3) evidence suggesting that the survey’s figure for violent deaths was extrapolated from two earlier surveys; 4) presence of a number of known risk factors for interviewer fabrication listed in a joint document of American Association for Public Opinion Research and the American Statistical Association; 5) a claimed field-work regime that seems impossible without field workers crossing ethical boundaries; 6) large discrepancies with other data sources on the total number of violent deaths and their distribution in time and space; 7) two particular clusters that appears to contain fabricated data; 8) irregular patterns suggestive of fabrication in claimed confirmations of violent deaths through death certificates and 9) persistent mishandling of other evidence on mortality in Iraq presented so as to suggest greater support for the survey’s findings from other evidence than is actually the case.

I haven't read Spagat's paper and so am offering no evaluation of my own (see here for some comments form a year or so ago), but the discussions of ethics and survey practice are fascinating. Social data always seem much cleaner when you don't think too hard about how they were collected! May I say it again: a great example for your classes...

P.S. As a minor point, I still am irritated at the habit of referring to a scientific publication by the name of the journal where it was published ("the Lancet study").

P.P.S. A reporter called me about this stuff a couple months ago, but I'm embarrassed to say that I offered nothing conclusive, beyond the statement that these studies are hard to do, and for some reason it's often hard to get information from survey organizations about what goes on within primary sampling units. (We had to work hard even to get this information from these simple telephone polls in the U.S.)

Eric Mazur is my hero (see also here). But I wonder about this abstract:

A growing number of physics teachers are currently turning to instructional technologies such as wireless handheld response systems—colloquially called clickers. Two possible rationales may explain the growing interest in these devices. The first is the presumption that clickers are more effective instructional instruments. The second rationale is somewhat reminiscent of Martin Davis’ declaration when purchasing the Oakland Athletics: “As men get older, the toys get more expensive.” Although personally motivated by both of these rationales, the effectiveness of clickers over inexpensive low-tech flashcards remains questionable. Thus, the first half of this paper presents findings of a classroom study comparing the differences in student learning between a Peer Instruction group using clickers and a Peer Instruction group using flashcards. Having assessed student learning differences, the second half of the paper describes differences in teaching effectiveness between clickers and flashcards.

Is it really the best choice to keep the answer hidden in this way? Suspense is great, but an abstract should tell you the answer, no?

P.S. I tried took a look at the paper but the link didn't work. If anybody finds out whether clickers worked better, please let me know...

I like John Tierney's New York Times column (for example, here), but sometimes he goes over the top in counterintuitiveness.

Here, for example, Tierney writes about someone who says, "in some circumstances it’s better to drive than to walk. . . . If you walk 1.5 miles, Mr. Goodall calculates, and replace those calories by drinking about a cup of milk, the greenhouse emissions connected with that milk (like methane from the dairy farm and carbon dioxide from the delivery truck) are just about equal to the emissions from a typical car making the same trip. . . . Michael Bluejay, who’s done some number-crunching at BicycleUniverse.info, says that walking is actually worse than driving if you replace the calories with food in the standard American diet and if the car gets more than 24 miles per gallon. . . ."

This is interseting to me because these guys are making a classic statistical error, I think, which is to assume that all else is held constant. This is the error that also leads people to misinterpret regression coefficients causally. (See chapters 9 and 10 of our book for discussion of this point.) In this case, the error is to assume that the walker and the driver will be making the same trip. In general, the driver will take longer trips--that's one of the reasons for having a car, that you can easily take longer trips. Anyway, my point is not to get into a long discussion of transportation pricing, just to point out that this seemingly natural calculation is inappropriate because of its mistaken assumption that you can realistically change one predictor, leaving all the others constant.

As we like to say, it's a great classroom example.

P.S. More here (also see discussion in the comments below).

Rachel and Tyler write:

The Columbia statistics graduate students are excited to announce a Symposium on Careers for PhD's in Statistics on April 4, 2008. In an effort to broaden our exposure to the various possibilities that our distinguished fields affords, we are inviting leaders from academia and industry for a frank discussion of the careers and lifestyles of statisticians.

Steven Levitt writes, "I wish that I was teaching intermediate microeconomics this term, because this would be a perfect exam question."

I have mixed feelings about cool exam questions. I used to put effort into making my exams really cool, but a few years ago I decided that it wasn't always clear to the students what they were expected to learn in my classes, so I switched to writing non-clever exams that more directly addressed key points in the course. I expect that the optimal exam depends on how the course is organized. (Also, of course, different exams are good for different students. Presumably clever exams are great for the top students.)

For intro statistics, I wish we used standardized tests so we could have a better sense of what (if anything) the kids are learning during the semester. Also, pre-tests at the beginning of the semester. The whole deal.

Alex F. commented here about problems with our Starbucks and Walmart data. Elizabeth Kaplan, who collected the data for me, replied:

Yeah Walmart was a bit of a pain to find the locations for as you can not search just by state on their website, like for Starbucks. In order to find the locations I relied on the yellow page results. Even though I looked through to eliminate double postings for walmarts with the same address, after I looked into it again tonight, it appears the yellow pages dramatically over represented the number of walmarts per state. I have attached the correct data. All of these numbers come from this website (http://www.walmartfacts.com/StateByState/) which I was unable to locate before.

As far as the data for starbucks that should be correct as I got it straight from their website. The one thing is that they don't list all affiliate stores (that is stores not own and operated by the company). There is no reliable source of data on affiliate stores by state, and obviously the yellow pages are not a good source. So the data I sent to you just includes Starbucks owned and operated stores.

Also for population I used the 2006 Census Bureau estimates.

This sort of thing happens all the time to me, so I certainly don't think Elizabeth should feel too bad about this. I'm just glad that Alex noticed and pointed out the problem. Anyway, here are the corrected maps:

and scatterplot:

And also, following Seth's suggestion, the scatterplot on the log scale:

And, following Kaiser's suggestion, a reparameterization showing people per store (rather than stores per million people):

Lee Arnold, a plumber from LA, has developed a wonderful graphical way of communicating the behavior of complex systems, inspired by Odum's work, which he calls Ecolanguage. Here is how he explains banking:

He also tackles some spicier political topics, such as Gore's Assault on Reason, or Social Security. Finally, he uses this graphic language to explain heavier philosophical issues such as semiotics in New Chart.

While the symbols have been used before, Lee's animations and narration make it a great way of communicating quantitative ideas. He should find a publisher and produce a DVD!

At the Harvard 50th anniversary celebration a few months ago, they showed a video of Fred Mosteller's TV lectures on statistics from 1960 or so. The thing that struck me was that Fred was speaking reallllly reallllly slowly. He was a slow talker in real life, but this was so slow that I'm pretty sure he was doing it on purpose, maybe following some specific advice to go slow. Fred was a great teacher (as I remember from being his T.A.), and it made me wonder if I should speak more slowly also. Probably the answer is yes.

Also, when you speak slowly, you have to think more carefully about what to keep in your lecture and what to leave out, which is probably a good idea too.

I just ran into this article by W. Michael Cox and Richard Alm on the comparison of incomes and spending of rich and poor:

The share of national income going to the richest 20 percent of households rose from 43.6 percent in 1975 to 49.6 percent in 2006 . . . families in the lowest fifth saw their piece of the pie fall from 4.3 percent to 3.3 percent.

Income statistics, however, don’t tell the whole story of Americans’ living standards. Looking at a far more direct measure of American families’ economic status — household consumption — indicates that the gap between rich and poor is far less than most assume.

The top fifth of American households earned an average of $149,963 a year in 2006. As shown in the first accompanying chart, they spent $69,863 on food, clothing, shelter, utilities, transportation, health care and other categories of consumption. The rest of their income went largely to taxes and savings.

The bottom fifth earned just $9,974, but spent nearly twice that — an average of $18,153 a year. How is that possible? . . . lower-income families have access to various sources of spending money that doesn’t fall under taxable income. These sources include portions of sales of property like homes and cars and securities that are not subject to capital gains taxes, insurance policies redeemed, or the drawing down of bank accounts. While some of these families are mired in poverty, many (the exact proportion is unclear) are headed by retirees and those temporarily between jobs, and thus their low income total doesn’t accurately reflect their long-term financial status.

So, bearing this in mind, if we compare the incomes of the top and bottom fifths, we see a ratio of 15 to 1. If we turn to consumption, the gap declines to around 4 to 1. . . .

Let’s take the adjustments one step further. Richer households are larger — an average of 3.1 people in the top fifth, compared with 2.5 people in the middle fifth and 1.7 in the bottom fifth. If we look at consumption per person, the difference between the richest and poorest households falls to just 2.1 to 1.

This would be a good example for an intro statistics class when the topic of measurement comes up. The challenge for a stat class is to focus on measurement issues--how to design a survey to estimate people's income, assets, and spending patterns, or how to design an experiment or observational study to estimate the effects of changes in income on spending.

From the economics perspective, the example confuses me--on one hand, it makes sense to use consumption, not income, as a measure of well-being. On the other hand, if I were given the choice between two options:

(a) Earning $100,000 next year and spending $50,000, or
(b) Earning $40,000 next year and spending $60,000,

I'd prefer option (a). So I don't really know how to think about this. This sort of thing always confuses me in discussions of the utility of money (which I teach in my decision analysis class): it's good to have more money, but, usually, it's not money that brings joy, it's the things that money buys that do it.

In the example above, it would certainly make sense to adjust income for taxes and transfer payments and probably for household size (even if not by simply dividing by the number of people). It's harder for me to think how whether to adjust for savings or for non-cash benefits such as health-insurance.

Gary sent along this news article from the Syracuse Post-Standard:

Dead heat: Obama and Clinton split the Syracuse vote 50-50

by Mike McAndrew

In the city of Syracuse, the strangest thing happened in Tuesday's Democratic presidential primary.

Sen. Hillary Clinton and Sen. Barack Obama received the exact same number of votes, according to unofficial Board of Election results.

Clinton: 6,001.

Obama: 6,001.

"Wow, that is odd," said Jay Biba, Clinton's Central New York campaign coordinator. "I never heard of that in my life."

The odds of Clinton and Obama tying were less than one in 1 million, said Syracuse University mathematics Professor Hyune-Ju Kim.

I recently taught a short course, and, at the end, the students in the class filled out evaluation forms where they filled in the little circles. I just received copies of the forms in the mail. Amazingly (to me), 4 of the 25 forms in the class were filled out in error, with people getting the direction of the questions reversed (filling in "strongly disagree" where they meant "strongly agree," etc.) Three of the four people caught themselves and scribbled over their mistakes (this wouldn't work for a machine reader, though), but one never seemed to notice at all!

Perhaps this will be a less important issue now that everything is moving online, but just a reminder that it's good to provide some confirmation of people's choices. Especially fin areas more important than teaching evaluations.

In response to my presentation linked to here, Ken writes, "Andrew is using the LaTeX and the beamer package to produce the presentations. Much better than Powerpoint, especially for equations and can directly include .eps from R. A good alternative to beamer is powerdot."

I agree that beamer is great--I've been using it ever since Jouni told me about it, a few years ago--but it's awkward for a presentation with lots of images, since first I have to convert each graph to .pdf, then I have to spend lots of time in trial-and-error moving and resizing the figures using the numbers inside the \includegraphics call to get the pictures in the right place. When I have a lot of pictures, I actually use Powerpoint! See here, for example. (This particular presentation actually looks cooler in "real life": when I converted to pdf for convenient downloading, the software added a white border--which I hate, but I can't get rid of--to each slide.)

Kenny passed on this link, which is related to a project that Jennifer and I are involved in, on comparing New York City public schools:

Thanks to heavy parent involvement and high test scores, Public School 321 in Park Slope, a yuppie neighborhood in Brooklyn, is considered a gem of New York City's public school system. In the eyes of New York's Department of Education, however, P.S. 321 deserved just a B in the city's first-ever school report cards, which are based largely on how students score on standardized tests. Such accountability efforts — widespread since the advent of the federal No Child Left Behind Act — have raised the hackles of parents and educators across the country. . . .

James Liebman, chief accountability officer for New York City schools, devised the grading system for the city's 1.1 million-pupil school system. Liebman said standardized tests are a good measure of whether students have learned what they should know. "If children can't read and they can't do math, then the educational system and their school have failed them," he said. . . . Liebman pointed to a Quinnipiac University poll in which voters said the grades were fair by a margin of 61 to 27 percent. "It's a system to provide information to parents to make their own judgments," he said.

I've talked with Jim about the school evaluations but I don't know exactly how they finally decided to do it. One of the challenges in doing this sort of rating is that the evidence seems to show that teachers, rather than schools, have the biggest effects on test scores. To first approximation, the effect of the school seems to be pretty much the average of its teacher effects.

Regarding criticisms of the evaluations: one way the evaluations themselves can be evaluated is to apply them retroactively and see how well they predict future performance, to estimate the answer to the following question: if you were to send your kid to a highly-graded or poorly-graded school, how much different would you expect his or her test scores to be in a year, or two years, or whatever.

Beyond this, I think one of the motivations for getting these evaluations out there is to put some pressure on the schools. I have to say that I think our own teaching at the university level would be improved if our students had to take standardized tests after each of our courses and we were confronted with evidence on how much (or how little) they learned.

I had various course titles floating around: my course at Columbia this spring is officially called Applied Statistics, and I had promised people that it would cover Bayesian statistics. At Harvard they asked me to teach Statistical Computing, but I wanted to focus on applied Bayesian methods. So I'm putting it all together in the title given above.

If you're interested in taking the class, let me know if you have any questions or just show up to the first few lectures; it's Wed Fri 9:00-10:30 at Columbia (if you're in New York), or Mon 11:30-2:30 (if you're in Boston).

Motivation:

Statistical computing is to statistics as statistics is to science: necessary but a distraction from the main event. I hate computing, yet I do it all the time. For those of us in this position, it makes sense to spend a bit more time thinking harder about how to compute efficiently. Learning statistical computation is an investment in becoming a more effective practitioner and researcher.

Overview:

We will cover topics in Bayesian computation, statistical graphics, and software validation, as well as special topics that interest the class.

There will be some homework (writing programs and making graphs in R) and a final project to be done in pairs.

The (tentative) syllabus is below.

This year it's about voting.

Cosma Shalizi (of the CMU statistics dept) and I had an exchange about the role of measure theory in the statistics Ph.D. program. I have to admit I'm not quite sure what "measure theory" is but I think it's some sort of theoretical version of calculus of real variables. I had commented that we're never sure what to do with our qualifying exam, and Cosma wrote,

I think we have a pretty good measure-theoretic probability course, and I wish more of our students went on to take the non-required sequel on stochastic processes (because that's the one I usually teach). I do think it's important for statisticians to understand that material, but I also think it's actually easier for us to teach someone how a martingale works than it is to teach them to be interested in scientific questions and to not get a freaked out, "but what do I calculate?" response when confronted with an open research problem. Here it's been suggested that we replace our qualifying exams with having the student prepare a written review of some reasonably-live topic from the literature and take an oral exam on it, which would be more work for us but come a lot closer to testing what the students actually need to know.

I replied,

I agree that it's hard to teach how to think like a scientist, or whatever. But I don't think of the alternatives as "measure theory vs. how-to-think-like-a-scientist" or even "measure theory vs. statistics". I think of it as "measure theory vs. economics" or "measure theory vs. CS" or "measure theory vs. poli sci" or whatever. That is, sure, all other things being equal, it's better to know measure theory (or so I assume, not ever having really learned it myself, which didn't stop me from proving 2 published theorems, one of which is actually true). But, all other things being equal, it's better to know economics (by this, I mean economics, not necessarily econometrics), and all other things being equal, it's better to know how to program. Etc. I don't see why measure theory gets to be the one non-statistical topic that gets privileged as being so requrired that you get kicked out of the program if you can't do it.

Cosma then shot back with:

I also don't think of the alternatives as "measure theory vs. how-to-think-like-a-scientist" or even "measure theory vs. statistics". My feeling --- I haven't, sadly, done a proper experiment! --- is that it's easier to, say, take someone whose math background is shaky and teach them how a generating-class argument works in probability than it is to take someone who is very good at doing math homework problems and teach them the skills and attitudes of independent research.

You say, "I think of it as "measure theory vs. economics" or "measure theory vs. CS" or "measure theory vs. poli sci" or whatever." I'm more ambitious; I want our students to learn measure-theoretic probability, and scientific programming, and whatever substantive field they need for doing their research, and, of course, statistical theory and methods and data analysis. Because I honestly think that if someone is going to engage in building stochastic models for parts of the world, they really ought to understand how probability _works_, and that is why measure theory is important, rather than for its own sake. (I admit to some background bias towards the probabilist's view of the world.) At the same time it seems to me a shame (to use no stronger word) if someone, in this day and age, gets a ph.d. in statistics and doesn't know how to program beyond patching together scripts in R.

P.S. I think measure theory should be part of the Ph.D. statistics curriculum but I don't think it should be a required part of the curriculum. Not unless other important topics such as experimental design, sample surveys, statistical computing and graphics, stochastic modeling, etc etc are required also. It's sad to think of someone getting a Ph.D. in statistics and not knowing how to work with mixed discrete/continuous variables (see Nicolas's comment below) but it seems equally sad to see Ph.D.'s who don't know what Anova is, who don't know the basic principles of experimental design (for example, that it's more effective to double the effect size than to double the sample size), who don't know how to analyze a cluster sample, and so forth.

Unfortunately, not all students can do everything, and any program only gets some finite number of applicants. If you restrict your pool to those who want to do (or can put up with) measure theory, you might very well lose some who could be excellent statistical researchers. It would be sort of like not admitting Shaq to your basketball program because he can't shoot free throws.

Someone writes,

I am currently looking at different grad school stats programs. I have a BA in Psychology (U. Southern California), but I am really interested in stats. I loved my stats classes in college but I was a bit of a naive wallflower back then and did not think to change course and pursue stats more, even though it was the favorite part of my degree. After I graduated, I worked as a research assistant where my PI quickly learned that I was happiest talking about and running the stats for her various projects. I worked with her for close to two years, then moved and now I'm a public school science teacher.

Michael Foster writes,

I was thinking of writing a book on Junk Science--"How Bad Research Scares the Crap out of Parents and Leads to Bad Public Policy"--there's a bunch of the stuff out there suggesting that day care makes kids more aggressive, tv causes attention problems and so on.

Do you know of a book that focuses on kids that already addresses these kinds of issues?

Sounds like a good book idea to me. Does anyone know what's out there?

Chris Paulse writes, "By accident I [Chris] discovered a book that has part of its focus on educational psychology. It's called Handbook of Competence and Motivation, Elliott and Dweck eds. A few recent articles have appeared in the NYT that seem to be sourced from material like this (one on self-regulation profiling the work of Roy Baumesiter, and another on learning from mistakes that quotes Carol Dweck). The Dweck chapter on self-concept is a fun read. I'd love to see a mixture model developed from survey data for the evaluation anxiety idea. Great for teachers."

Aleks sent along this link. I just don't have the patience to watch these sorts of videos but maybe someone out there will like it. . . .

Daniel Lippman sent me this news article which would be an excellent thing to give your statistics students to read if you're covering confidence intervals and sampling.

Frank Di Traglia writes,

I'm going to be teaching a three-week, introductory statistics course for local high school students next summer, and wanted to ask for your advice. I have two questions in particular.

First, I doubt that three weeks will be enough time to teach the usual Statistics 101 course. If you had only three weeks, what would you skip and what would you emphasize?

Second, since next year is an election year, I thought it might be fun to build the course around substantive examples from political science. Although I've enjoyed many of your poly-sci papers, my own background is not in this area (I did my masters in Statistics, and am currently pursuing a PhD in Economics). What would you consider to be the ten most interesting and accessible quantitative papers in this field?

My reply:

Seth posts this account by a college student who went back to her high school to give a guest lecture on depression to "Mr. Tinloy’s 3rd period psychology class." Her feelings in preparing and delivering her lecture were pretty similar to my own feelings before doing this sort of thing, and I've been doing it for over 20 years!

The college student's presentation seemed to go well--the students were polite, got involved in discussion a bit, and clapped at the end, and the teacher was helpful in keeping things focused--but when she talked with some friends afterward, one said "she was fighting to stay awake, because the topic did not interest her one bit," and another said that "it was boring because she wasn’t all that interested in what I was talking about, but it got more interesting toward the end when other students started to talk. `Nobody likes guest speakers, so it’s okay.'"

I have a few thoughts:

1. I suspect the student's presentation to the high school kids would've gone even better if she'd had them working in pairs to discuss the material. When students are working in pairs, they seem less likely to drift off, also with two students there is more of a chance that one of them is interested in the topic.

2. It's interesting but perhaps not so surprising that depression is not an interesting topic to the high school students. Maybe they'd be more interested if it were framed in terms of being happy or sad, or good moods and bad moods? Even those of us who feel far from "depression" get sad or demoralized on occasion.

3. My own lectures to outside audiences) seem to go well (in that people say nice things to me afterwards about the presentations) but I usually have difficulty getting people actively involved. It often seems that my talks don't have "hooks" to grab the audience and motivate them to ask questions and think hard. They more often sit there passively, enjoying it (I hope) but not actively engaged. Maybe I should have them work in pairs. I do this for college students and even grad students--it always surprises them, but they like it--but I've rarely had the nerve to try it with nonstudents.

4. In the continuing theme of not practicing what we preach, I should point out that my comments above (including the title of this blog entry) are not based on any systematic research, just on my informal observations of what seems to have worked and not worked for me in the past. (Although it does seem consistent with the literature on active learning, as I've abosrbed it by reading a few books on the topic.) What I'm missing is (a) careful experimentation (assigning treatments--different teaching methods--unconfounded with important variables such as characteristics of the class, and (b) outcome measures such as surveys of student satisfaction and performance on standardized tests.

David Kim writes,

Alex Tabarrok writes,

Ayres argues that large experimental studies have shown that the teaching method which works best is Direct Instruction (here and here are two non-academic discussions which summarizes much of the same academic evidence discussed in Ayres). In Direct Instruction the teacher follows a script, a carefully designed and evaluated script. As Ayres notes this is key:

DI is scalable. Its success isn't contingent on the personality of some uber-teacher....You don't need to be a genius to be an effective DI teacher. DI can be implemented in dozens upon dozens of classrooms with just ordinary teachers. You just need to be able to follow the script.

I'll buy this--it fits with my own experience (yes, the usual n=1 reasoning that we follow in our lives). My teaching has improved over the years as I've tried more and more to follow the script (a script that I write, but still...). I've also tried to encourage new teachers to follow the textbook more--even if it's a crappy textbook, it seems to work better to follow it rather than jumping around or going with improvised lecture notes.

Regarding Ayres's advice, I would add only that it helps to have student involvement. Thus, the "script" is not a pure lecture, it includes class-participation activities, students working in pairs, etc.

Finally, one of Alex's commenters writes, "DI might be effective, but it sounds inhumanely boring." No! Not at all. A good script allows for creativity. In fact, the creativity can be well spent in getting the students involved, rather than in the preparation of lecture materials etc.

Jeff Lax sends along this article:

Are the polls obscuring the reality that Barack Obama is beating Hillary Clinton in the race for the Democratic nomination for president? Drew Cline, the editorial page editor of New Hampshire's Union Leader thinks so.

Based on money-raising and visible support on the streets of New Hampshire, "the evidence shows that Obama has broader support than is being picked up by the polls," Cline writes at his Union Leader blog. "So the polls must be wrong."
. . .
"Think of it like a House, M.D. episode. When you have a test result you know is accurate (in this case, the fund-raising numbers) that contrasts with a symptom or test result you can't explain (the poll numbers), you go with what you know is right and keep testing the other one until they match." . . .

Much as I hate to contradict anyone named "Drew," I have to admit that a natural explanation for the discrepancy is that the visible support he's seen on the "streets of New Hampshire" does not represent a random sample of primary voters. Of course, as I never tire of saying, a poll is a snapshot, not a forecast, and things can definitely change.

Check out this (from Dan Goldstein):