Recently in Teaching Category

I was pleasantly surprised to have my recreational reading about baseball in the New Yorker interrupted by a digression on statistics. Sam Fuld of the Tampa Bay Rays, was the subjet of a Ben McGrath profile in the 4 July 2011 issue of the New Yorker, in an article titled Super Sam. After quoting a minor-league trainer who described Fuld as "a bit of a geek" (who isn't these days?), McGrath gets into that lovely New Yorker detail:

One could have pointed out the more persuasive and telling examples, such as the fact that in 2005, after his first pro season, with the Class-A Peoria Chiefs, Fuld applied for a fall internship with Stats, Inc., the research firm that supplies broadcasters with much of the data anad analysis that you hear in sports telecasts.

After a description of what they had him doing, reviewing footage of games and cataloguing, he said

"I thought, They have a stat for everything, but they don't have any stats regarding foul balls."

My (coauthored) books on Bayesian data analysis and applied regression are like almost all the other statistics textbooks out there, in that we spend most of our time on the basic distributions such as normal and logistic and then, only as an aside, discuss robust models such as t and robit.

Why aren't the t and robit front and center? Sure, I can see starting with the normal (at least in the Bayesian book, where we actually work out all the algebra), but then why don't we move on immediately to the real stuff?

This isn't just (or mainly) a question of textbooks or teaching; I'm really thinking here about statistical practice. My statistical practice. Should t and robit be the default? If not, why not?

Some possible answers:

10. Estimating the degrees of freedom in the error distribution isn't so easy, and throwing this extra parameter into the model could make inference unstable.

9. Real data usually don't have outliers. In practice, fitting a robust model costs you more in efficiency than you gain in robustness. It might be useful to fit a contamination model as part of your data cleaning process but it's not necessary once you get serious.

8. If you do have contamination, better to model it directly rather than sweeping it under the rug of a wide-tailed error distribution.

7. Inferential instability: t distributions can yield multimodal likelihoods, which are a computational pain in their own right and also, via the folk theorem, suggest a problem with the model.

6. To make that last argument in reverse: the normal and logistic distributions have various excellent properties which make them work well even if they are not perfect fits to the data.

5. As Jennifer and I discuss in chapters 3 and 4 of our book, the error distribution is not the most important part of a regression model anyway. To the extent there is long-tailed variation, we'd like to explain this through long-tailed predictors or even a latent-variable model if necessary.

4. A related idea is that robust models are not generally worth the effort--it would be better to place our modeling efforts elsewhere.

3. Robust models are, fundamentally, mixture models, and fitting such a model in a serious way requires a level of thought about the error process that is not necessarily worth it. Normal and logistic models have their problems but they have the advantage of being more directly interpretable.

2. The problem is 100% computational. Once stan is up and running, you'll never see me fit a normal model again.

1. Clippy!

I don't know what to think. RIght now I'm leaning toward answer #2 above, but at the same time it's hard for me to imagine such a massive change in statistical practice. It might well be that in most cases the robust model won't make much of a difference, but I'm still bothered that the normal is my default choice. If computation wasn't a constraint, I think I'd want to estimate the t (with some innocuous prior distribution to average over the degrees of freedom and get a reasonable answer in those small-sample problems where the df would not be easy to estimate), or if I had to pick, maybe I'd go with a t with 7 degrees of freedom. Infinity degrees of freedom doesn't seem like a good default choice to me.

After posting on David Rubinstein's remarks on his "cushy life" as a sociology professor at a public university, I read these remarks by some of Rubinstein's colleagues at the University of Illinois, along with a response from Rubinstein.

Before getting to the policy issues, let me first say that I think it must have been so satisfying, first for Rubinstein and then for his colleagues (Barbara Risman, William Bridges, and Anthony Orum) to publish these notes. We all have people we know and hate, but we rarely have a good excuse for blaring our feelings in public. (I remember when I was up for tenure, I was able to read the outside letters on my case (it's a public university and they have rules), and one of the letter writers really hated my guts. I was surprised--I didn't know the guy well (the letters were anonymized but it was clear from context who the letter writer was) but the few times we'd met, he'd been cordial enough--but there you have it. He must have been thrilled to have the opportunity to write, for an audience, what he really thought about me.)

Anyway, reading Rubinstein's original article, it's clear that his feelings of alienation had been building up inside of him for oh I don't know how long, and it must have felt really great to tell the world how fake he really felt in his job. And his colleagues seem to have detested him for decades but only now have the chance to splash this all out in public. Usually you just don't have a chance.

Looking for a purpose in life

To me, the underlying issue in Rubinstein's article was his failure to find a purpose to his life at work. To go into the office, year after year, doing the very minimum to stay afloat in your classes, to be teaching Wittgenstein to a bunch of 18-year-olds who just don't care, to write that "my main task as a university professor was self-cultivation"--that's got to feel pretty empty.

Early this afternoon I made the plan to teach a new course on sampling, maybe next spring, with the primary audience being political science Ph.D. students (although I hope to get students from statistics, sociology, and other departments). Columbia already has a sampling course in the statistics department (which I taught for several years); this new course will be centered around political science questions. Maybe the students can start by downloading data from the National Election Studies and General Social Survey and running some regressions, then we can back up and discuss what is needed to go further.

About an hour after discussing this new course with my colleagues, I (coincidentally) received the following email from Mike Alvarez:

If you were putting together a reading list on sampling for a grad course, what would you say are the essential readings? I thought I'd ask you because I suspect you might have taught something along these lines.

I pointed Mike here and here.

To which Mike replied:

I wasn't too far off your approach to teaching this. I agree with your blog posts that the Groves et al. book is the best basic text to use on survey methodology that is currently out there. On sampling I have in the past relied on some sort of nonlinear combination of Kish and a Wiley text by Levy and Lemeshow, though that was unwieldy for students. I'll have to look more closely at Lohr, my impression of it when I glanced at it was like yours, that it sort of underrepresented some of the newer topics.

I think Lohr's book is great, but it might not be at quite the right level for political science students. I want something that is (a) more practical and (b) more focused on regression modeling rather than following the traditional survey sampling textbook approach of just going after the population mean. I like the Groves et al. book but it's more of a handbook than a textbook. Maybe I'll have to put together a set of articles. Also, I'm planning to do it all in R. Stata might make more sense but I don't know Stata.

Any other thoughts and recommendations would be appreciated.

Xian points me to an article by retired college professor David Rubinstein who argues that college professors are underworked and overpaid:

After 34 years of teaching sociology at the University of Illinois at Chicago, I [Rubinstein] recently retired at age 64 at 80 percent of my pay for life. . . . But that's not all: There's a generous health insurance plan, a guaranteed 3 percent annual cost of living increase, and a few other perquisites. . . . I was also offered the opportunity to teach as an emeritus for three years, receiving $8,000 per course . . . which works out to over $200 an hour. . . .

You will perhaps not be surprised to hear that I had two immediate and opposite reactions to this:

1. Hey--somebody wants to cut professors' salaries. Stop him!

2. Hey--this guy's making big bucks and doesn't do any work--that's not fair! (I went online to find David Rubinstein's salary but it didn't appear in the database. So I did the next best thing and looked up the salaries of full professors in the UIC sociology department. The salaries ranged from 90K to 135K. That really is higher than I expected, given that (a) sociology does not have a reputation as being a high-paying field, and (b) UIC is OK but it's not generally considered a top university.

Having these two conflicting reactions made me want to think about this further.

Carol Cronin writes:

The new Wolfram Statistics Course Assistant App, which was released today for the iPhone, iPod touch, and iPad. Optimized for mobile devices, the Wolfram Statistics Course Assistant App helps students understand concepts such as mean, median, mode, standard deviation, probabilities, data points, random integers, random real numbers, and more.

To see some examples of how you and your readers can use the app, I'd like to encourage you to check out this post on the Wolfram|Alpha Blog.

If anybody out there with an i-phone etc. wants to try this out, please let me know how it works. I'm always looking for statistics-learning tools for students. I'm not really happy with the whole "mean, median, mode" thing (see above), but if the app has good things, then an instructor could pick and choose what to recommend, I assume.

P.S. This looks better than the last Wolfram initiative we encountered.

At the Statistics Forum, we highlight a debate about how statistics should be taught in high schools. Check it out and then please leave your comments there.

This recent story of a wacky psychology professor reminds me of this old story of a wacky psychology professor.

This story of a wacky philosophy professor reminds me of a course I almost took at MIT. I was looking through the course catalog one day and saw that Thomas Kuhn was teaching a class in the philosophy of science. Thomas Kuhn--wow! So I enrolled in the class. I only sat through one session before dropping it, though. Kuhn just stood up there and mumbled.

At the time, this annoyed me a little. In retrospect, though, it made more sense. I'm sure he felt he had better things to do with his life than teach classes. And MIT was paying him whether or not he did a good job teaching, so it's not like he was breaking his contract or anything. (Given the range of instructors we had at MIT, it was always a good idea to make use of the shopping period at the beginning of the semester. I had some amazing classes but only one or two really bad ones. Mostly I dropped the bad ones after a week or two.)

Thinking about the philosophies of Kuhn, Lakatos, Popper, etc., one thing that strikes me is how much easier it is to use their ideas now that they're long gone. Instead of having to wrestle with every silly think that Kuhn or Popper said, we can just pick out the ideas we find useful. For example, my colleagues and I can use the ideas of paradigms and of the fractal nature of scientific revolutions without needing to get annoyed at Kuhn's gestures in the direction of denying scientific reality.

P.S. Morris also mentioned that Kuhn told him, "Under no circumstances are you to go to those lectures" by a rival philosopher. Which reminds me of when I asked one of my Ph.D. students at Berkeley why he chose to work with me. He told me that Prof. X had told him not to take my course and Prof. Y had made fun of Bayesian statistics in his class. At this point the student got curious. . . . and the rest is history (or, at least, Mister P).

Zoe Corbyn's article for The Guardian (UK), titled Wikipedia wants more contributions from academics, and the followup discussion on Slashdot got me thinking about my own Wikipedia edits.

The article quotes Dario Taraborelli, a research analyst for the Wikimedia Foundation, as saying "Academics are trapped in this paradox of using Wikipedia but not contributing," Huh? I'm really wondering what man-in-the-street wrote all the great stats stuff out there. And what's the paradox? I use lots of things without contributing to them.

Taraborelli is further quoted as saying "The Wikimedia Foundation is looking at how it might capture expert conversation about Wikipedia content happening on other websites and feed it back to the community as a way of providing pointers for improvement."

This struck home. I recently went through the entry for latent Dirichlet allocation and found a bug in their derivation. I wrote up a revised derivation and posted it on my own blog.

But why didn't I go back and fix the Wikipedia? One, editing in their format is a pain. Second, as Corbyn's article points out, I was afraid I'd put in lots of work and my changes would be backed out. I wasn't worried that Wikipedia would erase whole pages, but apparently it's an issue for some these days. A real issue is that most of the articles are pretty good, and while they're not necessarily written the way I'd write them, they're good enough that I don't think it's worth rewriting the whole thing (also, see point 2).

If you're status conscious in a traditional way, you don't blog either. It's not what "counts" when it comes time for tenure and promotion. And if you think blogs don't count, which are at least attributed, what about Wikipedia? Well, encyclopedia articles and such never counted for much on your CV. I did a few handbook type things and then started turning them down, mainly because I'm not a big fan of the handbook format.

In that sense, it's just like teaching. I was told many times on tenure track that I shouldn't be "wasting" so much time teaching. I was even told by a dean at a major midwestern university that they barely even counted teaching. So is it any surprise we don't want to focus on teaching or writing encyclopedia articles?

Under the heading, "Why Preschool Shouldn't Be Like School," cognitive psychologist Alison Gopnik describes research showing that four-year-olds learn better if they're encouraged to discover and show to others, rather than if they're taught what to do. This makes sense, but it's not clear to me why this wouldn't apply to older kids and adults. It's a commonplace in teaching at all levels that students learn by doing and by demonstrating what they can do. Even when a student is doing nothing but improvising from a template, we generally believe the student will learn better by explaining what's going on, by having a mental model of the process to go along with the proverbial 10,000 hours or practice. The challenge is in the implementation, how to get students interested, motivated, and focused enough to put the effort into learning.

So why the headline above? Why does Gopnik's research support the idea that preschool should be different from school? I'm not trying to disagree with Gopnik here. I just don't understand the reasoning.

P.S. One more thing, which certainly isn't Gopnik's fault but it's pretty funny/scary, given that it's the 21st century and all. Slate put this item in the category "Doublex: What women really think about news, politics, and culture." What? It wasn't good enough for "Science"? No, that space was taken by "The eco-guide to responsible drinking." But, sure, I guess it makes sense: kids in school . . . that sounds like it belongs on the women's page, along with Six recipes to get your kids to eat their vegetables, etc.

Mark Palko points to a news article by Michael Winerip on teacher assessment:

No one at the Lab Middle School for Collaborative Studies works harder than Stacey Isaacson, a seventh-grade English and social studies teacher. She is out the door of her Queens home by 6:15 a.m., takes the E train into Manhattan and is standing out front when the school doors are unlocked, at 7. Nights, she leaves her classroom at 5:30. . . .

Her principal, Megan Adams, has given her terrific reviews during the two and a half years Ms. Isaacson has been a teacher. . . . The Lab School has selective admissions, and Ms. Isaacson's students have excelled. Her first year teaching, 65 of 66 scored proficient on the state language arts test, meaning they got 3's or 4's; only one scored below grade level with a 2. More than two dozen students from her first two years teaching have gone on to . . . the city's most competitive high schools. . . .

You would think the Department of Education would want to replicate Ms. Isaacson . . . Instead, the department's accountability experts have developed a complex formula to calculate how much academic progress a teacher's students make in a year -- the teacher's value-added score -- and that formula indicates that Ms. Isaacson is one of the city's worst teachers.

According to the formula, Ms. Isaacson ranks in the 7th percentile among her teaching peers -- meaning 93 per cent are better. . . .

How could this happen to Ms. Isaacson? . . . Everyone who teaches math or English has received a teacher data report. On the surface the report seems straightforward. Ms. Isaacson's students had a prior proficiency score of 3.57. Her students were predicted to get a 3.69 -- based on the scores of comparable students around the city. Her students actually scored 3.63. So Ms. Isaacson's value added is 3.63-3.69.

Remember, the exam is on a 1-4 scale, and we were already told that 65 out of 66 students scored 3 or 4, so an average of 3.63 (or, for that matter, 3.69) is plausible. The 3.57 is "the average prior year proficiency rating of the students who contribute to a teacher's value added score." I assume that the "proficiency rating" is the same as the 1-4 test score but I can't be sure.

The predicted score is, according to Winerip, "based on 32 variables -- including whether a student was retained in grade before pretest year and whether a student is new to city in pretest or post-test year. . . . Ms. Isaacson's best guess about what the department is trying to tell her is: Even though 65 of her 66 students scored proficient on the state test, more of her 3s should have been 4s."

This makes sense to me. Winerip seems to presenting this is as some mysterious process but it seems pretty clear to me. A "3" is a passing grade, but if you're teaching in a school with "selective admissions" with the particular mix of kids that this teacher has, the expectation is that most of your students will get "4"s.

We can work through the math (at least approximately). We don't know this teacher's students did this year so I'll use the data given above, from her first year. Suppose that x students in the class got 4's, 65-x got 3's, and one student got a 2. To get an average of 3.63, you need 4x + 3(65-x) + 2 = 3.63*66. That is, x = 3.63*66 - 2 - 3*65 = 42.58. This looks like x=43. Let's try it out: (4*43 + 3*22 + 2)/66 = 3.63 (or, to three decimal places, 3.636). This is close enough for me. To get 3.69 (more precisely, 3.697), you'd need 47 4's, 18 3's, and a 2. So the gap would be covered by four students (in a class of 66) moving up from a 3 to a 4. This gives a sense of the difference between a teacher in the 7th percentile and a teacher in the 50th.

I wonder what this teacher's value-added scores were for the previous two years.

John Sides points to this discussion (with over 200 comments!) by political scientist Charli Carpenter of her response to a student from another university who emailed with questions that look like they come from a homework assignment. Here's the student's original email:

Hi Mr. Carpenter,

I am a fourth year college student and I have the honor of reading one of your books and I just had a few questions... I am very fascinated by your work and I am just trying to understand everything. Can you please address some of my questions? I would greatly appreciate it. It certainly help me understand your wonderful article better. Thank you very much! :)

1. What is the fundamental purpose of your article?

2. What is your fundamental thesis?

3. What evidence do you use to support your thesis?

4. What is the overall conclusion?

5. Do you feel that you have a fair balance of opposing viewpoints?

Sincerely,

After a series of emails in which Carpenter explained why she thought these questions were a form of cheating on a homework assignment and the student kept dodging the issues, Carpenter used the email address to track down the student's name and then contacted the student's university.

I have a few thoughts on this.

- Carpenter and her commenters present this bit of attempted cheating as a serious violation on the student's part. I see where she's coming from--after all, asking someone else to do your homework for you really is against the rules--but, from the student's perspective, sending an email to an article's author is just a slightly enterprising step beyond scouring the web for something written on the article. And you can't stop students from searching the web. All you can hope for is that students digest any summaries they read and ultimately spit out some conclusions in their own words.

- To me, what would be most annoying about receiving the email above is how insulting it is:

The new R environment RStudio looks really great, especially for users new to R. In teaching, these are often people new to programming anything, much less statistical models. The R GUIs were different on each platform, with (sometimes modal) windows appearing and disappearing and no unified design. RStudio fixes that and has already found a happy home on my desktop.

Initial impressions

I've been using it for the past couple of days. For me, it replaces the niche that R.app held: looking at help, quickly doing something I don't want to pollute a project workspace with; sometimes data munging, merging, and transforming; and prototyping plots. RStudio is better than R.app at all of these things. For actual development and papers, though, I remain wedded to emacs+ess (good old C-x M-c M-Butterfly).

Favorite features in no particular order

• plots seamlessly made in new graphics devices. This is huge— instead of one active plot window named something like quartz(1) the RStudio plot window holds a whole stack of them, and you can click through to previous ones that would be overwritten and ‘lost’ in R.app.
• help viewer. Honestly I use this more than anything else in R.app and the RStudio one is prettier (mostly by being not set in Times), and you can easily get contextual help from the source doc or console pane (hit tab for completions, then F1 on what you want).
• workspace viewer with types and dimensions of objects. Another reason I sometimes used R.app instead of emacs. This one doesn’t seem much different from the R.app one, but its integration into the environment is better than floaty thing that R.app does.
• ‘Import Dataset’ menu item and button in the workspace pane. For new R users, the answer to “How do I get data into this thing?” has always been “Use one of the textbook package’s included datasets until you learn to read.csv()”. This is a much better answer.
• obviously, the cross-platform nature of RStudio took the greatest engineering effort. The coolest platform is actually that it will run on a server and you access it using a modern browser (i.e., no IE). (“While RStudio is compatible with Internet Explorer, other browsers provide an improved user experience through faster JavaScript performance and more consistent handling of browser events.” more).

It would be nice if…

• indents worked like emacs. I think my code looks nice largely because of emacs+ess. The default indent of two spaces is nice (see the Google style guide) but where newlines line up by default is pretty helpful in avoiding silly typing errors (omitted commas, unclosed parentheses
• you could edit data.frames, which I’ll guess they are working on. It must be hard, since the R.app one and the X one that comes up in emacs are so abysmal (the R.app one is the least bad). RStudio currently says “ Editing of matrix and data.frame objects is not currently supported in RStudio.” :-(

Overall, really great stuff!

Dikran Karagueuzian writes:

About 12 years ago Greg Wawro, Sy Spilerman, and I started a M.A. program here in Quantitative Methods in Social Sciences, jointly between the departments of history, economics, political science, sociology, psychology, and statistics. We created a bunch of new features for the program, including an interdisciplinary course based on this book.

Geen Tomko asks:

Can you recommend a good introductory book for statistical computation? Mostly, something that would help make it easier in collecting and analyzing data from student test scores.

I don't know. Usually, when people ask for a starter statistics book, my recommendation (beyond my own books) is The Statistical Sleuth. But that's not really a computation book. ARM isn't really a statistical computation book either. But the statistical computation books that I've seen don't seems so relevant for the analyses that Tomko is looking for. For example, the R book of Venables and Ripley focuses on nonparametric statistics, which is fine but seems a bit esoteric for these purposes.

Does anyone have any suggestions?

Antony Unwin writes:

I [Unwin] find it an interesting exercise for students to ask them to write headlines (and subheadlines) for graphics, both for ones they have drawn themselves and for published ones. The results are sometimes depressing, often thought-provoking and occasionally highly entertaining.

This seems like a great idea, both for teaching students how to read a graph and also for teaching how to make a graph. I've long said that when making a graph (or, for that matter, a table), you want to think about what message the reader will get out of it. "Displaying a bunch of numbers" doesn't cut it.

Val has reported success with the following trick:

Get to the classroom a few minutes earlier and turn on soft music. Then set everything up and, the moment it's time for class to begin, put a clicker question on the screen and turn off the music. The students quiet down and get to work right away.

I've never liked the usual struggle with students to get them to settle down in class, as it seemed to set up a dynamic in which I was trying to get the students to focus and they were trying to goof off. Turning off the music seems like a great non-confrontational way to send the signal that class is starting.

Jonathan Livengood writes:

There has been some discussion about the recent PISA results (in which the U.S. comes out pretty badly), for example here and here. The claim being made is that the poor U.S. scores are due to rampant individual- or family-level poverty in the U.S. They claim that when one controls for poverty, the U.S. comes out on top in the PISA standings, and then they infer that poverty causes poor test scores. The further inference is then that the U.S. could improve education by the "simple" action of reducing poverty. Anyway, I was wondering what you thought about their analysis.

My reply: I agree this is interesting and I agree it's hard to know exactly what to say about these comparisons. When I'm stuck in this sort of question, I ask, WWJD? In this case, I think Jennifer would ask what are the potential interventions being considered. Various ideas for changing the school system would perhaps have different effects on different groups of students. I think that would a useful way to focus discussion, to consider the effects of possible reforms in the U.S. and elsewhere. See here and here, for example.

P.S. Livengood has some graphs and discussion here.

Joan Nix writes:

Your comments on this paper by Scott Carrell and James West would be most appreciated. I'm afraid the conclusions of this paper are too strong given the data set and other plausible explanations. But given where it is published, this paper is receiving and will continue to receive lots of attention. It will be used to draw deeper conclusions regarding effective teaching and experience.

Nix also links to this discussion by Jeff Ely.

I don't completely follow Ely's criticism, which seems to me to be too clever by half, but I agree with Nix that the findings in the research article don't seem to fit together very well. For example, Carrell and West estimate that the effects of instructors on performance in the follow-on class is as large as the effects on the class they're teaching. This seems hard to believe, and it seems central enough to their story that I don't know what to think about everything else in the paper.

My other thought about teaching evaluations is from my personal experience. When I feel I've taught well--that is, in semesters when it seems that students have really learned something--I tend to get good evaluations. When I don't think I've taught well, my evaluations aren't so good. And, even when I think my course has gone wonderfully, my evaluations are usually far from perfect. This has been helpful information for me.

That said, I'd prefer to have objective measures of my teaching effectiveness. Perhaps surprisingly, statisticians aren't so good about measurement and estimation when applied to their own teaching. (I think I've blogged on this on occasion.) The trouble is that measurement and evaluation take work! When we're giving advice to scientists, we're always yammering on about experimentation and measurement. But in our own professional lives, we pretty much throw all our statistical principles out the window.

P.S. What's this paper doing in the Journal of Political Economy? It has little or anything to do with politics or economics!

P.P.S. I continued to be stunned by the way in which tables of numbers are presented in social science research papers with no thought of communication with, for example, tables with interval estimate such as "(.0159, .0408)." (What were all those digits for? And what do these numbers have to do with anything at all?). If the words, sentences, and paragraphs of an article were put together in such a stylized, unthinking way, the article would be completely unreadable. Formal structures with almost no connection to communication or content . . . it would be like writing the entire research article in iambic pentameter with an a,b,c,b rhyme scheme, or somesuch. I'm not trying to pick on Carrell and West here--this sort of presentation is nearly universal in social science journals.

John Cook noticed something:

I [Cook] was looking at the preface of an old statistics book and read this:

The Bayesian techniques occur at the end of each chapter; therefore they can be omitted if time does not permit their inclusion.

This approach is typical. Many textbooks present frequentist statistics with a little Bayesian statistics at the end of each section or at the end of the book.

There are a couple ways to look at that. One is simply that Bayesian methods are optional. They must not be that important or they'd get more space. The author even recommends dropping them if pressed for time.

Another way to look at this is that Bayesian statistics must be simpler than frequentist statistics since the Bayesian approach to each task requires fewer pages.

My reaction:

Classical statistics is all about summarizing the data.

Bayesian statistics is data + prior information.

On those grounds alone, Bayes is more complicated, and it makes sense to do classical statistics first. Not necessarily p-values etc., but estimates, standard errors, and confidence intervals for sure.

Sharon Otterman reports:

When report card grades were released in the fall for the city's 455 high schools, the highest score went to a small school in a down-and-out section of the Bronx . . . A stunning 94 percent of its seniors graduated, more than 30 points above the citywide average. . . . "When I interviewed for the school," said Sam Buchbinder, a history teacher, "it was made very clear: this is a school that doesn't believe in anyone failing."

That statement was not just an exhortation to excellence. It was school policy.

By order of the principal, codified in the school's teacher handbook, all teachers should grade their classes in the same way: 30 percent of students should earn a grade in the A range, 40 percent B's, 25 percent C's, and no more than 5 percent D's. As long as they show up, they should not fail.

Hey, that sounds like Harvard and Columbia^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H various selective northeastern colleges I've known. Of course, we^H^H^H they give a lot more than 30% A's!

P.S. In all seriousness, it does appear from the report that the school has problems.

Columbia College has for many years had a Core Curriculum, in which students read classics such as Plato (in translation) etc. A few years ago they created a Science core course. There was always some confusion about this idea: On one hand, how much would college freshmen really learn about science by reading the classic writings of Galileo, Laplace, Darwin, Einstein, etc.? And they certainly wouldn't get much out by puzzling over the latest issues of Nature, Cell, and Physical Review Letters. On the other hand, what's the point of having them read Dawkins, Gould, or even Brian Greene? These sorts of popularizations give you a sense of modern science (even to the extent of conveying some of the debates in these fields), but reading them might not give the same intellectual engagement that you'd get from wrestling with the Bible or Shakespeare.

I have a different idea. What about structuring the entire course around computer programming and simulation? Start with a few weeks teaching the students some programming language that can do simulation and graphics. (R is a little clunky and Matlab is not open-source. Maybe Python?)

After the warm-up, students can program simulations each week:
- Physics: simulation of bouncing billiard balls, atomic decay, etc.
- Chemistry: simulation of chemical reactions, cool graphs of the concentrations of different chemicals over time as the reaction proceeds
- Biology: evolution and natural selection
And so forth.

There could be lecture material connecting these simulations with relevant scientific models. This could be great!

This post is by Phil Price.

A reporter once told me that the worst-kept secret of journalism is that every story has errors. And it's true that just about every time I know about something first-hand, the news stories about it have some mistakes. Reporters aren't subject-matter experts, they have limited time, and they generally can't keep revisiting the things they are saying and checking them for accuracy. Many of us have published papers with errors -- my most recent paper has an incorrect figure -- and that's after working on them carefully for weeks!

One way that reporters can try to get things right is by quoting experts. Even then, there are problems with taking quotes out of context, or with making poor choices about what material to include or exclude, or, of course, with making a poor selection of experts.

Yesterday, I was interviewed by an NPR reporter about the risks of breathing radon (a naturally occurring radioactive gas): who should test for it, how dangerous is it, etc. I'm a reasonable person to talk to about this, having done my post-doc and several subsequent years of research in this area, although that ended about ten years ago. Andrew and I, and other colleagues, published several papers, including a decision analysis paper that encompasses most of what I think I know about radon risk in the U.S. In this case, the reporter had a good understanding of the fact that the risk is very small at low concentrations; that the risk per unit exposure is thought to be much higher for smokers than for non-smokers; and that the published estimates of radon deaths are based on the unrealistic comparison to people being exposed to no radon at all. He had a much more sophisticated understanding than most reporters, and perhaps more than some radon researchers! So I hope the piece will come out OK. But I gave him a lot of "on the one hand..., on the other hand..." material, so if he quotes selectively he could make me look extreme in either direction. Not that I think he will, I think he'll do a good job.

The piece will be on NPR's Morning Edition tomorrow (Friday), and available on their archives afterwards.

A student in my intro class came by the other day with a lot of questions. It soon became clear that he was confused about a lot of things, going back several weeks in the course. What this means is that we did not do a good job of monitoring his performance earlier during the semester. But the question now is: what do do next? I'll sign the drop form any time during the semester, but he didn't want to drop the class (the usual scheduling issues). And he doesn't want to get a C or a D. He's in big trouble and at this point is basically rolling the dice that he'll do well enough on the final to eke out a B in the course. (Yes, he goes to section meetings and office hours, and he even tried hiring a tutor. But it's tough--if you've already been going to class and still don't know what's going on, it's not so easy to pull yourself out of the hole, even if you have a big pile of practice problems ahead of you.)

What we really need for this student, and others like him, is a road to a B: a plan by which a student can get by and attain partial mastery of the material. The way that this, and other courses, is set up, is that if you do everything right you get an A, and you get a B if you make some mistakes and miss some things along the way. That's ok, but if you're really stuck, you want some sort of plan that will take you forward. And the existing plan (try lots of practice problems) won't cut it. What you need is some sort of triage, so you can nail down topics one at a time and do what it takes to get that B. And that's not something we have now. I think it needs to be a formal part of the course, in some way.

Gayle Laackmann reports (link from Felix Salmon) that Microsoft, Google, etc. don't actually ask brain-teasers in their job interviews. The actually ask a lot of questions about programming. (I looked here and was relieved to see that the questions aren't very hard. I could probably get a job as an entry-level programmer if I needed to.)

Laackmann writes:

Let's look at the very widely circulated "15 Google Interview Questions that will make you feel stupid" list [here's the original list, I think, from Lewis Lin] . . . these questions are fake. Fake fake fake. How can you tell that they're fake? Because one of them is "Why are manhole covers round?" This is an infamous Microsoft interview question that has since been so very, very banned at both companies . I find it very hard to believe that a Google interviewer asked such a question.

We'll get back to the manhole question in a bit.

Lacakmann reports that she never saw any IQ tests in three years of interviewing at Google and that "brain teasers" are banned. But . . . if brain teasers are banned, somebody must be using them, right? Otherwise, why bother to ban them? For example, one of her commenters writes:

I [the commenter] have been phone screened by Google and so have several colleagues. I can say that the questions are different depending on who is asking them. I went in expecting a lot of technical questions, and instead they asked me one question:

"If I were to give you $1000 to count all the manholes in San Francisco, how would you do it?"

I don't think you can count on one type of phone screen or interview from Google. Each hiring team probably has their own style of screening.

And commenter Bjorn Borud writes:

Though your effort to demystify the interview process is laudable you should know better than to present assumptions as facts. At least a couple of the questions you listed as "fake" were used in interviews when I worked for google. No, I can't remember ever using any of them (not my style), but I interviewed several candidates who had other interviewers ask some of these. Specifically I know a lot of people were given the two eggs problem. Which is not an entirely unreasonable problem to observe problem solving skills.

And commenter Tim writes:

I was asked the manhole cover question verbatim during a Google interview for a Datacenter Ops position.

What we seem to have here is a debunking of a debunking of an expose.

Who do we believe?

You'll be unsurprised to hear that I think there's an interesting statistical question underlying all this mess. The question is: Who should we believe, and what evidence are we using or should be using to make this judgment?

What do we have so far?

- Felix Salmon implicitly endorses the analysis of Laakmann (who he labels as "Technology Woman"). I like Salmon; he seems reasonable and I'm inclined to trust him (even if I still don't know who this Nouriel Roubini person is who Salmon keeps mocking for buying a 5 million dollar house).

- Salmon associated the "fake" interview questions with "Business Insider," an unprofessional-looking website of the sort that clogs the web with recycled content and crappy ads.

- Laackman's website looks professional (unlike that of Business Insider) and reports her direct experiences at Google. After reading her story, I was convinced.

- There was one thing that bugged me about Laackmann's article, though. It was the very last sentence:

Want to see real Google interview questions, Microsoft interview questions, and more? Check CareerCup.

I followed the link, and CareerCup is Laackmann's commercial website. That's fine--we all have to earn a living. But what bothered me was that the sentence above contained three links (on "Google interview questions," "Microsoft interview questions," and "CareerCup")--and they all linked to exact same site. That's the kind of thing that spammers do.

Add +1 to the Laackmann's spam score.

- I didn't think much of this at first, but then there are the commenters, who report direct experiences of their own that contradict the blog's claims. And I couldn't see why someone would bother to write in with fake stories. It's not like they have something to sell.

- Laackmann has a persuasive writing style, but not in the mellow style of Salmon (or myself) but more in the in-your-face style Seth Godin, Clay Shirky, Philip Greenspun, Jeff Jarvis, and other internet business gurus. This ends up being neutral for me: the persuasiveness persuades me, then I resist the pushiness, and the net is to be neither more or less convincing than if the article were written in a flatter style.

What do I think? I'm guessing that Laackmann is sincere but is overconfident: she's taking the part of the world she knows and is generalizing with too much certainty. On the other hand, she may be capturing much of the truth: even if these wacky interview questions are used occasionally, maybe they're not asked most of the time.

My own story

As part of my application to MIT many years ago, I was interviewed by an alumnus in the area. We talked for awhile--I don't remember what about--and then he said he had to go off and do something in the other room, and while I was waiting I could play with these four colored cubes he had, that you were supposed to line up so that the colors on the outside lined up. It was a puzzle called Instant Insanity, I think. Anyway, he left the room to do whatever, and I started playing with the cubes. After a couple minutes I realized he'd given me an impossible problem: there was no possible way to line up the cubes to get the configuration he'd described. When he returned, I told him the puzzle was impossible, and he gave some sort of reply like, Yeah, I can't figure out what happened--maybe we had two sets and lost a couple of cubes? I still have no idea if he was giving this to me as some kind of test or whether he was just giving me something to amuse myself while he got some work done. He was an MIT grad, after all.

Since this blog in November, I've given my talk on infovis vs. statistical graphics about five times: once in person (at the visualization meetup in NYC, a blog away from Num Pang!) and the rest via telephone conferencing or skype. The live presentation was best, but the remote talks have been improving, and I'm looking forward to doing more of these in the future to save time and reduce pollution.

Here are the powerpoints of the talk.

Now that I've got it working well (mostly by cutting lots of words on the slides), my next step will be to improve the interactive experience. At the very least, I need to allocate time after the talk for discussion. People usually don't ask a lot of questions when I speak, so maybe the best strategy is to allow a half hour following the talk for people to speak with me individually. It could be set up so that I'm talking with one person but the others who are hanging out could hear the conversation too.

Anyway, one of the times I gave the talk, a new idea came out: One thing that people like about infovis is the puzzle-solving aspect. For example, when someone sees that horrible map with the plane crashes (see page 23 of the presentation), there is a mini-joy of discovery at noticing--Hey, that's Russia! Hey, that's India! Etc. From our perspective as statisticians, it's a cheap thrill: the reader is wasting brainpower to discover the obvious. But I think most people like it. In this way, an attractive data visualization is functioning like a Chris Rock routine, when he says something that we all know, but he says it in such a fresh new way that we find it appealing.

Conversely, in statistical graphics we use a boring display so that anything unexpected will stand out. It's a completely different perspective. I'm not saying that statisticians are better than infovis people, just that we strive for different effects.

Another example are those maps that distort the sizes of states or countries to be proportional to population. Everybody loves these "cartograms," but I hate 'em. Why? Because the #1 thing these maps convey is that some states on the east coast have high population density and that nobody lives in Wyoming. People loooove to see these wacky maps and discover these facts. It's like being on vacation in some far-off place and running into Aunt Louise at the grocery store. The shock of the familiar.

(I'm not opposed to all such maps. In particular, I like the New York Times maps that show congressional and electoral college results within stylized states that include congressional districts as little squares. These maps do the job by focusing attention on the results, not on the cool processes used to create the distortions.)

This link on education reform send me to this blog on foreign languages in Canadian public schools:

Mark Palko comments on the (presumably) well-intentioned but silly Jumpstart test of financial literacy, which was given to 7000 high school seniors Given that, as we heard a few years back, most high school seniors can't locate Miami on a map of the U.S., you won't be surprised to hear that they flubbed item after item on this quiz.

But, as Palko points out, the concept is better than the execution:

From the Gallup Poll:

Four in 10 Americans, slightly fewer today than in years past, believe God created humans in their present form about 10,000 years ago.

They've been asking the question since 1982 and it's been pretty steady at 45%, so in some sense this is good news! (I'm saying this under the completely unsupported belief that it's better for people to believe truths than falsehoods.)

A couple months ago, the students in our Teaching Statistics class practiced one-on-one tutoring. We paired up the students (most of them are second-year Ph.D. students in our statistics department), with student A playing the role of instructor and student B playing the role of a confused student who was coming in for office hours. Within each pair, A tried to teach B (using pen and paper or the blackboard) for five minutes. Then they both took notes on what worked and what didn't work, and then they switched roles, so that B got some practice teaching.

While this was all happening, Val and I walked around the room and watched what they did. And we took some notes, and wrote down some ideas: In no particular order:

Who's holding the pen? Mort of the pairs did their communication on paper, and in most of these cases, the person holding the pen (and with the paper closest to him/herself) was the teacher. That ain't right. Let the student hold the pen. The student's the one who's gonna have to learn how to do the skill in question.

The split screen. One of the instructors was using the board in a clean and organized way, and this got me thinking of a new idea (not really new, but new to me) of using the blackboard as a split screen. Divide the board in half with a vertical line. 2 sticks of chalk: the instructor works on the left side of the board, the student on the right. On the top of each half of the split screen is a problem to work out. The two problems are similar but not identical. The instructor works out the solution on the left side while the student uses this as a template to solve the problem on the right.

Seeing one's flaws in others. It can be difficult to observe our own behavior. But sometimes when observing others, we can realize that we are doing the same thing ourselves. Thus I can learn from the struggles of our Ph.D. students and get ideas of how to be a better teacher myself (and then share these ideas with them and you).

Go with your strengths. One of our students (playing the role of instructor in the activity) speaks English with a strong accent (but much better than my accent when speaking French or Spanish, I'm sure). If your spoken language is hard to understand, adapt by talking less and writing more. You'll have plenty of chances to practice your speaking skills--outside of class.

Setting expectations. When a student comes in for office hours, he or she might have one question or two, or five. And this student might want to get out of your office as quickly as possible, or he or she might welcome the opportunity for a longer lesson. How you should behave will depend a lot on what the student wants. So ask the student: What are your expectations for this session? This needn't limit your interaction--it's perfectly fine for someone to come in with one question and then get involved in a longer exploration--but the student's initial expectations are a good place to start.

Any other thoughts?

If you have other ideas, please post them here. I've never been good at one-on-one teaching in introductory courses--I've always felt pretty useless sitting next to a student trying to make some point clear--but maybe with these new techniques, things will go better.

Aleks points me to this research summary from Dan Goldstein. Good stuff. I've heard of a lot of this--I actually use some of it in my intro statistics course, when we show the students how they can express probability trees using frequencies--but it's good to see it all in one place.

Posted by Phil Price:

A blogger (can't find his name anywhere on his blog) points to an article in the medical literature in 1994 that is...well, it's shocking, is what it is. This is from the abstract:

In Tai's Model, the total area under a curve is computed by dividing the area under the curve between two designated values on the X-axis (abscissas) into small segments (rectangles and triangles) whose areas can be accurately calculated from their respective geometrical formulas. The total sum of these individual areas thus represents the total area under the curve. Validity of the model is established by comparing total areas obtained from this model to these same areas obtained from graphic method (less than +/- 0.4%). Other formulas widely applied by researchers under- or overestimated total area under a metabolic curve by a great margin

The blogger finds this amusing, but I find it mostly upsetting and sad. Which is sadder: (1) That this paper got past the referees, (2) that it has been cited dozens of times in the medical literature, including this year, (3) that, if the abstract is to be believed, many medical researchers DON'T use an accurate method to calculate the area under a curve.

Things gets reinvented all the time. I, too, have published results that I've later found were previously published by someone else. But I've never done it with something that is taught in high school calculus. And --- I'm practically spluttering with indignation --- if I wanted to calculate something like the area under a curve, I would at least first see if there is already a known way to do it! I wouldn't invent an obvious method, name it after myself, and send it to a journal, without it ever occurring to me that, gee, maybe someone else has thought about this already! Grrrrrr.

I just gave my first Skype presentation today, and it felt pretty strange.

The technical difficulties mostly arose with the sound. There were heavy echoes and so we ended up just cutting off the sound from the audience. This made it more difficult for me because I couldn't gauge audience reaction. It was a real challenge to give a talk without being able to hear the laughter of the audience. (I asked them to wave their hands every time they laughed, but they didn't do so--or else they were never laughing, which would be even worse.)

Next time I'll use the telephone for at least one of the sound channels.

The visuals were ok from my side--I just went thru my slides one by one, using the cursor to point to things. I prefer standing next to the screen and pointing with my hands. But doing it this way was ok, considering.

The real visual problem went the other way: I couldn't really see the audience. From the perspective of the little computer camera, everyone seemed far away and I couldn't really sense their reactions. I wonder if next time it would be better to focus on just one or two people in the audience whom I could see clearly.

My overall feeling was that it was strange to give a talk in an isolation booth with no feedback. Also, the talk itself was a bit unusual for me in that very little of it was about my own research. It's my own ideas (joint with Antony Unwin) but almost all the graphs are by others.

John Haubrick writes:

Next semester I want to center my statistics class around independent projects that they will present at the end of the semester. My question is, by centering around a project and teaching for the different parts that they need at the time, should topics such as hypothesis testing be moved toward the beginning of the course? Or should I only discuss setting up a research hypothesis and discuss the actual testing later after they have the data?

My reply:

I'm not sure. There always is a difficulty of what can be covered in a project. My quick thought is that a project will perhaps work better if it is focused on data collection or exploratory data analysis rather than on estimation and hypothesis testing, which are topics that get covered pretty well in the course as a whole.

Xian pointed me to this recycling of a classic probability error. It's too bad it was in the New York Times, but at least it was in the Opinion Pages, so I guess that's not so bad. And, on the plus side, several of the blog commenters got the point.

What I was wondering, though, was who was this "Yitzhak Melechson, a statistics professor at the University of Tel Aviv"? This is such a standard problem, I'm surprised to find a statistics professor making this mistake. I was curious what his area of research is and where he was trained.

I started by googling Yitzhak Melechson but all I could find was this news story, over and over and over and over again. Then I found Tel Aviv University and navigated to its statistics department but couldn't find any Melechson in the faculty list. Next stop: entering Melechson in the search engine at the Tel Aviv University website. It came up blank.

One last try: I entered the Yitzhak Melechson into Google Scholar. Here's what came up:

Your search - Yitzhak Melechson - did not match any articles

Computing wrong probabilities for the lottery must be a full-time job! Get this guy on the Bible Code next.

P.S. If there's some part of this story that I'm missing, please let me know. How many statistics professors could there be in Tel Aviv, anyway? Perhaps there's some obvious explanation that's eluding me.

Sam Jessup writes:

I am writing to ask you to recommend papers, books--anything that comes to mind that might give a prospective statistician some sense of what the future holds for statistics (and statisticians). I have a liberal arts background with an emphasis in mathematics. It seems like this is an exciting time to be a statistician, but that's just from the outside looking in. I'm curious about your perspective on the future of the discipline.

Any recommendations? My favorite is still the book, "Statistics: A Guide to the Unknown," first edition. (I actually have a chapter in the latest (fourth) edition, but I think the first edition (from 1972, I believe) is still the best.

Don Coffin writes:

A colleague of mine and I are doing a presentation for new faculty on a number of topics related to teaching. Our charge is to identify interesting issues and to find research-based information for them about how to approach things. So, what I wondered is, do you know of any published research dealing with the sort of issues about structuring a course and final exam in the ways you talk about in this blog post? Some poking around in the usual places hasn't turned anything up yet.

I don't really know the psychometrics literature but I imagine that some good stuff has been written on principles of test design. There are probably some good papers from back in the 1920s. Can anyone supply some references?

Joe Blitzstein and Xiao-Li Meng write:

An effectively designed examination process goes far beyond revealing students' knowledge or skills. It also serves as a great teaching and learning tool, incentivizing the students to think more deeply and to connect the dots at a higher level. This extends throughout the entire process: pre-exam preparation, the exam itself, and the post-exam period (the aftermath or, more appropriately, afterstat of the exam). As in the publication process, the first submission is essential but still just one piece in the dialogue.

Viewing the entire exam process as an extended dialogue between students and faculty, we discuss ideas for making this dialogue induce more inspiration than perspiration, and thereby making it a memorable deep-learning triumph rather than a wish-to-forget test-taking trauma. We illustrate such a dialogue through a recently introduced course in the Harvard Statistics Department, Stat 399: Problem Solving in Statistics, and two recent Ph.D. qualifying examination problems (with annotated solutions). The problems are examples of "nano-projects": big picture questions split into bite-sized pieces, fueling contemplation and conversation throughout the entire dialogue.

This is just wonderful and it should be done everwhere, including, I hope, in my own department. I am so tired of arguments about what topics students should learn, long lists of seemingly-important material that appears on a syllabus, is taught in a class, and is never used again, and so forth.

(The exam problems described in the article are a bit on the theoretical side for my taste, but I presume the same ideas would apply to applied statistics as well.)

P.S. I have fond memories of my own Ph.D. qualifying exam, which I took a year before Xiao-Li took his. It was an intense 12-day experience and I learned a huge amount from it.

John Christie sends along this. As someone who owns neither a car nor a mobile phone, it's hard for me to relate to this one, but it's certainly a classic example for teaching causal inference.

Just in case you didn't notice it on the blogroll.

Here's a good one if you want to tell your students about question wording bias. It's fun because the data are all on the web--the research is something that students could do on their own--if they know what to look for. Another win for Google.

Here's the story. I found the following graph on the front page of the American Enterprise Institute, a well-known D.C. think tank:

My first thought was that they should replace this graph by a time series, which would show so much more information. I did a web search and, indeed, looking at a broad range of poll questions over time gives us a much richer perspective on public opinion about Afghanistan than is revealed in the above graph.

I did a quick google search ("polling report afghanistan") and found this. The quick summary is that roughly 40% of Americans favor the Afghan war (down from about 50% from 2006 through early 2009).

The Polling Report page also features the Quninipiac poll featured in the above graph; here it reports that, as of July 2010, 48% think the U.S. is "doing the right thing" by fighting the war in Afghanistan and 43% think the U.S. should "not be involved." This phrasing seems to elicit more support--I guess people don't want to think that the U.S. is not doing the right thing.

OK, so we have 40% support, or maybe 48% support . . . how did the AEI get the 58% support highlighted on its graph?

Now that September has arrived, it's time for us to think teaching. Here's something from Andrew Heckler and Eleanor Sayre. Heckler writes:

The article describes a project studying the performance of university level students taking an intro physics course. Every week for ten weeks we took 1/10th of the students (randomly selected only once) and gave them the same set of questions relevant to the course. This allowed us to plot the evolution of average performance in the class during the quarter. We can then determine when learning occurs: For example, do they learn the material in a relevant lecture or lab or homework? Since we had about 350 students taking the course, we could get some reasonable stats.

In particular, you might be interested in Figure 10 (page 774) which shows student performance day-by-day on a particular question. The performance does not change directly after lecture, but rather only when the homework was due. [emphasis added] We could not find any other studies that have taken data like this, and it has nice potential to measure average effects of instruction.

Note also Figure 9 which show a dramatic *decrease* in student performance--almost certainly due to interference from learning a related topic.

I love this kind of thing. The results are not a huge surprise, but what's important to me about this kind of study is the active measurement that's involved, which can be difficult to set up but, once it's there, allows the opportunity to discover things about teaching and learning that I think would be nearly impossible to find out through our usual informal processes of evaluation. Some time I'm hoping to do this sort of project with our new introductory statistics course. (Not this semester, though; right now we're still busy trying to get it all working.)

Paul Bleicher writes:

This simply screams "post-hoc, multiple comparisons problem," though I haven't seen the paper.

A quote from the online news report:

The findings revealed that kindergarten matters--a lot. Students of kindergarten teachers with above-average experience earn $900 more in annual wages than students of teachers with less experience than average. Being in a class of 15 students instead of a class of 22 increased students' chances of attending college, especially for children who were disadvantaged . . . Children whose test scores improved to the 60th percentile were also less likely to become single parents, more likely to own a home by age 28, and more likely to save for retirement earlier in their work lives.

I haven't seen the paper either. $900 doesn't seem like so much to me, but I suppose it depends where you stand on the income ladder.

Regarding the multiple comparisons problem: this could be a great example for fitting a multilevel model. Seriously.

People sometimes email asking if a solution set is available for the exercises in ARM. The answer, unfortunately, is no. Many years ago, I wrote up 50 solutions for BDA and it was a lot of work--really, it was like writing a small book in itself. The trouble is that, once I started writing them up, I wanted to do it right, to set a good example. That's a lot more effort than simply scrawling down some quick answers.

Some thoughts from Mark Palko:

Of all the subjects a student is likely to encounter after elementary school, mathematics is by far the easiest to teach yourself. . . .

What is it that makes math teachers so expendable? . . .

At some point all disciplines require the transition from passive to active and that transition can be challenging. In courses like high school history and science, the emphasis on passively acquiring knowledge (yes, I realize that students write essays in history classes and apply formulas in science classes but that represents a relatively small portion of their time and, more importantly, the work those students do is fundamentally different from the day-to-day work done by historians and scientists). By comparison, junior high students playing in an orchestra, writing short stories or solving math problems are almost entirely focused on processes and those processes are essentially the same as those engaged in by professional musicians, writers and mathematicians. Unlike music and writing, however, mathematics starts out as a convergent process. . . .

This unique position of mathematics allows for any number of easy and effective self-study techniques. . . . All you need is a textbook and a few sheets of scratch paper. You cover everything below the paragraph you're reading with the sheet of paper. When you get to an example, leave the solution covered and try the problem. After you've finished check your work. If you got it right you continue working your way through the section. If you got it wrong, you have a few choices. . . .

I have nothing to add to this except to agree that, yes, doing mathematical research (or, at least, doing mathematics as part of statistical research) really is like doing math homework problems! An oft-stated distinction is that homeworks almost always have a clear correct answer, whereas research is open-ended. But, actually, when I do math for my research, it surprisingly often does work out. Doing (applied) mathematical research is a little bit like waking through the woods: sometimes I get stuck and have to work around an obstacle, and I usually don't end up exactly where I intend to go, but I usually make some progress. And in many cases the math is smarter than I am, in the sense that, through mathematical analysis, I'm able to find a correct answer that is surprising, until I realize how truly right it is.

Also relevant is Dick De Veaux's remark that math is like music, statistics is like literature.

In discussing the ongoing Los Angeles Times series on teacher effectiveness, Alex Tabarrok and I both were impressed that the newspaper was reporting results on individual teachers, moving beyond the general research findings ("teachers matter," "KIPP really works, but it requires several extra hours in the school day," and so forth) that we usually see from value-added analyses in education. My first reaction was that the L.A. Times could get away with this because, unlike academic researchers, they can do whatever they want as long as they don't break the law. They don't have to answer to an Institutional Review Board.

(By referring to this study by its publication outlet rather than its authors, I'm violating my usual rule (see the last paragraph here). In this case, I think it's ok to refer to the "L.A. Times study" because what's notable is not the analysis (thorough as it may be) but how it is being reported.)

Here I'd like to highlight a few other things came up in our blog discussion, and then I'll paste in a long and informative comment sent to me by David Huelsbeck.

But first some background.

Skirant Vadali writes:

According to Kaiser:

EdLab at Teachers' College does a lot of interesting things, like creating technologies for the classroom and libraries, blogging, and analyzing data sets in the education sector.

They're 2 blocks from my office and I've never heard of them! And I even work with people at Teachers College. Columbia's a big place.

P.S. Kaiser also makes an excellent point:

The most intriguing and unexpected question was: to do well in this business, do you have to read a lot? This is where I stumbled into a spaghetti carbonara analogy while mixing metaphors with the gray flannel, with which I have already been associated. Basically, statistics is not pure mathematics, there is not one correct way of doing things, there are many different methodologies, like there are hundreds of recipes for making carbonara. What statisticians do is to try many different recipes (methods), and based on tasting the food (evaluating the outcomes), we determine which recipe to use. Because of this, statisticians need to be well-read, to keep up with what are the new methods being developed.

This is actually the kind of thing that I say--complete with a cooking example!--except that in this case it's his idea and not mine.

Sometimes I follow up on the links of commenters and it turns out they have their own blogs. Here's Joseph Delaney's, on which I have a few comments:

Delaney's co-blogger Mark writes:

There are some serious issues that need to be addressed (but almost never are) when comparing performance of teachers. Less serious but more annoying is the reporters' wide-eyed amazement at common classroom techniques. Things like putting agendas on the board or calling on students by name without asking for volunteers (see here) or having students keep a journal and relate lessons to their own life (see any article on Erin Gruwell). Things that many or most teachers already do. Things that you're taught in your first education class. Things that have their own damned boxes on the evaluation forms for student teachers.

These techniques are very common and are generally good ideas. They are not, however, great innovations (with a handful of exceptions -- Polya comes to mind) and they will seldom have that big of an impact on a class (again with exceptions like Polya and possibly Saxon). Their absence or presence won't tell you that much and they are certainly nothing new.

To which I must reply: Yes, but. They never taught me this stuff when I was in grad school. And our students don't learn it too (unless they take my class). Lots and lots of college teachers just stand up at the board and lecture. Maybe things are better in high school. So even if it's "certainly nothing new," it's certainly new to many of us.

And here's another one from Mark, reporting on a lawsuit under which a scientist, if he were found to have manipulated data, could have to return his research money--plus damages--to the state. This seems reasonable to me. I just hope nobody asks me to return all the grant money I've received for projects I've begun with high hopes but never successfully finished. I always end up making progress on related work, and that seems to satisfy the granting agencies, but if they ever were to go back and see if we've followed up on all our specific aims, well, then we'd be in big trouble.

Judah Guber writes about his new company:

What we have done with Ultrinsic is created a system of incentives for students to allow them to invest in their ability to achieve a certain grade and when they achieve that grade we reward them with a cash incentive on top of receiving their original investment. This helps remove one of the large barriers students have to studying and staying motivated over the course of long semesters of college by giving them rewards on a much more immediate basis.

We have been doing a pilot program in 2 schools, NYU and Penn, for the past year or so, and are currently in the process of a major roll out of our services to 37 schools all across the country. This is due to our popularity and inquiries from students in tons of schools all around the country regarding getting Ultrinsic's services in their school. In the Fall 2010 semester, Ultrinsic will be revolutionizing student motivation on a grand scale . This is the dream of many economists: to change the cost benefit analysis of people to cause them to improve themselves and to even allow them to put them in control of this change.

Our system is forever sustainable because we earn a profit on motivating students to improve their performance, there will always be a motivation to sustain this program. Another very important aspect to our market, the college student, is that it's one thing that's never going to go out of style.

Guber asked if I wanted to interview him, which I did by email. Here are my questions and his responses:

AG: How will your system make money?

Yesterday we had a spirited discussion of the following conditional probability puzzle:

"I have two children. One is a boy born on a Tuesday. What is the probability I have two boys?"

This reminded me of the principle, familiar from statistics instruction and the cognitive psychology literature, that the best way to teach these sorts of examples is through integers rather than fractions.

For example, consider this classic problem:

"10% of persons have disease X. You are tested for the disease and test positive, and the test has 80% accuracy. What is the probability that you have the disease?"

This can be solved directly using conditional probability but it appears to be clearer to do it using integers:

Start with 100 people. 10 will have the disease and 90 will not. Of the 10 with the disease, 8 will test positive and 2 will test negative. Of the 90 without the disease, 18 will test positive and 72% will test negative. (72% = 0.8*90.) So, out of the original 100 people, 26 have tested positive, and 8 of these actually have the disease. The probability is thus 8/26.

OK, fine. But here's my new (to me) point). Expressing the problem using a population distribution rather than a probability distribution has an additional advantage: it forces us to be explicit about the data-generating process.

Consider the disease-test example. The key assumption is that everybody (or, equivalently, a random sample of people) are tested. Or, to put it another way, we're assuming that the 10% base rate applies to the population of people who get tested. If, for example, you get tested only if you think it's likely you have the disease, then the above simplified model won't work.

This condition is a bit hidden in the probability model, but it jumps out (at least, to me) in the "population distribution" formulation. The key phrases above: "Of the 10 with the disease . . . Of the 90 without the disease . . . " We're explicitly assuming that all 100 people will get tested.

Similarly, consider the two-boys example that got our discussion started. The crucial unstated assumption was that, every time someone had exactly two children with at least one born on a Tuesday, he would give you this information. It's hard to keep this straight, given the artificial nature of the problem and the strange bit of linguistics ("I have two children" = "exactly two," but "One is a boy" = "exactly one"). But if you do it with a population distribution (start with 4x49 families and go from there), then it's clear that you're assuming that everyone in this situation is telling you this particular information. It becomes less of a vague question of "what are we conditioning on?" and more clearly an assumption about where the data came from.

I've recently decided that statistics lies at the intersection of measurement, variation, and comparison. (I need to use some cool Venn-diagram-drawing software to show this.) I'll argue this one another time--my claim is that, to be "statistics," you need all three of these elements, no two will suffice-.

My point here, though, is that as statisticians, we teach all of these three things and talk about how important they are (and often criticize/mock others for selection bias and other problems that arise from not recognizing the difficulties of good measurement, attention to variation, and focused comparisons), but in our own lives (in deciding how to teach and do research, administration, and service--not to mention our personal lives), we think about these issues almost not at all. In our classes, we almost never use standardized tests, let alone the sort of before-after measurements we recommend to others. We do not evaluate our plans systematically nor do we typically even record what we're doing. We draw all sorts of conclusions based on sample sizes of 1 or 2. And so forth.

We say it, and we believe it, but we don't live it. So maybe we don't believe it. So maybe we shouldn't say it? Now I'm working from a sample size of 0. Go figure.

Patricia and I have cleaned up some of the R and Bugs code and collected the data for almost all the examples in ARM. See here for links to zip files with the code and data.

For "humanity, devotion to truth and inspiring leadership" at Columbia College. Reading Jenny's remarks ("my hugest and most helpful pool of colleagues was to be found not among the ranks of my fellow faculty but in the classroom. . . . we shared a sense of the excitement of the enterprise on which we were all embarked") reminds me of the comment Seth made once, that the usual goal of university teaching is to make the students into carbon copies of the instructor, and that he found it to me much better to make use of the students' unique strengths. This can't always be true--for example, in learning to speak a foreign language, I just want to be able to do it, and my own experiences in other domains is not so relevant. But for a worldly subject such as literature or statistics or political science, then, yes, I do think it would be good for students to get involved and use their own knowledge and experiences.

One other statement of Jenny's caught my eye. She wrote:

I'm not sure how the New York Times defines a blog versus an article, so perhaps this post should be called "Bayes in the blogs." Whatever. A recent NY Times article/blog post discusses a classic Bayes' Theorem application -- probability that the patient has cancer, given a "positive" mammogram -- and purports to give a solution that is easy for students to understand because it doesn't require Bayes' Theorem, which is of course complicated and confusing. You can see my comment (#17) here.

Eric Aspengren writes:

I've been attempting to teach myself statistics and I've recently purchased your book Bayesian Data Analysis. I have a small problem, unfortunately. I tend to need to have things explained to me by an actual, physical person (or a video). I've been able to use MIT's online video courses to help with my learning and was wondering if Columbia may have videos of your lectures available. If not, maybe there is a professor who teaches using your text that might have videos of their lectures available somewhere. I tend to be quite thick-headed when learning math.

I'm utterly fascinated by Bayesian methods. I work in politics and I feel there tends to be a lack of quantitative bases for decision making in this field. Conventional wisdom holds sway in many circles and I'm attempting to change that in my area.

My quick reply: I'd recommend starting with my book with Jennifer before moving on to Bayesian Data Analysis. Beyond this, no, I don't have any lectures online (except these). Actually, some of the online material on Bayesian statistics doesn't make me so happy (recall our blog discussion on the relevant Wikipedia articles). So I think you have to be careful what you listen to. Or, to put it another way, there's probably a lot of good stuff out there, but be careful not to take something seriously, just because someone says it in an authoritative manner.

P.S. Also, I write good books and give good one-hour lectures, but I'm not always so great over a one-semester course. I think you're better off taking a course out of my book but from a different lecturer who can present it from his or her own perspective.

Tyler Cowen links to a news article about a biology professor who got removed from a class after failing most of her students on an exam.

The article quotes lots of people, but what I wonder is: what's on the exam? It would seem to be difficult to make a judgment here without actually seeing the exam in question.

This note on charter schools by Alex Tabarrok reminded me of my remarks on the relevant research paper by Dobbie and Fryer, remarks which I somehow never got around to posting here. So here are my (inconclusive) thoughts from a few months ago:

Elizabeth Green writes a fascinating article about Doug Lemov, a former teacher and school administrator and current education consultant who goes to schools and tells teachers how they could do better. Apparently all the information is available in "a 357-page treatise known among its hundreds of underground fans as Lemov's Taxonomy. (The official title, attached to a book version being released in April, is 'Teach Like a Champion: The 49 Techniques That Put Students on the Path to College.')." I'd like to see the list of 49 techniques right now, but maybe you have to buy the book.

Green writes:

Central to Lemov's argument is a belief that students can't learn unless the teacher succeeds in capturing their attention and getting them to follow instructions. Educators refer to this art, sometimes derisively, as "classroom management." . . . Lemov's view is that getting students to pay attention is not only crucial but also a skill as specialized, intricate and learnable as playing guitar.

A lot of this resonated with my own experience in several roles:
- Student
- Teacher
- Author of a book on teaching tricks
- Teacher of teachers

A student wrote:

Hello, I've been researching a career in the field of statistics. I'm writing a high school paper on my career field which requires interviews of people working in that field. Would you consider scheduling a fifteen minute phone interview to help me with the paper? Please let me know if you would be willing to participate and when you are available. Thanks in advance.

I asked him to send me questions by email and received the following:

Austin Lacy writes:

I read your post on school gardens [see also here]. A close friend of mine taught a grade in which this was part of the curriculum. After he changed grades his successor continued the program, but instead had the middle-schoolers plant and harvest tobacco, yes tobacco. Not sure if the ATF ever caught wind of it, but his reaction to your post is below.

Thanks for sending this along; I genuinely enjoyed the article very much, which surprised me a bit (that a statistics blog should so engage me). However, I do think that both Gelman (and, by extension, Flannagan) miss both an opportunity (to extol school tobacco gardens that provide both a highly salable and potentially lucrative crop and the opportunity to scientifically prove, before the product's sale, that tobacco is bad for you and stuff) and the larger point (ridiculousness=memorability=good pedagogy). But it's a good start. I especially liked the idea that the composition of recipes makes one more capable of insights into the Crucible. I assume that same holds true for Shakespeare, and will look to implement a lesson to this effect shortly. And in this school climate, the following conversation could actually get me a raise, rather than a demotion:

Concerned Parent One: Mr. is at it again. He's doing a unit asking students to devise and compose the perfect recipe for spinach and parmesan risotto. And this right after he insisted that watching DIEHARD 4 was a worthwhile method of discussing effective character development in Romeo and Juliet.

Concerned Parent Two: Yes, but little Jimmy's recipe IS delicious. And he's giving extra credit if students devise a way to make their recipe low-sodium.

Concerned Parent One: Well, I guess. And little Suzy HAS been more interested in seeking out genuinely organic produce lately . . .

Eavesdropping School Administrator in Carpool Line: Hmmmmm. I like what I'm hearing . .

.

Mark Glaser writes an interesting but confusing article about a journalism class at NYU where students aren't allowed to blog or twitter about the class content:

After New York University journalism student Alana Taylor wrote her first embed report for MediaShift on September 5, it didn't take long for her scathing criticism of NYU to spread around the web and stir conversations. . . . By Taylor's account, [journalism professor Mary] Quigley had a one-on-one meeting with Taylor to discuss the article, and Quigley made it clear that Taylor was not to blog, Twitter or write about the class again.

Glaser then corresponds with Prof. Quigley, who emails:

I [Quigley] will confirm that I asked the class not to text, email or make cell phone calls during class. It's distracting to both me and other students, especially in a small class seated around a conference table. This has always been my policy, and I would hazard a guess that it's the policy of many professors no matter the discipline.

However, I did say after the class session they were free to text, Twitter, blog, email, post on Facebook or whatever outlet they wanted about the course, my teaching, the content, etc.

Seems clear enough: Keep your thumbs to yourself during the class period then write it all down later. Makes sense to me. But then Glaser reports:

When I [Glaser] followed up and asked her whether that meant students still needed to get permission before writing about class, she said: "Yes, I would certainly require a student to ask permission to use direct quotes from the class on a blog written after class."

Huh? Didn't she just say "they were free to text, Twitter, blog, email, . . . whatever they wanted about the course"? At this point, I wish Glaser had gone back to Quigley one more time for a clarification.

P.S. I looked up Mary Quigley on the web and found this list of articles by her students--judging from the quick summaries, apparently Quigley teaches a class on feature writing--and
this homepage, which to me was suprisingly brief, but I suppose that journalists have a tradition of not giving our their work for free.

P.P.S. Without knowing more details than what is in the links above, I'm 100% in support of Taylor, the student who was told not to blog. But I can definitely sympathize with Quigley: I can well imagine a student in one of my classes blogging something like this:

At the halfway point in the class, Quigley lets us go on a break. In the bathroom I run into an old classmate who asks me if I am going to stay in the class. I ask her if she doesn't like it and she responds that she is worried of it being too "all-over the place" or "disorganized" or "confusing."

Ouch!

P.P.P.S. I was amused that Taylor wrote that "I like to think that having a blog is as normal as having a car." Where exactly does she park?

I just finished grading my final exams--see here for the problems and the solutions--and it got me thinking about a few things.

#1 is that I really really really should be writing the exams before the course begins. Here's the plan (as it should be):
- Write the exam
- Write a practice exam
- Give the students the practice exam on day 1, so they know what they're expected to be able to do, once the semester is over.
- If necessary, write two practice exams so that you have more flexibility in what should be on the final.

The students didn't do so well on my exam, and I totally blame myself, that they didn't have a sense of what to expect. I'd given them weekly homework, but these were a bit different than the exam questions.

My other thought on exams is that I like to follow the principles of psychometrics and have many short questions testing different concepts, rather than a few long, multipart essay questions. When a question has several parts, the scores on these parts will be positively correlated, thus increasing the variance of the total.

More generally, I think there's a tradeoff in effort. Multi-part essay questions are easier to write but harder to grade. We tend to find ourselves in a hurry when it's time to write an exam, but we end up increasing our total workload by writing these essay questions. Better, I think, to put in the effort early to write short-answer questions that are easier to grade and, I believe, provide a better evaluation of what the students can do. (Not that I've evaluated that last claim; it's my impression based on personal experience and my casual reading of the education research literature. I hope to do more systematic work in this area in the future.)

I just graded the final exams for my first-semester graduate statistics course that I taught in the economics department at Sciences Po.

I posted the exam itself here last week; you might want to take a look at it and try some of it yourself before coming back here for the solutions.

And see here for my thoughts about this particular exam, this course, and final exams in general.

Now on to the exam solutions, which I will intersperse with the exam questions themselves:

This is pretty funny. And, to think that I used to work there. This guy definitely needs a P.R. consultant. I've seen dozens of these NYT mini-interviews, and I don't think I've ever seen someone come off so badly. The high point for me was his answering a question about pay cuts by saying that he's from Philadelphia. I don't know how much of this is sheer incompetence and how much is coming from the interviewer (Deborah Solomon) trying to string him up. Usually she seems pretty gentle to her interview subjects. My guess is what happened is her easygoing questions lulled Yudof into a false sense of security, he got too relaxed, and he started saying stupid things. Solomon must have been amazed by what was coming out of his mouth.

P.S. The bit about the salary was entertaining too. I wonder if he has some sort of deal like sports coaches do, so that even if they fire him, they have to pay out X years on his contract.

In the middle of a fascinating article on South Africa's preparations for the World Cup, R. W. Johnson makes the following offhand remark:

Any minute now the usual groaning will be heard from teams which claim that they, uniquely, have been drawn in a 'group of death'. What is the point, one might ask, in groaning about a random draw? Well, the trouble starts there, for the draw is not entirely random. In practice, seven teams are seeded, according to how well they've been doing in international matches, along with an eighth team, the host nation, whose passage into the second round is thus made easier - on paper. The draw depends on which balls rise to the top of the jar and thus get plucked out first; but it's rumoured that certain balls get heated in an oven before a draw, thus guaranteeing that they will bubble to the top. The weakest two teams aside from South Africa and North Korea are South Korea and New Zealand. The odds are, of course, heavily against any two or more of these bottom four finding themselves in the same group. If they do, we will have to be deeply suspicious of the draw.

This got me wondering. What is the probability that the bottom four teams will actually end up in different groups?

Given the rules as stated above, eight of the teams (including South Africa) start in eight different groups. There are 24 slots remaining. Now let's assign the next three low-ranking teams. The first has a 21/24 chance of being in one of the seven groups that does not have South Africa; the next has a 18/23 chance of being in one of the six remaining groups, and the next has a 15/22 chance of being in one of the five remaining. Combining these, the probability that the bottom four teams are in four different groups is 1-(21/24)*(18/23)*(15/22) = 0.53. (Unless I did the calculation wrong. Such things happen.)

So, no, I don't think that if two of these teams happen to find themselves in the same group, that "we will have to be deeply suspicious of the draw."

P.S. The 53% event happened: the four bottom-ranked teams are in different brackets. So we can breathe a sigh of relief.

Jimmy pointed me to this news article. My reaction to this is that the standards in teaching are low enough that someone like Xiao-Li or me can be considered to be an entertaining lecturer. It would be a lot hard to get by in standup.

That's ok. New ideas are overrated. But this bit was just hilarious:

This is one I'd never thought about . . . Daniel Rubenson writes:

I'm an assistant professor in the Politics Department at Ryerson University in Toronto. I will be teaching an intro statistics course soon and I wanted to ask your advice about it. The course is taught to fire fighters in Ontario as part of a certificate program in public administration that they can take. The group is relatively small (15-20 students) and the course is delivered over an intensive 5 day period. It is not entirely clear yet whether we will have access to computers with any statistical software; the course is taught off campus at a training facility run by the Ontario Fire College.

Guilherme Rocha writes:

Here. Official opening is Monday but youall get to see it earlier.

After reading what Christian wrote about the solutions manual that he and his collaborator wrote for their Bayesian Core book, I'm reminded of my own efforts with the Bayesian Data Analysis solutions. Not long after the first edition of the book came out, nearly fifteen years ago, I wrote up solutions to fifty of the homework problems (out of a total of about 150). People sometimes ask for more solutions, but the difficulty is that, once you have official solutions, you want them to be correct, you want them to be clear, and you want them to illustrate good statistical practice. It's a lot of work. Somehow I was able to write up those fifty, back when I had more time on my hands, but, really, writing up another fifty would almost be the equivalent of writing a (short) book! Originally I thought I could quickly put together a complete or nearly complete set by gathering solutions from students, or people just emailing them in, but I quickly realized that this wouldn't work. I think it would be ok to post scanned-in versions of student solutions, but once I start typing them up, I need them to be cleaner, and that takes work. That's one reason I didn't even try to write a solution set for ARM.

Phil Turk wrote:

Carl Bialik reports on a website called the Book of Odds (really, as Carl points out, these are probabilities, not odds, but that's not such a problem because, at least to me, probabilities are much more understandable than odds anyway). It's pretty cool. I could give some examples here but I encourage you to just go to the site yourself and check it out. One thing I really like is that it gives the source of every number: right on the page it gives the country and date of the information, then you can click to get the details. Awesome.

The only thing that bothers me a little bit about the site is that it is almost too professional. When something's that slick, I worry about whether I can trust them.

In contrast, Nate Silver's website is respected but not particularly attractive. And the NameVoyager is just the coolest thing in the world, and, yes, it's professional and it's commercial--that's fine--but it doesn't have the suspicion-inducing hyper-professionalism of the Book of Odds. Seeing the all-so-appealing photo of the bright-eyed oldsters illustrating the "Will you live to be 100?" item that's currently featured on the site's main page, I just think--this is too slick to be trusted. (In case you're wondering, their data say that a randomly-chosen 90-year-old has only a 1-in-9 chance of living to 100. Actually, they say 1 in 8.85, but you know what I think about extra decimal points.)

In some way I prefer the charmingly and unabashedly commercial OK Cupid site to the Book of Odds, which looks so, so commercial but claims only purely altruistic goals. I just don't know what to think.

Anyway, whatever the true story happens to be, it's great stuff. Fun to browse, and a great teaching tool too, I'd think. Enjoy.

Helen DeWitt links to this British-style rant (much different from my own story), which reminds me that I never erase the board after my classes either. But the board is usually clean for me. Probably because I usually teach morning classes. Once, though, about 15 years ago, I taught in a classroom right after a biology professor, an elderly lady who used several colors of chalk to construct beautiful pictures on the blackboard every lecture. She was very good about erasing as I was coming in. That semester and the one after that, I remembered to erase the board after class, but I've since slipped to the more natural equilibrium.

One more thing. A few years ago I invited Frank Morgan to give a talk at Columbia on how to teach better. One of the little things he discussed was how to use the blackboard. One of his points was that students aren't always paying attention--and, even if they are, they are often lost in thought trying to make sense of something or another. As a result, they won't be following every step of yours on the blackboard.

To put it another way: to the lecturer, the blackboard is a space-time process in which things get written down, erased, modified, and so on. To many a student, the blackboard is a series of snapshots of a spatial process. And it helps if each of these snapshots makes sense on its own terms. To that end, Frank recommends that you write a header at the top of each section of the blackboard so that students can keep track of what is going on.

Colin Gillespie writes:

A couple of weeks ago I did your suggested exercise (from Teaching Statistics: A Bag of Tricks) on 'Guessing the age', with the additional twist that the people were actors/actresses out of CSI. As well as discussing the data in class, I used it for there first R lab, where they generated simple scatterplots, boxplots and histograms.

In case your interested, the main results where:

1. Watching CSI didn't seem to affect your guess

2. Females guessed better than males.

3. The vast majority of guesses where too low (unsurprising for actors), except for the youngest actor.

If you are interested you can find my slides/handouts here.

Cool!

Pour l'année sabbatique, à Sciences Po.

John Sides and Joshua Tucker link to a news article by Jeremy Peters that reports that former New York State governor Eliot Spitzer is teaching a course at the City University of New York for "$98.43 an hour, or about $4,500 for the semester." This comes out to 45 or 46 hours--let's say 3 classroom hours a week for 15 weeks.

I noticed a few interesting things in the article.

1. I think it's ridiculous to consider $4500 for a course to be a rate of $98/hour. Teaching isn't just lecturing. You also have to prepare the classes, meet with students, write exams, and grade homeworks. $98/hour sounds like a lot, but it's based on a low denominator.

(There are exceptions, though. I know of a professor who paid the T.A. $100 per lecture to teach the class when the prof was out of town. It happened several times during the semester.)

2. I thought it was interesting that the commenters identified in the news article seemed to think that $4500 was a high rate of pay. I mean, suppose you teach 8 courses a year at $4500 each. That's $36,000. Hardly Richie Rich territory. This point is made at the very end of the article ("The point is not that Spitzer is paid too much, but rather that most adjuncts are paid too little") but it didn't really come through at first.

3. Sides writes that "This isn't pretty." I don't see what's so bad about Spitzer teaching a class. He knows a lot about politics and would seem to be well qualified to be an adjunct professor. I thought that was the ideal, to have adjuncts who are working professionals who take time off to teach a class.

I received the following email: