December 2008 Archives

I was going to write that I used to write articles in Latex and now I write them in Word (for some examples, see here), and then speculate about how the change in format might change how and what I write.

But then I thought that more background could be useful.

1. The pencil-and-paper era. Back when I was a kid I used to write everything by hand. I was proficient with the eraser. In high school I developed a style where I'd outline the English compositoin first and then write it.

2. Typewriter. In college we had to type our papers. Which I did, using my little typewriter (which had been my sister's before that), until . . .

3. Mark's homemade word processor. My college roommate was a CS major and had an Atari computer. I persuaded him to write a word processor in 6502 assembler, and I ended up using it more than he did.

I took a couple of English classes that required a paper every week or so, and to get everything done, I developed a system whereby I first diagrammed my plan on a single sheet of paper using circles and arrows, then wrote a series of outlines, the last of which had at least one sentence per paragraph of the final paper. I'd then take that outline, sit at the computer, and type up the paper pretty much in one take.

4. Troff. I wrote my senior thesis on a campus computer which printed out really nice--not that yucky dot-matrix stuff. I formatted it using Troff.

5. Pencil and paper again. For homework assignments in graduate school I went back to pencil and paper. I was still doing graphs by hand on graph paper (but that's another story).

6. Latex. One of my colleagues told me about Latex, and this quickly became my standard. When I wanted to write an article, I'd take an old latex file and map out sections and subsections, then fill in different parts when I was ready. I did it this way for several books and a few zillion articles.

I have to admit, I've never learned Bibtex, so I spend lots of time cutting and pasting bibliographic references.

7. Html. A few years ago I started this blog (originally intended as a way for members of our research group to communicate with each other). I often use the blog to record thoughts that later are published more formally. Writing in Html puts these thoughts in a different shape than what was happening in Latex. More conversational, less locked into a formal structure.

8. Word. Recently, for some reason I've been writing articles (and our forthcoming book) in Word, which doesn't work so well when I have formulas but somehow seems smoother otherwise.

The medium does affect the message. In many ways I'm dissatisfied with my current approach of composing at the keyboard. Maybe I'll try pencil and paper outlining for awhile.

Oh, yeah, . . . happy new year.

P.S. There are a bunch of comments below, but none of the commenters addressed my point, which was the way in which the typesetting or word processing environment affects the style of writing (and the choice of what to write about). Lots of suggestions about Latex implementations, but this is really beside the point here.

Peter Woit relates a story about how four physicists did work that led to a Nobel Prize, but the rules only allowed it to be given to three of them, creating a motive for murder. The story is consistent with Andrew Oswald's finding that not getting the Nobel Prize reduces your expected lifespan by two years. The fited article frames it as that winning the prize increases your lifespan, but so many more eligible people don't get it than do (and the No comes year after year). I'd guess that it's a net reducer of scientists' lifespans. Even setting murder aside.

Jim Dannemiller writes:

I ran across your discussion on retrospective power and power calculations in general, and I thought that you might be interested in this manuscript [link fixed] that Ron Serlin and I are working on at present.

Their idea is to formally put costs into the model and then optimize, instead of setting Type 1 and Type 2 error rates ahead of time. I'm already on record as not being a fan of the concepts of Type 1 and Type 2 errors; that said, we do work in that framework in chapter 20 of our book, and my guess is that it does make sense to put costs into the model explicitly. So I imagine this article by Dannemiller and Serlin is a step forward.

P.S. Jim also said he liked our Teaching Statistics book!

Seth points to this article by Edward Vul, Christine Harris, Piotr Winkielman, and Harold Pashle, which begins:

The newly emerging field of Social Neuroscience has drawn much attention in recent years, with high-profile studies frequently reporting extremely high (e.g., >.8) correlations between behavioral and self-report measures of personality or emotion and measures of brain activation obtained using fMRI. We show that these correlations often exceed what is statistically possible assuming the (evidently rather limited) reliability of both fMRI and personality/emotion measures. The implausibly high correlations are all the more puzzling because social-neuroscience method sections rarely contain sufficient detail to ascertain how these correlations were obtained. We surveyed authors of 54 articles that reported findings of this kind to determine the details of their analyses. More than half acknowledged using a strategy that computes separate correlations for individual voxels, and reports means of just the subset of voxels exceeding chosen thresholds. We show how this non-independent analysis grossly inflates correlations, while yielding reassuring-looking scattergrams. This analysis technique was used to obtain the vast majority of the implausibly high correlations in our survey sample. In addition, we argue that other analysis problems likely created entirely spurious correlations in some cases.

This is cool statistical detective work. I love this sort of thing. I also appreciate that the article has graphs but no tables. I have only two very minor comments:

1. As Seth points out, the authors write that many of the mistakes appear in "such prominent journals as Science, Nature, and Nature Neuroscience." My impression is that these hypercompetitive journals have a pretty random reviewing process, at least for articles outside of their core competence of laboratory biology. Publication in such journals is taken much more of a seal of approval than it should be, I think. The authors of this article are doing a useful service by pointing this out.

2. I think it's a little tacky to use "voodoo" in the title of the article.

Jon Peltier saw this horrible graph that I'd discussed earlier:

Peltier writes:

Well, this is an eye-catching chart. It seems to show an inward spiral, but the overall trend is really not very clear. It also looks distorted, too tall and not wide enough, but I examined axis settings several times, then even physically measured the lengths of the January-July and April-October spokes, and everything lined up. This optical illusion was caused by plotting the data in a radar chart: April and October were the two largest numbers in 1929, stretching the curve vertically. The first step to improving this chart is to cut it between December and January, and unroll it. . . .

In the chart below, we can easily see the downward trend which started around the time of the stock market crash of October 1929. The trend was well underway even before the Smoot-Hawley Act was enacted in June 1930, because many international trade partners had instituted preemptive retaliatory tariffs of their own. By the middle of 1932, the volume of international trade had effectively plateaued at one-third of its high of 1929:

That's what I'm talking about.

P.S. Jason Roos writes:

In his aforementioned chapter, Stephen Senn writes:

"In order to interpret a trial it is necessary to know its power": This is a rather silly point of view that nevertheless continues to attract adherents. A power calculation is used for planning trials and is effectively superseded once the data are in. . . . An analogy may be made. In determining to cross the Atlantic it is important to consider what size of boat it is prudent to employ. If one sets sail from Plymouth and several days later sees the Statue of Liberty and the Empire State Building, the fact that the boat employed was rather small is scarcely relevant to deciding whether the Atlantic was crossed.

I used to think this too, but after writing my paper with David Weakliem, I've changed my stance on the relevance of retrospective power calculations. In that article, Weakliem and I discussed the problem of Type M (magnitude) errors, where the true effect is small but it is estimated to be large. One problem with underpowered studies is that, when they do turn up statistically significant results, they tend to be huge compared to the true effect sizes.

On the other hand, large studies can be a huge waste of effort, so I don't really know what I would recommend for medical research.

Following this discussion of his statistical advice, Stephen Senn sent me this chapter on "Determining the Sample Size", which has a great beginning:

Clinical trials are expensive, whether the cost is counted in money or in human suffering, but they are capable of providing results which are extremely valuable, whether the value is measured in drug company profits or successful treatment of future patients. Balancing potential value against actual cost is thus an extremely important and delicate matter and since, other things being equal, both cost and value increase the more patients are recruited, determining the number needed is an important aspect of planning any trial. It is hardly surprising, therefore, that calculating the sample size is regarded as being an important duty of the medical statistician working in drug development.

I'll pile on the references by linking to chapter 20 of my book with Jennifer, which, compared to Senn's book, has a bit more calculation and a bit less discussion. I suspect many readers will benefit from reading both. (Full link to the book here.)

Finally, here's a link with a couple more references, including a great little article by Russ Lenth.

Aleks links to "The Manga Guide to Statistics":

and commenter David Warde-Farley links to the similar-looking "Cartoon Guide to Statistics."

My thoughts

Based on the example shown above, the point of the comic-book format seems to be to allow a punchy, power-point sort of delivery. The picture conveys essentially no content, which would suggest that the entire contents of a 222-page comic book could be presented in a 10-page pamphlet of text. The remaining 212 pages are essentially a reader-friendly trick to get students to turn the pages. It's the printed analogy to a power-point presentation.

So . . . let's take the customers' word for it that these cartoon guides are good. If so, this suggests that the useful content of a typical introductory statistics book can be captured in 10 pages. And, if this is the case, it in turn suggests that textbook writers could do a better job with those other 212 pages. Maybe it would be better to have a 10-page textbook and 212 pages of examples? Presumably a good textbook author could do better than those silly cartoons.

It's a tricky issue. Thinking about my own 600-to-700-page textbooks, it's hard for me to see what I would cut to bring it down to 30 pages. At the same time, the actual material that students learn in the class can probably be written in 30 single-spaced pages, so maybe it would be a good idea to try to pull that out.

(My books aren't directly comparable to these comic books, as I'm covering higher-level material. But the issues of presentation can't be that different.)

I read the recent story by Colson Whitehead in the New Yorker. It was hilarious. I'd never read anything by Whitehead before. Now I want to go back and read whatever else he's written.

We recently uploaded on to CRAN multiple imputation package "mi" which we have been developing.

The aim of package mi is to make multiple imputation transparent and easy to use for the user. Hence there are few characteristics that we believe are valuable.
1. Graphical diagnostics of imputation models and convergence of the imputation process.
2. Use of bayesglm to treat the issue of separation.
3. Imputation model specification is made similar to how you would fit a regression model in R.
4. It automatically detects some problematic characteristics in the given dataset and alerts the user.

Please give it a try if you have any dataset that has missingness.

Also we are still in the process of improving the package, thus your input is most welcome.

One caution is if you are using big dataset with large number of missingness across many variables, it may take some time for process to converge. We admit, it is not the fastest imputation package on the market.

However, once we can get the basics down, speeding things up is not so difficult. So please bare with it for now.

There are future directions we plan to expand such as imputation of time-series cross-sectional data, hierarchical data, etc. But for now these features are not part of the package.

Happy Holidays!!

Sergey Aksenov pointed me to this article by Deepak Hegde and David Mowery:

Pablo Pinto and Boliang Zhu write:

The effects of foreign direct investment (FDI) on government corruption are conditional on the host country's underlying economic and political climate. . . . The effect of FDI on corruption is positive in authoritarian and poor countries, and turns negative as countries develop and become more democratic.

Here's their reasoning:

The underlying structure of the economy determines the possibility of extracting rents that could be distributed among foreign investors and the incumbent. Political development, on the other hand, determines the level of accountability of the incumbent, and creates a check on the incumbent's ability to appropriate those rents, and the probability of getting caught and sanctioned for engaging in corrupt behavior. Hence, we [Pinto and Zhu] argue, FDI will be associated with higher corruption levels in political and economic environments with restricted competition. In more competitive political systems with diversified economies, on the other hand, FDI inflows are likely to reduce the ability of the incumbent to engage in corrupt behavior.

Dan Goldstein pointed me to this op-ed referring to the work of Arthur Brooks on charitable contributions of Democrats and Republicans. For convenience, I'll repeat my comments from two years ago:

I was thinking more about a framework for understanding these findings by Arthur Brooks on the rates at which different groups give to charity. Some explanations are "conservatives are nicer than liberals" or "conservatives have more spare cash than liberals" or "conservatives believe in charity as an institution more than liberals." (My favorite quote on this is "I'd give to charity, but they'd spend it all on drugs.")

But . . . although I think there's truth to all of the above explanations, I think some insight can be gained by looking at this another way. Lots of research shows that people are likely to take the default option (see here and here for some thoughts on the topic). The clearest examples are pension plans and organ donations, both of which show lots of variation and also show people's decisions strongly tracking the default options.

For example, consider organ donation: over 99% of Austrians and only 12% of Gernans consent to donate their organs after death. Are Austrians so much nicer than Germans? Maybe so, but a clue is that Austria has a "presumed consent" rule (the default is to donate) and Germany has an "explicit consent" rule (the default is to not donate). Johnson and Goldstein find huge effects of the default in organ donations, and others have found such default effects elsewhere.

Implicit defaults?

My hypothesis, then, is that the groups that give more to charity, and that give more blood, have defaults that more strongly favor this giving. Such defaults are generally implicit (excepting situations such as religions that require tithing), but to the extent that the U.S. has different "subcultures," they could be real. We actually might be able to learn more about this with our new GSS questions, where we ask people how many Democrats and Republicans they know (in addition to asking their own political preferences).

Does this explanation add anything, or am I just pushing things back from "why to people vary in how much they give" to "why is there variation in defaults"? I think something is gained, actually, partly because, to the extent the default story is true, one could perhaps increase giving by working on the defaults, rather than trying directly to make people nicer. Just as, for organ donation, it would probably be more effective to change the default rather than to try to convince people individually, based on current defaults.

P.S. More Arthur Brooks links are at item 5 here.

Steve Buyske points me to this:

Boy do I hate this. A straight time series would do so much better. They should also follow the general principle of extending the series, going back before 1929 and after 1933. But my main feeling is that this spiderweb action is just horrible.

This article, by Jim Stallard, is just hilarious. It's at the intersection of politics and basketball. There are just so many funny lines here, I just don't know where to start.

Alessandra Stanley writes in the New York Times, regarding the protagonists of two recent economic and political scandals:

Even their names, Madoff and Blagojevich, have a Dickensian ring, like Skimpole or Pardiggle.

Madoff? Blagojevich? These don't sound very Dickensian to me! I have to admit that I've only read a few of Dickens's books, so maybe I'm missing something. But these names sound more "ethnic" (as we used to say) than Dickensian. I think the real question is, if Alessandra Stanley things Madoff and Blagojevich have a Dickensian ring, are there any names that wouldn't sound Dickensian to her?

P.S. By comparison, here's a list of names that really do sound like they belong in a Dickens novel.

Christian Grose sent me this article, along with the comment that their paper is somewhat relevant to my article with Edlin and Kaplan. Grose writes, "Our argument differs from yours in that we examine (1) voting when there is no secret ballot, such as during the Iowa caucus; and (2) we examine costs and benefits to the individual that are socially induced." Here's the abstract of the Grose and Russell article:

Citizen vote choice in modern democracies is almost always a private act, yet the act of turning out to vote is a social and public activity. Does voting in public increase or depress turnout? We present a theory of the effect of social pressure and ostracism on political participation, arguing that both social costs and social benefits can affect individual behavior. No theoretical work on voter turnout has argued that social costs deter participation, but it is a fundamental explanation of the decision not to participate. We test our theory by conducting a randomized field experiment during the 2008 Iowa Democratic presidential caucus, sending mailers to registered Democrats suggesting different reasons for voting. We include three treatments: (1) one telling citizens of the date, time, and location of the caucus; (2) one telling citizens the caucus is a public meeting where neighbors and friends will be attending; and (3) one telling citizens that the caucus does not have a secret ballot and that their neighbors and friends will be attending. We find that citizens are more likely to vote when information costs are reduced and when they are told the caucus is a public meeting. However, we find that turnout is reduced when citizens are told their vote choices must be revealed to their neighbors, thus increasing social costs. Unlike nearly all other field experiments of voter turnout, our experiment is one of the only to find a suppression of the vote as a result of a treatment. These findings provide insights into individual behavior in a social context, a rejection of one explanation for heavy voter turnout in 19th century America, and practical insights for campaigns interested in mobilizing voters to presidential caucuses.

Check it out:

This should be interesting:

Jamie forwarded this paper by Craig Garthwaite and Tim Moore:

Candidates in major political contests are commonly endorsed by other politicians, interest groups and celebrities. Prior to the 2008 Democratic Presidential Primary, Barack Obama was endorsed by Oprah Winfrey, a celebrity with a proven track record of influencing her fans' commercial decisions. In this paper, we use geographic differences in subscriptions to O! The Oprah Magazine and the sale of books Winfrey recommended as part of Oprah's Book Club to assess whether her endorsement affected the Primary outcomes. We find her endorsement had a positive effect on the votes Obama received,increased the overall voter participation rate, and increased the number of contributions received by Obama. No connection is found between the measures of Oprah's influence and Obama's success in previous elections, nor with underlying local political preferences. Our results suggest that Winfrey's endorsement was responsible for approximately 1,000,000 additional votes for Obama.

This looks interesting--I'll have to read it...

Mike Larsen, editor of Chance magazine, passed along the announcement for a graphics contest. Entries are due 15 Jan 2009, and there is a specific requirement, which is that they display the data described below (and also here).

CHANCE GRAPHIC DISPLAY CONTEST: Burtin's antibiotic data

The year 2008 marks the 100th anniversary of the birth of Will Burtin (1908-1972). Burtin was an early developer of what has come to be called scientific visualization.
In the post World War II world antibiotics were called "wonder drugs" for they provided quick and easy cures for what had previously been intractable diseases. Data were being gathered to aid in learning which drug worked best for what bacterial infection. Being able to see the structure of drug performance from outcome data was an enormous aid for practitioners and scientists alike. In the fall of 1951 Burtin published a graph showing the performance of the three most popular antibiotics on 16 different bacteria.

Slashdot has a review of "The Manga Guide to Statistics". Here is a snippet:

The story is silly and sets up some humorous examples of how to use statistics. Ramen noodle prices get graphed, Rui looks at grading on a curve and explores why her and a class mate get different grades for identical scores. Cramer's coefficient is used to examine how boys and girls prefer to be asked out. I thought that this was helpful not only because it helps to keep the readers interest but because it also moves the problems from the abstract to more concrete applications.

I haven't seen the book, but I like the tagline: "Statistics with heart-pounding excitement!"

Bob writes:

You should to team up with these guys to promote your book. This one looks like it could come out of one of your papers with its shared horizontal scale, no ticks, and no vertical scale. It was even done with the help of a NY prof.

I don't really know what's going on here, but if anyone wants to go with this, be my guest...

From Chance News come these quotations from Stephen Senn's book Statistical Issues in Drug Development:

"Most Trials are unethical because they are too large." Page 178.

"Small Trial are unethical." Page 179.

"A significant result is more meaningful if obtained from a large trial." Page 179.

"A given significant P-value is more indicative of the efficacy of a treatment if obtained from a small trial." Page 180.

"For a given P-value, the evidence against the null hypothesis is the same whatever the size of the trial." Page 182.

Perhaps Stephen can explain? He's a funny guy, so I don't really know if these are jokes or if there are more serious points he's trying to make by these contradictions.

After seeing this discussion of Bill Easterly's discussion of international development and my graphical summary of some of Page Fornta's work on conflict resolution, Andrew Mack, director of the Human Security Project at Simon Fraser University, sent me this political science take on Paul Collier's rejection of grievance-based explanations of the drivers of armed conflict:

The dismissal of political and economic grievances as drivers of civil war is one of the most contentious findings to emerge from quantitative research on armed conflicts. Critics of Collier and his co-author Anke Hoeffler have argued that the proxy measures they use are inappropriate, that a number of other assumptions are problematic, and that other quantitative studies, plus a mass of case study evidence, demonstrate that grievances are indeed important risk factors for armed conflict.

But there is a more profound reason for contesting the claim that grievances don't matter in explaining the onset of civil wars--one that cannot be rebutted by creating more appropriate proxy measures, better cross-national data, or using different statistical significance tests.

John Side posts this graph from Lee Drutman of lobbying expenditures and bailout funds for several financial firms:

This is fun, and I recognize that this graph, in John's words, "a back of the envelope analysis" and "only subjective," but . . . isn't there a problem with selection bias here--you're only considering cases that had bailouts. I'd like to see a lot more data points here: firms with many different levels of lobbying but zero bailout.

But the graph raises an interesting theoretical point: larger firms will be giving more total lobbying dollars and thus may be more likely to be bailed out? Even beyond any "too big to fail" argument based on economics.

This whole exchange is interesting and is closely related to our current research on prior distributions and Bayesian inference for varying-intercept, varying-slope models.

It all started when Sean Zhang asked,

I am using your book to self-teach myself using R for multilevel modeling.

One question I have is that why lmer cannot provide var-cov matrix of estimates of random components. You mentioned in your book that lmer can only provide point estimates for variance component and that is one of the reasons to go Bayesian and use Bugs.

I am running a simple random intercept model using SAS glimmix and found that glimmix provides standard error for variance estimate of random intercept. I looked into glimmix document(see attachment, page 121, theta contains random effect parameters) and can imagine that SAS may use hessian or outer produce of gradient (see page for their likelihood function) to get them.

My question is then why lmer cannot do similar thing as SAS to report var-cov matrix of estimates of random components?

I replied,

This discussion from Keynes (from Robert Skidelsky, linked from Steve Hsu) reminds me of a frustrating conversation I've sometimes had with economists regarding the concept of "risk aversion."

Nolan McCarty writes:

Saxby Chambliss won reelection in the Georgia Senate run-off by a somewhat surprising margin 57-43% margin over Democrat Jim Martin. . . . there seems to be an emerging pattern of the newly elected president's party losing in run-off elections. Of course, the closest parallel was in Georgia in 1992 when republican Paul Coverdell beat incumbent Democrat Wyche Fowler following Bill Clinton's presidential victory. . . . Political scientists and economists such as Alberto Alesina, Howard Rosenthal, and Mo Fiorina have offered a "balancing" explanation as to why the new president's party performs poorly in these special elections and in midterm elections generally. The basic idea is that most voters are more ideological moderate than the two parties and therefore would like to balance them through divided government. . . . in a special or midterm election, voters have a clear opportunity to promote balance by voting against the president's party.

Isn't there a simpler explanation? Runoff elections generally have lower turnout than general elections (especially if the general election has the president on the ballot). Lower-turnout elections generally favor Republicans and conservatives. Chambliss won a plurality in the primary election, then you go to a lower-turnout runoff and you'd expect him to do even better, which he did. (Similarly for Republican candidate Coverdell in 1992.)

Is "balancing" really needed to explain this at all?

P.S. I agree with McCarty that the whole 60 votes thing has been overemphasized. Realistically there's a limit to how many times the minority will want to filibuster against legislation that is popular enough to be passed by a majority in the House and Senate.

A couple weeks ago I posted an analysis of rich and poor voters in rich and poor states from exit polls in 2008, and a commenter ("Audacious Epigone") picked up on Larry Bartels's observation that, among whites, the Republican advantage among richer voters compared to poorer voters is not larger in poor than in rich states. As "Audacious" writes:

We talk a lot about inference from Markov chain simulation (Gibbs sampler and Metropolis algorithm), convergence, etc. But inference from simple random samples can be nonintuitive as well.

Consider this example of inference from direct simulation. The left column shows five replications of 95% intervals for a hypothetical parameter theta that has a unit normal distribution, each based on 100 independent simulation draws. Right column: five replications of the same inference, each based on 1000 draws. For both columns, the correct answer is [-1.96, 1.96].

Inferences based on         Inferences based on
100 random draws            1000 random draws
[-1.79, 1.69]               [-1.83, 1.97]
[-1.80, 1.85]               [-2.01, 2.04]
[-1.64, 2.15]               [-2.10, 2.13]
[-2.08, 2.38]               [-1.97, 1.95]
[-1.68, 2.10]               [-2.10, 1.97]

From one perspective, these estimates are pretty bad: even with 1000 simulations, either bound can easily be off by more than 0.1, and the entire interval width can easily be off by 10%. On the other hand, for the goal of inference about theta, even the far-off estimates above aren't so bad: the interval [-2.08, 2.38] has 97% probability coverage, and [-1.79, 1.69] has 92% coverage.

So, in this sense, my intuitions were wrong and wrong again: first, I thought 100 or 1000 independent draws were pretty good, so I was surprised that these 95% intervals were so bad. But, then, I realized that my corrected intuition was itself flawed: actually, nominal 95% intervals of [-2.08, 2.38] or [-1.79, 1.69] aren't so bad at all.

This is an example I came up with for my chapter that's going into our forthcoming Handbook of Markov Chain Monte Carlo (edited by Galin Jones, Xiao-Li Meng, Steve Books, and myself).

A potential applicant writes:

I am considering applying to the postdoc positions at the Applied Statistics Center advertised at http://applied.stat.columbia.edu/postdocads.php. From the description in that page, it is not clear to me whether the selected postdocs are expected to use part of their time in their own projects. There is no information, either, on what the postdoc positions offer, in terms of salary and benefits. Lastly, there is no advertised application deadline. I would appreciate it a lot if you could give me information on these issues.

My reply:

1. Yes, postdocs are expected to spend time on their own projects, possibly in collaboration with others here.

2. Salary and benefits are competitive with salary and benefits for statistics postdocs elsewhere.

3. There is no application deadline. We consider the applicants as they come in.

4. I'll be in sabbatical next year so I'm not sure if I'll hire someone here--it depends on what grants are funded. But the Applied Statistics Center at Columbia includes other researchers, and when you apply for our postdoc, faculty in various departments might take a look at your application.

AT writes:

A Facebook friend pointed me to this Gladwell piece discussing how you can('t) predict whether a teacher will be successful, but more importantly, on the range of advancement of a class depending on a teacher's ability.

The claim that a good teacher can make as much as a full year's difference in their students' advancement isn't too surprising to me [AT]. What I want to know is whether there are good studies around that look at the interaction effects between teachers and their students' backgrounds. In particular, I'd say that there are a number of teachers I've had who are very polarizing -- some make their students advance far, some stall them in their paths and make them give up the discipline.

My quick reply: From the work of Jonah Rockoff and others, I am convinced that teacher effects are real and they are large. And school effects are mostly the composition of teacher effects. I'm not sure about how large the interactions are (i.e., if some teachers do better with good students and others do better with poor students). Jennifer and I have talked about estimating such interactions (with a big multilevel model, of course) but I don't know what's up with that.

And, of course, this has no implications for university teaching, where as we know the sole qualification is to publish technical articles in obscure journals.

Lane Kenworthy writes (link from here and here):

The notion that political parties are a key determinant of income inequality has been around for a long time. I suspect many non-academics take its truth for granted. Among American scholars, the notion is perhaps most closely associated with Douglas Hibbs . . .

[In his recent book, Unequal Democracy], Larry Bartels suggests that a key part of the story is different policies pursued by Democratic and Republican presidents. . . . Bartels' argument, while by no means novel, is very much a fresh one. It is based on extensive empirical analysis of the post-World War II period. Is he correct? I think Bartels probably has it right for part of this period, but I'm not convinced that his hypothesis holds up for the other part. . . .

This relates to some ideas I had after seeing Bartels speak on his work at Columbia a couple of years ago; see here and here. In particular, in that last link, I wrote the following:

After seeing Larry Bartels present his findings on how the economy has done better, for the poor and middle class, under Democratic presidents than Republican presidents, I was puzzled. Not that it couldn't be true, but it seemed a little mysterious, given the general sense that presidents don't have much control over the econony--business cycles just seem to happen sometime.

But the general perceptions about Presidents and the economy have changed over time.

I might be wrong here, not having lived through the entire postwar period, but my perception is that, during most of this time, "competence" was not an issue; rather, there was a general belief that the president could do some things, most notably help labor (for the Democrats) or business (for the Republicans).

The exception here was the 1976-1996 period, during which there was a real sense of economic incompetence or powerlessness of some presidents (Ford with his Whip Inflation Now, Carter with stagflation, the residual view of Democrats being incompetent for the economy, George H.W. Bush with the deficit and the regression, perhaps extending to Dole in 1996). Then, since 2000, we've returned to the general attitude that both parties have essential competence but have different goals. (Not that everyone agrees on the "competence" issue, but it seems to me that the battle is more being fought on priorities than competence--in contrast to 1992, for example.)

So, the conventional wisdom based on the 1976-1996 period is that presidents can't do much, they're at the mercy of the business cycle, etc., which makes Bartels's results seem like some sort of fluke, or a perhaps meaningless juxtaposition of one-off results. But taking the 1948-1972 and 2000-2004 perspectives, Bartels's graph makes a lot of sense. From this perspective, the Democrats did their thing, and the Republicans did theirs, and you'd expect to see a big difference at the low end of the income scale. (Again, this is inherently short-term reasoning, not long-term, but as Larry pointed out in his talk, the evidence is that voters are susceptible to short-term inferences.)

In summary: we're used to thinking of presidents as fairly powerless surfers on the global economy, able to tinker with tax rates but not much more--but thinking about the entire postwar period, there's certainly been at least the perception that presidents can deliver the economic goods to their constituencies. So from that perspective, Larry's curves should not be much of a surprise--at least in that the slope for Democrats goes down (i.e., poor people do better under Democratic presidents) and the slope for Republicans goes up (i.e., rich people do better under Republican presidents). The relative positions of the lines is another story, which perhaps corresponds to random alignments of the business cycle.

Perhaps Kenworthy can connect this thinking more directly to his arguments. My time frames don't quite align with his, but it's a similar idea of breaking the period into smaller segments.

And, to comment on my comments . . . when posting the above in 2006, I wrote, "since 2000, we've returned to the general attitude that both parties have essential competence but have different goals. . . . we're used to thinking of presidents as fairly powerless surfers on the global economy, able to tinker with tax rates but not much more. . ." Things sure have changed in 2 1/2 years!

Sometimes people will email me that their comments aren't published on the blog. It's a good idea to be a registered user to prevent this from happening - as we have tens of thousands of spammy messages, and one sees unspeakable things there. So it was interesting to see a visualization (developed by some famous open source developers) of where blog spam comes from:

It's a great visualization, except for the colors: the USA is bright red. But what does this tell us? That the USA has the highest number of computers on the World Wide Web, and the total number of blog comments posted? We know that already! The visualization should provide information that isn't known already.

So should one just present the ratio between spammy and hammy comments for each country? That would be valid, but it would involve ad-hoc modeling. Instead, one has to build a model that removes the influence of variables that are already known to influence the outcome, such as the number of computers, the number of all comments posted, and so on. I'll write more about how to do this another day.

This is cute too. I suggest, in addition:

1. Plotting the y-axis on the log scale.

2. Normalizing by total number of words (of, if that's hard to find, something easier such as total number of articles).

Regarding my article on the boxer, the wrestler, and the coin flip, Steve Hsu writes:

A world class wrestler would easily demolish a top boxer in a no holds barred fight. This has been verified by in many experiments (Inoki-Ali doesn't count)!

Steve has more details in this blog entry from 2007:

Ultimate fighting has grown from obscurity to unbelievable levels of popularity. It will soon surpass boxing as the premier combative sport. And it will soon be widely recognized that the baddest man on the planet is not a boxer, but an ultimate fighter. . . .

Unarmed single combat -- mano a mano, as they say -- has a long history, and is a subject which fascinates most men, both young and old. As a boy, I can remember serious discussions with my friends concerning which style was most effective -- karate or kung fu, boxing or wrestling, etc. How would Muhammed Ali fare against an Olympic wrestler or Judo player? What about Bruce Lee versus a Navy Seal? Of course, these discussions were completely theoretical, akin to asking whether Superman could beat Galactus in arm wrestling. There was scarcely any data available on which to base a conclusion.

However, thanks to the recent proliferation of "No Rules" or "No Holds Barred" (NHB) fighting tournaments, both in the U.S. and abroad, we finally have some interesting answers to this ancient question.

Darryl Caldwell writes:

I enjoyed your response to Satoshi Kanazawa's statistical data on sex ratios. I have a quick question for you. Did he respond? How was his response?

My reply: I sent him an email but he did not respond. I assume he must be aware of my comment on his article, in any case. I was disappointed in his lack of response and even more disappointed that he then wrote an entire book on this stuff without addressing these concerns. (Search this blog for Kanazawa for more than you want to know about this topic.) The upside is that I got a publication in the Journal of Theoretical Biology, something which probably otherwise never would've happened!

Also relevant is this paper I wrote with David Weakliem, a paper which I will soon devote an entire blog entry, if for no other reason than to highlight some really quotable bits that David threw in to the revision.

Helen DeWitt writes:

Jesse Bering writes:

Researchers have found that, at least when it comes to what goes on in our own heads, there's not much of a conflict between religion and science. Sure, that bad case of strep throat your kid got right before your scheduled vacation to Barbados was caused by her chewing on a virus-laden pencil she'd borrowed in math class. . . . But that doesn't mean God's not trying to tell you something by--what's the best word here--'authoring' these events. . . . this way of thinking as "co-existence reasoning," where natural, scientific forces are viewed as directly causing a certain event, but supernatural forces are perceived simultaneously as somehow blowing life into this science. Another way to say this is that science and God often co-exist harmoniously in the same mindset, with science acting 'proximally' and God acting 'distally.' . . .

This looks interesting but I can't quite figure out what the experimental findings are. I'll have to try to track down the researcher who did the study.

Not to pick on Greg Mankiw, but is he saying that it was a good thing that the stock market rose 50% during the two years he was in the Council of Economic Advisors? I'm not at all trying to blame him for what happened, but in retrospect wouldn't one want to regret the big climb in the stock market that preceded the fall? This is completely out of my area of expertise so I defer to others on this. It looks like Mankiw is making a joke of some kind but I don't know enough about the background to really understand the point. (Probably this is the same reaction that many readers to this blog get when I make a statistics joke.)

Mankiw calculates that McCain's tax plan would tax him at a marginal rate of 83%, while Obama's would tax his marginal dollar at 93%. He concludes:

The bottom line: If you are one of those people out there trying to induce me [Mankiw] to do some work for you, there is a good chance I will turn you down. And the likelihood will go up after President Obama puts his tax plan in place. I expect to spend more time playing with my kids. They will be poorer when they grow up, but perhaps they will have a few more happy memories.

I don't quite follow Mankiw's reasoning on the marginal tax rates, except I do get his point that his marginal dollars are all ultimately going to his kids--none of it will be spent in his lifetime, so in that sense he's talking about different varieties of an estate tax.

I'm more interested in the decision implications.

To start with, it does sound like Mankiw's kids are already well provided for, and, although I'm sure they'd disagree with me on this, it's not clear that they would benefit from having more money in the bank when their parents are gone.

So, from that point of view, the question is why Mankiw isn't already spending more time playing with his kids? I can't speak for him, but for me, I have to say that it can be fun to work (or even to write blog entries). But, more than that, I feel a sense of obligation to get things done. At some level, getting paid is part of the motivation, but in any particular example I'm not quite sure how it fits in. I do lots of work things that pay me $0; I think they're important, so I do them.

On the other hand, if I really, really didn't need the money, I could set my salary to $0 and spend the money on extra postdocs. That would be pretty cool but I can't really live on $0 and keep my current lifestyle.

For Mankiw, I'm not sure; maybe he makes enough from his textbooks that he doesn't need much of his academic salary and could possibly do more by converting it into postdocs and research assistants. Or maybe he already has more research assistants than he knows what to do with; I don't know. But his division of waking hours into "working" or "playing with kids" is, I would guess, not very sensitive to the marginal tax rate.

There are two aspects of a presidential election that can be predicted: the national popular vote and the relative positions of the states. The national popular vote can be forecasted months ahead of time given the economy and other predictors. for example using Doug Hibbs's model:

.

(As I wrote a few months ago, "the incumbent party sometimes loses but they never have gotten really slaughtered. In periods of low economic growth, the incumbent party can lose, but a 53-47 margin would be typical; you wouldn't expect the challenger to get much more than that.")

The relative positions of the states don't actually change much from election to election:

You can do slightly better by using polls. As Matthew Yglesias puts it, "the large number of public polls on something like a presidential election makes the outcomes quite easy to forecast based on crude measures. What's more, even absent polling, Presidential election outcomes seem to be pretty predictable based on nothing more than macroeconomic variables."

Actually, even the February polls turn out to be pretty good--when combined with previous election results--to pin down the relative positions of the states.

Bayesian combination of state polls and election forecasts

Here's the revised version of my article with Kari Lock in which we forecast the election using Hibbs for the national popular vote, and a weighted average of last election (corrected for incumbency) and the February polls to get the relative positions of the states.

Lots fo fun stuff there, including this prediction (based on February Clinton-McCain and Obama-McCain polls) of which states Clinton or Obama were expected to win in November:

Back when I used to teach at Berkeley, I used to run into non-Bayesian hardliners--the kind of people who would say no to prior information even if it were wrapped nicely, placed on a warm plate, and served with a delicious pile of crisp fries. I don't run into such people much anymore but then recently Matthew Yglesias linked to my mention of economy-based election forecasts. I read the comments, one of which says:

The problem with this sort of analysis is that we're working with very few data points. We've only ever had 50 some odd Presidential elections. 12 in the television era. 2 in the internet era. It's very very hard to generalize anything from a small dataset, and even correlations don't mean much - who's to say that the macroeconomic correlations are any more meaningful than the Redskins game?

I agree with the bit about the small sample size--as I often tell people--95% intervals on the national election outcome don't mean much considering we wouldn't even try to apply a single model to 20 successive elections--but . . . "who's to say that the macroeconomic correlations are any more meaningful than the Redskins game?" ???

The answer is: Everyone can agree that macroeconomic correlations are more meaningful than the Redskins game. For one thing, voters when surveyed say that the economy is important to their voting. They don't say that the Redskins game is important. Also most people don't care about the Redskins. Etc etc. In statistics we call this prior information. Anyway, I'm not trying to pick on a blog commenter--not everyone's an expert in every field of endeavor--it was just funny to see such a pure non-Bayesian in the wild, so to speak.

Somebody asked me for the golf putting data from Don Berry's book, which Deb and I use as an example for nonlinear modeling in this article and our Teaching Statistics book. Here they are:

I went to the webpage of physicist / computer scientist David MacKay and found that he had written a book on energy policy for general audiences. It's basically a physics book where he computes the energy costs of different aspects of our lifestyles and then estimates the potential for getting power from various non-carbon-emitting sources. It's a fun read and I recommend taking a look. I don't know enough to offer any serious endorsement or criticism of his claims, but he presents his reasoning very clearly, which I like. He has lots of graphs, and I view his book as being somewhat in the spirit of Red State, Blue State, as organizing a bunch of information so that the reader is in a better position to make his or her own judgments. (Again, I'm in no position to endorse or criticize MacKay's specific recommendations.)

My main suggestion is that MacKay follow up on one of his suggestions and connect his work to that of advocates on different sides of the issue. He begins his book as follows:

I [MacKay] recently read two books, one by a physicist, and one by an economist. In Out of Gas, Caltech physicist David Goodstein describes an impending energy crisis brought on by The End of the Age of Oil. . . .

In The Skeptical Environmentalist, Bjørn Lomborg paints a completely different picture. "Everything is fine." Indeed, "everything is getting better." Furthermore, "we are not headed for a major energy crisis," and "there is plenty of energy." How could two smart people come to such different conclusions? I had to get to the bottom of this.

This sounded good, and I was looking forward to the resolution. But in all the rest of the book, MacKay never mentioned Goodstein or Lomborg again (except once in a brief aside to say that their books are "full of interesting numbers and back-of-envelope calculations," and once to cite Lomborg's estimate of bird deaths caused by wind turbines)!

This was a letdown. I think MacKay's argument would be stronger if he could loop back and address the arguments of Goodstein, Lomborg, and others.

This is a great opportunity to work in an expanding research group including my collaborator Jennifer Hill. We're working on a bunch of interesting projects and it would be great if this job could be filled with someone with innovative statistical ideas and broad applied interests.

Here's a description:

The NYU Steinhardt School seeks an applied statistician with an active program of research in one or more of the following areas: public health, health policy, behavior/prevention research, human development, and health psychology/behavioral medicine. Specialty areas that are of particular interest to the departments include: (1) epidemiological network modeling; (2) latent class models and clustering; (3) longitudinal data analysis, including survival analysis; (4) randomized experiments and quasi-experiments; (5) spatio-temporal modeling.

Responsibilities: The successful candidate would join a growing applied statistics center to foster cross-disciplinary collaborative research throughout New York University and would play a key role in providing methods training for NYU's two Master's programs in Global and Community Public Health (the Global program is described here: www.nyu.edu/mph). He/she should be prepared to teach introductory and more advanced Biostatistics courses that cover topics such as categorical data analysis, non-parametrics, repeated measures, and research design.

Aleks pointed me to this article by Brendan O'Connor comparing the fields of machine learning and statistics. I don't have much to add, except to say:

1. Healthy competition is fine, but I think it's great to have separate fields of statistics and machine learning. It's like having more space on the supermarket shelf to sell our common product. In general, I'd like to see more statistics, CS, applied math, etc., and less pure math. Pure math is fine but I think it occupies way too many resources and draws way too many students, given what comes out of it.

2. My impression is that computer scientists work on more complicated problems and bigger datasets, in general, than statisticians do. That's fine--each of us has our niche--but I think our machine learning cousins deserve our respect for being able to make progress on hard problems.

Bill Easterly is speaking in our seminar this Thursday--the title is "Free the Poor! Ending Global Poverty from the Bottom Up," and it will be in 711 IAB from 11-12:30 (it's open to all)--and I thought I'd prepare by reading his recent article in the New York Review of Books, a review of "The Bottom Billion: Why the Poorest Countries Are Failing and What Can Be Done About It," by Paul Collier. (I haven't read the Collier book so that puts me on an even footing with most of the others in the audience, I think.)

Easterly has some pretty strong criticism of Collier, setting him up with some quotes that set of alarms to me as a statistician. For example, Collier writes:

Aid is not very effective in inducing a turnaround in a failing state; you have to wait for a political opportunity. When it arises, pour in the technical assistance as quickly as possible to help implement reform. Then, after a few years, start pouring in the money for the government to spend.

and

Security in postconflict societies will normally require an external military presence for a long time. Both sending and recipient governments should expect this presence to last for around a decade, and must commit to it. Much less than a decade and domestic politicians are liable to play a waiting game rather than building the peace.... Much more than a decade and citizens are likely to get restive for foreign troops to leave the country.

These are the kind of precise recommendations that make me suspicious. I mean, sure, I know that real effects are nonlinear and even nonmonotonic--too much of anything is too much, and all that--but I'm just about always skeptical of the claim that this sort of "sweet spot" analysis can really be pulled out from data. (See here--search on "Shepherd"--for an example of a nonlinear finding I didn't believe, in that case on the deterrent effect of the death penalty.)

Easterly continues with some more general discussion of the role of statistics in making policy decisions. Here's Easterly:

Alas, as a social scientist using methods similar to Collier's in my research, I [Easterly] am painfully aware of the limitations of our science. When recommending an action on the basis of a statistical correlation, first of all, one must heed the well-known principle that correlation does not equal causation. . . . [Collier] fails to establish that the measures he recommends will lead to the desired outcomes. In fairness to Collier, it is very difficult to demonstrate causal effects with the kind of data we have available to us on civil wars and failing states. . . .

Of course, governments take many actions even when social scientists are unable to establish that such actions will cause certain desirable outcomes. Presumably they use some kind of political judgment that is not based on statistical analysis.

I'm not so sure about this. Bill James once said something along the lines of: The alternative to "good statistics" is not "no statistics," it's "bad statistics." (His example was something like somebody who critizicized On Base Percentage because it counted a walk the same as a hit, but then said critic ended up relying on Batting Average, which has lots more serious problems.)

Similarly, if we don't have a fully credible causal inference, I'd still like to see some serious observational analysis.

An example of this is Page Fortna's work on the effectiveness of peacekeeping. I might be garbling the details here, but my recollection is that she compared outcomes in countries with civil war, comparing countries with and without international peacekeeping. The countries with peacekeeping did better. A potential objection arose, which was that perhaps peacekeepers chose the easy cases--maybe the really bad civil wars were so dangerous that peacekeepers didn't go to those places. So she controlled for how bad off the country before the peacekeeping-or-no-peacekeeping decision was made, using some objective measures of badness--and she found that peackeeping was actually more likely to be done in the tough cases, and after controlling for how bad off the country was, peacekeeping looked even more effective. Here's the graph (which I made from Page's data when she spoke in our seminar a few years ago):

The red points are countries with peacekeeping, the black points are countries without, and the y-axis represents how long the countries have gone so far without a return of the conflict. In an example like this, the real research effort comes from putting together the dataset; that said, I think this graph is helpful, and it also illustrates the middle range between anecdotes of individual cases, on one hand, and ironclad causal inference on the other. I have no idea where Collier's research fits in here but I'd like to keep a place in policy analysis for this sort of thing. I'd hate to send the message that, if all we have is correlation, that the observational statistics must be completely ignored.

P.S. Easterly also writes, "To a social scientist, the world is a big laboratory." I would alter this to "observatory." To me, a laboratory evokes images of test tubes and scientific experiments, whereas for me (and, I think, for most quantitative social scientists), the world is something that we gather data on and learn about rather than directly manipulate. Laboratory and field experiments are increasingly popular, sure, but I still don't see them as prototypical social science.

P.P.S. See here for some comments from Chris Blattman, who knows more about this stuff than I do.

Chris Zorn pointed me to this news article by Julie Rehmeyer.

Given the context, the graph is impressive and important. But given what we know today, it would've been even better as a line plot. (Rehmeyer suggests a bar graph but I assume that's just because she doesn't know about line graphs; see here for a simple example.)

Following a suggestion of Hober Short, I replotted the voting-by-age data with time on the x-axis. I also took this opportunity to go back to 1988 (the earliest for which I could effortlessly pull exit poll data off the web). Here's what happened:

Bill Clinton in 1992 and 1996 did well among young voters--like Barack Obama, he was a young Democrat facing older Republican opponents--but not so well as Obama in 2008.

As in many political settings, the largest gains in the graph come from incorporating additional data--in this case, the comparison of 2008 with earlier years, the comparison on young voters with those of other ages, and the comparison of the three other age groups with each other (with the lack of variation in this last comparison being a motivation to focus on trends among young voters in particular).

I majored in math and physics in college. I knew all along that math was a dead end for me (in high school, I was in the U.S. Math Olympiad program and learned that there were kids who were much better at math than me. My impression of math was that in every century there are the Cauchys and Fouriers who do the real stuff and then a bunch of other guys who pretty much spin their wheels--and I didn't want to be one of these other guys) but physics was cooler. (I ended up deciding that I didn't understand physics well enough to continue with it, but that's another story.)

One of the requirements for the physics degree was to do an undergraduate thesis. There was a booklet listing the faculty who would take research assistants. My junior year I went to a few of these physics professors but they told me that I should wait until my senior year when I was ready to do a thesis. Then as a senior I went back to these places and was told that they only took students who'd been working with them earlier. (To be fair, though, maybe I wasn't trying so hard. I worked in physics labs in summer jobs all through high school and college, and while it was interesting and I learned a lot--among other things, I became an expert at programming the finite element method for thermal analysis--it never really seemed to be me.)

So I broadened my search and found a professor of political science who accepted undergraduates and did research in game theory. (Although the undergraduate physics degree required a thesis, it did not have to be in physics. And I'd taken a couple of political science classes already.) Game theory sounded interesting so I went to Prof. Alker's office and he told me about a recent book called The Evolution of Cooperation by a political scientist named Robert Axelrod. Alker told me to buy the book, read it, and come back to him with some research ideas. I did so, and we had our next discussion a week or two later.

My ideas were a bunch of pretty technical game theoretic questions involving different prisoner's dilemma strategies, and Alker, to his eternal credit, pointed me in a better direction. Axelrod had a chapter on First World War trench warfare: did his model make sense there? Alker pointed me to a book by Tony Ashworth--Axelrod's main source--and also the book, Men Against Fire, by S. L. A. Marshall (see here for a recent overview), and The Face of Battle, John Keegan's recent historical overview of combat.

These were great leads. Over the next few weeks I read these books and realized that (at least to me) Axelrod's application of game theory to First World War trenches didn't hold up. Alker felt that criticism wasn't enough and pointed me toward recent political science literature on cooperative games, which allowed me to place the trench warfare example in this more general framework.

I liked my undergraduate thesis but never thought to submit it for publication for another fifteen years or so--too bad, I think it could've been influential back in 1986! After a couple of submissions to different journals, I didn't have the energy to try to revise further but luckily had a convenient opportunity to put the article in a book I was writing and editing (it's coming out next year, under the title A Quantitative Tour of the Social Sciences, edited by Jeronimo Cortina and myself). To keep it clean, I took out the alternative models and just focused the chapter on an exposition and criticism of Axelrod's model. The book chapter is fun because I also quoted from, and responded to, some of the referee reports I got from the journals. I also posted the article on my website.

(Just to be clear: I'm a big fan of Axelrod's book, which has been rightfully influential . Even if he overstretched the applicability of his model in one case, this isn't meant as any kind of devastating criticism of his book as a whole.)

Anyway, a year or so ago I got an email from the editor of an Italian sociology journal saying that he liked my article and could he publish it in his journal, QA-Rivista dell'Associazione Rossi-Doria (whatever that means)? I immediately responded yes, as I had no plans to try to go through the submission-and-revision process. And so the article duly appeared. It has a nice title: Methodology as Ideology.

The funny thing is that I thought it was so cool that the journal wanted to publish my paper. No effort needed on my part! On the other hand, why was I so happy to give them my work for free? I mean, suppose I ran into somebody on the street and said, "I really like your bike--could I have it?" Would I say Yeah, sure? But with intellectual property, I'm so eager to give it away! Sure, the article was already posted on the web, but allowing someone else to publish it is slightly different.

P.S. When undergraduates want to work with me, I just about always say yes. Not that it always works out--often I'll give a project to a student and then never hear back from him or her--but I'll give them the chance.