April 2009 Archives

David Fox writes:

As a 'classically' trained statistician who works on 'real' problems (mainly environmental ones) I have come to appreciate the utility and benefits of working within a Bayesian framework. I would not classify myself as a 'convert' but prefer to have an array of statistical tools from which I can select the most appropriate one for the job at hand. As they say - if all you've got is a hammer, then the whole world's a nail! On the issue of choice of priors, I believe this is an absolute strength in the evaluation and setting of environmental regulatory limits. In situations characterized by high levels of data paucity but rich with expert knowledge (albeit diverse), why would you choose to ignore the latter?

However, I should get to the real purpose of this email. A rather fierce debate has been taking place among academics in our departments of Botany and Mathematics and Statistics about the use of a 'new' form of decision-making under extreme uncertainty. It is called Info-Gap (short for information gap) Theory and owes its existence to Prof. Yakov Ben-Haim at Technion in Israel (Ben-Haim 2006). Yakov is well known to the aforementioned academics - he visits here regularly and has done a remarkably good job at 'selling' his product - to the extent that some staff and students in our Botany department and The Australian Centre of Excellence in Risk Analysis (http://www.acera.unimelb.edu.au) have enthusiastically (and some would say, blindly) embraced this 'new' paradigm for decision-making under extreme uncertainty. I must plead mea culpa, having been swept up in the initial enthusiasm and published a couple of papers which use info-gap. However, I have a growing unease that IG is not 'new' but in fact a variant of existing methodologies." While not wishing to draw you into our local debate, I was wondering if you have ever heard of info-gap theory and if you have, do you have an opinion? Prof. Ben-Haim has recently launched his own web site (http://www.info-gap.com) presumably in response to the 'hi-jacking' of the Wikepedia entry (http://en.wikipedia.org/wiki/Info-gap_decision_theory) by IG's most strident local critic, Moshe Sniedovich. Sniedovich has also established a web site (http://info-gap.moshe-online.com/) and a quick look will demonstrate the ferocity of the debate.

Just today, the following paragraph in a paper I was reading [Hickey, G.L., Craig, P.S., and Hart, A. (2009) On the application of loss functions in determining assessment factors for ecological risk. Ecotoxicology and Environmental Safety, 72, 293-300] caught my attention:

"There do exist other forms of risk measurement. However, by a very well-known theorem of Wald (1950), any admissible decision rule is a Bayes rule with respect to some prior distribution (possibly an improper prior distribution), whereby admissibility is defined to mean that no other decision rule dominates it in terms of risk. It is therefore argued by many, for example, Bernardo and Smith (2000) that it is pointless to work in decision-theory outside the Bayesian framework".

This accords with my own gut feeling that IG Theory is in fact a Bayes Rule with a non-informative prior.

My reply: I had never heard about Dr. Ben-Haim or his methods before receiving this email. I checked out the links but couldn't really see the point in this approach. The mathematics looked complicated and appeared to be a distraction from the more important goals of modeling the decision problems directly.

For some of my thoughts on Bayesian decision analysis, see chapter 22 of Bayesian Data Analysis (second edition). Bayesian decision analysis is a lot more flexible than people realize, I think, especially when used in the context of hierarchical modeling. See here for a brief discussion of my idea of "institutional decision analysis" and here for an example of Bayesian decision analysis in action.

In my article on the boxer, the wrestler, and the coin flip, I discuss some fundamental difficulties with Bayesian robusness and similar approaches.

Finally, I don't know that I'd agree with the statement that it's "pointless" to work in non-Bayesian decision theory. For me, I've found the Bayesian approach to do the job, but I can imagine there are settings where other methods can be useful. I'm not, however, a fan of those 1950's-style alternatives such as "minimax regret" and all the reat. I offer no comment on Info-Gap since I didn't put in the effort to try to understand exactly what it is.

Students Plotted to Kill Springbrook High Staff

A Quantitative Tour of the Social Sciences has just come out. The book is edited by Jeronimo Cortina and myself, and it is intended to give the reader a sense of how research is done in different areas of social science. It is not a book of statistical methods, nor is it that sort of academic book that has a zillion little chapters of things that people submitted because they couldn't get them accepted into journals. Rather, it is a set of in-depth examples and discussions of social science research from a variety of perspectives.

I think the book should be especially useful for courses for graduate students or advanced undergraduates in social science, who typically aren't familiar with the way people think in neighboring fields. For example, a political science student might know a little bit about economics but nothing about psychology. Or a sociology student might not know much about historical data collection. And so forth.

Here's the table of contents:

I. Models and Methods in the Social Sciences (Andrew Gelman)
1. Introduction and overview
2. What's in a number? Definitions of fairness and political representation
3. The allure and limitations of mathematical modeling: Game theory and trench warfare

II. History (Herbert Klein and Charles Stockley)
1. Historical background of quantitative social science
2. Sources of historical data
3. Historical perspectives on international exchange rates
4. Historical data and demography in Europe and the Americas

III. Economics (Richard Clarida and Marta Noguer)
1. Learning from economic data
2. Econometric forecasting and the flow of information
3. Two studies of interest rates and monetary policy

IV. Sociology (Seymour Spilerman and Emanuele Gerratana)
1. Models and theories in sociology
2. Demographic explanations of social disturbances in the 1960s
3. Studying the time series of lynchings in the South
4. Attainment processes in a large organization

V. Political Science (Charles Cameron)
1. What is political science?
2. The politics of Supreme Court nominations: the critical role of the media environment
3. Modeling strategy in congressional hearings

VI. Psychology (E. Tory Higgins, Elke Weber, and Heidi Grant)
1. Formulating and testing theories in psychology
2. Some theories in cognitive and social psychology
3. Signal detection theory and models for tradeoffs in decision making

VII. To Treat or Not to Treat: Causal Inference in the Social Sciences (Jeronimo Cortina)
1. The potential-outcomes model of causation; propensity scores
2. Some statistical tools for causal inference with observational data
3. Migration and Solidarity

The cover is an adaptation of this image that was sent to us from Chris Albon last year after we asked for cover ideas on the blog. Thanks, Chris. You're getting a free copy!

There's a lot of good stuff here (from Thomas Lumley). It's all classical stuff--no small-area estimation, no Mister P, etc.--but the classical stuff is still pretty useful. The "survey" package for R looks pretty good; in particular, it allows you to specify the survey design, which is a big step beyond simply specifying survey weights.

I'd also like to recommend Sharon Lohr's book from 1999. When's the second edition coming out?

Mark Johnson writes:

Suppose we want to build a hierarchical model where the lowest level are multinomials and the next level up are Dirichlets (the conjugate prior to multinomials). What distributions should we use for the level above that? I'm not a statistician, but aren't Dirichlet distributions exponential models? If so, Dirichlets should have conjugate priors, which might be useful for the next level up in hierarchical models. I've never heard anyone talk about conjugate priors for Dirichlets, but perhaps I'm not listening to the right people. Do you have any other suggestions for priors for Dirichlets?

My reply: I'm not sure, but I agree that there should be something reasonable here. I've personally never had much success with Dirichlets. When modeling positive parameters that are constrained to sum to 1, I prefer to use a redundantly-parameterized normal distribution. For example, if theta_1 + theta_2 + theta_3 + theta_4 = 1, with all thetas constrained to be positive, I'll use the model,

theta_j = exp(phi_j)/(exp(phi_1)+exp(phi_2)+exp(phi_3)+exp(phi_4), for j=1,2,3,4.

If you give each of the phi_j's a normal distribution, this is a more flexible model than the Dirichlet: it has 8 parameters (four means and four variances). Well, actually 7, because the means are only identified up to an arbitrary additive constant.

Frederic Bois and I used this distribution for a problem in toxicology, modeling blood flows within different compartments of the body--these were constrained to sum to total blood flow.

In this article, I used a fun stochastic approximation trick to compute reasonable values for the mean and variance parameters for the phi's.

P.S. Dana Kelly points to this article on the topic by Aitchison and Shen from 1980.

Another talk in NYU's Statistics in Society series. It looks interesting:

Corey Yanofsky pointed me to this:

This is a fun one--it has so many flaws, I hardly know where to begin.

This one is also interesting, in that they seem to have decided retroactively to blame the Democrats as of January, 2007. Fair enough, I guess. In retrospect, I'm surprised the Congressional Republicans and the McCain campaign didn't try this tactic in the 2008 campaign--to say that, sure, the economy is a disaster but it's the fault of the Congressional Democrats. Maybe the "economy is fundamentally strong" pitch wasn't such a good idea.

It's hard to see that arguments about the national debt will convince many people right now. But, stepping back a bit, the role of the Republican campaign right now can't really be to change policy or even to convince a majority of Americans that Obama and the Democrats are doing things wrong. Rather, the short-term goal has got to be to keep up morale among the base. I don't know that the deficit is a great talking point there either, but maybe it is; I imagine they've done some polling.

P.S. Yes, I'm happy to comment on silly Democratic graphs too--just send 'em in!

John Sides has an excellent discussion of a recent article by Jeremy Bailensen, Shanto Iyengar, Nick Yee, and Nathan Collins, who find that survey respondents are slightly more likely to have positive feelings about a candidate if they are shown a picture of the candidate that has been "morphed" to look more like the respondent him or herself.

Sides discusses how and when these effects might actually be important. As he puts it, "to broaden their results from the impact of facial similarity to that of physical appearance more generally: if 'faces matter,' they are likely to matter more in precisely the kinds of races where, ironically, voters are less likely even to know what the candidates look like in the first place."

To me, the most interesting implications involve ethnic and racial politics. I hope the Bailensen et al. results get lots of attention from perspectives similar to Sides, where the value of the study is recognized without it being presented with a spin of: "voters are silly and only care about appearances."

When demonstrating his Alice in Wonderland example, Brad Paley showed how the words in the center of the display were located by grabbing a word with his mouse, clicking to show its connections with places in the text, and then moving the word, showing the lines of the connections stretching, then letting go to show the word bounce back. The image of the word connected to the places using rubber bands was clear.

What I want to know is, can somebody rig a robot arm to do this so I could feel the pull? Imagine a robot arm that can be moved within a 30cm x 40cm box. You could use this to feel the springiness of the connections in Brad's diagram; the idea is that you'd say a word (for example, "Alice"); the pointer of the robot arm would move to the position of the word in the display, and then you could--with effort--move the robot arm away from this place. When you let go or relax your grip, the pulling of the virtual rubber bands would return the arm back to its original place, and you could feel the strength of the pull.

The arm could also be used to feel a curve (for example, a nonlinear regression or spline, or a mathematical function such the logarithm or the normal distribution curve), as follows: the arm would start at one end of the curve and the user could grip it and move it along, with the motion physically constrained so that the arm would trace the curve.

In displaying several curves--for example, level curves indicating indifference curves - the arm could start on one of the curves and be programmed to stay on that curve, unless the user pushes hard, in which case there would be resistance during which the arm moves between curves. It would then lock into the next curve, which the user could again trace until he or she pushes hard enough to get the arm unstuck again.

More generally, the robot arm could be used for exploring three-dimensional functions such as physical potentials, likelihood functions, and probability densities. From any point in the two-dimensional box, a "gravitational force" would pull the arm toward a local minimum (or, for a likelihood or probability density, the maximum) of the function. Then with moderate effort the user could move the arm around and, by feeling the resistance, get a sense of gradients, minima, and constraints.

(for example, a nonlinear regression or spline, or a mathematical function such the logarithm or the normal distribution curve), as follows: the arm would start at one end of the curve and the user could grip it and move it along, with the motion physically constrained so that the arm would trace the curve.

In displaying several curves--for example, level curves indicating indifference curves - the arm could start on one of the curves and be programmed to stay on that curve, unless the user pushes hard, in which case there would be resistance during which the arm moves between curves. It would then lock into the next curve, which the user could again trace until he or she pushes hard enough to get the arm unstuck again.

More generally, the robot arm could be used for exploring three-dimensional functions such as physical potentials, likelihood functions, and probability densities. From any point in the two-dimensional box, a "gravitational force" would pull the arm toward a local minimum (or, for a likelihood or probability density, the maximum) of the function. Then with moderate effort the user could move the arm around and, by feeling the resistance, get a sense of gradients, minima, and constraints.

An example of how I'd like to use the robot arm together with a visual graph of data and model fit

I was originally thinking of this as a statistical tool for blind people, but I'd actually like to have one of these myself, for example to understand the sensitivity of a model fit to changes in parameter values. I'm thinking of twp graphs next to each other: a graph of parameter space and a graph of data with fitted curves. The robot arm would be pointed to the posterior mode or maximum likelihood estimate in parameter space. As I moved the robot arm around, I'd feel resistance--it would be difficult to move far in parameter space without feeling the increase in -log(density)--and at the same time the curve would be moving in the fitted curve + data plot. The muscular resistance information on one graph and the visual information on the other graph would together give me a sense of what aspects of the data are determining the model fit.

Here's an example of what it might look like:

I'd also like to be able to use the robot arm to pull on the fitted curve and feel the resistance as I move it away from the data.

P.S. Here's the R code for the above graphs:

Ian Ayers refers to the research by Brett Pelham, Matthew Mirenberg, and John Jones that people are likely to have names that are related to their occupations, places of birth, etc. Pelham et al. write:

Taken together, the names Jerry and Walter have an average frequency of 0.416%, compared with a frequency of 0.415% for the name Dennis. Thus, if people named Dennis are more likely than people named Jerry or Walter to work as dentists, this would suggest that people named Dennis do, in fact, gravitate toward dentistry. A nationwide search focusing on each of these specific first names revealed 482 dentists named Dennis, 257 dentists named Walter, and 270 dentists named Jerry.

In his blog, Ayres referred to this finding but wrote:

To be honest, I [Ayres] am not fully persuaded that either of these results is true.

I think that Ayres is saying this because the effect sounds so large: Even if there really were something going on, could it really explain the difference between 482 and 257, nearly a factor of 2?

Let me repost a simple conditional probability calculation that might put Ayres's mind at ease:

There were 482 dentists in the United States named Dennis, as compared to only about 260 that would be expected simply from the frequencies of Dennises and dentists in the population. On the other hand, the 222 "extra" Dennis dentists are only a very small fraction of the 620,000 Dennises in the country; this name pattern thus is striking but represents a small total effect. If we assume that 222 of these Dennises are "extra" dentists--choosing the profession just based on their name--that gives 221/620000= .035% of Dennises choosing their career using this rule. I can certainly believe that the naming effect could be as high as .035%.

What percentage of people pick their job based on their name?

And here is my quick calculation that approximately 1% of Americans choose their career based on their first name:

Medical researcher Nicholas Christakis and political scientist James Fowler sent me an advance copy of their book, "Connected: The Surprising Power of Social Networks and How They Shape Our Lives." Christakis and Fowler are best known for their work connecting social networks and epidemiology, in particular the fact that obese people are more likely to have obese friends. As one wag put it, they find that obesity is environmental and voting is genetic. I guess that sort of interpretation is the inevitable outcome of man-bites-dog reporting, with the real story being that obesity is more determined by social behavior than we might have thought, while voting behavior is more tied to genes than we might have thought.

Anyway, I like their new book a lot. They bring in many different research findings. It's a popular science book but with a much higher meat-to-filler ratio. Considering myself as the ideal reader, I wish they had more discussion of methodology, of the strengths and potential weaknesses of each of the findings they cite. I understand why they didn't want to clutter a popular book with such discussion, but in retrospect I wish we'd put in more methods talk within our own Red State, Blue State book. My impression from talking with people is that (a) they respect open discussions of strengths and weaknesses of an argument, and (b) many people find methods fun and enjoy chewing on issues such as representativeness of sampling, etc. In our own book, I think we tried to hard to be reader friendly and wish we'd laid more of the struggle out there for the reader to see. On the other hand, their book has many advantages over mine, being from a major publisher and also covering so many different topics that there's something for everyone. It's actually much more general in scope than its title indicated. It's really about all of social science. I'm not sure how a reader who isn't familiar with all this work would think of the book, but I enjoyed seeing all this stuff in one place.

Henry thinks that, if you a review an article for an academic journal, they should email you a copy of their letter deciding what to with the manuscript. I review lots and lots of articles and occasionally get these letters, which I always immediately delete. I guess it's not really a bad thing when they send these emails--they're easy enough to remove--but I certainly don't see why it would upset someone not to be bothered by them.

I have noticed a big problem recently, though: with electronic publishing, I see fewer and fewer hardcopy journals lying around, and this removes a key way for me to keep up with what's going on in statistics.

A couple days ago I asked what was it that Brad Paley did to get such active participation in his seminar. I can get active participation, but it takes work: I have to ask students to work in pairs to prepare questions, etc. Brad didn't need such mechanical tricks; the participation came naturally.

So what did he do? Brad writes:

I'm glad my "seminar style" presentation worked for you--sometimes I'm a little worried that people won't follow the digressions, but I think the material holds together well in itself and I try hard to shape/steer the digressions back to the "main topic." BTW: I write "main topic" in quotes because the outline of my talks is typically pretty strategically defined, and I let people steer me towards the things they want to get from me, figuring that what they ask will always stick better in their minds and be a better "gift" from me to them than if I just plowed through some pre-arranged plan. (It's important that the material--really anything I do--is a gift to the audience rather than a plea for attention; and when I realize it's about them, not me, there's less pressure to push my point.)

Peter Selb writes:

I was mentioned on ESPN (sort of). Take that, Hal Stern!

Aurélien Madouasse writes:

One of the things Brad Paley talked about the other day was the computer program he used to make a visualization of the text of Alice in Wonderland [link fixed]. (Click on the "Alice in Wonderland" link; it's really cool.)

My first question when I saw this was, why is the book presented as a circle rather than a line? The circle places the end of the book at the same place as the beginning. There are some reasons this might make sense--after all, Alice wakes up from her dream at the very end of the book, returning to where she was at the start--but, overall, I don't see the circularity making sense. I asked Brad during his talk, but he did not have time to respond (too many questions were being asked, a problem I'd love to have at my own talks!). He indicated that he did have a good reason, though, so if he lets me know I'll report it here.

People asked what was the point of the TextArc display (other than it looking pretty), and Brad gave a bunch of examples of what the plot showed. In some way it was similar to some of my statistical research efforts, in that the results were impressive but ended up confirming things that made sense and that, ultimately, we already knew. In my case, my colleagues and I found that American Indians are not randomly distributed in the social network; in Brad's case, he found that Alice is a central character in Alice in Wonderland, that the words "Mock" and "Turtle" go together, and so forth. (See here for more.)

When pressed further, Brad justified TextArc as a souped-up index. This made a lot of sense to me: his graph tells you lots of information that's not in a conventional index and also allows you to map straight back to the original text. I agree that it's silly to criticize the program for what it doesn't do. It's an automatic program and does a lot. I'm also impressed by any program written more than 5 years ago that still works!

Anyway, one of Brad's remarks about using this tool to understand text made me think that there are two kinds of books:
1. Books that you want to read straight through, from beginning to end.
2. Books that you use for reference, flipping through and looking for what you need.
The horrible thing is that I write all my books as if they will be read from beginning to end, but I'm pretty sure most people read them as reference books. For most people--even most statisticians--reading Bayesian Data Analysis from beginning to end would be like me reading the instruction manual for my washing machine. I pick up the instruction manual when I need it, and then I look for what I need.

Anyway, I thought this might be relevant to TextArc and similar projects. Maybe Alice in Wonderland is not the best example; it might make more sense to use TextArc for a book such as Bayesian Data Analysis that has a sequence but is primarily used for reference. (I went to the TextArc site but can't find the program; at least, there's no easy way to feed in a book and have it produce the TextArc picture.)

I recently read Nicholas Chistakis and James Fowler's Connected, and now everything I see makes me think of social networks.

For example, Richard Florida links to a research article by Bart Bronnenberg, Sanjay Dhar, and Jean‐Pierre Dubé, who write:

We [Bronnenberg et al.] document evidence of a persistent "early entry" advantage for brands in 34 consumer packaged goods industries across the 50 largest U.S. cities. Current market shares are higher in markets closest to a brand's historic city of origin than in those farthest. For six industries, we know the order of entry among the top brands in each of the markets. We find an early entry effect on a brand's current market share and perceived quality across U.S. cities. The magnitude of this effect typically drives the rank order of market shares and perceived quality levels across cities.

I haven't read the article, but assuming it's findings are correct, could some of this be the effect of employees and investors in the company, as well as local pride? I doubt Heinz Ketchup currently employs a lot of people in the Pittsburgh area, but over the years it must add up to a lot of people. Then add in their friends and relatives, along with people who get business from Heinz (suppliers and the like), and that's a whole bunch of Pittsburghers with some connection to Heinz.

The social network bit is the idea that the employees and the like are multiplied by their friends. Beyond this, of course, people are creatures of habits, tastes can get established young, and so forth.

Also, Heinz ketchup is something that anyone can buy. The very fact that it's (a) substitutable with other items and (b) just different enough to be distinguishable (it doesn't taste _exactly_ like other ketchups, it's not a pure commodity), might make it particularly susceptible to this sort of effect. It may be no coincidence that Bronnenberg et al. found this effect in the area of low-cost packaged foods.

Nathan Yau makes some good points in response to my belated comments on his "5 Best Data Visualization Projects of the Year."

First off, I'd like to apologize for saying the projects "suck," That was just rude. Would I like it if somebody said that the examples in Bayesian Data Analysis "suck" because they're not completely realistic, or if somebody said that the demos in Teaching Statistics "suck" because they're not tied closely enough to the lecture material? A better thing for me to say would've been: "I don't particularly like these as data displays, but I'm impressed by the effort that went into them, and I'm glad to see these sort of data-based displays getting a broad audience."

In the interest of constructive discussion, I'd like to make a few points.

The political website Talking Points Memo is featuring a discussion of Red State, Blue State this week. The discussants so far have included software developer / political activist Aaron Swartz, historian Eric Rauchway, political scientist Nolan McCarty, journalist Steve Sailer, and Jeronimo and myself. It's interesting to see all these different ways of looking at things.

As Swartz perceptively points out, our book doesn't really have much of a persepective--it's more of a guided data dump--so there aren't really a lot of things for people to argue about. It's not like I'm George Lakoff saying that the Democrats should do X, and other people can say, no, they should do Y. Nonetheless, the discussants manage to bring up some interesting points.

I led off the discussion with this brief summary of the book and the questions it left unresolved.

My recent post, responding to the discussions so far, is here. I like what I wrote, but it's a little ridiculous that I spent 2 hours on it, thus violating the academic norm to spend so much time on something that's not a peer-reviewed article. All I can say is, I hope some people read it! Just to be on the safe side, I'll repeat it here:

Click over and check it out.

Via epc I came across Jonathan Soma's Triptrop NYC, which practically in real time estimates how long it's going to take to travel from a location in NYC by subway to any other location, and paints this graphically as an overlay on a map. Here's Manhattan, starting from Columbia's Statistics Department:

The walking distance is 3mph. Distances in number of minutes are color-coded and above the map. Jonathan says he used SciPy and curve fitting, along with a precalculated database of over 120000 distances between locations.

There are other "time travel" tools for London and UK, but Triptrop NYC is the first one I've come across that allows you to enter your own precise location. As for similar visualizations, here's my post on housing prices in New York.

Programmer/designer W. Bradford Paley spoke yesterday for the data visualization group here at Columbia. He gave an amazing talk, one of the best I've ever seen. One reason I say this is that about half the talk was devoted to an application he built for Wall Street trading--something I just couldn't care less about, it's hard for me to imagine a topic I'm less interested in--and, even so, I liked the talk a lot.

The seminar participants--a mixture of architects, computer scientists, and some other people, including a psychologist and even a statistician--had lively discussion throughout. In fact, there was so much going on, that I'll spread my comments through several blog entries over the next few days.

Right now I want to talk about Paley's speaking style, which was great in so many ways, but what really got to me was how he managed to get so many questions and comments from the audience---so much that people had to ask him to stop taking questions so he could move forward with the material. This was amazing. When I speak, I always struggle to get audience participation. Usually I get a few questions at the end, but not this kind of barrage all the way through.

What can I do to involve the audience more? I've always thought I need more "hooks" but have not been sure how to do it. After seeing Paley's talk, my new idea is to devote more of my talks to process. I typically present results without a lot of detail on how I got there. But maybe it would be better to talk more about what I did. At least, that worked for Paley.

The funny thing is that I love answering questions, and I think I'm good at it. That's one reason I get so frustrated that I don't get more questions when I speak. People typically think my talks are entertaining, informative, and thought provoking--at least, that's what they tell me--but I'm lacking the hooks that draw people in.

P.S. More here on Paley's talk.

In a long review of David Boyd Haycock's "Mortal Coil: A Short History of Living Longer," Steven Shapin discusses historical and recent proposals for extending the human lifespan. Shapin's article seems off to me: he just seems to spend too much time mocking the idea of extending life. He keeps bringing up silly examples such as the biblical Methuselah, Mel Brooks's 2000 year old man, and Old Tom Parr, who lived in the 1600s and claimed to have lived 150 years old. (Shapin didn't even need to go back that far; I remember as a child reading of a bunch of Russians who claimed to have lived to about 150--as I recall, they ate a lot of yoghurt.)

This is all fine--after all, Shapin's a historian and is reviewing a history book--but he seems a bit too eager to laugh at modern life-extenders such as Roy Walford, who promoted the caloric restriction diet but died at age 79. Connecting to the Bible and even Old Tom Parr is fine, but why does Shapin keep bringing them up in his review? Not to mention bringing up the "Groundhog Day worry about endless boredom . . . the meaninglessness of life in a world without death . . ." I mean, talk about weak arguments in favor of mortality!

My guess is that to Shapin--as to me--the potential of much longer life is scary. I'd love to live to 150 or beyond (I think); certainly I'm not happy about the idea that my life is more than half over!--but, still, there's something scary here, and not just because of issues such as environmental devastation, global inequities, and so forth. I think the scary thing is: What if the calorie-restricters and vitamin-poppers are right? What if we could live to 150 if we only lived right? Then when we die peacefully in our beds at 80, we can be torn up about the 70 years that we're going to miss. I mean, who really wants this guy (with his "private blog" and all the rest) to be right? Far better to laugh it off or just not think about it. That's what I do.

P.S. I was also surprised that Shapin didn't discuss the theory (which I first read in Plagues and Peoples, I believe) that premodern hunter-gatherers lived healthier lives than those in agricultural societies, at least until recently. This would relate the historical stories of the ancients having long lives.

James Heckman recently posted this article, which is based on a paper from 1980. (This sort of thing happens; for example, I just published an article based on work from 1986.) Heckman's tongue-in-cheek article begins:

This paper uses data available from the National Opinion Research Center's (NORC) survey on religious attitudes and powerful statistical methods to evaluate the effect of prayer on the attitude of God toward human beings.

He sets up a model for the intensity of prayer, given its effectiveness. The key assumption is as follows:

Accept on faith that the conditional density of x [the intensity of prayer in the population] given y [God's attitude arrayed on a scale ranging from 0 to 1] is of the form g(x|y) = a(y) exp(xy).

That is, the higher y is, the more prayer we'd see, which makes sense. (Heckman labels the function a(y) as "unknown," but, unless I'm missing something, a(y) is a normalizing constant that can be calculated in closed form by integrating exp(xy) over x. Perhaps this mistake, if it is one, can be caught before the article appears in press.)

Given the reasonable enough model above, Heckman points out that you can differentiate the density of x and learn something about the distribution of y, the effectiveness of prayer.

What does it all mean?

Of course Heckman is joking, but it appears he might be making a more serious point when he comments:

Provided conditional density (1) is assumed, we do not need to observe a variable in order to compute its conditional expectation with respect to another variable whose density can be estimated. For example, one can extend current empirical work in a variety of areas of economics to estimate the effect of income on happiness or the effect of income inequality on democracy.

I don't think this is literally an issue. True, all four of the variables Heckman mentions--income, happiness, income inequality, and democracy--can only be measured with error, but certainly they can be (and are) measured when they are studied empirically.

But I got a little worried that maybe there's something more going on here, some reason I should be giving a little less credence to studies linking economics to psychology and political science. Is Heckman implying that those cross-disciplinary studies have, at bottom, no more foundation than his argument on the effectiveness of prayer?

So I went back to Heckman's article to try to find the flaw in the reasoning. (By "flaw," I don't mean that Heckman was making a mistake; rather, I'm speaking of the hidden logical flaw that makes the reasoning flow, just as in those mathematical arguments where you "prove" 1=0 by means of a series of algebraic expressions that include a division-by-zero.)

Rereading carefully, I found the flaw. I actually think this article would be a good one for a take-home exam in a theoretical statistics class. I'll give the answer below.

A friend writes:

Does this stuff suck? Or am I missing something?

My reply: Yes, I agree. They all suck (for the purpose of data display).

Presidential elections have been closer in the past few decades than they were for most of American history. Here's a list of all the U.S. presidential elections that were decided by less than 1% of the vote:

1880
1884
1888
1960
1968
2000

Funny, huh? Other close ones were 1844 (decided by 1.5% of the vote), 1876 (3%), 1916 (3%), 1976 (2%), 2004 (2.5%).

Four straight close elections in the 1870s-80s, five close elections since 1960, and almost none at any other time.

At the congressional level, however, NY-20 notwithstanding, close elections are less and less likely to be close. Here's a graph showing, for each decade of the past century, the proportion of elections each to the House and Senate that were closer than 51.0%-49.9% of the two-party vote:

Close elections (in percentage terms) have always been more common in the Senate than the House. (We can't take the comparison back before the 1910s, because it was during that decade that direct election for senators was implemented.) In addition, the rate of close elections in the House has declined steadily over the century. If you count closeness in terms of absolute votes rather than percentages, then close elections become even rarer, due to the increasing population. In the first few decades of the twentieth century, there were typically over thirty House seats each election year that were decided by less than 1000 votes; in recent decades it's only been about five in each election year.

The decline of House elections each year is no surprise; as Nate and I discuss, the increasing incumbency advantage has reduced the number of close elections; beyond this, localities are more politically homogeneous than they used to be, and with the nationalization of political parties, it is harder for candidates to tack to the center in individual district races. For the Senate, these factors are present, but to a lesser extent. Gerrymandering sounds like a potential explanation--after all, House districts get redrawn and Senate districts don't--but there's actually no evidence that redistricting reduces competitiveness of legislative districts on the average. (I'll refer you to Ansolabehere and Snyder's 2002 paper for more discussion of these issues.)

P.S. We got our data from various sources, including some old data files that I can't remember who prepared, also from CQ, see for example here.

Slate has a beautiful animated rendering of the job gains/losses over the past 2 years. It would be very difficult to show the trends without animation.

Two other things I like: The quantity circles are so much more informative than using color to paint states: we all know that most job losses are in NY and CA, because they're the biggest! Those circles help control for state population density.

The animation helps control for job gains in the previous period: it hides the cities that are relatively stable, but it nicely shows boom-bust cities (NYC) and stagnation-bust cities (Detroit).

(Via Peter's Twitter.)

Carpenter's 1981 movie Escape from New York was a dystopian vision of the 1997. There was plenty of technology shown, but I was struck by Lee Van Cleef making a call using a two-handed cell phone:

For comparison, an actual 1997 cell phone on the right (via Niels Hoffmeyer).

Science fiction tends to imagine new technological developments along a particular track, but in reality those developments have been made cheaper and widespread. William Gibson has a good quote: "The future is already here - it is just unevenly distributed." It's true that we've lived through 50 years of development guided by consumer economy. To be fair to the science fiction authors, it was military and government that guided development before that, and that's where they got their extrapolations from. Things might change.

Another pattern seen in science fiction is that the negative trends tend to get extrapolated and throwing the world off balance. In doing this, they scare the people and get them to change their ways, often overcompensating and driving the trends in the exact opposite direction. Escape from New York warns about crime running wild, but in reality it was the opposite, it became safer. When people think it's bad, it's never that bad, because they're making things better. When people think it's good, it's never that good, because they're making things worse.

Science fiction authors should know their statistical modeling.

Via infosthetics, I came across a new and very nice web application for data analysis, Verifiable. Among their featured graphs, there's a very nice one displaying the association between politics and religion:

This graph also shows how the often-hated bar charts can be effective. In all, the graphs coming out of Verifiable look like some of the best I've come across. Previously, I've written about ManyEyes, which is quite versatile and allows many data types, and Swivel, which was among the first. Nicely done.

[Several commenters have pointed out (thanks!) that the selection of colors is not good, and that some religions in the list are very similar, or too small to be interesting. When it comes to selecting good colors, I stand by ColorBrewer.]

John Sides and I posted something yesterday on the yawning gap between economic perceptions of Democrats and Republicans, to which we received some interesting comments (see also here and here) that I'd like to reply to.

Adam Taylor writes:

One of the criticisms of Bayesian statistics seems to be that, as generally practiced, it relies on strong distributional assumptions. I'm wondering if it's possible to come up with a posterior distribution on the mean of a bunch of IID samples from an unknown distribution [I think he means "a posterior distribution on an unknown distribution given the mean of an independent sample from that distribution" -- AG], and to do it such that you don't have to make strong assumptions about what the unknown distribution is. I.e. I'm looking for some kind of nonparametric or semi-parametric Bayesian approach to this problem. Does something like this exist?

My reply: Yes, you can do this. See, for example, the articles, "On Bayesian analysis of mixtures with an unknown number of components," by Sylvia Richardson and Peter Green, Journal of the Royal Statistical Society B, 59, 731-792 (1997), and Bayesian density regression, by David Dunson, N. S. Pillai, and J-H. Park, Journal of the Royal Statistical Society B, 69, 163-183.

The short answer it's not trivial to solve the problem in reasonable generality. There are classical methods such as kernel density estimation that are much simpler, but they have problems when sample size is small.

Beyond this, my intuition is that the way to proceed is to think hierarchically. You're almost never really just analyzing a sample from one distribution; realistically, you'll be applying your method repeatedly on related problems. This returns us to the connections between hierarchical modeling and the frequency evaluation of statistical procedures.

Tyler Brough writes:

I am currently a PhD student in Finance. I was explaining my research today to a very senior scholar at a well known eastern business school, who remarked that the econometric methods I am using are "way too complex, and that unless you just do OLS no one will believe it anyway." What are your thoughts about that comment? If I know that the data generating process violates all of the assumptions underlying the classical linear regression model then I have to use more complex methods do I not?

My reply: I agree with the senior scholar that it's important--even necessary--to do the simpler linear regression in addition to the more elaborate model. If a more elaborate model gives a different answer than the least-squares regression, this doesn't necessarily mean that the more elaborate model is wrong, or even too complex--but it does mean that you need to understand what's happening in the transition from the simple to the complex model.

Sometimes I prefer the simpler model and I think the more complex model is giving misleading results and implausible extrapolation.

Other times I like the complicated model, and I put in the effort to understand why it differs from the simple model. Ultimately you have to go to the data and to the underlying problem being studied.

From "the fundamentals are still strong" to "the worst economic crisis since the Great Depression" . . . from "the economy doesn't really needs saving" to "the crash of 2009" . . . from crisis to "glimmers of hope" . . .

What do John McCain, Casey Mulligan, and Barack Obama have in common? Ask Joe Bafumi, Larry Bartels, Alan Gerber, and Greg Huber.

Maybe because I spend so much time working with numbers, I'm as interested in the process of statistics as much as in its outcomes.

A couple months ag I told you about my struggles with the GDP of Russia and how I had inadvertently become entangled with the question.

More recently, I heard about Dick Morris's claim that, "In the last five months, according to the Federal Reserve Board, the money supply in the United States has increased by 271 percent."

271%??? Where did that implausible-looking number come from? Bill Peterson traced this to a 27.1% (note the decimal point) annualized rate of growth in M1 reported on a Federal Reserve website. So it sounded like a simple case of innumeracy (compounded by some partisan foolishness on Morris's part that, I argued here, doesn't do the Republican Party any favors).

But then an anonymous commenter wrote, "Dick Morris was referring to the Federal Reserve Adjusted Monetary Base which did, in fact, grow by a multiple between 2.5x-3x in the five months spanning October, 2008 through March, 2009." The commenter provided a couple of links and concluded,

In short, Dick Morris is right and you are wrong. I believe it is called a cruel irony when you publicly mock someone's intelligence only to find out subsequently that they are correct and you, well, you stepped in it.

I've made mistakes before and so it hardly shocked me that I got something wrong again! Apparently I'd been too quick to believe the Chance News entry that had gotten me started on this. In retrospect, it seemed pretty silly that I was so quick to trust the zero-budget Chance News while disparaging the well respected newspaper, The Hill (where Morris's column had originally appeared).

At this point, I really wanted to see the "271%" so I could issue a full-throated retraction. Unfortunately (or, maybe, fortunately, in the sense that it led to this story), when I followed the links supplied by the commenter, I could not find a 271% growth in the money supply anywhere! Which led back to the original puzzle of where the number came from. Was it simply a mis-transcribing of the 27.1%, or was there something else going on?

I was reminded of a legal consulting project I once worked on, where the statistician on the other side had done an analysis which I had then replicated, getting completely different results. But I didn't feel confident about my own claims until I tracked down how the other guy had done it wrong. It took me 2 hours to get the correct answer myself and to check it to my satisfaction [amusingly, I first typed "statisfaction" there], and 6 hours to get into the problem in enough depth to figure out what the other statistician had done wrong. (I bill by the hour so I remember these time totals. And, believe me, the other guy billed lots lots more than 8 hours to get his wrong answer!)

OK, back to Dick Morris's 271%. The latest insight came from Robert Waldmann, who commented as follows:

I [Waldmann] think I understand how he missed the damned dot, overlooked the concept of "annualised" and decided to call a 271% increase "tripling" not "almost quadrupling".

He mixed up H and M1. The monetary base has roughly tripled I think (and if I'm wrong well Morris is ignorant too).

If he didn't know about money multipliers, the money supply process, fractional reserve banking and my mother's maiden name (all equally certain) he might think this meant the money supply tripled. So he sends his long suffering research assistant to find the proof that the money supply tripled. The poor unfortunate guy came back with the number which Morris miss read due to the fact that "He puts ideology first and the [data] a distant second."

This story has the ring of truth to it: the research assistant was sent to do an impossible task, and Morris's ideology blocked him from realizing the mistake. (And, presumably, nobody edits his column at The Hill.) Interesting.

I remain ignorant regarding the money supply. One of the few things I remember from economics class in 11th grade is that "the money supply" is not well defined because of the presence of nonmonetary assets such as stocks, bonds, real estate, etc., as well as checking accounts and the like.

P.S. I'm still waiting for the anonymous commenter to come back to me with more data. I still think it's possible that there's a 271% in there (or, at least, "a multiple between 2.5x-3x," as the commenter claimed) that makes sense and that I just didn't know where to look.

I received the following email from a student at a university in another state:

After this semester I will be transfering into the college of science with the major of Statistics. My assignment right now for my undecided class is to interview a statistician, learn about the type of work statisticians do, and about statistics as a career in general. If you could take the time and complete the enclosed interview at your earliest convenience, it would be greatly appreciated.

Here's the form with my responses interpolated:

Even the pros are afflicted. See here.

Andy has frequently noted how conclusions can change depending on time frame...

http://www.good.is/post/number-of-pirates-killed-by-obama-in-two-graphs/

Just to follow up on Jeff's recent post . . . see here and here (see story #6).

Here.

I'm sure there is a better way to convey this information...

http://bigcrush.tumblr.com/post/95865650

This looks great to me:

Patrick Egan and Jack Citrin, "When the Supreme Court Decides, Does the Public Follow?"

Robert Erikson and Laura Stoker, "Vietnam Draft Lottery Status and Political Attitudes"

William Jacoby, "Individual Value Choices: Hierarchical Structure Versus Ambivalence and Indifference"

It's on Sat 25 Apr at 10.30am at 110 E 28 St, 2nd floor.

I use R for just about everything, including exploratory data analysis and graphics. The only other package with which I have any familiarity is Mathematica. I've been generally satisfied with R graphics, although there are things that I always struggle with, such as:

1. using expression() to get symbols and math expressions the way I want them;


2. writing line labels at an angle so that they lie along the line (and then having to re-do this if I change the dimensions of the plot, e.g. by changing the margins);


3. setting margins when I have multiple plots on a single figure, so that the axis labels fit but there is still enough room for the data;
   
4. placing labels or legends where they don't get in the way of the plot.
   

At least in my normal course of business, all of these issues only come up when I'm making publication-quality figures (or at least presentation-quality), not when I'm exploring the data or comparing the data to predictions. So I've always thought of R as being excellent for exploratory data analysis, and fair or poor for making publication-quality output. But sometimes I do find myself taking a lot of time on an exploratory plot (such as the example here), which is frustrating.
View image
And then a friend mentioned that he thinks R is good for publication-quality graphics --- you have precise control over everything --- but is terrible for exploratory graphics, which is exactly the opposite of the way I think of it! He pointed that, aside from some crude things you can do with identify(), R's graphics are non-interactive: you can't click to remove bad data points, or zoom in and out, or click on a line and change its color or width. He said good exploratory graphics programs let you do all of these things. But here's the kicker: he couldn't name a good exploratory graphics program! He says he knows they exist, but he doesn't know what they are.

So: what's worth a look, besides R and Mathematica? Am I using R just because it's what I'm used to (and it's free), or is it actually the best thing out there, as I have always assumed?

Jeff and John were bugging me about this so I thought I should give a quick summary:

Statistical Modeling, Causal Inference, and Social Science: Most of my stuff goes here. We also have several other contributors who unfortunately don't blog very often. When they do blog, they usually have something good to say. I used to ask my students and postdocs to blog here when I was on vacation, but then I found out they were spending hours writing these blog entries, which seems to be contrary to the spirit of the thing.

Fivethirtyeight.com: I've recently started posting political things on Nate Silver's site. Nate's super-cool--recall that one of the reasons I became a statistician was from reading Bill James, and Nate is definitely of that breed--and also this allows me to reach a different audience than I'm reaching here.

Newmajority.com: David Frum's conservative site. This started with an article I wrote a few months ago on the lessons for conservatives from the 2008 election. I am occasionally posting here when I have something relevant for this audience, whether it be a bit of number crunching specifically relevant to something newsworthy, or a more general point about political polarization or whatever. I do not see my research as inherently liberal or conservative (or moderate, for that matter), and I (naively, I'm sure) feel that politics would improve if all sides have a clearer view of public opinion.

The Monkey Cage: A blog run by John Sides, Lee Sigelman, and others, focusing on political science research. I post there sometimes (generally crossposting on this blog) and also participate in discussions there.

Overcoming Bias: I post here sometimes because it's fun (albeit frustrating) to try to communicate with a bunch of people with whom I probably disagree with on 95% of all issues. Robin Hansen is an interesting guy and it seems worth keeping these communication lines open, even though I feel I'm speaking a different language from most of the participants there.

Red State, Blue State, Rich State, Poor State: Boris set up this website for our book and we kept a blog going here, with frequent posts in the month leading up to the election and the month or so following. Since then I've been putting all my political posts here at Statistical Modeling (or at the other sides mentioned above), so there's no need to keep up with that one.

Rachel and I also set up a blog for my course last semester; that worked pretty well for communicating to students, and I think I'll do it again, but that's not relevant to discussions of public blogs here.

I think I can handle the first 5 blogs above. I can't really see reducing beyond that, given the goal of reaching diverse audiences. Of course what I really want is for everyone to read the Statistical Modeling blog--but I recognize that not everyone is fascinated by discussions of statistical graphics, R code, causal inference, and so forth. I'll tell you one thing: this is the only blog where you'll get my musings about literature!

Igor Carron forwarded this from Ed Tufte:

The Recovery Accountability and Transparency Board was created by the American Recovery and Reinvestment Act to coordinate and conduct oversight of funds distributed under this law in order to prevent fraud, waste and abuse. . . . The Board has a series of functions and powers to assist it in the mission of providing oversight and promoting transparency regarding expenditure of funds at all levels of government. . . . The Board is also charged under the Act with establishing and maintaining a user friendly website, Recovery.gov, to foster greater accountability and transparency in the use of covered funds. The job of the Recovery Accountability and Transparency Board is to make sure that Recovery.gov fulfills its mandate -- to help citizens track the spending of funds allocated by the American Recovery and Reinvestment Act.

That is all.

Jonathan Nagler writes:

Morgan Ryan sent me this quote:

"Perhaps the most perplexing part of the study lay in the attitude of the statisticians, who showed no enthusiastic confidence in their own figures. They should have reached certainty, but they talked like other men who knew less. The method did not result in faith. . . . at last, a scholar, fresh in arithmetic and ignorant of algebra, fell into a superstitious terror of complexity as the sink of facts." --Education of Henry Adams

Via Andraz's Twitter feed, I came across the following:

Using Natural Science and Engineering Research Council Canada (NSERC) statistics, we show that the $40,000 (Canadian) cost of preparation for a grant application and rejection by peer review in 2007 exceeded that of giving every qualified investigator a direct baseline discovery grant of $30,000 (average grant).

This would lead to an explosion in the number of "qualified investigators," and bring many lazy and mediocre ones in and drive most of the good and driven ones out. Also, the EU does compensate the preparation of grants. The preparation of a proposal is not all wasted effort: a proposal requires a researcher to organize his ideas.

Now, I don't want to come out of this post as a defender of the grant system - after all, the grant system has been one of the main centrifugal forces pulling me away from a career in research. Three things have been most problematic:

1. Lack of transparency: Arbitrary decisions by anonymous reviewers without the opportunity to address their criticism give them the power to make essentially political decisions. Grant picking should be a transparent public process - and research should strive towards something good for humanity.
2. Lack of accountability: Once the project has been approved, there is little pressure on the PI to actually achieve goals. Consequently, research often ends with the grant proposal, and research for a new grant proposal then begins. It would be better to spend most money for awards recognizing past research than to spend practically all of it for vapor and smoke.
3. Lack of a productive environment: To execute successful projects one needs the freedom to pick the best people with the right skills, competing with the industry. Good work requires focus, and there are not many people who can both do quality research and quality teaching. I cannot do both at the same time myself. Moreover, many research institutions have become internally rigid, slow and top-heavy, and the overhead is suffocating.

It went well, I got some laffs, which is what it's all about. One person asked a bunch of good questions about asymmetries between the Democrats and Republicans. I thought I gave some good answers but she appeared to be dissatisified. It was also fun because I got to meet a bunch of their PhD students beforehand. We had a fascinating conversation, almost all of which I've forgotten, unfortunately. I have to remember to take notes; my memory is not what it once was.

Here are the slides. Almost no overlap with my red state, blue state talk (which I'd given elsewhere at Harvard in the fall). At some point I'll put up a post discussing the arguments and open questions in some detail.

Here [link fixed]. I love this stuff.

Juned Siddique writes:

I have a question regarding a paragraph in your paper, "Prior distributions for variance parameters in hierarchical models."

In the paper, you write, "We view any noninformative or weakly-informative prior distribution as inherently provisional--after the model has been fit, one should look at the posterior distribution and see if it makes sense. If the posterior distribution does not make sense, this implies that additional prior knowledge is available that has not been included in the model, and that contradicts the assumptions of the prior distribution that has been used. It is then appropriate to go back and alter the prior distribution to be more consistent with this external knowledge."

I don't quite understand this passage, especially the part where you write, "this implies that additional prior knowledge is available that has not been included in the model," and was hoping to get more explanation.

My situation is that I am fitting a random-effects probit model and using posterior predictive checking to check the fit of the model. One way to get the model to fit the data well is to use an informative prior that I arrived at by iterating between posterior predictive checking and making my prior more informative. While changing one's model to make it fit the data better is standard in statistics, it seems like I should be changing the likelihood, not the prior. One the other hand, my "model" is my posterior distribution which also includes the prior.

My reply:

1. You should certainly feel free to change the likelihood as well as the prior. Both can have problems.

2. With hierarchical models, the whole likelihood/prior distinction becomes less clear. In your example, you have a probability model for the data (independent observations, I assume), a nonlinear probit model with various predictors, a normal model (probably) for your group-level coefficients, and some sort of hyperprior for what remains

3. My point in the quoted passage is that in the phrase "the posterior distribution does not make sense," the "does not make sense" part is implicitly (or explicitly) a comparison to some external knowledge or expectation that you have. "Does not make sense" compared to what? This external knowledge represents additional prior information.

Last month I reported on a statistical analysis by Josh Millet at Criteria Corp., suggesting that the economic climate for small business is improving. Millet now has an update (posted on 1 Apr but I assume it's serious):

With the final March numbers now in, the Hiring Activity Index nudged upwards very slightly again this month, to 62.3% from 61.4% in February. To me [Millet] this is an encouraging sign that the February jump in hiring activity by small businesses was not just a blip. If the data we're seeing means anything, the hiring situation for small and medium-sized businesses has begun to rebound.

Here's the graph I made of his numbers:

Millet also answers a bunch of potential criticisms of his measure:

There were some interesting comments and questions about the HAI and its potential utility as a leading economic indicator. We [Criteria Corp.] do sell our software on a subscription basis, and someone pointed out that if non-active subscribers didn't renew because of the downturn, this could artifically inflate the HAI because it is based on the percentage of our customer base that is actively doing pre-employment testing in a given month. This is a legitimate point, but I [Millet] will say that while low levels of use are a reason that customers sometimes do not renew, we haven't see non-renewal rates climb much since November, when the HAI dropped by 10 points. It was also suggested that higher numbers of job-seekers may result in applicants for positions that may not have been desirable previously--this is theoretically possible, but I don't see much evidence for it. What is most certainly true is that companies are getting far more applicants per open positon, as I previously blogged about here. However, since the HAI is based on the percentage of companies testing in a month, not the overall volume of tests, this shouldn't influence the HAI unduly, and wouldn't in any case explain the plunge in November and (partial) rebound in February.

I programmed a Bugs model (see below) and got this horrible and useless error message: Bugs opened a "Trap: incompatible copy" window, and there was nothing in the log window but this:

display(log)
check(C:\DOCUME~1\ANDREW~1\LOCALS~1\Temp\RtmpMCdEbN/pewpoststrat2.bug.txt)

No syntax error, no nothing. My first thought, of course, was that there was something wrong with the default directory, but when I copied the "schools" example over, it worked fine. So I went through the laborious process of debugging, stripping down the code until I could get it to run and then gradually adding back lines until it crashed. I still couldn't figure out what was wrong so I brought Matt over and he figured it out at a glance.

Here was the original code:

This is cool, but what I'd really like is software that allows me to allocate the voting booths among polling places . . .

In a comment on my entry on why I don't like so-called Bayesian hypothesis testing, Stephen Senn writes:

Bayesian hypothesis tests are the work of Harold Jeffreys who realised that you could not proceed using vague priors for parameters unless you have a means of choosing between simpler and more complex models. Also he was keen to find ways of proving that scientific laws are true. If you think you can do this and want to do it you need Bayesian hypothesis tests.

I, personally, don't like Jeffreys's approach. However, I think that it is a tribute to his genius that he realised that such a system had to be part and parcel of any attempt to be semi-objective in the use of Bayes. Unfortunately we now have many so-called Bayesians who think they can use uninformative priors without a system of deciding between simpler and more complex hypotheses. This is not possible.

My reply: When deciding between simpler and more complex hypotheses, I generally prefer the more complex hypothesis. When I choose the simpler hypothesis, I view this as a combination of labor-saving device and approximate Bayes, pooling a parameter estimate all the way to zero instead of merely pooling it most of the way. I certainly don't see Bayes factors having any relevance, given the oft-noted problem that Bayes factors can depend decisively on aspects of the prior distribution that have no influence on the posterior distribution under each of the individual models.

Antonio Ramos writes:

In most social sciences, maximum likelihood is taught before topics such as multilevel modeling and or Bayesian statistics more generally. However, in the preface of your multilevel book you say that basic statistics and regression classes are sufficient for one to study your book. So may question is: should we learn maximum likelihood first or it is just historical convention, without much pedagogical basis?

My reply: Maximum likelihood is fine; we discuss it in chapter 18, I believe it is, where we discuss Bayesian methods as a generalization of maximum likelihood. All of this is important to learn, but I think you can get started in serious applied statistics (which is what our book is about) without necessarily already knowing it.

Dan Kahan writes:

Hi. I'm wondering if you -- and readers of your blog -- have a take on how to preempt the mistake of construing overlapping confidence intervals as indicating that distinct predictors (e.g, two treatments in an experiment) do not have a significantly different effect. See Schenker, N. & Gentleman, J.F. On Judging the Significance of Differences by Examining the Overlap Between Confidence Intervals. Am. Stat. 55, 182-186 (2001). The mistake is common enough to make me fret about using really nice bar plots w/ CIs when the CIs overlap. One can always point out in the text that it is a mistake to see overlapping CIs as indicating lack of a significant difference, and then report the relevant difference of the relevant point estimates & the CI associated with *that* difference, but since having to use additional text to explain how to interpret a figure undermines the whole point of using a figure, I'm wondering if there are better reporting or graphic-display strategies I'm unaware of.

My reply: I'm not as worried as you might expect by this, as statistical significance is pretty arbitrary anyway. I'm more worried about people not realizing that the difference between "significant" and "not significant" is not itself statistically significant. Ultimately, if there's a particular comparison you want people to make, you have to make it yourself, and if there are any comparisons that you don't want people to make, it's best to explicitly tell them not to do it.

Richard Posner defended the rationality of people who bought stocks during the bubble, writing:

People buy common stock when stock prices are rising. They (notoriously) bought houses during the early 2000s when house prices were rising. Since almost no one can predict the ups and downs of the stock market or the housing market, these purchases must have been motivated, Akerlof and Shiller argue, by something other than a rational investment strategy. But this is not at all obvious . . . Stocks have generally been a good investment, at least when held for a considerable period. . . .

I agree with Nate, who disagrees with Richard Posner by pointing out that, in fact, there was evidence that stocks were overpriced during the early 2000s, even at the time.

I'd like to add one comment. During all these bubble years, the experts were telling us over and over again how we should be buying stocks, how stocks were the best investment over the long term, and how we were all irrational for not putting more of our money into the stock market.

What's the logic here? People were being irrational by hesitating to buy stocks when they were going up, then they were finally being rational by buying stocks when they had very high prices?

I think all this discussion is hindered by the overloading of the term "rational." I imagine that just about everybody takes his or her money management seriously, and I'm sure people are trying to behave rationally with their investments. The trouble is that there are lots of rules out there to follow, so there's more than one way to be rational. I agree with Nate that Posner's implicit assumption--that people were following expert advice, and so they must have been applying (prospectively) good judgment--is misguided.

This is great. I'm not commenting one way or another on the science--it's not something I know anything about. Rather, it's just funny to see the phrase "researchers in Brooklyn" in a newspaper article. Brooklyn's usually a punchline but this time they're serious.

From What Colleges Should Learn From Newspapers' Decline:

Newspapers are dying. Are universities next? The parallels between them are closer than they appear. Both industries are in the business of creating and communicating information. Paradoxically, both are threatened by the way technology has made that easier than ever before.[...]

And it would be a grave mistake to assume that the regulatory walls of accreditation will protect traditional universities forever. Elite institutions like Stanford University and Yale University (which are, luckily for them, in the eternally lucrative sorting and prestige business) are giving away extremely good lectures on the Internet, free. Web sites like Academic Earth are organizing those and thousands more like them into "playlists," which is really just iPodspeak for "curricula." Every year the high schools graduate another three million students who have never known a world that worked any other way.

There are four value propositions of universities:

1. Helping students familiarize themselves with a topic (an informative, entertaining lecture showing why someone should care and up-to-date pointers)
2. Helping students master the applications of a topic (project supervision, in-depth tutorials, apprenticeship)
3. Helping students contribute to a topic and push the envelope (research apprenticeship, collaboration, leadership)
4. Network building among students of similar capability, similar or complementary interests

The first value proposition can be done more effectively with the use of the WWW, but one should be careful: if this content isn't compensated, the compensation will take the form of sometimes hidden advertising (did you notice the image on this post?). The second can partly be done remotely with a lower cost - at places like Columbia, it's often the grad students who tutor, while professors do research. The third requires excellence, dedication and research funding - if there is no funding, the research will become confidential and proprietary. The fourth requires sorting and community organization.

It is always a good idea to rapidly adopt new technology, some institutions and individuals are pushing the edge with either video lectures, or with course materials. We're trying to innovate with blogging.

[Updated after reading thoughtful comments by hal, yolio and Igor Carron. This blog is really a community.]

Aleks forwarded this to me. It looks interesting. I'm disappointed that the readings don't include anything by Bill Cleveland, but on the plus side the course appears to be incredibly well organized. I'm sure I'd love it. They sure didn't have classes like this when I was in college.

Iain writes:

I [Iain] remember Dennis Cook used to have a multiple choice question in an exam for a regression class that asked simply "if in doubt do what?" with correct answer "take the log."

I just want to know what the other options were in the multiple choice.

Different Sorts of Political Polarization in the United States

Monday, April 6, 2009, 12:00-1:45 p.m.
(Buffet lunch available at 12:00 -- Presentation begins at 12:15)

Location: Harvard Kennedy School | Allison Dining Room (Taubman 5th floor)

I'm not sure exactly what I'll talk about. I gave them the link to this article with Delia but I think there's other stuff I want to discuss too. I gave the Red State, Blue State talk at Harvard a few months ago so I don't need to talk so much about that stuff. Probably I'll throw a lot of data at 'em without any single coherent storyline. If you show up, please think of some tough questions: the format is that I speak for 45 minutes and then there's 45 minutes of discussion.

P.S. Here's what happened.

The Ph.D. students of Columbia's statistics department have arranged a one-day conference centered on student research, in memory of Minghui Yu, our student who tragically died last year. Conference information is here.and the schedule of speakers and topics is here. It looks like an interesting mix of topics.

Democracy is alive and well in the United States, at least when there is no incumbent running for reelection and voters have a choice between two clear alternatives (witness the recent closely-contested House election in upstate New York). Partisans of all persuasions are dissatisfied with the process, in particular how votes are counted. Democrats still remember Florida 2000, when George W. Bush won the majority of the votes counted, even though analyses accounting for overvotes and the notorious "butterfly ballot" in Palm Beach County showed that tens of thousands more voters in the state were intending to vote for Al Gore. The memory of this carried over to 2004, when more questionable claims were made about Ohio's vote count. On the other side, Republicans have expressed concern about ballot fraud dating back to the old-style big-city political "machines." And, of course, supporters of third parties of left, right, and center struggle against restrictive ballot laws and the difficulties of state-by-state registration for national campaigns.

In a new book, "The Democracy Index: Why Our Election System is Failing and How To Fix It," Yale law professor Heather Gerken reviews the problems of our election systems and suggests an intriguing way to improve things: as she puts it,

We should create a Democracy Index that ranks states and localities based on election performance. The Index would function as the rough equivalent of the U.S. News and World Report rankings for colleges . . . It would focus on issues that matter to all voters: how long did you spend in line? How many ballots were discarded? How often did voting machines break down? The Index would tell voters not only whether things are working in their own state, but how their state compares to its neighbors.

Heather Gerken worked on Barack Obama's campaign team, and unsurprisingly her suggestions focus on issues of particular concern to many Democrats. That said, I think her idea could -- and should -- be of interest to Republicans as well. Americans of all political persuasions have an interest in our major institutions -- including business, the military, education, the news media and, yes, government too -- retaining the confidence of the people.

What makes Gerken's proposal particularly appealing is its feature of using open sharing of information to create incentives for states and localities to improve their electoral systems, by setting up specific targets that voters can follow.

I like her idea -- a lot -- and just have a couple of concerns and suggestions.

First, as noted above, Gerken compares her proposed ratings to the U.S. News rankings of colleges. She immediately disassociates herself from the particulars of the U.S. News rankings -- which are notorious for being "gamed" by colleges, for example by manipulating early admissions -- but this brings to mind another problem, which is that rankings probably won't change much from year to year. I would've liked to see a ranking of the 50 states based on their current voting systems. Illinois and Ohio aside, would the traditional good-government metric of "closeness to Canada" be a good predictor? I can see why Gerken didn't include such a ranking in her book -- if you're trying to sell a new idea to the nation, it doesn't make sense to start off by disparaging the 25 states that would be on the bottom half of such a list -- but a baseline would've given me more of a sense of what the ratings would mean.

My second suggestion to Gerken and fellow reformers is that they broaden their list of concerns. On page 123, Gerken writes that the numbers in the index should "evaluate whether (1) every eligible voter who wants to register can do so, (2) every registered voter who wants to cast a ballot can do so, and (3) every ballot cast is counted properly." One thing she does not mention here is voter fraud. According to my Barnard College colleague and urban politics scholar Lorraine Minnite, "voter fraud is extremely rare"; nonetheless, fraud is certainly a real political concern. In her report, Minnite writes that "Better data collection and election administration will improve the public discussion of voter fraud and lead to more appropriate policies," and so it would seem to be a win-win policy to include some measure of voter fraud in the Democracy Index.

In summary, Gerken's proposals are interesting and appealing, and I hope that the fact that they come from a member of the Obama campaign team does not stop Republicans from taking her ideas, adding to them, and working with Democrats and supporters of minor parties (who might very well have their own reasonable additions to the Democracy Index) to set up a system in which the flaws of state and local election systems are made public in a nonpartisan way that would encourage innovation and improvement.

J. Robert Lennon writes that he is "rarely disappointed by a book's ending. Almost never, in fact. If I like a book all the way through, I almost always like the way it ends, too . . ."

I know what he means, sort of, but what about The Bonfire of the Vanities? I loved that book while I was reading it, but the ending was so weak that, for me, its lameness sort of leaked backwards into the body of the book, so that, once it was all over, I didn't retrospectively like the book at all! Partly this was because what I liked about the beginning and middle of the book was the sense that I was seeing all these different facets of the world, and somehow the ending took something away from this.

I have a problem that comes up a lot in programming, and I'm sure the experts can help me out. It has to do with giving names to objects that I'm working with. Sometimes the naming is natural; for example, if I have a variable for sex in a survey data, I can define "male" as 1 for men and 0 for women and then continue with names such as "male00" for data in the 2000 survey, "male04" for the 2004 survey, and so forth, or else use a consistent set of names and define things within dataframes. In either case, this is no problem.

What is more of a hassle is coming up with names for temporary varaibles. For example, should I use "i" for every looping index, with "j" for the nested loops, "k" for the next level, and so forth? This can get confusing when referring to array indexes (sometimes you end up needing things like a[j,i]), or should I go for mnemonics, such as "i" to index survey respondents, "s" for states, "e" for ethnic groups, "r" for religious denominations, etc? Sometimes I've tried to reserve "j" for the lowest level of poststratification cel, but that isn't always convenient.

At other times I've tried more descriptive names, for example n.state for the number of states (50 or 51, depending on whether D.C. is included) and i.state for state indexes, thus giving loops such as "for (i.state in 1:n.state)" or "for (i.state in states)," where "states" has been pre-defined as the vector of state numbers. This approach is currently my favorite--I tried to stick to something like it in my book with Jennifer--but can also create its own difficulties, as I have to remember the names of all the indexing variables.

Beyond looping indexes, there are all sorts of temporary variables I'm creating all the time; for example, after fitting a multilevel model, pulling out coefficient estimates: "fix <- fixef (M2000.all)." There's always a tension in naming these temporary quantities: on one hand, I don't want meaningless names such as "temp" floating around; on the other, every new variable name is another thing to remember, which is annoying if it's only being used in two lnes of the program.

I guess the real solution is to ruthlessly compartmentalize: in R, this essentially means that you turn every paragraph of code into a function and get rid of all globally-defined variables. I haven't always had the discipline to do this, but maybe I should try.

Some useful ideas:

With trendy letters like Z and Q commanding 10 points a piece, parents across the country are rethinking their naming expenditures. If you're looking to maximize style but minimize points, try these tips for cool, cost-effective baby-names. . . .

Perhaps Nan and Sue will make a comeback?

Jeff pointed me to this graph from congressmember Paul Ryan:

Ryan is actually being generous to the Democrats here. You can't imagine how things are going to look around 2150 or so!

Graphs are gimmicks, substituting fancy displays for careful analysis and rigorous reasoning. It's basically a tradeoff: the snazzier your display, the more you can get away with a crappy underlying analysis. Conversely, a good analysis doesn't need a fancy graph to sell itself. The best quantitative research has an underlying clarity and a substantive importance whose results are best presented in a sober, serious tabular display. And the best quantitative researchers trust their peers enough to present their estimates and standard errors directly, with no tricks, for all to see and evaluate. Let's leave the dot plots, pie charts, moving zip charts, and all the rest to the folks in the marketing department and the art directors of Newsweek and USA Today. Here at this blog we're doing actual research and we want to see, and present, the hard numbers.

To get a sense of what's at stake here, consider two sorts of analyses. At one extreme are controlled experiments with clean estimate and p-value, and a well-specified regressions with robust standard errors, where the p-values really mean something. At the other extreme are descriptive data summaries--often augmented with models such as multilevel regressions chock full of probability distributions that aren't actually justified by any randomization, either in treatment assignment or data collection--with displays of all sorts of cross-classified model estimates. The problem with this latter analysis is not really the modeling--if you state your assumptions carefully, models are fine--but the display of all sorts of numbers and comparisons that no way are statistically significant.

For example, suppose a research article with a graph showing three lines with different slopes. It's natural for the reader to assume, if such a graph is featured prominently in the article, that the three slopes are statistically significantly different from each other. But what if no p-value is given? Worse, what there are no point estimates are no standard errors to be found? Let alone the sort of multiple comparisons correction that might be needed, considering all the graphs that might have been displayed? Now, I'm not implying any scientific misconduct here--and, to keep personalities out of this, I've refrained from linking to the article that I'm thinking about here--but it's sloppy at best and statistical malpractice at worst to foreground a comparison that has been presented with no rigorous--or even approximately rigorous--measure of uncertainty. And, no, it's not an excuse that the researchers actually "believe" their claim. Sincerity is no defense, There's a reason our forefathers developed p-values and all the rest, and let's remember those reasons.

The positive case for tables

So far I've explained my aversion to graphs as an adornment to, or really a substitute for, scientific research. I've been bothered for a while by the trend of graphical displays in journal articles, but only in writing this piece right here have I realized the real problem, which is not so much that graphs are imprecise, or hard to read, or even that they encourage us to evaluate research by its "production values" (as embodied in fancy models in graphs) rather than its fundamental contributions, but rather that graphs are inherently a way of implying results that are often not statistically significant. (And all but the simplest graphs allow so many different visual comparisons, that even if certain main effects actually do past the p-value test, many many more inevitably won't. Some techniques have been developed to display multiple-comparisons-corrected uncertainty bounds, but these are rarely included in graphs for the understandable reason that they magnify visual clutter.)

But enough about graphs. Now I'd like to talk a bit about why tables are not merely a necessary evil but are actually a positive good.

A table lays down your results, unadorned, for the readers--and, most importantly, scientific peers--to judge. Good tables often have lots of numbers. That's fine--different readers may be interested in different things. A table is not meant to be read as a narrative, so don't obsess about clarity. It's much more important to put in the exact numbers, as these represent the most important summary of your results, estimated local average treatment effects and all the rest.

It's also helpful in a table to have a minimum of four significant digits. A good choice is often to use the default provided by whatever software you have used to fit the model. Software designers have chosen their defaults for a good reason, and I'd go with that. Unncessary rounding is risky; who knows what information might be lost in the foolish pursuit of a "clean"-looking table?

There is also the question of what words should be used for the rows and columns of the table. In tables of regressions, most of the rows represent independent variables. Here, I recommend using the variable names provided by the computer program, which are typically abbreviations in all caps. Using these abbreviations gets the reader a little closer to your actual analysis and also has the benefit that, if he or she wants to replicate your study with the same dataset, it will be clearer how to do it. In addition, using these raw variable names makes it more clear that you didn't do anything shifty such as transforming or combining your variables before putting them in your regression.

We'd do well to take a lead from our most prominent social science colleagues--the economists--who have, by and large, held the line on graphics and have insisted on tabular presentations of results in their journals. One advantage of these norms is that, when you read an econ paper, you can find the numbers that you want; the authors of these articles are laying it on the line and giving you their betas. Beyond this, the standardization is a benefit in itself: a patterned way of presenting results allows the expert readers--who, after all, represent the most important audience for journal articles--to find and evaluate the key results in an article without having to figure out new sorts of displays. Less form, more content: that's what tables are all about. If you've found something great and you want to share it with the world, sure, make a pretty graph and put it on a blog. But please, please, keep these abominations out of our scientific journals.

B-b-b-but . . .

Yes, you might reply, sure, graphics are manipulative tricks and tables are the best. But doesn't the ambitious researcher need to make graphs, just to keep up with everybody else, just to get his or her research noticed? It's the peacock's tail all over again--I don't want to waste my precious time making fancy 3-D color bar charts, but if I don't, my work will get lost in the nation's collective in-box.

To this I say, No! Stand firm! Don't bend your principles for short-term gain. We're all in this together and we all have to be strong, to resist the transformation of serious social science into a set of statistical bells and whistles. Everything up to and including ordered logistic regression is OK, and it's fine--nay, mandatory--to use heteroscedasticity-consistent standard errors. But No to inapporpriate models and No to graphical displays that imply research findings where none necessarily exist.

Beyond this, even in the short term I think there are some gains from going graph-free.
And, finally, the time you save not agonizing over details of graphs can be instead be used to think more seriously about your research. Undoubtedly there's a time substitution: effort spent tinkering with graphs (or, for that matter, blogging) is effort not spent collecting data, working out models, proving theorems, and all the rest. If you must make a graph, try only to graph unadorned raw data, so that you're not implying you have anything you don't. And I recommend using Excel, which has some really nice defaults as well as options such as those 3-D colored bar charts. If you're gonna have a graph, you might as well make it pretty. I recommend a separate color for each bar--and if you want to throw in a line as well, use a separate y-axis on the right side of the graph.

Damn. It felt good to get that off my chest. I hope nobody reads this, though, or I might be able to fool people with graphs much longer.