Status and statistical graphics

I recently posted on statistical graphics, making the following points:

1. “Exploratory” and “confirmatory” data analysis (that is, statistical graphics and statistical hypothesis tests) are not opposites; rather, they go together and represent two ways of summarizing model checks, two ways of comparing data to a fitted model. (In Tukey’s writings on EDA, the models tend to be hidden and implicit, but I think they’re there nonetheless, underlying the choices of graphs.)

2. Statistical graphics are useful, especially when used in combination with complex models. Graphs of raw data are fine, but graphs can be much more effective when model-guided. And, conversely, complex models can be much more effective when accompanied by useful graphs.

3. Researchers in social science, even otherwise very good researchers, appear to be unaware of points 1 and 2 above. Instead, they tend to plot the raw data (if they do any plotting at all), then jump to the model and never look back. The result, it seems to me, is (a) models that do not fit the data–and thus, do not learn from the data–as much as they could, and (b) a much reduced ability to find interesting unexpected patterns, compared to what could be learned from post-model exploratory graphics.

4. Good statistical graphics are hard to do, much harder than running regressions. I made last point in response to Seth, who’d argued that graphics are easy to do, that scientists avoid graphs (and statisticians avoid research on graphical methods) because graphs are so easy and thus graphics are low in status. Seth claimed that when something (such as, in this case, statistical graphics) is useful, it will be low in prestige.

Economist and former co-blogger Robin Hanson picked up on my remark that, in my experience, statisticians–even those who teach at universities!–generally prefer to do work that is useful rather than useless. I think Robin misunderstands the fundamentally methodological nature of statistical research–and he also seems to have missed my point about graphics and modeling going together–but you can read our back-and-forth and draw your own conclusions.

In any case, my predominant interest here is not in academics and status-seeking, but rather in exploratory graphics. My only point in bringing up the issue of the prestige of graphical methods was to dispute Seth’s claim that graphs are easy to do. Once we recognize that it is hard to make good graphs, we get some insight (perhaps) into why exploratory statistical methods are not used as often as they should be.

I wanted to emphasize this so as to focus the discussion back to statistics and graphics and away from less interesting (to me, although not to Seth or Robin) arguments about the purported status benefits of useless research.

P.S. Regarding status-seeking and discussions thereof, this comment by “Popeye” says it all, I think.

7 thoughts on “Status and statistical graphics

  1. well, there are many parts of our practice that are actually "hard" to do "correctly," but, when performed in a mediocre way, seem invisible. i think computing is another example. i'm all for a rebalancing of priorities to that these invisible parts of our practice get the respect (publications, air time in phd curricula, etc) they deserve.

  2. On graphs, recall some discussion by Brian Ripley on S news years ago about the need to distinguish the purpose of graphs – i.e. to aid discovery versus disseminate a discovery widely…

    Also to display lack of model fit versus the fit of the model (Frank Harrel has a nice recent paper on displaying covariate effects for survival data).

    There is an evidence based literature of graphics but much is just qualitative and opinions about what are good graphs

    I do believe good graphs are very hard to make – and often even harder to get into publications (and of course some graphs are just meant/suitable for the research team inself)

    Keith

  3. All else equal I'm sure people prefer to do useful research, but their could often be other considerations. The question is: how well are status, career, financial, aesthetic and other other awards aligned with doing useful research?

    The answer would be useful.

  4. Count me in the "graphics are hard" camp. Of course simple stuff is usually simple — if you just want a scatterplot, hey, no problem — but often even a modest level of sophistication takes a lot of effort. In R, for example, I often have to spend way more time than I would like, just getting the axis ranges set the way I want them. For instance, I plot y vs x, then overlay a plot of z vs x. Except, whoops, the range of z is wider than the range of y, so I have to re-plot in the other order, or else specify the y-axis range explicitly. And then if I decide to put q vs x on the same plot, the issue might come up again. And once I have a few lines on the plot, I'd like to label them — I prefer that to a legend, because with a legend you have to keep looking back and forth between the legend and the lines — but there's no easy way to automatically label the lines, so again, I have to spend time to specify exactly where the labels go. And all of these are things that I often want to do even with exploratory graphics; once you get to publication quality, it's even more of a pain. I guess I would agree that this stuff is "hard" in the sense of being "tedious and a pain to do", not "conceptually difficult and thus indicating great creativity", so maybe I'm not saying it's high status (or should be). But I do know that I respect people who make good graphics and I have some disdain for people who don't.

    As far as "everyone is status-seeking", or "everyone is such-and-such": A lot of people who try to understand the world just latch onto one effect and magnify its importance, sometimes to an absurd degree. It's like the blind men and the elephant, except that the blind men get to choose which part of the elephant to feel. To some economists, people are utility-maximizing machines. To some psychologists, people are obsessed with status because that's how you get a mate to pass along your genes. To some political scientists, all actions can be interpreted as attempts to increase one's power. Marxists see class struggle everywhere they look. And so on.

    Of course, all of those things are true to some extent and apply to most people to some degree. The thing I think is funny — perhaps it shouldn't surprise me — is that people who are themselves obsessed with money think that everybody is motivated by money; people who are obsessed with power think that everybody is motivated by power; people who are obsessed with status think that everybody is motivated by status; and so on. It seems obvious to me that _some_ people are status-seekers, some people are fairly rational utility-maximizers, some people are very concerned about projecting high status, some people are motivated by this, that, or the other. Sure, people have some similarities, but when it comes to what motivates peoples' behavior, people are very very different.

  5. I agree that good graphs are hard to do and have low status. Here are three reasons why they have low status:

    1. They don't look hard, ex post. The whole point is to puzzle until you find a way to brilliantly display the main point of the data in a simple way. Naturally, that looks simple.

    2. Since graphs have low status and are hard, people don't think about them much. As a result, people don't even realize what makes one graph brilliant and another one stupid and unclear. Illiterates don't rate good novels higher than bad ones.

    Economists like me think about regressions a lot. As a result, we can tell the difference between a good one and a bad one very easily, and we reward people who do good ones with praise.

    3. Regressions are all basically alike, but good graphics vary a lot. Thus, it is harder to come up with a standard way to evaluate quality in a graphic.

Comments are closed.