Sample size and power calculations

Russ Lenth (Department of Statistics, University of Iowa) wrote a great article on sample size and power calculations in the American Statistican in 2001. I was looking for it as a reference for Barry’s comment on this entry.

Anyway, I saw that Lenth has a webpage with a power/sample-size calculator and also some of the advice from his article, in easily-digestible form
. Perhaps this will be helpful to some of youall. I’m not happy with most of what’s been written on sample size and power calculations in the statistical and biostatistical literature.

Also, here are some of my ramblings on power calculations.

2 thoughts on “Sample size and power calculations

  1. My interests in this area started in computing — writing algorithms for the noncentral beta, F, and t distributions; then expanded to power; then to sample size; and now, I spend a lot of time thinking about effect size. I hope that message comes out strongly in my website's points about the right things you can do — which I feel pretty good about. The upshot is that I think people need to focus a lot less on asterisks and a lot more on the actual numerical effects that are observed in statistical data, with units of measurement attached.

    It's interesting to examine why people in the pharmaceutical industry and in the social sciences drive me batty, but for opposite reasons. Many pharmaceutical people refuse to transform data, no matter what. Taking logs, say, moves you away from the original scale of measurement, and they don't want to have to explain that to other people (especially the FDA).

    Social scientists, on the other hand, seem to have built up a tradition whereby they refuse to think about the original scale of measurement, no matter what. It is in the social sciences that we see the emphasis on standardized regression coefficients; where effect size is always discussed on a standardized scale; and where Cohen's determinations of what is small, medium, or large on a standardized effect-size scale are considered benchmarks for sample-size problems. I have written a lot about why I think that is wrong, and will not elaborate on it here.

    I am not deeply involved in social-science applications. But what has recently occurred to me is that perhaps the reason my anti-Cohen opinions are such an uphill battle is that maybe social scientists believe that standardization is the true path toward such ideals as "objectivity" and "validity" — that somehow, making a judgment about the actual size of an effect, on the actual scale of measurement, is somehow wrong. Is this true? I'd be interested in hearing from people who do serious social-science work.

  2. You're right about the effect certainly and quite possibly about it's origin as well. Callous disregard for the distinction between statistical and practical significance is an intellectual vice of the social sciences; Deirdre McCloskey has been shouting about it for years.

    I think that part of the problem might also be that unlike the experimental sciences, the social sciences often deal in data about historical, unrepeatable events, so if you're going to have a science at all you have to start standardising things. But it's an intellectual vice for sure.

Comments are closed.