I’ve been thinking about power calculations recently because some colleagues and I are designing a survey to learn about social and political polarization (to what extent are people with different attitudes clustered in the social network?). We’d like the survey to be large enough, and with precise enough questions, so that we can have a reasonable expectation of actually learning something useful. Hence, power calculations.

(For some simple examples of power calculations, see this page at UCLA. Briefly, “power” is the probability that a study will detect a nonzero effect of some specified and reasonable size.)

The funny thing about this is that I’ve done a lot of power calculations but this is the first time that I’ve really done it for the nominal reason that we want to design a study with enough power.

The usual reason I do power calculations is for funded public health studies in which I am collaborating. The NIH always requires power calculations–basically you need to have an 80% chance of detecting a specified effect at the 95% level, which in the usual setting means that your effect should be at least 2.8 (= 1.96 + .84) standard errors away from 0. The way this always happens is that the investigator has an idea of a feasible sample size, and we play around with the power calculations to see how large an effect can be reasonably estimated. This is all kosher, but it starts with the sample size, not with the hypothesis (which is how a power calculation would nominally be done).

Anyway, this new power calculation is interesting because we have lots of options in the survey design–it’s not like I’m studying a treatment that has already been picked by someone else. One option is to use the “how many X’s” idea of Killworth, McCarty, and others, a survey that we’ve had some success reanalyzing.