« Axes that extend below 0 or above 1: actually a bigger issue involving how statistical variables are stored on the computer | Main | A more formal rant »
May 13, 2008
The folk theorem of statistical computing
I mentioned this in class all the time this semester so I thought I should share it with the rest of you. The folk theorem is this: When you have computational problems, often there's a problem with your model. I think this could be phrased more pithily--I'm not so good in the pithiness department--but in any case it's been true in my experience.
Also relevant to the discussion is this paper from 2004 on parameterization and Bayesian modeling, which makes a related point:
Progress in statistical computation often leads to advances in statistical modeling. For example, it is surprisingly common that an existing model is reparameterized, solely for computational purposes, but then this new con guration motivates a new family of models that is useful in applied statistics. One reason why this phenomenon may not have been noticed in statistics is that reparameterizations do not change the likelihood. In a Bayesian framework, however, a transformation of parameters typically suggests a new family of prior distributions.
Posted by Andrew at May 13, 2008 12:37 AM
Trackback Pings
TrackBack URL for this entry:
http://www.stat.columbia.edu/~cook/movabletype/mt-tb.cgi/1587
Comments
I'm a software engineer. In the face of a difficult debugging problem that was dragging out, I was once counseled, "If you can't win the game, change the rules." That seems a corollary of this your theorem.
Posted by: Chris at May 13, 2008 8:36 AM.
Posted by: Russell Almond at May 16, 2008 3:30 PM.