In response to my comments here, Kevin writes:

On Andrew’s point about interpretation: Bob Carpenter is correct in his post that we were trying to convey the idea that our method is (in our opinion) an improvement over previous approaches that tended to be quite imprecise. Dan and I would not claim that our method is arbitrarily precise.

On Andrew’s point about the MCMC run length: our choice of 4 million scans is more than necessary. We simply did this because we had the available computer time.
Diagnostics looked fine on a 1 million scan run and continue to look fine on the longer run. Results don’t change from the 1 million scan run to the 4 million scan run.

On Andrew’s point about parameterization: I think the suggestion is to do something similar to what Andrew and coauthors suggested here (Section 2). I think one could do something like that here although we really tried to keep the model as simple as possible while still being an accurate representation of the data. As a side note, we did a number of things (including posterior predictive checks) to assess model fit– only some of these things made it into the paper.

I think the appendix of the paper deals with Bob Carpenter’s question about data collection. See also the discussion about data collection in this paper that is forthcoming at Stanford Law Review.

Zubon is right to note that what counts as “centrist” depends entirely on what the references points are. We try to be as clear as possible about this in the body of the paper. Note that the 0 position on the scale is arbitrary– there’s no reason to think that 0 is “the center”. The data can speak only to relative locations.

BTW, the version of the paper that Bob Carpenter linked to here is the final QJPS version. This should be freely available to everyone.