Gregor sent another question:

The following came up in my recent disscussion in the department. I have some data on several variables that are of interest for the analysis, say the difference between treatments. We can do analysis of each variable separately and infer the differences. I had a comment that Bonferroni or any procedure like it should be applied. How does this scale with Bayesian approach and posterior probabilities that difference is bigger/lower than 0? Should multivariate analysis of all variables in one run perform better in terms of this correction?

My (quick) response: I hate that Bonferroni stuff. In a Bayesian context, it’s more appropriate to model multiple comparisons using a hierarchical model. More generally, it’s ok for me that 5% of my 95% intervals will not contain the true value. For more on how I think of these multiple comparisons things, see these 2 papers:

[2000] Type S error rates for classical and Bayesian single and multiple comparison procedures. {\em Computational Statistics} {\bf 15}, 373–390. (Andrew Gelman and Francis Tuerlinckx)

The difference between “significant” and “not significant” is not itself statistically significant. (Andrew Gelman and Hal Stern)