DIC (the Deviance Information Criterion of Spiegelhalter et al.) is a good idea and, I think, the right way to generalize AIC (the Akaike Information Criterion) when trying to get a rough estimate of predictive accuracy for complex models. We discuss DIC in Section 6.7 of Bayesian Data Analysis (second edition) and illustrate its use with the 8-schools example.

However, some practical difficulties can arise:

1. In the examples I’ve worked on, pD and DIC are computationally unstable. You need a lot more simulations to get a good estimate of pD and DIC than to get a good estimate of parameters in the model. If the simulations are far from convergence, the estimates of pD and DIC can be particularly weird.

Because of this instability, I don’t actually recommend using DIC to pick a model. Actually, I don’t recommend the use of any automatic criterion to pick a model (although, if I had to choose a criterion, I’d prefer a predictive-error measure such as DIC, rather than something like BIC that I don’t fully understand). But I can see that DIC could be useful for understanding how a set of models fit together.

2. bugs and bugs.R use different formulas for pD. bugs uses the formula from Spiegelhater et al, whereas bugs.R uses var(deviance)/2 Asymptotically, both formulas are correct, but with finite samples I really don’t know. I’d expect that the Spiegelhalter et al. formula is better–I say this just because they’ve thought harder about these things than I have, and I assume they came up with their formula for good reasons! The reason why I used a different formula is that the bugs output does not, in general, provide enough info, in general, for me to compute their formula.