Michael Papenfus writes,

I have been using your ARM book to learn multilevel modeling and I have a question which I cannot seem to find answer to in your book. How can I assess the overall significance of a random effect (varying intercept)?
Let say I estimate

M1 <- lmer(y ~ 1 + x (1|country)) and obtain the results from your book which include Error terrms: Group Name Std.Dev. county (Intercept) 0.33 Residual 0.76 Is there a way to test for the overall significance of the random effect (varying intercept)? I don't believe that the se.ranef is what I am looking for. I believe that some packages such as Stata xtmixed provide standard errors to the variance components.

My reply: the mcsamp() function does approximate posterior simulations for the multilevel model and will give you a sense of the uncertainty in the variance parameters. To test the statistical significance, though, you have to fit the model with and without the variance parameters and check some measure of fit. I haven’t thought too much about this problem because I usually keep in whatever I can, discarding variance parameters only for convenience (in which case it’s usually clear in practical terms that a variance component is small enough that it can be ignored). Ultimately, I think all the variance components exist at some level, although they can be small enough that they can be ignored. That said, you can compare models with and without variance components using predictive accuracy. See the table on the top of p.526 for an example.