Michael Kubovy writes,

Can you point me to a model report of empirical research (preferably of a designed experiment) using mixed models?

As you know, the pattern in psychology is to have a stultifying paragraph listing which effects and interactions were or were not significant: “… the a by b interaction was significant — F(n1, n2) = 23.3456, p ≤ .00023 … ” — followed by an interaction plot that summarizes the main results.

But suppose I [Kubovy] need to report the process that led me to settle on txt.lmer2:

> anova(txt.lmerM, txt.lmer1M, txt.lmer2M, txt.lmer3M)
Data: txt
Models:
txt.lmerM: duration ~ vis * display + pitch * display + aud * pitch + (1 | subj)
txt.lmer1M: duration ~ vis * display + pitch * display + aud * pitch + (1 + aud | subj)
txt.lmer2M: duration ~ vis * display + pitch * display + aud * pitch + (1 + aud + pitch | subj)
txt.lmer3M: duration ~ vis * display + pitch * display + aud * pitch + (1 + aud + pitch + vis | subj)
Df AIC BIC logLik Chisq Chi Df Pr(>Chisq)
txt.lmerM.p 12 27216 27287 -13596
txt.lmer1M.p 14 27168 27252 -13570 51.45 2 6.7e-12 ***
txt.lmer2M.p 21 26601 26727 -13280 580.77 7 < 2e-16 *** txt.lmer3M.p 26 26607 26762 -13278 4.17 5 0.52 --- Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 where *M means that method = 'ML', and then plot the complex pattern I believe best summarizes my data with CIs, but here I need to say that these were obtained using REML. All this is so unfamiliar to readers and my students that there is be resistance. They are accustomed to ANOVAs done with OLS in which effect E is tested against the E by subject interaction. So: how do I make the report of my data analysis best fit people's pre-conceived ideas of what such a report should look like, w/o misrepresenting what was done?

My (brief) reply: I would plot coefficient estimates graphically (see, for example, pages 306, 307, 312, 313, 328, 338, 341 of our new book). These graphs all look different, which indicates that I don’t have any systematic way of doing this yet. I’m hoping this will come, as a product of further research. Also, I don’t really like formal model comparisons using AIC, BIC, etc. (but, of these, I much prefer AIC because at least it has a direct interpretation in terms of predictive error). I like summarizing complex Anova-type models in graphical displays as shown in the Anova chapter of our book.