Shravan adds an index entry to our book:
In the index at the back, please add crossed varying intercepts and crossed random effects as an entry for the pilot data discussed on p. 289. That kind of structure is very common in psychology/psycholinguistics, as you probably know, and many people will be looking for crossed random factor specificiations in your book but won’t find it unless they know to look under non-nested models in the index.
Shravan also writes:
Also, I really think the code needs to be cleaned up to make it usable generally. It’s really a lot of work to try to figure out which parts are missing in each of the code files. I will give you a more structured example and some suggestions for improvement soon. You may lose a lot of impatient people if you don’t focus on clean code for the next edition of the book (conversely, you will gain a wider readership if you get the code cleaned up).
OK, OK, we’ll do it…
This is for Andrew Gelman, or anybody else with an opinion on this.
Psychologists are starting to take notice of the benefits of mixed effects models for analyzing experimental data. Like multiple regression analysis of experimental data, however, there seems to be some confusion about how to interpret the results.
Consider a common type of experiment, wherein a group of randomly selected subjects each encounter an identical set of words, and the subjects perform some type of speeded judgment on each word.
Standard practice for creating the set of words in this kind of experiment is to a) define 1-3 independent variables (say, the frequency of a word in the English language), b) split these variable(s) into discrete conditions (e.g., low vs. high), and match the means of several other potentially confounding variables (e.g., word length) across the cells of the independent variable(s). Importantly, the natural distributions of the confounding variables at each level of the independent variable are typically only partially overlapping and partially balanced. For example, high frequency words tend to be shorter in length than low frequency words.
The resulting data from these experiments are then typically analyzed with repeated measures analysis of variance (RM-ANOVA).
Compare this approach to the following. Another researcher is also interested on the effect of word frequency on reaction time. She collects a random sample of words, measured on the same variables, which serve as stimuli in an identical experiment. The resulting data are analyzed as follows. All variables (confounding variables and independent variables) are simultaneously entered into a crossed (i.e., non-nested) subject x word mixed effects model (i.e., random by-subject and by-word intercepts).
Now, most Psychologists would characterize the first approach as a controlled, randomized experiment. As such, they would be willing to make a causal inference about the effects of the independent variable on reaction time (or brain activity, or whatever). How about the second approach?
Ben Amsel
PhD Candidate
Dept. of Psychology
University of Toronto