Jeff pointed me to this paper by Brandon “not Larry” Bartels on using multilevel modeling for time series cross-sectional data. I agree with Bartels’s recommendations, which are:
– Use a multilevel model to allow intercepts to vary by groups. This is more reliable than estimating intercepts by least squares or not allowing the intercepts to vary at all.
– Also allow slopes to vary. (Bartels doesn’t emphasize this so strongly but I think this is important advice also.)
– Include as group-level predictors the group-level averages of important individual-level predictors. This will in many settings capture some of the otherwise unexplained group-level variation, as Joe Bafumi and I discuss.
Bartels also recommends representing individual-level predictors by their deviation from group averages. This is ok but I don’t think it’s necessary. It depends on the context. For example, if you have a predictor that is 1 if you’re African American and 0 otherwise, I wouldn’t want to subtract that from its state average. In that case you’d be better off including the individual predictor and state % African American as two predictors in the model. In other settings, Bartels’s recommendation to center the predictor for each group makes more sense. Either way, this doesn’t affect his main recommendation to fit a multilevel model, including important predictors in their group averages as well.
Individual and group-level predictors
Finally, I recommend my 2006 Technometrics paper, “Multilevel (hierarchical) modeling: what it can and cannot do,” which begins:
Multilevel (hierarchical) modeling is a generalization of linear and generalized linear modeling in which regression coefficients are themselves given a model, whose parameters are also estimated from data. We illustrate the strengths and limitations of multilevel modeling through an example of the prediction of home radon levels in U.S. counties. The multilevel model is highly effective for predictions at both levels of the model, but could easily be misinterpreted for causal inference.
In particular, see the discussion in Section 2.4 of my paper on the interpretation of a group-level predictor. You have to be careful about calling such coefficients “effects” or interpreting them causally.
The claims made on pp.12-13 of the paper seem to me to be misguided.
If the assumption that Cov(X,u)=0 is controversial (as Bartels accepts) then the assumption that Cov(Xbar,u)=0 must surely also be controversial – and if it is violated, then the tests for cluster confounding would seem to be invalid. Am I missing something here?
As an aside, the treatment of dynamics here also seems odd (though admittedly not a main concern of the paper). It's well-established that simply including a lagged dependent variable – suggested as standard practice by Bartels – gives biased coefficients for both OLS and FE (though the biases operate in different directions) and has led economists to prefer GMM approaches to estimating dynamic panel data models.