Michael Bader writes:
What is the best way to examine interactions of independent variables in a propensity weights framework? Let’s say we are interested in estimating breathing difficulty (measured on a continuous scale) and our main predictor is age of housing. The object is to estimate whether living in housing 20 years or older is associated with breathing difficulty compared counterfactually to those living in housing less than 20 years old; as a secondary question, we want to know whether that effect differs for those in poverty compared to those not in poverty. In our first-stage propensity model, we include whether the respondent lives in poverty. The weights applied to the other covariates in the propensity model are similar to those living in poverty compared to those who are not. Now, can I simply interact the poverty variable with the age of construction variable to look at the interaction of age of housing and poverty on breathing difficulty? My thought is no — in order to get the true interaction, I would need to have the propensity model estimate the joint probability of being poor and living in older housing (as well as the other three cells in the 2×2 table) because without doing this, the main effect of being poor is being washed out by the propensity weights in the first step. On the other hand, if the weights are comparable across the model when we stratify on poverty, I’m not sure whether it will have much of an effect. Or, I could be totally incorrect and running the interaction with the poverty variable is sufficient.
I [Bader] am happy to read up on the subject; but when I tried doing a search, all I could find were debates about adding interactions into the propensity model itself, not looking at interactions of separate independent variables in the model.
My reply:
I don’t think it’s a good idea to frame this in terms of weights or weighting. I think of propensity scores as just one particular method for the more general problem of constructing similar groups in a treatment/control comparison. (See chapter 10 of ARM for further discussion of this point.) In the example you describe above, you could compare people who lived in housing 20 years older to people who lived in more recent housing, matching on other variables including their previous poverty status. Then you can include the relevant interactions in your model. The whole propensity-weighting thing seems like a distraction from your real goals here.
I'm not clear on the "weights" language, but the examples I've seen of "interactions" in propensity score analysis simply entirely separate matches and comparisons within each group. Divide the sample into n many cells as dictated by the interaction(s); run n matching procedures; compare outcomes within each cell.
In this case, that would be two separate propensity score analyses. The first takes those in poverty, predicts old housing, matches groups, compares breathing. The second is the same, but for those not in poverty. Unless you care about the main effect of poverty on breathing (and not just its moderation of housing age), there's no reason to try to estimate the probability of poverty.
So, for example, King, Massoglia, and MacMillan (2007, Criminology) estimated the treatment effect of marriage on crime separately for men and women. One analysis for men, another for women. Available here: http://www.udel.edu/soc/faculty/parker/SOCI836_S0…
Harding (2003, American Journal of Sociology) had one binary moderating variable of interest (race: black vs. nonblack) as well as a ternary treatment (poverty: low, medium, or high)… so that came out to six separate propensity matches and comparisons: black low vs. medium, black medium vs. high, black low vs. high, non-black low vs. medium, etc. Available here: http://www.jhsph.edu/bnrc/Research_Libr/Harding20…
…but I don't really know what is meant by "propensity weights" here, so maybe I'm talking about a totally different kind of analysis.
Sam-
What if you want to compare, say, black low with non-black low? I think that's the kind of interaction Michael is looking for…
Mike
I think Sam nails it here. So long as the propensity scores create balance across treatment conditions at levels of the background characteristic, you are fine. Sam's stratification method is a way to do it without making assumptions about how other things may be the same or differ depending on the background covariate of interest. If that results in sparseness problems, you can start thinking about ways to model the propensity scores with only some interactions with the background characteristic, rather than the fully interacted model, which is equivalent to the stratification approach. (I am using background characteristic/covariate to refer to the particular covariate that you want to interact with the treatment.)