Last year we discussed an important challenge in causal inference: The standard advice (given in many books, including ours) for causal inference is to control for relevant pre-treatment variables as much as possible. But, as Judea Pearl has pointed out, instruments (as in “instrumental variables”) are pre-treatment variables that we would not want to “control for” in a matching or regression sense.
At first, this seems like a minor modification, with the new recommendation being to apply instrumental variables estimation using all pre-treatment instruments, and to control for all other pre-treatment variables. But that can’t really work as general advice. What about weak instruments or covariates that have some instrumental aspects?
I asked Paul Rosenbaum for his thoughts on the matter, and he wrote the following:
In section 18.2 of Design of Observational Studies (DOS), I [Rosenbaum] discuss “seemingly innocuous confounding” defined to be a covariate that predicts a substantial fraction of the variation in treatment assignment but without obvious importance to the outcomes under study.
The word “seemingly” is important: it may not be innocuous, but only seem so. The example is drawn from a study (Silber, et al. 2009, Health Services Research 44: 444-463) of the timing of the discharge of premature babies from neonatal intensive care units (NICUs). Although all babies must reach a certain level of functional maturity before discharge, there is variation in discharge time beyond this, and we were interested in whether extra days in the NICU were of benefit to the babies who received them. (The extra days are very costly.) It is a long story, but one small part of the story concerns two “seemingly innocuous covariates,” namely the day of the week on which a baby achieves functional maturity and the specific hospital in the Kaiser family of hospitals. A baby who achieves maturity on a Thursday goes home on Friday, but a baby who achieves maturity on Saturday goes home on Tuesday, more or less. It would, of course, be ideal if the date of discharge were determined by something totally irrelevant, but is it true that day-of-the-week is something totally irrelevant?
Should you adjust for the day of the week? A neonatologist argued that day of the week is not innocuous: a doc will keep a baby over the weekend if the doc is worried about the baby, but will discharge promptly if not worried, and the doc has information not in the medical record. Should you adjust for the day of the week? Much of the variation in discharge time varied between hospitals in the same chain of hospitals, although the patient populations were similar. Perhaps each hospital’s NICU has its own culture. Should you adjust for the hospital?
The answer I suggest in section 18.2 of Design of Observational Studies is literally yes-and-no. We did analyses both ways, showing that the substantive conclusions were similar, so whether or not you think day-of-the-week and hospital are innocuous, you still conclude that extra days in the NICU are without benefit (see also Rosenbaum and Silber 2009, JASA, 104:501-511). Section 18.2 of DOS discusses two techniques, (i) an analytical adjustment for matched pairs that did not match for an observed covariate and (ii) tapered matching which does and does not match for the covariate. Detailed references and discussion are in DOS.
"We did analyses both ways, showing that the substantive conclusions were similar"
This seems, to me, to be the easy case. I think you should always try the analysis several different ways and see what the impact of changing approaches is (sensitivity analysis). What I find much more challenging is what to do when different approaches give different results.
I always seem to end up having to pick one approach and argue for it using substantive reasons. I am never as happy trying to pick the best estimate that we can given the data when there are several competing estimates that would lead to different interpretations. Even showing all of them seems confusing more than anything else.
Just uncertainty about the least wrong model to assume/stick with?
K?
This entire conversation seems to be entirely backwards. If the goal is to determine the causal effect of longevity of stay in the NICU on future health outcomes of the baby, isn't the "confounding factor" of which day of the week the patient was released actually a valid instrument for identifying this causal effect? If the story above is true, then patients who are discharged on monday or tuesday are more likely to have stayed a day or two extra in the NICU (unrelated to actual needs of care). Therefore, if there is a causal effect of extra days on health outcomes, we should see future health outcomes positively correlated with monday/tuesday discharge as opposed to thursday/friday discharge. A simple IV estimator would recover the causal effect (where day of the week is the excluded instrument). Instead, the authors in the study (Silber et al) utilize a matching approach that cannot separate potentially unobserved information that the doctors use in their decision to retain children in the ICU.
Nathan: I am always careful with proposed instruments where we can't be sure that the instrument is really independent of the outcome. Day of the week is tricky as the MD is making a decision (based on patient information) as to when to discharge. These could make the IV assumptions suspect.
That is not to say that I might not do the analysis and see what it yields but that I'd interpret with caution.
On using this as instrument, here a related paper by Joe Doyle and Doug Almond
After Midnight: A Regression Discontinuity Design in Length of Postpartum Hospital Stays
"…Our identification strategy exploits insurance rules that reimburse a predetermined number of "days" in the hospital, counted as the number of midnights in care. A newborn delivered at 12:05 a.m. will have an extra night of reimbursable care compared to an infant born minutes earlier… The results suggest that for uncomplicated births, extended minimum insurance levels are associated with substantial costs without an improvement in major health problems."
http://www.mit.edu/~jjdoyle/After_Midnight.pdf
A revision to Nathan's point: I thought the excluded instrument should be "day of functional maturity", not day of discharge. Day of discharge probably violates the exclusion restriction (since Monday or Tuesday releases seem to be caused in part by unobserved "reasons for worry"). But "day of functional maturity" would seem to satisfy the exclusion restriction. That being the case, then I agree that we want that as an excluded instrument. Matching may still serve a purpose though to the extent that it makes the exclusion restriction more plausible (e.g., by having us restrict attention to Thursday and Saturday day of functional maturity babies). Or maybe that wasn't measured? Or, is day of functional maturity also subject to some judgment call or manipulation?