Sensitivity Analysis of Joanna Shepherd’s DP paper

Just a quick note on what we’re doing with Shepherd’s paper, and why…

There’s a long standing economics literature (beginning with Ehrlich in 1975) on the question of whether the death penalty deters murders. Since the death penalty is used in some states and not others, and has been used to differing extents over the years, it tends to be treated as a natural experiment.

However, capital punishment is not implemented at random — be it the political climate, current crime rates, or inherent differences across states, there’s something that drives some states to legalize the death penalty, and others not to. Furthermore, the “deterrent effect” of the death penalty varies by state and time, making it difficult to make a single causal claim of deterrence or the lack thereof.

Since there are such differences across states and time, the question of deterrence may lend itself to a multilevel model. Most modern papers look at data by state and year, and a multilevel model allows some flexibility in the appropriate level of aggregation.

Joanna Shepherd’s recent paper (available on the Wiki) uses a series of equations to predict murder rates by county and year, as a function of deterrent (and other) measures, and predict the probabilities of arrest, death sentences, and executions, based on murder rates and a number of other factors.

She finds that in some states, executions “deter” crime, in others there is no effect, and in still others, executions “cause” crime. We plan to test the sensitivity of these findings to changes in model specifications.

We don’t have her data yet; at the moment I’m using Stata to play with some state-level data to try and get a handle on her model. The county-by-year data might add more to a multilevel model than state-level data, since the death penalty is legalized by state, but deterrent effects may be felt more locally.

We’re also planning to test some of the other model specifications — the reliance on publicity, differences between high-execution and low-execution states, and so forth.

I’m meeting with Jeff on Friday, who has some ideas of what we should be testing, and will relay that conversation…

4 thoughts on “Sensitivity Analysis of Joanna Shepherd’s DP paper

  1. Sam commented:

    Some ideas from the economics literature might be relevant here. Alberto Abadie, a professor at the Kennedy School at Harvard, has done work on estimating the effects of terrorism in the Basque country by constructing "control" regions from other parts of Spain. The idea is to use a weighted average of other regions of Spain so that the Basque country and the constructed "control" region look similar with respect to covariates such as population density, income, education, etc. Sort of a small-sample alternative to propensity scores. Maybe something similar could be done to construct control states to compare with states that have the death penalty. The Abadie paper is Abadie and Gardeazabel (2003) "The Economic Costs of Conflict: A Case Study of the Basque Country", The American Economic Review, pgs. 113-132.

  2. Andrew commented:

    To get back to the multilevel modeling issue: we'll fit a varying-intercept, varying-slope model:

    y_st = alpha_s + theta_s*T_st + (X beta)_st + gamma_t + error,

    where

    s = state

    t = time (year)

    T_st = the "treatment" predictor corresponding to the death penalty (e.g., existence of the d.p., or #executions in the previous year, or whatever Shepherd uses)

    theta_s = the estimated "effect" of executions in state s

    (X beta)_st = the linear predictor for everything else in the model Shepherd is controlling for

    gamma_t = otherwise-unexplained time variation

    Including the treatment effect as theta_s rather than simply theta is the varying slope which should correspond to Shepherd's finding that executions deter crime in some states and cause it in others.

    We suspect that including all these varying coefficients, which capture state-level variability, will increase standard errors. Perhaps the results are not actually statistically significant when considering the moderate sample size of only 34 states with executions.

    In addition, as Jeff pointed out, there are difficulties with using executions as a causal predictor since they are "endogenous" as the economists say. Presumably Shepherd and others have thought about this issue also.

  3. Amanda commented:

    Thanks for the thoughts —

    Andrew, I think one of the most interesting things that a multilevel model might add to this data in particular is that we have information by county. So we (theoretically; have not seen the data yet) can predict

    y_ct = alpha_s + theta_s*T_st + (X beta)_ct + gamma_t + delta_c + error

    This is a step beyond what we had with Mocan's data, because we'll now have more opportunities among which pooling can occur. (Because if I remember correctly from what we did over the summer, most of the state stuff was not being pooled very much, but I'd imagine there's a lot of pooling that happens from county to state.)

    I'll take a look at the data today and see what we've got…

    And Sam, I will check out that article. I was thinking some sort of propensity score view might be interesting…

  4. Jeff commented:

    Unfortunately, endogeneity is something that Shepherd has some difficulty incorporating into her analyses. Don't know why.

    Sam's idea was subjected to a limited test by Dick Berk. Using Mocan's data, he did some rough case-control comparisons of death penalty states of similar size and with similar murder rates, but with (highly) varying execution rates: TX-CA, FL-PA, and one or two other comparisons. He finds that the deterrent effects disappear when such comparisons are incorporated.

Comments are closed.