In the spirit of Dehejia and Wahba:
Three Conditions under Which Experiments and Observational Studies Produce Comparable Causal Estimates: New Findings from Within-Study Comparisons, by Cook, Shadish, and Wong.
Can Nonrandomized Experiments Yield Accurate Answers? A Randomized Experiment Comparing Random and Nonrandom Assignments, by Shadish, Clark, and Steiner.
I just talk about causal inference. These people do it. The second link above is particularly interesting because it includes discussions by some causal inference heavyweights. WWJD and all that.
Recalling Plato’s elephant in the cave, the emphasis in these empirical case studies of bias reduction should not be a serious assessment of how well we did – i.e. was it a snake or tree?
We only know if it’s a snake or a tree if we make abstract assumptions that imply either an (abstract) snake or tree and use mathematical analysis or simulation to work this out.
These empirical case studies can be immensely elucidating at many other things.
I created a simple model of Plato’s cave to draw attention the dangers apparent in some published advice David Cox gave many years ago about a possible rule for combining estimates from RCTs and observations studies – essentially he suggested one only combine them if the appeared similar enough. I presented it at a meeting in Montreal in 1997 and a number of times since and the only criticism I have received was from some one saying “I don’t like your model – I think it’s a stupid model” but they refused to elaborate any further.
It’s easily stated and displayed in this R simulation (lessthan needs to replaced with right symbol).
> bias=1;close.enough=.1;small.sd=.1;big.sd=3;rep=10000
> rx=rnorm(rep,0,big.sd)
> obs=rnorm(rep,bias,small.sd)
> mean(rx)
[1] 0.003723854
> mean(obs)
[1] 1.000040
> comb=(rx + obs)/2
> comb.if.close=comb[abs(rx – obs) lessthan close.enough]
> mean(comb)
[1] 0.5018817
> mean(comb.if.close)
[1] 1.005977
Contrary to David almost always giving good advice – for this model its really bad advice.
K?
p.s. this is also the model that lead me to George Casella's very practical 1992 paper on relevant subsets entitled Conditional Inference from confidence sets.
Some mangled history of philosophy here.
I guess the elephant in the cave is the one that was not in the room. So that's where it went!
But if there was an elephant in the cave the people in Plato's example who could only see shadows on the wall were sadly sense-deficient. Could they not hear it and smell it and feel it stomping up and down?
K?, you are better off with your favourite philosopher, Peirce.
Nick – you did catch me on a mangled memory from my undergrad days.
Someone adapted it, maybe Allan Bloom, and in the adaption there was a blind man in the cave who either felt the Elephant's leg or trunk and thought it was a tree or snake.
But it was minor analogy to draw attention to the human condition of never knowing the "realities" behind the appearances.
The major analogy (model) was given in the R program and does not require a good schooling in philosophy to see what it represents (or I hoped so).
Thanks for at least prompting me to glance at http://en.wikipedia.org/wiki/Allegory_of_the_Cave and its very unlikely I will read the Republic any time soon.
K?
I've heard the story of the elephant feeling like a tree, snake, wall, etc. in various versions. I can't give a first date, but I guess it long predates Allan Bloom and for all I know has very long roots in Eastern philosophy.
You're thinking of the story (usually presented as an Indian fable) of the five blind men.
This was brilliantly parodied by a New Yorker cartoon (wish I could find a copy) which had five blind men holding, respectively, a snake, a rope, a tree, a palm frond and a wall and all saying "It's an elephant."
Perhaps a little off topic but here's the whole thing:
1.
It was six men of Indostan
To learning much inclined,
Who went to see the Elephant
(Though all of them were blind),
That each by observation
Might satisfy his mind.
2.
The First approached the Elephant,
And happening to fall
Against his broad and sturdy side,
At once began to bawl:
"God bless me!-but the Elephant
Is very like a wall!"
3.
The Second, feeling of the tusk,
Cried: "Ho!-what have we here
So very round and smooth and sharp?
To me't is mighty clear
This wonder of an Elephant
Is very like a spear!"
4.
The Third approached the animal,
And happening to take
The squirming trunk within his hands,
Thus boldly up and spake:
"I see," quoth he, "the Elephant
Is very like a snake!"
5.
The Fourth reached out his eager hand,
And felt about the knee.
"What most this wondrous beast is like
Is mighty plain," quoth he;
"'Tis clear enough the Elephant
Is very like a tree!"
6.
The Fifth, who chanced to touch the ear,
Said: "E'en the blindest man
Can tell what this resembles most;
Deny the fact who can,
This marvel of an Elephant
Is very like a fan!"
7.
The Sixth no sooner had begun
About the beast to grope,
Than, seizing on the swinging tail
That fell within his scope,
"I see," quoth he, "the Elephant
Is very like a rope!"
8.
And so these men of Indostan
Disputed loud and long,
Each in his own opinion
Exceeding stiff and strong,
Though each was partly in the right,
And all were in the wrong!
Moral.
So, oft in theologic wars
The disputants, I ween,
Rail on in utter ignorance
Of what each other mean,
And prate about an Elephant
Not one of them has seen!
Look around and you'll find many spot on cartoons on this theme which was exactly what I did when stumbling in here.