Dan Kahan writes:
Here is a very interesting article form Science that reports result of experiment that looked at whether people bought a product (picture of themselves screaming or vomiting on roller coaster) or paid more for it when told “1/2 to charity.” Answer was “buy more” but “pay lots
less” than when alternative was fixed price w/ or w/o charity; and “buy more” & “pay more” if consumer could name own price & 1/2 went to charity than if none went to charity. Pretty interesting.
But . . .
What’s odd, I [Kahan] think, is the measure used to report the result. The paper (written by some really amazingly good social psychologists; I know this from other studies) goes on & on, w/ figures & tables, about how the amusement park’s “revenue,” “revenue per ride” & “profit” went up by large amount when it used “name your own price & 1/2 to charity.” Yet that result is dominated by random effects — the marginal cost & volume of sales are peculiar to the product being sold & have nothing to do w/ the experimental treatment.
This seems very very puzzling; I mean who exactly is supposed to find *this measure* edifying? A social scientist won’t–b/c again, the measures are dominated by random effects; one has to *remove* these to figure out how the 4 different options actually effected behavior! A business person might find the measures easy to interpret but will recognize instantly that he or she can’t draw any conclusions about the sorts of products that will benefit from this sort of pricing scheme, since all the arguments in the expected rate of return are specific to the product being sold not only is it impossible to figure out if this would be a good way to sell other products– a car, e.g., or a comic book– it’s not clear that “pay what you want w/ 1/2 to charity” was even profit-maximizing for the roller coaster operator, since it might well have made still more money if it lowered the “fixed price” w/o charity, an option that isn’t included in the experimental conditions.
In fact, if one reads carefully, one get a good bit of the information that is actually relevant, at least to social scientists, who presumably are ones reading Science: “The results of the two fixed $12.95 price conditions reveal low and similar purchase rates (0.50% without charity, 0.59% with). [Pay-what-you-want] substantially increased purchase rates to 4.49% in the PWYW + Charity treatment and to 8.39% in the simple PWYW treatment. In the PWYW treatments, buyers paid significantly more per photo when half of the revenues went to charity (M = $5.33 per photo) than with no charity [M = $0.92; t(3535) = 43.24, P < 0.001 (9)]. Figure 1 presents the resulting profits." So people didn't pay *nearly* as much when they could name price w/ 1/2 to charity as when fixed price was charged; but they bought a lot more often, & paid more, than they would have if there had been "name your own price" regime w/ no charity (seems like they were indeed charging way too much for a scream-or-barf photo). Or in other words, if you *bundle* good w/ contribution to charity, people are willing to pay more for it than if you don't bundle. That's not earth shattering news, although it certainly isn't what one would expect from very spare "rational choice" model, which would likely see bundling as adding no value & potentially imposing dead weight loss relative letting consumer buy the photo for less & donate difference to charity of his or her own choosing. I doubt, though, that this would have been as catchy a way to present the paper, which in fact has gotten a lot of coverage. Again, I think these are great social psychologists. But I'm not surprised that they are teaching people marketing at a business school!
It’s been awhile since anyone’s sent me an email with the word “barf.”
reading my "barf" msg I realize I actually breezed through the key sentence — the only one that it seems to me actually gives informatoin on what the experimental effect was — and didn't quite get the findings. There are 4 conditions: (1) fixed price (FP); (2) fixed price, proprietor donates 1/2 to specified charity (FP+C); (3) name-own-price (NOP; and (4) name-own-price, proprietor dontats 1/2 to specified charit (NOP+C). The result is that people almost never buy in FP or FP+C; they buy often for small price in NOP; and they buy less often but pay middling price (much less than FP) in NOP+C. (I misread & thought there was higher volume of sale in FP+C, not just higher price). Make of that what you will (it's something). But the finding that FP+C maximized profit — the big big big emphasis of paper, w/ lots of Figures & Tables — doesn't tell you anything b/c it is all a happenstance of elements of the design unrelated to the experimental treatment. Can't be sure, but it seems reasonable to think profit maximizing scheme would have been FP & no charity if the stupid amusement park charged a lower, reasonable price. Anyway, was struck that the authors (very good scholars, too) would pick what strikes me as an uninformative outcome measure (albeit one that makes for good headline, "LETTING CONSUMER PICK PRICE & GIVING HALF TO CHARITY MAXIMIZES BUSINESS's PROFIT!!!!"– & that reviewers & Science apparently thought that was okay.
Its been my experience that many of the papers published by the "top tier" scientific journals in my field (ie crystallography published in Science) are somewhat lacking in rigor. I'm not sure whether this is due to non-experts doing the research (not relevant to the above article), under-informed reviewers, a rush to publish resulting in taking short-cuts, or another reason. My guess is that this is often the case and mainly recognized by researchers reading articles in their own field — others (and the media) probably won't recognize some of the failings.
I think if I understand Dan correctly, he is suggesting a follow-up experiment with the fixed price being determine by the previous experiment with pay what you want + 1/2 goes to charity. Call this 2c.
We could find out whether a)c or b) 2c and half going to charity makes any difference.
I am not sure that I understand the criticism about whether this effect will work other products. I read the authors to be making a framing point – you don't know what the worth of a product to you is, but if it were framed as a purchase towards your favourite charity, then "problem" of determining its value is now given to you.
Just to make sure I understand the criticism — the complaint is that each treatment was assigned to the roller coaster for a span of two days, and different days might just have different types of people. So really we should treat this as more like N=2 per treatment (with respect to the "random effects" of different days) than N=30,000 (people).
Except, that doesn't sound like what Kahan is saying.
Yet that result is dominated by random effects — the marginal cost & volume of sales are peculiar to the product being sold & have nothing to do w/ the experimental treatment.
Huh? The marginal cost is constant across treatments, and the volume of sales of course depends on the experimental treatment (but also, potentially, the day-to-day random effects). So if you run a good experiment, then you could learn the Revenue, Revenue per Ride, and Profit from randomly assigning people to treatments. Moreover, they seem like relevant and interesting measurements to me, even if we might not think that any of the results generalize outside of this particular environment.
So, I'm confused.
Michael and Alex — I think these are the points here:
First, the headlines/punchlines are over-claiming by extrapolating the findings for roller coaster pictures to a general statement about profit maximization. There's little reason to suspect that goods that are less overpriced, or goods that are larger investments, or goods that are purchased by a small-but-dedicated group, or goods that are necessities, or goods that are purchased regularly, etc. etc., will experience the same experimental effect as the roller coaster pictures. The experiment plays on a pretty specific purchasing model but makes somewhat general-sounding conclusions. (Simply: If you drew caricatures on the street for money, would you switch to PWYW + 1/2 Charity? Maybe. If you sold cars, would you? If you do, please let me know!)
Relatedly, the study's results could be explained by an unwisely inflated fixed price for the photos. We have no idea from this experiment alone whether you would get the same effects if the photos initially were priced to maximize profits. Maybe if you just fixed the photo price at $8 or $2 instead of $13, you'd do much better than you would in any of these experimental situations! All they show is that for roller coaster photos, their SSR strategy beat the one they were using before (and a couple other possibilities).
DK, please correct me if I've misrepresented.