December 2007 Archives

In writing an entry motivated by Justin Wolfers's article on prediction markets, I had another thought, which was a bit tangential to my main point (a poll is a snapshot, not a forecast, etc.), so I'm putting it separately here.

Justin writes,

It is the accuracy of market-generated forecasts that led the Department of Defense to propose running prediction markets on geopolitical events. While political rhetoric about "terrorism futures" led the plug to be pulled on that particular experiment . . .

I don't know exactly why the plug was pulled on that program, but I seem to recall that it was being run by convicted criminal John Poindexter--a guy who was actually involved in what were arguably terrorist activities (the project of secretly sending weapons to Iran in the 1980s). So, yeah, maybe we should be a bit suspicious! Labeling this as "political rhetoric" dodges the real concerns about this program.

Beyond concerns about foxes guarding the henhouse, etc., even the "good guys" (however you define them) are subject to all sorts of cognitive biases, and I don't know that I want somebody in a position of power being in the position to make money if there is a terrorist attack. Even if it's small amounts of money, this sort of thing can affect people's judgment. Maybe it would affect people's judgment in a good way, I don't know, but it's certainly not obvious to me that the "terorrism futures" thing is a good idea.

Here's an article by Bob Erikson and Chris Wlezien on why the political markets have been inferior to the polls as election predictors. Erikson and Wlezien write,

Election markets have been praised for their ability to forecast election outcomes, and to forecast better than trial-heat polls. This paper challenges that optimistic assessment of election markets, based on an analysis of Iowa Electronic Market (IEM) data from presidential elections between 1988 and 2004. We argue that it is inappropriate to naively compare market forecasts of an election outcome with exact poll results on the day prices are recorded, that is, market prices reflect forecasts of what will happen on Election Day whereas trial-heat polls register preferences on the day of the poll. We then show that when poll leads are properly discounted, poll-based forecasts outperform vote-share market prices. Moreover, we show that win-projections based on the polls dominate prices from winner-take-all markets. Traders in these markets generally see more uncertainty ahead in the campaign than the polling numbers warrant—in effect, they overestimate the role of election campaigns. Reasons for the performance of the IEM election markets are considered in concluding sections.

I was motivated to post this after reading Justin Wolfers's Wall Street Journal article, "Best Bet for Next President: Prediction Markets," where he writes, "Experimental prediction markets were established at the University of Iowa in 1988, and they have since amassed a very impressive record, repeatedly outperforming the polls."

As I wrote a couple of years ago,

Prediction markets do a good job at making use of the information and analyses that are already out there--for elections, this includes polls and also the information such as economic indicators and past election results, which are used in good forecasting models. The market doesn't produce the forecast so much as it motivates investors to find the good forecasts that are already out there.

As an aside, people sometimes talk about a forecasting model, or a prediction market, "outperforming the polls." This is misleading, because a poll is a snapshot, not a forecast. It makes sense to use polls, even early polls, as an ingredient in a forecast (weighted appropriately, as estimated using linear regression, for example) but not to just use them raw.

But . . .

That said, I like Justin's work a lot, including his paper with Zitzewitz on prediction markets. (And I'm a big fan of the idea of betting--we have an example of football point spreads in chapter 1 of Bayesian Data Analysis.) I think Justin's Wall Street Journal article is just fine--I understand that it's necessary to simplify a bit to reach a general audience amid space limitations. I'm just wary of overselling them or of misunderstanding of what is learned from polls.

P.S. More discussion (including comments from Wolfers and Erikson) here.

Daniel Lippman sent me this article, which states, "A dose of perspective for the poll junkies out there: 45 percent of Americans say they 'have not read or heard anything about public opinion polls about the upcoming presidential election.' That's according to a new poll (what else?) from AP and Yahoo!"

The circularity of this reminded me of a study I heard about a few years ago (when reviewing an article for a journal, I think it was Public Opinion Quarterly) where people were polled and asked, "How many times were you surveyed in the past year?" That particular study was trying to get at the idea of the country being divided between "professional survey participants" who actually answer these surveys, and the rest of us who just hang up.

I'm one of the hanger-ups myself, but I justify it by feeling that this just creates more job opportunities for statisticians who analyze missing data. Anyway, I think there are too many polls.

It's 27 identical blocks, each of which is 4x5x6, to be placed inside a 15x15x15 box. 24<25, and so it should be possible to put all the pieces in the box with room to spare. And indeed it is possible, but it's tough--it took me a couple hours to figure out how to arrange the pieces. I think it's just a beautiful puzzle because all the pieces are identical.

Some guy showed me this puzzle once--he pointed out that the 2-D version (with 4 identical rectangles) is trivial and he claimed that the 4-D isn't hard but that he didn't know if the 5-D version had a solution. Anyway, 3-D is hard enough for me.

I'm sure someone manufactures it, but I don't know who, so I took a board and sawed it into 27 little pieces one day and made my own version, where it sits in my office inside a little plastic box. If I had one of those digital cameras, and if I were in my office, I'd post a picture of it here.

P.S. It's only the second-coolest puzzle because Rubik's cube is the coolest.

Aleks pointed me to this new product by the people behind the Baby Name Wizard. Here's the description:

How do we go from name inspirations to new name ideas? Nymbler combines human expertise and artificial intelligence to sift through thousands of names and find the ones that fit your taste.

Laura Wattenberg is a name expert and the author of The Baby Name Wizard. Her specialty is analyzing name style-sorting out the dozens of influences that might distinguish Marvin from Mason or Daphne from Darlene. Through years of research she has compiled a huge portfolio of name information. Each name is categorized on everything from ethnicity to historical popularity to soap opera appearances.

Icosystem is a Cambridge, Massachusetts technology company. Their Hunch EngineTM solves the dilemma of searching "when you don't really know what you are looking for, but you'll know it when you find it." The Hunch EngineTM is a smart system based on a genetic algorithm. Given information on options and taste, it identifies subtle patterns and makes personalized suggestions.

Together, these two experts set out to create an intelligent baby-naming tool. They trained the Hunch EngineTM on Laura's storehouse of name data and analysis. They taught it to understand names, to pick up on the trends and style cues that you'd notice yourself-and maybe some you wouldn't.

The result is Nymbler, a unique personal naming assistant. It's an expert system that learns about your taste and helps guide you to new ideas. We hope you'll enjoy exploring this name landscape as much as we do.

I wonder if they could get some data on names of parents and children, or names of siblings. This would give correlation info that could make things really interesting.

What's the deal with Drazen Prelec's "Bayesian Truth Serum"? There's something about this catchy name that makes me suspicious, but he and his colleagues claim to have results:

The Bayesian ‘truth serum’ (BTS) is a scoring method that provides truthtelling incentives for respondents answering multiple-choice questions about intrinsically private matters (opinions, tastes, past behavior). The method requires respondents to supply not only personal answers but also percentage estimates of how other respondents will answer that same question. The scoring formula then assigns high scores to answers whose actual frequency is greater than their predicted frequency. It is a theorem that truthful answers maximize expected score, under fairly general conditions. We describe four experimental tests of the truthtelling property of BTS. In surveys with varied content (personality, humor, purchase intent) we assess whether some identifiable category of respondents would have attained a higher score had they engaged in a systematic deception strategy, i.e., given a non-truthful answer according to some algorithm. Specifically, we test whether respondents would have achieved a higher score had they replaced their actual answer with the answer that they believe will prove the most popular (or least popular); whether they would have done better by misreporting their demographic characteristics (gender); and whether they would have done better by simulating the answers of some other person “that they know well.” We find that all types of deception are associated with substantial losses in score, for the majority of respondents. Hence, BTS can function as a truth-inducing scoring key even in settings where only the respondent only knows the actual truth.

I like the idea of trying to harness the "wisdom of crowds" in an anticipatory way. I really should look into this and try to understand exactly what's going on.

The Information Pump

Prelec certainly has a knack for naming things!

Following up on this entry, Ubs writes,

It's not always an obvious call who counts as a "major nominee", particularly in the multi-candidate races and in the early elections before the 12th Amendment.

I'm counting three candidates in 1912, only Washington in 1792, two candidates in every other race, and no one at all in 1789. In 1856 I'm counting Breckenridge but not Douglas, and in 1824 I'm excluding both Crawford and Clay. I think that's cutting it pretty tight, but even so I still get 106 nominees total, so I have no idea how she gets it down to 105 -- especially considering that her "215 years" implies she does count Washington in 1789, and the mention of Strom Thurmond suggests she's counting him, too, neither of which I count.

Anyway, here's my figures. First list counts candidates once per election; second list counts them once per person.

James: 11
John: 11
William: 11
George: 7
Thomas: 7
Franklin: 5
3 each: Andrew, Charles, Richard, Stephen
2 each: Abraham, Adlai, Alfred, Benjamin, Dwight, Henry, Herbert, martin, Ronald, Theodore, Ulysses, Winfield
1 each: Albert, Alton, DeWitt, Gerald, Horace, Horatio, Hubert, Lewis, Lyndon, Michael, Robert, Rufus, Rutherford, Samuel, Walter, Warren, Wendell, Zachary
Total: 106

James = 8
John = 8
William = 5
George = 5
Thomas = 3
2 each: Alfred, Charles, Franklin, Winfield
1 each: Abraham, Adlai, Albert, Alton, Andrew, Benjamin, DeWitt, Dwight, Gerald, Henry, Herbert, Horace, Horatio, Hubert, Lewis, Lyndon, Martin, Michael, Richard, Robert, Ronald, Rufus, Rutherford, Samuel, Stephen, Theodore, Ulysses, Walter, Warren, Wendell, Zachary
Total: 68

Either way, her 1/3 figure for James + John + William + George holds with room to spare.

I'm counting Stephen Grover Cleveland (x3), Thomas Woodrow Wilson (x2), and John Calvin Coolidge (x1) by their actual first names, by the way.

Amusing trivia: In 1916 a Thomas-Thomas ticket ran against Charles-Charles.

I'll answer just about anybody's question for free. Then I'll post the question and answer on the blog (after checking that it's ok with you). Since I've done it for free, my answer is open-source, and I'd like to share it with as many people who might be interested. I also work on projects that are publicly funded, or help people who are working on such projects. Again, your tax dollars already paid for this, so I'll post on the blog as appropriate. Finally, sometimes I'll consult for money, in which case you can tell me if you don't want me to immediately spread our findings to the world in this way.

Frederick Crews is writing about selective serotonin reuptake inhibitors" (SSRIs):

Hence the importance of David Healy's stirring firsthand account of the SSRI wars, Let Them Eat Prozac. Healy is a distinguished research and practicing psychiatrist, university professor, frequent expert witness, former secretary of the British Association for Psychopharmacology, and author of three books in the field. Instead of shrinking from commercial involvement, he has consulted for, run clinical trials for, and at times even testified for most of the major drug firms. But when he pressed for answers to awkward questions about side effects, he personally felt Big Pharma's power to bring about a closing of ranks against troublemakers. That experience among others has left him well prepared to puncture any illusions about the companies' benevolence or scruples.

. . .

The most gripping portions of Let Them Eat Prozac narrate courtroom battles in which Big Pharma's lawyers, parrying negligence suits by the bereaved, took this line of doubletalk to its limit by explaining SSRI-induced stabbings, shootings, and self-hangings by formerly peaceable individuals as manifestations of not-yet-subdued depression. As an expert witness for plaintiffs against SSRI makers in cases involving violent behavior, Healy emphasized that depressives don't commit mayhem. But he also saw that his position would be strengthened if he could cite the results of a drug experiment on undepressed, certifiably normal volunteers. If some of them, too, showed grave disturbance after taking Pfizer's Zoloft—and they did in Healy's test, with long-term consequences that have left him remorseful as well as indignant—then depression was definitively ruled out as the culprit.

Healy suspected that SSRI makers had squirreled away their own awkward findings about drug-provoked derangement in healthy subjects, and he found such evidence after gaining access to Pfizer's clinical trial data on Zoloft. In 2001, however, just when he had begun alerting academic audiences to his forthcoming inquiry, he was abruptly denied a professorship he had already accepted in a distinguished University of Toronto research institute supported by grants from Pfizer. The company hadn't directly intervened; the academics themselves had decided that there was no place on the team for a Zoloft skeptic.

That doesn't make the research institute look so good, although maybe there's another side to the story.

Hey, did he just say what I think he said???

Crews continues:

Undeterred, Healy kept exposing the drug attorneys' leading sophistry, which was that a causal link to destructive behavior could be established only through extensive double-blind randomized trials—which, cynically, the firms had no intention of conducting. In any case, such experiments could have found at best a correlation, in a large anonymous group of subjects, between SSRI use and irrational acts; and the meaning of a correlation can be endlessly debated. In contrast, Healy's own study had already isolated Zoloft as the direct source of his undepressed subjects' ominous obsessions.

Thanks partly to Healy's efforts, juries in negligence suits gradually learned to be suspicious of the "randomized trial" shell game. . . .

I agree that randomized trials aren't the whole story, and I'll further agree that maybe we statisticians overemphasize randomized trials. But, but, . . . if you do do a randomized trial, and there are no problems with compliance, etc., then, yes, the correlation does imply causation! That's the point of the randomized design, to rule out all the reasons why observational results can be "endlessly debated."

The New York Review of Books needs a statistical copy editor! I don't know anyone there (and I don't know Crews), but maybe someone can pass the message along. . . .

P.S. Maybe I'm being too hard on Crews, who after all is a literary critic, not a statistician. I assume he wrote this thing about correlation and causation because he misinterpreted what some helpful statistician or medical researcher had to say. Sort of like how I might sound foolish if I tried to make some pronouncement about Henry James or whatever.

P.P.S. Typo fixed (thanks, Sebastian).

Joe Bafumi and Michael Herron write,

We consider a fundamental question about the elected American political institutions: do they work? . . . Any given set of democratic institutions may aggregate preferences fairly—i.e., the associated aggregation process yields an outcome that reflects an appropriately designated representative constituent—or it may fail to do so—i.e., the aggregation process leads to distortion between its outcome and a representative constituent. Thus, to discern whether the elected American political institutions fairly aggregate preferences, we must address the question, who precisely is represented by these institutions and, importantly, is this individual representative of Americans writ large?

Here's what they find:

The scale has liberals on the left (the negative numbers) and conservatives on the right (the positive numbers). As is well known, voters tend to be more moderate than representatives, but the median of the voters is not far from the median of the current House and Senate. (Joe and Michael unfortunately ignored my advice and labeled the congresses by number rather than year.)

Joe and Michael made the graph by taking a large national survey and putting in questions asking about attitudes on a bunch of issues that were also voted on by the House and Senate. They then used an ideal-point model (see, for example, chapter 14) to line up voters and congressmembers on a common scale. They also did some adjustment to match the sample to the general population. Good stuff.

Questions about the distribution of voters

Getting distributions of congressmembers is standard now (Poole and Rosenthal, etc), but getting voters on the same scale is new. I just have two questions about those cool distributions of voters.

First, I wonder about the bimodality. There seem to be two things going on. On issue attitudes, voters are basically unimodal, with more people in the center and some in the extremes. On party identification, voters are bimodal, with many strong Democrats and strong Republicans. Bafumi and Herron put this all together and end up with a bimodal distribution, but I wonder how sensitive this is to their particular methods.

Second, I wanted to point out the asymmetry in their graph. According to the analysis, something like 20% of the people are more liberal than the median Democratic congressmember, but only about 5% are more conservative than the median Republican congressmember. An some basic level, this is hard for me do believe, but I suspect it has to do with the issues that congress votes on. It would be interesting to see this broken down, issue by issue.

The seats-votes curve

Regarding the point in the paper about 2006, it's worth noting that, for various reasons (including incumbency, gerrymandering, and simple geography), congressional elections have shown a big partisan bias in the seats-votes curve in favor of the Republicians. (Before then there was a bias in favor of the Democrats).

So this explains part of what they found, I think.

See Figure 1 in this paper (to appear next year in PS): From 1996 through 2004, the Dems were getting 50% or more of the vote just about every year but getting clearly less than 50% of the seats.

See Figure 2 of that paper for estimated seats votes curves since 1958.

Even in 2006, the Dems didn't get their fair share of the seats (compared to what the Reps would've gotten with that vote share).

The Bafumi and Herron paper is great but I think it would be strengthened by including the role of the seats-votes curve.

OK, OK, I had to say it . . .

Some minor comments:

Figures 5,6: Please, please, please don't order these states alphabetically!!! It would be much more informative to order them from most conservative to most liberal. Also, I'd suggest putting the two graphs side by side on the same page. Similarly with Figures 7,8.

Similarly, Fig 9 should not be alphabetical either!!!!!!!!!!!!!!!!!!!!!!!!!! Other than that, though, the pictures are really pretty.

And don't get me started on the tables.

Aleks sent along this, which raises interesting statistical as well as legal, economic, ethical, political, etc. questions.

I'm a sucker for this sort of silly thing:

Maybe we can look at income and voting within each of these regions.

Aleks points to two papers that might be relevant for our statistical computing efforts: state-of-the-art optimization for laplace-prior logreg and bayesian regularization vs leave-one-out.

Just as a reminder, here's our project that relates to the above topics:

Aleks labels this "if clicks were votes," but maybe it would be more accurate to label it "clicks aren't votes" . . . .

Here are some graphs Yu-Chieh made from data from the National Election Studies, extending Manza and Brooks's analysis of voting by occupation up to 2004.

Within each occupatoin class, we're plotting Republican presidential vote (relative to the national mean each year); you can see some striking trends, with professionals (doctors, lawyers, teachers, etc) going to the Democrats and business owners going to the Republicans, among other patterns.

Our next step is to throw income into the analysis and see income tracks with Republican vote more in some occupation categories than others. (A quick analysis found the difference in voting patterns of rich and poor to be similar in the different occupation categories.)

P.S. There was some problem with the coding for 1972. We'll have to go back and see what we did wrong for that year. (I'm hoping the other years are ok. They're roughly consistent with what Manza and Brooks found, so I expect they're fine.)

In explaining why she picked "Barack" as the 2007 Name of the Year, Laura Wattenberg spits out the following stunner:

In 215 years of American electoral history encompassing 105 major nominees, the overwhelming majority of candidates have had traditional English names. In fact, the names George, James, John and William alone account for more than a third of all nominees.

She continues:

Some of you will remember that a few months ago this blog mentioned Errol Morris's New York Times article about two famous old photographs, both of which show the same stretch of road on the Crimean peninsula. One photograph shows the road covered with cannonballs, with additional cannonballs strewn around the ground on both sides of the road; the other shows the road clear of cannonballs. As Morris discusses, it has long been assumed that the photo with the clear road --- the "off the road" picture --- was taken first, and that the photographer and his crew then moved a bunch of cannonballs onto the road to take the "on" picture. In his article, Morris questioned whether this ordering was in fact correct.

As Morris discusses in another article, the traditional wisdom was in fact correct: "Off" came first. This can be determined pretty conclusively by looking at the cannonballs that are lying around on the ground: many of them have shifted position slightly, and in every case they are slightly farther downhill in the On photo than in the Off photo. The only story that makes sense is that the Off photo was taken, and then these cannonballs were disturbed (presumably by the photographer and his team

Before the answer was known for sure, Morris asked his readers to send in their opinions and reasons. In a new article, Morris summarizes the reasons, using some of the worst statistical graphics I have seen in 2007 (it's worth taking a look). And he likes them (the graphics, I mean)!

If anyone would like to make a better display, here are Morris' data. (Sorry, he doesn't really discuss what the reasons mean, so you'll just have to work with what's here). The first line is a header line; subsequent lines give the reason, the number of people who cited this reason in describing why they think "On" came first, and the number who cited this reason in describing why "Off" came first. ("Off" is the right answer).

Reason,On,Off
Shadow,149,23
Gravity,3,5
# and Position,155,75
Camera/Exposure,20,10
Topography/Climate,33,17
Character/Artistic,20,37
Ball Properties,17,8
Practical Concerns,60,25
Shelling,10,30
Rocks,13,19
Physics,2,2

Below are 50 little graphs showing the 90th percentile and 10th percentile in income, within each state, for the past forty years. The patterns are pretty striking: the high end has increased pretty consistently in almost all the states, and the low end increased a lot in poor states, especially for the first half of the series. I don't really know what more to say about this--we made the graphs because we are trying to understand the differences between rich and poor states in the past 20 years, and what has made them into "blue" and "red" states--but the graphs are full of interesting patterns. Incomes are inflation-adjusted and presented on a logarithmic scale (with a common scale for all the graphs), and the states are ordered from poorest to richest.

OK, here's the picture:

Maybe someone can do some record-linkage on this list and the list linked here .

Actually, I find the campaign list more interesting than the drugs list!

Hans van Maanem scrivt op deze manuscript hier van Michael Foster. Hans scrivt:

I [Hans] wrote about the paper when it came out -- not as thorough, I am afraid, but enough to warn readers against this kind of science. Maybe your colleagues are heartened by the fact that not everybody took it at face value! Could you please forward them my column from May 2004 in De Volkskrant (and maybe translate it...)?

I just reviewed the second edition of Jeff Gill's book for Amazon (5 stars). It's a fun book: the tone, a sort of theoretically-minded empiricism that is hard for me to characterize exactly but strikes me as a style of writing, and of thinking, that will resonate with the social science readership. It's great to see this sort of book that really puts a lot of thought into the "why" as well as the "how" (which is what we statisticians tend to focus on).

I gave comments on an earlier draft, and I'll put those comments below, but first I wanted to rant a bit about Bayesian methods before the Gibbs sampler came into play.

Some (reconstructed) history

Given that Gill does talk about history, I would've liked to have seen a bit more discussion of the applied Bayesian work in the "dark ages" between Laplace/Gauss in the early 1800s and the use of the Gibbs sampler and related algorithms in the late 1980s. In particular, Henderson et al. used these methods in animal breeding (and, for that matter, Fisher himself thought Bayesian methods were fine when they were used in actual multilevel settings where the "prior distribution" corresponded to an actual, observable distribution of entities (rather than a mere subjective statement of uncertainty)); Lindley and Smith; Dempster, Rubin, and their collaborators (who did sophisticated pre-Gibbs-sampler work, published in JASA and elsewhere, applying Bayesian methods to educational data); and I'm sure others. Also, in parallel, the theoretical work by Box, Tiao, Stein, Efron, Morris, and others on shrinkage estimation and robustness. These statisticians and scientists worked their butt off getting applied Bayesian methods to work _before_ the new computational methods were around and, in doing so, motivated the development of said methods and actually developed some of these methods themselves. Writing that these methods, "while superior in theoretical foundation, led to mathematical forms that were intractable," is a bit unfair. Intractable is as intractable does, and the methods of Box, Rubin, Morris, etc etc. worked. The Gibbs sampler etc. took the methods to the next level (more people could use the methods with less training, and the experts could fit more sophisticated methods), but Bayesian statistics was more than a theoretical construct back in 1987 or whenever.

In response to this, Carrie and Michael had an interesting exchange on possible junk science.

Aleks sent me this link.

(I haven't looked at this myself; note the "?" in the title above.)

The author of the great Santa Land Diaries is reduced to telling a non-story about his business-class trip to Paris??? I have to admit it was always a concern, that an author whose shtick is to tell his true-life stories, that at some point he'd run out of material. I don't really know the solution. Maybe he could suck it up and try to write some fiction based on serious reflections on his life experiences? Maybe he could do some journalism and get good stories out of other people?

My guess it, the New Yorker would be only doing him a favor by rejecting some of these pieces which are so far below his standard.

Then again, we're talking about a magazine that defines Dennis Miller as the king of comedy . . .

Douglas Hibbs's latest paper on his "bread and peace" model (predicting presidential election voting based on the economy, with corrections for wartime) is here. It's clear and worth a read. Even more amusing (to me, but probably not to most of you) is this exchange, which Hibbs posted on his webpage.

[Scroll down to the bottom to see my latest version of the graph.]

Here's my first version of the killer graph (adapted from Hibbs's to include his 2004 data):

This looks pretty good. The big exceptions are 1952 (Korean war) 1968 (Vietnam war). Beyond that, as Hibbs noted, 1996 and 2000 are the biggest outliers.

This stuff isn't news--we all heard about it in grad school. But at another level, it's always news.

Which picture do you prefer?

Here was my first try at the graph:

I put in some effort to make the more elaborate graph higher up on the page as a template for a sort of enhanced scatterplot that provides more information about each data point. I've seen this sort of dotplot before, and it seemed to make sense, since the points were roughly evenly spaced along the "economic growth" axis anyway.

I prefer the top graph on the page. But . . . there's something compelling about the scatterplot--forecasting y from x--that seems to be lost in the more elaborate dotplot. So I'm not quite sure what to do.

P.S. In his article, Hibbs convincingly explains why his model is much better than the much-ballyhooed Ray Fair model.

A new contestant

Following Sandapanda's comment below, I redid the upper graph width different proportions so that the area with the dots is larger. I also gave it a cleaner title:

Is it better now?

Or maybe I should plot them in decreasing order of avg income growth (i.e., 1964 and 1984 at the top, down to 1992 and 1980 at the bottom). This might be clearer because the dots would show a satisfying increasing relationship (rather than the more confusing negative slope seen here).

OK, OK, . . .

Here's the newest version:

I wasn't thrilled with this last version because it didn't make it clear what is being predicted from what. Here's a new version (see below) which puts the economic growth first, followed by the election outcome:

Michael Foster writes,

I was thinking of writing a book on Junk Science--"How Bad Research Scares the Crap out of Parents and Leads to Bad Public Policy"--there's a bunch of the stuff out there suggesting that day care makes kids more aggressive, tv causes attention problems and so on.

Do you know of a book that focuses on kids that already addresses these kinds of issues?

Sounds like a good book idea to me. Does anyone know what's out there?

Tall political scientist David Park and tall economist Robin Hansen point to this paper by Mankiw and Weinzierl on the idea of taxing tall people. I have no problem with the paper--it's wacky, but intentionally wacky, one might say, using height as an example of a variable that is correlated with income in the general population. Height isn't actually correlated very much with income (together, height and sex predict earnings with an R-squared of only 9% (see, for example, page 63 of our book), so in some way it's not a great example, but maybe that's part of their point too.

Anyway, David and Robin both quote a news article that states that Mankiw and Weinzierl "say that height is a 'justly acquired endowment': it is not unfairly wrested from anyone else, so the state has no right to seize its fruits."

Huh?

I think there's something I'm missing here, but . . . who ever said that you can only tax something that was "unjustly wrestled from someone else"? Haven't they ever heard of the sales tax? Even a society with no unjust wrestling at all would still need taxes. Setting redistribution aside entirely, there are reasons for higher taxes on the rich (they can more easily afford it) and reasons for not taxing the rich (depending on your mathematical model for the efficiency of the economy), but I don't really see how something being "justly endowed" means that it can't be taxed. The tax money needs to come from somewhere.

Why taxes?

The same article also says, "By the same logic, they imply ... the government has no right to force someone with the 'justly acquired endowment" of entrepreneurial genius to pay a higher tax rate." This also confuses me since I hadn't heard that anyone was proposing a tax on "entrepreneurial genius."

Maybe this is a difference between how economists and political scientists view the world. Mankiw and Weinzierl seem to view taxes as a way to punish people, whereas I see taxes as a way to raise money?

Lee Sigelman writes,

In a brief article (abstract here) in the current issue of Current Directions in Psychological Science, Dan Ariely and Michael Norton analyze the wide gap currently separating psychologically- and economically-based experimental research — a gap clearly perceptible in experimental work within political science, a heavy borrower from both psychology and economics.

“Psychologists have not traditionally been interested in the efficiencies and design of markets,” Ariely and Norton note, “while experimental economists have not customarily focused on emotion, memory, or implicit cognition.”

In addition to studying different topics, the two fields differ in their research methods as well:

It’s not just that psychologists enjoy lying to people while economists enjoy paying them. To find out what they want to find out, psychologists have to give their experimental subjects a “cover story” and transport them into a particular situation, for which purposes deception is often necessary. By contrast, economists want to know about experimental subjects’ ability to make informed decisions, and for that purpose deception would be counterproductive. At the same time, economists want to motivate their subjects to behave “normally,” so they explicitly define incentives to enable subjects to evaluate the costs and benefits of a particular course of action.

Finally, Sigelman quotes Ariely and Norton, who write,

Experimental economists might shift from asking whether deception is good or bad — a moral question — to exploring whether deception helps or harms social scientists’ ability to understand human behavior. Psychologists’ aversion to incentives, on the other hand, might be addressed by taking a broader view of what experimental economists are trying to accomplish with them: making people care about their behavior as much in the lab as they do in the real world.

Aversion to incentives?

I just have two things to add.

1. I applaud the call for researchers to become more aware of what is being done in other fields, but at some point, different people have different areas of expertise. (See my remarks here on why we shouldn't be disturbed that economists don't spend more time studying romance and here on different views of rationality.)

2. I question whether psychologists really have an "aversion to incentives." Giving incentives to research participants is one strategy out of many. It's natural for economists to privilege financial incentives--that's what they study--but maybe not so natural for others and maybe not always so relevant to the "real world." Many important real-world phenomena--such as political particpation--have little or no financial incentives at all!

The 2008 primary election season is just beginning, and, amid the debates between Hillary Clinton, Barack Obama, John Edwards and others, there's been active discussion about whether the country is looking for a centrist "New Democrat" in the Bill Clinton mode or someone from Howard Dean's "Democratic wing of the Democratic party."

One way to get a handle on this question is to consider the strategies considered in 2004. Could John Kerry have gained votes in the recent Presidential election by more clearly distinguishing himself from George Bush on economic policy? At first thought, the logic of political preferences would suggest not: the Republicans are to the right of most Americans on economic policy, and so it would seem that the optimal strategy for the Democrats would be to stand infinitesimally to the left of the Republicans. The "median voter theorem" (as political science jargon puts it) suggests that each party should keep its policy positions just barely distinguishable from the opposition.

In a multidimensional setting, however, or when voters vary in their perceptions of the parties' positions, a party can benefit from putting some daylight between itself and the other party on an issue where it has a public-opinion advantage (such as economic policy for the Democrats). Is this reasoning applicable in 2004 (or today)?

What we did

Our paper has two parts. In the theoretical part, we set up a plausible model in which the Democrats could achieve a net gain in votes by moving to the left on economic policy, given the parties' positions on a range of issue dimensions. In the data-analysis part, we fit a set of regression models based on survey data on voters' perceptions of their own positions and those of the candidates in 2004.

For example, here is a graph based on National Election Study data from 2004. Each dot represents where a survey respondent places him or herself on economic and social issues: positive numbers are conservative and negative numbers are liberal, and "B" and "K" represent the voters' average placements of Bush and Kerry on these scales:

Most voters tend to place themselves to the right of the Democrats on economic and on social issues, and most voters tend to place themselves to the left of the Republicans in both dimensions. Having (approximately) located the voters, we fit a model estimating the probability that each person would vote for Bush or Kerry, given the person's distance from each of the two candidates on economic and social issues. We can then artificially imagine moving the candidates to the left or right and seeing what would happen to their votes.

What we found

Under our estimated model, it turns out to be optimal for the Democrats to move slightly to the right but staying clearly to the left of the Republicans' current position on economic issues.

First, here are the estimated results based on one-dimensional shifts; that is, Kerry or Bush shifting to the left or the right on economic or social issues. Positions on the economy and on social issues are measured on a -9 to 9 scale, and a -8 to 8 scale, respectively, so shifts of up to 3 points to the left or right are pretty large (see the scatterplot above to get a sense of where the voters stand, and how they rate the candidates). For all shifts, the graphs show the estimated change in Bush's share of the vote.

Based on this model, Kerry should've shifted slightly to the right in both dimensions, Bush should've shifted slightly to the left on social issues and a great deal to the left on economic issues. (The curves are slightly jittery because of simulation variability.)

Now here are the estimated results for two-dimensional shifts, in which a candidate can change his position on economic and social issues:

According to this model, the optimal strategy for Kerry is to move 1 point to the right in both dimensions; in contrast, Bush would benefit by moving about 2 points to the left on social issues and nearly 3 points to the left on the economy. (Recall that the scales go from -9 to +9.)

In summary . . .

The answer to the question posed by the title of the paper appears to be No, Kerry should not have moved to the left on economic policy. Conventional wisdom appears to be correct: Kerry would have benefited by moving to the right, and Bush by moving to the left. The optimal shifts for Bush are greater than those for Kerry, which is consistent with the observation that voters are, on average, closer to the Democrats on issue attitudes.

Does this make sense?

Could the Democrats really move a bit to the left or the right? I think they could; there's been various debate along these lines in the 2008 primary season, with Hillary Clinton generally viewed as the more centrist candidate and John Edwards being more to the left on economic issues.

Could the Republicans move strongly toward the center, as recommended by our calculations? This seems less likely, given that the debates among the Republicans in the primaries seem to be focusing more on establishing the candidates' conservative credentials.

We conclude with a reminder that candidates need not, and perhaps should not, necessarily follow these seemingly optimal strategies. For one thing, the recommendations are only as good as the models, and the very act of a candidate trying to move to the left or to the right could affect voters's attitudes and other ways. For another thing, you only need 50%-plus-one electoral votes to win the election, and sometimes that can be done with a position that is less than optimal, as with George Bush's successful 2004 campaign, where he did fine without having to move to the center. He might have won more states with more centrist positions, but he didn't need to.

Reference

This is based on the article, "Should the Democrats move to the left on economic policy?" by Andrew Gelman and Cexun Jeffrey Cai, to appear next year in the Annals of Applied Statistics.

Richard Gunton writes,

Chris Paulse writes, "By accident I [Chris] discovered a book that has part of its focus on educational psychology. It's called Handbook of Competence and Motivation, Elliott and Dweck eds. A few recent articles have appeared in the NYT that seem to be sourced from material like this (one on self-regulation profiling the work of Roy Baumesiter, and another on learning from mistakes that quotes Carol Dweck). The Dweck chapter on self-concept is a fun read. I'd love to see a mixture model developed from survey data for the evaluation anxiety idea. Great for teachers."

I received the following email:

Hello, Dr. Gelman,

I am working on person-fit index for reading comprehension test using PPMC. I am pretty new for Bayesian, WinBUGS, and PPMC. . . .Could you take a look at my WinBUGS code for me? I calculated likelihood for posterior and predictive posterior and PPP-value . . .

Good question. I don't have time to look at people's code but I responded that you can check things by simulating fake data from the model and then checking that the estimated parameters are close to the true values; see Section 8.3 of my recent book with Hill. I'm usually too lazy to do this fake-data checking, but when I get around to it, it often is helpful.

Sam and Don and I wrote a paper with a systematic approach to fake-data checking. Often, though, just simulating one fake dataset and fitting the model is enough to reveal problems.

Pietro Panzarasa sent me this paper. From the abstract:

Aleks sent along this link. I just don't have the patience to watch these sorts of videos but maybe someone out there will like it. . . .

I got this in the email:

Someone who wishes to remain anonymous writes,

I have a question for you. Count models are to be used when you have outcome variables that are non-negative integers. Yet, OLS is BLUE even with count data, right? I don't think we make any assumptions with least squares about the nature of the data, only about the sampling, exogeneous regressors, rank K, etc. So why technically should we use count if OLS is BLUE even with count data?

My reply:

1. I don't really care if something is BLUE (best linear unbiased estimator) because (a) why privilege linearity, and (b) unbiasedness ain't so great either. See Bayesian Data Analysis for discussion of this point (look at "unbiased" in the index), also this paper for a more recent view.

2. Least squares is fine with count data, it's even usually ok with binary data. (This is commonly known, and I'm sure it's been written in various places but I don't happen to know where.) For prediction, though, you probably want something that predicts on the scale of the data, which would mean discrete predictions for count data. Also, a logarithmic link makes sense in a lot of applications (that is, E(y) is a linear function of exp(x)), and you can't take log of 0, which is a good reason to use a count data model.

Andrew Garvin writes,

The CFNAI is the first principal component of 85 macroeconomic series - it is supposed to function as a measure of economic activity. My contention is that the first principal component of a diverse set of series will systematically overweight a subset of the original series, since the series with the highest weightings are the ones that explain the most variance. As an extreme example, say we do PCA on 100 series, where 99 of them are identical - then the first PC will just be the series that is replicated 99 times.

Shravan adds an index entry to our book:

In the index at the back, please add crossed varying intercepts and crossed random effects as an entry for the pilot data discussed on p. 289. That kind of structure is very common in psychology/psycholinguistics, as you probably know, and many people will be looking for crossed random factor specificiations in your book but won't find it unless they know to look under non-nested models in the index.

Shravan also writes:

Also, I really think the code needs to be cleaned up to make it usable generally. It's really a lot of work to try to figure out which parts are missing in each of the code files. I will give you a more structured example and some suggestions for improvement soon. You may lose a lot of impatient people if you don't focus on clean code for the next edition of the book (conversely, you will gain a wider readership if you get the code cleaned up).

OK, OK, we'll do it...

Shang Ha writes,

"Instrumental variables" is an important technique in applied statistics and econometrics but it can get confusing. See here for our summary (in particular, you can take a look at chapter 10, but Chapter 9 would help too).

Now an example. Piero spoke in our seminar last Thursday on the effects of defamation laws on reporting of corruption in Mexico. In the basic analysis, he found that, in the states where defamation laws are more punitive, there is less reporting of corruption, which suggests a chilling effect of the laws. But there are the usual worries about correlation-is-not-causation, and so Piero did a more elaborate instrumental variables analysis using the severity of homicide penalties as an instrument.

We had a long discussion about this in the seminar. I originally felt that "severity of homicide penalties" was the wackiest instrument in the world, but Piero convinced me that it was reasonable as a proxy for some measure of general punitiveness of the justice system. I said that if it's viewed as a proxy in this way, I'd prefer to use a measurement-error model, but I can see the basic idea.

Still, though, there was something bothering me. So I decided to go back to basics and use my trick for understanding instrumental variables. It goes like this:

The trick: how to think about IV's without getting too confused

Suppose z is your instrument, T is your treatment, and y is your outcome. So the causal model is z -> T -> y. The trick is to think of (T,y) as a joint outcome and to think of the effect of z on each. For example, an increase of 1 in z is associated with an increase of 0.8 in T and an increase of 10 in y. The usual "instrumental variables" summary is to just say the estimated effect of T on y is 10/0.8=12.5, but I'd rather just keep it separate and report the effects on T and y separately.

In Piero's example, this translates into two statements: (a) States with higher penalties for murder had higher penalties for defamation, and (b) States with higher penalties for murder had less reporting of corruption.

Fine. But I don't see how this adds anything at all to my understanding of the defamation/corruption relationship, beyond what I learned from his simpler finding: States with higher penalties for defamation had less reporting of corruption.

In summary . . .

If there's any problem with the simple correlation, I see the same problems with the more elaborate analysis--the pair of correlations which is given the label "instrumental variables analysis." I'm not opposed to instrumental variables in general, but when I get stuck, I find it extremely helpful to go back and see what I've learned from separately thinking about the correlation of z with T, and the correlation of z with y. Since that's ultimately what instrumental variables analysis is doing.

Here's some material on causal inference from a regression perspective. It's from our recent book, and I hope you find it useful.

Chapter 9: Causal inference using regression on the treatment variable

Chapter 10: Causal inference using more advanced models

Chapter 23: Causal inference using multilevel models

Here are some pretty pictures, from the low-birth-weight example:

and from the Electric Company example:

See here.

Bill Jefferys pointed me to this note by Bob O'Hara on difficulties with DIC (the deviance information criterion for evaluating predictive model fit). I'd also point out that DIC can be numerically unstable. But I think it's basically a good idea.

Also here's some discussion of DIC from Brad Carlin, Nicky Best, and Angelika van der Linde.

Manuel Spínola writes,

Matt pointed me to this paper by Robert Vanderbei:

We describe and illustrate various ways to represent election results graphically. The advantages and disadvantages of the various methods are discussed. While there is no one perfect way to fairly represent the outcomes, it is easy to come up with methods that are superior to those used in recent elections.

The coolest thing in the paper are some 3-color maps. Here's 1992: blue is Clinton, red is Bush, and green is Perot:

It has the usual problem that large sparsely-populated areas are overrepresented but otherwise is ok, and certainly provides some interesting information. Vanderbei has some interesting discussion of the choice of colors for displaying these scales.

My other thoughts on the paper:

1. What's with the lower-case "democratic" and "republican"? It's standard to write these in caps.

2. I really hate those so-called "cartograms" (p.10 of the paper) since they draw attention to the distortion (the distribution of population) rather than the votes, which is really what we want to see.

3. I still like this map, which unfortunately isn't in the paper:

4. For the maps by Congressional district (page 11 of the paper), I'd prefer to put one dot per district rather than shading. The shading overemphasizes large areas (as usual) and also adds another distracting feature of drawing attention to the shapes of the districts, which is not the main point of interest.

That said, I do often present colored-state maps myself, because it is a clear way of presenting the information, despite all the problems in interpretation.

Martin James writes,

Here.

John Sides says yes, or maybe yes, linking here to a couple papers on the topic here. My comments on the Alvarez, Bailey, and Katz paper are here. And also Leighey and Nagler's research on the possible effects of increasing turnout.

Futurescanner is a website full of forecasts. (I heard about this from an unsolicited email but it looks interesting.) It would be fun (at least for me) to see forecasts about statistical methods. The challenge would be in stating the problems clearly enough you could unambiguously state when they were solved.

Jerry Fodor puts some effort here into shooting down evolution, or more precisely adaptationism, the idea (in human terms) that we adapted for a stone-age environment and that's why we have difficulties in the modern world, or (in more general terms) that natural selection is the key aspect of the evolution of species.

What I'm interested in here, though, is not the scientific issue about evolution (which I'm certainly not competent to judge) but some of the related political issues. I'd like to write something longer on this (if I could figure out exactly what to say) but the idea is that the big underlying issue is politics. Fodor is (I assume) a political liberal in U.S. terms, meaning that he supports some combination of income redistribution, feminism, gay rights, environmental protection, a nonmilitaristic foreign policy, etc. And he's opposing adaptationism partly, I think, because it is associated with conservatives--people who want to keep traditional social and economic arrangements and support open markets, traditional religious values, minimal regulation, an active military, and so forth. The conservative arguments typically have the flavor of, "Human nature is the way it is, we can't change it so don't try. Clever efforts at reform end up being too clever by half and have unintended consequences." Adaptationism fits here as a scientific basis for "human nature," supplying what Fodor (quoting Gould) labels as "just-so stories" about why men should be the boss, why people are inherently aggressive so we need a strong defense, why family ties are important, etc. As Steven Pinker and others have noted, adaptation doesn't need to have any particular political implications. (For example, if men are naturally killers because of our stone age evolution, this could be an argument for accepting some violence (yeah, I'm talking about you, Michael Vick), or an argument for rigorous laws to stop the violence.) And, even accepting adaption, there's still room for lots of debate on the details.

That said, I think that Fodor is reacting to the current vogue for adaptation as an explanation/motivation for conservative ideas by what he would view as modern-day Herbert Spencers.

Let me be clear here. I'm not saying that adaptationism is some sort of hard truth that Fodor doesn't accept because he doesn't like it's political implications. Rather, I'm saying that adaptation is a tricky scientific question, and I suspect that one reason for Fodor's interest in it is that it's been used by conservatives to support their political positions. Maybe the political dimension is one reason it's so difficult for me to follow the discussions (for example, go here and scroll down to "Why Pigs Don't have Wings").

Daniel Lakeland writes with a question about lmer() that is more generally a question about partially nested multilevel models:

David Brooks wrote a column today on "the dictatorship of talent" in China, which reminds me of a big problem with all discussion of "meritocracy," which is that it's actually a self-contradiction. I learned this several years ago from a wide-ranging and interesting article by James Flynn (the discoverer of the "Flynn effect", the steady increase in average IQ scores over the past sixty years or so). Flynn's article talks about how we can understand variation in IQ within populations, between populations, and changes over time.

At the end of his article, Flynn gives a convincing argument that a meritocracatic future is not going to happen and in fact is not really possible. He first summarizes some data showing that America has not been getting more meritocratic over time. He then presents the killer theoretical argument:

The case against meritocracy can be put psychologically: (a) The abolition of materialist-elitist values is a prerequisite for the abolition of inequality and privilege; (b) the persistence of materialist-elitist values is a prerequisite for class stratification based on wealth and status; (c) therefore, a class-stratified meritocracy is impossible.

Basically, "meritocracy" means that individuals with more merit get the goodies. From the American Heritage dictionary: "A system in which advancement is based on individual ability or achievement." As Flynn points out, this leads to a contradiction: to the extent that people with merit get higher status, one would expect they would use that status to help their friends, children, etc, giving them a leg up beyond what would be expected based on their merit alone.

Flynn also points out that the promotion and celebration of the concept of "meritocracy" is also, by the way, a promotion and celebration of wealth and status--these are the goodies that the people with more merit get. That is, the problem with meritocracy is that it's an "ocracy". As Flynn puts it:

People must care about that hierarchy for it to be socially significant or even for it to exist. . . . The case against meritocracy can also be put sociologically: (a) Allocating rewards irrespective of merit is a prerequisite for meritocracy, otherwise environments cannot be equalized; (b) allocating rewards according to merit is a prerequisite for meritocracy, otherwise people cannot be stratified by wealth and status; (c) therefore, a class-stratified meritocracy is impossible.

He also has some normative arguments which you could take or leave, but the social-science analysis is convincing to me.

Bob the biologist sent me the link to this paper by Evans and Shakhatreh:

We [E & S] ]consider various properties of Bayesian inferences related to repeated sampling interpretations, when we have a proper prior. While these can be seen as particularly relevant when the prior is diffuse, we argue that it is generally reasonable to consider such properties as part of our assessment of Bayesian inferences. We discuss the logical implications for how repeated sampling properties should be assessed when we have a proper prior. We develop optimal Bayesian repeated sampling inferences using a generalized idea of what it means for a credible region to contain a false value and discuss the practical use of this idea for error assessment and experimental design. We present results that connect Bayes factors with optimal inferences and develop a generalized concept of unbiasedness for credible regions. Further, we consider the effect of reparameterizations on hpd-like credible regions and argue that one reparameterization is most relevant, when repeated sampling properties and the prior are taken into account.

I should read this--it seems related to the "weakly informative priors" stuff. But for now I'll stick it on the blog where I hope I won't forget it. At least it gets it out of my inbox!

From the Judgment and Decision Making list, I saw this interesting article by Scott Armstrong:

I [Armstrong], along with Kesten Green and Willie Soon, audited the forecasting methods used by the authors of the government's administrative reports to support their strategy to list polar bears as an endangered species. As it turns out, the forecasts were based primarily judgmental methods. We concluded that the forecasts of polar bear populations were not derived from scientific forecasting procedures. It would be irresponsible to classify polar bears as endangered on the basis of such forecasts.

Bob Clemen replied with some more general questions about how to evaluate forecasting methods:

Aleks noticed this interesting-looking course:

Boris noticed this report from Gallup:

Republicans are significantly more likely than Democrats or independents to rate their mental health as excellent, according to data from the last four November Gallup Health and Healthcare polls. Fifty-eight percent of Republicans report having excellent mental health, compared to 43% of independents and 38% of Democrats. This relationship between party identification and reports of excellent mental health persists even within categories of income, age, gender, church attendance, and education.

The basic data -- based on an aggregated sample of more than 4,000 interviews conducted since 2004 -- are straightforward.

One could be quick to assume that these differences are based on the underlying demographic and socioeconomic patterns related to party identification in America today. . . . But an analysis of the relationship between party identification and self-reported excellent mental health within various categories of age, gender, church attendance, income, education, and other variables shows that the basic pattern persists regardless of these characteristics.

[Similar graphs follow by sex, age, and church attendance, followed by a multiple regression.]

This comes as a surprise to me. I would've expected the opposite--I associate Democrats with the "self-esteem" concept and Republicans with a grimmer, more conservative view of the world. I wonder what the time trends are on this.

P.S. As Matt suggests in comments, this might be a regional thing. I'd also like to see it broken down by urban/suburban/rural.

This is a long one because it has a lot of information (collected and analyzed by others, not ourselves). To start with, some data from a study by Baldassare of Californians:

But political scientists generally hold that voters and nonvoters aren't really so different

This 1999 article by Highton and Wolfinger summarizes the basic political-science view of nonvoters and election outcomes: