June 2008 Archives

Here's a graph of the 50 states (actually, I think Alaska and Hawaii are missing), showing the average economic and social ideology of adults within each state. Each of these is scaled so that negative numbers are liberal and positive are conservative; thus, people in Massachusetts are the most liberal on economic issues and people in Idaho are the most conservative:

West Virginians are on the liberal side economically but are extremely socially conservative, whereas Vermont is about the same as West Virginian on the economic dimension but is the most socially liberal of all the states. Coloradans are economically conservative (on average) but socially moderate (or, perhaps, socially divided; these are averages only).

How do these rankings fit with our usual rankings of states? Here's a plot showing average economic and social ideology for each state, plotted vs. George W. Bush's vote share in 2000:

Democrats and Republicans separately

The next step is to break these voters down into Democrats and Republicans (based on self-reported party identification and following the usual practice among political scientists of throwing the "leaners" into the regular party categories). In the graph below, each state is shown twice: the avg social and economic ideologies of Democrats in the state are shown in blue, the avgs for Republicans in red.

We made these graphs during the primary election season, and one thing we noticed was that South Carolina ("SC") is in the middle of the pack among Democrats and among Republicans, but it's one of the most conservative states overall. My take on this: South Carolina is a strongly Republican state, and the moderates in South Carolina are likely to identify as Republican. This pulls the Republican average to the left (as they includes the moderates) and also pulls the Democratic average to the left (as they are not including so many moderates).

But the big thing we see from the graph immediately above is that Democrats are much more liberal than Republicans on the economic dimension: Democrats in the most conservative states are still much more liberal than Republicans in even the most liberal states. On social issues there is more overlap (although in any given state, the average Republican is more conservative than the average Democrat).

Details on data

David Park and I made these graphs from the Annenberg pre-election survey from 2000 (with its huge sample size), creating indexes based on issue opinions, giving each respondent an economic and social ideology score. We scaled these so that each had a national average of 0 and standard deviation of 0.5. (We used these scales in our Red State, Blue State book, but these particular graphs never made it into the book.)

P.S.

Yes, I know the graphs could be better. We made them a few months ago and haven't organized them into any final form.

P.P.S. More info here.

This is ok, but I like my solution better.

John Cook's theory of why the t distribution was discovered at a brewery:

Beer makers pride themselves on consistency while wine makers pride themselves on variety. That’s why you’ll never hear beer fans talk about a “good year” the way wine connoisseurs do. Because they value consistency, beer makers invest more in extensive statistical quality control than wine makers do.

(On the other hand, Seth thinks that "ditto foods" are so late-twentieth-century, and that lack of uniformity in taste is ultimately healthier.)

Ubs writes:

A TV journalist's career success is strongly correlated to how well-known he is to the audience, which in turn is strongly correlated to how much face time he gets. When you watch an interview on TV, if most of what you see are is person being interviewed, you won't remember the journalist so much. If more of your time is devoted to watching and hearing the interviewer talk, he'll be more recognizable next time. The latter probably does not make for a better interview, but it does make for a better chance of the journalist getting more gigs.

See here.

There are some changes in arm. Most of them are related to the big changes lme4 has made. If your arm version number is > arm_1.1.8, then you might want to read the followings (or if you are a reader of Data Analysis Using Regression and Multilevel/Hierarchical Models, you should read this):

Drew Conway pointed me to this article by Chris Anderson talking about the changes in statistics and, by implication, in science, resulting from the ability of Google and others to sift through zillions of bits of information. Anderson writes, "The new availability of huge amounts of data, along with the statistical tools to crunch these numbers, offers a whole new way of understanding the world. Correlation supersedes causation, and science can advance even without coherent models, unified theories, or really any mechanistic explanation at all."

Conway is skeptical, pointing out that in some areas--for example, the study of terrorism--these databases don't exist. I have a few more thoughts:

1. Anderson has a point--there is definitely a tradeoff between modeling and data. Statistical modeling is what you do to fill in the spaces between data, and as data become denser, modeling becomes less important.

2. That said, if you look at the end result of an analysis, it is often a simple comparison of the "treatment A is more effective than treatment B" variety. In that case, no matter how large your sample size, you'll still have to worry about issues of balance between treatment groups, generalizability, and all the other reasons why people say things like, "correlation is not causation" and "the future is different from the past."

3. Faster computing gives the potential for more modeling along with more data processing. Consider the story of "no pooling" and "complete pooling," leading to "partial pooling" and multilevel modeling. Ideally our algorithms should become better at balancing different sources of information. I suspect this will always be needed.

Eddie Randolph writes,

I was wondering if you had any thoughts on the World Rock Paper Scissors Contest currently being held. Do you think it will be won by someone who plays intuitively, or a master strategist? If you think the strategist will win, do you think they will employ strategies from the book you pointed to in your blog?

My reply: I love rock-paper-scissors but I'm afraid I have no deep theories. I'd guess that it's pretty random, that whoever wins one year wouldn't have much better than a random chance of doing well the next year.

Jennifer pointed me to this site, which states that "white people hate math" but "are fascinated by 'the power of statistics' since the math has already been done for them." I'd like to believe this is true (the part about white people liking statistics, not the part about the math having already be done to them) but I'm skeptical. Everywhere I've ever taught, there have been a lot more math majors than stat majors, and I'm pretty sure this is true among the subset of students who are white. But it might be true that the business majors, the poli sci majors, the English majors, etc.--not to mention the people who don't go to college at all--prefer statistics to mathematics. Actually, I think most of these people should prefer statistics to mathematics. But I fear that a more likely reaction would be something like, "math is cool, statistics is boring."

P.S. I looked further down, and this "Stuff White People Like" site is just weird. "With few exceptions, white people are actually fond of almost any dictator not named Hitler"?? Huh? I mean, I can see that the site is a parody, but this is just weird.

Sunshine Hillygus and Todd Shields just came out with a book, "The Persuadable Voter: Wedge Issues in Presidential Campaigns," where they argue that "persuadable voters are not a homogeneous group of unsophisticated and indifferent policy moderates, as has often been believed. Rather, persuadable voters hold diverse policy preferences, making it less clear which candidate offers a better match." Hilligus and Shields point out that many many voters disagree with their party lines on important issues (see also my paper with Delia on this topic; correlations between party identification and individual issue attitudes have increased over the past few decades but are still only about 0.3 on a -1 to 1 scale).

The discussion of campaigns trying to exploit voters' cross-pressure reminds me of Dave Krantz's research on how people process information. Suppose you are evaluating hypotheses A and B, exactly one of which is true (for example, the suspect committed the crime or did not). You can imagine four kinds of evidence: (1) evidence supporting A, (2) evidence making A less likely, (3) evidence supporting B, and (4) evidence making B less likely. Dave et al. did some lab experiments manipulating these conditions, and found that people treat them differently: for example, people react differently if you give them evidence of the form (1)+(2), (1)+(3), or (2)+(4). Hillygus and Shields's work focuses this idea by considering an area--political campaigns--where there is a lot of effort being made on both sides to persuade people. Their recommendation to the news media is to look beyond broadcast ads and speeches, to monitor microtargeted messages and direct mail, to make it more difficult for a political campaign to send different messages to different audiences.

Boris forwarded to me this passage from The Audacity of Hope which was noted by Jim Geraghty:

Increasingly, I [Obama] found myself spending time with people of means - law firm partners and investment bankers, hedge fund managers and venture capitalists. As a rule, they were smart,interesting people, knowledgeable about public policy, liberal in their politics, expecting nothing more than a hearing of their opinions in exchange for checks. But they reflected, almost uniformly, the perspectives of their class; the top 1 percent or so of the income scale that can afford to write a $2,000 check to a political candidate. They believed in the free market and an educational meritocracy; they found it hard to imagine that there might be any social ill that could not be cured with a high SAT score. They had no patience with protectionism, found unions troublesome, and were not particularly sympathetic to those whose lives were upended by movements of global capital. Most were adamantly prochoice and were vaguely suspicious of deep religious sentiment...

I know that as a consequence of my fund-raising I became more like the wealthy donors I met, in the very particular sense that I spent more and more of my time above the fray, outside the world of immediate hunger, disappointment, fear, irrationality, and frequent hardship of the other 99 percent of the population - that is, the people I'd entered public life to serve.

Juli sent me this article by Robert Adler, John Ewing, and Peter Taylor arguing that "impact factors" journals are overrated. I'm sympathetic to this argument because my articles are typically published in low-impact-factor journals (at least in comparison with psychology). I also like the article because it has lots of graphs, no tables. Here's a graph for ya:

Adler et al. also criticize the so-called h-index and its kin; as they write, "These are often breathtakingly naive attempts to capture a complex citation record with a single number. Indeed, the primary advantage of these new indices over simple histograms of citation counts is that the indices discard almost all the detail of citation records, and this makes it possible to rank any two scientists. . . . Unfortunately, having a single number to rank each scientist is a seductive notion – one that may spread more broadly to a public that often misunderstands the proper use of statistical reasoning in far simpler settings."

Michael Arrington talks about a new model he created to assess the market value in online social networks. As hard as it can be to place a market value on a person, it's almost more complicated to place such a value on an online social network. Arrington looks at MySpace, Facebook, Bebo, Hi5, and LinkedIn, among other sites, and creates a model that includes the number of unique visitors the site gets, the number of worldwide users, and the site's total revenue.

He points out that one of the issues with his model is that each social networking site is different in its focus; I would add to this that each site has a very different user base. (He does point this out for LinkedIn, which is purely a place to connect virtually and not a site on which you can play games or babble back and forth.) I have profiles on all the sites I listed above (which is partially because I used to be a high school teacher and partially out of personal interest), but Facebook is definitely the only one I use with some regularity. There is obviously a lot of money to be made from advertising on the sites, which in turn might make some people interested in actually purchasing the sites, but it still seems shaky to equate any "values" between sites since they are all so different in nature. People on each site have different buying powers, people spend different amounts of time on each site, people go to each site with different motives, etc.

I should add here that I am part of one of Andrew's projects on political polarization, and we ran a survey on Facebook as part of that project. It's not cheap to target Facebook users; it is, however, a nice way to target users with specific interests because they list their interests in their profiles. Then again, Facebook users are overwhelmingly within the same age range and at a similar level of education, so it's not a hugely diverse or population-representative respondent group. We paid for it though, and maybe that's exactly the point.

Random imputation is a flexible and useful way to handle missing data (see chapter 25 for a quick overview), but it's typically taken as a black box. This partly is a result of confusion over statistical theory. Structural assumptions such as "missingness at random" cannot be checked from data--this is a fundamental difficulty--but this does not mean that imputations cannot be checked. In our recent paper, Kobi Abayomi, Mark Levy, and I do the following:

We consider three sorts of diagnostics for random imputations: displays of the completed data, which are intended to reveal unusual patterns that might suggest problems with the imputations, comparisons of the distributions of observed and imputed data values and checks of the fit of observed data to the model that is used to create the imputations. We formulate these methods in terms of sequential regression multivariate imputation, which is an iterative procedure in which the missing values of each variable are randomly imputed conditionally on all the other variables in the completed data matrix.We also consider a recalibration procedure for sequential regression imputations.We apply these methods to the 2002 environmental sustainability index, which is a linear aggregation of 64 environmental variables on 142 countries.

The article has some pretty pictures (and some ugly pictures too; hey, we're not perfect). I don't know how directly useful these methods are; I think of them as providing "proof of concept" model checking for imputations is possible at all, and I'm hoping this will spur lots of work by many researchers in the area. Ultimately I'd like people (or computer programs) to check their imputations just as they currently check their regression models.

This is another example of why defaults matter a lot.

I got an email of Evan Cooch forward by Matt, saying that there exists a trick to speed up R matrix caculation. He found that if we replace the default Rblas.dll in R with the proper one. It can boost R's speed in doing matrix caculation.

The file is here (This file only works under Windows). For Mac and Linux users, see here.

I just read an article on a faster Gibbs/slice sampler algorithm and it sparked the following thoughts:

1. Faster convergence is not just about speed, it's also about having confidence that your results are correct. The worst thing about not having a good algorithm is never knowing whether I've really fit the model I'm trying to fit. The next worst thing is that then I don't have the confidence to go to the next, more complicated, model I'd like to try.

2. The researchers run things for 10,000 iterations. In statistical simulation research, 10,000 is a small number of iterations. But in practice, 10,000 can take a long time. If the algorithm is as efficient as claimed, wouldn't 1000 be enough? This might sound silly but in big problems--or in routine analyses--you don't necessarily want to wait for 10,000.

Lots of stories for little kids have kings and queens, not many seem to have presidents, prime ministers, mayors, etc. I don't fully understand this. I mean, I see that these stories are traditional, or imitate traditional forms, and so it makes sense that you'd have a king or queen rather than a president. But there are lots of other traditional forms of government. You can see some examples in children's literature, but they're clearly exceptions. (For example, the wolves in The Jungle Book have a tribal council, and the animals in Winnie the Pooh don't have any government at all.) I guess what I'm asking is: How did the standard storybook world become codified, the world with a kingdom, a king and a queen living in a castle riding horses etc? Even in the late Middle Ages in Europe when, I suppose, such places really existed, there were lots of other, different, sorts of places nearby. How and when did the storybook kingdom became canonical? Maybe Jenny can answer this question--it seems to fall within her bailiwick.

P.S. More discussion in the comments to Mark Thoma's blog here. My favorite comment is the first one: "If Mr Gelman doesn't like kings and queens in childrens' stories maybe he should write some stories himself." You'd think that a commenter to an economics blog would've heard about the division of labor! I tell stories to kids, but I write for adults.

More to the point, there are lots and lots of stories without kings and queens, from "And to Think That I Saw It on Mulberry Street" on down. What struck me, though, was how kingdoms are canonical. For example, Sesame Street is filled with original stories--not folktales or anything like that--and by default they are often set in kingdoms.

Dan Lakeland posts this graph:

Drew Conway links to this site that simulates text for Garfield cartoons using a Markov model. I don't actually think these are funny, but it strikes me that they could be a good demo for a lesson on Markov chains.

The discussion here on the climate change attitude mystery reminded me of a funny thing about how we think when we classify people by education, or income, or religion.

James O'Brien writes,

Just wondering if you had any thoughts on testing a random intercepts model in multilevel logistic regression. In my present study, the results of a Wald test (which Twisk, for example, suggests as a kind of approximation for testing whether parameter variance is effectively zero), produce a p-observed of .06.

I've noticed in the MLWin manual as well, they argue that p<.1 for the Wald is evidence that slopes and intercepts (in that particular case) indeed vary. Are they perhaps willing to relax the cut-off because of the approximate nature of the test?

Am I perhaps missing something? Perhaps I need to go back to MLWin and do some bootstrapping.

My reply: I've been known to bootstrap (see my 1992 paper in the Journal of Cerebral Blood Flow and Metabolism, and my recent paper with Alessandra analyzing the storable votes data), but in this case I don't think you have to go to that level of effort.

The short answer is that these true variances are never zero (at least, not in social science applications); a non-significant test doesn't mean the variance is zero, it just means that the data are consistent with the zero variance model. So, if you want, you can keep the variance component in the model and ignore the test. Conversely, if the variance is low, it might not hurt you much to just exclude it from the model (i.e., to set it to zero). It depends what you're doing. If you're just including this variance component to help estimate some other parameter in the model, you can probably just get rid of it, but if you're specifically trying to compare particular coefficients here, or to make predictions for new groups, you better keep it in. In that case, you might want to do fully Bayes (something like the 8-schools model) so you're not conditioning on a particular imperfect estimate of that variance parameter.

This issue actually comes up a lot. Fortunately, it's when the variance parameter is least important (when it could in practice be replaced by zero) that it's trickiest to estimate. A rare bit of good news in statistical inference.

A reporter asked me, "Do people run for VP, who in the past, how, has it worked or failed?"

My reply: I haven't looked at this recently, but I recall when studying election forecasting 15 years ago, that the estimated effect of VP choice was something like +3 percentage points in the VP's home state, so nothing huge.

What about national effects? In 1988, I recall that polls found that Bush alone (in a Bush vs. Dukakis matchup) did about 2 points better than Bush-Quayle vs. Dukakis-Bentsen. But this is probably an upper bound:
- Quayle was a horrible candidate
- And probably, when it came down to the voting booth, it's my guess that less than 2% of people decided not to vote for Bush on the basis of Quayle.

So probably the biggest effect of VP is that this is a person who's likely to become president. (I don't have the stats on this, but the total probability must be pretty high.) If I were choosing, I'd pick the person I'd most like as a future president and probably not worry so much about electoral calculations, fun though they are to think about.

I just finished two novels that both take place in London and were written by demographically similar authors. The Post-Birthday World is a book that I'd read an amusingly dismissive review of (I can't remember where; I thought it was the London Review of Books but it wasn't) but then I saw the paperback in the train station and flipped through it and it looked pretty good. It was indeed pretty good. I can't say that the characters really ever seemed like real people, but it was hard to put down and also thought-provoking. I liked that it moved slowly enough that the characters had a chance to mull things over. The Last Samurai was recommended to me by Rachel. I have to admit that I skipped all the parts of the book that were in foreign languages but I suppose they added to the atmosphere nonetheless. The two main characters seemed completely real, also the book was very funny. There was also some of that cool unreliable-narrator thing going on, where one of the two main characters (the adult one) seemed self-deluded. Highly recommended.

Apparently, Friday the 13th is not unlucky, according to Dutch researchers: link to article.

I would like to see a parallel psychological study, to see if people are more careful on Friday the 13th, go out less, drive less (or just shorter distances) - and if people considering criminal activity hold off until the next day. I also wonder if there is an upswing in the types of "bad luck" they chose to survey on Saturday the 14th...

This entry received the following comment:

You can't compare each round as a parallel test because they move the tee boxes and hole locations every day. This makes the course much more difficult on some days and that is what separates the best from the worst.

a href="http://www.sporthaley.com" Women's Golf Clothing /a

I don't know what to make of this sort of thing. It's hard for me to believe it's pure spam--could a bot really read the entry and make that comment? But what person would sign his or her comment as "Women's Golf Clothing"? We get this kind of semi-spam on the blog comments now and then, and I'm never sure what to think about it.

P.S. Just to be clear, here's another comment that clearly is 100% spam. It's for the blog entry on "Baby Names," it's by "baby boy," and it says, "I can think of "yang" names, you can never know." That's clearly spam, unlike the above comment which was human-generated.

Seeing Nate's discussion here on random walks, bounces, and trends, I was reminded of a paper that Joe Bafumi, David Park, and I wrote a few years ago.

Basically, general election opinion polls can be modeled well with a "mean reversion" model, in which the outcome is predictable and the polls will eventually converge to this predictable outcome. But journalists and observers tend to implicitly assume a "random walk" model which starts at the current position of the polls and then moves from there. Here's the paper, here's the abstract:

Scholars disagree over the extent to which presidential campaigns activate predispositions in voters or create vote preferences that could not be predicted. When campaign related information flows activate predispositions, election results are largely predetermined given balanced resources. They can be accurately forecast well before a campaign has run its course. Alternatively, campaigns may change vote outcomes beyond forcing predispositions to some equilibrium level. We find most evidence for the former: opinion poll data are consistent with Presidential campaigns activating predispositions, with fundamental variables increasing in importance as a presidential election draws near.

And here is a key graph showing votes becoming more predictable during the election campaign:

Finally, here's my article with Gary from 1993, "Why are American Presidential election campaign polls so variable when votes are so predictable?" This article gives lots of evidence supporting the idea that people ultimately decide how to vote based on their enlightened preferences.

Here are my thoughts, to appear in the American Statistician:

1. Introduction
2. Teaching Bayesian statistics to social scientists, including a discussion of what is Bayesian about making graphs to get a better understanding of the deterministic part of a model
3. Other thoughts on teaching statistics to non-statisticians
4. A case study: the sampling distribution of the sample mean
5. Starting an (implicity) Bayesian applied regression course: two weeks of classroom activities
6. How is there time, in a course with class participation, to cover all the material?

1. Introduction

I was trying to draw Bert and Ernie the other day, and it was really difficult. I had pictures of them right next to me, but my drawings were just incredibly crude, more "linguistic" than "visual" in the sense that I was portraying key aspect of Bert and Ernie but in pictures that didn't look anything like them. I knew that drawing was difficult--every once in awhile, I sit for an hour to draw a scene, and it's always a lot of work to get it to look anything like what I'm seeing--but I didn't realize it would be so hard to draw cartoon characters!

This got me to thinking about the students in my statistics classes. When I ask them to sketch a scatterplot of data, or to plot some function, they can never draw a realistic-looking picture. Their density functions don't go to zero in the tails, the scatter in their scatterplots does not match their standard deviations, E(y|x) does not equal their regression line, and so forth. For example, when asked to draw a potential scatterplot of earnings vs. height, they have difficulty with the x-axis (most people are between 60 and 75 inches in height) and having the data consistent with the regression line, while having all earnings be nonnegative. (Yes, it's better to model on the log scale or whatever, but that's not the point of this exercise.)

Anyway, the students just can't make these graphs look right, which has always frustrated me. But my Bert and Ernie experience suggests that I'm thinking of it the wrong way. Maybe they need lots and lots of practice before they can draw realistic functions and scatterplots. They'll certainly need lots of practice to learn Bayesian methods.

Here's an amusing story. I can understand why the guy computed his test from scratch, but I agree with Dan that the two-page appendix is kind of over the top.

I appreciate the effort, but I fear that the message that many have taken from Tufte is "graphs should be cool" rather than "graphs should be clear." As Yu-Sung put it, "I am still figuring out how to read it."

Andrew Oswald sent me this paper by Amanda Goodall, Lawrence Kahn, and himself, called "Why Do Leaders Matter? The Role of Expert Knowledge." Here's the abstract:

Why do some leaders succeed while others fail? This question is important, but its complexity makes it hard to study systematically. We draw on a setting where there are well-defined objectives, small teams of workers, and exact measures of leaders characteristics and organizational performance. We show that a strong predictor of a leader's success in year T is that person's own level of attainment, in the underlying activity, in approximately year T-20. Our data come from 15,000 professional basketball games and reveal that former star players make the best coaches. This expert knowledge effect is large.

My first thought upon seeing this paper was: What about Isiah Thomas? But a glance through reveals that their data end at 2004, before Isiah took up his Knicks coaching job.

More seriously, Goodall et al.'s findings seem to contradict the conventional wisdom in baseball that the best managers are the mediocre or ok players such as Earl Weaver and Casey Stengel rather than the superstars such as Ted Williams and Ty Cobb. I'd be interested to hear what the authors think about this.

Scatterplot, please! It's not just about an eye-catching result; it's about building confidence in your findings

I won't bother to give my comments on the tables and graphs (except to note that the figures are hard to read for many reasons, starting with the fact that these are bar graphs with lower bounds at 0.4 (?), 0.6 (??), etc.).

What I will say, though, is that I'd like to see a scatterplot, with a dot for each coach/team (four different colors for the four categories of coaches), plotting total winning percentage (on the y-axis) vs. winning percentage in the year or two before the coach joined the team (on the x-axis). This is the usual before-after graph, which can then be embellished with 4 regression lines in the colors corresponding to the four groups of coaches.

When reading such an analysis, I really, really want to see the main patterns in the data. Otherwise I really have to take the results on trust. This is related to my larger point about confidence building.

David Runciman writes,

Followed day by day, the race for the Democratic nomination has been the most exciting election in living memory. But viewed in retrospect, it is clear that it has been quite predictable. All the twists and turns have been a function of the somewhat random sequencing of different state primaries, which taken individually have invariably conformed to type, with Obama winning where he was always likely to win (caucus states, among college-educated and black voters, in the cities), and Clinton winning where she was likely to win (big states with secret ballots, among less well-educated whites and Hispanics, in rural areas).

"Predictable in retrospect"?? This seems like a contradiction. I agree with Runciman that there are patterns in the election results, but I'd only call it "predictable" if you actually predict it ahead of time, which I certainly didn't! He continues:

From Andrew Sullivan. More here. I love this stuff.

Following up on our link to an article about educational measurement, Eric Loken pointed me to this:

On the Criteria Corporation blog we [Loken] just posted a look at golf tournament scores. If you take the four rounds as if they were four repeats of the same test, or four parallel items on a test, the usual psychometric analyses would yield a terrible reliability coefficient. The problem of course is restriction of range of true scores among the world's best golfers. We figured since the US Open (this weekend) is sometimes called the Ultimate Test we'd offer a little psychometric analysis of golf.

Despite having published an article on golf, I know almost nothing about the sport--I've never actually played macro-golf--so I'll link to Eric's note without comment.

J. Michael Steele explains why he doesn't like the above saying (which, as he says, is attributed to statistician George Box). Steele writes, "Whenever you hear this phrase, there is a good chance that you are about to be sold a bill of goods."

We went by the pool in Central Park today and it was drained of water. Wha . . . ? It was only about 100 degrees out there today.

Aaahhhh, I see: "Pool Opens July 1st and closes after Labor Day". (Note also that in the picture on the website, the pool has no people in it. That's no coincidence: when there are people in it, it's jam-packed with people.)

OK, I have an idea: no A/C in city offices until the all the swimming pools open. If the city doesn't want to open the pool until July 1st, no problem: it's probably not so hot outside, and I'm sure the desk workers can make do with fans.

And, just in case they decide to do what they did in previous years and open up only half of the pool (I'm not kidding, they roped half the pool off so that a few zillion people were crammed into 50% of the space), then they can do the same thing in city offices: half of the offices can have A/C, half won't. I'm not sure what to do about the mayor. Maybe give him A/C half the time?

P.S. When open, the pool hours are from 11-3 and 4-7. Of course, this means that city offices should also be air conditioned only during these times.

Gal Elidan writes:

I am starting as a faculty next year in the statistics department at the Hebrew University, Israel. As it may be interesting to both the computer science and statistical community, I plan to give a course a course next year on Bayesian data analysis. My (still in its early stages) plan is to give a course based on your book along with some relevant topics/applications that have seen light in the computer science community in recent years (e.g. the Chinese restaurant process). I would greatly appreciate it greatly if you could share with me any material that you have used in the past in teaching this course. Since I have little experience estimating work load, I could use help in knowing how many problems you assigned each time.

My reply:

After seeing my note on education, partisanship, and views on climate change, Jason Reifler sent me this paper he wrote with Brendan Nyhan, which begins:

An extensive literature addresses citizen ignorance, but very little research focuses on misperceptions. Can these false or unsubstantiated beliefs about politics be corrected? Previous studies have not tested the efficacy of corrections in a realistic format. We [Nyhan and Reifler] conducted four experiments in which subjects read mock news articles that included either a misleading claim from a politician, or a misleading claim and a correction. Results indicate that corrections frequently fail to reduce misperceptions among the targeted ideological group. We also document several instances of a “backfire” effect in which corrections actually increase misperceptions among the group in question.

That's scary stuff!

P.S. Nice graphs. Tables 1-4 could be made into graphs too (by adapting coefplot()), but, still, the displays are pretty good.

David Dunson forwarded me this article for
a book that is coming out on Nonparametric Bayes in Practice. I think David's work is great but I keep encountering it in separate research articles and never in a single place which explains when to use each sort of model. I'll have to read the article in detail, but it seems like a good start. I suggested to David that he write a book but he pointed out that nobody reads books. But do people read articles in handbooks? I don't know. I guess what's really needed is a convenient software implementation for all of it. In the meantime, this article seems like the place to go.

Catherine Farry writes,

I'm late to the party on this one, but as I was catching up on my blog reading I came across your 18 April post ("Coalition Dynamics") and was a bit surprised to see that nobody spoke up in the comments about the actual political issue (as opposed to the model) addressed in the paper you criticize.

Really, we were just making our own independent decision. Just another example of unexplained group-level variance.

Um . . . no, "if the general election were held today" is not a particularly interesting question. The polls can move a lot in 5 months. Remember President Dukakis? See here (from our 1993 paper):

The triangles on the right side of each plot are the actual election outcomes, and the little arrows on each graph show the dates of the Democratic and Republican conventions in each year. As you can see, polls this early are in many cases not even close to the outcome.

I'm sure that Dr. Tyson means well, and I'm a big fan of Nova, but, really, he should talk with some political scientists before glibly writing about politics and concluding, "The political analysts need to take it from here." We've taken it pretty far already, dude! Tyson has every right to speculate about politics--I wouldn't claim that you need some sort of political science affiliation as a "union card" to do political science research--but it would make sense to ask around a bit, right? I mean, if a couple of political scientists wrote a paper on astrophysics in a journal called Mathematical and Computer Modeling . . . well, before trumpeting it in the New York Times I'd first go up to the 10th floor and ask my friend David, the astronomer, whether it's for real.

Andrew Smith sends in this:

He writes, "I think it beats the pie chart you referenced in your previous blog post! My brain still hurts trying to parse it."

In all seriousness: yes, a scatterplot would be better. And they gotta work on their axis labeling. "61.8?"

P.S. In contrast, the photographic height/weight chart is excellent.

Robert Rohrschneider writes:

I [Rohrschneider] am trying to gain an understanding of the pitfalls of multi-level analyses in my work which typically requires that I merge country data with surveys of individuals, usually in Europe. I wonder whether you could reduce my confusion about one issue of multilevel modeling in political science. It appears that the simple two-stage regression approach to multilevel data structures (i.e., a variant of which you and Jennifer Hill describe on p. 240 in your 2007 book) is gaining in popularity in political science.

We've chosen the winners of the ASC art contest!

Chris Zorn writes, http://graphics8.nytimes.com/packages/flash/politics/20080603_MARGINS_GRAPHIC/margins.swf

He's clearly a man of few words. I'll give it as a link. You can play with it, click on things, see all sorts of fun stuff.

What I'd really like to do is pipe this through a hierarchical model to smooth out the inevitable survey fluctuations. Also, it would be good to subtract off main effects. For example, in the graph below, are well-educated Arkansans particularly strong Clinton supporters, or is this just a combination of Arkansas being a Clinton state and small-sample fluctuation?

Anyway, I'm not complainin, just suggesting even more things that could be done with these data and this software. The first thing to do is to run it with the 2000 and 2004 exit polls. This app would go great with our Red State, Blue State book.

I don't know if I agree with this. I enjoy writing, but what I really enjoy is goofing off. If my books could write themselves, I'd choose that option.

But it is true (for me) that only through the writing does the content come out. I don't really know exactly what I want to say until I write it.

In their article, "High-Stakes Testing in Higher Education and Employment Appraising the Evidence for Validity and Fairness," Paul Sackett, Matthew Borneman, and Brian Connelly write:

As young adults complete high school in the United States, they typically pursue one of three options: continue their education, enter the civilian work force, or join the military. In all three settings, there is a long history of using standardized tests of developed cognitive abilities for selection decisions. In these domains, the tests themselves often are very similar. For example, Frey and Detterman (2004) reported a correlation of .82 between scores on the SAT, widely used for college admissions, and a composite score on the Armed Services Vocational Aptitude Battery.

The question is: are these tests any good? The authors say yes:

The authors review criticisms commonly leveled against cognitively loaded tests used for employment and higher education admissions decisions, with a focus on large-scale databases and meta-analytic evidence. They conclude that (a) tests of developed abilities are generally valid for their intended uses in predicting a wide variety of aspects of short-term and long-term academic and job performance, (b) validity is not an artifact of socioeconomic status, (c) coaching is not a major determinant of test performance, (d) tests do not generally exhibit bias by underpredicting the performance of minority group members, and (e) test-taking motivational mechanisms are not major determinants of test performance in these high-stakes settings.

Their key methodological point:

One thing I learned in econ class in 11th grade was that government policy should be counter-cyclical (spending more in recessions and cutting back in boom times), but that there’s a lot of pressure to be pro-cyclical, which will tend to exacerbate business cycles. (Except I suppose they didn’t say “exacerbate” in 11th grade.) At a personal level, too, it’s natural to spend more when we have more and cut back when we aren’t doing so well. Every now and then you hear about a “rainy day fund” but my general impression is that these are never big enough to counter the business cycle.

Political parties seem to apply a similar pro-cyclical behavior in their congressional election campaigns. Consider 2008. . .

Via Craig Newmark, I saw a column by John Lott summarizing his 1999 paper with Lawrence Kenny, "Did women's suffrage change the size and scope of government?" Lott and Kenny conclude Yes, by comparing the spending and revenue patterns of state governments before and after women were allowed to vote. I haven't looked at the analysis carefully and would need a little more convincing that it's not just a story of coinciding time trends (they have a little bit of leverage because women were given the vote sooner in some states than others), but the story is plausible, at least from the perspective of voting patterns nowadays.

On the other hand . . .

Here's the new paper by John Kastellec, Jamie Chander, and myself, and here's the abstract:

This paper predicts the seats-votes curve for the 2008 U.S House elections. We document how the electoral playing field has shifted from a Republican advantage between 1996 and 2004 to a Democratic tilt today. Due to the shift in incumbency advantage from the Republicans to the Democrats, compounded by a greater number of retirements among Republican members, we show that the Democrats now enjoy a partisan bias, and can expect to win more seats than votes for the first time since 1992. While this bias is not as large as the advantage the Republicans held in 2006, it is likely to help the Democrats win more seats than votes and thus expand their majority.

Here are our estimated seats-votes curves for 2006 and 2008:

As you can see, there used to be a strong Republican bias; now we estimate a Democratic bias. The change comes from the incumbency advantage (which we estimate to be about 8%, on average), which tends to lock in party control except in big swing years such as 1994 and 2006.

And here are some possibilities for 2008 in historical perspective:

The paper is a sequel to this article about the 2006 election.