September 2006 Archives

Paul Gronke writes,

I conducted the 100 pieces of candy demonstration and it did not work! Why?

I made one error that made it possible. Rather than using 80 pieces of small candy and 20 large, I allowed some "medium" (Halloween sized) candy bars.

But the real problem was this: one of the students was vegan and did not want to win. So she purposely chose five of the life savers in order do a "bad" job.
She ended up underestimating and getting closest ... thus winning the candy!!

P.S. The candy demo is described in Section 3 of this paper (and also in my book with Deb Nolan). I met Paul at this workshop on teaching statistics to political science students. One thing I remember about this workshop is that the participants, who taught political science at small colleges, seemed to have about 4 kids each.

Pre-Doctoral Clinical Research Fellowship at MSKCC

The Department of Psychiatry & Behavioral Sciences of Memorial Sloan-
Kettering Cancer Center (MSKCC; www.mskcc.org) invites applications for a
part-time pre-doctoral clinical research fellowship in the behavioral
aspects of cancer prevention and control.

A couple of debates seem to never stop: nature vs nurture, ability versus luck, role of society vs personal responsibility. The fundamental problem in these discussions is that one group of people considers one of the causes more important than the other one, and the other group disagrees. In this entry, I will attempt to show an explanation of this problem with my interaction analysis framework.

I have taken the "rodents" dataset. Cases are apartments in New York City, the covariates are the number of defects, the poverty score and the race for the apartment, whereas the outcome is whether there were rodents found in the building. The result of the analysis in the form of an interaction graph is as follows:

The defects are clearly by far the best predictors of rodents (13.2% of explained variation), this is followed by race (7.9%) and then by the poverty score (7.1%). What is important is that none of the covariates is explained away by the others. The links between covariates indicate the correction that is necessary as both covariates provide in part the same information about the outcome. In particular, should we predict rodents using poverty and race, the actual amount of variance explained would be 7.1+7.9-3.0=12.0%.

The trouble is that -3.0 factor. If race and poverty weren't correlated, it would be zero (or positive). But as they are correlated, there is ambiguity with respect to what is primary, race or poverty, in predicting the rodents. In particular, one could say that the increased frequency of rodents among minorities can be explained by poverty. With this, we would assign 7.1% of explained variance to poverty and 7.9-3.0=4.9% to race.

On the other hand, we could say that minorities have a cultural bias, an example of which is that don't keep as many pets like cats and dogs that prey upon rodents. Thus, cultural biases can explain an increased likelihood of rodents, along with, say, racist landlords that refuse to fix cracks in an apartment of a householder of the wrong race. Poverty could also be a consequence of these cultural biases (preferring one profession to another) or even race directly, either in terms of innate ability, in terms of discrimination or in terms of the "poverty trap". With such an interpretation we would allocate 7.9% of explained variance to race, a proxy for culture, and 7.1-3.0=4.1 to poverty.

Same data, same model, but two interpretations: because of the correlation between race and poverty, we do not know how to divide the 3% of shared information among the two variables. People will continue to disagree. Sometimes it is possible to resolve this dilemma when one variable completely explains away the other one, but this isn't the case here. What to do?

Jonathan Gardner and Andrew Oswald write,

One of the famous questions in social science is whether money makes people happy. We [Gardner and Oswald] offer new evidence by using longitudinal data on a random sample of Britons who receive medium-sized lottery wins of between £1000 and £120,000 (that is, up to approximately US$ 200,000). When compared to two control groups – one with no wins and the other with small wins – these individuals go on eventually to exhibit significantly better psychological health. Two years after a lottery win, the average measured improvement in mental wellbeing is 1.4 GHQ points.

Here's the paper. (Yes, Tables 2 and 3 should be graphs).

This paper has a funny history. I'd read an article by Jan de Leeuw (see here) that was pretty critical of Bayesian multilevel modeling, and I had the thought of writing a paper with Jan where we lay out where we agree and disagree on the topic. The idea would be to give the reader some idea of our overlap, which presumably would represent some safe zone, falling between Jan's complete skepticism and my naive faith. I told Jan I'd write a draft of an article with my perspective, then I could send to him and he could add his part. So I wrote my half, but then when I sent it to him, he said he actually agreed with what I wrote, so I should submit it as is. So I did. Perhaps I had internalized his critical view while writing the article.

Anyway, here's the abstract:

Multilevel (hierarchical) modeling is a generalization of linear and generalized linear modeling in which regression coefficients are themselves given a model, whose parameters are also estimated from data. We illustrate the strengths and limitations of multilevel modeling through an example of the prediction of home radon levels in U.S. counties. The multilevel model is highly effective for predictions at both levels of the model, but could easily be misinterpreted for causal inference.

and here's the article.

I don't know if I really believe this article by Lustick and Miodownick, but it's interesting. Following the analysis of simulations from a simple mathematical model, they write,

In this paper's introduction we [Lustick and Miodownick] mentioned the contrast between predictions by American policy-makers that a powerful cascade among Iraqis toward democracy and against the Saddam regime would be triggered by the American-led invasion. That did not occur. What cascades of change did occur included, in the first instance, local tips toward looting and, over subsequent years, convergence of networks of clans, Jihadi fundamentalists, angry patriots, and former Saddam loyalists, on patterns of violent resistance collectively known, now, as "the insurgency." Our findings lead us to conjecture that the expectation of a rapid and powerful tip toward revolution against Saddam's regime and toward democracy following the arrival of US troops was in part the result of ignoring the effects of spatiality. Convinced that the overwhelming majority of Iraqis would benefit from the fall of Saddam and that same majority would recognize this, American strategists expected ambitious Iraqis to launch strikes against Saddam's forces, thereby leading other Iraqis to join quickly in the battle so as to be identified with the new order. But as in our models, so too in Iraq, operative zones of knowledge were smaller--operating saliently at the clan, religious sect, tribal, ethnic, or regional levels. This helped produced differently directed cascades among different networks, a pattern that made the future that did ensue, featuring political and regional fragmentation and contrary beliefs among different segments of the population regarding the likely outcome of current political struggles, if not inevitable, than much more likely than the hoped for future of a universal tip toward a democratic, pro-American Iraq.

A similar application of our line of analysis could help Kuran explain why the dictatorial rule of Aleksander Lukashenko is still intact in Belarus. Although Lukashenko is enormously unpopular, fearful Belarussians prefer living the lie to risking personal loss by joining a "denim revolution" that might not succeed in toppling him. Recent reporting suggests that the smallness of the zones of knowledge of individual Belarussians deprive them of virtually all communication opportunities apart from word of mouth techniques among close friends and acquaintances. Until these zones are expanded by samizdat or other techniques, Lukashenko can sustain his dictatorship by winning phony "election" majorities, despite the activities of brave activists and the true preferences of the mass of citizens.

As I said, I don't know how much to believe it, but it would be good if these agent-based models could give insights into these sorts of contingent processes. Lustick will be speaking on the paper this Wednesday at noon.

Alan Reifman sends along a link to his webpage on weighting survey results to adjust for party ID.

Ben Hansen sent me this paper by Donald Pierce and Dawn Peters, about which he writes:

I [Ben] stumbled on the attached paper recently, which puts forth some interesting ideas relevant to whether very finely articulated ancillary information should be conditioned upon or coarsened. These authors' views are clearly that it should be coarsened, and I have the impression the higher-order-asymptotics/conditional inference people favor that conclusion.

The background on this is as follows:

1. I remain confused about conditioning and testing. I hate the so-called exact test (except for experiments that really have the unusual design of conditioning on both margins; see Section 3.3 of this paper from the International Statistical Review).

2. So I'd like to just abandon conditioning and ancillarity entirely. The principles I'd like to hold (following chapters 6 and 7 of BDA) are to do fully Bayesian inference (conditional on a model) and then use predictive checking (based on the design of data collection) to check the fit.

3. But when talking with Ben on the matter, I realized I still was confused. Consider the example of a survey where we gather a simple random sample of size n, fit a normal distribution, and then test for skewness (using the standard test statistic: the sample third moment, divided by the sample varicance to the 3/2 power). The trick is that, in this example, the sample size is determined by a coin flip: if heads, n=20, if tails, n=2000. Based on my general principles (see immediately above), the reference distribution for the skewness test will be a mixture of the n-20 and the n=2000 distribution. But this seems a little strange, for example, what if we see n=2000--shouldn't we make use of that information in our test?

4. In this particular example, I think I can salvage my principles by considering a two-dimesional test statistic, where the first dimension is that skewness measure and the second dimesion is n. Then the decision to "condition on n" becomes a cleaner (to me) decision to use a particular one-dimensional summary of the two-dimensional test statistic when comparing to the reference distribution.

Anyway, I'm still not thrilled with my thinking here, so perhaps the paper by Pierce and Peters will help. Of course, I don't really care about getting "exact" pvalues or anything like that, but I do want a general method of comparing data to replications from the assumed model.

A commented pointed out this note by Kevin Drum on this cool paper by Alan Gerber and Neil Malhotra on p-values in published political science papers. They find that there are suprisingly many papers with results that are just barely statistically significant (t=1.96 to 2.06) and surprisingly few that are just barely not significant (t=1.85 to 1.95). Perhaps people are fuding their results or selecting analyses to get significance. Gerber and Malhotra's analysis is excellent--clean and thorough.

Just one note: the finding is interesting, and I love the graphs, but, as Gerber and Malhotra note,

We only examined papers that listed a set of hypotheses prior to presenting the statistical results. . . .

I think it's kind of tacky to state a formal "hypothesis," especially in a social science paper, partly because, in many (most?) of my research, the most interesting finding was not anything we'd hypothesized ahead of time. (See here for some favorite examples.) I think there's a problem with the whole mode of research that focuses on "rejecting hypotheses" using statistical significance, and so I'm sort of happy to find that Gerber and Malhotra notice a problem with studies formulated in this way.

Slightly related

In practice, t-statistics are rarely much more than 2. Why? Because, if they're much more than 2, you'll probably subdivide the data (e.g., look at effects among men and among women) until subsample sizes are too small to learn much. Knowing this can affect experimental design, as I discuss in my paper, "Should we take measurements at an intermediate design point?"

Tian links to a document showing hundreds of shades of colors in R. I don't think I would've listed them alphabetically, but it is convenient to see them all in one place. When picking out colors, don't forget that they look different on the computer, projected onto a screen, and on paper.

Allan Cohen writes,

Here (link from Jenny Davidson). Is there something similar for other countries?

P.S. Lotsa good links in the comments below.

Fredric Jameson writes,

Take the new definition of the superego. No longer the instance of repression and judgment, of taboo and guilt, the superego has today become something obscene, whose perpetual injunction is: ‘Enjoy!’

There's some truth to this unexpected statement, I think. But the funny thing is that I read the article at all, and that, having started the article, I got far enough to reach this quote. An article by Fredric Jameson on Slavoj Zizek has gotta be the last thing I'd want to read.

I've been told recently that there is actually no good evidence that alcohol is good for your heart. But it may be good for your wallet. Gueorgi sent me this:

Drinking Alcohol Can Lead to Fatter Pay Checks, Study Says

Sept. 15 (Bloomberg) -- Drinking alcohol can fatten your pay check, according to a Reason Foundation study published in the Journal of Labor Research.

Men who visit a bar at least once a month to drink socially bring home 7 percent more pay than abstainers, and women drinkers earn 14 percent more than non-drinkers, according to the study by economists Bethany Peters and Edward Stringham.

"Social drinking builds social capital," Stringham, a professor at San Jose State University, said in a press release. "Social drinkers are out networking, building relationships, and adding contacts to their BlackBerries that result in bigger paychecks."

The report, published in Los Angeles, questions the economic effects of anti-alcohol legislation at sports stadiums and festivals.

"Instead of fear-mongering we should step back and acknowledge the proven health and economic benefits that come with the responsible use of alcohol," Stringham said.

Here's the report, and here's the link to the full article in the Journal of Labor Research. I had no ideat that drinkers (in the U.S.) make 10% more money than nondrinkers, but this is apparently a well-known fact with a literature going back nearly 20 years. 10% more is a lot! In this paper, Peters and Stringham actually find that drinkers make 20% more than nondrinkers, on average. After controlling for age, ethnicity, religion, education, marital status, parents' education, number of siblings, and region of the country, they find a coefficient of drinking of over 10%. That would seem to more than cover the cost of the drinks themselves (for example, two $5 drinks per week comes to only 1.7% of a $30,000 (after-tax) salary).

The difference between "significant'' and "not significant'' is not itself statistically significant

The researchers also find that people who attend a bar at least once per month (which they perhaps misleadingly call "bar-hoppers") to have higher earnings than other drinkers, again after controlling for the other variables. The coefficient for "bar-hopping" is higher for men than for wonen, in fact significant for men but not for women, but the difference between this coefficient for the two sexes is not statistically significant.

I don't really want to pick on Peters and Stringham here, since this is such an extremely common mistake (which also wasn't picked up by the referees of the paper). The comparison between men and women is also a small part of the study. It's just funny to me to see this mistake here, where I wasn't really looking for it. It's one of those perception things, like if you get a dog, then all of a sudden you notice that everybody in the neighborhood seems to have dogs. I'm just super attuned to this particular statistical point, having just written a paper on it. (I'd also comment that the tables would be better displayed graphically but I fear I've worn out my welcome by now.)

How to think about this?

I don't really know. Obviously lots of criticisms could be made, most notably that these social people might make more money and also drink more, but maybe they'd make more money even if they didn't drink as much. On the other hand, the pattern appears to be there in the data. I guess I'd be more cautious about the causal interpretation, but, causal or not, it's an interesting finding, as is the connection to social capital.

Similarly, the policy recommendations are interesting, but the research could be taken in different directions. The article says,

Our analysis leads to a number of policy implications. Most importantly, restrictions on drinking are likely to have harmful economic effects. Not only do anti-alcohol policies reduce drinkers’ fun, but they may also decrease earnings.

But, another way to say it is that drinkers are richer than nondrinkers, and so restrictions on drinking harm the relatively well-off and are thus not such an onerous social burden. In any case, the issues of public healh, individual liberty, and economic effects have to be balanced in some way in deciding about laws like this, and this paper seems relevant to the debate. I assume that economists have also done city- or country-level analyses to estimate the effects of alcohol restrictions on the local economy.

Unintended consequences?

Peters and Stringham write, "One of the unintended consequences of alcohol restrictions is that they push drinking into private settings." I'm just wondering: was that really unintended? My recollection from 8th grade history or whatever is that the temperance crusaders were no fans of saloons, and they may have actually felt that drinking behind closed doors would not be so bad.

One more thing . . .

The authors use the General Social Survey and conjecture about social networks. We have some questions on the 2006 General Social Survey to estimate network size (using the method described here). I don't know if the 2006 GSS has any questions on drinking (or, for that matter, other aspects of drug use) but if it does, I think there's room for a followup study making use of our social network information. As with the current study, we wouldn't know to what extent drinking expands the social network, and to what extent already-popular people are drinking. But it would be interesting to see the data.

Eugene Lavely writes,

I work at BAE Systems AIT here in the Boston area. We have a very interesting research program for imaging building interiors that is just starting. I believe the unique features of this inverse problem leads one to MCMC methods such as RJMCMC. The specific statistical techniques that we would like to apply are described in the second paragraph of the ad below. We seek candidates with outstanding expertise in these areas. Our position is available for immediate start. BAE has a very academic atmosphere,and so academic scientists fit in here very well, and have significant opportunities to develop and apply innovative research.

In the discussion of this entry by Peter Woit on open-access publishing, John Baez writes about the motivations of scientists who write for academic journals. He has some interesting things to say, but I think he's missing some big issues. Here's Baez:

So, even as the university library is crying for help, struggling to pay the ever-growing costs of journals run by the big media conglomerates, the science faculty continues to publish in these journals, because their careers depend on it.

The science faculty also work as editors for these journals, typically for no pay - just for the prestige of being on the editorial board. They also work without pay refereeing articles for these journals. They write papers that appear in volumes published by the same media conglomerates, again just for the prestige of having a paper in a prestigious volume. And, they write books for presses owned by the same conglomerates.

I [Baez] find these activities to be a bit more craven, because I haven’t seen people getting hired just because they do these things. But, just as millionaires work their ass off to become billionaires, a lot of the most prestigious scientists engage in these activities to polish their reputations to an ever finer sheen. This is especially true of people who have given up trying to do original research.

I [Baez] get lots of invitations to write books and papers for various collections, because people know I can write. These days I almost always turn them down. I’ve learned a key fact: when someone gives me an honor, it’s usually a way to get me to do work for free. I still give lots of talks, because I get free travel out of it, and I really enjoy explaining stuff. But writing review papers for volumes published by prestigious publishers - that’s something I’ve come to really dislike.

Each time I turn such an offer down, I feel a little ache, because I know I’ll miss out on a little piece of prestige. For example, I could have contributed to the forthcoming “Princeton Companion to Mathematics”. I almost did - what a great opportunity! But I didn’t. I’d rather do whatever the hell I want on a given day - usually thinking about math and physics. I’m in an incredibly lucky position where I can afford to do this; it seems insane not to.

In short, to understand what’s going on, you have to realize: big companies care about profits, academics care about prestige.

My comments:

Yes, prestige (buffing our reputations to "an ever finer sheen") is certainly a big issue, maybe partly because we've been trained to do this ever since we were in high school and trying for that perfect grade point average that would get us into the right college, etc. (I imagine it's even worse in poorer countries where there's major competition to get into college at all.)

But I think the motivation is more altruistic. I don't know what's in the Princeton Companion to Mathematics, but in general I don't think that much, if any, prestige, comes from writing encyclopedia articles. When I do such things, it's more out of a sense of service to the community or from some desire to communicate: my thought is, if people are going to read an encyclopedia article on "statistical graphics" or "Bayesian statistics" or whatever, I'd like them to learn the real thing (i.e., my perspective on it). These things take time, though, so I can understand why Baez said no to the invitation to contribute to the volume.

Ultimately the reason that I (and others, I think) do much of our research (and why we give talks) is: if I've gone to the trouble of figuring something out, I'd like others to hear about it and make use of it. Basically an altruistic (or evangelical) motivation.

That said, I agree with much of Baez's criticism of academic publishing. It does seem a bit weird to me that authors and referees provide services for free. Nowadays the role of journals seems to be to give a stamp of approval on research, not to actually "publish," so it's not clear why the system should be so expensive. Peer review is another screwed-up system, but that's another story.

A dermarcation?

A cynic might say, how can we distinguish between "prestige" (Baez's claim) and "altruism" (my claim). One demarcation would be to consider your reaction if an idea or paper, equivalent to yours, is published by someone else. You get no prestige from this, but the outcome to the scientific field is the same as if you had published it yourself. Thinking about it, I admit that, all things equal, I'd prefer to have the article published under my name (although, still, I don't think of this as "prestige" but more as "credit"), but I'd still be happy to have it out there. I certainly feel that way about review articles (which is one reason that XIao-Li and I edited a volume of review articles ourselves).

Jenny Davidson writes,

I [Davidson] find myself only very infrequently really loving regular contemporary grown-up literary short stories . . . but both as a child and in my adult life I have had a complete passion for what might be called tales of the uncanny. . . Robert Louis Stevenson, the more thrilling efforts of Henry James, Saki's stories (which are funny rather than primarily uncanny, but it's the exception that proves the rule), Joan Aiken's absolutely wonderful tales and also, and most particularly, the stories of Roald Dahl . . and Poe . . . and Sherlock Holmes. . . . that kind of story is how you get from reading children's books to grown-up ones; crime fiction also provides a useful bridge.

I'm with her on Stevenson and would also add John Collier, who wrote somewhat low-grade versions of these twisty stories in the first half of the last century. I have this book called Bedside Tales from around 1940 (I found it at a tag sale, I think) that's full of really fun short stories, including very readable ones from Fitzgerald, Hemingway, etc., as well as John Collier and others (and also that story that always found its way into anthologies, "The Most Dangerous Game," about the rich guy who hunts people for sport). The book also had a silly, yet perfect, Peter Arno cover, but unfortunately I lent it to someone who lost the dust jacket. Who are these people who think that the dust jacket doesn't matter??

In a comment on this entry on active learning in large classes, Bill Tozier links to "a somewhat more ambitious program, which takes active learning principles in a more forcefully collaborative direction." It's worth reading--I can't really figure out how to excerpt it, so you can just follow the link--the basic idea is that the students in his (hypothetical course) are required to collaborate on dozens of homework assignments throughout the semester. The hard part, I think, is coming up with that long list of homework assignments, but in any case I like the idea.

It also reminds me of the general principle that just about any teaching method (or, for that matter, research method) can work well, as long as (a) you put in the effort to do it right, and (b) keep in mind the ultimate goal, which is for the students to have certain skills and certain experiences by the time the class is over. Related to these is (c) the method should be appropriate for your own teaching style. Even old-fashioned blackboard lecturing is fine--if you can pull it off in a way that keeps the students' brains engaged while you're doing it. I developed a more active teaching style for myself because that was the only way I could keep the students thinking.

Rebecca pointed me to this interesting article by Ben Hoyle in the London Times, "Helmeted cyclists in more peril on the road." Hoyle writes:

Cyclists who wear helmets are more likely to be knocked off their bicycles than those who do not, according to research.

Motorists give helmeted cyclists less leeway than bare-headed riders because they assume that they are more proficient. They give a wider berth to those they think do not look like “proper” cyclists, including women, than to kitted-out “lycra-clad warriors”.

Ian Walker, a traffic psychologist, was hit by a bus and a truck while recording 2,500 overtaking manoeuvres. On both occasions he was wearing a helmet.

During his research he measured the exact distance of passing traffic using a computer and sensor fitted to his bicycle.Half the time Dr Walker, of the University of Bath, was bare-headed. For the other half he wore a helmet and has the bruises to prove it.

He even wore a wig on some of his trips to see if drivers gave him more room if they thought he was a woman. They did.

He was unsure whether the protection of a helmet justified the higher risk of having a collision. “We know helmets are useful in low-speed falls, and so are definitely good for children.”

On average, drivers steered an extra 3.3 in away from those without helmets to those wearing the safety hats. Motorists were twice as likely to pass “very close” to the cyclist if he was wearing a helmet.

Not just risk compensation

This is interesting: I was aware of the "risk compensation" idea, that helmeted riders will ride less safely, thus increasing the risk of accident (although the accident itself may be less likely to cause serious injury), as has been claimed with seat belts, antilock brakes, and airbags for cars. (If it were up to me, I would make car bumpers illegal, since they certainly seem to introduce a "moral hazard" or incentive to drive less carefully.)

But I hadn't thought of the idea that the helmet could be providing a signal to the driver. From the article, it appears that the optimal solution might be a helmet, covered by a wig . . .

The distinction between risk compensation altering one's own behavior, and perceptions altering others' behavior, is important in making my own decision. On the other hand, my small n experience is that I have a friend who was seriously injured after crashing at low speed with no helmet. So it's tricky for me to put all the information together in making a decision.

Attitudes

The news article concludes with,

He [Walker] said: “When drivers overtake a cyclist, the margin for error they leave is affected by the cyclist’s appearance. Many see cyclists as a separate subculture.

“They hold stereotyped ideas about cyclists. There is no real reason to believe someone with a helmet is any more experienced than someone without.”

I don't know the statistics on that, but I do think there's something to this "subculture" business. People on the road definitely seem to have strong "attitudes" to each other based on minimal information.

Self-experimentation

Finally, Rebecca pointed out that this is another example of self-experimentation. As with Seth's research, the self-experimenter here appears to have a lot of expert knowledge to guide his theories and data collection. Also amusing, of course, is that his name is Walker.

Peter has some interesting thoughts on open access publishing:

There’s a big debate within the scientific community in general about how and whether to move away from the conventional model of scientific publishing (journals supported by subscriptions paid by libraries, only available to subscribers) to a model where access to the papers in scientific journals is free to all (”Open Access”). The main problem with this is figuring out how to pay for it. . . .

The CERN task force proposes raising $6-8 million/year over the next few years to start supporting the half of the journals (not including Elsevier ones) that it has identified as ready for Open Access. . . . What is being proposed here is basically to give up on what a lot of people have hoped would develop: a model of free journals, whose cost would be small since they would be all-electronic, small enough to be supported by universities and research grants. Instead the idea here is to keep the current journals and their publishers in place, just changing the funding mechanism from library subscriptions to something else, some form that would fund access for all. . . .

Peter gives some reasons why he doesn't think this plan will work, and the subsequent discussion has some thoughts about the whole system in which scientists submit articles for free and review papers for free, then publishers make money selling their work. I have more thoughts on this, which I'll try to organize at some point, but for now let me just say that things in statistics and political science seem a bit better than in physics. Our major journals are organized by the American Statistical Association, Midwest Political Science Association, and so forth, so at least we don't have to worry so much about the interests of the publishers (which apparently is a big deal lin physics, to judge from Peter's comments.)

This is important to researchers for (at least) two reasons: (1) the format and availability of publishing affects who sees our work and thus affects the course of future research; (2) those of us who write for refereed journals waste a lot of time tailoring articles to the desires (or the perceived desires) of the referees.

Carrie writes,

File this one under News of the Weird:

Health Journal: Hip government exercise campaign looks for its next move

The story is about the apparent success of the Center for Disease Control's "verb" ad campaign -- designed to fight obesity among children and teens. A recent study in the journal Pediatrics found that kids who had seen the Verb campaign "reported one-third more physical activity during their free time than kids who hadn't."

Carrie expresses skepticism since it's hard to see that cryptic ads could really make such a difference in bahavior. In addition, it's an observational study: the ads were shown everywhere, then they compared kids who recalled seeing the ads to kids who didn't. They did a baseline study, so they could control for pre-treatment level of exercise, but they didn't do much on this. I would have liked to see scatterplots and regressions.

Here's the article in the journal Pediatrics reporting the comparison of exercise levels for kids who recalled or didn't recall the ad campaign. Perhaps an interesting example for a statistics or policy analysis class. As usual, I'm not trying to shoot down the study, just to point out an interesting example of scientific ambiguity. I'd think there's lots of potential for discussion about how a future such study could be conducted.

I once asked Don Rubin if he was miffed that some of his best ideas, including the characterization of missing-data processes ("missing completely at random," "missing at random," etc.) and multiple imputation are commonly mentioned without citing him at all. He said that he actually considers it a compliment that these ideas are so accepted that they need no citation. Along those lines, he'd probably be happy to know that we're now getting unsolicited emails of the following sort:

Nadia Urbaniti, a professor in the political science department here, just published a book, Representative Democracy: Principles and Genealogy, with the following thesis:

It is usually held that representative government is not strictly democratic, since it does not allow the people themselves to directly make decisions. But here, taking as her guide Thomas Paine's subversive view that "Athens, by representation, would have surpassed her own democracy," Nadia Urbinati challenges this accepted wisdom, arguing that political representation deserves to be regarded as a fully legitimate mode of democratic decision-making-and not just a pragmatic second choice when direct democracy is not possible.

I haven't read the book yet, but, based on this abstract, I like what I see so far. My impression from the work in social and cognitive psychology on information aggregation is that representative democracy will work better than dictatorship or pure democracy. (See also the discussion here of democracy and its alternatives.)

William Huang's comment on this entry reminds me of a question that sometimes arises, which is how to do active learning in large classes with more than 50 students. I've never actually taught a class with more than 50, but what I'd like to do is teach a large intro statistics class (200-300 students) with about 6 or 10 teaching assistants who would be required to come to class. As the lecturer, I'd keep things going, start the activities, and use the blackboard as appropriate. The T.A.'s would circulate and make sure the students are clear on what they should be working on. T.A.'s would also be able to answer questions and help out.

I plan to try this out in a couple years (once I get the course all set up). Perhaps others have relevant experience along these lines? One reason I've worked so hard on class-participation activities is that I used to have a lot of diffiiculty getting students to speak up in class. The demonstrations and activities have made a big difference (or so I say in the spirit of self-experimentation, not actually having done a formal study or experiment on my teaching methods). My next step is to make it more clear what the students are expected to learn, and to actively monitor their learning. (See here for an inspirational model I'd like to follow.)

Stuart Buck has an interesting story (linked from Tyler Cowen and Jane Galt of a map that was published in the newspaper showing gains and losses in median household incomes. Apparently the graph (from the Detroit Free Press) was mistaken. Buck writes,

Let's take my home state of Arkansas. According to the Census Bureau's page, Arkansas' 1999 median household income -- in 2005 dollars -- was $34,770. Then in 2005, the median household income was $36,658. That's an increase of 5.4%, as opposed to the 7.2% decrease that the Detroit Free Press claims to have found.

How about another state: Utah. In 1999 (again, in 2005 dollars): $53,943. In 2005: $54,813. That's a rise of 1.6%, not a decline of 10.5% as the Free Press claims. . . .

The first journalist then followed up and explained further that the 1999 data came from the 2000 Census (it's available here). They used the inflation calculator recommended by the Census Bureau. And then the 2005 data came from the American Community Survey (here). . . .

Estimates from any one survey will almost never exactly match the estimates from any other (unless explicitly controlled), because of differences such as in questionnaires, data collection methodology, reference period, and edit procedures.

Most importantly here, the American Community Survey seems, for whatever reason, to produce lower results than the official Census figures. For example, in one detailed analysis comparing ACS to the Census in a couple of counties, the Bureau reported:

There were significant differences in the estimation of median household income. In Tulare County, the Census reported a value of $33,983 compared to the ACS estimate of $31,467. This is consistent with Census Bureau research in other ACS sites that generally found lower income values reported in the ACS . . . .

This seems like a great example for a statistics (or policy analysis) class. Of course, the ultimate solution is not to give up but to get parallel series of both surveys (if possible) to better adjust for differences in making comparisons.

The other thing to be considered is uncertainty. Looking at the linked webpage, I see some big standard errors. For example, considering Stuart Buck's example of Arkansas, we see $36,700 +/- 1400 (for 2005) and $34,800 +/- 1200 (for 1999). Assuming independent surveys (which maybe isn't right), the difference is $1900 +/- 1800. That is, a difference of 5.4% +/- 5.2%. With numbers like these, it seems a little silly to be looking at individual states.

There is a statistical message here, too, which is that differences are hard to estimate precisely (unless they are studied using a panel design which keeps the data comparable from year to year).

P.S. See here for a table showing how variable the state estimates are--with color and two significant digits included to make the noise be even more visible! There are many comments on that blog entry, and they all seem to be taking the numbers at face value.

Aleks pointed me toward this pretty picture:

I'd prefer colored dots rather than shading. (Ideal would be something like one dot per 10,000 people of each religion, or something like that, I think.) But still, it's interesting. I'd like to see the one for each decade going back 100 years.

A finance professor writes,

I am currently working on a project and am looking for a test. Unfortunately, none of my colleagues can answer my question. I have a series of regressions of the form Y= a + b1*X1 + b2*X2. I am attempting to test whether the restriction b1=b2 is valid over all regressions. So far, I have an F-test based on the restriction for each regression, and also the associated p-value for each regression (there are approximately 600 individual regressions). So far, so good.

Is there a way to test whether the restriction is valid "on average"? I had thought of treating the p-values as uniformy distributed and testing them against a null hypothesis that the mean p-value is some level (i.e. 5%).

I figure that there should be a better way. I recall someone saying that a sum of uniformly distributed random variates is distribted Chi-squared (or was that a sum of squared uniforms?). In either case, I can't find a reference.

My response: if the key question is comparing b1 to b2, I'd reparameterize as follows:
y = a + B1*z1 + B2*z2 + error, where z1=(X1+X2)/2, and z2=(X1-X2)/2. (as discussed here)
Now you're comparing B2 to zero, which is more straightforward--no need for F-tests, you can just look at the confidence intervals for B2 in each case. And you can work with estimated regression coefficients (which are clean) rather than p-values (which are ugly).

At this point I'd plot the estimates and se's vs. some group-level explanatory variable characterizing the 600 regressions. (That's the "secret weapon.") More formal steps would include running a regression of the estimated B2's on relevant group-level predictors. (Yes, if you have 600 cases, you certainly must have some group-level predictors.) And the next step, of course, is a multilevel model. But at this point I think you've probably already solved your immediate problem.

Matt Salganik is teaching introductory statistics this year and would like to do lots of class-participation activities. He came up with the idea of giving each student a mini-whiteboard (the little ones that you can affix to a refrigerator door) and marker, so that when questions come up in class, each student can sketch his or her answer and hold it up for him to see. This seems like a great idea to me. I had only two suggestions:

1. Make it one whiteboard for each pair of students. I've found that students can focus better in class when they are working in pairs--it's harder for them to just gaze off into space and give up.

2. Hand the boards out at the beginning of each lecture and have the students hand them back at the end of lecture. If you let the students keep the boards, they'll inevitably forget to bring them to class, have to borrow from each other, etc.

There's an interesting article on Wikipedia by Aaron Swartz. Swartz comes to a similar conclusion as I did--Wikipedia and traditional encyclopedias are actually structured similarly--but he approaches the question from an encyclopedia editor's point of view, whereas I was generalizing from my experience as an encyclopedia contributor. Here's what Swartz reported:

So did the Gang of 500 [central Wikipedia participants] actually write Wikipedia? Wales decided to run a simple study to find out: he counted who made the most edits to the site. . . it turns out over 50% of all the edits are done by just .7% of the users ... 524 people. ... And in fact the most active 2%, which is 1400 people, have done 73.4% of all the edits. . .

Curious and skeptical, I [Swartz] decided to investigate. I picked an article at random ("Alan Alda") to see how it was written. . . Wales seems to think that the vast majority of users are just doing the first two (vandalizing or contributing small fixes) while the core group of Wikipedians writes the actual bulk of the article. But that's not at all what I found. Almost every time I saw a substantive edit, I found the user who had contributed it was not an active user of the site. . .

If you just count edits, it appears the biggest contributors to the Alan Alda article (7 of the top 10) are registered users who (all but 2) have made thousands of edits to the site. Indeed, #4 has made over 7,000 edits while #7 has over 25,000. In other words, if you use Wales's methods, you get Wales's results: most of the content seems to be written by heavy editors. But when you count letters, the picture dramatically changes: few of the contributors (2 out of the top 10) are even registered and most (6 out of the top 10) have made less than 25 edits to the entire site. In fact, #9 has made exactly one edit -- this one! With the more reasonable metric -- indeed, the one Wales himself said he planned to use in the next revision of his study -- the result completely reverses. . . .

When you put it all together, the story become clear: an outsider makes one edit to add a chunk of information, then insiders make several edits tweaking and reformatting it. In addition, insiders rack up thousands of edits doing things like changing the name of a category across the entire site -- the kind of thing only insiders deeply care about. As a result, insiders account for the vast majority of the edits. But it's the outsiders who provide nearly all of the content. . . . Other encyclopedias work similarly, just on a much smaller scale: a large group of people write articles on topics they know well, while a small staff formats them into a single work.

I would add only one comment (besides what I wrote before). Swartz writes:

And Wikipedia should too. Even if all the formatters quit the project tomorrow, Wikipedia would still be immensely valuable. For the most part, people read Wikipedia because it has the information they need, not because it has a consistent look. It certainly wouldn't be as nice without one, but the people who (like me) care about such things would probably step up to take the place of those who had left. The formatters aid the contributors, not the other way around.

My response: I know what he's saying here, but I don't know if it's so true in general. I imagine the common format is a big part of Wikipedia's appeal--it sort of makes it into the McDonald's of information sources. I suspect that the clean and uniform format is a large part of Wikipedia's air of authority.

O'Malley and Zaslavsky recommend a scaled-inverse-Wishart model, as we discuss in Section 13.3 of our forthcoming book.. (We took the idea from an earlier draft of the O'Malley/Zaslavsky paper.) The idea is to break up the covariance matrix into a diagonal matrix of scale parameters and an unscaled covariance matrix which is given the inverse-Wishart distribution. This larger miodel is still conditionally conjugate on the larger space. The model can be thought of as a generalization of the half-t model that I describe here.