Results matching “R”

I just today learned about an organization called SourceWatch--they have an article on the tobacco connections of the well-known sociologist Peter Berger. Beyond the inherent interest of the topic, I was fascinated by the way that the Sourcewatch webpage mimicked Wikipedia:

This is a smart move, I think: for better or worse, Wikipedia is generally considered to be authoritative.

But then I was thinking . . . is this the beginning of the end for Wikipedia. I don't know anything about Sourcewatch, if they're good guys or bad guys or whatever--but if they can mimic Wikipedia, I'm sure lots of other organizations could do so too. And, when they do it, all of a sudden there will be a lot of authoritative-looking Wikipedia-like pages floating around, a sort of counterfeit money devaluing the "real" 'pedia, which will then have to respond by branding itself--"100% real Wikipedia, accept no imitations"--and so on. Not a bad thing, perhaps, but not what we have now.

From Aaron Swartz, a link stating that famous sociologist Peter L. Berger was a big-time consultant for the Tobacco Insitute:

Peter L. Berger is an academic social philosopher and sociologist who served as a consultant to the tobacco industry starting with the industry's original 1979 Social Costs/Social Values Project (SC/SV). According to a 1980 International Committee on Smoking Issues/Social Acceptability Working Party (International Committee on Smoking Issues/SAWP) progress report, Berger's primary assignment was "to demonstrate clearly that anti-smoking activists have a special agenda which serves their own purposes, but not necessarily the majority of nonsmokers."

Masanao sends in this.

(The oven that Masanao was referring to is described here.)

Not any more.

Kavalier and Clay impressed me right away, even before I ever even held a copy in my hand, because for a year or so after it came out, I kept seeing people reading it. In the subway, in the park, everywhere. This was a book that people really wanted to read. So I bought it and read it and was duly impressed. It's a great book: DeLillo without the irony, if you will. The book Don DeLillo might have written had he been Michael Chabon and had the interests Chabon had instead of the interests that DeLillo has. Whatever.

More recently, I read (most of) Chabon's first book of stories, A Model World. I was into it for about a story and a half, and then I realized that these were all John Updike stories. Don't get me wrong here--Updike is my hero, and it's pretty impressive to me that he continues to write stories even after he's no longer around. And i don't really hold it against him (Updike) that he, like Gore Vidal, couldn't come up with good book titles. Chabon, though, he's a good titler. But I couldn't get into his book. It was just too weird that he was writing Updike stories.

But then I read Chabon's second book of stories, Werewolves in their Youth. Much better. Good to see someone getting better at what he does.

Lane Kenworthy, Yu-Sung Su, and I write:

Income inequality in the United States has risen during the past several decades. Has this produced an increase in partisan voting differences between rich and poor? We examine trends from the 1940s through the 2000s in the country as a whole and in the states. We find no clear relation between income inequality and class-based voting.

This article will appear in a special issue of Social Science Quarterly on the topic of "Inequality and Poverty: American and International Perspectives." We have some pretty graphs, some of which appeared in the Red State, Blue State book and some of which didn't.

P.S. "We find no clear relation . . .": That works great in an academic article but I don't think we'll be grabbing the headlines anytime soon.

After reading what Christian wrote about the solutions manual that he and his collaborator wrote for their Bayesian Core book, I'm reminded of my own efforts with the Bayesian Data Analysis solutions. Not long after the first edition of the book came out, nearly fifteen years ago, I wrote up solutions to fifty of the homework problems (out of a total of about 150). People sometimes ask for more solutions, but the difficulty is that, once you have official solutions, you want them to be correct, you want them to be clear, and you want them to illustrate good statistical practice. It's a lot of work. Somehow I was able to write up those fifty, back when I had more time on my hands, but, really, writing up another fifty would almost be the equivalent of writing a (short) book! Originally I thought I could quickly put together a complete or nearly complete set by gathering solutions from students, or people just emailing them in, but I quickly realized that this wouldn't work. I think it would be ok to post scanned-in versions of student solutions, but once I start typing them up, I need them to be cleaner, and that takes work. That's one reason I didn't even try to write a solution set for ARM.

In suggesting "a socially responsible method of announcing associations," AT points out that, as much as we try to be rigorous about causal inference, assumptions slip in through our language:

The trouble is, causal claims have an order to them (like "aliens cause cancer"), and so do most if not all human sentences ("I like ice cream"). It's all too tempting to read a non-directional association claim as if it were so -- my (least) favourite was a radio blowhard who said that in teens, cellphone use was linked with sexual activity, and without skipping a beat angrily proclaimed that giving kids a cell phone was tantamount to exposing them to STDs. . . . So here's a modest proposal: when possible, beat back the causal assumption by presenting an associational idea in the order least likely to be given a causal interpretation by a layperson or radio host.

Here's AT's example:

A random Google News headline reads: "Prolonged Use of Pacifier Linked to Speech Problems" and strongly implies a cause and effect relationship, despite the (weak) disclaimer from the quoted authors. Reverse that and you've got "Speech Problems linked to Prolonged Use of Pacifier" which is less insinuating.

It's an interesting idea, and it reminds me of something that really bugs me.

Phil Turk wrote:

Bill Browne sends in this interesting job possibility. Closing date for applications is 30 Oct 2009, so if you're interested, let him know right away!

Here.

Song Qian writes:

I am very pleased to see your comment on not analyzing data without context. Would you please elaborate the reasons on your blog? I have been teaching an intro data analysis class to our professional masters students since 2005. One thing I have emphasized is the understanding of the underlying scientific problem before conducting any data analysis. This point is not always well-taken. Thanks.

My response: From a Bayesian point of view, it's pretty clear: no context = no prior information. It's really more than that, though, since the context structures the model itself, not just the numerical information that you use to regularize parameter estimates. For the climate change example, Bill Jefferys provides a good discussion here on what you can get from substantive knowledge.

Richard Morey asks:

I was wondering if you or your blog readers have any experience using cloud computing to do simulations or analyses. The idea of packaging a simulation and having many copies of it running on a cloud (like, say, Amazon's EC2) is appealing. And using it for storage would be nice too.

I'd like to get the opinion of statisticians who might have used this approach before spending time figuring it out.

I have no idea. Any suggestions are welcome.

Thanks for all the suggested titles. My current favorite remains, "If You Don't Buy This Magazine, We'll Kill This Blog." Although, I have to admit, "Super-Duper-Freakanomics" [sic] wasn't bad either. And, as much as I like the idea of calling it "Mister P," I can't quite pull the trigger on that one.

To respond to some of your comments:

1. No, I can't just post the general-interest entries at the new blog. That would take a lot of the fun out of the current blog. And the Science Blog people don't want me to cross-post more than 4 items per month. I will, of course, link to the new items from the current blog, but it's not as good if I can't cross-post them.

2. I agree that Science Blogs isn't the same as what I'm doing here, that's why I just wanted to post some stuff there, to reach the different audience, without losing what we have here.

3. I don't plan to be doing anything extra with this new blog; I see it more as a place to post a few things that I was going to post somewhere anyway.

4. Someone commented that it's strange for me to ask for a title before deciding on a topic. I thought it was implicit that, by asking for a title, I'm also asking for suggestions on a topic. I guess I'll try two or three posts a week and see how it goes.

Finally, in all seriousness, if nobody comes up with a better title, I'm going to call it "Applied Statistics." And I'll kick it off with a few posts about literature. Consider yourselves warned.

This news article has made a bit of a splash: Seth Borenstein sent around a temperature time series to four statisticians--just sending the numbers without saying where they came from--and the statisticians uniformly concluded that there were no consistent temperature declines over time:

"If you look at the data and sort of cherry-pick a micro-trend within a bigger trend, that technique is particularly suspect," said John Grego, a professor of statistics at the University of South Carolina.

I don't have anything to add on the temperature series--there's only so much you can learn from a context-free data analysis, and I don't think anyone would want to take this particular set of blind statistical analyses as being at all informative about the science. But there's more going on here.

I'd heard this before, but good advice is typically worth repeating. For those of you who program in R, I'd also recommend that scripts be written so it can run from scratch in an empty R environment. Many many times I've found my R scripts and environments to be palimpsests whose meanings are difficult to unravel. (The official recommendation, I guess, is to put everything in R packages, but I've never actually learned how to do this.)

When I search Gelman in Google, I'm right up there at #2. With Bing, I'm not even on the front page. Heck, I'm not even on the second page! Or the third, or the fourth, or the fifth, or the sixth, . . . OK, enough already! I know, I know, I shouldn't be searching myself anyway, but I had a legitimate reason . . . I had to find my talk on the web today from someone else's computer.

P.S. OK, I take it all back about Bing. I searched my name on Yahoo, and again my homepage did not appear in the first seven pages of search listings. So, really, I shouldn't be blaming Bing, I should be thanking Google for being so nice to me.

Thanks, Google!

David Park sent this along. I haven't really been following basketball statistics lately, but some of you might find it interesting.

Gustaf Granath writes:

I am an ecologist. I have been struggling with a problem for some time now and even asked some statisticians about this. It would be interesting for me (and maybe other people reading your blog) to hear your opinion. So far, I have not received a satisfying answer from anyone.

I am doing a meta-analysis (in ecology with normal dist. data) using two different apporaches. My first approach is a frequentist mixed-model, assuming independence of each sample. The second approach is a hierarchical Bayesian model, modelling the dependence structure in the data set (e.g multiple outcomes from each study). I want to investigate if my covariates are important, and since I have many candidate covariates, I need to do some kind of model selection. My questions is then: is there a model selection tool that can be applied on both approaches??

We've been invited to start a blog at Science Blogs. This seemed like a good idea, a way to reach a new set of readers. At the same time, I didn't want to abandon the Mother Blog right here. Recently we've been overflowing with entries, so we decided to start a new blog at Science Blogs and just link back and forth between this blog and that one. Those of you with RSS can just get both feeds. (The Monkey Cage, 538, and New Majority are less of an issue since I can just crosspost.)

Anyway, we have two things to decide. First, what should the new blog be called; second, what sorts of things should we be posting there. Any suggestions? Thanks in advance for your help.

P.S. Yes, I know it would be logical to just move the entire blog over to the Science Blogs platform. But I just can't bring myself to do that. Science Blogs is a bunch of bloggers, which is fine, but I'd like my own blog to be centered on my research and teaching, which is here.

Lorraine Denby and Colin Mallows write:

It is usual to choose to make the bins in a histogram all have the same width. One could also choose to make them all have the same area. These two options have complementary strengths and weaknesses--the equal-width histogram oversmooths in regions of high density and is poor at identifying sharp peaks; the equal-area histogram oversmooths in regions of low density and so does not identify outliers. We describe a compromise approach which avoids both of these defects. We argue that relying on asymptotics of the Integrated Mean Square Error leads to inappropriate recommendations.

I'm so glad they wrote this article (it appeared recently in the Journal of Computational and Graphical Statistics)! I've thought for a long time that (a) histogram bars are typically too wide (for example, as set by default in software packages such as S and R), and (b) that the underlying problem was that people think of the goal of the histogram as to closely approximate the density function.

A key benefit of a histogram is that, as a plot of raw data, it contains the seeds of its own error assessment. Or, to put it another way, the jaggedness of a slightly undersmoothed histogram performs a useful service by visually indicating sampling variability. That's why, if you look at the histograms in my books and published articles, I just about always use lots of bins. I also almost never like those kernel density estimates that people sometimes use to display one-dimensional distributions. I'd rather see the histogram and know where the data are.

Denby and Mallows go far beyond my vague thoughts by considering histograms with varying widths and coming up with a particular algorithm. I'd like to try out their method on my own problems. Is there R package out there?

Margarita Alegría, Glorisa Canino, Patrick Shrout, Meghan Woo, Naihua Duan, Doryliz Vila, Maria Torres, Chih-nan Chen, and Xiao-Li Meng, write:

Although widely reported among Latino populations, contradictory evidence exists regarding the generalizability of the immigrant paradox, i.e., that foreign nativity protects against psychiatric disorders. The authors examined whether this paradox applies to all Latino groups by comparing estimates of lifetime psychiatric disorders among immigrant Latino subjects, U.S-born Latino subjects, and non-Latino white subjects.

The authors combined and examined data from the National Latino and Asian American Study and the National Comorbidity Survey Replication, two of the largest nationally representative samples of psychiatric information.

In the aggregate, risk of most psychiatric disorders was lower for Latino subjects than for non-Latino white subjects. Consistent with the immigrant paradox, U.S.-born Latino subjects reported higher rates for most psychiatric disorders than Latino immigrants. However, rates varied when data were stratified by nativity and disorder and adjusted for demographic and socioeconomic differences across groups. The immigrant paradox consistently held for Mexican subjects across mood, anxiety, and substance disorders, while it was only evident among Cuban and other Latino subjects for substance disorders. No differences were found in lifetime prevalence rates between migrant and U.S.-born Puerto Rican subjects.

Winston Churchill said that sometimes the truth is so precious, it must be attended by a bodyguard of lies. Similarly, for a model to be believed, it must, except in the simplest of cases, be accompanied by similar models that either give similar results or, if they differ, do so in a way that can be understood.

In statistics, we call these extra models "scaffolding," and an important area of research (I think) is incorporating scaffolding and other tools for confidence-building into statistical practice. So far we've made progress in developing general methods for building confidence in iterative simulations, debugging Bayesian software, and checking model fit.

My idea for formalizing scaffolding is to think of different models, or different versions of a model, as living in a graph, and to consider operations that move along the edges of this graph of models, both as a way to improve fitting efficiency and as a way to better understand models by making informative comparisons. The graph of models connects to some fundamental ideas in statistical computation, including parallel tempering and particle flitering.

P.S. I want to distinguish scaffolding from model selection or model averaging. Model selection and averaging address the problem of uncertainty in model choice. The point of scaffolding is that we would want to compare our results to simpler models, even if we know that our chosen model is correct. Models of even moderate complexity can be extremely difficult to understand on their own.

My colleague Boris Shor has performed some analysis (jointly with Nolan McCarty) on the ideological positions of state legislators. The estimates are based on state legislative voting, which might make you wonder how you could possibly compare legislators in one state with those in another. The trick is that some state representatives (for example, Barack Obama) also end up in Congress. There are enough of these overlap cases that you can put legislators from all 50 states on a common scale.

Boris and Nolan most recently applied their method to compare Deirdre Scozzofava, a state assemblywoman running on the Republican ticket in special election in New York's 23rd congressoinal district. Boris writes:

Keith points me to this article by Gretchen Chapman and Jingjing Liu:

Previous research has demonstrated that Bayesian reasoning performance is improved if uncertainty information is presented as natural frequencies rather than single-event probabilities. A questionnaire study of 342 college students replicated this effect but also found that the performance-boosting benefits of the natural frequency presentation occurred primarily for participants who scored high in numeracy. This finding suggests that even comprehension and manipulation of natural frequencies requires a certain threshold of numeracy abilities, and that the beneficial effects of natural frequency presentation may not be as general as previously believed.

Sounds interesting. Unfortunately the article has no killer graph to make the point. In psychology, the killer graph often takes the form of a plot with two lines that cross, thus demonstrating the interaction of interaction of interest. Maybe Chapman and Liu could do this for their next article.

P.S. I gotta say, it would be pretty cool to be named "Jingjing." Sort of a Boutros Boutros or Mike Michaelson thing going on here.

Bob writes:

I've been meaning to follow up two comments you made in passing about priors:

1. You said you didn't like Dirichlet priors for multinomials because they didn't model covariance. What alternative do you suggest?

2. When I told you I was using the prior from the hierarchical binomial survival example from [page 128 of] your BDA book, you said you didn't like that prior any more. Why and what would you suggest as an alternative?

The book model reparameterized Beta(a,b) in terms of mean a/(a+b) which got a uniform prior, and scale a+b with a Pareto(1.5) prior [p(a+b) proportional to (a+b)**-2.5].

It works fairly well in practice, though it does lead to a fair number of large scale (a+b) samples.

I used your prior for baseball batting average estimation; the post includes the raw data (2006 AL position players) in tsv form, BUGS code, and the R calling harness.

I also use your prior for a hierarchical model of diagnostic test accuracy in epidemiology (or other data coding tasks).

I have longer versions of that paper with more analysis, simulations, data, alternative item-response type models, and pointers to all the code and data.

The basic epidemiology model keeps getting rediscovered. I'm still the only one who's drunk
enough of your Kool-Aid to go the full Bayesian hierarchical model route.

My reply:

Some computational and modeling issues for hierarchical models

How can we fit a complex statistical model and have confidence in our results? There are several challenges, including (a) setting up models that are complicated enough to reflect the aspects of reality that we want to study, (b) regularization or partial pooling to get stable estimates for the resulting large number of parameters, (c) actually fitting the model (in Bayesian terms, getting a point estimate or posterior simulations, (d) checking the fit of the model to data, (e) attaining confidence that the fitting procedure is bug-free, and (f) understanding the fitted model. We discuss these in the context of nonnested varying-intercept, varying-slope multilevel logistic regression models that we have been using to estimate public opinion in demographic and geographic subgroups of the U.S. population.

Mon 26 Oct, 9.30 on the ground floor of the Latarjet building at International Agency for Research on Cancer (IARC), 150 Cours Albert Thomas, Lyon. This is where Martyn Plummer (the JAGS guy) works.

Que penser de ces affirmations insolites parues récemment dans des revues scientifiques sérieuses? Par exemple: «Les ingénieurs ont plus de garçons, les infirmières ont plus de filles », « Les hommes violents ont plus de garçons», ou encore « Les parents séduisants ont plus de filles ». On les doit à Satoshi Kanazawa, un chargé de cours en management et méthodologie de recherche à l'École d'économie de Londres. . . .

[No, we didn't write it this way; it was translated for us. I still like our original title, "Of Beauty, Sex, and Power," but maybe it doesn't make so much sense in other languages.]

The sequel is already assured of box-office success, so now's the time to start thinking about what's gonna be in volume 3. Here are a few models that Levitt and Dubner could consider, in no particular order:

Robin Blackburn in Port-au-Prince:

The conference was opened by the prime minister, Michele Pierre-Louis, who was appointed despite a scurrilous campaign by opposition forces, who argued that appointing a lesbian to such a prominent position was a violation of Haitian manhood. Pierre-Louis had been the director of an NGO known as Fokal (Fondasyon Konesans ak Libete). In choosing her, Preval was thought to have made an adroit move, pleasing the NGO and donor communities: Fokal is supported by George Soros and various Canadian charities.

Carl Bialik reports on a website called the Book of Odds (really, as Carl points out, these are probabilities, not odds, but that's not such a problem because, at least to me, probabilities are much more understandable than odds anyway). It's pretty cool. I could give some examples here but I encourage you to just go to the site yourself and check it out. One thing I really like is that it gives the source of every number: right on the page it gives the country and date of the information, then you can click to get the details. Awesome.

The only thing that bothers me a little bit about the site is that it is almost too professional. When something's that slick, I worry about whether I can trust them.

In contrast, Nate Silver's website is respected but not particularly attractive. And the NameVoyager is just the coolest thing in the world, and, yes, it's professional and it's commercial--that's fine--but it doesn't have the suspicion-inducing hyper-professionalism of the Book of Odds. Seeing the all-so-appealing photo of the bright-eyed oldsters illustrating the "Will you live to be 100?" item that's currently featured on the site's main page, I just think--this is too slick to be trusted. (In case you're wondering, their data say that a randomly-chosen 90-year-old has only a 1-in-9 chance of living to 100. Actually, they say 1 in 8.85, but you know what I think about extra decimal points.)

In some way I prefer the charmingly and unabashedly commercial OK Cupid site to the Book of Odds, which looks so, so commercial but claims only purely altruistic goals. I just don't know what to think.

Anyway, whatever the true story happens to be, it's great stuff. Fun to browse, and a great teaching tool too, I'd think. Enjoy.

Dubner defends himself here. No word on the drunk driving advice, but he has some backstory on the interviews that he and Levitt did regarding global warming. It seems pretty clear that their approach to writing Freakonomics 2 was much different than the original book: the first Freakonomics was all about Levitt's work, whereas the most prominent part of the sequel is a discussion of the ideas of others. As I noted yesterday, this creates a huge selection issue--how did they decide whom to interview?--which is much less present in the first book. I'm also still confused that Dubner describes global warming as "a very difficult problem to solve," given that on his blog the other day he seemed to be endorsing the view that future trends are "virtually assuring us of about 30 years of global cooling."

My guess is that Levitt/Dubner's views on the topic are not completely coherent (by which I mean, not that Levitt and Dubner disagree with each other, but that between them they have a bunch of partly conflicting attitudes on the topic). As a political scientist, I'm the last person to criticize attitudes for being incoherent, and given that neither Levitt nor Dubner is an expert on climate change, it's probably a good thing that their attitudes are fluid and not so easy to pin down. The difficulty comes when they feel the need to defend everything that they've written so far. Again, this is tougher to do here than in the Freakonomics 1 examples, partly because Levitt was much more of an expert on his own research than on others' research, and partly, I suppose, because you'll get a lot more flak in the major news media if you question global warming than if you write about the beneficial consequences of abortion.

P.S. But see the second blurb here!

The above title is a joke. I haven't actually seen the book. As a big-time blogger, I get some books in the mail to review, but maybe this one is sitting in my NYC office. Anyway, the backlash has begun, so maybe this is the right time to buy low and be the first to offer the contrarian claim that, despite what everybody's saying, the book is awesome.

From a short-term economics standpoint, the controversy has gotta be good for the book. So far, Levitt and Dubner have put the words "GLOBAL COOLING" on the cover of their book, they've endorsed a report saying that future trends are "virtually assuring us of about 30 years of global cooling," and that "even if man is warming the planet, it is a small part compared with nature," and they've written that "we believe that rising global temperatures are a man-made phenomenon and that global warming is an important issue to solve." That last bit should do the job of ticking off anybody who was with them so far! (I was actually surprised when reading the comments on that last quote--where Levitt assures us that they do believe in global warming--that pretty much all the global-warming-skeptics among the commenters still seem to think that Levitt is on their side. I guess half a loaf is better than none at all, politically speaking, but I'm surprised that more of them didn't get angry at Levitt for saying that.)

I don't want to get into the substance of climate models, a subject on which I've worked on only a little bit. (The paper we wrote a few years ago never got published--actually, we never finished it enough to submit it anywhere--and our current work on the topic is still in the research-and-writing-up stage.) But I do want to speculate a bit on the political angle.

As I'm sure you know by now, I'm interested in differences between rich and poor. Higher-income people are more likely to vote Republican, and we've seen this in many different subgroups of the population. Among whites, among blacks, among religious attenders, etc., the poorer voters among these subgroups are more Democratic and the richer ones are more Republican.

This got me wondering: What are the subgroups of the population for which this isn't true? Or, more generally, how do rich and poor differ in their voting patterns, in different subgroups of the population.

Here's we found, courtesy of the 2000 and 2004 Annenberg surveys. For each group, we're looking at Republican share of the two-party vote intention among people in the upper third of family income, minus Republican share ... among people in the lower third of family income:

(Click on any of these graphs to see larger versions.)

A striking pattern. The differences between rich and poor are much larger among conservative, Republican groups than among liberal, Democratic groups. At the very bottom of the graph above, you see a few groups where richer people are more likely to vote Democratic. All of these are groups that are mostly liberal and Democratic.

Harker Rhodes writes:

I'm looking for a chapter in someone's textbook titled "Bayesian analysis of case-control studies". Or a chapter with any title covering that subject.....

Here's his longer story:

Just following up . . . this time Dr. McWilliams includes many qualifiers: "I suggested . . . I also suggested . . . Of course, this is only a possibility. I have no numbers to draw on. . . . In any case, it's just a thought."

This helps. As I said before, I have no problem with this sort of op-ed-style reasoning; it just seems out of place on Freakonomics. Anyway, this was part 3 of 3, so I'll have no more to say on the topic.

Kathy Dopp pointed me to this analysis she did regarding the Afghan election. I don't have the time/energy to look into this myself right now but I thought I'd pass this along so that others can comment if they'd like.

Stephen Kershaw writes:

From Eric Loken:

And I expect we'll see some comments here.

Dan Lakeland writes:

Apropos your recent posting of the Churchill/Roosevelt poster, there has been a bit of a controversy over the effect of smoking bans in terms of heart attack rates. Recent bans in the UK have given researchers some plausible "experiments" to study the effect on a larger scale than the famous "Helena Montana" study. For example, this.

On the other hand, when looking for info about this to follow up your poster I found a variety of usually rather obviously biased articles such as this one. But that's no reason to ignore a point of view if it can be backed up by data. The second link at least attempts (poorly) to display some data which suggests that an existing downward trend could be responsible for the reductions, and if poorly done the statistical research could have missed this.

Have you looked at the statistical methodology of any smoking ban studies? it seems like an area ripe for Bayesian modeling, and could be a subject along the lines of the fertility and beauty more girls/more boys research that you recently meta-analyzed.

My reply:

Yes, I imagine that some people have looked into this. I would guess that a smoking ban would reduce smoking and thus save lives, but of course it would be good to see some evidence.

Smoking behavior is a funny thing: It can be really hard to quit, and I've been told that the various anti-smoking programs out there really don't work. It's really hard to make a dent in smoking rates by working with smokers one at a time. On the other hand, rates of smoking vary a huge amount between countries and even between U.S. states:

And smoking bans might work too. Thus, smoking appears to be an individual behavior that is best altered through societal changes.

Aleks points me to this blog by Neuroskeptic, who reports on some recent research studying the placebo effect:

Someone who knows that I hate the so-called Fisher exact test asks:

I was hoping you could point to a Bayesian counterpart or improvement to "Fisher's exact test" - for 2 x 2 categorical, contigency tables with possibly very small numbers (too small to do a chi-square.) I see that you had a blog post on it before (1) but there are several issues i'm unclear about:

(i) What would a full applied Bayesian analysis look like of this type of problem, in general? I have seen one beta-binomial like analysis but never in practical/applied examples. Any practical examples you may have for this, e.g. papers or code examples you've used in teaching, would be great.

(ii) What if we add the twist that the data from the two populations for our 2 x 2 test is paired? e.g. we have several male and several female patients, and the two conditions are drug / no drug. But, each male and female are paired as they are twins (which breaks the independence of the samples obviously.) How is this modeled from a Bayesian perspective?

(iii) Less important: when in practice is it ok to use Fisher's exact test if you're open to Bayesian analysis? 'Never' is a reasonable answer but i'd like to understand practical reasons why you think this. Finally, if all of our data counts are greater then 10, do you think its legitimate to use a chi-square?

My reply:

(i) The basic analysis is pretty simple, it goes like this:

y1 ~ Binomial (n1, p1)

y2 ~ Binomial (n2, p2)

We need a prior distribution on (p1,p2), and we usually assume that n1,n2 provide no information about p1,p2. (This latter point depends on the design of the study, but I'm keeping it simple here.) What's a good prior distribution depends on the problem, but in many cases, a simple uniform distribution on (p1,p2) will be fine. Whatever your prior is, you then throw in the likelihood and you get posterior inference for (p1,p2). Draw 1000 simulations and then use these to get inference for p1-p2. That's it. With moderate or large sample sizes, this is basically equivalent to the standard t-test.

If you have many tables, you can set up a hierarchical model for the p's. We have an example near the end of chapter 5 of Bayesian Data Analysis.

(ii) With paired data, you can fit a logistic regression. Call the data y_ij, where i=1 or 2 and j is the index for the pairing. Then you can model Pr(y_ij=1) = invlogit (a_i + b_j), with a hierarchical model for the b_j's, something like b_j ~ N (mu_b, sigma_b^2), with weakly informative or flat prior distributions on mu_b, sigma_b.

(iii) The only case I could even imagine using the so-called Fisher exact test is if the data were collected so that the row and column margins were both pre-specified. The only example I can think of with this design is Fisher's tea-tasting experiment. In all cases I've seen, at most one margin is preset by design. Also, I'd never do a chi-squared test in this setting. See chapter 2 of ARM for an example where I thought a chi-squared test was OK.

Fernando Hoces De La Guardia writes:

Last night we did the traditional first year econ phd student's skit nite @ Penn.

One particular thing that I noticed was that we had less public that what the upper years told us to be prepared for.

Somebody suggested that it was due to Passover and Good Friday. My immediate reaction was "science & religion don't go usually together". By this I meant a prior of mine that the fraction of religious people is a lot less within a scientific discipline than among the rest of the population.

Two things pop out of my head this morning:

- in which data base can I check that prior?

- if true, are economists more religious than other scientists?

My reply: Usually people look these things up at the General Social Survey, which has a convenient web interface. Good luck!

Richard Morey writes:

I don't know if you are into this sort of thing, but I came across it on the web and thought it was entertaining. Essentially, it is a music video made up of visualizations of quantitative information. It follows a day in a workers life. I suspect some of the data is real. Anyway, it is a creative use of data visualization. I don't know anything about the artist(s).

From David Madigan:

The Observational Medical Outcomes Partnership (OMOP) seeks new statistical and data mining methods for detecting drug safety issues through the OMOP Cup Methods Competition.

Here. That was fun. I never knew what the dude looks like before. Now I know that the has lighter skin and darker hair that I do. Or maybe it's just the lighting. The conversation was fun; I hope to have another chance to do this.

Last year I did a bloggingheads with Will Wilkinson.

Bear with me. I've got a lot of graphs here (made jointly with Daniel Lee). Click on any of them to see the full-size versions.

I'll start with our main result. From the 2004 Annenberg surveys:

Providing health insurance for people who do not already have it--should the federal government spend more on it, the same as now, less, or no money at all?

The maps below show our estimated percentages of people responding "more" (rather than "the same," "less," or "none") to this question:

Increased government spending on health was particularly favored by people under 65 and those in the lower end of the income distribution. Older and higher-income people are much more likely to be in opposition. And, yes, there's some variation by state--you can see a band of states in the middle of the country showing opposition--but age and income explain a lot more.

A few days ago I posted a note about a Freakonomics blog by James McWilliams, who asked, "Do Farmers' Markets Really Strengthen Local Communities?" I was disappointed to see that he offered a historical discussion but no quantitative data or analysis, merely a barrage of subjective impressions and rhetorical questions of the "Who is to say?" sort.

I was hoping for something more in the next installment, but Part Two is unfortunately more of the same. Lots of qualitative quotes but still no data. We get this sort of thing: "Building on this suspicion, she acknowledges that many small farms are indeed more sustainable than larger ones, but then reminds us that "Small scale, 'local' farmers are not inherently better environmental stewards."

"Not inherently better"? That's the best he can do??

Again, if this were an ordinary magazine article or posted in an ordinary blog, it would be fine. Personal impressions make the world go round. But I expect something more when I turn to Freakonomics. Hard-edged data analysis is what makes Freakonomics special. Otherwise it's just the sort of opinionating that anyone can do in their sleep.

Part Three is forthcoming. Maybe we'll see some economic analysis there.

I'd love if someone else were to write my article, tentatively titled "Better than a boxplot," with the following abstract: "We demonstrate graphical options that dominate the boxplot. We hope that, once these alternatives are understood, boxplots are never used again." But I have a horrible feeling I'm going to have to write this article itself.

Awhile ago I was invited by Keying Ye to contribute to a book of essays, Frontier of Statistical Decision Making and Bayesian Analysis, in honor of the great Jim Berger. Here's my chapter, which begins:

Jim Berger has made important contributions in many areas of Bayesian statistics, most notably on the topics of statistical decision theory and prior distributions. It is the latter subject which I shall discuss here. I will focus on they applied work of my collaborators and myself, not out of any claim for its special importance but because these are the examples with which I am most familiar. A discussion of the role of the prior distribution in several applied examples will perhaps be more interesting than the alternative of surveying the gradual progress of Bayesian inference in political science (or any other specific applied field).

I will go through four examples that illustrate different sorts of prior distributions as well as my own progress--in parallel with the rest of the statistical research community--in developing tools for including prior information in statistical analyses . . .

Following up on some links, I came across this:

As a beneficiary of indoor smoking bans, I can't say that I agree with the sentiment, but the poster is pretty clever, and it got me thinking. Imagine Churchill on his regular dose of alcohol but without the moderating influence of the tobacco. Maybe would've been a disaster. Seems like a joke, but maybe we'd all be blogging in German right now. I'd like to think, though, Churchill would've switched to chewing tobacco and all would be ok. A spitoon in the corner is a small price to pay for freedom.

Christopher Rhoads writes:

Interested to know what your comment would be on the following article, which includes the following lines:

If he corrects Marshall's bar graphs he does...

HIMYM on the visualization of data

My total knowledge of all foreign language is a constant. Back when I learned (some) Dutch, I quickly forgot a corresponding amount of French. Then I learned Spanish and forgot more French and almost all my Dutch. Now that I'm relearning French, I'm forgetting my Spanish. At some point I think I was in an unstable equilibrium in which I knew equal amounts of all three.

P.S. Just to calibrate for you: I'm pretty bad. I can't read the newspaper in any language other than English.

P.P.S. No, this doesn't apply to computer languages. The last time I programmed in Fortran (a few years ago), it didn't cause me to forget any R. And I think if I ever learned Python or whatever, it would only help with these other languages. And, no, I don't plan to ever learn C. I've programmed in assembly language already (for the 6502 in my college roommate's Atari), no need to go back to that.

Hamdan Yousuf writes:

I was reading your Kanazawa letter to the editor and I was interested in your discussion of multiple comparisons. This might be an elementary issue but I don't quite understand when the issue of multiple comparisons arises, in general. To give an example from research I have been involved in, assume I am trying to fit a linear regression on a response variable (PR: placebo responsivity score, continuous, experimentally measured) and am assessing 100-200 potential predictors (mostly scores on psychological scales.) The predictors are highly multicollinear such that it is difficult to build a "model" using more than 1 of them, and the matter simplifies to picking the single predictor that optimally explains variance in my response variable. Note that my number of observations (subjects) is small, about 40.

Is this considered a situation with multiple comparisons? That is, I am simultaneously looking at p-values for correlation between my response and each potential predictor. In practice, a handful of the variables yield very good p-values (.001-.005), and these variables make sense scientifically. However, should I be using a correction for MCs, say Bonferonni, with p=.05/200=.0005, in which case nothing is significant. Or am I misinterpreting the idea of multiple comparisons to begin with?

My reply: No, I don't think you should be using classical multiple comparisons methods in your problem. See here and here for further discussion. For your example, maybe it would make sense to combine a bunch of your potential predictors into a single combined scale. I'm guessing that your real question is not, "Are any of these 200 potential predictors correlated with the outcome in the population," but rather "How good are these predictors?" I think you'd be better off with a multilevel model in which you handle the uncertainty using partial pooling.

Lee Sigelman writes that Senator Coburn of Oklahoma is proposing to zero out funding for the National Science Foundation's political science program. It's hard for me to believe this will even come close to happening, and conflict of interest prevents me from saying anything at all trustworthy on the subject--I've had nearly continuous NSF funding for the past 23 years--but I'll tell ya this: I clicked through to Sen. Coburn's list of NSF-funded projects that he'd like to cut, which included:

- $91,601 to conduct a survey to determine why people are for or against American military conflicts.

- $8,992 to study campaign finance reform, with the stated intent of providing "a basis for assessing future proposed changes to campaign finance regulations.

- $958 for a direct mail survey of the residents of Celebration, Florida regarding their feelings of living in privately operated city.

Following my comments on their article on U.S. military funding and conflict in Colombia, Oeindrila Dube and Suresh Naidu wrote:

Thanks for the comments on our paper. It seemed that you viewed the correlations in the anaysis as an interesting descriptive exercise, but not interpretable as causal. We agree with you that the most interesting social science is often causal, and in this case in particular the causal claims are the main results. The paper's punchline is that military aid needs to be reconsidered when there is collusion between the army and non-state armed groups, and we couldn't make this claim if we thought the results were purely descriptive.

In the paper, we do a lot of sample splitting and parametric time controls to rule out the possibility that this is a spurious effect. For example, our results are robust to including a base-specific time trend, along with a base-specific post-2001 dummy.

Possibly the best evidence against a strict "conflict" time-series interpretation is that there is no effect (positive or negative) of US military aid on guerrilla attacks near Colombian military bases. In other words, its not just an increase in conflict on all sides, but an increase in paramilitary attacks in particular.

The "differential time trend" that could drive our effect would have to be a) steeply nonlinear b) only applicable to paramilitaries in base municipalities, and c) would have to be fairly unique to the base municipalities, given the wide variety of alternate control groups we examine. So we think this is not a likely alternative explanation that can account for the effects.

To which I replied:

First off, I still would prefer associational language followed by causal speculation. But I can respect your different choice of emphasis. Now to get to details: my basic alternative model goes as follows: - Conflict in Colombia increased during the early 2000's. - U.S. military aid, in the U.S. and elsewhere, increased during that period also. - Most of the paramilitary attacks (and, thus, most of the increase in paramilitary attacks) occurred near military bases. Thus, I'm not so impressed by the "differential time trend" argument. It's unsurprising (but nonetheless worth noting, as you do) that there are fewer guerilla attacks near military bases. But that doesn't mean that the paramilitary attacks wouldn't have increased in the absence of U.S. aid.

None of the above really contradicts your main political story, which is that the Colombian military is involved in paramilitary attacks, and that U.S. aid is an enabler for this sort of violence.

My story above is consistent with your causal story--more U.S. aid, more resources for the military, more paramilitary attacks. It's also consistent with a different causal story, which goes like this: more conflict, more paramilitary attacks, also more U.S. aid which actually serves to stop the situation from getting worse. The argument is, yes, the U.S. is giving weapons to the bad guys, but by doing so, it co-opts them and restrains their behavior.

OK, I'm not saying this latter argument is true, but I think your strongest argument against it is to say something like: "Sure, it's possible that things would be getting even worse in the absence of U.S. military aid. But given that, during the time that aid was higher, violence was also higher--and we're talking here about violence being done by the allies of the recipients of the aid--well, maybe aid isn't such a good idea." That is, you can put the burden of proof on the advocates of aid. Hey, it costs money and it's going to some unsavory characters. You shouldn't have to prove that aid is hurting; I think it would be more defensible, from a statistical/econometric point of view, to show the association and put the ball in their court.

P.S. Just to be clear: I don't have any strong feeling that you're wrong or any goal of "debunking" your paper. It's interesting and important work and I'm trying to understand it better.

And then they shot back with:

Regarding the stylistic point about associations and causal claims, we think this is perhaps discipline-specific, as the style in economics seems to be to make a causal claim and then rule out all the alternative causal stories as much as possible. I'm sure this is probably one of many idiosyncrasies that irks non-economists.

The substantive question is why paramilitary attacks (and paramilitary attacks specifically, rather than other measures of conflict), increase more in places near bases. The account we put forward is that this occurs because the Colombian military funnels a share of its resources to paramilitary groups. Thus, if US military aid translates into more resources for the military which are shared with paramilitary groups, the implication is that in the absence of increases in US military aid, paramilitary attacks would not have increased by as much as they did.

Now the alternative account you put forward is "more conflict, more paramilitary attacks, also more U.S. aid which actually serves to stop the situation from getting worse. The argument is, yes, the U.S. is giving weapons to the bad guys, but by doing so, it co-opts them and restrains their behavior."

It seems like you have two distinct things in mind, that overall conflict is a source of bias, and an associated conjecture that this omitted variable (overall conflict) upward biases our main coefficient since it is positively correlated with paramilitary attacks and positively correlated with the aid shock. First, we explicitly address and rule out potential omitted variables using a number of empirical specifications. But, even if there is an omitted variable correlated with U.S. military aid that differentially affects paramilitary attacks in base municipalities, it is not clear whether the direction of the bias would be positive. As an example, say a change in Colombian government leads the state to become more effective in fighting the guerilla insurgency, and the US rewards the state with more military aid, while paramilitary activity declines differentially in base regions, as this activity becomes less necessary with greater military effectiveness. In this case, the omitted variable (stronger Colombian state) is negatively correlated with paramilitary attacks and positively correlated with the aid shock, and this would lead us to underestimate the true effect of U.S. aid on paramilitary activity.

Moreover, we think we do a good job ruling "conflict in general" at the national, state, or municipality level as a confounding variable. "Overall conflict" variation at the country level is absorbed by year fixed effects, and conflict at the department level is absorbed by the department x year fixed effects. At the municipal level, it is NOT the case that we observe increases in overall conflict, such as total number of clashes amongst all armed actors at the municipal level. (In out data, attacks are one-sided events carried out by a particular group. The fact that we see paramilitary attacks increase means we are specifically observing increases in events that involve only paramilitary groups - e,g, the paramilitaries attack a village or destroy some type of infrastructure. ) Also, in every specification we find no effect on the guerrilla attacks, and we think you are not taking the non-effect sufficiently seriously in terms of countering the overall conflict account. The guerilla non-effect actually provides very robust evidence that the U.S. military aid is not just correlated with any type of conflict, but rather with attacks by a particular group (which has no regional spillovers).

In addition, our base-specific linear trend and post-2001 dummy specification should convince you that our effect is not merely a post-2001 increase in conflict that manifests particularly as paramilitary attacks in base municipalities.

Your alternative account suggests that more aid to paramilitary organizations could actually result in less violence. While it is challenging to know what the counterfactual would have been in the absence of increased aid, Figure 2 shows that when aid rises sharply in 1999 there is a differential increase in aid in the base regions, and when aid decreases in 2001, there is a corresponding closing of differential decrease in the base regions. This seems inconsistent with the idea that lower aid translates into more paramilitary activity. Also, after 2002, when aid rises again, the differential increases yet another time. It is difficult to explain this pattern with the account you put forward, which would have to require additional coincidental reasons why paramilitary attacks should increase more in base regions precisely in 1999, then decline in 2001, and then rise again in 2002. This is possible, but seems unlikely.

We were thinking of some ideas that would be consistent with your alternative account, of why more aid to paramilitary organizations could actually lower violence. One story here could be deterrence - that stronger paramilitaries deter the guerillas resulting in fewer attacks by guerillas or fewer clashes between guerillas and paramilitaries. But, our results do not show a fall in guerilla attacks or clashes amongst the two groups; rather the coefficient on these other variables is close to 0 and they are statistically insignificant, which is inconsistent with the deterrence account.

Another reason could be dependence, that in the short run U.S. aid increases paramilitary violence, but it also induces paramilitary reliance on the Colombian military for supplies, which increases the sway the government has vis-à-vis this group, potentially leading to future demobilization. Thus in the long-run, U.S. military aid reduces paramilitary violence. While this process could take "long and variable lags" to manifest, it is important to note that we see a dramatic increase in paramilitary activity in 2005, despite a half-decade of huge U.S. military transfers to Colombia. Thus we do not see evidence of this dependence account in our data.

The economy isn't going so well, but there are some interesting possibilities here at Columbia University. One such option that you should be thinking about is the Earth Institute Fellowship, which pays well, includes a research stipend, and puts you in an exciting interdisciplinary community of faculty and postdoctoral researchers. The Earth Institute at Columbia brings in several postdocs each year--it's a two-year gig--and some of them have been statisticians (recently, Kenny Shirley and Leontine Alkema). We're particularly interested in statisticians who have research interests in development and public health. It's fine--not just fine, but ideal--if you are interested in statistical methods also. The EI postdoc can be a place to do interesting work and begin a research career.

If you're a statistician who's interested in this fellowship, feel free to contact me directly--you have to apply to the Earth Institute directly (see link above), but I'm happy to give you advice about whether your goals fit into our program. It's important to me, and to others in the EI, to have statisticians involved in our research.

Deadline for applications is 1 Dec.

Keith Ellis writes:

I've been wondering about are the use of sophisticated mathematical techniques to discover what are the real-world political ideologies, starting without conventional preconceptions.

The core idea I had when this came to many years ago was by way of reading some technical articles about color vision. I was struck by one paper, which I could barely understand, which attempted to determine the "spatial" dimensionality of color vision...I recall vaguely that the conclusion was that it is best described in a 28 or so dimensional space. This connected up, conceptually in my imagination, with what was then the nascent specialty of the stuff involved in the Netflix prize--I can't recall the technical term...preference modeling?

Denis Cote writes:

I am reviewing a paper using logistic regression and I am uncertain about the way they coded their inputs.

They have different ordinal variables coming from self-report questions. For example, self-perceived health" with its answer choice: excellent, very good, good, fair, poor.

Or weight coded as underweight, normal, overweight and obese. They entered the answers as categorical-binary variables (unsure about the precise coding).

Shouldn't they have kept a single ordinal variable? What would be the best practice with ordinal variables?

I think I would not have asked this question if I hadn't read and applied your 2 standard deviations technique!

My reply:

Wow.

Nice graph--especially good that they go back to 1980 (it would be better to go back even earlier but maybe it's not so easy to get the data). One could argue that the numbers would be better per-capita, but the patterns are clear enough that I don't think there's any need to get cute here.

My only criticism of the graph is . . . what's with all the fine detail on the y-axis? 0, 5 million, 10 million, 15 million: that would be enough. What do we gain by seeing 2.5, 7.5, 12.5, 17.5 on the graph? Nuthin. Really, though, this is a very minor comment. It's a great graph.

Helen DeWitt links to this British-style rant (much different from my own story), which reminds me that I never erase the board after my classes either. But the board is usually clean for me. Probably because I usually teach morning classes. Once, though, about 15 years ago, I taught in a classroom right after a biology professor, an elderly lady who used several colors of chalk to construct beautiful pictures on the blackboard every lecture. She was very good about erasing as I was coming in. That semester and the one after that, I remembered to erase the board after class, but I've since slipped to the more natural equilibrium.

One more thing. A few years ago I invited Frank Morgan to give a talk at Columbia on how to teach better. One of the little things he discussed was how to use the blackboard. One of his points was that students aren't always paying attention--and, even if they are, they are often lost in thought trying to make sense of something or another. As a result, they won't be following every step of yours on the blackboard.

To put it another way: to the lecturer, the blackboard is a space-time process in which things get written down, erased, modified, and so on. To many a student, the blackboard is a series of snapshots of a spatial process. And it helps if each of these snapshots makes sense on its own terms. To that end, Frank recommends that you write a header at the top of each section of the blackboard so that students can keep track of what is going on.

Culture wars, voting, and polarization: divisions and unities in modern American politics

I will be discussing various recent work, including material that has appeared in this article and this book.

And here's the video from the last time I gave this lecture.

*Ecole Doctorale *
*salle de réunion du 3ème étage *
*199, boulevard Saint-Germain*

*Lundi 5 Octobre 2009 à 17h30*

Enjoy.

I was in the library the other day and saw a new book, Why are Jews Liberals?, by O.G. neoconservative Norman Podhoretz. This is right up my alley, research-wise, and so I took a look. I don't think Podhoretz's book will match the sales of Thomas Frank's similar record of frustration, "What's the Matter with Kansas?"--there are very few writers out there who can match Frank's skill with the perceptive quip--but this new book has something to offer, if nothing else by presenting the view of an influential battler in the world of political ideas.

Podhoretz's argument (here's a quick summary) goes as follows. Jews in America vote overwhelmingly for Democrats, even though you'd expect from their income levels that Jews would lean Republican. Expanding this, Podhoretz gives three reasons why Jews should vote for Republicans:

I enjoy reading the Freakonomics blog, but as I've noted previously, I remain puzzled by the presence of two appealing but, to my mind, incompatible forms of reasoning that seem to be used more generally in the world of "freakonomics" (which I'm using in lower-case to indicate not just the famous book and blog, but the larger world of empirical microeconomic analyses intended for a popular audience).

Carlisle Rainey writes:

In an earlier blog post, you suggest: "...do a global search-and-replace to change 'DV' to 'outcome' and to change 'OLS' to 'linear regression'." Would you provide a quick explanation why or point me somewhere to find the answer myself?

My reply:

1. I don't like the term "dependent variable" because of confusion with dependence of random variables. To me, "outcome" makes it clearer that you are choosing which variables to use as predictors and which as outcome. "Predictee" would be ok too, I guess.

2. "OLS" focuses on the optimization task; "linear regression" focuses on the model. I think the model is more important that how it's estimated. To put it another way, "OLS" generalizes to weighted least squares, least absolute deviation, etc. "Linear regression" generalizes to logistic regression, nonlinear regression, etc. I find the latter set of generalizations more important and interesting.

Ole Rogeberg writes:

Saw your comments on rational addiction - thought you might like to know that some economists think the "theory" is pretty silly as well. It's worse than you think: They assume people smoke cigarettes, shoot up heroin etc. at increasing rates because they've planned out their future consumption paths and found that to be the optimal way to adjust their "addiction stocks" in the way maximizing discounted, lifetime utility. To quote Becker and Murphy's original article: "[I]n our model, both present and future behavior are part of a consistent, maximizing plan."

Yeah, right

Here's Ole's article, "Taking Absurd Theories Seriously: Economics and the Case of Rational Addiction Theories," which begins:

Rational addiction theories illustrate how absurd choice theories in economics get taken seriously as possibly true explanations and tools for welfare analysis despite being poorly interpreted, empirically unfalsifiable, and based on wildly inaccurate assumptions selectively justified by ad-hoc stories. The lack of transparency introduced by poorly anchored mathematical models, the psychological persuasiveness of stories, and the way the profession neglects relevant issues are suggested as explanations for how what we perhaps should see as displays of technical skill and ingenuity are allowed to blur the lines between science and games.

I agree, and I'd also add that this problem isn't unique to economics. Political science and statistics also have lots of silly models that seem to have a life of their own.

Bill Ricker points me to this blog from Mark Liberman on whether (and how much) managers are more likely to use management jargon. Or, to be more precise, whether knowing that someone uses management jargon in their speech gives you information on how likely they are to be a manager. The motivation was this quote from Peter Taylor:

I [Peter Taylor] argue that the first question to ask is whether hearing someone use the phrase "At the end of the day" conveys information on whether they are likely to be a manager...

Much Bayesian inference follows. My only comment here is not on the Bayesian inference but rather on the idea that "managers" are dweeby Dilbert characters who talk using management jargon. I was thinking about it, and I realized that I'm a manager. I manage projects, hire people, etc. But of course I don't usually think of myself as a "manager" since that's considered a bad thing to be.

For another example, Liberman considers a "spokesperson for a manufacturer of sex toys" as a manager. I don't know what this person does, but I wouldn't usually think of a spokesperson as a manager at all.

To me, the most interesting linguistic phenomenon here is the floating definition of "manager."

P.S. Lots and lots and lots of discussion here. Somehow I think that Mark Liberman gets a lot more readers on his blog than I do on mine!

I just assumed they already were doing this. Did they really used to charge the same price for flights on every day of the year? That would be silly, no? It doesn't make sense to me for people to be angry about differential pricing.

Comments on the linked blog suggest that the problem is a lack of information in the communication of ticket prices. Consumers (such as myself) don't really have any idea what a ticket will cost--we either have to just buy something blind or else do informal statistical inference by running a lot of queries on Expedia or whatever. As a result of this ignorance, airlines have an incentive to advertise super-low fares, which then leads to surcharges etc. What a mess.

On the other hand, I never feel comfortable complaining about airport/airline experiences. I fly a lot and as a result am a big polluter. So, really, anything that makes flying more of a pain in the ass is probably a net benefit to the world.

FInd out on Thurs 1 Oct at 11:15 am in Kimmel 900 at NYU: Dr. Michael Foster from UNC will present the 4th Statistics in Society lecture, entitled: "Does Special Education Actually Work?" This talk will explore the efficacy of current special education policies while highlighting the role of new methods in causal inference in to helping answer it. It is jointly sponsored by the Departments of Teaching and Learning and Applied Psychology, and by the Institute for Human Development and Social Change.

I'd definitely go to this if I were in town.

I'd much much rather have the Washington Post have a competition for America's Next Great Reporters. We have enough Next Great Pundits as it is.

"The cab driver - who witnesses said was talking on his cell phone and appeared distracted - slowed briefly but then tried to speed away . . ." (link from Streetsblog). (And this other source says he "has several traffic violations on his driving record."

Something similar happened to us not long ego (although, luckily, nobody was hurt). This time the cops told the driver (who, again, had to be stopped by people on the street so she couldn't drive off) not to worry about it.

P.S. I'm not saying the drivers should go to prison. The more appropriate sanction would be to get them out of the driver's seat of a car. For the guy who killed the kid and tried to escape, perhaps forbidding him to drive for 20 years would be appropriate. But really it should never have reached that point: if each of the "several traffic violations" had resulted in his car being taken away and him being forbidden to drive for some period of time, then it's likely he wouldn't have been on the road that day and the kid would still be alive.

On the other hand, if hitting a kid and driving away is considered OK, then of course you'll still see people driving that way.

Fivethirtyeight commenter TGGP links to a news article about zillionaire financier Peter Theil, who "predicted which firms would be bailed out based on whether they leaned Republican or Democratic." In the words of reporter Peter Robinson, Theil "possesses a preternatural ability to spot patterns that others miss."

I'll repeat a bunch of Theil's reasoning, because on one level if's interesting while on another level I find it hard to take completely seriously as it stands..

Macartan Humphreys sends along this article where he proves that the requirement of "compactness" in districting, if interpreted as requiring districts to be convex, does not by itself stop a majority party from gerrymandering:

Gerrymandering--the manipulation of electoral boundaries to maximize constituency wins|is often seen as a pathology of democratic systems. A commonly cited cure is to require that electoral constituencies have a `compact' shape. But how much of a constraint does compactness in fact place on would-be gerrymanderers? By applying a theorem of Kaneko, Kano, and Suzuki (2004) to the two party situation we show that a gerrymanderer can always create equal sized convex constituencies that translate a margin of k voters into a margin of at least k constituency wins. Thus even with a small margin a majority party can win all constituencies. Moreover there always exists some population distribution such that all divisions into equal sized convex constituencies translate a margin of k voters into a margin of exactly k constituencies. Thus a convexity constraint can sometimes prevent a gerrymanderer from generating any wins for a minority party.

Chris Blattman reports on a study by Seema Jayachandran and Ilyana Kuziemko that makes the following argument:

Medical research indicates that breastfeeding suppresses post-natal fertility. We [Jayachandran and Kuziemko] model the implications for breastfeeding decisions and test the model's predictions using survey data from India. . . . mothers with no or few sons want to conceive again and thus limit their breastfeeding. . . . Because breastfeeding protects against water- and food-borne disease, our model also makes predictions regarding health outcomes. We find that child-mortality patterns mirror those of breastfeeding with respect to gender and its interactions with birth order and ideal family size. Our results suggest that the gender gap in breastfeeding explains 14 percent of excess female child mortality in India, or about 22,000 "missing girls" each year.

Interesting. I wonder what Monica Das Gupta would say about this study--she seems to be the expert in this area.

Huh?

The only thing that really puzzles me about Jayachandran and Kuziemko's article is that, on one hand, they produce an estimate of 14%, but on the other, they write:

In contrast to conventional explanations, excess female mortality due to differential breastfeeding is largely an unintended consequence of parents' desire to have more sons rather than an explicit decision to allocate fewer resources to daughters.

But they just said their explanation only explains 14%. Doesn't that suggest that the other 86% arises from infanticide and other "explicit decisions"? The difference between "14%" and "largely" is so big that I think I must be missing something here. Perhaps someone can explain? Thanks.

John reports on an article by Oeindrila Dube and Suresh Naidu, who ran some regressions on observational data and wrote:

This paper examines the effect of U.S. military aid on political violence and democracy in Colombia. We take advantage of the fact that U.S. military aid is channeled to Colombian army brigades operating out of military bases, and compare how changes in aid affect outcomes in municipalities with and without bases. Using detailed data on violence perpetuated by illegal armed groups, we …find that U.S. military aid leads to differential increases in attacks by paramilitaries . . .

It's an interesting analysis, but I wish they'd restrained themselves and replaced all their causal language with "is associated with" and the like.

From a statistical point of view, what Dubey and Naiduz are doing is estimating the effects of military aid in two ways: first, by comparing outcomes in years in which the U.S. spends more or less in military aid; second, by comparing outcomes in cities in Colombia with and without military bases.

Gregor Gorjanc writes:

Colin Gillespie writes:

A couple of weeks ago I did your suggested exercise (from Teaching Statistics: A Bag of Tricks) on 'Guessing the age', with the additional twist that the people were actors/actresses out of CSI. As well as discussing the data in class, I used it for there first R lab, where they generated simple scatterplots, boxplots and histograms.

In case your interested, the main results where:

1. Watching CSI didn't seem to affect your guess

2. Females guessed better than males.

3. The vast majority of guesses where too low (unsurprising for actors), except for the youngest actor.

If you are interested you can find my slides/handouts here.

Cool!

My friend Seth wrote:

A few months ago, because of this blog, I got a free heart scan from HeartScan in Walnut Creek. It's a multi-level X-ray of your heart and is scored to indicate your heart disease risk. . . . What's impressive about these scans is three-fold:

1. The derived scores are strongly correlated with risk of heart disease death. . . . Here is an example of the predictive power. . . .

2. You can improve the score. Via lifestyle changes.

3. The scans provided by HeartScan are low enough in radiation that they can be repeated every year, which is crucial if you want to measure improvement. In contrast, a higher-tech type of scan (64 slice) is so high in radiation that it can't be safely repeated. . . .

Heart scans, like the sort of self-experimentation I've done, is a way to wrest control of your health away from the medical establishment. No matter what your doctor says, no matter what anyone says, you can do whatever you want to try to improve your score. . . .

This looked pretty good. Heart attacks are the #1 killer, maybe I should be getting a heart scan. On the other hand, Seth's references are to a journal article from 2000 and a news article from Life Extension magazine, hardly a trustworthy source. So I didn't know what to think.

I contacted another friend who works in medical statistics, who wrote:

I don't know any of this literature but the fact that his source publication dates back to 2000 while the screening method has clearly not gained widespread traction is an indicator that the cost/benefit ratio is not very favorable (though it's no doubt very favorable to HeartScan who make money out of doing the scanning).

I found this more recent (though skimpy) review, "CT-Based Calcium Scoring to Screen for Coronary Artery Disease: Why Aren't We There Yet?" which casts doubt on the whole idea (and given that it's written by radiologists it has some credibility because they would normally be the first to promote a radiology-based screening technique). There were also some links to reviews of the potential dangers (carcinogenic) of repeated CT scans.

From this information, I wouldn't try to talk Seth out of getting heart scans, but I won't rush out to get one of my own.

Tom Holbrook writes:

I just saw your post on the generic ballot and thought you might be interested in something I posted just the other day. My post was stimulated something Charlie cook had written a couple of weeks ago, and I hadn't yet seen the Bafumi, Erikson, and Wlezien article. Anyway, I find that there isn't much connection between generic ballots and midterm results this far (14 months) out from the election. My analysis doesn't break down by in-party and out-party, and it uses data much farther out than Bafumi et al. used, so it not directly comparable to their work; but I thought you might find it interesting.