Results matching “mister p”

Thanks for all the suggested titles. My current favorite remains, "If You Don't Buy This Magazine, We'll Kill This Blog." Although, I have to admit, "Super-Duper-Freakanomics" [sic] wasn't bad either. And, as much as I like the idea of calling it "Mister P," I can't quite pull the trigger on that one.

To respond to some of your comments:

1. No, I can't just post the general-interest entries at the new blog. That would take a lot of the fun out of the current blog. And the Science Blog people don't want me to cross-post more than 4 items per month. I will, of course, link to the new items from the current blog, but it's not as good if I can't cross-post them.

2. I agree that Science Blogs isn't the same as what I'm doing here, that's why I just wanted to post some stuff there, to reach the different audience, without losing what we have here.

3. I don't plan to be doing anything extra with this new blog; I see it more as a place to post a few things that I was going to post somewhere anyway.

4. Someone commented that it's strange for me to ask for a title before deciding on a topic. I thought it was implicit that, by asking for a title, I'm also asking for suggestions on a topic. I guess I'll try two or three posts a week and see how it goes.

Finally, in all seriousness, if nobody comes up with a better title, I'm going to call it "Applied Statistics." And I'll kick it off with a few posts about literature. Consider yourselves warned.

The National Election Study is hugely important in political science, but, as with just about all surveys, it has problems of coverage and nonresponse. Hence, some adjustment is needed to generalize from sample to population.

Matthew DeBell and Jon Krosnick wrote this report summarizing some of the choices that have to be made when considering adjustments for future editions of the survey. The report was put together in consultation with several statisticians and political scientists: Doug Rivers, Martin Frankel, Colm O'Muircheartaigh, Charles Franklin, and me. Survey weighting isn't easy, and this sort of report is just about impossible to write--you can't help leaving things out. They did a good job, though, and it's great to have this stuff put down in an official way, so that people can work off it of it when going forward.

It's a lot harder to write a procedure for general use than to do a single analysis oneself.

Some corrections

I have a few corrections to add to the report that unfortunately didn't make it into the final version (no doubt because of space limitations):

A political scientist writes:

Here's a question that occurred to me that others may also have. I imagine "Mister P" will become a popular technique to circumvent sample size limitations and create state-level data for various public opinion variables. Just wondering: are there any reasons why one wouldn't want to use such estimates as a state-level outcome variable? In particular, does the dependence between observations caused by borrowing strength in the multilevel model violate the independence assumptions of standard statistical models? Lax and Phillips use "Mister P" state-level estimates as a predictor, but I'm not sure if someone has used them as an outcome or whether it would be appropriate to do so

First off, I love that the email to me was headed, "mister p question." And I know Jeff will appreciate that too. We had many discussions about what to call the method.

To get back to the question at hand: yes, I think it should be ok to use estimates from Mister P as predictor or outcome variables in a subsequent analysis. In either case, it could be viewed as an approximation to a full model that incorporates your regression of interest, along with the Mr. P adjustments.

I imagine, though, that there are settings where you could get the wrong answer by using the Mr. P estimates as predictors or as outcomes. One way I could imagine things going wrong is through varying sample sizes. Estimates will get pooled more in the states with fewer respondents, and I could see this causing a problem. For a simple example, imagine a setting with a weak signal, lots of noise, and no state-level predictors. Then you'd "discover" that small states are all near the average, and large states are more variable.

Another way a problem could arise, perhaps, is if you have a state-level predictor that is not statistically significant but still induces a correlation. With the partial pooling, you'll see a stronger relation with the predictor in the Mr. P estimates than in the raw data, and if you pipe this through to a regression analysis, I could imagine you could see statistical significance when it's not really there.

I think there's an article to be written on this.

Fancy statistical analysis can indeed lead to better understanding.

Jeff Lax and Justin Phillips used the method of multilevel regression and poststratification ("Mister P"; see here and here) to estimate attitudes toward gay rights in the states. They put together a dataset using national opinion polls from 1994 through 2009 and analyzed several different opinion questions on gay rights.

Policy on gay rights in the U.S. is mostly set at the state level, and Lax and Phillips's main substantive finding is that state policies are strongly responsive to public opinion. However, in some areas, policies are lagging behind opinion somewhat.

A fascinating trend

Here I'll focus on the coolest thing Lax and Phillips found, which is a graph of state-by-state trends in public support for gay marriage. In the past fifteen years, gay marriage has increased in popularity in all fifty states. No news there, but what was a surprise to me is where the largest changes have occurred. The popularity of gay marriage has increased fastest in the states where gay rights were already relatively popular in the 1990s.

In 1995, support for gay marriage exceeded 30% in only six states: New York, Rhode Island, Connecticut, Massachusetts, California, and Vermont. In these states, support for gay marriage has increased by an average of almost 20 percentage points. In contrast, support has increased by less than 10 percentage points in the six states that in 1995 were most anti-gay-marriage--Utah, Oklahoma, Alabama, Mississippi, Arkansas, and Idaho.

Here's the picture showing all 50 states:

I was stunned when I saw this picture. I generally expect to see uniform swing, or maybe even some "regression to the mean," with the lowest values increasing the most and the highest values declining, relative to the average. But that's not what's happening at all. What's going on?

Some possible explanations:

- A "tipping point": As gay rights become more accepted in a state, more gay people come out of the closet. And once straight people realize how many of their friends and relatives are gay, they're more likely to be supportive of gay rights. Recall that the average American knows something like 700 people. So if 5% of your friends and acquaintances are gay, that's 35 people you know--if they come out and let you know they're gay. Even accounting for variation in social networks--some people know 100 gay people, others may only know 10--there's the real potential for increased awareness leading to increased acceptance.

Conversely, in states where gay rights are highly unpopular, gay people will be slower to reveal themselves, and thus the knowing-and-accepting process will go slower.

- The role of politics: As gay rights become more popular in "blue states" such as New York, Massachusetts, California, etc., it becomes more in the interest of liberal politicians to push the issue (consider Governor David Paterson's recent efforts in New York). Conversely, in states where gay marriage is highly unpopular, it's in the interest of social conservatives to bring the issue to the forefront of public discussion. So the general public is likely to get the liberal spin on gay rights in liberal states and the conservative spin in conservative states. Perhaps this could help explain the divergence.

Where do we go next in studying this?

- We can look at other issues, not just on gay rights, to see where this sort of divergence occurs, and where we see the more expected uniform swing or regression-to-the-mean patterns.

- For the gay rights questions, we can break up the analysis by demographic factors--in particular, religion and age--to see where opinions are changing the fastest.

- To study the "tipping point" model, we could look at survey data on "Do you know any gay people?" and "How many gay people do you know?" over time and by state.

- To study the role of politics, we could gather data on the involvement of state politicians and political groups on gay issues.

I'm sure there are lots of other good ideas we haven't thought of.

P.S. More here.

Stephen Senn quips: "A theoretical statistician knows all about measure theory but has never seen a measurement whereas the actual use of measure theory by the applied statistician is a set of measure zero."

Which reminds me of Lucien Le Cam's reply when I asked him once whether he could think of any examples where the distinction between the strong law of large numbers (convergence with probability 1) and the weak law (convergence in probability) made any difference. Le Cam replied, No, he did not know of any examples. Le Cam was the theoretical statistician's theoretical statistician, so there's your answer.

The other comment of Le Cam's that I remember was his comment when I showed him my draft of Bayesian Data Analysis. I told him I thought that chapter 5 (on hierarchical models) might especially interest him. A few days later I asked him if he'd taken a look, and he said, yes, this stuff wasn't new, he'd done hierarchical models back when he'd been an applied Bayesian back in the 1940s.

A related incident occurred when I gave a talk at Berkeley in the early 90s in which I described our hierarchical modeling of votes. One of my senior colleagues--a very nice guy--remarked that what I was doing was not particularly new; he and his colleagues had done similar things for one of the TV networks at the time of the 1960 election.

At the time, these comments irritated me. But, from the perspective of time, I now think that they were probably right. Our work in chapter 5 of Bayesian Data Analysis is--to put it in its best light--a formalization or normalization of methods that people had done in various particular examples and mathematical frameworks. (Here I'm using "normalization" not in the mathematical sense of multiplying a function by a constant so that it sums to 1, but in the sociological sense of making something more normal.) Or, to put it another way, we "chunked" hierarchical models, so that future researchers (including ourselves) could apply them at will, allowing us to focus on the applied aspects of our problems rather than on the mathematics.

To put it another way: why did Le Cam's hierarchical Bayesian work in the 1940s and my other colleague's work in 1960s not lead to more widespread use of these methods? Because these methods were not yet normalized--there was not a clear separation between the math, the philosophy, and the applications.

To focus on a more specific example, consider the method of multilevel regression and poststratification ("Mister P"), which Tom Little and I wrote about in 1997, then David Park, Joe Bafumi and I picked back up in 2004, and then finally took off with the series of articles by Jeff Lax and Justin Phillips (see here and here). This is a lag of over 10 years, but really it's more than that: when Tom and I sent our article to the journal Survey Methodology back in 2006, the reviews said basically that our article was a good exposition of a well-known method. Well-known, but it took many many steps before it became normalized.

These are all important methods and concepts related to statistics that are not as well known as they should be. I hope that by giving them names, we will make the ideas more accessible to people:

Mister P: Multilevel regression and poststratification.

The Secret Weapon: Fitting a statistical model repeatedly on several different datasets and then displaying all these estimates together.

The Superplot: Line plot of estimates in an interaction, with circles showing group sizes and a line showing the regression of the aggregate averages.

The Folk Theorem: When you have computational problems, often there's a problem with your model.

The Pinch-Hitter Syndrome: People whose job it is to do just one thing are not always so good at that one thing.

Weakly Informative Priors: What you should be doing when you think you want to use noninformative priors.

P-values and U-values: They're different.

Conservatism: In statistics, the desire to use methods that have been used before.

WWJD: What I think of when I'm stuck on an applied statistics problem.

Theoretical and Applied Statisticians, how to tell them apart: A theoretical statistician calls the data x, an applied statistician says y.

The Fallacy of the One-Sided Bet: Pascal's wager, lottery tickets, and the rest.

Alabama First: Howard Wainer's term for the common error of plotting in alphabetical order rather than based on some more informative variable.

The USA Today Fallacy: Counting all states (or countries) equally, forgetting that many more people live in larger jurisdictions, and so you're ignoring millions and millions of Californians if you give their state the same space you give Montana and Delaware.

Second-Order Availability Bias: Generalizing from correlations you see in your personal experience to correlations in the population.

The "All Else Equal" Fallacy: Assuming that everything else is held constant, even when it's not gonna be.

The Self-Cleaning Oven: A good package should contain the means of its own testing.

The Taxonomy of Confusion: What to do when you're stuck.

The Blessing of Dimensionality: It's good to have more data, even if you label this additional information as "dimensions" rather than "data points."

Scaffolding: Understanding your model by comparing it to related models.

Ockhamite Tendencies: The irritating habit of trying to get other people to use oversimplified models.

Bayesian: A statistician who uses Bayesian inference for all problems even when it is inappropriate. I am a Bayesian statistician myself.

Multiple Comparisons: Generally not an issue if you're doing things right but can be a big problem if you sloppily model hierarchical structures non-hierarchically.

I know there are a bunch I'm forgetting; can youall refresh my memory, please? Thanks.

P.S. No, I don't think I can ever match Stephen Senn in the definitions game.

My quick take on the Souter replacement is that, with 59 Democratic senators and high popularity, Obama could nominate Pee Wee Herman to the Supreme Court and get him confirmed. But I'm no expert on this. The experts are my colleagues down the hall, John Kastellec, Jeff Lax, and Justin Phillips, who wrote this article on public opinion and senate confirmation of Supreme Court nominees. They find:

Greater public support strongly increases the probability that a senator will vote to approve a nominee, even after controlling for standard predictors of roll call voting. We also find that the impact of opinion varies with context: it has a greater effect on opposition party senators, on ideologically opposed senators, and for generally weak nominees.

There's a lot of good stuff here (from Thomas Lumley). It's all classical stuff--no small-area estimation, no Mister P, etc.--but the classical stuff is still pretty useful. The "survey" package for R looks pretty good; in particular, it allows you to specify the survey design, which is a big step beyond simply specifying survey weights.

I'd also like to recommend Sharon Lohr's book from 1999. When's the second edition coming out?

The other day I was reading a story in the New Yorker that had what I consider the now-standard pattern of starting the reader with no information about the key characters so that it takes awhile to figure out who the narrator is and how he relates to the scene. (After a few pages I got the sense that he was a well-off doctor in his fifties or sixties on a vacation with his wife and some friends.)

Anyway, here's my beef. I've always found this sort of style annoying, in comparison to the more traditional opening ("Once upon a time there was a well-off doctor in his sixties named James. One day he went on a vacation with his wife and some friends . . ."). At the same time, I've been conditioned to think that the "New Yorker"-style opening is better, more true to life--after all, in real life, people aren't generally introduced to you with a "Once upon a time"!

But then I was thinking that maybe this New Yorker style isn't so natural. These stories are generally told from one character's perspective--and, from that perspective, you would actually know someone's name, age, etc. It's not so natural at all to have to spend the first part of a story figuring out who's talking to whom.

My new take on this is that this style is a cheat, a way of creating a feeling of mystery and suspense without doing the work to create actual mystery and suspense. Actual mystery is when there's a situation you should be able to understand, but you don't, there are some missing pieces that you're trying to figure out. Actual suspense is when you want to know what happens next. Fake mystery and suspense is when you're just confused and don't know what's happening.

For example, the movie North by Northwest is actually mysterious and suspenseful. But not because it's a cheat and everyone's in a fog and you don't know who's who; it's because you're in the position of a character who knows who he is, but he doesn't know what's going on around him. That's a little different, in my opinion. Similarly with, say, John Le Carre: there's lots of things that, as a reader, you don't understand, but you're clear right away on who's saying what.

Or, for that matter, Mister New Yorker, John Updike, who begins a story with, "The Maples had moved just the day before to West Thirteenth Street, and that evening they had Rebecca Cune over, because now they were so close." Lots of hidden meaning there, but none of this artificial confusion where you're basically thrown into someone's brain at a random moment and not given any background. Following John Updike (or, for that matter, John O'Hara), I think the real challenge is giving the right amount of background--not too much, and not too little. Zero is not usually a serious option, in my opinion.

But, if you're writing a story that really has no mystery and no suspense, then starting by giving the reader no information can be a good way to give the illusion of depth.

P.S. Just to be clear, I'm not complaining about the "start in the middle" approach where the story begins and then you use flashbacks or other revelations to give a sense of how things all got started. That makes a lot of sense to me. What I'm bothered by is the particular trick of not identifying anything explicitly about the main characters so that the first part of the story involves the reader having to figure out the basics.

P.P.S. Sorry for ranting again. Yes, I know, I know, nobody's forcing me to read this story. But these questions of style interest me.

P.P.P.S. These issues also arise when writing statistics books.