It was the usual paradox of measurement: I was confusing reliability with validity, as they say in the psychometric literature.
]]> In retrospect, it worked out well for me to learn (even if falsely) that there were 15 or 20 kids my age better than me in math. This made me realize that a career as a "mathematician" (to the extent I understood what this meant, based on my experiences up to the age of 16) was not for me. Given what I know now, I think I would've wanted to be a statistician even if I'd been the #1 kid at the Olympiad. Luckily this didn't happen to me.And now for the most important part
One of the tricks that I didn't know about in the math olympiad training program is to plod along without giving up. Sometimes the direct approach works, solving a problem by eliminating all alternatives. That's a "trick" that's useful in a lot of areas of academic life. For years I've been trying to get this message out to students: If you get stuck right away, don't just stare at your desk and give up. Instead, work actively. This is a point I made in chapter 19 of the ARM book (in particular, the graph on page 416). My theory is that students at top universities have succeeded pretty well by being able to solve problems quickly; they haven't really needed to develop the tools to solve problems systematically by brute force, the way I like to do it.
I think they did try to explain this principle to us at the olympiad program (How to Solve It, and all that), but I didn't ever get the point, partly I think because the problems were so artificial that there only seemed to be a point to solving them if it could be done easily or through some clever trick.
]]>On the Ask Metafilter site, someone asked the following:How does statistical analysis differ when analyzing the entire population rather than a sample? I need to do some statistical analysis on legal cases. I happen to have the entire population rather than a sample. I'm basically interested in the relationship between case outcomes and certain features (e.g., time, the appearance of certain words or phrases in the opinion, the presence or absence of certain issues). Should I do anything different than I would if I were using a sample? For example, is a p-value meaningful in this kind of case?
My reply:
This is a question that comes up a lot. For example, what if you're running a regression on the 50 states. These aren't a sample from a larger number of states; they're the whole population.
To get back to the question at hand, it might be that you're thinking of these cases as a sample from a larger population that includes future cases as well. Or, to put it another way, maybe you're interested in making predictions about future cases, in which case the relevant uncertainty comes from the year-to-year variation. That's what we did when estimating the seats-votes curve: we set up a hierarchical model with year-to-year variation estimated from a separate analysis. (Original model is here, later version is here.)
So, one way of framing the problem is to think of your "entire population" as a sample from a larger population, potentially including future cases. Another frame is to think of there being an underlying probability model. If you're trying to understand the factors that predict case outcomes, then the implicit full model includes unobserved factors (related to the notorious "error term") that contribute to the outcome. If you set up a model including a probability distribution for these unobserved outcomes, standard errors will emerge.
]]>
But I was curious about the interview with Arthur Jensen. I've never met him--he gave a talk at the Berkeley statistics department once when I was there, but for some reason I wasn't able to attend the talk. But I've heard of him. As the interviewers (Daniel Robinson and Howard Wainer) state:
]]> Dr. Jensen has authored over 435 articles, books, and book chapters and is perhaps best known for his controversial 123-page article that appeared in the Harvard Educational Review in 1969. In the article, Dr. Jensen concluded that the differences between Whites and Blacks on IQ tests were attributable to inherent intellectual differences between the two races. In 1980, his Bias in Mental Testing book concluded that intelligence tests were not biased against Blacks, resulting in even more controversy.The interview had some interesting bits. First, something on education and individual differences:
The problems of schooling illustrate the first and second laws of individual differences. I call them laws because they are demonstrated without exception both in the psychological laboratory and in "real life." Unfortunately, they happen to contradict the popular faith in education as the "great leveler." The first law is that individual differences in learning and performance increase as task complexity increases. The second law is that individual differences in performance increase with continuing practice and experience, unless the particular task imposes an artificially low ceiling on proficiency.One notable consequence of these laws is that successful attempts to raise performance by improving methods and amounts of instruction raises the overall mean of the treated group but at the same time widens the distribution of individual differences. The very same effect also applies to group differences. A benefit of raising the overall educational level of the whole population is that it moves a greater proportion of the population above the threshold levels of knowledge and skill required for gainful employment. The downside is the resulting increase in individual and group differences.
Then there are his ruminations on "g":
The educated public today knows of Newton's law of gravitation, Darwin's natural selection, and Einstein's equivalence of mass and energy. They should also know about Spearman's g. Discovered in 1904, g is an essential concept for understanding variation in human abilities. . . . At the top of the factor hierarchy is g, the most general factor. Every cognitive ability that shows individual differences is loaded on the g factor. Tests differ in their g loadings, but their g loadings are not related to any particular knowledge or skills assessed by the various tests. So the possible indicators of g are of unlimited diversity. . . .It is also important to understand what g is not. It is not a mixture or average of a number of diverse tests representing many different abilities. Rather, it is a distillate, representing the single factor that all different manifestations of cognition have in common. In fact, g is not really an ability at all. It does not reflect the tests' contents per se, or any particular kind of performance. It defies description in psychological terms. [italics added] Actually, it reflects some properties of the brain that cause diverse forms of cognitive activity to be positively correlated, not only in psychometric tests but in all of life's mental demands. IQ scores are an attempt to estimate g. But because IQ is just a vehicle for g, it inevitably reflects other broad factors as well, such as verbal, numerical, and spatial abilities, and the specific properties of the particular IQ test. Yet, g is the sine qua non of all IQ tests.
A bit over the top, no? I mean, I'm a political scientist and I think party id and ideology are important, and I even talk about our conceptual model in which each person has a position on a left-right scale and can get shifted by valence issues etc etc--but we know not to take that stuff too seriously!
The most interesting part of the interview, from a historical perspective, was Jensen's discussion of the reaction to his papers. He tells a story in which he was giving a professional lecture in Chicago that was disrupted by 100 protesters who had infiltrated themselves into the crowd--and then he was rescued from the demonstration by a group of 10 police officers who had infiltrated themselves into the infiltrators.
]]>My understanding of median voter theorem is that each election has its own median voter, and that the median voter's influence is limited to the outcome of that election only. I don't understand, then, why the graph in your post is evidence that the median voter has little influence. It seems to me that there are two elections being considered in that graph, with two different median voters. The graph appears to consider "moderation" to be having a moderate voting record in Congress, but it seems to me that the median voter in Congress is likely quite different from the median voter in any particular Congressional district. The power of the median voter in Congress, it seems to me, is to affect the outcome of Congressional votes, not to improve his own chances for re-election, which are determined by his proximity to the median voter in his district. Thus, I'm not sure why we would expect moderation, as measured by the median Congressional voter, to translate into electoral success, which we would expect to be determined by the median district voter.
My reply:
]]> Yes, there are two medians: the median congressmember (or maybe the 60th-most-liberal senator), and the median voter in any congressional district or state.I definitely agree with your point about the median congressmember. As I wrote in the blog entry you cited, "Certainly the median congressmember is important: by definition, it's that marginal vote you need to get a majority. But where do the median congressmember's positions come from?"
What our graph showed was that it's not as important as you might think for a congressmember to be near the median voter in his or her congressional district. This was the point that I was focusing on, because this was the point being made by various pundits: Ben Nelson can't be too liberal because he's representing the people of Nebraska; or, Many Democrats in Congress represent moderate-to-conservative districts, so therefore they can't be too liberal; or, There's no way Olympia Snowe can get away with voting against Obama all the time, given that Maine is a strongly Democratic state; etc. These arguments have some force--Ben Nelson, Olympia Snowe, etc., certainly could lose their seats--but the evidence shows that the benefits from moderation aren't huge.
]]>Is Sanford a cad for bolting his family on Father's Day weekend? Of course, but that is a private, moral failing, rather than a failure of public duty. . . .I [Schaller] oppose most of what Mr. Sanford stands for politically. His showy rejection of federal stimulus money targeted for his state was a crass publicity stunt designed to garner national attention for Mr. Sanford at the expense of his constituents, many of whom are struggling economically. . . . Should Mr. Sanford's ambitions founder on the shoals of a personal scandal, however, yet another opportunity will be lost to establish the long-overdue separation between private comportment and public service. So here's hoping he doesn't resign or, if he does, it is a matter of personal choice rather than him bowing to political pressure.
I see where Schaller is coming from. Lots of people have complicated personal lives, and it's not clear at all that these difficulties have much if anything to do with governing. But I don't know if I agree with him on the wall of separation between private comportment and public service.
Consider the Sanford case. Schaller's a Democrat, so he can evaluate Sanford on his policies. But if Schaller were a Republican, he might very well want Sanford out of there because he tarnishes the brand, makes the party a laughingstock, etc. Also makes it harder for Sanford to convincingly follow a "family values" agenda which Schaller (if he were a Republican) might want. These are legitimate concerns for a Republican to have. Even if you don't think Sanford's personal indiscretions are important, you might want him gone and replaced by a more effective Republican. Just as, from the other direction, a Democrat would've preferred a zipped-fly version of Bill Clinton.
]]> But the first thing I noticed in Schaller's otherwise excellent post were the ugly pie charts. Boy are they ugly. Damn! Some quick points:These aren't hard-and-fast rules. The real point is that it's hard for me to just look at the pie charts and see what's going on. There are too many colors, legends, numbers, etc., floating around. When all is said and done, I guess the charts aren't horrible, but they're the graphical equivalent of meandering, hard-to-follow paragraphs.
]]>
Today I was impressed by a much cleaner and Tuftier variant on the theme by Mike Bostock, called Dependency Tree:
Click on the link, it's interactive. Jeff Heer and Bostock also have a new JavaScript visualization toolkit out ProtoVis, which simplifies the creation of such stuff. The computer scientist in me finds this development very cool. But I still like my correlation matrices.
]]>
For some perspective on where this is coming from: In our office is a map from 1924 titled "Good Roads Everywhere" that shows a proposed system of highways spanning the country, "to be built and forever maintained by the United States Government." The map, made by the National Highways Association, also includes the following explanation for the proposed funding system: "Such a system of National Highways will be paid for out of general taxation. The 9 rich densely populated northeastern States will pay over 50 per cent of the cost. They can afford to, as they will gain the most. Over 40 per cent will be paid for by the great wealthy cities of the Nation. . . . The farming regions of the West, Mississippi Valley, Southwest and South will pay less than 10 per cent of the cost and get 90 per cent of the mileage." Beyond its quaint slogans ("A paved United States in our day") and ideas that time has passed by ("Highway airports"), the map gives a sense of the potential for federal taxing and spending to transfer money between states and regions.
P.S. Yes, I posted this last year, but without the pretty map image (click on it for higher resolution, which unfortunately still isn't quite good enough to make out the text)..
]]>On of the fun parts is this story from his days as an assistant professor:
]]> Soon after I [Wainer] arrived, a parade of students and faculty came to my door asking for help. . . . It wasn't long before every spare minute was used up doing analyses for others. I felt useful, but a bit overwhelmed. About mid-year I was back in Princeton having lunch with Harold [Gulliksen], and when he asked about my research, I grimaced and told him that there was no time. He asked what was taking it all up, and I explained. His advice was sage and practical. He told me that I should remember that my goal was not to help the students get their projects done, but rather help them learn something. He suggested a 4-step solution:1. Ask all who come for a consultation to prepare first a one-paragraph description of their problem and give it to me a day or two in advance, so I might be able to think about it (this alone cut back on the line by 30-50%).
2. Prepare an annotated bibliography.
3. Check off the appropriate reading on the bibliography and give that to the student.
4. Only if I didn't know an appropriate reading should I meet face-to-face with the student.
I [Wainer] followed this advice and found that, once students realized that they would have to do something themselves, the torrent of help-seekers shrank to a trickle.
But the "prepare an annotated bibliography" step seems like a lot of work! How did he find the time to do that?
]]>For example, in mid-September, John McCain notoriously said, "The fundamentals of our economy are still strong." But then in early March, he said that the American people "want to know how we got into this ditch--the worst economic crisis since the great Depression." Based on these two statements, the slide into the ditch apparently occurred sometime between September 16 and March 3.
Similarly, University of Chicago economist Casey Mulligan spent the end of 2008 arguing that the economy is just not that bad, but then changed course in March, writing that "the crash of 2008 did not bother me" but "the crash of 2009 is more worrisome . . . So far productivity has been good in this recession, but 2009's stock market could well see that changing."
It's no surprise that John McCain and Casey Mulligan's views on the economy differ from those of Rahm Emanuel and Paul Krugman, or for that matter Barack Obama, who just last week was beginning to see "glimmers of hope" in the economy. . . .
I don't know about John McCain and Barack Obama, but I recently checked on Casey Mulligan, and I'm pleased to report that he does not seem to have shown a partisan tack in his statements about the economy. For example, some recent posts:
Real Disposable Personal Income per Capita Higher than Ever The BEA reported that real disposable personal income was $2478 per person in May 2009. The only month in U.S. history higher than that was May 2008 ($2499). Based on the recent trends, I [Mulligan] expect that June 2009 (which is almost over) will have the highest real disposable personal income ever.
$592 per person Compared to a world in which real GDP remained at the (thusfar) all time high (achieved in 2008 Q2), the BEA's report this morning shows that through 2009 Q1 the U.S. economy had lost $181 billion (measured at 2008 Q4 prices). $181 billion is equivalent to:- $592 per person, which is equivalent to- 4.6 days of GDP (that is, we are producing like we took 3 weeks of vacation per year, instead of two)
Mulligan clearly has a partisan perspective, but his take on the economy--that things aren't going so badly--has been broadly consistent since the fall of 2008. He does not seem to have changed this view or applied any partisan filters in response to the change in power in Washington.
I know nothing about macroeconomics--even less than I know about the EM algorithm (that's an inside joke; Xiao-Li can explain it to you)--and I am not trying in any way to agree or disagree with Mulligan's analyses (not that my position on this matter would mean anything, anyway). I just wanted to follow up on my earlier offhand remark that had implied that Mulligan had changed course following the change in administration.
]]>My colleague, who's an untentured professor, said that was understandable because she makes less money than I do, so she can better relate to the students' lifestyles. That's a pretty scary thought--it should really go the other way, right? I get paid more so I should be able to afford to be more generous. But maybe she's right; if so, it's a sobering insight.
]]>I was on an NIH panel a couple of years ago with about 25 other scientists, reviewing something like 90 grants. It was pointless. 25 people is just too many to make a decision. What happened was that there were 3 or 4 people who were experienced in the process, who ended up guiding the entire discussion.
The highlight--or, I should say, lowlight--was when we were reviewing a proposal involving the study of the carcinogenic effects of hookah (water pipe) smoking. I asked if this was really such a big deal, and one of the panel members told me that smoking tobacco through a hookah is something like 10 times worse than smoking a cigarette. If so, the public health consequences could be pretty serious, even if not so many people did it. I said this sounded like a reasonable point to me. Then this guy across the table from me spoke up and said that he knew somebody who was 80 years old, had been smoking with a hookah all his life and was none the worse from it. At this point, I blew up. I couldn't believe that the "my elderly aunt smokes and she didn't get cancer" argument could be brought up at an NIH panel!
]]>But the the usual sort of evidence for major problems is a discrepancy between the overall election outcome and what was expected from polls or from extrapolation from other elections. A notorious example is Patrick Buchanan's votes on the "butterfly ballot" in Palm Beach in 2000, which were inconsistent with patterns in other Florida counties in that year.
For the Iran election, the natural step is to compare to previous election returns and look for large changes, as was done by political scientist Walter Mebane. Striking patterns found in such a comparison to not prove fraud but can be useful in giving people a sense of where to focus attention if they want to look further.
Scacco and Beber's analysis is based on the idea that, if there is election fraud, the cheaters are probably acting in a hurry and with various constraints on what numbers they can actually manipulate. As a result, the fake numbers might show some patterns that would be highly unlikely to be seen in tallies of real votes. Again, it is hard for such circumstantial evidence to be entirely convincing on its own, but the patterns they find can support particular theories of how the vote totals came to be.
Another way to calibrate our understanding of such statistical tests is to apply them to a large number of actual elections to see where apparent anomalies appear. Are anomalies happening pretty much at random, as might be expected if one were simply trawling through the data looking for patterns, or do they actually coincide with elections known to be suspicious or fraudulent on other grounds?
Even if statistical tests cannot prove fraud, they can help the news media and observers on the ground to focus their inquiries.
]]>Our mi package in R has several features that allow the user to get inside the imputation process and evaluate the reasonableness of the resulting models and imputations. These features include: flexible choice of predictors, models, and transformations for chained imputation models; binned residual plots for checking the fit of the conditional distributions used for imputation; and plots for comparing the distributions of observed and imputed data in one and two dimensions. In addition, we use Bayesian models and weakly informative prior distributions to construct more stable estimates of imputation models. Our goal is to have a demonstration package that (a) avoids many of the practical problems that arise with existing multivariate imputation programs, and (b) demonstrates state-of-the-art diagnostics that can be applied more generally and can be incorporated into the software of others.
We've made lots of improvements since listing the package last year (here). There's still a lot more work to do, in many different directions (including multilevel models, nonignorable models, the self-cleaning oven, and making the program run faster in sorts of ways), and we keep improving it. But it's good to have something out there.
To actually get the R package, just open your R window, click on Packages, Install packages, and grab mi.
]]>In particular, I [Pinchas] would like to highlight the sentence, six lines into the email, in which Biden says that "[these] stories make the case far better than any statistics ever could." I find it troubling that policymakers who are advocating "evidence based medicine," trump anecdotal evidence over that which is anchored in statistics.
Here's the email:
