September 2005 Archives

Brett Pelham (whose research on names and life choices is discussed here and here--based on the work of Pelham and his collaborators, we crudely estimate that about 1% of people in the U.S. choose a career based on their first name) wrote a quick email in response.

Jouni pointed me to this page, at Harvard's Bok Center for Teaching and Learning, on teaching by having students work in groups. As Jouni says, and I agree,

It seems to me that from the "liberal" (in the U.S. politics) perspective, man [humans] used to be the "rational animal" but is now the "irrational computer," and this worries me a bit.

The rational animal

For an example of the first view, here's a quote I just googled::

"We believed . . . that man was a rational animal, endowed by nature with rights, and with an innate sense of justice; and that he could be restrained from wrong and protected in right, by moderate powers, confided to persons of his own choice, and held to their duties by dependence on his own will." -- Thomas Jefferson, 1823

The idea being that our rationality is what separates us from the beasts, either individually (as in the Jefferson quote) or through collective action, as in Locke and Hobbes. If the comparison point is animals, then our rationality is a real plus!

The irrational computer

Nowadays, though, it seems almost the opposite, that people are viewed as irrational computers. To put it another way, if the comparison point is a computer, then what makes us special is not our rationality but our emotions.

I was thinking about this when reading in n+1 magazine the review by Megan Falvey of the book "Freakanomics."

Our description of the rational self supports the real-world conditions under which some futures seem more attainable than others. It coaxes us into wholehearted, personally felt participation with capitalist regulation. Levitt’s calculating individual is the ideal subject of contemporary neoliberal economic reform, in particular the expansion of the market into all possible areas of life.

The idea seems to be that "the description of the rational self" excludes warmer aspects of human nature. That I'll definitely believe. But I still think rationality is a good thing--perhaps my bias as a scientist.

Decoupling rationality and selfishness

Rationality can serve other-directed as well as selfish goals. Yes, I can rationally try to get the best deal on a new TV, but the Red Cross can also use rationality (for example, in the form of mathematical optimization) to deliver help to as many people as possible. Or Novartis can use rationality (in the form of up-to-date biostatistical methods) to increase the chance of developing an effective drug--this can serve both selfish and unselfish purposes.

The decoupling of rationality and selfishness is a point we made here, in the context of considering voting as a rational way to attempt to improve the well-being of others as well as oneself.

To get back to Falveys' book review: I'm not attempting to address the details of her disagreements with Levitt and Dubner, just to express my distress that she sees rationality to be a problem. Considering the alternatives, I think rationality is pretty good. But it is useful to think about the goals to which the rationality is directed.

From Chance News, submitted to Chance News by Bill Peterson, based on a posting from Joy Jordan to the Isolated Statisticians e-mail list:

Exploiting the gender gap New York Times, 5 September, 2005, A21 Warren Farrell

Farrell is the author of Why Men Earn More: The Startling Truth Behind the Pay Gap -- and What Women Can Do About It (AMACOM, 2004)

This article was published for Labor Day, and it opens by citing a demoralizing, often-heard statistic: women still earn only 76 cents for each dollar paid to their male counterparts in the workplace. Farrell maintains that such comparisons ignore important lurking variables. He claims to have identified twenty-five tradeoffs involving job vs. lifestyle choices, all of which men tend to resolve in favor of higher pay, while women tend to seek better quality of life.

Here are some the factors discussed in the article. Men more readily accept jobs with longer hours, and Farrell reports that people who work 44 hours per work earn twice as much as people who work 34 hours per week. Similarly, he finds that men are more willing to relocate or travel, to work in higher risk environments, and to enter technical fields where jobs may involve less personal interaction. Each of these choices is associated with higher pay.

Even head-to-head comparisons of men and women working in the “same job” can be tricky. Farrell observes, for example, that Bureau of Labor Statistics data consider all medical doctors together. But men opt more often for surgery or other higher paid specialties, while women more often choose general practice.

As indicated by the subtitle of his book, however, Farrell intends to provide some positive news for women. He claims that in settings where women and men match on his 25 variables, the women actually earn more than men. He also identifies a number of specific fields where women do better. One of these is statistics(!), where he reports that women enjoy a 35 percent advantage in earnings.

I haven't read the book so can't comment on the analysis, but it seems like a great discusison topic for class.

Jim Hodges, Yue Cui, Daniel Sargent, and Brad Carlin completed their paper on "smoothed Anova". The abstract begins: "We present an approach to smoothing balanced, single-term analysis of variance (ANOVA) that emphasizes smoothing interactions, the premise being that for a dependent variable on the right scale, interactions are often absent or small. . . ."

The topic is hugely important, I believe (see also here): especially in observational studies, regression models work because they can handle multiple inputs (for example, see Michael Stastny's quick discussion here). Once we have multiple inputs, you gotta look at interactions, which quickly leads to a combinatorical explosion, the usual "solution" to which is to ignore high-level interactions. But in some problems--including many decision analyses and many research studies in psychology--interactions are what we really care about. (Here's an example from our own work where we would have liked to include more interactions than we actually did.)

Anyway, I think this is one of the major unsolved problems in statistics. It can be attacked in several ways, including regression/Anova (that's where Hodges et al. and I are working), neural nets, nonparametric models, etc etc. My best published method so far of handling high-level interactions isn't so great, and I think that Hodges et al. are doing interesting stuff.

I hope lots of people read the article, try out the methods presented there, and take the ideas even further.

I don't really want to go on and on about this, but since it's a current research topic of ours . . .

I'm trying to integrate class-participation activities into the Applied Regression and Multilevel Modeling course I'm teaching this semester. We have a whole bunch of these activities for introductory statistics (in my intro class I have at least one demo and one other activity per lecture) but I've never before tried to consistently use them for a more advanced class.

I'll tell you how things have been going so far and then update occasionally.

The first 2 weeks: what they need to learn

The first several weeks of the class are a review of classical (non-multilevel) regression, with a focus on understanding the model, particularly the deterministic part (that is, y=a+bx, with less of a focus on the distirbution of epsilon). This is also a time for the students to get familiar with R, which they'll have to use more of when working with more complicated models--especially when trying to use inferences beyond simply looking at parameter estimates and standard errors. The first two homework assignments involve fitting simple regressions in R, graphing the data and the fitted regression lines, and building a multiple regression to fit Hamermesh's beauty and teaching evaluations data. Jouni, as T.A., has to spend a lot of time helping students get started with R. The main mathematical difficulties are learning and understanding linear and logarithmic transformations.

The first 2 weeks: in the classroom

Lecture 1 starts with some motivating examples, including roaches, rodents, and red/blue states. I stop and give the students a few minutes to work in pairs to come up with explanations for the patterns of income and voting within and between states. I describe the roach study and the rodent study and then give the students a minute to discuss in pairs to see if they can figure out the key difference between the two studies. (The difference is that the roach study has the goal of estimating a treatment effect--integrated pest management compared to usual practice--and the rodent study is descriptive--to understand the differences between rodent levels in apartments occupied by whites, blacks, hispanics, and others. We'll get back to causal inference in a few weeks.) I yammer on a bit about the skills they'll learn by the time the course is over, and how I expect them to teach themselves these skills. Analogies between statistics and child care, sports, and policy analysis. Cautionary examples of Dan Marino and Cal Ripken. The beauty and teaching evaluations example. I give the equation of the regression line, the students have to work in pairs to draw it. Use the computer to fit some regressions in R and plot the data and fitted regression lines. (No residual plot for now, no q-q plot: we're focusing on the important things first.)

Lecture 2 starts with the cancer-rate example. I hand out Figure 2.7 from BDA and give the students a few minutes to work in pairs to come up with explanations for why the 10% of counties with highest kidney-cancer deaths are mostly in the middle of the country. I write various explanations on the blackboard and then hand out Figure 2.8. We discuss: this is a motivator for multilevel models. I was going to bring up the example of the test with 1 or 100 questions but forgot to mention it--maybe I'll do it in class in a few weeks. I then give them the regression of earnings (in 1993) on height (in inches): y = -61000 + 1300*height + error, with residual sd of 19000. In pairs, they must draw the line and hypothetical data that would lead to this estimated regression. This is a toughie--the students have to realize that heights are mostly between 60 and 75 inches, and that the data must be skewed to all fit above the y=0 line. We talk transformations for a bit--some more activities in pairs (for example, what's the equation of the regression line if we first normalize x by subtracting its mean and dividing by its sd). Discussion of appropriate scale of the measurements and how much to round off. Comparisons of men to women: adding sex into the regression model. In pairs: what's the difference in earnings between the avg man and the avg woman (it's not the coef for sex, since the two sexes differ in height). Why it's better to create a variable called "male" than one called "sex."

Lecture 3 starts with answering questions. What are outliers and should we care about them? (My answer: outliers are overrated as a topic of interest.) Why is it helpful to standardize input variables before including interactions? Long discussion using the earnings, height, and sex example. Standardize earnings by subtracting mean and dividing by 2*sd. Standardize sex by recoding as male=1/2, female=-1/2. Lots of working in pairs drawing regression lines and figuring out regression slopes. Understanding coefficients of main effects and interactions. Categorized predictors, for example modeling age as continuous, with quadratic term, using discrete categories. Start talking about the logarithm. The amoebas example--at time 1, there is 1 amoeba; at time 2, 2 amoebas, at time 3, 4 amoebas; etc. In pairs: give the equation of #amoebas as a function of time. Then give the linear relation on the log scale. (I should have had this example starting at time 0. Having to subtract time=1 is a distraction that the students didn't need.) Graph of world population vs. time since year 1, graph on log scale. Interpreting exponential growth as a certain percentage per year, per 100 years (in pairs again).

Lecture 4: all about logarithms. On the blackboard I give the equation for a cube's volume V as a function of its length L. Then also log V = 3 log L. Then, in pairs, they have to figure out the corresponding formulas for surface area S as a function of volume. It's not so easy for students who haven't used the log in awhile. Then we discuss the example of metabolic rate and body mass of animals. We then go to interpreting log regression models. Log earnings vs. height. Log earnings vs. log height. Interpreting log-regression coefficients as multiplicative factors (if the coef is 0.20, then a 1-unit difference in x corresponds to an approximate 20% difference in y). Interpreting log-log coefficients as elasticities (if the coef is 0.6, then a 1% increase in x corresponds to an approximate 0.6% increase in y). All these are special cases of transformations. Also discuss indicator variables, combinations of inputs, and model building. How to interpret statistical significance of regression coefficients. We did some more activities in pairs but I can't quite remember what they were.

How do I have time to cover the material?

People have often told me that they'd like to do group activities but they can't spare the class time. I disagree with that line of thinking. My impression is that students learn by practicing. A lecture can be good because it gives students a template for their own analyses, or because it motivates students to learn the material (for example, by demonstrating intersting applications or counterintuitive results), or by giving students tips on how to navigate the material (e.g., telling them what sections in the book are important and what they can skip, helping them prepare for homework and exams, etc.). The lecture room also can be a great way to answer questions, since when one student has a question, others often have similar questions, and the feedback is helpful as the class continues.

But I don't see the gain in "covering" material. I don't need to do everything in lecture. It's in the book, and they're only going to learn it if its in the homeworks and exams anyway. The class-participation activities allow the students to confront their problem-solving difficulties in an open setting, where I can give them immediate feedback and help them develop their skills. And having them work in pairs keeps all of them (well, most of them) focused during my 9-10:30am class.

Summary (so far)

This has been pretty exciting so far. We'll see how it works for the whole semester. At this point, I don't even think I'm capable of doing straight lectures, so it's good that the activities are working. But maybe . . . maybe . . . this could transform the teaching of statistics! It's a hope (or distant goal).

Here's the revised version of our paper on why and how it's rational to vote, and here's the abstract:

.For voters with "social" preferences, the expected utility of voting is approximately independent of the size of the electorate, suggesting that rational voter turnouts can be substantial even in large elections. Less important elections are predicted to have lower turnout, but a feedback mechanism keeps turnout at a reasonable level under a wide range of conditions. The main contributions of this paper are: (1) to show how, for an individual with both selfish and social preferences, the social preferences will dominate and make it rational for a typical person to vote even in large elections; (2) to show that rational socially-motivated voting has a feedback mechanism that stabilizes turnout at reasonable levels (e.g., 50% of the electorate); (3) to link the rational social-utility model of voter turnout with survey findings on socially-motivated vote choice.

What's cool about the social-benefit model is it not only explains why it is rational to vote (and to participate in politics in other ways, such as by making small contributions to political campaigns) but also makes it clear that, to the extent it is rational to vote, it is rational for the choice of whom to vote for to depend on social rather than selfish preferences.

For more on this and related topics, see these earlier blog entries here, here, and here. And some stuff here on voting and social networks.

My entry on the boxer and the wrestler sparked some interesting discussion here and here. In order to understand the distinction between randomness and uncertainty in a probability distribution, one has to embed that probabability into a larger structure with potentially more information. As Aki Vehtari pointed out, Tony O'Hagan made this point in a nice article published last year.

Dempster-Shafer not a solution. Neither is robust Bayes

Anyway, one of the other comments on my post alluded to belief functions (Dempster and Shafer's theory of upper and lower probabilities) as a solution to the boxer/wrestler paradox. Actually, though, the boxer/wrestler thing was part of something that Augustine Kong and I came up with 15 years ago as a counterexample, or paradox, for Bayesian inference, robust Bayes, and also belief functions. For this particular example, the answer given by belief functions doesn't make much sense.

Here's my recent paper on the topic, and here's the abstract to the paper:

Bayesian inference requires all unknowns to be represented by probability distributions, which awkwardly implies that the probability of an event for which we are completely ignorant (e.g., that the world's greatest boxer would defeat the world's greatest wrestler) must be assigned a particular numerical value such as 1/2, as if it were known as precisely as the probability of a truly random event (e.g., a coin flip).

Robust Bayes and belief functions are two methods that have been proposed to distinguish ignorance and randomness. In robust Bayes, a parameter can be restricted to a range, but without a prior distribution, yielding a range of potential posterior inferences. In belief functions (also known as the Dempster-Shafer theory), probability mass can be assigned to subsets of parameter space, so that randomness is represented by the probability distribution and uncertainty is represented by large subsets, within which the model does not attempt to assign probabilities.

Through a simple example involving a coin flip and a boxing/wrestling match, we illustrate difficulties with pure Bayes, robust Bayes, and belief functions. In short: pure Bayes does not distinguish ignorance and randomness; robust Bayes allows ignorance to spread too broadly, and belief functions inappropriately collapse to simple Bayesian models.

Mark Liberman at Language Log has traced the pirate's "Rrrrr" to a 1950 movie version of Treasure Island. Which reminded me of something. I read Treasure Island a few years ago and was just delighted and amazed by its readability. That plot really moved. Really un-put-downable. (Unfortunately its ending is weak--things get wrapped up a bit too quickly--but otherwise I'd say the book is perfect.) I was also amused that it had all the cliches of the pirate genre--X marks the spot and all that. But of course they weren't cliches back then--or were they?? I seem to recall reading somewhere that much of Treasure Island was ripped off from a book from the 1820s or so. (I can't remember the details.) This disturbed me, but then I decided that novels back then were like movies and TV today--it was all about doing a good job, not about originality. I mean, nobody criticizes Martin Scorsese or Steven Spielberg etc. of ripping off old movies--that's just beside the point.

On a related topic, I found Dr. Jekyll and Mr. Hyde also to be incredibly readable, and also very suspenseful. Yes, I knew that the Dr. and the Mr. were the same person, but there was a lot of suspense about what would happen next. This was also an interesting book because I did not find its individual sentences to be well-written--they were foggy, much like the London weather that pervades the book--but on the whole the paragraphs whipped by. In contrast, Moby Dick was just full of sparkling sentences, yet each page was a struggle to read.

In tomorrow's Applied Micro seminar, Regina Almeyda Duran speaks on "Proximate Literacy, Inter and Intrahousehold Externalities and Child Health Outcomes: Evidence from India." Here's the abstract:

In the course of studying social and political polarization, I have been thinking about the perceptions of political polarization over the past few decades. In the 1970s, there was much worrying about the decline of political parties, and a general concern that voters were deciding based on slick advertising and personality-based campaigns rather than on firmer grounds such as party affiliation. Since 2000, many political scientists, sociologists, and commentators have been disturbed by increasing political polarization (the differences between so-called red and blue America, and so forth), and a general concern that Democrats and Republicans can't communicate with each other.

The funny thing is, the commentators were concerned about declining party ID in the 1970s and increased party alignment in the 2000s. Shouldn't one of these have made them happy? Well, there are a lot of ways of looking at this, but one perspective is that most of the scholars and commentators have been Democrats. In the 1970s, most voters were Democrats, but the Republicans were doing pretty well in Presidential elections. It would be natural for a scholar to think: if only voters were sensible and followed their party ID, all would go well. . . . Contrariwise, in the 2000s, the voters are split between the parties, with more identified as conservative than as liberal--but they agree with the Democrats on many specific issues, especially economic issues. Thus, it's natural for a commentator to feel that if only the voters were following issues, rather than the liberal/conservative label, all would go well. . . .

Not that I'm saying there should be no cause for concern. As far as I can tell from books I've read, parties in the 1960s and earlier had a large local component, and voting by party involved a long chain of personal connections. Whereas voting by ideology now, whether liberal or conservative, is often more abstract and media-driven. So I think that it's possible to argue that parties then were beneficial in a way that ideologies now are not. All the same, it's interesting to see how these trends are perceived.

The following arrived in the email yesterday:

Seth Roberts is guest-blogging at Freakanomics with lots of interesting hypotheses about low-budget science, or what might be called "distributed scientific investigation" (by analogy to "distributed computing").

One of the paradoxes of Seth's self-experimentation research is that it seems so easy, but it clearly isn't, as one can readily see by realizing how few scientific findings have been obtained this way. Reading Seth's article in Behavioral and Brain Sciences gave me a sense of how difficult these self-experiments were. They took a lot of time (lots of things were tried), required discipline (e.g., standing 8 hours a day and setting up elaborate lighting systems in the sleep experiments) and many many measurements, and were much helped by a foundation of a deep understanding of the literature in psychology, nutrition, etc.

Also, for those interested in further details of Seth's diet, I'll cut-and-paste something from his blog entry.

Seth writes:

I read a couple of psychology papers recently and was impressed by their thoroughness. Each of these papers (one by Pelham, Mirenberg, and Jones, and one by Roberts) had 10 separate studies covering different aspects of their claims. The standard in applied statistics seems much lower: what's expected is that we do one good data analysis, along with explorations of what might have happened had the data been analyzed differently, assessment of the importance of assumptions, and so on.

Standards are lower in applied statistics

The difference, I think, is that in a statistics paper--even an applied statistics paper--the goal is to study or demonstrate a method rather than to make a convincing scientific case about the application under consideration. I mean, the scientific claims being presented should be plausible, but the standards of evidence seem quite a bit lower than in psychology.

What about other fields? Biology and medicine, oddly enough, seem more like statistics than psychology in their "convincingness" standards. In these fields, it seems common for findings to be reversed on later consideration, and typically a research paper will present the result of just one study. (In medicine, it is common to have review articles that summarize 40 or more studies in an area, and it seems accepted that individual studies are not supposed to be convincing in themselves.)

Political science, economics, and sociology seem somewhere in between. Research papers in these fields will sometimes, but not always, include multiple studies, but there is also often a requirement for a theoretical argument of some sort. It's not enough to show that something happened, you also have to explain how it fits in (or refutes) some theoretical model.

The paradox of importance

Getting back to statistical research, one thing I've noticed is that the most elaborate research can be done on relatively unimportant problems. If there is a hurry to solve some important problem, then we'll use the quickest methods at hand--we don't have the time to waste on developing fancy methods. But if it's something that nobody cares about . . . well, then we can put in the effort to really do a good job! In the long run, these new methods we develop can become the quick methods for the important problems of the future, but meanwhile we often see cutting-edge applied statistical research on problems that are of little urgency.

Matthew Kahn asks, "Why are there so few economists in elected office?" (link from Arnold Kling). He states that 45% of Congressmembers are lawyers (which seems a bit much, I must say!) but only some small number of economists (Matthew names two).

I was curious about this so I looked up some statistics--not on Congress but on the workforce. According to the 1001 Statistical Abstract of the United States (within arms reach of my computer, of course!), there were 139,000 economists employed in the United States, which reprsented 0.1% of the employed population. 1% of 535 is about 1/2, so with at least two economists in Congress, the profession is hardly unrepresented.

Serge Galam of the Laboratoire des Milieux Desordonnees et Heterogenes (the social sciences just sound cooler when they're in French) is speaking here on "Contrarian deterministic effects on opinion dynamics: "the hung elections scenario'". The paper is here.

I'm skeptical of physicists doing social science, but on the other hand, Galam's paper seems somewhat related to some (much simpler) work of mine on coalition-formation as a potential explanation for political instability.

Galam's talk is Wed 14 Sept in 1219 International Affairs Building.

Zhiqiang Tan recently wrote two papers on the theory of causal inference: see here and here. Here are the abstracts:

Scott de Marchi writes, regarding the "blessing of dimensionality":

One of my students forwarded your blog, and I think you've got it wrong on this topic. More data does not always help and this has been shown in numerous applications -- thus the huge lit on the topic. Analytically, the reason is simple. Just for an example, assume your loss function is MSE; then, the uniquely best estimator is E(Y | x) -- i.e., the conditional mean of Y at each point X. The reason one cannot do this in practice is that as the size of your parameter space increases, you never have enough data to span the space. Even if you change the above to a neighborhood around each x, the volume of this hypercube gets really, really ugly for any value of the neighborhood parameter. The only way out of of this is to make arbitrary restrictions on functional form, etc. or derive a feature space (thus "tossing out" data, in a sense).

As I said, there's a huge number of applications where more is not better.
One example if face recognition --increasing granularity or pixel depth
doesn't help. Instead, one must run counter to your intuition and throw
out most of the data by deriving a feature space. And, face recognition
still doesn't work all that well, despite decades of research.

There's a number of other issues -- in your comments on 3 "good" i.v.'s
and 197 "bad" ones, you have to take the issue of overfitting much more
seriously than you do.

My reply: Ultimately, it comes down to the model. If the model is appropriate, then Bayesian inference should deal appropriately with the extra information. After all, discarding most of the information is itself a particular model, and one should be able to do better with shrinkage.

That said, the off-the-shelf models we use to analyze data can indeed choke when you throw too many variables at them. Least-squares is notorious that way, but even hierarchical Bayes isn't so great when the large number of parameters have structure. I think that better models for interactions are out there for us to find (see here for some of my struggles; also see the work of Peter Hoff, Mark Handcock, and Adrian Raftery in sociology, or Yingnian Wu in image analysis). But they're not all there yet. So, in the short term, yes, more dimensions can entail a struggle.

Regarding the problem with 200 predictors: my point is that I never have 200 unstructured predictors. If I have 200 predictors, there will be some substantive context that will allow me to model them.

Seth Roberts's work on self-experimentation is the subject of the Freakanomics column in this Sunday's New York Times. Regular readers of this blog will recall discussions of Seth's work here and here. Also a related study here.

The publicizing of Seth's work also is an interesting example of information transmission. Seth published a paper in Behavioral and Brain Sciences--a top journal, but not enough to get the work much publicity. I posted a link to it on our blog (circulation 200/day), it was picked up by Alex at Marginal Revolution (circulation 10,000/day) and from there was noticed by a columnist for the New York Times (circulation ~ 2 million/day). But I think the high quality of Seth's article in BBS, with all its experimental data and scientific context, was crucial, in convincing the two levels of gatekeeper--Alex and Stephen--that the work could be taken seriously.

Andy Nathan had a question about ordered predictor variables:

Craig Newmark writes:

Some people think Katrina will be bad for Republicans and for conservatives generally. . . . I disagree. . . . I think Katrina will ultimately redound to the benefit of conservatives. The pointed, effective question that conservatives used to pose: "Do you want the same people that run the Post Office and the DMV and the IRS running [fill in the blank] for you?" will now become "Do you want the same folks--local, state, and/or federal bureaucrats, whoever you prefer to blame--that responded to Katrina doing [fill in the blank] for you?"

This is an interesting point. If true, it suggests there is an inherent "moral hazard" for conservative politicians, in that they have a long-term incentive to perform poorly in order to discredit government performance more generally. (I wouldn't think the moral hazard holds in the short-term, since I'd assume that poor performance in office leads to a greater probability of losing the next election. But for a farseeing conservative who is willing to lose the next election, it seems that this moral hazard exists.

Not that moral hazards, or perverse incentives, in politics, are limited to conservatives. It's also been said that liberal politicians have an incentive to maintain poverty (to continue getting the votes of the disaffected poor), that anti-abortion politicians have an incentive to keep abortion legal, and so forth. I'm not quite sure what to make of this, or how to study it empirically. At some level we just have to assume that politicians are motivated by doing the right thing. But it's a little scary to think that a slow response to a disaster could be considered a plus.

Speaking of trendiness . . . in the Collective Dynamics Group this Friday aft:

Speaker: Matt Salganik Affiliation: Graduate Student, Sociology, Columbia

Title: Experiments on the collective generation of superstar cultural objects

The Baby Names site of Laura Wattenberg (which we mentioned here) also has a blog. Lots of fun stuff; for example see the recent entry on short and long names:

This sounds interesting, and it's highly statistical:

Organizing the world's information (the world is bigger than you think!) Craig Neville-Manning Engineering Director and Senior Research Scientist Google Inc.

Today's applied micro lunch seminar (which unfortunately I won't be able to attend):

When: Tuesday, September 6th 1:10-2:00pm

Where: International Affairs Building, Room 1027

Speaker: Olga Gorbachev (Graduate Student)

Title: "Have Our Lives Become More Unstable? An Investigation of
Individual Volatility of Welfare in the U.S. over
1980-2000."

Abstract:

Has the individual volatility of welfare changed and if so how? What
events led to these changes and what are the implications for public
policy? We examine the evolution of individual volatility of welfare
over 1980-2000 using data from two surveys: Panel Study of Income
Dynamics (PSID) and Consumer Expenditure Survey (CEX). We find that
on average, micro level data follows macro trends. But, when
specific groups are considered, substantial differences are
observed. Older generations, those born between 1915 and 1944,
experienced increasing levels of volatility over 1980-2000 period,
and those born between 1960 and 1974, encountered decreasing
volatility, independent of their educational attainment. Those born
between 1945 and 1959 saw a decrease in volatility only if they had
some college education otherwise, they experienced an increased
volatility. We propose several reasons for the divergence of the
patterns and conclude by estimating social cost to the society and
to individual groups from changes in volatility measured over
1980-2000 period.

Awhile ago I discussed the Flynn effect and Seth Roberts's view that the writing in newspapers and magazines had become more sophisticated in the past 50 years--an idea that was consistent with Steven Johnson's book finding increased complexity in TV shows.

Seth just sent me something interesting along these lines. Seth writes:

I saw this in a NY Times article:

On Dec. 8, 1941, the day after the Japanese attack on Pearl Harbor, Representative Charles A. Eaton, Republican of New Jersey, made his case in the House for why the nation should enter the Second World War.

"Mr. Speaker," his speech began, "yesterday against the roar of Japanese cannon in Hawaii our American people heard a trumpet call; a call to unity; a call to courage; a call to determination once and for all to wipe off of the earth this accursed monster of tyranny and slavery which is casting its black shadow over the hearts and homes of every land."

Last year, Senator Sam Brownback, Republican of Kansas, made the case for war in Iraq this way:

"And if we don't go at Iraq, that our effort in the war on terrorism dwindles down into an intelligence operation," he said. "We go at Iraq and it says to countries that support terrorists, there remain six in the world that are as our definition state sponsors of terrorists, you say to those countries: we are serious about terrorism, we're serious about you not supporting terrorism on your own soil."

The linguist and cultural critic John McWhorter cites these excerpts in his new book, "Doing Our Own Thing: The Degradation of Language and Music and Why We Should, Like, Care" (Gotham Books). They not only are typical of speeches made in Congress on both occasions, he argues, but also provide a vivid illustration of just how much the language of public discourse has deteriorated.

Notice that what Brownback said is considerably more conceptually complex than what Eaton said, even though the number of words is about the same.

On first glance (and with all due respect to my complete ignorance of the field of linguistics), I agree with Seth on this one: in terms of content, Brownback's statement is much more sophisticated than Eaton's, and I don't see this as "deterioration" at all.

Seth continued with:

Something similar: A few days ago I read a talk by Richard Hamming called "You and Your Research" (google the title to find it) in which he notes:

John Tukey almost always dressed very casually. He would go into an important office and it would take a long time before the other fellow realized that this is a first-class man and he had better listen. For a long time John has had to overcome this kind of hostility.

Relatively complex ideas in a relatively casual package (Brownbeck, Tukey) causing negative feelings in listeners (McWhorter, "the other fellow").

Perhaps my self-experimentation suffers from a similar problem: How dare he measure his own weight! Or his own mood. It's too casual! This analogy suggests history is on the side of self-experimentation. Business dress, like the speeches of congressmen, has become more casual.

Interesting thoughts. I assume this is the same John McWhorter who contributes to the cool Language Log website. I wonder what McWhorter think of Seth's comments on the complexity of public statements.

P.S. Mark Liberman of Language Log has a long and interesting response here (for some reason, I can't get these things to show in Trackback). Here's what I have to say in response to his comments:

Regarding the particular issue in my posting, I think that Seth was responding to the NYtimes article referrong to how "The language of public discourse has deteriorated." Seth is arguing that, if content increases while style becomes simpler--well, that's not deterioration at all, but rather an improvement on two counts.

Also, I like the graphs in Mark's post. I think they'd be sligltly imporved by extending the y-axes down to zero.

I got an email about something called "Adventures in Statistics: Cartoon Learning Modeles." I don't know anything about this but thought it might interest some people. Here's the link.

There's really no need to link to Junk Charts anymore, since if you're interested in data display, you're probably reading it already . . . but here's a nice one:

A. J. P. Taylor wrote (in the Journal of Modern History in 1977), "Once, when I applied for an appointment at Oxford which I did not get, the president of the College concerned said to me sternly: 'I hear you have strong political views.' I said: 'Oh no, President. Extreme views weakly held.'"

Reading this set me to thinking of how such a position would fit into the usual "spatial models" of voting and political preferences, where an individual is located based on his or her views on a number of issues or issue dimensions. How does "extreme views weakly held" compare to "weak views strongly held," etc.? One approach would be to have the view and its certainty on two different dimensions, but that certainly wouldn't be right, as the spatial model is intended to represent the view itself. Another modeling approach would be to put Taylor in an extreme position corresponding to his views, but give him a large "measurement error" to allow for his views to be weakly held. But I don't think this is appropriate either; "measurement error" in such models corresponds to possible ignorance or different interpretations of particular issue positions, not to uncertainty about one's views.

The problem seems similar to the characterization of uncertain probabilities, as in this puzzle from Bayesian statistics.

P.S. Taylor was unusual, at least in the context of current debates over history, in combining left-wing political views with a focus on the role of contingency in history.

Linking to an Onion story is really too easy, but I think I'm allowed to do it just this once since it relates to our own research . . .

In Bayesian inference, all uncertainty is represented by probability distributions. I remember in grad school discussing the puzzle of distinguishing the following two probabilities:

(1) p1 = the probability that a particular coin will land "heads" in its next flip;

(2) p2 = the probability that the world's greatest boxer would defeat the world's greatest wrestler in a fight to the death.