It’s great to know that Shakespeare scholars are reading this blog. Alan Farmer (Department of English, Ohio State University) wrote to me several months ago:

I’ve recently co-authored an article that relies on statistical analyses of the early modern English book trade (specifically, the publication of plays from 1576 to 1660), and that article has elicited a response from someone in my field. In our rebuttal to that response, we want to point out that the chi square test addresses one of the respondent’s central concerns and proves our point with a high degree of confidence, and we need to do so for an audience of literary critics (I believe this will be the first time the phrase “chi square test” has ever appeared in “Shakespeare Quarterly”).

What I was hoping is that either I could ask you or you could refer me to someone who might be able to read over the two relevant paragraphs in our rebuttal (which are fewer than 500 words together) and confirm both that we’re using the test properly and that we’re describing its results accurately. We think we are, but we’d feel greatly reassured if a specialist confirmed that for us.

In case it helps, I’ve included below a short description of the statistics we’re analyzing.

The details (omitting the details of our counting methods):
Among the total population of books published in England from 1576 to 1640, we’re comparing how often playbooks and books of sermons were reprinted.

For playbooks, we get this: 83 of 208 were reprinted.
For sermon-books, we get this: 188 of 972 were reprinted.

We conclude that we can be confident that these figures suggests these two types of books actually performed differently in the book trade.

Our respondent believes it is not justified to compare percentages when the sample populations are too different and declares this is the case in the above example. He then throws this out attempted reductio ad absurdum:

For plays by Shakespeare, 8 of 23 reach a third edition.
For plays by Edward Sharpham, 2 of 2 reach a third edition.

He concludes that it’s just as ridiculous to point to the difference in reprint rates b/w sermons and plays as it is to point to this difference in the plays of Sharpham and Shakespeare. We want to say, in his second example, we cannot conclude that this difference in reprint rates is statistically significant (but in the first one it is).

As I wrote above, we have two paragraphs making this same point in a more polished and elegant way.

My response:

Your response is fine. Indeed, sample size is key.

Alan asked me at the time not to post this on the blog until the paper had apperared. Now it has. He writes:

At long last I’m getting back to you with links to the articles related to the question I sent you this past summer. To refresh your memory, I’ve co-authored two articles on the popularity of printed playbooks during from 1576 to 1660 that rely on statistical measurements of the economic performance of different types of books during that period. Our first article was answered by a critique from Peter Blayney, which we then responded to in a third piece and in which we laid out the methodological, statistical, and historical underpinnings of our argument.

The section that you might find most relevant comes on pp. 208-10 of the third article, which is where we discuss the chi-square test and where we thank you. The first article also has five graphs in case you’re curious how those in English are now employing the visual display of quantitative information (Franco Moretti has recently been generating a fair amount of press for his use of “Graphs, Maps, and Trees” in studying the 19th-century novel).

1) Alan B. Farmer and Zachary Lesser, “The Popularity of Playbooks Revisited,” Shakespeare Quarterly 56 (2005): 1-32.

2) Peter W. M. Blayney, “The Alleged Popularity of Playbooks,” Shakespeare Quarterly 56 (2005): 33-50.

3) Alan B. Farmer and Zachary Lesser, “Structures of Popularity in the Early Modern Book Trade,” Shakespeare Quarterly 56 (2005): 206-13.

A statistical controversy in Shakespeare studies! I love it. My default, of course, is to side with the person who reads our blog.