You can’t put Pandora back in the box

Rajiv Sethi writes:

I suspect that within a decade, blogs will be a cornerstone of research in economics. Many original and creative contributions to the discipline will first be communicated to the profession (and the world at large) in the form of blog posts, since the medium allows for material of arbitrary length, depth and complexity. Ideas first expressed in this form will make their way (with suitable attribution) into reading lists, doctoral dissertations and more conventionally refereed academic publications. And blogs will come to play a central role in the process of recruitment, promotion and reward at major research universities. This genie is not going back into its bottle.

And he thinks this is a good thing:

In fact, the refereeing process for blog posts is in some respects more rigorous than that for journal articles. Reports are numerous, non-anonymous, public, rapidly and efficiently produced, and collaboratively constructed. It is not obvious to me [Sethi] that this process of evaluation is any less legitimate than that for journal submissions, which rely on feedback from two or three anonymous referees who are themselves invested in the same techniques and research agenda as the author.

I don’t disagree with these sentiments, although I do think that, if blogging every becomes important in statistics, it will come much slower than in economics, political science, computer science, or even mathematics. Blogging has been big since 2003–that’s the year when, it seems to me, the general practice of trackbacking and blogrolling started to go away, leaving new blogs to be found on their own rather than through a network of references. But in all these years, very few statistics blogs have achieved much attention.

Sethi points out that, compared to journal articles, blog entries can be subject to more effective criticism. Beyond his point (about a more diverse range of reviewers), blogging also has the benefit that the discussion can go back and forth. In contrast, the journal reviewing process is very slow, and once an article is published, it typically just sits there. I personally like to publish discussion papers (that is, articles where others discuss and then I write a rejoinder), but most published journal articles don’t have that format. (In that way, statistics may be better than economics. The field of statistics appears to accept that articles are attempts rather than realizations of perfection, whereas my impression is the economists worship at the altar of the “home run,” the article that is such a perfect jewel that it is beyond criticism.

Can/should the blogosphere replace the journal-sphere in statistics? I dunno. At times I’ve been able to publish effective statistical reactions in blog form (see, for example, my skepticism about reported statistical significance in a brain-scan study) or to use the blog as a sort of mini-journal to collect different viewpoints (for example, our discussion with Pearl, Dawid, and others on causal inference). And when it comes to pure ridicule (I think you know who I’m thinking of here), maybe blogging is actually more appropriate than formally writing a letter to the editor of a journal.

But I don’t know if blogs are the best place for technical discussions. This is true in economics as much as in statistics, but the difference is that many people have argued (perhaps correctly) that econ is already too technical, hence the prominence of blog-based arguments is maybe a move in the right direction. Even technical types such as Paul Krugman and Greg Mankiw have become much more talky and less algebraic as bloggers than as authors of journal articles.

Statistics, though, is different. Setting aside debates about whether Ph.D. students in statistics should learn the strong law of large numbers (I think you can guess my position on that one), even the applied stuff that I do is pretty technical–algebra, calculus, differential equations, infinite series, and the like.

Can this sort of highly-technical material be blogged? Maybe so. Igor Carron does it, and so does Cosma Shalizi–and both of them, in their technical discussions, clearly link the statistical material to larger conceptual questions in scientific inference and applied questions about the world. But this sort of blogging is really hard–much harder, I think, than whatever it takes for an economics professor with time on his or her hands to regularly churn out readable and informative blogs at varying lengths commenting on current events, economic policy, the theories of micro- and macro-economics, and all the rest.

I think few will disagree that the most effective statistics blogging, by a longshot, has been Nate Silver’s polling and election analysis on fivethirtyeight.com. Here,
Nate and I have actually published a couple of journal articles based on material related to the blog, but (a) the journal articles are quite a bit more technical than the blog entries, and (b) as journal articles, they don’t represent major research efforts–rather, they fall into the “fun applications” category. Ultimately, Nate’s blogging succeeds because it is news, not because of its research content. From another direction, I think Scott Sumner’s econ blogging succeeds because it is well-written and because it supports a fiscally-conservative position that many people want to hear. Krugman’s blog works because it’s linked to a popular New York Times column and he takes a strongly partisan political stance, and Levitt and Dubner’s blog succeeds along the same lines as Cowen and Tabarrok’s–readers are getting a mix of news, provocation, and bite-sized analyses. All of these play important roles, but not quite the roles of journal articles.

On the other hand, the current system of scientific journals is, in many ways, a complete joke. The demand for referee reports of submitted articles is out of control, and I don’t see Arxiv as a solution, as it has its own cultural biases. I agree with Sethi that some sort of online system has to be better, but I’m guessing that blogs will play more of a facilitating informal discussions rather than replacing the repositories of formal research. I could well be wrong here, though: all I have are my own experiences, I don’t have any good general way of thinking about this sort of sociology-of-science issue.

9 thoughts on “You can’t put Pandora back in the box

  1. I know she caused lots of mischief, but can't we forgive Pandora rather than trying to lock her in that box? :)

    I haven't seen many mathematically rigorous blogs, and the ones I have run into I've passed over. Of the tens of thousands of research papers published a year, there's only a handful I've got the knowledge and interest to actually slog through. Perhaps its different within academia, but a real research blog wouldn't have enough highly germane material to keep me coming back.

    Which isn't to say some blog-like features wouldn't be nice in a formal paper. Some comments at the end so I can get another perspective on the work.

    In my experience blogs work best at letting us share a view of what's going on in a field. I've learned plenty about vulcanology, statistics, C.S. Theory, and a handful of other fields from practitioners' blogs.

    If anything, blogs are replacing pop-sci books for me. Things like Hawking's 'A Brief History of Time', that say "this is what we know about the world" while skipping over the nitty-gritty of "this is how we know it". It's pretty remarkable to have this much exposure to this many experts. I think the Internet excels at being wide, rather than deep.

  2. I guess economics and political science are a bit special in that much interesting or useful comment is interesting or useful because it is _timely_. Clearly, no academic journal can match the instant publication of blogs. But in this respects blogs don't differ much from newspapers, television etc. And in lots of academic disciplines really rapid publication is of little consequence. When it is, preprint publication has long preceded blogs.

    I think in many respects many academic disciplines have yet to catch up with something that started several decades ago, writing computer code. In many disciplines or institutions there is a serious lack of respect for contributions in the form of software. So, the possibility that universities, which are in many ways ultra-conservative institutions, will suddenly change their values and that some academics will shortly surf to success on a blog wave, and not much else, seems highly far-fetched to me.

  3. A.: Another culture clash… Actually, Pandora had a jar and not a box, of the kind in which Diogenes was living, so she could have gone inside the jar, indeed.

  4. Andrew,

    Thanks for the plug. In my case, the blog is really an exercise in making sense of the new body of literature on a specific subject that spans different fields. Each community has its own culture and most authors who publish on that subject rarely venture outside their own field (i.e. there are no common journals where authors in each of these fields would publish).

    I am sure you have seen the same thing in some fashion or another but in the course of running the blog, I have seen several instances of traditional "peer review" in action where authors would point out that new materials were either wrong or not novel. My initial finding was that the blog was also bringing a "lot" of traffic to the papers being featured there. By a lot, I mean a paper would be read by more than 10 to 100 people in one day. To some it is a two order magnitude increase :-)

    But a much more interesting dimension has emerged from it as well. For instance, I have seen:

    – different sorts of people getting together and publishing because of the blog,

    – people changing their academic trajectory as a result of reading the blog.

    – non-specialists applying compressed sensing techniques in their field without a long incubation period (example: Compressed Genotyping,…)

    – open discussions on several highly technical points that only about ten people understand at one point in time. Access to that discussion used to traditionally be dependent on attending the right conference and following the right people at that conference.

    – people getting "ah-ah" moments because of the dumb questions I asked authors. I sometimes run Q&As because authors have generally more to say than what could be mentioned in the abstract/introduction of their papers.

    Finally, the blog allows for some thought provoking write-ups that just could not make it into a traditional engineering or applied math paper. Because they are inspiring to a larger set of people, these write-ups tend to be correlated with an increase in readership.

    In short, replacing traditional journals, I am not sure. Changing the dynamics in a field, I'd tend to think that this is already happening.

    Igor.

  5. The PLoS journals have blogesque comment options for each article. Mostly these sit unused, but some articles attract quite a few comments. See, for example, Ioannidis' PLoS Med article Why Most Published Research Findings Are False. Reviews are also quick (relative to traditional journals). PLoS One now publishes more articles than any other journal, has a good impact factor (4.3), and takes stats papers, so if you want to combine the best of blogs and journals, maybe that's the place to publish?

  6. My best guess at the future re: blogs and academic publishing is that authors will publish their own works which will be aggregated by several organizations.

    Since the cost of publishing an article is effectively zero now, the journal serves no purpose but to aggregate articles. Authors can use this to their advantage but publishing their own works, keeping the copyright, and allow many (or any) organization to aggregate their content. That is, if I were sitting on an epic paper in cell biology, why would I settle for it to be published in Science when I could have it appear in Science AND Nature.

    With this model there is still value in the aggregator who finds relevant works, authors benefit from potentially better exposure of their works, and "quality" control is done interactively by scientists through comments, likes, responses, etc. that link to the original source documents

Comments are closed.