In a wonderful post at Slate, Carl Zimmer describes the one wonderful thing about the whole #arseniclife paper published last November, and has identified what will probably be its lasting contribution: The reaction to that paper both catalyzed and revealed the power of more open peer review — a fast, post-publication peer review by the wider scientific community — to accelerate and make open the sort of slow, more private post-publication peer review that has always been the real adjudicator of a paper’s significance.
Both NASA and the authors tried to play the bloggers-in-their-pajamas card, but it was a losing hand. For one thing, the people who were talking on blogs and Twitter were not in their pajamas… [They are scientists] starting to carry out a new way of doing science, known as post-publication peer review. Rather than leaving the evaluation of new studies to a few anonymous scientists, researchers now debate the merit of papers after they have been published. The collective decision they come to stays open to revision.
Post-publication peer review—and open science in general—is attracting a growing number of followers in the scientific community. But some critics have argued that it’s been more successful in theory than in practice. The #arseniclife affair is the one of the first cases in which the scientific community openly vetted a high-profile paper, and influenced how the public at large thought about it.
This sort of open, community-driven evaluation helps take over one of the functions that has long been tied up in the consolicated functions and power of the scientific journal paper, as I described in Free Science, One Paper at a Time. That power of review, and to establish the first impression of legitimacy and significance, has been guarded jealously by many scientific publishers, as it was in this case by Science. But as Zimmer notes, “The center of gravity has shifted.”
It’s a splendid piece, and essential to understanding what this episode means and how much it reveals about how science is changing.
See also:
Science Publishes “Arsenic is Life” Critiques. Game On.
Free Science, One Paper at a Time
The Wrong Stuff: NASA Dismisses Arsenic Critique Because Critical …
Arsenic Author Dumps Peer Review, Takes Case to TED | Wired …