Archive for November 14th, 2011

Pitfalls of Peer Rerview

Monday, November 14th, 2011

In 2006, Nature tried an experiment. The journal receives about 10,000 manuscripts a year and sends 40% of them out for traditional peer review. In the trial, the editors asked authors if they would also submit their paper for open peer review where any scientists could leave signed comments. 71 authors agreed.

The journal promoted the experiment heavily on their website, through e-mail blasts, and with targeted invitations to scholars in the field. After four months, they reviewed the results. nature05535-i2Despite sizable web traffic to the site, 33 papers received no comments, and the most heavily commented on paper received only 10 replies.

Nor did the editors find the comments influential in their decisions whether to publish. They found that although many scientists approved of the idea of open review, very few would perform it.

Their experiment demonstrates both the promise and the pitfalls of social media. It opens up the possibility for dialogue, but it depends on self-motivated users to enrich the content.