Nature Responds to ACS Nano Editorial, “Gets” Post-Publication Review

October 31st, 2013

nchemfc_kit_250The editors of Nature just penned an editorial promoting post-publication review of papers like the kind that takes place on blogs and sites dedicated to these sorts of discussions. PubPeer is a prime example of such a site, and PubMed just opened comment threads for papers. The folks at Nature point to the muzzling of scientific criticism recently advocated by the editors of ACS Nano as “likely to backfire”:

Alarmed at the tenor of criticism when it concerns misconduct, some editors have tried to rein in online discussion, and to bring it within the limits of conventional debate. But attempts to dictate terms are likely to backfire. In a recent editorial in the journal ACS Nano, for example, the editors asserted that “the numbers of blogs, twitter messages, etc. in which individuals accuse others of academic fraud are steadily rising” — although they did not provide evidence for this. And they asked that suspicions of plagiarism or data manipulation be reported directly to a journal, rather than posted openly online (W. J. Parak et al. ACS Nano 7, 8313–8316; 2013). It was others’ “privilege” to be able to comment on a journal’s decision on a blog afterwards, the editorial added.

Although written with concern for the fair treatment of scientists who suffer damage to their reputation when comments are made irresponsibly, the editorial raised the hackles of chemistry bloggers who have pointed out egregious examples of image manipulation in papers — and who understandably consider that it is they, as much as the journals, who are doing the community the service (see go.nature.com/cplnfd). It is better to ask that debate be civil, responsible and courteous, than that it not appear online at all.

While the ACS Nano editorial did raise my hackles, the Nature editorial warmed the cockles of my heart. The editors of Nature obviously “get it”, for lack of a better description. The ACS Nano editorial was ridiculous because the editors wanted to turn back the clock on Web 2.0 technology, which has democratized scientific debate by giving a platform to less-established scientists whose natural voices would ordinarily be lost in the crowd.

Rather than stifle Internet discussion, the editors of Nature argue that editors should embrace it. They point out that this commentary represents useful information, and when it is fragmented across all sorts of different Web sites, it can’t be managed and preserved. Journal sites—with comment threads dedicated to each paper—are logical venues for these discussions. If journals set up systems that allow post-publication review, editors can manage it to ensure the discussion is fair and civil.

I’ve argued for a long time that journals should open comment threads on papers (in addition to publishing things like referee reports), because in addition to promoting scientific discussion and openness, such a system would allow young researchers to learn what kinds of flaws expert referees find in experiments and what data experts think are necessary to support a scientific conclusion. It’s a shame that this valuable analysis is wasted after the (pre-publication) review of a paper is complete.

H/T @DrStelling and @stuartcantrill on Twitter.


15 Responses to “Nature Responds to ACS Nano Editorial, “Gets” Post-Publication Review”

  1. Umbisam Says:

    I knew it was coming. It will be interesting if ACS Nano responds to Nature. As long as you don’t commit outright fraud, one should be relatively safe. People can attack you online for your interpretation or sloppy technique, but this happens anyways. Plus, it allows those of us who painstakingly reproduce results and take the time you use instruments correctly, consider the uncertainty in the measurements, etc. to hopefully more effectively compete with people who are more concerned about quantity over quality. One of the downsides to all of this is I don’t have time to read pubpeer to see if I’m getting flogged online.

  2. Umbisam Says:

    One nice thing about adding comments to papers: instead of emailing the corresponding author when he did work that is nearly the same as mine but didn’t site my paper, I can just post the link to my paper in the comments section.

  3. milkshake Says:

    Embarrassed journal editor is not any different from an embarrassed politician. Of course they would love to have the debate about those unpleasant revelation on their own terms – preferably none at all !

  4. Allison (@DrStelling) Says:

    @Umbisam: The citation thing could get pretty interesting, since those numbers are used in evaluations for a lot of different things. (Well, in the USA at least, and this depends on the field a bit. As usual.)

    Now funding agencies and panels can look at the paper’s comments, and find related works quite easily….perhaps from “competing” labs…..

  5. See Arr Oh Says:

    Paul: To more succinctly verse your thoughts, would you say Nature really “grokked” post-pub review? : )

  6. Erick Carlson Says:

    Paul:

    First off, love the site and have been following it for awhile now. In your post though you mention the following:

    “If journals set up systems that allow post-publication review, editors can manage it to ensure the discussion is fair and civil.”

    It’s that last bit that gets me. How do you expect editors, very busy scientists themselves, to manage these threads for each article. This surely seems impossible and would require a huge amount of time, effort, and possible money if the task is to be doled out to others qualified.

  7. Paul Bracher Says:

    @Erick: I think this task would have to be assigned to an editorial assistant. As the Nature editorial notes, post-publication analysis does not (yet) generate a whole lot of activity. I imagine that ACS Publications could hire one manager of social media to handle the moderation tasks for all of the journals.

  8. Is it Erick or Erin? Says:

    @”Erick” Carlson:

    Are you actually Erin Carlson, who is moving from IU to Minnesota? or are you someone else? Just curious! And sorry for the digression folks.

  9. Dave Fernig Says:

    It is indeed an interesting and very positive editorial. There remains the question of anonymity, which is allowed at PubPeer, but generally not at journals. This is important in science, simply because the majority of scientists are not that secure in their positions. They may not have tenure and/or they are very dependent on panels/committees for progression, funding and indeed publishing. One false move in many countries where decisions on career progression and funding are not that transparent and your career is effectively dead. The same might apply for publishing. I have served on panels and acted on their behalf in quite a few different countries and I would say that despite the claims , transparency is far from universal.

  10. Erick Carlson Says:

    I am indeed Erick. Not Erin. I’m not sure why I would hide under such a silly way to spell Erik. I mean really, who spells there name this way.

  11. John Spevacek Says:

    Posting reviews could get interesting, very interesting. We could see which papers got rubber-stamped and which ones got put through the wringer – twice.

    The only downside I see is that the call would then go out for releasing the reviewers names. While there are some valid reasons for such requests, I think the overall effect would be negative. As bloggers and commentators, we know the power of remaining anonymous, both good and bad. I seldom go anonymous, but would hate to see that level of security taken away.

  12. Evil ACS editor Says:

    Despite the vast resources of ACS Pubs, they could not possibly hire editorial assistants to manage forums for very paper. Consider how many papers they publish per year.

    I would also like to point out that “peer review anonymous” and “blog commenter anonymous” are not the same thing. A reviewer’s identity is known to the journal editors and staff. Their comments indirectly affect their reputation, at least with editors, who are in their field. I think scientists are more careful of what they say in a review than what they write on a blog comment.

    Now I must get back to doling out favors for my friends and peddling influence. Muwah ha ha ha!!

  13. Paul Bracher Says:

    @Evil ACS Editor: While the ACS publishes ~40,000 papers per year, if you look at how many comments are generated on sites that allow commenting, it’s not that many. The Nature editorial mentions this as well. If commenting becomes more popular, the ACS should be able to raise revenue through advertising to support hiring more moderators.

  14. Uncle Al Says:

    The greatest danger to managed, as opposed to pursued, science is some unregulated upstart discovering what is Officially untrue. Two gits say something remarkably stupid[1]. An ethnic confrère twists arms to fund and perform the experiment[2], whose impossible observation is confirmed[3]. Half of particle physics must be rewritten.

    Properly managed research had made the identical discovery 30 years earlier[4]. It was discarded as being insubordinate to the proper order of things. Management is about process not product. Woe unto anybody who discovers something new rather than further parameterizes what is properly believed.

    The ACS keeps chemistry monstrously over-subscribed to renormalize salaries and facilitate turnover to cheaper green graduates. The EPA and Homeland Severity keep busy incrementally outlawing every aspect of chemical education and industrial practice. (Try having a chem lab in your garage.) On the bright side, this does rebound staff crystal meth synthesis and hash oil extraction – both of which are well-paying, tax-free employments.

    [1] Phys. Rev. 104(1) 254 (1956), http://prola.aps.org/pdf/PR/v104/i1/p254_1
    [2] Phys. Rev. 105(4) 1413 (1957), http://prola.aps.org/pdf/PR/v105/i4/p1413_1
    [3] Phys. Rev. 105(4) 1415 (1957), http://prola.aps.org/pdf/PR/v105/i4/p1415_1
    [4] PNAS 14(7) 544 (1928), http://www.pnas.org/content/14/7/544.full.pdf+html

  15. Senyawa Organik Says:

    While the ACS Nano editorial did raise my hackles, the Nature editorial warmed the cockles of my heart. The editors of Nature obviously “get it”, for lack of a better description. The ACS Nano editorial was ridiculous because the editors wanted to turn back the clock on Web 2.0 technology, which has democratized scientific debate by giving a platform to less-established scientists whose natural voices would ordinarily be lost in the crowd.


Leave a Reply

*