The Heavy Reliance of Chemical Academia on Opinion
Since the last post, several other chemistry bloggers have weighed in on Professor Murray’s editorial in Anal. Chem. (here, here, here, here, here, here, and here). The feedback has been almost universally negative, which you would probably expect given that: i) these are responses from bloggers and blog commenters, and ii) the arguments in the editorial were neither substantiated nor well reasoned.
What makes Murray’s editorial especially troubling has as much to do with its source as its content. If some generic person had posted the text of his editorial as a comment on a blog or a letter-to-the-editor for C&EN, few people would have given it much attention. But this editorial came from Royce Murray. Royce Murray: a superstar of chemistry who’s built a wildly successful scientific career at a fantastic department. If such an experienced and brilliant leader in the worlds of chemistry and scientific publishing can produce an analysis that is so far off-base as his editorial on blogs, it’s not unreasonable to hypothesize that similar lapses (that we don’t see) occur on a regular basis.
The thoughts of respected professors are routinely solicited for such important decisions as accepting papers for publication, assigning grades in classes, funding grants, and hiring young scientists for faculty positions. The opinions proffered in these scenarios—unlike editorials in popular ACS journals—are invisible to the community. While occasional errors in judgment probably won’t disrupt the balance of the Force, they can easily have grave consequences for individual chemists.
It’s a little bit disconcerting to think about how much weight our field seems to place in the opinions of small sets of scientists. For faculty applications, n = 3 letters; for submitted papers, n = 1-3 referee reports; for grants, n = a dozen people on a panel; and for the undergraduate science fair I judged on Saturday, n = 2 judges. So long as this practice is the norm in chemistry, playing politics will always be viewed as important. You never know whether your success could hinge on whether someone isn’t playing with a full deck or whether you’ve made nice with one or two particular scientists. The door is wide open for payback…shielded by the anonymity guaranteed by the institutions of scientific publishing and funding.
The thing is, I’m not sure that there’s a better system. Assessment of success in chemistry is not easily boiled down to objective, quantitative metrics (e.g., publications or citations). Subjective analysis is pretty much unavoidable, and correspondingly, personal biases and botched analyses are bound the enter the equation. In some cases (e.g., faculty applications) at least recommendations are only one piece of a larger puzzle, but for the other things, a bad bit of luck in terms of who gets assigned to assess you can really ruin your day.
Over time, I’d like to see more open forms of assessment enter the picture. Journals (like Anal. Chem.), you can start: Publish referee reports alongside the Supporting Information and create comment threads for each published paper. We’re big boys and girls; we can (or at least, should be able to) handle criticism.
October 18th, 2010 at 7:27 AM
If anyone is so inclined, you can leave comments on Anal Chem articles here:
http://www.chemfeeds.com/analchem.php
October 18th, 2010 at 7:50 AM
I just found out that the journal of the European Molecular Biology Organization publishes a “review process file” which summarizes all author-referee correspondence. The reviewers are still anonymous.
October 18th, 2010 at 8:20 AM
Ed’s looking a little Gandalf-esque there.
October 18th, 2010 at 8:57 AM
Thanks for the magnets. My toddler has been running around the house, putting them all over everything.
Q: “Who’s that?”
A: “Eded the Dogggg.”
October 18th, 2010 at 9:29 AM
excimer, he’s actually looking Royce-esque!
October 18th, 2010 at 10:04 AM
Royce is looking rather Old Ed-esque.
October 18th, 2010 at 3:56 PM
Reading blogs I have noticed that, as soon as disussion extends beyond their area of expertise, several scientist do not provide sources and do regurgitate unquestioningly received opinion. Annoying value system.
October 18th, 2010 at 4:20 PM
[...] The Heavy Reliance of Chemical Academia on Opinion [...]
October 18th, 2010 at 10:06 PM
@Chemjobber: I’ve always thought the readership of ChemBark skewed young, but this is ridiculous.
@Joel: That is a major step forward by the people at Nature Publishing Group. Now readers can see the anonymous referee reports and the authors’ responses. (Incidentally, this was part of feature #1 in my 2007 proposal for what I’d do if I became the editor of JACS). These documents will certainly be helpful in letting those less experienced in a subject area become more familiar with experimental pitfalls and such. I hope the Nature Chem editors follow suit. Bravo to NPG.
October 19th, 2010 at 2:31 PM
you always pay for what you are publishing, in one sense, or the other
(remark the Harvard Komma)
October 19th, 2010 at 2:36 PM
it means : breath always in the right position
October 20th, 2010 at 10:56 AM
Wolfie is a has-been in the race for best crazy commenter. Uncle Al is the king.
October 20th, 2010 at 2:37 PM
Uncle Al sounds like the love child of Harvey Mansfield and Anne Coulter with a chemistry degree and some time spent in New Mexico and Montana, but he’s definitely coherent. Wolfie just sounds incoherent, like a radio that can’t stay tuned but flips through the stations.
One advantage of Uncle Al is that he hasn’t seemed to exhaust his welcome – after awhile people like Scott in Oregon just seem to unite everyone in the opinion that they have nothing to say and should be saying it on an ice floe in the South Atlantic to dead penguins, but not Uncle Al.
October 20th, 2010 at 9:29 PM
Uncle Al is to the chemblagosphere what Robert Anton Wilson was to novels
October 22nd, 2010 at 1:04 PM
Hmm… didn’t know that EMBO-J did that with reports and correspondence. I don’t know if a similar thing has been discussed in relation to the Nature-branded titles (note that EMBO-J is published by NPG on behalf of the European Molecular Biology Organization, but it is editorially independent of NPG). In principle I’m not against doing this sort of thing (speaking personally rather than with my NChem hat on here), but it’s not something a Nature-X journal can do overnight… and it would be something decided by people much further up the food chain than myself. It would also require editorial resources, which is another issue – but not necessarily an insurmountable one. And of course, you’d miss out on a lot of juicy details about all the papers that get rejected since you can only publish this detail for accepted papers…
At least we always feedback to the referees what happens with a particular manuscript – and we send all of the referees all of the reports on a paper they have reviewed for us, so in some ways it serves to educate and calibrate referees to some extent.