The Heavy Reliance of Chemical Academia on Opinion
Since the last post, several other chemistry bloggers have weighed in on Professor Murray’s editorial in Anal. Chem. (here, here, here, here, here, here, and here). The feedback has been almost universally negative, which you would probably expect given that: i) these are responses from bloggers and blog commenters, and ii) the arguments in the editorial were neither substantiated nor well reasoned.
What makes Murray’s editorial especially troubling has as much to do with its source as its content. If some generic person had posted the text of his editorial as a comment on a blog or a letter-to-the-editor for C&EN, few people would have given it much attention. But this editorial came from Royce Murray. Royce Murray: a superstar of chemistry who’s built a wildly successful scientific career at a fantastic department. If such an experienced and brilliant leader in the worlds of chemistry and scientific publishing can produce an analysis that is so far off-base as his editorial on blogs, it’s not unreasonable to hypothesize that similar lapses (that we don’t see) occur on a regular basis.
The thoughts of respected professors are routinely solicited for such important decisions as accepting papers for publication, assigning grades in classes, funding grants, and hiring young scientists for faculty positions. The opinions proffered in these scenarios—unlike editorials in popular ACS journals—are invisible to the community. While occasional errors in judgment probably won’t disrupt the balance of the Force, they can easily have grave consequences for individual chemists.
It’s a little bit disconcerting to think about how much weight our field seems to place in the opinions of small sets of scientists. For faculty applications, n = 3 letters; for submitted papers, n = 1-3 referee reports; for grants, n = a dozen people on a panel; and for the undergraduate science fair I judged on Saturday, n = 2 judges. So long as this practice is the norm in chemistry, playing politics will always be viewed as important. You never know whether your success could hinge on whether someone isn’t playing with a full deck or whether you’ve made nice with one or two particular scientists. The door is wide open for payback…shielded by the anonymity guaranteed by the institutions of scientific publishing and funding.
The thing is, I’m not sure that there’s a better system. Assessment of success in chemistry is not easily boiled down to objective, quantitative metrics (e.g., publications or citations). Subjective analysis is pretty much unavoidable, and correspondingly, personal biases and botched analyses are bound the enter the equation. In some cases (e.g., faculty applications) at least recommendations are only one piece of a larger puzzle, but for the other things, a bad bit of luck in terms of who gets assigned to assess you can really ruin your day.
Over time, I’d like to see more open forms of assessment enter the picture. Journals (like Anal. Chem.), you can start: Publish referee reports alongside the Supporting Information and create comment threads for each published paper. We’re big boys and girls; we can (or at least, should be able to) handle criticism.