Suspicious Data in a JACS Paper from 2009

November 7th, 2013

Covers of JACSOn Tuesday, the Journal of the American Chemical Society published a retraction notice for a paper titled “Bimetallic Effects for Enhanced Polar Comonomer Enchainment Selectivity in Catalytic Ethylene Polymerization” by Brandon A. Rodriguez, Massimiliano Delferro, and Tobin J. Marks (J. Am. Chem. Soc. 2009, 131, 5902−5919).

The retraction notice reads:

The authors have been unable to reproduce the synthesis and spectroscopic characterization of the ethylene/acrylate copolymer described in this article. Accordingly, the authors are retracting this publication due to concerns over the validity of the aforementioned data. The authors regret any confusion that may have been created by publication of this work.

A casual reader might encounter this notice and think this is exactly how the system should work: a scientist told of a problem with his published procedure went back, attempted to repeat the work, found he couldn’t, and published a notice to inform the community.

But a closer inspection reveals that there is more to this story that the retraction notice would indicate.

ChemBark was alerted to problems with this paper in August by an astute chemist who wishes to remain anonymous. One day earlier, the source had contacted Peter Stang, the editor-in-chief of JACS, to alert the journal to abnormalities with data in the paper. ChemBark respected the wishes of the source and allowed a traditional editorial review of the paper to be completed before reporting the matter on the blog. With the publication of the retraction notice yesterday, the authors have publicized that their work is problematic. It would appear that an initial review of the paper is complete.

The authors’ wording of the retraction does not indicate specifically what was wrong with the paper, but a brief inspection of the data (including those in the Supporting Information) reveals the following abnormalities:

Figure 3: The baselines of these spectra are peculiar in that they are thick but very straight. The thicknesses and shapes of the peaks are also peculiar. Some of the tall peaks lack the wide bases you would expect of them. Finally, some of the peaks appear to have a different background (i.e., noise) and line thickness (as if they originated from a different source).

marks_retraction_figure_3

Figure 6: The baselines (i.e., noise) of the two NMR spectra appear to be identical. The peaks look irregular—like solid lines.

Figure 7: Again, some of the peaks have lines that are much thicker than the rest of the spectrum. They look unnatural.

Figure S19 (in the SI): The NMR spectra taken at various temperatures have identical noise in their baselines, which would be very, very unusual. The peaks of interest appear irregular and have different line shapes.

marks_retraction_figure_s19

Figure S21: The background of the peaks appears different from the background of the rest of the spectrum, as if the peaks were cut-and-pasted into the spectrum.

Figure S22: Same as above. The spectrum in Figure S21 and the bottom spectrum of Figure S22 appear to have the same baseline noise even though these spectra are reported to be of two entirely different compounds.

marks_retraction_figure_s22

That’s not everything that looks peculiar, but do not simply take my word for it. Go to the paper, have a look, and judge for yourself (paper, SI). ChemBark does not know why these data look the way they do, but they do look unnatural.

On Tuesday, ChemBark contacted Professors Tobin Marks (the corresponding author on the retracted paper) and Peter Stang (the Editor-in-Chief of JACS) by e-mail for comment.

A response was received on Wednesday from Alan K. Cubbage, the chief communications officer for Northwestern University. In the interest of fairness, we are reporting this e-mail its entirety:

Mr. Bracher, your inquiry to a Northwestern University professor, Tobin Marks, was referred to me, as I am the chief communications officer for Northwestern.

As you note in your email, two Northwestern University faculty members and a former Ph.D. student have retracted a publication that appeared several years ago in the Journal of the American Chemical Society. The authors are Brandon A. Rodriguez, who received a Ph.D. from Northwestern in 2009, Tobin Marks, a professor of chemistry, and Massimiliano Delferro, a research assistant professor of chemistry. The article was retracted because the authors were unable to reproduce a portion of the data described in the article.

Northwestern University has established processes and procedures for reviewing issues relating to research integrity.  If concerns were to be raised regarding the data that was the subject of the retraction, the University would use those procedures in its review.  Part of those procedures, which follow the steps mandated by the federal government, is that any review remains confidential.

Thank you for your interest in Northwestern.

Best wishes,

Alan Cubbage

ChemBark also received an e-mail response from Peter J. Stang, the Editor-in-Chief of JACS. In addition to asking about the possibility of fabrication and the specific concerns listed above, we asked for his thoughts on Paul Weiss’s recent editorial in ACS Nano. Dr. Stang’s message is also copied below in its entirety, with the exception of one paragraph (that will be communicated in the next post on the blog, a follow-up to this story):

Dear Dr. Bracher:

Thank you for your communication regarding  Bimetallic Effects for Enhanced Polar Comonomer Enchainment Selectivity in Catalytic Ethylene Polymerization   authored by Brandon A. Rodriguez, Massimiliano Delferro, and Tobin J. Marks [http://dx.doi.org/10.1021/ja900257k] that has been retracted {10.1021/ja409590r] from the Journal of the American Chemical Society (JACS).

As you know, concerns with this paper were brought to my attention in late August by an anonymous source.  Apparently this source also communicated the same concerns to you.  I am disappointed that you felt obligated to  not communicate these concerns  to JACS directly  at that time and deferred to your source to do so. Please know that JACS takes all allegations regarding the validity of  data reported in published articles very seriously.

These concerns were shared with the author and a  thorough editorial review of the article and accompanying information was conducted by JACS. The authors have retracted the article – it would not be proper to speculate on some of the questions you have posed.

(Omitted paragraph — to be discussed in next post)

ACS and its Editors adhere to the principle that the observance and practice of high ethical standards is vital to the entire scientific enterprise.  Toward that end, guidelines for a course of conduct by those engaged in the publication of chemical research, specifically, editors, authors, and manuscript reviewers are set forth in ACS Ethical Guidelines to Publication of Chemical Research -http://pubs.acs.org/userimages/ContentEditor/1218054468605/ethics.pdf.

ACS Publications  is a member of the Committee on Publication Ethics (COPE). ACS Editors adhere to  a process established by COPE to review suspicious/fabricated data -http://publicationethics.org/files/u7140/Flowchart%20Fabricated%20B%20revised.pdf

With regard to the editorial in ACS Nano, I believe that it is well-reasoned and articulated. Public speculation and  finger-pointing before all of the facts are gathered, assessed and decided upon is, in my opinion, counterproductive.  A confidential rigorous review of allegations based on the COPE  prescribed process is the best means to determine an appropriate action in response to those allegations.  As you know, outcomes of these reviews may very well have a lasting impact on the researcher(s) involved and we need to let the facts dictate those outcomes.

Thank you for advising that ChemBark will post a story on the retractionhttp://pubs.acs.org/doi/abs/10.1021/ja409590r. I offer my assistance in reviewing a draft and through cc have alerted  Dr. Tobin, the corresponding author.

Peter J. Stang

 

ChemBark could not locate an e-mail address for Dr. Brandon Rodriguez, the first author on the paper, but did send him a message seeking comment through Facebook. We received no reply. We will publish any comments from Dr. Rodriguez as soon as possible after they are received, and we note that he (and everyone) is welcome to post comments in the discussion thread below. We also note that the community owes a debt of gratitude to the chemist who brought these concerns to light in a professional and considerate manner.

So…here we have a case where a suspicious paper from a very high-profile group was allowed to go through the “traditional” private review process espoused by scientists like the editor-in-chief of ACS Nano rather than public review on a blog. ChemBark did nothing to interfere with the investigation or the actions of the editors at JACS.

Is this how the system should work?

 

Editor’s Notes: Neither Dr. Stang nor anyone at Northwestern were allowed to view an early draft of this story. The spectra used to construct the images above were taken from figures in J. Am. Chem. Soc. 2009, 131, 5902−5919 (paper, SI). The next post on ChemBark will deal with how the journal handled a specific aspect of this review.


161 Responses to “Suspicious Data in a JACS Paper from 2009”

  1. Per-Ola Norrby Says:

    It may well be as the system should work, in particular if the behind the scenes actions include identifying the culprit and notifying the current employer. As a PI, I could never have caught something as subtle as this. I’m trusting, and will not change that. I applaud you for noting it, I probably could not have done it as a referee either.

    In my view, retraction is correct, and no more should be done against the PI. But I do hope the confidential process actually gets to the root of this, and identify who did the actual fabrication, because that is intentional fraud, and there I would not want to limit punishment in any way.

  2. Ton Spek Says:

    As a crystallographer (crystal structure validator) my attention was drawn to this post.
    I downloaded the two CIF’s for the two reported crystal structures from the supplementary material.

    The CIF ja900257k_si_003.cif refers to a monoclinic structure and is thus different from the either of the two structures in Table 1.

    The CIF ja900257k_si_002.cif refers to the second structure in Table 1. Interestingly the radiation used is CuKa and not the strange (mistyped?) 0.71703 Angstrom (~ MoKalpha)
    The ORTEP constructed from this CIF (with the same cell dimensions as for the second structure in table 1) has only a distant resemblance with Fig 2. Interestingly, P1 in the CIF has a high ‘thermal motion’ (i.e a large ellipsoid) and a distance to Ni of about 2.04 Angstrom, quite different from the reported values in Table 2 (Thus indicating that ‘P1′ is not Phosphorus in this structure)

  3. Shawn Says:

    At this stage, this looks like the work of an unscrupulous student. It would be difficult for a PI to pick up on this because one almost always expects more their students. If the student doesn’t have a pattern of scientific misconduct, one would not be looking for attempts to deceive.

    I note that the JACS designation of retracted articles has come a long way from the days when the correction notice was the only evidence of a problem. I’d still favor a large “RETRACTION” in red, shadowed in the background of each page, but something on the abstract page is reasonable. Two things are a little troubling in the bigger picture. First, I don’t anticipate Northwestern will issue a press release about the outcomes of their investigation. Not different from what any university would do to avoid to avoid embarrassment. Hence, nothing would be learned. Second, why is Stang is “disappointed that you felt obligated to not communicate these concerns to JACS directly at that time and deferred to your source to do so”? This seems weird. The anonymous source found the problem, so why isn’t it most proper for them to make the report? Is he looking for 2 reports? This just sounds strange. Perhaps this is addressed in the redacted paragraph for the future blog post?

  4. Andre Says:

    I find it interesting how Dr. Stang touts the confidential process of the review in the same letter that he chastises you for not writing JACS about this issue when someone else reported this problem to the journal. I’m sure he would have been displeased with you if you had blogged about it at the time and now he is “disappointed” that you did not take an action on the matter (but only his preferred action, of course).

  5. John Says:

    i do agree the spectra looks odd, but i think is it next to impossible to pick that up during the review process.

  6. Graet Chem Says:

    @John – completely agree as under normal circumstances a reviewer wouldn’t be subjecting the figures to such forensic scrutiny (there simply isn’t the time to devote to such an unpaid activity), whereas someone trying to use this paper as a basis for their own work and not being able to repeat it probably would.

    I think, in terms of the work, the retraction is the right out come, but it is the lessons learned about the processes and procedures of generating such work that need to be transparently reported and that doesn’t appear to be a prospect in this case.

  7. Mike Says:

    @Shawn – I too would first cast my suspicion to the student, but this is where the reporting done by Chembark is important. By not releasing the full story behind this retraction, the editors and university are leaving the incrimination up to everyone’s imaginations. What if my personal biases are wrong and Dr. Rodriguez isn’t to blame for these spectra?

    Additionally, it’s important to know if a researcher has committed scientific misconduct (I certainly don’t want to read papers from someone who fudges spectra). Without giving us the full story, we also miss out on learning from this experience. Was there something about the lab environment that caused this to happen? Are there safe-gaurds that could have been implemented in the lab to prevent this? By allowing the journal and university to hide behind their editorial process, we learn nothing from these cases and are left just as exposed to future bad science.

  8. charles Says:

    The student who caught this did the right thing. Do the synthesis. If it doesn’t work, check the supp. If the data doesn’t match then there is a problem and dig deeper. The wrong thing to do is fly off the handle and blab over social media about how the authors faked their data without any real evidence other than some suspicious looking spectra.

  9. student Says:

    Am I the only one annoyed by the patronizing, father-knows-best tone of these journal editors?

    Paul, I have absolutely no idea how your muck-raking is going to affect your career, but I am not sure it will be in a net negative way. I have been traveling a ton lately to different schools giving talks as part of my tenure tour, and your blog frequently comes up in discussions. Scientists may be cowards as a rule, but they love being involved in a good fight, particularly one they can wage by proxy and at no personal risk to themselves. You might be happy to know you have a lot of friends in low places who think you are doing the right thing. Your next JACS submission may be scrutinized a little more closely at the editorial letter, but I think you will find a lot of scientists who will support you in general and be your advocate (pray you get one of these people on your NSF panel).

  10. JeffB Says:

    While the NMR anomalies would be difficult for a referee to spot, the questions about the crystallography raised by Prof. Spek would have been EASILY found by a dedicated crystallographic referee. It seems in this case that JACS, Northwestern and Prof. Marks could have been saved some embarrassment, since questionable crystallography would likely have triggered a closer examination of the other data.

    I wonder if ACS will ever wake up and realize the importance of crystallographic review. Shouldn’t their flagship journal require the highest standards for all results reported?

  11. anonymous coward Says:

    Charles – if there’s another explanation for any of the journal anomalies discussed on ChemBark other than intentional falsification/intent to falsify, I’m sure a lot of people would be interested. Badly photoshopped chopsticks, Photoshopped spectra, and NMR spectral peaks with no peak widths (Figure 3 above) don’t spontaneously happen. We may not know who manipulated them, but the output leaves no room for concluding that they were not manipulated by someone. Bringing them up in public seems like a reasonable idea – maybe someone will find a different explanation, but at least people will know that there may be something wrong of which they should be aware.

    Professor Stang – Please look at the response to Prof. Sames’s retracted C-H activation papers again and tell me with a straight face that the papers in question were dealt with in a timely and open fashion. If you can do that, I think an Oscar nomination is in order. If you can’t, I suggest that the response advocated by the editors of ACS Nano is insufficient, both in substantive results (making sure it doesn’t happen again, or at least so obviously) and in information value (transparency in dealing with article problems and in informing readers that there is a problem and what it is or might be). If you want respect for your institutions, then they need to treat both authors and readers with respect. “Shut up, we’ll handle it” does not qualify, particularly when the definition of “handle it” is analogous to a cat burying poop as a method of getting rid of it.

  12. Umbisam Says:

    While I think that the decisions of journal editors and their investigations shouldn’t be pressured by online commentary, there’s not much they can do about blogs, just as the Supreme Court can’t stop a blogger from commenting about a case under review before a decision is reached. The number of blogs is likely to increase, and the one topic that brings the most activity is scandal.

  13. The Iron Chemist Says:

    With respect to the “why didn’t you report it to me” portion of Stang’s statement: I think that Stang was implying that you shouldn’t assume that the anonymous source was going to proceed with his/her plan to contact the journal. If the source did back out, for whatever reason, no one at JACS would be the wiser about this. With respect to the Sezen affair, when did people start to contact the journals about those articles? If this was done significantly after the fact, then JACS should not be blamed here.

    And no, I’m not Peter Stang.

  14. eugene Says:

    Hey, I think I met that Rodriguez guy at a conference. He was probably even the one who even told me that using thorium (or some non uranium actinide) as a catalyst in polymer synthesis is justified as basic science research. They were all crowding around that poster from the group, but it wasn’t his project. I think I answered that if that is so, then just do one reaction, instead of optimizing and figuring out that it is ultimately a bad catalyst that can’t compete with 1st row metals and just do chemical reactivity studies instead of publishing essentially a polymerization paper. There is no way that in this solar system anyone is going to use thorium as a polymerization catalyst and it’s not a good use of resources to optimize a radioactive catalyst down to 1% loading.

    I think they stopped talking to me for the rest of the conference after that… Well, maybe they were right, but I still think my argument was right. Anyways, the bottom line is that I met the guy (unlike the previous cases)!

  15. Nick Says:

    From Dr. Stang:
    “Public speculation and finger-pointing before all of the facts are gathered, assessed and decided upon is, in my opinion, counterproductive.”

    I think we will all probably agree that finger pointing and speculation are not good. But what about what bringing facts to the table (the *public* table)? Any self respecting journalist doesn’t just (but should also) report stories to the authorities. They report the facts. Facts and data from published sources can and should be discussed publicly. To say that this is not OK flies in the face of what this country stands for and borders on laughable. Why can’t these editors acknowledge this?

    If some troll wants to throw accusations around, he/she is going to. No editor is going to stop them, EVER. I suspect you will see Paul’s opinions on his blog (e.g., something looks unusual), but you will probably not see him accusing people of anything, either because he is a good journalist and/or because he has some sense of self-preservation. There is very important difference there.

    Also, why was Dr.Stang ‘disappointed’ if Paul knew his source was already reporting the issue to JACS? Does having two people report the same incident to the same editors have any benefit?

  16. Nick Says:

    There’s a reason JACS is know in the crystallography community as the Journal of Awful Crystal Structures!

    I don’t know why ACS and Wiley don’t have dedicated crystallographic referees in the same way that RSC journals do. Especially because people know that RSC have crystallography referees, so if they have ropey structures they’ll avoid RSC journals, and publish in ACS/Wiley/somewhere else instead. This can’t be a good thing for these publishers.

  17. eugene Says:

    Anyways, there is no way Tobin Marks could have caught this, especially since he runs a mega group of 40 people and lots of grants with dedicated lieutenants and probably doesn’t know a lot of what’s going on. Probably a regular PhD student could go only with a few formal meetings with the ‘boss’ before graduation. So he’s definitely innocent in this whole affair because at that level you just manage stuff and designate who writes grants and don’t really read the SI over carefully or talk to the student a lot.

    P.S. Also you don’t really read blogs since you don’t have time, so I’m safe in assuming he won’t come after me with a sawed-off shotgun (I read a Sherlock Holmes mystery where that was the murder weapon)

    P.P.S. Did I mention that I think mega-groups are bad for science? I must have forgot to say that…

  18. Olivier Roubeau Says:

    For sure there are (and were at the time of submission of this work) means to spot incorrect crystallography. In a publication such as JACS that should be systematically checked.
    In addition to the comment of Nick, I would add that a structure (the triclinic one) with a goodness of fit above 3 and wR2 above 0.6 (and not commenting any possible reason) should not even require a dedicated crystallographic referee. Of course these figures are not those reported in Table 1, but a mere fast read-through of the cif files would have indicated there could be a problem with work.

  19. Greg Says:

    It kind of looks like these are spectra that were scanned in from a paper copy. This would leave an imperfect white/grey background with smudges and marks from imperfections in the paper (and somewhat explain the terrible resolution). Maybe someone tried to clean up the image but used mspaint and absolutely botched the job?

    For example, S21 is really poor resolution but has x-axis labels that were clearly added in later. They have better resolution then everything else and do not all line up with the major tick marks above them. The spectrum itself looks like someone tried to use the paint brush tool to make the background perfectly white instead of the off white color that was scanned in. Some of the peaks look clipped and this might be from getting a little wild when recoloring the white background.

    I can’t think of any explanation for the mess at the top of S22 though. The peaks look like they were added in by hand. My favorite part of this figure is where the big peak from the bottom spectrum overlaps with the baseline from the top one. It looks like this top spectrum was pasted in last when constructing this figure and it deleted part of the peak stretching up from the bottom. The solution, apparently, was just to use the paintbrush tool to draw a black line connecting the top and bottom of this peak.

  20. Rhenium Says:

    Northwestern: This is embarrassing and therefore confidential. Please kindly stop drawing attention to it. You won’t be hearing from Tobin Marks.

    Stang: This is embarrassing and since I should be receiving awards right now instead of dealing with this I’ll consider it your fault, not mine, my sub-editors, or the reviewers. Please stop embarrassing ACS, because I’ll make certain that you never get NSF funding, and I hope you like publishing in Elsevier.

  21. Duvane Says:

    The “disappointment” statement seemed very strange to me on first read; that’s fairly strong language considering the tone of the letter. It seems that there are two potential motivations for it.

    One, as The Iron Chemist pointed out, is simply that the editor is worried that had the anonymous source chickened out or been hit by a bus that this might never have seen the light of day (which is not a totally unreasonable worry). The obvious counter to that is that the source did in fact come forward to JACS, and so there is no way to know whether Chembark would have eventually contacted JACS itself had it become apparent that the source had not. If that was the motivation, a more appropriate statement (in my opinion) would have been to express that concern directly, but people don’t always think carefully about how every little thing they say might be parsed, so I could still see this being an explanation.

    Two, is that the disappointment is (obliquely) in reference to the editor’s anticipation of exactly what Chembark has done: essentially to allow the procedure to proceed to completion as it would have ten years ago, and then to analyze it after the fact (quite nicely leaving comparisons to recent complaints about the effect of the blogosphere on the process as an exercise for the reader). This reading makes even more sense in light of the offer of “assistance in reviewing a draft” of the blog post. If this is the case, then the tone makes more sense; its about as strong a statement as could be included without directly stating the objection. This is not to say that the editor’s reaction is necessarily nefarious; it may have more to do with irritation at essentially being used as a guinea pig (which, if unreasonable, is understandable). But it certainly could be a “mind your business” attitude.

  22. Kenrod Says:

    Stang “alerted Dr. Tobin”!? Sounds like he’s distancing himself from this already.

  23. anon Says:

    ironic: http://dx.doi.org/10.1007/s11422-010-9268-4

    “this public forum of review and dialogue does not exist in the world of science journals, which favor a more ‘cloak-and-dagger’ process”

  24. Johan Hoogboom Says:

    It is extremely worrying that academic institutes, editors and companies always try to hide behind privacy concerns in these matters, even going so far as lawyering up. A ridiculous reason, as science will always benefit most from complete honesty and transparent discussion. Whether honest mistake, deliberate fraud or something in between, these issues should be dealt with openly.

  25. isotopeeffect Says:

    Maybe there is more back story to the interactions between Stang and ChemBark, but I’m not seeing a *whole* lot wrong with his e-mail. It’s a conservative response, as expected for the editor of the flagship ACS journal, and it expresses a general distaste for the blog approach to peer science. Don’t get me wrong, I love this blog and have read it for years, but I think I can understand where he’s coming from. Peer reviewing necessarily has a limited number of people involved; if every chemist in the country had to approve a paper before it was published nothing would get published (most of the time it’s hard to get three people to agree). I can understand the desire to ensure there are measures of control on the official process (which, of course, raises the question: *Who* is in control?). That said, this blog and the (new?) process it represents is also important for too many reasons to enumerate in a comment thread. I’m very pleased that the discussion of the data quality in this paper occurred *after* the paper was retracted, since it meant that the authors had a chance to see if the original data was reproducible.

    However, I concur with the sentiment that it’s unnerving that the given reason for the retraction was irreproducibility. After looking at the paper and SI, I’m not surprised the data wasn’t reproducible, since I doubt any spectrometer is capable of generating such spectra as the ones illustrated. Also, is the baseline composed of the same repeating segment in Fig S22??

  26. Chao Says:

    I saw this retraction yesterday… interested in follow-up story. Again, hopefully JACS will not come up with some editorial similar to ACS Nano.

  27. XrayJay Says:

    On RSC and ropey crystal structures – If you SAY in the caption/.cif etc something along the lines of “while the diffraction data was of low quality precluding a detailed description of the metrical parameters, the identity of amazing compound X is clear and in agreement with spectroscopic/EA/other data etc.” the RSC crystallographic referees seem perfectly happy. Its trying to pass of your shat structure as high quality that bothers them…

  28. Dr. VanNostrand Says:

    Total speculation, but it sounds to me like the anonymous tip was that the spectra were faked, there was some sort of discussion between the prof. and the journal, and there was plenty of time to check the results. Because the results could not be duplicated, he prof. would be able to pass it off as a retraction due to unreproducable data” instead of falsified NMRs.

  29. isotopeeffect Says:

    It’s also worth pointing out there is an addition/correction on this article in 2010 on one of the x-ray structures, so there was a missed opportunity for greater scrutiny back then.

  30. Chris Fellows Says:

    I disagree that the dodginess in the spectra is “next to impossible to pick up in the review process”. I can’t imagine *anyone* looking at S19 and thinking it was plausible. It is clear that reviewers just did not bother to look at the supplementary material.

  31. Bill Says:

    Journals are a corrupt cartel proppping up an entrenched elite that utilise some of the worst labour conditions to funnel ever more resources to a smaller group of people. The actions of the editors should not suprise anyone, they are trying to preserve their position in the elite (the fact that journal editors are also scientists who compete for grants on the basis of publications is a clear conflict of interest that everyone overlooks) by silencing the masses. But the Academy as it exists is literally and metaphorically “the walking dead”, its time is up it just doesn’t realize it.

  32. Synthetic Dave Says:

    Well, I can definitely see why problems were found. I have no idea what to make of Figure 3. If I were the PI, I’d want to see the fid file. Also, if you zoom in (like 350x) on figure S19, you’ll see there’s no break in the baseline, and there’s a very small gap between the top of the baseline and the peak that is being claimed at approx. -4 ppm. That would definitely have me wanting the original fid files to examine.

    Another glaring problem is that the NMR spectra are low resolution images. This is consistent with exporting the data as a pdf or jpeg from Bruker Topspin, or scanning the spectra at low resolution (under 600 dpi). What’s weird is instead of just printing and scanning the spectra at high resolution, whoever put it in the SI took the time to make it look like high resolution at first glance, although upon close inspection it is definitely still low resolution. This kind of thing makes superimposed spectra such as S21, suspicious. I don’t think that it was a copy-paste job, but rather an attempt at making a low resolution image look high resolution. What’s even more confusing is that all of the GPC traces are high-resolution scans.

  33. Renee Says:

    The retraction notice says they are unable to reproduce the ethylene/acrylate polymerizations. So, what exactly was being analyzed in the samples reported in Table 7 of the paper, that are supposed to be copolymers? Were they just simple ethylene homopolymer, without the acrylate or methacrylate monomer incorporated? The is quite a lot of explanation in the paper about why they thought they were getting incorporation of the acrylate and methacrylate monomers.
    The paper also reports monomer reactivity ratios – are these also not valid?
    To my polymer chemist eyes, there GPC chromatograms look somewhat odd. Figure S13 shows the GPC of the ethylene/methyl methacrylate copolymer, which elutes between 15 and 24 minutes. But the GPC of the same polymer in the paper (Fig. 8) shows the elution times between 36-40 minutes. Figure 8 also has much less detail that the GPCs shown in the supplemental information.
    I think the scientific community deserves more information about what exactly has been found to be irreproducible.

  34. Special Guest Lecturer Says:

    I am aware of researchers who have been discussing this paper for a while now, and it definitely caught their attention because the spectra were obviously faked. I have no idea if these people were the ones who alerted JACS and/or Paul.

    I am disappointed that JACS allowed the retraction notice to state that the chemistry was irreproducible rather than manipulated. “Irreproducible” implies that some unknown and difficult-to-identify variable in the experiment has a dramatic impact on the outcome. It does not mean “We can’t reproduce these results because they were made up in the first place.”

  35. GiantOctopus Says:

    I feel bad for Prof. Tobin Marks and his group as they do outstanding research and some rotten apple has spoiled a small portion of it. That said, if I were to be offered a similar research output to Prof. Marks over the next 30 years of my career for the price of one retraction, I’d bite your hand off.

    I also think that Prof. Marks has handled ChemBark’s e-mail superbly, by passing it to the dedicated communications officer of the university, rather than getting involved in correspondence that could impact negatively. Whilst ChemBark’s posts regarding data integrity and scientific fraud are justified, it must be quite difficult and frustrating for senior scientists with grant applications to write, colleagues to advise, articles to review and teaching etc. to devote time to a blog which is going to focus the attention of the worldwide chemistry community on the negative aspect. With all due respect, replying to ChemBark is probably quite low on Prof. Marks list of priorities, and I sense that some of the less-than-welcoming responses ChemBark receives are borne out of frustration as another thing lands on the in-tray. Nothing in life is ever as simple as we wish it were.

    The people you are e-mailing for comment can no doubt see the issues perfectly well and so can the journals themselves, they just probably don’t want to be splashed all over the tabloid press. Also, I don’t think the older generation of scientists are really as comfortable with the intrusive nature of the internet (which probably became mainstream when they were in their 40’s?) when compared to those of us who grew up with it and accept it as part of society, so maybe people should bear that in mind when criticizing them. Some people are extroverts and enjoy attention, some are introverted and want to get on with their work with the minimum of fuss. It is unfortunate that a career in science now seems to be more about publicity rather than making genuine progress, and it increasingly seems to be attracting these extroverted types that write a lot and talk a lot and make a few valid criticisms here and there but don’t get the primary data that tests hypotheses or creates something new or improved i.e the real work. I’m not referring to the Editor of this blog or indeed blogs in general, rather a certain type of attitude among some PhD students and some post-docs that is starting to appear among laboratories and will eventually spill over into the literature if we are not careful. We do research to benefit society, not so we can say how much smarter we are than our peers.

    Anyhow, what lesson can we learn from this and how can PI’s prevent this sort of thing happening? My former supervisor explained to me that results were not published until they had been reproduced by other persons in the group. That meant two or more people had to be able to synthesize the same compound, or generate identical reaction kinetics that fitted on the same line. Furthermore, well-established past experiments were repeated as an internal measure, to check that things were operating correctly and that new students could get good at generating results. When one person made the catalyst that worked perfectly and the other made the same catalyst that gave slower kinetics, the results were not accepted until batch-to-batch reproducibility was achieved. I can think of one very high profile case where following these guidelines could have saved an awful lot of trouble. In any case, the result was fewer publications, but the higher data quality had benefited the research group long-term as past results could be re-visited as new discoveries were made. In some cases, 10 year old data had ended up in high impact journals when a mathematical model explaining the trends was developed.

    When I see paper’s published with a large number of authors, I would like to assume it is because they have ALL done some experimental work and checked each other’s results, possibly over a long period of time. You never know who might repeat your experiments in future because if they are genuinely worth doing they will be performed again.

  36. sdgaerh Says:

    This technology is patented under US Patent # 8,236,907. How serious is it to submit fraudulent data as part of a patent application? Potential jail time for Tobin Marks?

  37. eugene Says:

    Also, even though they say that the Organometallics paper results are reproducible, I find the spectra S4-S5-S6 to be basically the same except there is a small peak in S4. The noise also appears to be the same. pubs.acs.org/doi/suppl/10.1021/om800208f/suppl_file/om800208f-file001.pdf
    It’s not hard for me to imagine that ethylene will polymerize with ‘mononuclear’ ‘dinuclear’ and whathaveyou, but really if the work was reproduced then we could hope to get a correction and three different spectra from whoever was the poor sap doing the reproduction.

  38. Hoya Says:

    The anomalies in the spectra are definitely not difficult to spot – for some of the spectra you don’t even have to look closely to see the, uh, interesting artifects without magnification (those of you who haven’t read past Figure 3 in the original paper really shouldnt comment on how “difficult” they are to spot). It’s interesting that it was not spotted by the referees and not least people who supposedly know the work best before submission.

    On that note, I find it disturbing that, after all the recent reports of interesting results in the literature, many of us still immeidately direct the blame to the students involved and/or make baseless claims that the PI is innocent. Even if the PI has no part in the production of interesting results, having missed anomalies as glaring as those in this particular case is not an excuse. A PI’s name holds a great amount of responsibilities (to the research group, the insitution, funding bodies… etc.) and is meant to be an indication of quality and integrity of the work being presented; if something as simple as this, for example, was not picked up then she/he should be punished for being careless and irresponsible at the very least.

    This is what we all should do when we make mistakes: 1. admit to it; 2. apologise to the people affected; 3. deal with it *personally* with honour and integrity.

    @Special Guest Lecturer:
    Completely agree! I also find the reason (irreproducible results) listed by JACS is very tale-telling. Suppose that they really couldn’t repduce the results and that the PI had no idea of the funny results, any good scientist would try to figure out why it’s not reproducible as well as chase up the person who originally did the work. JACS… well, I suppose one can’t really say anything about the statement in this case because they may or may not have known that it’s fishy. One way or the other, they don’t seem to be a nice bunch these days; I mean… http://chemjobber.blogspot.de/2010/10/jacs-rejections-just-got-tougher.html.

    @GiantOctopus:
    Not least you are unreservedly defending the PI without any evidence for such painfully obvious anomalies in the paper in question; the fact that you are okay with having one retraction like this in return for a sizable publication record worries me.

    All that you have said about how the PI in question responded in this case were just excuses that you made up for him. Has it not occured to you that he may be running away from the community? If he knows his science then he should already have known what was going on when he asked JACS to retract the paper (if he didn’t then he’s just simply a bad scientist) – so why was it reported as reproducibility issues?

  39. Graet Chem Says:

    @Hoya – it’s easy to say that these things are easy to spot when you’ve been tuned into them by someone else’s commentary. In reality, I think it would be easy to dismiss the ‘artifects’ (sic) as minor reprographic errors rather than data manipulation, and I did read past Fig. 3, especially if you were reviewing a printed copy and couldn’t magnify the spectra (for example on the plane to a conference). We also don’t know what revisions the reviews insisted on either, who knows what the initial submitted manuscript looked like? Perhaps there were more glaring errors that drew the attention of the reviewers as requiring amendment? In the end, deliberate attempts to deceive sometimes work, this is a fact of life.

  40. Graet Chem Says:

    @GiantOctopus

    Prof. Marks is responsible for the output of his research group and so if his checks and balances aren’t effective then he will reap what he sows. I’m afraid I have little sympathy for the poor big-wig professor who has to contend with the fallout from a retraction of his work published in a major journal. Prof. Marks should see this as it is: A serious issue for his research group and one that does need to be prioritised for the sake of his reputation.

    However, I agree with your comments regarding double-checking syntheses. Sounds like an eminently sensible and useful safety net. It also ensures that people don’t get their names on papers for making the tea or simply being in the lab at the same time as the first author. This thinking really goes beyond just organic chemistry, all experimental scientists should be demanding to see evidence of reproducibility from their students. I learnt very quickly that if I did not turn up to a meeting with my supervisor with a set of raw data and spectra from a triplicate repetition of a particular experiment I was wasting my time.

  41. Allison (@DrStelling) Says:

    GO: “It is unfortunate that a career in science now seems to be more about publicity rather than making genuine progress, and it increasingly seems to be attracting these extroverted types that write a lot and talk a lot and make a few valid criticisms here and there but don’t get the primary data that tests hypotheses or creates something new or improved i.e the real work. I’m not referring to the Editor of this blog or indeed blogs in general, rather a certain type of attitude among some PhD students and some post-docs that is starting to appear among laboratories and will eventually spill over into the literature if we are not careful. We do research to benefit society, not so we can say how much smarter we are than our peers.”

    I actually used the term “overly verbose” when describing the biomedical literature in general once in one of my referee responses. I’ve heard another term used for this which is not as nice and I shall repeat it.

    Part of the reason I took upper division physical chemistry courses in college rather than cell biology was due to the smaller class sizes…….

  42. Umbisam Says:

    There has been quite a lot of talk of fraud in research lately, ranging from blogs to the Economist. I hate to say this but maybe the most utilitarian thing to do is accept it as par for the course and move on. Is it better to focus time and energy on on fraudulent papers, or focus on doing original research? With photoshopping software, etc. it will be difficult to catch all the cheats. It’s almost as if a rare shark has been caught and we’re poking it with sharp objects, delight in its twitches. There will be cheaters, but if I’m spending all my energy on the witch hunt then I’m not focusing on developing sustainable solutions to current challenges. I think this is why the journal editors get annoyed; they know there are kinks in the system, but is it worth investing so much time in fixing it, or is it better to move on and do great science. One good thing that might come out of this is hopefully people will judge scientists by the quality of their work and not the number of publications. Of course, you could argue that the bigwigs don’t have time for this, but the bloggers do, so let the bloggers take the burden upon themselves, but don’t expect a Full Professor or Journal Editor to post or even reply to emails. Just some thoughts.

  43. SGPrilliman Says:

    This is one of those cases in which double blind review might have been helpful. If you know Prof. Marks’ work you may, as a reviewer, not give it quite the same diligence if it comes from an unknown source.

    This also, yet again, raises the point that reviewers have to be more careful AND more suspicious in their reviews for both fraud and plagiarism. I’ve taken to using Google to search for random phrases, and caught someone plaguing a web source in one instance.

  44. grad student Says:

    Umbisam–“is it worth investing so much time in fixing it, or is it better to move on and do great science.”

    A large part of the outrage is that these papers are not published in a vacuum, so it is difficult for many people to just “move on.” The results from fraudulent work like this may lead to frustrated grad students who waste weeks or months trying to reproduce it, PIs who think their students are incompetent as a result, time wasted by other groups pursuing projects using the fraudulent results as a design element, denial of grant funding to PIs whose proposals conflicts with the data presented here, etc.

  45. KeepItSimple Says:

    After reading the article posted by anon, I’m thinking this retracted paper was a call for revolution by Rodriguez, but it didn’t come into play until years later. A lot of the points that he raised in that article seem to directly correlate with what we are seeing here.

    Perhaps this is all an intricate performance piece on the ills of peer review in chemistry.

  46. Umbisam Says:

    @grad student – true, but what if we just assume some percent (let’s say 40%) might be problematic, so as a rule we won’t be surprised when something doesn’t work. If everybody took this approach, then people’s grants wouldn’t get denied when proposals conflict with data, grad students wouldn’t get fired, etc. In Biotech the general rule is that about half the published work is not reliable. Don’t spend weeks or months trying to reproduce something (unless it’s truly frontieir). Move on and find something that actually works. For me the biggest drag is that if not caught, things like nanochopsticks could trump my work and some goof gets the coveted position. But I’m not seeing any dependable solution to stop this non-sense. I’m not against outing people, just questioning whether time spent getting ticked off with clowns is worth more than time spent reading papers by more “reliable” scientists. Maybe some high court should be appointed who are paid to do nothing but read what has been published and look for badly pasted nanotubes or spectra with odd noise patterns. It would kind of be like how restaurants claiming to be kosher get inspected. Certain journals could get the seal of approval. Journals that are generally considered to be lousy would not be inspected. This would include many of the scam open access journals that are popping up.

  47. eugene Says:

    Umbisam – I went to a lecture by a Swiss social scientist on this where they did games involving Swiss Francs and trust. The highest rewards were available to the group if nobody cheated and invested money for the next round to get the highest return, but a cheater could make twice as much money by stealing investments and the rest would get none. The system would then break down as nobody invested anymore. In any given society, about 10-15% are altruists who will work for the common good will always invest, 10-20% will take advantage and game the system. The rest can be swayed either way to cheat, or to be altruists if they can punish the cheaters. If there is no credible punishment system (the others could penalize the cheater in the next round by taking their money away), then the game breaks down and no one makes any money. Societies that utilize the punishment system in that game are Swiss, Japan, USA and most of Europe. Those where the punishment of cheaters was available but still failed because no one trusted each other anyways, are Arab states and Greece.

    If 40% of research is problematic, it means that we are Greece here in a big way. Why the hell do I want to keep playing this stupid chemistry game if so many other are playing me for a sucker? You think I like being a frier? I’m either going to join them or take them all down, and I prefer the latter option. Glad I’m not in biotech academia research if this is the case, as it sounds like a loser’s or cheater’s game. Good thing I think your entire argument of burying your head because of too much fraud(sloppy science) in the sand is bunk, or I would have quit the field a lot sooner. If you don’t spend energy and time at rooting out cheaters, then the worth of your own clean work will be significantly diminished. The utilitarian thing to do is double down on the effort to root this out. In the friendliest possible terms, what you are suggesting is a perversion of how science should be done. I’m shocked to read things like “If everybody took this approach, then people’s grants wouldn’t get denied when proposals conflict with data, grad students wouldn’t get fired, etc.” despite knowing that you have good intentions.

    Obviously you’re trying to engage a bigger, philosophical issue here and we’re not talking about the Rodriguez paper anymore. But I think the ideas you present are a disaster.

    (Still, what if there are problems in my thesis that I forgot about? Again, Paul, don’t read my thesis due to it being extremely boring)

  48. GiantOctopus Says:

    Hi again all, hope everyone is having a productive day. Seems my last post generated a few responses, as I thought it might.

    Firstly @DrStelling : You forgot to repeat your other phrase! Maybe you meant (with justification) that I am overly verbose? In any case I was referring to a lot of papers I see published that seem to consist of discussing other people’s results and not containing any new real / novel data or development. Yes, we need articles to clean up literature, clarify points, highlight results and provide perspective…but if that is the ONLY type of article that a researcher publishes (and I can think of a couple of groups that do it) and they publish 100’s of them, and none of their own experiments, ideas or data appear, then I think there is a bit of a problem. It is doubly problematic when these articles are criticisms of the work of others, and triply problematic when these criticisms are not supported by any meaningful data and can be answered with a a few well-performed experiments. Reviews and highlight articles are quite prestigious and should be earned as a reward for good quality experimental work.

    Secondly @ GraetChem: Yes, Prof. Marks is the PI, the buck stops with him. His group should have had a structure in place to ensure that people are not writing nonsense and are generating reliable results regardless of whether or not someone is going to commit research fraud. It was poor management on part of the PI, but perhaps a post-doc could have realized that the students were not being adequately supervised by him and stepped up to the plate. Generally, good post-docs are the ones that do the day-to-day things like interact with the PhD students, advising them how to to do good science, how perform the most relevant experiments, how to write articles and will check spectra and results on behalf of the PI, so that the PI only needs to see the major progress. They work as part of a team, and they are able to work out that they should be doing these sorts of things if the PI can’t, without the PI having to ask them. In my opinion, rubbish post-docs are the ones that treat students like they are personal slaves or put their head down and focus only on their one area of project so that they end up caring only for themselves rather than the team overall. Such an attitude is what is being developed by these new PhD students and post-docs that disappoints me. Maybe people will disagree with me, but isn’t training other people part of a post-doc’s remit? It is good preparation for running a research group, afterall.

    Thirdly @Hoya: I didn’t “unreservedly defend” Prof. Marks without evidence. I reservedly defended Prof. Marks. My evidence? Prof. Marks works at Northeastern University, a top university, has been full Professor for 35 years, has won numerous awards for his work, has published over 1000 papers (including very high profile journals) and written lots of patents, etc. etc. PEOPLE THAT CAN OBTAIN THAT SORT OF CV GENERALLY TEND TO KNOW WHAT THEY ARE DOING!!! Now there is no doubt that something has gone very badly wrong here, but that is once in over 40 years. Of particular importance to this argument is that Prof. Marks has been recognized throughout his career, and is not just a flash-in-the-pan one hit wonder i.e runs a research group that consistently shows REPEATED SUCCESS. That does not happen by accident! Wouldn’t you like to run a research group that shows repeated success and develops something new? I would. Now, this retraction business takes him and his group into new territory and will be stuck on his record, but the key points are (a) I judge Prof. Marks on his entire work and specifically his best papers, and not one retraction or his weakest papers (we all write something that is not so good from time to time). Does JACS 2009 invalidate his work from the 80s, 90s, 2000s ? Now, if we were to re-investigate all this work and find similar problems there might be a different story, but people don’t continually fund research that does not work to a certain extent. (b) As yet there is still no established formal protocol for dealing with this sort of thing, and it seems to be done on a case by case basis. Why should anyone engage with ChemBark or anyone else who is not directly involved? They are not answerable to “the community” but to their employers. Besides, “the community” responds via funding agencies ceasing to provide money for research and journals not sending out manuscripts for review. The only parties this concerns that I can see are the University, JACS, the funding agency, any patent agency and Prof. Marks research group. Based on the past 40 odd yearssm of professionalism, I strongly suspect Marks will be able to work out how to respond appropriately to this new situation.

    There is one word you use that I don’t like – “punish”. Is that what interests you, punishing people?

    Also, you criticize me (perhaps understandably) for saying I would accept a retraction for “a large publication record”. As I stated in my earlier post, I try to do science to develop something that benefits society. Turning your criticism on its head, I find it very worrying that your top priority is a “publication record” and not “potential solution to world’s energy, water and health issues”.

    In my experience, people who bitch the most tend to publish the papers that interest me the least, if they manage to publish anything at all.

  49. anon Says:

    i agree with GiantOctopus. Sure a PI is responsibly for what’s published under their name, but by no means is being taken in by a student’s fraud an equal offense to knowingly committing that fraud. At some point a PI has to trust that he’s not being lied to by his students rather than going over experimental data with a fine-toothed comb. You can always do a better job of faking data. The student could have easily done a better photoshop job on those spectra (Assuming that this was indeed fraud, which seems somewhat reasonable), or, coming from a polymer lab, he could have easily obtained a sample of copolymer produced by another system and used it for his analysis.

    What can we learn from this? I’m not exactly sure. NU already requires their students to perform a multisession training course on academic honesty. You can do your best to instill a culture of academic honesty, but ultimately you will have to accept the fact that there will be one or two rotten apples that slip through with copied exam answers or phony data. Grad school can be very stressful, research can be frustrating, and students with some character flaw will respond to that stress in the wrong way.

    As for the criticism of large, “super labs” – there are definite tradeoffs, but an outsider from a smaller group shouldn’t just assume the worst. From what i understand Prof. Marks is much more accessible to his students than “a few meetings before graduation”. Even so, I have heard that in his later career, R. B. Woodward hardly spoke to some of his students a few times between their joining the lab and their graduation, yet they performed excellent science during their time under him and went on to become good organic chemists.

    Finally, even though i don’t feel particularly sympathetic towards the student, I do think innocence should be presumed until guilt is proven. Has “mens rea” really been demonstrated here? maybe he spilled coffee on the original spectra and his dog ate the USB stick with the electronic copy

  50. eugene Says:

    “Does JACS 2009 invalidate his work from the 80s, 90s, 2000s”

    Funny you should mention that. See the article here: http://www.nature.com/srep/2013/131106/srep03146/full/srep03146.html

    Apparently the answer is yes, it does invalidate earlier work by 7%. Of course, I doubt that will be the case for Marks, because the group is so large and has separate divisions with research professors who are glorified postdocs directing troops, that people will separate the PI and others from this one person, so I’m guessing it will probably be 1%, if at all. It will correctly be seen as an isolated case. Now if another isolated case shows up…

  51. eugene Says:

    “As for the criticism of large, “super labs” – there are definite tradeoffs, but an outsider from a smaller group shouldn’t just assume the worst. From what i understand Prof. Marks is much more accessible to his students than “a few meetings before graduation”. ”

    That’s just the least of it. I’ve got plenty of more reasons where that one came from, but maybe it shouldn’t be discussed on this thread.

  52. Allison (@DrStelling) Says:

    “Firstly @DrStelling : You forgot to repeat your other phrase! Maybe you meant (with justification) that I am overly verbose? ”

    I really should not post prior to coffee; sorry! Not you in particular (although I find my Editors like it when I have more paragraph breaks); the biomedical literature in general could do with some trimming and re-organizing. I enjoy reading all this brain tumor literature, but a lot of the stuff I read is not what I would call “well written”. I’ve made it a hobby to spot typos as well, though these are present in much of the literature.

    I also meant to say “*not* nice term”. It involved a scatological reference. Ahhh physicists; they can have a way with language.

    “Reviews and highlight articles are quite prestigious and should be earned as a reward for good quality experimental work.”

    What I think would be neat, is if we gave out “Moderator-ships” to scientists like this to handle post publication commentary. It could be a sign of support from your scientific community that could be taken as “service” when committees are deciding on promotion, for example. I’d of course want an Editor to see my MS before I show the whole community, but it’d be nice to have one on the other side of things as well. Maybe a few could even be full time positions?

  53. Umbisam Says:

    @eugene: but you play the game anyways, regardless of the size of the problem (which at this point we don’t know). And I believe it was you who once said that in so many years all of the research currently done will be useless, so what does it matter if you are playing alongside hustlers, as long as you try to do something meaningful yourself. But if you do succeed in exposing real culprits, more power to you. I’m not opposed to punishment, the question is more about how much time to spend trying to dole out punishment. I’m all for punishment to those who cut to the front of the line, but I’m not really sure how much punishment some of these characters get when caught. The original villain still gets grants and is still tenured. I suppose those who want to chase the robbers should do that (as long as it doesn’t turn into a witch hunt, which in general I don’t think it has yet), and those who want to focus on research and get the summary of the latest scandal on C&EN. I’ve always found these stories fascinating because I’ve always had serious doubts about many things I see, but not enough evidence to convict (admittedly I’m probably overly skeptical). What I’ve written is not a perversion, it’s the way most successful scientists are currently operating. Focusing on doing great research is not burying one’s head in the sand, which I think goes along with Weiss’ editorial. I suppose at this point I’m obligated take a break from all this to do just that.

  54. GiantOctopus Says:

    Interesting discussion. I’ll add more paragraph breaks.

    @DrStelling: I think scientists in general have a very hard time explaining their thoughts and ideas in ways that other scientists (and that includes experts in the field) can understand. Don’t forget that many articles are written in an authors second or third language.

    @Eugene: I don’t think number of citations is a good way to judge the quality of an article. Also, I don’t think a drop in citation number means that an experiment is not reproducible. Getting lots of citations should not be the primary goal of the scientist (although the world is certainly heading that direction).

  55. Allison (@DrStelling) Says:

    “Don’t forget that many articles are written in an authors second or third language.”

    I actually insist that publications with my East German collaborators go through me first. (I was over there for 4 years, just got back, I’m much more aware of multinational issues in science now!). The older scientists learned Russian in school over there, so English is their 3rd/4th language.

    I’ve gotten pretty good at guessing which country the author is from based on the grammar :). (“Good” German and Russian = very precise, long sentences. “Good” English = very short sentences, and short paragraphs. Unless you’re the next James Joyce, in which case you can get away with it. These differences could cause confusion in the older scientists, when the Nature editors says “needs to re-written for English”…..)

    “I’ll add more paragraph breaks.”

    Vielen Dank! Remember, Editors read for most of their day- anything to shorten or break up the ideas does help!

  56. Renee Says:

    I’m a polymer chemist who will limit my comments to two paragraphs in the article in question, starting at the bottom of the right column on p.5913. This section is a discussion of the monomer reactivity ratios of ethylene and methyl methacrylate, both from this study and from published results of free radical polymerizations. These ratios tell how likely a monomer is to homopolymerize with itself (large value of r, greater than 1) versus how likely a monomer is to copolymerize with a different monomer (small value of r, less than 1). The values of r1 and r2 for these two monomers, from the present study using the nickel catalyst, are reported as r1 = 0.34 and r2 = 12.2, as compared to the published free radical copolymerization values of r1= 0.2 and r2 = 17.26. The section concludes with the statement “These results argue strongly against significant radical pathway contributions to product formation.”

    The results do no such thing. The values of 0.34 vs. 0.2, and likewise 12.2 vs 17.26 are not significantly different, when one is discussing the copolymerization rates between two monomers.

    What is more, there are no data present in the paper as to how the values of 0.34 and 12.2 were arrived at. There are no tables of data, and no figures, either in the article or in the supplemental information. There is simply a statement that an NMR assay was run to generate these reactivity ratios.

    How did the final concluding statement make it into this section of the paper, in a paper that is about copolymerization using a novel catalyst? How did the two professors in charge of this work, and the reviewers of this article, allow this statement to pass, as well as not noticing any supporting experimental information?

    This isn’t about a harried post-doc not paying attention to a graduate student – this is about drawing defensible conclusions from the purported data. No wonder there have been questions for some time about this work.

  57. GiantOctopus Says:

    @Renee: Since you are a polymer chemist and thus probably know something about GPC, what do you make of the data in Figure S10 in the supporting info?

  58. Chemist Says:

    You said you tried to message B. R. via Facebook. Why don’t you just write to him in Linkedin? In Facebook when someone that is not on your friends list writes you a message it goes into the “other” section within the messages section and the user does not get notified of anything that goes into that “other” section.

    By the way, based on what has already been shown, data have clearly been manipulated and made up. I hope when the university concludes their review they revoke B. R.’s Ph.D. degree.

  59. Chemist Says:

    Here is B. R.’s linkedin page:

    http://www.linkedin.com/pub/brandon-rodriguez/18/570/953

  60. eugene Says:

    @umbisam

    “What I’ve written is not a perversion, it’s the way most successful scientists are currently operating.”

    While I try to go about doing my research and I don’t look through articles to find suspicious images (like on the Science Fraud or Retraction Watch blogs), I don’t operate on the assumption that 40% of the colleagues I meet at meetings are doing sub-standard science or that I can’t trust 40% or results in a top journal.

    “And I believe it was you who once said that in so many years all of the research currently done will be useless, so what does it matter if you are playing alongside hustlers, as long as you try to do something meaningful yourself.”

    I do remember saying something about it not mattering 100 years from now. Just look at Jackass articles from 100 years ago. But I don’t remember the last part; maybe it was paraphrased wrong? Well, if not then I want to take it back or clarify things. While 100 years from now people will laugh at the crude techniques you used and all your great ‘discoveries’, I’m alive now and not in the future. It is not in my interest to see my field of research devalued in the now as I have money to earn to pay for offspring so that my great grandchildren can someday laugh at my work like I do now with my own great grandfather who volunteered to fight for the Red Army during the revolution and died of tuberculosis. Hah! If only he knew how that would all turn out…

    “Focusing on doing great research is not burying one’s head in the sand, which I think goes along with Weiss’ editorial. I suppose at this point I’m obligated take a break from all this to do just that.”

    If you don’t have someone to police the literature and punish fraud, then your own ‘great research’ gets devalued. The journals are not an effective police presence yet. Anyways, I’m writing an article and I procrastinate terribly, so I’ll be around the blogs for the next little while. Have a good one.

  61. Wow Chem Says:

    @GO: How do you mean it that it’s just 1:1000 of Prof Tobin’s papers are bad, so it’s okay….
    Here is the fallout
    1) B.R. has 4 other Tobin papers and a patent…which others have faked data.
    2) I doubt he did all the primary data since it’s so diverse and so much work (though he could of), what about reviewing the res. asst profs papers
    3) the fact is B.R. Lied and has a 100k/yr job now. Some non-lying schmuck didn’t get that job. Someone was really hurt by this fraud. Tobin got federal funds and raises. Easily in hundreds of thousands in damage via this fraud.
    4) if you’re too busy as a researcher to view your own labs data maybe you should not be on a paper or listed as a researcher. That’s like saying I’m such an important CEO I shouldn’t be held responsible if My company lies to beat Wall Street valuation every so often. Once and your out as CEO

  62. anonymous Says:

    Great post, but I did want to make one correction – technically, it is reproducible. You start by opening photoshop…

  63. Chemist Says:

    I do hope Dow leaders read this story. Particularly those who B. R. worked for. They should go back and re-check all his data. Pressure does not stop with grad school; there is significant pressure in any industrial job to be better than your peers, to stand out and get faster promotions, bigger bonuses, etc. How could Dow trust this guy anymore, or whatever he has done/published at Dow?

  64. Paul Bracher Says:

    So, is everyone comfortable with this post’s coverage of a suspicious paper? The paper was allowed to proceed through editorial review, and when the retraction was issued, we sought comments from the main parties involved before posting a deeper analysis of the data.

    I think this is *exactly* what Paul Weiss asked for in his editorial, right?

    As you could probably gather from the post, I am disappointed the editors allowed the retraction to be based on simple irreproducibility versus manipulation. They’ve had 2.5 months with the paper/investigation, and the retraction notice provides the community with little information. JACS never told the community what happened with the Sames-Sezen retractions; I had to dig this information out using a FOIA request of government records after 5 *years*.

    I am perfectly happy to let journals handle these cases by themselves, but it would seem these examples demonstrate we cannot trust all journals to be forthright about cases of data manipulation and fabrication. I think the community deserves better. How can people improve the system if all of the details are hidden?

  65. Umbisam Says:

    “If you don’t have someone to police the literature and punish fraud, then your own ‘great research’ gets devalued.” This is why I suggested that there be some kind of high court appointed to do this. 2nd part of that sentence wasn’t meant to be attributed to you, only the first part. I would say that quite a lot of things discovered 100 years ago are still important, even if research methods were more crude. We have more sophisticated methods, but that doesn’t devalue previous research. As an example, it’s great that organic molecules have been imaged with AFM, but the AFM results didn’t change the way we think about molecular structure. I suppose it would be fun to list things discovered/developed 100 years ago. Periodic table, that idea of atoms being linked together to form molecules, etc.

  66. Umbisam Says:

    @Chemist: Could happen. Sezen’s Ph.D got revoked. In the end it might all be up to Marks. If he agrees there was data manipulation. If he says he’s not sure then they won’t revoke it.

  67. yikes Says:

    Dow researchers are well aware of the situation, especially those who work with Brandon closely…

  68. Chemist Says:

    Check this one out. It also has an e-mail address at the bottom.

    http://rd.springer.com/article/10.1007%2Fs11422-010-9268-4/lookinside/000.png

  69. Chemist Says:

    Here is the second page:

    http://rd.springer.com/article/10.1007%2Fs11422-010-9268-4/lookinside/001.png

  70. Umbisam Says:

    @Chemist: I’m not sure that it’s clear which of the three did the manipulation if any, but if it was the first author, people are going to be even more upset when they see the Dow affiliation with so much competition for jobs. This will be enough for people to email Prof. Stang to find out what’s going on. I respect a man who knows what it’s like to flee Nazis, but what about the chemists who did their time in grad school and are now trying to flee the awful beast of unemployment. Instead of paying people 7 figure salaries to talk about how great the employment situation is for chemists, why not people to serve as a high court to judge the data integrity of papers and be the people for anonymous sources or blogs to tip off.

  71. Chemist Says:

    @Umbisam: I fully agree. Regarding the first part, I have a hard time imagining that Tobin or that asst. prof. would have been part of this fraud.

  72. Umbisam Says:

    @Chemist: I feel the same way, but if they don’t say anything, out of fairness I can’t publicly assume the first author to be the one who figured out how to get strange noise patterns.

  73. Rhenium Says:

    Chembark @11:58 am,

    No, I’m not at all satisfied with the journal response. As usual it was glacial in its response (as opposed to geological speed e.g. the nano-chopsticks paper). It also completely side-steps the fact (that you pointed out) that this is manipulation, fabrication and fraud, as opposed to irreproducibility. ACS is an ossified society and this is another embarrassment to its members.

    Sadly, going and embarrassing high ranking members of the chemistry community is likely the only way we will ever see progress on this front. I will stick to my quiet little area of the chemistry world rather than publish bunk and get away with it, as seen above for the first author.

    Rhenium

  74. Chemist Says:

    @Umbisam: you are correct.

  75. Chemist Says:

    Well said, Rhenium.

  76. undaground Says:

    ChemBark, delaying this open discussion about the JACS paper is what provided those involved with the opportunity to address only “reproducibility” in the retraction. It benefited only them, not the community. Is this how YOU think it should work?

    When can we expect the follow up story? Is the delay for similar reasons?

  77. Paul Bracher Says:

    @undaground: The delay (of mere days) is because I have a job. Also, I was on a trip this week to Washington for an ACS meeting. I hope to have the follow-up post ready for Monday.

  78. Chemist Says:

    Personally, I’m happy Chembark even exists. Otherwise all we would have is a meaningless retraction notice. Thank you, Chembark.

  79. Dave Fernig Says:

    @Hoya Blaming the underling may or may not be right. It is a common defence, but it isn’t always a full reflection of reality. I would imagine that if there was some sort of breakdown in a lab, that the PI would do as full an investigation as possible to figure out where the data came from and the conclusions of this work would then go to the University for action. One would also would want to reconsider how the lab functions to figure out a way to avoid a repeat in the future.

    @Umbisam @Grad student We do have a problem in science and it is quite simple: science does not work the way we say it does. That is, the perception is that science is very open, we put evidence forward, weigh, and if evidence is found wanting, we say so. Instead, we have a career system built on papers and journals, where unethical behaviour may get you ahead. This is particularly demoralising for the young, who generally are more idealistic than their more cynical elders. I also think the hypocrisy of institutions, who enforce correctly strict rules on misconduct in their teaching activities, but are less willing to do so when it comes to their own is dangerous. When a system has a different ethical standards for the leaders and the led, it can only collapse. Though this can take a while, even the Iron Curtain fell.
    Given that science is one of the pillars of a civilised secular society, it would be a tragedy if we were complicit through our silence in the downfall of science.

  80. horde Says:

    Hey paul, I don’t have a problem with how you’ve reported this one — Nor any of the other stories you’ve broken in the past. Clearly there are differences between the two approaches, it doesn’t make a lot of difference to me the reader, but I do tend to prefer that such matters are aired in public as soon as matters for public concern are noticed.

    The clear disadvantage of waiting for whatever official investigation to take place and conclude before publishing the story is that the authorities that be will take their precious time over them (years sometimes, as you well know), during which time of course all concerned proceed as though nothing was wrong, and you would be sitting on the story for all that (unknown length of) time. B, there may well of course be no such investigation in the absence of a public outcry — such institutions have large carpets and may often believe that the avoidance of scandal is the right thing to do.

    In this case, Jacs had a chance to deal with it. They dealt with it badly (by failing to acknowledge misconduct), and they somehow manage to have a go at you — I find the point in the letter addressed to you where stang expresses his disappointment in your behaviour astonishing. The plus is that they did what they did quite quickly.

  81. Renee Says:

    A previous poster has asked my thoughts as a polymer chemist on the GPC chromatogram in Fig. S10. I’ve taken a more careful look at this one, as well as two other relevant ones, and these are my observations:

    Figure S6 is the GPC for the ethylene/methyl methacrylate (MMA) copolymer made with the nickel catalyst FI2-Ni2. This chromatogram is so fuzzy that it is illegible, as is the MW data posted to the right of the chromatogram. It is the only chromatogram out of eleven in the article that is difficult to read. Even with zooming in at 400X, I could not make out the MW data.

    Fig. S10 is the GPC for the physical mixture of commercial poly MMA (MW 120,000) and commercial polyethylene (MW 70,000). No MW data is posted to right of this chromatogram (unlike most others), and there are no legends to show which peaks are poly MMA and which are polyethylene, so one cannot tell if there is good separation between these two polymers, although the caption states that there is. Also, there are serious baseline issues in this GPC, with large negative peaks evident for minutes 0 to 15, unlike the other chromatograms. The caption for this figure notes that it should be compared to Fig. S6, but S6 is illegible, as noted above.

    As a comparison, one can look at Fig. S9 to see a run with a smooth baseline, and with MW data posted to the right of the figure.

    Figure S13 – This is the same ethylene/MMA copolymer made with the nickel FI2-Ni2 catalyst as in Fig. S6, but after washing the copolymer with DCM to remove any MMA homopolymer that might be present. As with Fig. S10, the MW data has been cut off from this chromatogram. The reader is asked to compare Fig. S13 to Fig. S6. However, as noted before, S6 is very fuzzy, with MW data that is illegible.

    Still, looking at both chromatograms at higher zoom, the copolymer does indeed look essentially unchanged after washing, as the caption to Fig. S13 states. Perhaps a bit too unchanged. I invite others to look carefully at these two chromatograms, as we the readers have been invited by the authors to compare them. I would be interested in hearing what others have to say about them.
    [Please note – these chromatograms were published with slightly different sizes, so it helps to look at them at slightly different zoom levels in order to make the axes the same size.]

  82. Chemist Says:

    Additional comment about Fig 7. Go the the paper and look at it at 800% magnification. It is crystal clear that solid bars were placed in as peaks. They are indeed very unnatural in that not only they have no broadening at the bottom, but they are darker, way thicker, and sharper than the sorrounding peaks. In addition, some of them show a little tip sticking out at the top as though some were placed there (very poorly) to cover off existing peaks in an effort to make all major peaks look similar. All that hassle whilst leaving some of the other peaks alone, creating a very significant inconsistency even at 100% zoom… Ridiculous. Actually, rather sad.

  83. Chemist Says:

    @Renee: Figs S13 and S6 do look VERY identical. I did a screen capture on both and placed them into Word, where I changed the brightness and contrast on them. They show identical defects (small random black dots and smudges here and there). You could also see that the peak and baseline shapes/patterns are identical. I am no forensic chemist, but I sure cannot tell the difference between the two. This certainly explains why Fig S13 does not have MW info on the side and that S6 is made so unusually fuzzy.

  84. Chemist Says:

    I may be completely wrong on this one, but the DSC in Fig 9 looks strange to me as well. First of all, the curves look like they were drawn by a 5 yr old. Also, upon magnifying the image the lines/peaks look strange a bit. My question is why did the authors not use the Universal Analysis program that makes professional looking plots and it normally comes with TA Instruments machines? That program allows one to create nice overlays, etc. Strange at least. The figure caption is wrong, too, since it mentions Tg’s instead of Tm’s, also, it says “second scan data”, yet you see doubling of the lines for some peaks. So do these curves represent two heating/cooling cycles or just one? If two, then it is weird how there is an almost complete overlap for the two cycles for both samples except for only a few peaks, but hey, I am not a professional DSC chemist, so I will let someone more skilled in the art chime in.

  85. Curt F. Says:

    One question I have been wrestling with is: should journals require submission of all data in digital form? That is, should journals require that any NMR spectrum shown by the authors also be provided as a .csv file or similar?

    In general, I think most readers would welcome digital versions of all the data in the paper. Authors would probably find the practice tedious, especially older labs which still rely on analog printouts of data.

    But when it comes to detecting and outing fraudulent articles, I wonder if a requirement for digital data would just force the fraudsters to fake the digital data first, then prepare far less suspicious figures from that. It would be much harder for casual readers to detect “correct” figures prepared from fake data. So requiring digital data could, perversely, make fraud harder to detect for readers.

    (It could make automated fraud detection easier, since data auditing programs could automatically scan these raw data, but that’s a different topic and I don’t know how well poised they would be to implement anything like that anyway.)

  86. Graet Chem Says:

    This discussion has moved on a bit since I could last contribute, but I’d like to cast back to a response to my last post:

    @Giant Octopus:

    It seems that you are looking around for options to justify your apologist stance on this issue. Try to blame a post-doc who may or may not exist is a bit of a stretch to me. A PI should be thoroughly reading and reviewing every paper that comes out of their group this is a really important core part of their job. I know senior professors with large research groups who manage to do this so there is no excuse in my opinion.

  87. GiantOctopus Says:

    @Renee, @Chemist : Based on the captions, Figure S5 and S6 should be the same sample, and are probably different chromatograms when looking at the baseline from 0 mins to 15 mins. It could be that they forgot to put “Entry 18 after washing” in the caption for Figure S6, and then decided to repeat the Figure for S13 (they are probably identical, looking at baseline noise) which has caused some confusion. Therefore, I would compare Figure S13 to Figure S5, which is better resolution and has the molecular weight data.

    That said, I would not trust that molecular weight data as we can see the baseline the experimenter has used in the chromatogram. They have selected only the low molecular weight fraction of the sample, so they are getting an artificial result. There is absolutely no way that polymer has Mw/Mn = 1.43 according to that chromatogram, and we see that in the tables of data reported in the publication (Mw/Mn = 2.7 and higher in most cases) and Figure S8 is analyzed correctly and gives Mw/Mn = 3.0.

    My point about Figure 10 is that the baseline is noisy, with variations of 20 mVs, that the data is unusable. A good chemist would have spent a 40 mins re-running that sample when the instrument was functioning correctly.

    Even if the bad NMR data was impossible to spot (and I certainly would not have spotted it) and even if the experiments were reproducible, that chromatogram should have alerted the other two authors that the experimenter had a such low standard concerning data quality that all of his work should have been checked.

    Furthermore, the data for S5 and S6 is labelled Entry 18 in Table 1. I’ve looked through the paper and tried and find this entry (even looked at Entries labelled 8 in case there was a mis-typing) and nothing seems to match. Maybe I’m missing something?

    I still don’t think Marks and Delferro are dishonest, just that they really should have done better.

  88. chem Says:

    If Rodriguez don’t answer try asking the other co-author because probably he is the one who reviewed the article before the JACS submission and it is impossible that he didn’t notice the fake NMRs.

  89. anonymous coward Says:

    anon @1:37 – There are a lot of places that digital data could have been gummed up (so that without further investigation, I wouldn’t know who to blame), but I don’t see why making data up because you lost the originals is an understandable and reasonable alternative. You could be making things up for lots of less honorable reasons, and no one else really knows them, and so can’t know if they can trust your future work. The faked data (if used in place of real data) might also be missing artifacts which might be important even if the person making up the data (or recalling it from memory) didn’t know were important. I’m sure there are other reasons, but being naive and sloppy with data isn’t a reasonable excuse.

  90. Umbisam Says:

    When one can’t reproduce something, one should look back at the data that was used to support the original conclusion. Shouldn’t have missed it twice.

  91. Hoya Says:

    @Garet Chem

    To avoid my comment being a product of hindsight bias, I actually thought about it very carefully (and checked with different methods of veiwing without magnification, as I pointed out) before I made the comment about whether or not the interesting artifects are easy to spot.

    Regardless, I don’t think we should spend more time arguing about whether or not it was easy to spot or not – my concern, from the beginning, lies in the problem that those spectra in the main text have repeatedly escaped expert scrutinies and that, once again, the problem was poorly handled by all parties and officies involved. Then again, that is based on my personal opinion that the artifects are easy to spot, perhaps in this case we will have to agree to disagree.

    @Dave Fernig

    Agree. The thing that disappoints me the most with all the recent reports on potential academic misconduct is that barely anyone has barvely dealt with it with honour and transparency. At the very list I don’t seem to be able to recall any of such cases. Is it really that difficult to be completely honest about it to the community – particularly when one has no part with the actual fabrication and can only be held accountable for an oversight? I would actually admire someone who deals with situations like this with honesty and trust his science even more, although it appears that everyone seems to think the opposite.

    Hide the facts, blame the underlings. That’s what I’m seeing from supposedly reputable scientists who, and institutions and publishers that, are involved in muddy situations like this.

    @GiantOctopus

    Thank you for taking the time to respond to my comment, although I think you have misinterpreted what I said (and implied) and made unjust accusations and insults.
    First of all, I would like to point out that I have absolutely nothing personal against the Prof. – I was just commenting on how a PI should and/or should not act and be treated by the community in cases like this and, obviously, using this specific case as an example. As I understand, this is what happened:

    1. They were not artifects that could not have been easily spotted;
    2. There were many of the said artifects;
    3. Not to mention other incongruencies that others have spotted and not elaborated upon;
    4. The authors claimed that they could not reproduce the results and asked for a retraction;
    5. Paper retracted by the journal, stating that it’s simply a problem with reproducibility.

    Now, I agree with you that the Prof. most likely knows his science. In fact, if you read again what I said originally I was implying that he very likely knows his science but he didn’t handle the situation with the honesty that I think a good scientist should have. So, let me ask you again, why was the reason for retraction merely an inability to reproduce results? One can only spectulate but I’ll help you with a few explanations (after filtering out the ones where we assume the PI is not very bright) that I came up with that fits the current scenario:
    1. PI knows his science. PI is innocent. PI found, or was alerted to, reproducibility issues. (PI contacted the student who originally did the work). PI realized that there is a big problem but did not dare to find out what it was. PI simply asked the journal to retract the paper using problems with reproducibility as a reason.
    2. PI knows his science. PI is innocent. PI found, or was alerted to, reproducibility issues. (PI contacted the student who originally did the work). PI realized that there is a big problem. PI located the problem. PI hid the fact that they are interesting results hopefully by retracting the paper things will die down quietly. PI simply asked the journal to retract the paper using problems with reproducibility as a reason.

    3. PI knows his science. PI is not innocent. PI simply asked the journal to retract the paper using problems with reproducibility as a reason, hoping that it would quiet down after that.

    Clearly there are many other possible scenarios as well. Although I would like to challenge you to come up with one in which the PI knows his science, is innocent and is completely honest. One may be tempted to simply shift the blame to the journal though…
    Interestingly, you pointed out that you judge a PI based on his publication record (by the way, that’s a self-contradiction to what you falsely accused me of how I judge a scientist – more later) and recognition. Have you seen this? http://www.nature.com/news/through-the-gaps-1.11427 I would not like to be a sheep who just blindly believes everything that is published by prominent scientists – that’s the attitude that hinders the progress of science.

    You said that “… people don’t continually fund research that does not work to a certain extent.” That is very well worded and almost correct, things only need “[to be shown to] work to a certain extent” to get funded – what’s the point funding something that already works? Here’s an example for when problems sometimes occur: http://retractionwatch.wordpress.com/2013/11/04/jbc-issues-correction-for-paper-by-khachigian-who-has-had-four-others-retracted/

    You further contradicted yourself by rhetorically asking me the following: “why should anyone engage with ChemBark or anyone else who is not directly involved? They are not answerable to ‘the community’ but to their employers.” Anyone who publishes in the open literature has a duty to be responsible to the chemistry community; 2. moreover, as I have pointed out before, a PI is responsible not only to himself and his employers, but also his colleagues, students, collaborators, funding bodies and the public. Isn’t that everyone? Well, it is. Not convinced? Let’s use the Khachigian case from above as an example: http://www.abc.net.au/news/2013-08-12/trials-of-skin-cancer-drug-dz13-suspended-amid-misconduct-claims/4881622

    I apologies for using the word “punish” if it doesn’t fancy you. Although I have no interest in inflicting punishment on anyone, I was simply saying that one should simply accept appropriate consequences for making mistakes and neither should the community nor the PI/all parties involved just let it slide. Is recieving punishment for when you do something wrong such a bad thing? I personally don’t think so – it’s great for learning faster and character building. Then again, you may be thinking that I’m masochist.
    Finally, in your response, I think you have unfairly insulted my moral standards, which I do not deserve, and that brings me to the next point. I critized you for saying “… if I were to be offered a similar research output to Prof. Marks over the next 30 years of my career for the price of one retraction, I’d bite your hand off.” Aren’t you the one who initially proposed it’s appropriate to prioritize quantity over quality? It was I who expressed concerns of you thinking exactly that in my first response. You should apologize, at the very least, for accusing me saying something that I have never said or implied, not least then you seem to have then went on to suggest something harsher than that.

    You have said things, made false accusations and interpretations, shifting your view ever so slightly every other post, and dished out insults as you pleased, which are far worse than my intendedly-neutral use of the word “punish”. I hope you made those mistakes simply because you are too busy finding solutions to the world’s problems.

  92. Synthetic Dave Says:

    @Hoya: “Hide the facts, blame the underlings. That’s what I’m seeing from supposedly reputable scientists who, and institutions and publishers that, are involved in muddy situations like this.”

    It’s not surprising in the least, especially when you have stories quietly circulating around academia about professors of high repute not reacting well to be told their papers are incorrect, to the point where in one case the career of the young professor, who has been submitting papers showing that the papers put out by a professor of high repute are incorrect, is ruined – as in, he can’t get published, can’t get funding, because he’s essentially been blackballed. Then there’s former editors who have held up submissions from less well-known groups just so their own students don’t get scooped. So, would I expect someone in these same circles to try hide evidence that would make their lab look bad? Absolutely.

  93. GiantOctopus Says:

    @Synthetic Dave : Very good!! Although in some cases though the young professor thinks up a simple yet devastating experiment, and just so happens to get the perfect opportunity to do it. :)

    @Hoya: Thanks for the reply. I tend to speak my mind and my writing style can be rather aggressive, so apologies if you are somewhat put out. Don’t take it personally. Any insults are not targeted at you per se, but I tend to get confrontational. My style is to cause absolute mayhem, rather than being bland and insipid. If my view appears to shift it is because I read every post on here and have my perspective altered or thought about things in a way I hadn’t thought of before, or gained some new information. That is how we do science, after all. In this particular instance, I gave the now retracted paper a more thorough read-through than previously.

    Firstly if you really think that my ego believes it is solving all the world’s problems with synthetic chemistry then you are very much mistaken. My work is about understanding a very commonly used synthetic tool. When you are building a machine, you would like the optimum toolkit, wouldn’t you?

    Also, my comments and posts (calculated) were deliberately provocative to actually try and generate a genuine response. Consider me a Troll if you like.

    My point of Trolling you? Consider it a lesson that words (such as on a blog, or in Reviews and Perspectives) can only sting a target and annoy or offend it. However, actions (in the form of proper experiments and reproducible data) have far greater resonance. This blog is widely read, and some of the other scientists that might be looking at it would do well to learn the lesson that consistently and repeatedly deliberately misrepresenting the views of others in the literature, as well as being a dishonest reviewer, can lead to a very unpleasant response. I get the feeling that Synthetic Dave knows exactly what I am talking about.

    “I would not like to be a sheep who just blindly believes everything that is published by prominent scientists – that’s the attitude that hinders the progress of science.” – I could not agree more!!

    A situation where the PI is knows his science, is innocent and is completely honest? That is a tricky one, how about this?

    The PI has a theory that a certain type of catalyst will work for a particular reaction. The student makes a range of compounds to test, but accidentally mislabels the samples. The student uses the catalyst in bottle Labelled A, thinking that they have Catalyst A, and reaction X works well, meaning that he (or she) thinks they have achieved what they set out to do. The compound in the bottle A is actually Catalyst B. They keep using said catalyst to make compounds or generate reproducible kinetics and genuine data. Because the student does not repeat the synthesis of the catalyst, or re-check that the compound in the bottle is what the label says it is, they think they have a working Catalyst A. The kinetics and data generated using this mislabeled compound are all genuine.

    The PI is pleased that the theory that the catalyst works is correct. The PI asks if the data are reproducible. The student replies that the kinetic data have been performed a number of times, and the same kinetics results has been obtained every time. The PI is satisfied that the data are reproducible. (This is the key misunderstanding)

    Because the student does not have adequate guidance from a post-doc or lab manager, the student is not aware of the standards of reproducibility required by high-level science and is under the false impression that their data is correct. The kinetics may be reproducible, but would not have been if the catalyst had been made a few times and tested. The PI believes the data are correct, and has assumed that the student has made the catalyst a few times in batches. The word “reproducibility” has meant different things to different people.

    Under time pressure to get results and publications, funding etc. and not wanting to be scooped, the PI decides to publish the data. “Reaction X is catalyzed by Catalyst A (from Bottle A)” appears in the literature and all the authors genuinely believe the results, including the person who got them. However, what should really written is “Reaction X is catalyzed by Compound B” since that was what was really in the mislabeled bottle.

    Time passes. There is an issue with reproducibility informed by another group. The PI finds out and gets the student to repeat the reaction. The student gets mislabeled Bottle A. The student thinks they have Catalyst A (but in reality they have Catalyst B) and repeats the experiment. In the process, the bottle A is finished off. The experiment works as before. The professor notes the criticism, assumes the other group’s experimenter was not very good, allows the student to graduate, and thinks no more of it.

    More time passes and the reproducibility issue flares up again. The PI asks a new student to repeat the paper. The paper says “Reaction X is catalyzed by Catalyst A”. The student makes Catalyst A, because that is what the paper says to do, and the reaction fails to work. The student tries again but gets the same result. The PI believes the student may not very good at doing experiments so asks a post-doc, but they don’t get the results in the paper.

    Rather than spending too much time trying to work out what has gone wrong and issuing a major correction, the PI decides that the paper is not reproducible and retracts it. One criticism of the reproducibility of a paper is unfortunate. Two is pretty bad. Three is atrocious. How can anyone trust any conclusion coming from that?

    Everyone assumes the student is guilty of fraud and making up results, when in reality they are guilty of not doing experiments properly under time pressure. Put it down to inexperience. If the original student had made batches of catalyst a few times and tested it, or checked what the compound in Bottle actually is, this problem could have been avoided. They even repeat the error when asked to reproduce the experiment. The PI is innocent because they have not knowingly committed fraud (i.e manipulated spectra, data, kinetics etc.) and the PI is honest because as soon as an issue is caused they ask the student to repeat the reaction and they get the same result, and when they repeat it a third time and can’t reproduce it they decide to pull the paper rather than pretend the data are good and bury their heads in the sand.

    Is that any good? I can try again if you are not satisfied or disagree.

    Lets scale this problem up. The student makes a series of catalysts and mislabels a number of them (e.g Refers to Entry 18 in Table 1 when there is no such entry) but generates what they, the post-doc and the PI think are genuine data. The student can’t make figures for toffee so everyone is going to criticize them even if they happen to be genuine because they look so terrible.

    What happens then?

  94. Graet Chem Says:

    @Giant Octopus

    Interesting hypothetical there but where is the justification of manufacturing spectra, as has appeard to happen in this case?

    If you have a student who thinks it is acceptable to cut and paste peaks from one spectrum onto another (without stating that they have done this) then they have not been a) taught properly as an undegraduate, b) not guided properly by the PI and senior members of his or her team.

    There simply isn’t a get-out clause here as PI’s have a managerial duty to the members of their labs. I lead a team in industry and it is very obvious to me that in that commercial environment if my staff screw up, I’ve screwed up. As their manager I am responsible not only for ensuring that their work is good enough, but also that they are trained and competent to take on the responsibilities I have assigned them. If I have a team member who can’t construct clear and unambiguous graphs and spectra then I need to train them, not just let their sub-standard work out of the door and wash my hands of it.

    Of course mistakes do happen, but as a leader you have to be prepared to take the hit if they happen on your watch.

    Since leaving an academic career I have been strongly critical of the lack of decent training in management skills given to academics. I’m not talking about the buzz-word laden voodoo that seems to pervade executive training but the practical hands-on methodologies of how to get the best out of people and yourself. I have seen some absolutely atrocious management styles as a result in academia (these seem to be seen as part of the myths and legends of many chemistry departments).

    So where were the real major failings in your hypothetical that led to the publication of incorrect data? They were with the PI:

    a) Failure to establish clear lab rules regarding the standards of reproducibility expected on novel syntheses.
    b) Failure to train a student correctly in how to conduct experiments and present their results.
    c) Failure to ask clear and unambiguous questions to ensure standards are being adhered to e.g. not ‘are the results reproducible?’ but ‘did you test the kinetics with at least two different batches of catalyst?’

    Anyone of these failures, if addressed properly, could have prevented the hypothetical scenario you described from occuring, all are within the direct control of the PI.

  95. GiantOctopus Says:

    @Graet Chem – thanks for reply and comments. I hope Hoya also considers my response.

    I’ve been busy all day and am far too tired to write substantial anything now, but will try to find time before the weekend. Suffice to say that manufacturing spectra is not acceptable if this is what has occurred here, but if I had been scrutinizing this data I would not have noticed anything untoward (I would have asked for a better quality figure though) and I would not have wanted to stake my reputation on considering whether or not they were fraudulent.

    It would be interesting to know if the anonymous source of the story alerted the journal because they saw unusual spectra, or because they tried reproducing the experiments and failed.

    I will adapt my hypothetical situation using a couple of examples that I remember people actually talking about when they presented their research.

  96. Justin Bieber fan Says:

    I like how Brandon’s linkedin profile shows the following interests: career opportunities, new ventures, job inquiries. Well, he is thinking ahead already I guess. The good news is Jack in the Box is hiring!

  97. Su Says:

    Comment in moderation for implication of suspicious data.

  98. Paul Bracher Says:

    @Su: Wow. That is a really nice catch. Did you raise these concerns with the journal?

  99. Paul Bracher Says:

    Comment in moderation for implication of suspicious data.

  100. WMG Says:

    Do you all see how excited Paul get’s when he sees blood in the water? Jesus. Paul, I hope you realize that every time you jump up and down for joy at the fall of one of these papers/people (you remember they are people right Paul?) there is an entire network of alumni and people connected to each of these professors that begins to resent you. Do you understand that? Do you understand that every paper you ever publish should and will be scrutinized with a fine tooth comb? Until I read your blog I have never ever blown a figure up to 800%, but since you and your Columbo sidekicks have taught me the art I, and many people I know, will surely be doing that to each and every one of your papers. Best of luck in your long and illustrious career Paul.

  101. Bryan Says:

    @WMG: Are you saying that if you are capable of making data “look better”, by fixing the baseline or removing peaks or any other means, to a point where you need to “blow up a figure up to 800%”, that means it is ok?

  102. Chemjobber Says:

    Nothing quite like making direct threats on (what’s believed to be) a blog with an IP address tracker.

  103. Franz Says:

    Wow WMG, the implication of your comment is, this is something people do, it’s unseemly to point it out, and we’re all going to be waiting to catch you when you do it. I tend to believe that this is NOT something most people do, and that the occasional example needs to be brought to the community’s attention. I would imagine that Paul welcomes your promised scrutiny.

    Do you think it would be better to just keep quiet about digitally manipulated spectra that have been published?

  104. isotopeeffect Says:

    @su and @Paul: I’m confused, I don’t see a smoking gun here. It looks like those spectra were processed with relatively few points and the program that draws lines did some bad interpolation. Unless I’m looking at the wrong data…

    @WMG: I mean, I think I understand part of the sentiment, and certainly some people in the community (of varying degrees of power/authority) will respond as you have, but seriously? You’re implying that checking data is a bad thing? This isn’t a a Glenn Beck-style “I’m only asking questions!” witchhunt, it seems to me that it’s the expression of concern that published papers are treated as ironclad documents. You’re saying that Paul had better watch his back because academics *might be* capricious? Wow, big shocker. I bet you rocked his world.

  105. MarkR Says:

    @WMG
    I don’t know if your post can possibly be serious. Maybe, I’ve missed the subtleties of your humor.
    If there is an “entire network of alumni and people” that resents someone for publicizing the possible fabrication of data or spectra, then that is a problem with system. Surely, it is the person responsible for the fabrication that should be resented?
    Although I couldn’t possibly comment on Paul’s concerns about every paper he submits being scrutinized with a fine tooth comb, I don’t see any problem with that. If the science is worthy of publication and the data stands up to scrutiny, then surely it will be published? If the data is accurate, then there is nothing to fear from scrutiny post-publication either. You’re not suggesting that journal editors or reviewers would discriminate against someone are you. This is from the ACS ethical guidelines:

    “An editor should give unbiased consideration to all manuscripts offered for publication, judging each on its merits without regard to race, religion, nationality, sex, seniority, or institutional affiliation of the author(s).”

    Lastly, I also take issue with your statement about “Columbo sidekicks”, are you suggesting that people should accept anything that is published without question? If there is more fabricated data out there (and recent high-profile examples would suggest that there is), then it is clearly better that it is found and retracted. In the VT 31P NMR spectra in the Rodriguez–Marks paper, it is clear that the noise on each baseline is exactly the same. This does not need to be blown up by 800% and should have been noticed by the reviewers.

    I do concede that there should not be excitement when another case of fabrication is discovered. Rather, there should be sadness. However, I don’t “see” any evidence of excitement in this quote, “Wow. That is a really nice catch. Did you raise these concerns with the journal?”

  106. anonymous Says:

    Academics are capricious. But I think a small number are vindictive, which is what I think WMG is implying.

  107. WMG Says:

    @MarkR and Franz

    No I am certainly not saying that falsifying data is right and of course I think all work should be properly
    scrutinized. My comments were born out of frustration from the glee I perceive Paul takes in making these posts (see “wow. Nice catch” comment, the “before I get scooped” comment”, and the tease of the second Rodriguez post which seemed to be designed to only get more web hits).

    I actually agree more with Paul Bracher than I do with Paul Weiss, but I do think there is a point to be made that the way the internet works is that as soon as something like this is published online then all of the parties related get blame tossed on them which is unlikely to wash off (except maybe for a powerful PI). In regards to Clare Francis it seemed to be agreed upon that if she gets it right 1 out of 2 or 3 times then she would be doing a service to the field. What happens if/when Paul’s record gets to 1 out of 3? Is it worth it then?

    @Chemjobber. I would hardly call the promise to review his papers the same way he seems to expect all papers be reviewed a threat. Though mine was born out of frustration and probably went overboard, I don’t doubt that this is the kind of thing Paul will be dealing with his entire career.

  108. Chemjobber Says:

    @WMG: Without condoning your initial response, I see where you are coming from. Fair enough.

  109. Paul Bracher Says:

    @WMG: Who’s jumping up and down for joy? My “nice catch” comment was not an expression of excitement, but one of appreciation that someone picked up on something so hard to notice.

  110. anonymous coward Says:

    Schadenfreude is present in a lot of crime journalism, or at in the choices of coverage (either “look at that stupid evil idiot who got busted – I’d never be that dumb.” or “look at that celebrity who thought they were better than us get brought to their knees by their own failures.”) I didn’t think Paul’s joy was at the potential fraud, but that someone was paying attention, but in any case, the joy doesn’t negate the fact that the coverage (of crimes and of potential fraud) is necessary.

    I think the lack of justice from many of the fraud cases (Sames/Sezen, where everyone who wasn’t the prof got screwed, including people who proved it, or LaClair’s *cough* synthesis of hexacyclinol) also factors into the commentary. People want to feel that the system is fair, and nailing people who do wrong is one way people can feel so. Sometimes, criminal prosecution is motivated by the desire to make *someone* pay for a terrible act – while it is execrable and wrong, it does not negate the entire system. Hopefully it gets people to do better at enacting justice. I hope the same happens here.

  111. Umbisam Says:

    Wow, there’s a lot of competitors for this year’s chemical villain award. You might have to make two new honors: Best Photoshop Job, Worst Photoshop Job. The latter obviously goes to the world’s smallest chopsticks.

  112. Anonymous Says:

    Comment in moderation for implication of suspicious data.

  113. Paul Bracher Says:

    OK…two commenters today (or maybe one from two IPs) have left comments calling attention to questionable data in six different papers.

    Whatever system we have for dealing with this sort of stuff, part of it can’t be that comments threads become dumping grounds for accusations or calls for me to investigate things.

    I’ve placed any comments from today that call attention to specific papers into moderation such that the comments are not visible to readers. They are not deleted; they are simply “on hold” until I can figure out a fair way of dealing with them.

    If you want to draw my attention to questionable data, please e-mail me (paul at chembark). If you want to call the community’s attention to concerns, please use a site like PubPeer, where you’ll have a dedicated thread.

    Finally, whatever you do, please write to the journal editors with your concerns. You can do this anonymously. For the papers called into question today, I would appreciate it if the commenters would e-mail me to let me know that they have contacted the editors. I won’t “out” you anywhere. Also, if you are worried about my disclosing your identity, you can always make an pseudonymous e-mail account and use that to contact me or the editors.

    The next post on ChemBark will outline a policy for how we’ll deal with reporting papers that contain suspicious data and moderating comment threads. I’ll need a little time to figure this out. Suggestions are welcome.

  114. WMG Says:

    Paul, I am happy to see that my slight outburst seems to have at least caused you to pause for a moment to see that *some* checks and balances and a call for caution might be necessary. Thank you for that much at least. For the sake of sound Science and potentially innocent individuals let us hope we get no more “Wow. Nice catch” moments.

  115. Paul Bracher Says:

    To be fair to the person/people who posted, I think the concerns with the spectra are legitimate, but I can’t thoughtfully process a comment that calls attention to five different papers with huge SI files in a timely manner. Rather than let the comment fester, I took it (and its sister comment) down temporarily.

    But, again, to the person/people who found those six papers…please write the journals too.

    These spectra appeared to have *very* subtle unnatural features, but there were several of them. I don’t see how you could fault any reviewer or PI for missing them, but when they’re pointed out to you, they don’t make sense.

  116. Allison (@DrStelling) Says:

    @Paul:

    I super-zoomed in on a few of the NMRs from one of the aforementioned papers within the supplemental file. Now, I’m more a Raman/FTIR gal, but……yeah. Look. Take your time, that’s a lot of info to sort.

  117. Paul Bracher Says:

    @Allison:

    I saw a few odd-looking sawed-off peaks and a *lot* of places where the amount of noise abruptly disappeared in favor of a completely noiseless baseline. Is this what you saw? Also, the color of some of the lines at these points seemed to go from jet black to a very dark grey.

    Very subtle. If “Su” or “anonymous” wants to weigh in without explicitly naming the papers again, please do. Or better yet, drop me an e-mail. Again, you are welcome to remain anonymous.

  118. Franz Says:

    Paul, I agree with your decision. In all the brouhaha about “lynch mobs” and such, I have agreed with the alarmists on but one issue: comments sections can start getting into piling on mode, and people’s reputations can be defamed in a careless and casual manner. So if someone wants to raise their concerns about the data in a heretofore undiscussed paper, they should either bring it to your attention through other channels and let you decide whether to devote a post to it, or else start their own blog and have at it.

    I did look at the spectra in that first paper mentioned by Su, and I agree that there are some very strange things going on with the baseline noise. When the individual spectra are imported to Adobe Illustrator, it becomes more obvious that certain segments are *different* to the rest of the spectrum. I honestly do not know whether this is due to some sort of spectral editing, or some other phenomenon, and will leave that you and other experts.

    If I were to evaluate the various data discussions on this blog over the last few months (Dorta, Cossy, and Marks), I would say I had the least discomfort with the way the last one played out. You delayed posting on it until JACS dealt publicly with it. (That said, I would have supported your decision to out this business anyway, if the journal had dragged their feet on dealing with it.) While I do think that these matters need to be much more transparent than they traditionally have been, I became a bit concerned about the content of the comment threads in the Dorta and Cossy posts. The upshot is, you may have to become more activist in your moderation of comments.

  119. Paul Bracher Says:

    @Franz: Thanks for the input.

    For the record, I absolutely loathe moderating comments. If I am going to do it, I feel I owe readers a policy for when and how it will be done. I’m working on the policy now, probably for a post on Monday. The policy will more closely resemble how the Marks paper was handled vs. the Dorta or Cossy papers.

  120. Synthetic Dave Says:

    “Paul Bracher Says:
    November 13th, 2013 at 7:06 PM

    I saw a few odd-looking sawed-off peaks and a *lot* of places where the amount of noise abruptly disappeared in favor of a completely noiseless baseline. Is this what you saw?”

    That’s what I saw. At first glance, it looked like the spectra was just processed in Mnova, but yeah – when I zoomed in on the baseline I saw that completely noiseless baseline. This is why it should be policy to submit the fids as part of the supporting info files. It won’t completely eliminate stuff like this, but it will cut down on it significantly.

  121. Bob Sacamano Says:

    Any NMR-types out there know if standard processing packages allow one to define signals and noise=0 for a segment of a transformed spectrum?

    I can understand how one ‘shaves noise’ from a spectrum in photoshop, but the most recent examples looked as if a smoothing function had been applied to discrete stretches of some of the spectra.

    I can’t understand why someone would want to do this sort of thing, but I do want to understand how they are doing it and advise students to be mindful of potential misuse.

  122. Allison (@DrStelling) Says:

    @Paul: “I saw a few odd-looking sawed-off peaks and a *lot* of places where the amount of noise abruptly disappeared in favor of a completely noiseless baseline.”

    Yeah. The ones I saw seemed to be …. not continuous lines right at the base of the peaks, if that makes sense. Like a stepwise function. Might just be a bit of overprocessing….

  123. Allison (@DrStelling) Says:

    Ah! What Bob S. said.

    And I’m with Synthetic D. on submitting original data files with your MS.

  124. Synthetic Dave Says:

    If it really was a smoothing function of a program, I don’t think you would suddenly see so much noise suddenly pop up near a peak, but I could be wrong on that. I know NMR pretty well for being a synthesis guy, but I’m no NMR expert. As for why a grad student would be inclined to use a ssmoothing function, all it takes is his pi complaining enough about signal to noise to he tempted to use a smoothing function.

  125. eugene Says:

    I’m not an ‘NMR person’ but I use it all the time. Topspin and Mestrec don’t have that option, to define baseline noise as 0 in a part of the spectrum. Maybe there are secret buttons in those programs, but I haven’t found them. Also, window functions are used on the time decay spectrum before Fourier Transform and thus they apply to pretty much to all the peaks. I can’t think of a window function that would affect the spectrum in that way.

    Post Fourier Transform… you can define the baseline by points I guess and if you define every point as a baseline in a certain region, then it will go to 0 for a nice straight line, but the aforementioned programs don’t let you do that so easily. That’s a few thousand points to pick by hand for every spectrum. I think that it shouldn’t be too hard to program something that will set all intensities to 0 in a region that your define after the Fourier Transform. Maybe also transform it back into the time domain for extra, ultimate cheating. Of course noiseless baselines would definitely be a sign of heavy manipulation.

  126. Umbisam Says:

    I’ve always been against smoothing data. Too much room for creating/attenuating signal or making peaks that don’t exist. If you have used the instrument properly just show the data as it is. Otherwise, change your parameters, do more scans, etc. If one is going to smooth data, then you had better include the unsmoothed spectra and clearly indicate that the spectra were smoothed.

  127. Graet Chem Says:

    @Umbisam – as a spectroscopist I completely agree. Smoothing makes things look nice, but it has no sound scientific basis and doesn’t use any reliable statistical approaches to ensure that the smoothed curve is representative thus meaning that any quantitative analysis is right out the window. It is probably just about okay for qualitative analysis but even then it can be used to hide features and make poor spectra look better than they are so I wouldn’t ever use it.

    As a spectroscopist it is de rigour to describe in detail any post-processing of spectra undertaken including any background subtraction and what backgrounds were used (e.g. Shirley, etc.) and any functions used for peak fitting or integration. It seems to me that some in the synthetic community kind of take this for granted a bit as a kind of ‘black box’ or common knowledge that doesn’t need explaining and I am struggling to see a justification for that.

  128. Hoya Says:

    @GiantOctopus

    Many apologies for the delayed response and thank you, once again, for taking the time to reply. I have been busy with work and haven’t had the opportunity to make a proper response since I read yours a few days ago.
    Admittedly, at the beginning I didn’t quite know how to feel when I found out that I was getting trolled. In retrospect, it was probably a mixture of confusion and disappointment (before the eventual realization) because I made “genuine” responses. However, one particular bit in your response stuck with me since I read it:
    “Consider it a lesson that words (such as on a blog, or in Reviews and Perspectives) can only sting a target and annoy or offend it. However, actions (in the form of proper experiments and reproducible data) have far greater resonance. This blog is widely read, and some of the other scientists that might be looking at it would do well to learn the lesson that consistently and repeatedly deliberately misrepresenting the views of others in the literature, as well as being a dishonest reviewer, can lead to a very unpleasant response.”

    Now that I’ve had a few days to think about that, I think what you said is true – almost annoyingly true – on many levels. While I believe that we should continue to openly discuss problems in the literature (not only limited to potential data fabrication), perhaps, if you would excuse me for paraphrasing you, we would all do well to learn to do a bit better when it comes to both making and taking criticisms. After all, we are witnessing the beginning of something interesting!

    I actually don’t have much more to say now that I understand your view on this (unless I’m getting trolled again, but I don’t think you would be cruel enough to do it with a response like that). Although, I do have to offer further apologies for a few things. Firstly, I have wasted your time by not explicitly pointing out that I was referring to this particular JACS paper, as Graet Chem has then pointed out. Secondly, when I said ” I hope you made those mistakes simply because you are too busy finding solutions to the world’s problems”, I wasn’t judging your work (not least because I had no idea what you do!) – it was just the first retort that I came up with brash and rash. Last but not least, my comment about you “shifting your view ever so slightly” is embarassingly stupid.

    I love to argue (and clearly sometimes to hasty with my retorts), particularly when it comes to values that I strongly believe in. While I was clearly a bit too hasty with some of my retorts (clearly not very clever) I hope you haven’t taken it too personally either!

  129. GiantOctopus Says:

    @Hoya: No, not taken personally. I think the real lesson is that, thanks to the internet, we are ALL reviewers now. A scientific article’s life begins when it gets published, and “getting it past the reviewers” doesn’t mean getting the thing into print but having the ideas, theories and hypotheses accepted by the wider chemistry community at large.

    Interesting that you picked up on that particular comment because I thought long and hard about how to write that. You never know who might be reading these things, and might understand that the message was meant for them. Consider the context of Synthetic Dave’s comments above.

    @Graet Chem: I remember two things (discussed during talks) that could be added to the hypothetical situation above that could affect reproducibility. In the first example only a post-doc (PD) could get a reaction to work and nobody else could. It turned out that, when the PD was sampling the reaction he was a bit sloppy and opened the flask up to air and, as it turns out, oxygen was in some way involved in the reaction.

    In the second example, a student was making a catalyst. It worked beautifully and the student did the appropriate reproducibility studies. Some months later the same student made the same compound and the kinetics were slow and made the product in low yield. It ultimately turned out that drying glassware in a 200 degree oven (rather than in a 50 degree oven) solved the problem, so it was assumed that some trace water affected the catalyst. These tiny details could have significant implications for results, and I think answers your points (a), (b) and (c) – if a good, honest student spent 6 months searching for these details when trying to repeat experiments on their own paper, what chance would anybody else have?

    Furthermore, I don’t think all of those things are directly in control of the PI. The student should have the intelligence to work some things out for themselves and it should go without saying that “don’t make up your results” is one of the first lessons that people learn. A researcher who will deliberately fabricate spectra is impossible to train.

    “The PI should make clear what the levels of reproducibility required are” – some students don’t listen, or are willfully ignorant to cut corners and save time. What if the PI asks “did you make three batches of catalyst and test them all?” and the student says yes, when the real answer is no. Is that the PI’s fault?

    “They need to be trained to do experiments properly and present results accordingly” – the PI is not going to put on his/her lab coat, get in the lab and show the student how to do an experiment. They have post-docs to do that sort of lab management. A post-doc should not be so focused on their building own career that they won’t help train more junior researchers/students, and it is not fair to then be an author on those very same researchers/students papers without looking at the data they have generated to see if it is valid.

    I suppose, on reflection, it is far better that everyone criticizes the established big name PI who has a distinguished 40 year career and has a good reputation with the people who fund his work, rather than the young assistant Prof. who might not have a career or be able to get funding in future due to the retraction.

  130. wolfie Says:

    My goodness. The Assistant Professor is still playing police.

    Go to jax.org, if you need some reflection. You can really get it there, and I speak from experience.

  131. wolfie Says:

    What would the Jesuits comment to some self-reflectivity ????

    ohhh….

  132. Boston Legal Says:

    High quality investigation Paul. Your readers may find the below interesting. The economics of academia.

    “How Academia Resembles a Drug Gang”

    https://alexandreafonso.wordpress.com/2013/11/21/how-academia-resembles-a-drug-gang/

  133. Umbisam Says:

    @Boston Legal: In terms of physical sciences, a lot of the commentary in the article and in the comments section is off base. I don’t have time to go into detail. But one obvious point: Adjuncts/lecturers aren’t in the running to become a professor, so they are really not analogous to low-level members of a drug gang. In fact, these kinds of positions are beneficial as they increase the number of jobs available, or allow people to focus on teaching rather than research. Many of the people leaving comments seem to think the only purpose of a professor is to teach classes. It is not. Further, there are jobs outside of academia for chemistry Ph.D.s, although they are getting harder to find.

  134. Anonymous Says:

    interesting recent Erick Carreira’s JACS paper with manipulated spectra!!
    http://pubs.acs.org/doi/abs/10.1021/ja412119q

  135. anon2 Says:

    wow -this has to be the most high-profile case to date?

  136. Umbisam Says:

    So in other words, to ensure your chances of getting accepted in JACS, manipulate spectra so that refs don’t think your a slob and then issue a correction confessing to manipulation. Or manipulate so that your boss lets you submit, and then issue a correction. Better than wasting time in the lab trying to purify your compound. A nice job at Dow awaits you.

  137. Umbisam Says:

    What sounds better: Post-doctoral Research Associate or Research Associate? If one believes there is a deficit of qualified chemists in industry and you believed one way to solve it would be to give fellowships to PhD students, would you select someone who erases impurities rather than purify or concede that they just can’t get it pure? Or is it not a problem and we should just accept manipulation as part of the game?

  138. Anonymous Says:

    Very interesting NMR spectra!!
    https://pubpeer.com/search?q=Yasuyuki+Kita&sessionid=BD0115FA8FD938E42459&adv=none

  139. Anonymous Says:

    looks like there’s another one from carreira lab with manipulated spectra http://onlinelibrary.wiley.com/doi/10.1002/anie.201309901/pdf

  140. bad wolf Says:

    I thought we were waiting for Paul to issue guidelines on accusations of improper spectra/whatever?

    By the way, is purifying your own compounds really a thing in industry? I thought everyone just tossed stuff on the Biotage and came back later. Has anyone had to run a column and show clean separation of two compounds as part of their on-site?

  141. Anonymous Says:

    anybody has any insights to what happened in Carreira lab recently regarding the corrections?

  142. Anonymous Says:

    @bad wolf
    this is an admission from the authors, rather than an accusation

  143. bad wolf Says:

    @anon–sorry, i meant the Kita lab reference. Yes, if it is a retraction, then no reason not to discuss it, i suppose.

  144. Anon Says:

    Why has Paul been totally silent on this front? Any chance someone lawyered up?

  145. Paul Bracher Says:

    I’ve just been busy, and serious issues like these demand thoughtful responses. I will circle back and catch up when the semester and wedding are done.

  146. Catalysis guy Says:

    This topic is extremely important, but it has gotten much too quiet. In an attempt to breathe life in it, here is: 1) what I know, and 2) what I suspect.

    1) Marks’ work is grossly overhyped. Despite the Northwestern propaganda mill, Marks’ (reproducible) work has had minimal impact at best. Those who have worked with Marks describe him as a unrelenting tyrant (note that NONE of his students have written to support him). If Rodriguez has fabricated this on his own, Marks would have hung him out to dry by now. Marks is Northwestern great white hope, and they are circling the wagons, hoping this goes away. The idea that he is exonerated because he has a large group is ridiculous. The buck stops with the PI. If Marks has too many people to mentor, then the funding agencies have a duty to the taxpayers to better allocate funding. I assume those young faculty out there struggling to get their first grant agree.

    2) My speculation is that Marks badgered Rodriguez until he ‘made the case’ that the bimetallic systems did something ‘interesting’, assuming he wanted to graduate. Many of us in the field spotted the fake NMRs when published, but feared calling Marks out – shame on us. ChemBark/Bracher should be congratulated for their role in sorting this out. Marks has learned well from politicians – the public has a very short memory and if he remains quiet this will just ‘go away’. It is up to us to keep this alive and let the truth be told.

    A message to Tobin from the field – you have been silent too long. You need to clear the record now or forever be suspected of being implicit in this fraud.

    A message to the community – what do you know, and what do you suspect? Let’s not let this topic die until the truth is known!

  147. Reykjavik Says:

    Is C&EN going to mention the Marks retraction or are they are afraid of Marks?

  148. Alum Says:

    Ok Catalysis Guy I’ll bite.

    1.) Major businesses disagree with your assertion that Marks is overhyped. I would argue that bringing in over $100,000,000 in a career is pretty impressive, as is being named one of the top inventors in a major US city.

    (http://www.chicagobusiness.com/article/20130427/ISSUE02/304279997/mr-innovation)

    Dow might also disagree as they produce several billion dollars worth of plastics a year based on technology he developed.

    2.) Do you honestly think a professor of his stature would fabricate data to get one paper? Tobin is a smart guy. He knows that he risks law suits from Rodriguez if he goes out and says “this guy fabricated data.” So he did what any rational person would do, he contacted Northwestern’s lawyers to begin an investigation. These things take him. Can you imagine going through every lab notebook and email from every person involved in any project Rodriguez had his hands on? You are right on one front. The buck does stop with him and he is responsible for everything his group puts out. This situation is on him ultimately. But don’t you think these things should be dealt with in an organized manner (rather than group lynching as seems to often be the case in comment sections)?

    I might add that it is laughable that you describe him as a tyrant as anyone who has worked with or near him knows he is nothing of the sort. I can assure you that your speculation regarding how Rodriguez was “badgered” is completely off base.

    Again I say give it time. A real investigation is ongoing. To your last question I can say I know much more than you, and you know nothing.

  149. Umbisam Says:

    @alum: I would say publishing over 950 papers and having an H-index of 114 supports your claim. Out of curiosity, what is this billion dollar technology you speak of?

    Interestingly, if you Google Tobin Marks + DOW the sixth result is Brandon Rodriguez’s LinkedIn page.

  150. Allison (@DrStelling) Says:

    OK I was just going to stay the heck out of this one but ….. @alum: “‘Childish impatience’ pays off” in the link about Marks’s patents.

    Really? “Impatience” is a GOOD quality in a PI? Not from what I’ve seen. I’m pretty young, I’m the first to admit. But I have been in labs like these. I know how they work. I talked with their graduate students, when I was one- and later on. (If you really want to know what the PI’s like, always talk to the students.) There’s a very good reason I did my PhD with a boss known for being strict- but not totally draconian. (Ah, the Brits….)

    And above all else, certainly not someone who comported themselves like an impatient child when doing federally funded research.

    Since you appear to have interacted with him as a PI, how was he in your opinion? How hierarchical was the lab structure? How much time did each student get, one-on-one, with him to discuss their projects with him directly?

    Perhaps if the feds starting limiting the total numbers of postdocs and PhD students per PI, oversights like this would be less frequent.

    I’m sure Marks is a very smart guy. Most of the chemists that age are; it’s why I like talking to them. But I suspect if we looked at the ratio of students per PIs vs “degree quality”, we might see interesting trends.

  151. Alum Says:

    @umbisam – The work with Dow is a cocatalyst for polyolefin production.

    @Allison – I’m going to chalk the “childish impatience” up to glossy writing for a business magazine. I’ve seen similar pieces on other big names. My point of posting that was the point out that he is not overhyped based on how much money he has brought in for various things.

    I do agree with you that big labs can leave students behind and the average degree quality might be lower as compared to a great boss in a small group. That always depends on the group. Professor Marks’ lab is not very hierarchical.* The lab is essentially managed by a mix of postdocs and senior grad students. Postdocs are not in charge of grad students. The exception (*) is that there are two research professors who act as essentially coPIs for the two sides of the group (separated by research topic). He isn’t asking for meetings all the time, but there are multiple group meets a week and he is very easy to get a one-on-one meeting with. The fact that he was described above as an “unrelenting tyrant” is simply laughable. He is a nice guy, a smart scientist, and will no doubt weather this storm. I suspect the same cannot be said for Mr. Rodriguez.

  152. NMR jock Says:

    Such peaks are well known to experts in NMR and can be seen when all of the following conditions prevail:

    (1) T2>T1

    (2) Using special pulse sequences such as Proton Enhanced Nuclear Induction Spectroscopy

    shame on you Tobin Marks!

  153. Allison (@DrStelling) Says:

    @Alum- Yeah, I know- but I think we are reaching the day and age where scientists can take a bit more control over their PR and tone done the hype a notch. Given USA patent law, I’m not terribly impressed by having “lots of patents” unless someone lists what they are *for*.

    I am thinking the fact that Rodriguez graduated from this lab says quite a bit about quality control in the group.

    I’ve seen a lot of super-groups like this. They work far better in institutes, and when the PIs hire professional MS/PhDs to get the work done. (The Germans have a ton of them- some definitely work better than others. This depends less on the PI and more on team cohesion. We used to have labs like this in the States, but then CEOs figured out they could do the development work by using cheap student labor and overworking the university PIs.)

    If the co-PIs are truly “co”, do they sign off on the dissertations for the PhD students? Do they take responsibility for the quality of the student’s work, and face repercussions if they failed in their supervisory duties?

    A PhD is supposed to be a mentored degree, not a certification of skill. Yes, you must be independent; but you also need guidance, an external check on your work, and help to find your place in a sea of literature.

    @NMR jock: Heh

  154. BIP sat morning Says:

    My take.

    1. Catalysis guy’s take is way overheated. His one valid point is that PIs should take the fall given they reap the rewards. But I doubt Tobin pushed false data. It doesn’t make sense risk reward and it’s not what I’m used to from him.

    2. Marks is very smart (not super math physicist, but close) and is a polymath (orgo, metallo-organic, polymers, solid state, thin films, etc). Into all the sexy chemistries and a cut above the glitzy nano-types (he can bring real hard core chemical smarts to the party).

    3. There was a history of safety accidents in his lab. He didn’t like it. But he still did not do enough to prevent it. At a company, he would have been terminated. However, academia accepts injuries to students much more than industry does to employees.

    4. yeah, basically he ran a super group with all the issues (low ratio of student to professor, higher Chinese percentage, occasional rivalries and competition within the group, the accidents). Not a total slave labor camp. But…yeah…in that direction. He’s pretty cool to talk to if you are smart and not a part of the entity. Not super evil or anything…but I would not want him as an advisor. [But I would collaborate or fund him, or do a grant or take a class, or even just brainstorm with him in a heartbeat.] I could see him squeezing a grad student or a postdoc. But I don’t think he is the type for instance that would dig in the feet and defend wrong results (try to stop a retraction for instance).

    5. He is interested in applications (in a good way), which partially explains high patents. However, his wife was (is?) in charge of the NU department that decides what inventions to spend money on for patent applications. I have to think that has some positive impact on his proposals for patents marching through.

  155. Alex Spokoyny Says:

    “Those who have worked with Marks describe him as a unrelenting tyrant (note that NONE of his students have written to support him).”

    Being a person who worked with Mark directly, and also being in a relationship with a former Ph.D. student of his, I would like to strongly disabuse Catalysis guy of this fundamentally uninformed and incorrect position. I also think, that while open discussion of issues related to academic integrity (i.e. Rodriguez case) is critical for our community, unsubstantiated personal attacks of this sort from individuals who do not possess sufficient credibility (as well as class allowing them to reveal their real name on this blog) are counter-productive and very distracting.

  156. Antoine Lavoisier Says:

    My concern every time this happens (i.e., a retraction is published) is the same one I have when I see DEA agents catching a small shipment of cocaine: for each shipment they catch, how many pass undisturbed?

  157. Antoine Lavoisier Says:

    Why doesn’t JACS give the names of the reviewers?

  158. Deport Bieber Says:

    Ladies and Gentlemen,

    Here’s the latest one from superchemist, shining star of Dow Chemical *drumroll* Mr. Rodriguez. Enjoy.

    http://pubs.acs.org/doi/abs/10.1021/om401131m

  159. eugene Says:

    Someone told me that I was the ‘anonymous source’ for those S4-S6 spectra based on the comments on this post. I forgot all about it by now and had to search for my original comment, but hey, pretty cool. It’s nice to know people take pseudonymous blog comments seriously.

    I’ll try to write with more gravitas from now on and I’m seriously considering not calling the journal of the american chemical society, Jackass.

  160. Rhenium Says:

    Interesting to see that the “corregidum” does not address the fake peaks or reuse of the base line.

  161. Somebody Says:

    The “best” person to represent Dow Chemical:
    http://www.bitlifesciences.com/GCC2014/program-5.asp
    (Day 3: Afternoon, Tuesday, September 23, 2014, 13:55 pm)
    I’ll bet it will be a high quality talk.


Leave a Reply

*