You know that arsenic-loving bacteria briefly mistaken for an alien? The bug, which roiled the science press earlier this week when it rode an unusually high and steep hype parabola, endured a firey re-entry today when University of British Columbia bacteriologist Rosie Redfield ripped into both the research and the paper — and was quickly backed quickly by several scientists, including heavyweight extremophile-bacteria researcher Jonathan Eisen.

I can’t claim to follow the technicalities of this argument, so can’t pass judgment on the research itself. Fortunately for clarity’s sake, Redfield’s critique, though it gets quite technical, does not leave you wondering what the author really thinks of the paper under the microscope. On Twitter she calls it “shamefully bad science.” She’s just as frank in the analysis on her blog:

Bottom line: Lots of flim-flam, but very little reliable information.… If this data was presented by a PhD student at their committee meeting, I’d send them back to the bench to do more cleanup and controls.

There’s a difference between controls done to genuinely test your hypothesis and those done when you just want to show that your hypothesis is true. The authors have done some of the latter, but not the former.…

I don’t know whether the authors are just bad scientists or whether they’re unscrupulously pushing NASA’s ‘There’s life in outer space!’ agenda. I hesitate to blame the reviewers, as their objections are likely to have been overruled by Science’s editors in their eagerness to score such a high-impact publication.

I don’t know Redfield, so didn’t know exactly what to make of her critique. But her opinion was quickly seconded in the blog’s comments and on Twitter by many sharp scientists. UC Davis’s Jonathan Eisen, who knows his extremophile bacteria and and his evo-sci quite well, tweeted that he “was gonna write about bad science & arsenic but no need – Rosie Redfield has all you need.” Evolutionary biologist John Hawks thought the Science paper clearly lacked the necessary controls to draw its conclusions. And Alex Bradley concluded that “this study lacks any real evidence for arsenate-based DNA; unfortunately these exciting claims are very very shaky.”

If the paper is as weak as these critiques hold, NASA appears to have been not just overzealous but reckless — and Science not only went along for the ride, cheering wildly, but put all the gas in the car.

I imagine — and certainly hope — that we’ll get a better picture of whether that’s the case as the technicalities get batted about. Meanwhile, what does this say about the state of science journalism? I think the answer there depends on what happens from here on out. The overhyped stories that appeared before and just after the embargo lifted were obviously a problem. But much of the press — both outlets-formerly-known-as-mainstream such as Nature News and the New York Times and high-quality-science-writers-formerly-known-as-just-bloggers, such as Ed Yong, did a good job correcting these. These same outlets tended to extend a benefit of the doubt to the paper, as seems reasonable enough if they hadn’t run into sharp doubts. True, most of these measured accounts tended to fall into a tone of “damned interesting, and I mean damned interesting … with caveats,” and you might want to complain that they hadn’t caught the slop. But as a version of my old favorite “Interesting if true,” I can accept that — especially given the constrictions imposed by the short-turn embargo system, which lets journalists have exclusive access to papers a few days, but only a few days, before they are published, in exchange for promising they won’t publish on them till a certain date and time. (I’d feel differently if I knew a reporter heard but dismissed truly sharp criticism such as that that Redfield has leveled, if it came from someone as reputable as Redfield appears to be.)

Here’s the problem: When a paper is still under embargo and we journalists call an outside expert to get comment on it, the expert has often not actually seen the paper yet, since, well, it’s under embargo. If time allows (often not, since one usually has only a few days and everyone is busy) then you can send the expert(s) the paper, and they can read the paper and get back. But as the experts usually lack time to compare impressions with peers, few will go out on a limb and really lay into a paper under those circumstances. You usually get either “This looks interesting, with a few caveats I’d like to note” or “I’d rather not comment.” You’ll rarely get an outright dismissal. They lack the time and probably the taste for the trouble it’ll make.

So we get what we got: some measured check on the hype from the best journalists, but mainly, since the claimed findings are ambitious, an esepcially fascinated version of “This looks damned interesting, and is truly interesting if true, and [this next part usually doesn’t get actually written] I sort of hope it is true, ‘cuz it’s cool.” I don’t think there’s much wrong with that if you follow through and cover objections that rise down the road.

What to make of this? First, this is one of those reminders — every science writer gets one now and then — that any of us can get excited enough to overlook some slop, especially if the slop is technical in nature. The test is in how you react afterwards.

It also attests to the immense value of having scientists that blog, despite that they get little credit (and often some flak) for doing so. No matter how this sugars out, Redfield’s use of her blog to call BS when she sees it has put science’s self-correcting feature — far too often asleep these days — into high gear.

Finally, as Ivan Oransky and many others have pointed out, this episode illustrates how utterly ridiculous and toxic the embargo system can be, especially when it is systematically and intentionally exploited to drum up hype. The NASA press release shamelessly used words like “astrobiology” and “extraterrestrial life” to tease the biggest response they could get. True, NASA never said they’d discovered extraterrestrial life. But if you think that you can throw language like that around in a press release that announces a high-profile news conference with five scientists, including the director of the astrobiology program, and not seriously risk generating speculative stories about aliens, then you’re not thinking empirically.

It was entirely predictable that they’d get stories about alien life. You could repeat this experiment 40 times and you’d get the same result. To complain that, Well dammit, the press release didn’t say “We found aliens!”, and then argue that the real problem was that some of the more impatient quarters of the press, Can you believe it?, jumped to that conclusion and overhyped the story anyway, is to indulge in fantasy. It’s like laying out chocolate cupcakes in front of a table of hungry two-year-olds, leaving the room saying, “I’m just fetching the broccoli for you little darlings, so hold on now, you know what’s good for you, save those cupcakes for later” — and then professing surprise and disappointment when you return to find that … “Why, the cupcakes are just GONE!” Please to give me a break.

We thought we were getting cupcakes. Some of us wanted cupcakes. Who doesn’t want cupcakes? Now everybody’s got humble pie in front of them, quite a bit to eat yet, and no dessert on the menu.