6/23/18, "Some science journals that claim to peer reveiw papers do not do so," Economist.com
"One estimate puts the number of papers in questionable journals at 400,000."
"WHETHER to get a promotion or merely a foot in the door, academics have long known that they must publish papers, typically the more the better. Tallying scholarly publications to evaluate their authors has been common since the invention of scientific journals in the 17th century. So, too, has the practice of journal editors asking independent, usually anonymous, experts to scrutinise manuscripts and reject those deemed flawed—a quality-control process now known as peer review. Of late, however, this habit of according importance to papers labelled as “peer reviewed” has become something of a gamble. A rising number of journals that claim to review submissions in this way do not bother to do so. Not coincidentally, this seems to be leading some academics to inflate their publication lists with papers that might not pass such scrutiny.
"WHETHER to get a promotion or merely a foot in the door, academics have long known that they must publish papers, typically the more the better. Tallying scholarly publications to evaluate their authors has been common since the invention of scientific journals in the 17th century. So, too, has the practice of journal editors asking independent, usually anonymous, experts to scrutinise manuscripts and reject those deemed flawed—a quality-control process now known as peer review. Of late, however, this habit of according importance to papers labelled as “peer reviewed” has become something of a gamble. A rising number of journals that claim to review submissions in this way do not bother to do so. Not coincidentally, this seems to be leading some academics to inflate their publication lists with papers that might not pass such scrutiny.
Experts debate how many journals falsely claim to engage in peer review. Cabells, an analytics firm in Texas, has compiled a blacklist of those which it believes are guilty. According to Kathleen Berryman, who is in charge of this list, the firm employs 65 criteria to determine whether a journal should go on it—though she is reluctant to go into details. Cabells’ list now totals around 8,700 journals, up from a bit over 4,000 a year ago. Another list, which grew to around 12,000 journals, was compiled until recently by Jeffrey Beall, a librarian at the University of Colorado. Using Mr Beall’s list, Bo-Christer Björk, an information scientist at the Hanken School of Economics, in Helsinki, estimates that the number of articles published in questionable journals has ballooned from about 53,000 a year in 2010 to more than 400,000 today. He estimates that 6% of academic papers by researchers in America appear in such journals.
Behind all this is a change in the way a lot of journals make their money. Over the past decade, many have stopped selling subscriptions. Instead, they charge authors a publication fee and permit people to read the result for nothing. This “open access” business model has the advantage of increasing the dissemination of knowledge, but it also risks corrupting the knowledge thus disseminated.
Trouble with lichen
The truth of this was shown as far back as 2013, in an experiment conducted by John Bohannon, a journalist with a doctorate in molecular biology. Dr Bohannon set up a sting operation by writing versions of a paper falsely claiming that a molecule found in lichens inhibits cancer. The papers featured, he says, “laughably bad” methodology and a shocking conclusion that the molecule is “a promising new drug” despite an absence of clinical trials. He attributed the papers to fictional biologists at made-up African medical institutes and then submitted them to open-access journals. Of 121 chosen from a blacklist, 69% offered to publish the paper for a fee, and even when he turned to journals on a whitelist of supposedly trustworthy open-access journals, 38% of the 167 he approached fell into the trap.
The truth of this was shown as far back as 2013, in an experiment conducted by John Bohannon, a journalist with a doctorate in molecular biology. Dr Bohannon set up a sting operation by writing versions of a paper falsely claiming that a molecule found in lichens inhibits cancer. The papers featured, he says, “laughably bad” methodology and a shocking conclusion that the molecule is “a promising new drug” despite an absence of clinical trials. He attributed the papers to fictional biologists at made-up African medical institutes and then submitted them to open-access journals. Of 121 chosen from a blacklist, 69% offered to publish the paper for a fee, and even when he turned to journals on a whitelist of supposedly trustworthy open-access journals, 38% of the 167 he approached fell into the trap.
Dr Bohannon’s experiment did lack a true control—submission to journals that still charge subscriptions. Nevertheless, his findings were worrying and since then, he says, “things have only gotten darker”.
One aspect of that darkness is that compiling a blacklist has itself become risky. Mr Beall stopped adding to his last year and left his job at the University of Colorado in March. He claims to have been subjected to pressure from a superior, to a research-misconduct investigation by the university and to threats of lawsuits by publishers. The university, for its part, says that no pressure was put on him to take down the list. As far as it is aware that decision was his, and his job was never in jeopardy because of his work researching open-access journals. It cannot, however, disclose whether or not there was a research-misconduct investigation. Disclosure happens only after a finding has been made in such an investigation. Mr Beall’s list has been taken up by another researcher who has since appended 690 new journals to it. But this new custodian refuses to be named.
Meanwhile, at Cabells, Ms Berryman reckons the publishers of bogus journals are getting ever cannier. She has seen cases of journals she regards as suspect claiming to be on whitelists, fabricating citation scores for papers, stating plausible time frames for peer review (claims of rapid review are often associated with questionable journals) and brazenly listing as sitting on their editorial boards scholars who are not in fact doing so.
Ms Berryman says, too, that some websites copy wording and graphics used by legitimate journals. Other sites go further, assuming a name that is confusingly similar to that of a reputable journal. And according to Ivan Oransky, co-founder of Retraction Watch, a blog that monitors such matters, questionable journals now also occasionally retract articles in a bid to appear responsible, in what can only be described as a superb piece of subterfuge.
None of this would matter if institutions hiring academics were appropriately vigilant about checking candidates’ publication histories against blacklists, and similarly inquisitive about the publications of those already employed. But some, apparently, are not. According to Brian Nosek, head of the Centre for Open Science, a not-for-profit organisation in Virginia that studies the matter, many institutions that hire and promote researchers seem unconcerned about where those researchers have been publishing—a problem made worse by recent requirements by the American and Canadian governments that taxpayer-funded research must be published in open-access journals.
Unsurprisingly, this is an area in which data are hard to come by. But one academic has been prepared to stick his neck out and investigate his own institution. Last year Derek Pyne, an economist at Thompson Rivers University’s business school, in British Columbia, published a paper in the Journal of Scholarly Publishing, itself published by the University of Toronto Press. In it, he reported that many of the business school’s administrators, and most of its economics and business faculty with research responsibilities, had published in journals on Mr Beall’s blacklist.
Dr Pyne also claimed that these papers seemed to further their authors’ careers. Of the professors who had published in the blacklisted journals, 56% had subsequently won at least one research award from the school. All ten instructors promoted to full professor during the study period had published in a journal on Mr Beall’s list.
Subsequently, Dr Pyne told school officials that an administrator up for promotion had published widely in blacklisted journals. This earned Dr Pyne an e-mail from the university’s human-resources department on June 15th, threatening him with disciplinary action for “defamatory language and accusations”. When asked, the university declined to comment.
Review peer review
What can be done about all this is hard to say. Dr Pyne thinks part of the problem is that too many academic administrators have no research experience, and so either cannot tell good publications from bad, or do not care. Few researchers, though, thrill to the idea of a career in administration, so changing that might be difficult.
An extreme reaction, albeit one supported by a growing minority of researchers who think the peer-review system is anyway creaking under the weight of publication pressure, would be to abandon anonymous peer review altogether, and make the process open and transparent. This could be done (as sometimes happens already) by publishing unreviewed papers on special servers and then inviting criticism conditional on the name of the critic being public. That, though, brings other risks. Anonymous critics often find it easier to be honest, especially in fields where most researchers know each other.
One far-fetched solution is a return to journal subscriptions. These have for so long been excoriated as rent-seeking profit-inflators restricting the flow of information that a change of course would now be unthinkable. But those who pushed for their elimination might be wise to pause for thought. As the old proverb has it, be careful what you wish for. You might get it."
No comments:
Post a Comment