PubPeer as an anonymous peer review tool

Since 2012, the PubPeer online forum has been playing the role of 4chan for science, allowing everyone to post anonymous comments about research. Initially, it was intended to discuss methods and results, but now it has become one of the most well-known information and analytical centers for accusations of scientific errors, fakes and misbehavior. Thanks to him, the journals produced corrections, withdrew materials, demolished careers, and eventually involved the site in a trial where his lawyers were a team from the American Human Rights Association (ACLU), who worked with Edward Snowden.



Critics believe PubPeer supports an uncensored stream of irresponsible accusations. But supporters say PubPeer is perhaps the only effective way to expose errors and fakes in the current scientific system. It works during the silent crisis of science and scientific journals, when the community is concerned with the problems of originality of results - the so-called. " crisis of reproducibility ", and the number of withdrawn works increases . The traditional system of expert assessments does not cope with these problems.



“We launched the project because we wanted to get more detailed arguments on science and were shocked at how many problems there are with scientific work, including very controversial research practices and quite obvious abuses,” says Brandon Stell [Brandon Stell], Center for Scientific Research in Paris, the creator of PubPeer.

')

And there is nothing to argue. According to the Retraction Watch blog, which tracks corrections, errors, and fakes in scientific work, at least three highly rated scientists have been revoked by journals in recent months after their data have been questioned by anonymous commentators from PubPeer.



One of the scientists, Fazlul Sarkar, is suing several commentators. His lawyers argue that the site should give the identity of users who have damaged his career - he lost his job at Mississippi University. PubPeer refused to do this. Google and Twitter wrote a text for the court in support of the site, which is now pro bono defended by lawyers from the ACLU.



Perhaps this is one of the most interesting cases related to online privacy, of those that you have not heard. All this grows out of frustration among scientists with an incomprehensible policy of publications and expert evaluations of works.



In fact, PubPeer is a site that allows everyone to comment on any scientific article in the state PubMed database , both anonymously and with a signature. It is very simple, but the possibility of anonymous comments makes it a safe place for scientists - especially young ones - to discuss and criticize research without fear of punishment. And they really want to do this: since its launch, there are already more than 55,000 anonymous comments.



In October 2013, someone started discussing about 20 published papers by Fazlul Sarkar, a cancer researcher from Wayne State University in Michigan. The papers have been published for more than 10 years, and describe various complex molecular signaling systems for cancer. But the questions raised by the comments were direct: commentators argued that the images in different works were copied and slightly modified, which hinted at the possibility of the absence of real experiments - or that the experiments could lead to other results.



Stell noted that in order to maintain order and law in the debate, PubPeer requires commentators not to blame the authors directly for fraud. Comments are moderated, so such accusations are always rather quiet.



But this does not make self-appointed watchmen less effective. The worst thing a researcher can see at PubPeer is “there are problems”.



The discussion of “problems” in Sarkar’s works quickly grew when it became clear that commentators found a gold mine: according to a database from the State Institute of Health and PubMed, Sakar received more than $ 12 million in funding, and published over 500 works. Commentators work quite carefully - they are known for often digging up dissertations in decades in search of "dirt". In general, the forum can be found 77 works Sarkar, subjected to careful study. If you compare them with each other, trends appear. For example, one user claims that the same set of images was reproduced 54 times in 13 papers over 3 years. Apparently, the public has revealed an active near-science photoshop, which for ten years has not been noticed by committees, editors and experts of prestigious journals.



Many site visitors adopt this type of investigation. “I came to PubPeer partly because of disappointment,” says Elizabeth Beek, a microbiologist at Stanford University and a regular PubPeer user. She says that she comments on her own behalf discussing the scientific advantages of the article, but remains anonymous in another area of ​​activity: the detection of manipulations and fakes of images in the works. Bick says that in her discussions she did not touch the works of Sarkar.



“In some cases, I complained to the editors of magazines,” she says. “It was the most honest way, which also makes it possible for the authors to answer, but I was disappointed to see that many editors did not answer me, or answered something like“ We are investigating this question, ”and then silence.”



Other scientists talked to me about the same thing, and they did not leave the feeling that they were trying to cover the one whom they criticized.



In response to our question about the concerns of scientists, the representative of Nature Research responded by e-mail that the work submitted to them "undergo careful expert control" and it is assumed that the work was performed "in good faith." They are automatically checked for plagiarism and image manipulation, and editors, together with experts, can correct something or withdraw work in case of suspicion.



“We take seriously all suspicions about the published works, they come from specific people or from anonymous sources, and we consider all cases separately,” they wrote.



A Cell Press spokesman also argues that editors seek to correct scientific records, noting that "because we maintain the confidentiality of investigations, we do not disclose their details to people who shared their suspicions."







Many observers do not believe that magazines and other intermediaries do everything they can. “For decades, researchers and editors have ignored anonymous and critical comments until they sought identification,” says Ivan Oransky of Retraction Watch. He says that critics and informers are constantly turning through his website, but the volume of these complaints is too large for a small staff of journalists. According to him, PubPeer provides an outlet for critics.



Charging a forgery is not a toy for you, especially for the accused. The accusations of Sarkar were not just idle exercises: as is often the case with popular threads on PubPeer, people sent the problems found through all the official channels that could do at least something, including the journals and employees that published these studies.



Unfortunately for Sarkar, this happened at a critical moment in his career. According to court documents obtained by Retraction Watch , he resigned from Wayne State University in June 2014, accepting a prestigious proposal from the University of Mississippi that included opening a laboratory and a salary of $ 350,000. But only 11 days before the proposed start of work, when he I looked after a house in Oxford, the offer was withdrawn. And all because of the investigation, which began at PubPeer.



According to court documents, in a letter dated June 19, 2014 from Larry Walker, director of the State Research Center for Natural Products at the University of Mississippi Cancer, and Sarkar’s main contact, said that Walker had received quotes from an anonymous (?) Source PubPeer.com, concerning several works of Sarkar. The letters were allegedly sent to other researchers at the university, with the result that all of Sarkar's new colleagues were in the know before he even appeared in the workplace.



Sarkar, in response, sued the university for withdrawing the proposal and to several anonymous commentators with PubPeer for defamation. In January 2016, the judge closed the case against the University of Mississippi, but the case against PubPeer is still pending. Sarkar's lawyers claim that commentators did not just discuss his work, but attacked him personally. They sent PubPeer a court request for issuing names and IP addresses of commentators.



PubPeer started as a hobby of one young scientist and his colleagues, so he was not well prepared to defend in court. But his credo as an anonymous platform for informers led to the fact that he was on the wave of debates about privacy and the disclosure of personal data, simmering a year after the first release of the NSA documents Snowden.



“I received this letter from an anonymous writer who was writing off with Ben Wiesner, and it will be interesting for Ben to talk to us about it,” says Stell. Ben Wiesner, director of the ACLU “Speech, Privacy and Technology” project and one of the lawyers for Edward Snowden. “They told me that he wanted to help, and I was terribly happy!” The ACLU quickly organized a team that started protecting PubPeer for free.



“It’s not that people are thrown into the dungeons, but scientific discussions suffer if scientists cannot speak freely without fear that their current or future bosses will find out about this,” said Alexander Abdo, chief lawyer at the ACLU and one of the team members working on a PubPeer case. “You need to burn all the bridges before you begin public criticism. Anonymity gives room for maneuver. ” “I think PubPeer is trying to use anonymity for the same reason that the constitution protects it. Anonymity protects you from those who could punish you for your views, ”he adds.



Stell, founder of PubPeer, agrees. “If we try to express our opinion about the work of our colleagues, this can lead to serious consequences for our careers,” he says.



And Nicholas Roumel, a partner in NachtLaw, a Michigan-based company representing Sarkar, argues that anonymous commentators are better protected than their goal, which cannot even address them by name. “Of course, anonymous commentators can be defended by the First Amendment. The problem is that it is necessary to protect anonymous evidence more than non-anonymous. Cowards who post anonymous accusations deserve protection more because they fear persecution — but they undoubtedly destroy careers in the way that happened to Dr. Sarkar from an ambush. This is hypocrisy. ”



Rumel, who indicated that he is a member of the ACLU and the chairman of the local committee of the ACLU, argues that his client is judged harsherly than his accusers, and that by sending pamphlets in support of PubPeer, giants like Google and Twitter agree with the idea that he considers alarming. “They say that victims like Dr. Sarkar should have a harder time proving their innocence by judging anonymous,” he says. Sarkar himself did not comment on this story.



So far, the case is in a stalemate. In March 2015, the judge rejected a request for the issuance of data on PubPeer users - true, except for one. Sarkar’s lawyers considered his contribution to the discussion to be the most harmful. PubPeer ordered to give all the information on this user, including the IP address.



The ACLU appealed this decision, and Sarkar’s lawyers appealed the judge’s decision to reject requests for information on other users. Hearings are scheduled for October.



What is interesting, according to Sarkar's attorneys, that malicious user used an email from Claire Francis. Francis is a pseudonym, using which, various scholars from about 2010 have already sent hundreds (and possibly thousands) of anonymous letters to the editors of journals. The fruitfulness of the pseudonym forced the giant publisher Elsevier to issue special rules for editors, prescribing the order of processing these letters. Undoubtedly, this was influenced by the habit of "Francis" to send a copy of letters to the New York Times.



It turns out that this legal case could potentially uncover one of the most long-lived mysteries of scientific publications and reveal the person or persons responsible for the phenomenon called "Claire Francis."



Although it slows down, Sakar's career is damaged. Already thirteen of his works were withdrawn from the journals that published them (six of them at once — in one month ), and Sarkar’s name is on the list of those who quit Wayne University this year.



But many members of the scientific community are not in a hurry to trust too much the possibilities of anonymous society. The editors of two major journals oppose such criticism . Michael Blatt, the editor of Plant Physiology, directly addressed PubPeer. In the editor's column, Blatt agreed that there should be a way to combat fraud in his field, plant biology. At the same time, he says that “anonymity is not an answer if your goal is legal methods and protection of innocent people”.



Many PubPeer users have expressed concern that anonymous commentators may pursue personal goals or make mistakes. “There are definitely cases in which it is clear that the commentator has ulterior motives,” one said. Another suggested that in some cases, a former student or employee may make unsubstantiated accusations due to poor collaboration experience.



But in the case of sincere intentions there is the danger of erroneous accusations. Bick, a microbiologist from Stanford, said that she, once trying to expose manipulation of images through PubPeer, was mistaken. The simplest cases of false accusations are quickly disassembled either by the commentators themselves or by the authors of the works. But the authors are unpleasant that these discussions in public access are related to their works. Bick says that has since become more accurate.



But scientific frauds may be more common than we think. Bik turned his interest in exposing image manipulations into a separate topic for research, and recently even published a paper on it . Together with the co-authors, they showed that almost 4% of the images from the 20,000 works reviewed are “problem”, many of them contain signs of manipulation. In their work, they write that if the author has one such image, it is likely that there are others, while in some journals this problem is more common than in others.



So far it turns out that the best way to pay attention to such problems is an anonymous message. As Orange of Retraction Watch pointed out, PubMed itself recently appeared a commenting system tied to email addresses of scientists, allowing them to leave comments on their real name under the works. But it does not give comparable results. “There are no cases of recall of work associated with the comments. Anonymity is the key. ”



Stell says that although he hopes to inspire people to use real names and develop scientific discussions, he still plans to support PubPeer as an effective platform for informants. He says that the site increases the anonymity of users, keeping as little information about them as possible, making it easier to access the site through Tor, and is working to inspire more people to use real names.



The idea is that when all scientists, and not just selected experts, can evaluate research in a public forum, fakes and useless data will no longer be such a problem. But as long as the founders of PubPeer lead users from a rigid hierarchy of scientific journals to better scientific discussions, the community can continue its work as a restless conscience for the powerful, using anonymity and the Internet to clean up science in ways available to them.