Suspicious Statistical Patterns Found in Recent Clinical Trials: Implications for Medical Journals

Review of Clinical Trial Data in Medical Journals

Dozens of recent clinical trials contain suspicious statistical patterns that could indicate incorrect or falsified data, according to a review of thousands of papers published in leading medical journals.The study, which used statistical tools to identify anomalies hidden in the data, has prompted investigations into some of the trials identified as suspect and raises new concerns about the reliability of some papers published in medical journals.The analysis was carried out by John Carlisle, a consultant anaesthetist at Torbay Hospital, who previously used similar statistical tools to expose one of the most egregious cases of scientific fraud on record, involving a Japanese anaesthesiologist who was found to have fabricated data in many of his 183 retracted scientific papers.In the latest study, Carlisle reviewed data from 5,087 clinical trials published during the past 15 years in two prestigious medical journals, Jama and the New England Journal of Medicine, and six anaesthesia journals. In total, 90 published trials had underlying statistical patterns that were unlikely to appear by chance in a credible dataset, the review concluded.“This raises serious questions about data in some studies,” said Carlisle. “Innocent or not, the rate of error is worrying as we determine how to treat patients based upon this evidence,” he added.Dr Andrew Klein, the editor-in-chief of Anaesthesia, which has published the new analysis, called for the studies identified as potentially flawed to be reviewed urgently.“It’s very scary that we may be treating patients based on false evidence.” he said. “It may be the case that certain treatments may need to be withdrawn from use.”The tool works by comparing the baseline data, such as the height, sex, weight and blood pressure of trial participants, to known distributions of these variables in a random sample of the populations.If the baseline data differs significantly from expectation, this could be a sign of errors or data tampering on the part of the researcher, since if datasets have been fabricated they are unlikely to have the right pattern of random variation. In the case of Japanese scientist, YoshitakaFuji, the detection of such anomalies triggered an investigation that concluded more than 100 of his papers had been entirely fabricated.Klein said that in some cases the anomalies could be put down to “misinterpretation, statistical error or plain simple mistakes”, such as transcribing data wrongly or mislabelling a column.The latest study identified 90 trials that had skewed baseline statistics, 43 of which with measurements that had about a one in a quadrillion probability of occurring by chance.The review includes a full list of the trials in question, allowing Carlisle’s methods to be checked but also potentially exposing the authors to criticism. Previous large scale studies of erroneous results have avoided singling out authors.Relevant journal editors were informed last month, and the editors of the six anaesthesiology journals named in the study said they plan to approach the authors of the trials in question, and raised the prospect of triggering in-depth investigations in cases that could not be explained.Klein called for journals to adopt similar screening tools, in the interests of scientific integrity. “There is no excuse for any medical journal reporting randomised controlled trials not to [do so],” he said.Sir David Spiegelhalter, professor for the public understanding of risk at the University of Cambridge, said that the outlier papers ought to be looked at closely, but cautioned against attributing a cause for the discrepancies at this stage. “This is a real danger of a classic ‘prosecutor’s fallacy’,” he said. “Just because fabricators had a good chance of producing non-random data, does not mean that non-randomness means a good chance of fabrication.”A spokeswoman for the New England Journal of Medicine said they had not had access to the list of papers cited until publication, but that the editors were taking the issue seriously and would carefully review all information.Howard Bauchner, the editor-in-chief of Jama, also responded, saying: “We receive numerous allegations about various issues related to the articles we publish. After we assess the validity. We certainly believe authors have a right to respond to allegations that are important.”Question 1(a)Discussat least FIVE(5)possiblereasons why researchers may falsify their research data. Your answer should includeadditional reasons not found in the article.((b)You are workingas a research assistantunder a well-liked research scientist. While going through the clinical data collected, you found that he may have falsified the research data in order to get the research published. Examine the ethical issuespresent in helpingyou decide towhether or notreport him to the authorities.(c)Ifyou arefrom a government agency in charge of scientific research, recommendthe various stepsyou would take to prevent such incidents from happening in the future.