How deep does the misreporting rabbit-hole go?

A new paper from Ben Goldacre and nine colleagues has been published in BMC journal Trials, examining discrepancies between pre-specified and reported outcomes.

The paper, titled COMPare: a prospective cohort study correcting and monitoring 58 misreported trials in real time documents the extent to which misreporting occurs in published papers in five journals endorsing Consolidated Standards of Reporting Trials (CONSORT): New England Journal of Medicine, The Lancet, Journal of the American Medical Association, British Medical Journal, and Annals of Internal Medicine)

The study assessed all trials over a six-week period to identify every correctly and incorrectly reported outcome. The group attempted to determine whether correction letters could be published for every misreported trial and tracked the responses from journal editors to understand why outcome misreporting persists despite public commitments to address it.

The authors declare this "the first empirical study of major academic journals’ willingness to publish a cohort of comparable and objective correction letters on misreported high-impact studies."

The paper suggests interventions to improve the effectiveness of dealing with misreporting, such as; changes to correspondence processes, post-publication peer review, changes to CONSORT’s mechanisms for enforcement, and strategies for research on methods and reporting.

The implication here is that such a severe problem in the top medical journals may indicate a much worse problem in others that don’t have the resources of these titles.


Goldacre et al. COMPare: a prospective cohort study correcting and monitoring 58 misreported trials in real time. Trials (2019) 20:118
https://doi.org/10.1186/s13063-019-3173-2

Comments