“The first thing you feel when a trial fails is a sense of shame,” says physician Siddhartha Mukherjee. Then comes the “exhilarating — and dangerous” instinct to slice and dice the data to resurrect the drug’s potential.
Here Siddartha discusses the lure to make more of the science than is perhaps there, when stakes are high and we are desperate to find hope for patients who are out of treatment options.
As we drive towards more personalized medicine, he reiterates the importance of identifying study subgroups before a trial takes place and not to rely on post-hoc analyses that may not be statistically robust. I particularly enjoyed the example he gives of the paper in which he submitted an “astrological subgroup” analysis showing a drug worked better in Capricorns, when pressed by the publishing Editorial Board to include post-hoc assessment of age- and gender-specific subgroups.
This also leads to the question of what to do with the evidence you have just demonstrated and the importance of making sure that negative trial results are appropriately recorded and made publicly available. This is vital for transparency around individual drugs as well as the wider understanding of the potential efficacy and safety of similar drugs. In fact, without the balance of negative results you could risk veering into coin-flipping territory. Here you can read about how under-reporting of negative trial results can dramatically skew the assessment of a drug and endanger patients. Some regulatory bodies, such as the US FDA, now mandate the publication of negative trial results to a formal registry e.g. ClinicalTrials.gov, but it is important to research global trials from multiple international registries to get the full picture. Interestingly, industry seems to be leading the way in the public dissemination of all trial results. It should be noted, however, that what is shown in the registry is not always subsequently reflected in the final publications of a study, in particular in relation to serious adverse events. In this analysis only 11% of trials were published with numbers matching those reported in ClinicalTrials.gov.
So, check out these three articles to make sure you are unearthing biology not astrology in your research.
Image credit: jackaldu / Getty Images / iStock