ISIS Report 01/11/07

Actonel, Dog that Did Not Bark in the Night

Data suppression and manipulation, victimisation of scientist, abuse of science and more. Prof. Peter Saunders

A fully referenced version of this article is posted on ISIS members’ website. Details here

An electronic version of this report, or any other ISIS report, with full references, can be sent to you via e-mail for a donation of £3.50. Please e-mail the title of the report to: report@i-sis.org.uk

An elusive ‘threshold’

Over a year ago, we reported on the controversy surrounding a drug commonly prescribed for osteoporosis [1] (Actonel: Drug Company Keeps Data from Collaborating Scientists, SiS 30).  The issue was whether Actonel made by Proctor & Gamble (P&G), is as good as its competitor, Novartis’ Fosamax. Both act by inhibiting cells involved in bone turnover, with very similar side effects (see Box), but it is generally accepted that Fosamax is the more effective [2]. That does not necessarily make Fosamax better at reducing fractures, because it might be that once the turnover rate has been brought down to a certain threshold level, there is nothing to be gained by reducing it further.

In 2003, a group of scientists at Sheffield University led by Richard Eastell and funded by Proctor and Gamble (P&G), claimed to have found just such a threshold [8].  The work soon became controversial, largely because while the crucial result hinged on the existence (or not) of a threshold, the senior author, when asked at an important meeting in 2002, was unable to explain what evidence it was based on. Only one of the authors, a statistician employed by P&G, had seen the actual data.

Aubrey Blumsohn, who was to lead the next stage of the research, was uneasy at having his name on papers when he had seen neither the raw data nor the statistical calculations. Eastell therefore suggested to P&G that the Sheffield group should be given the data and allowed to analyse it themselves in parallel with the company. This was refused on the obviously spurious grounds that it would not allow the industry to “demonstrate its ability to be a true partner in scientific endeavours.” They also said it was standard practice for industry not to allow access to raw data, even to the people who had collected it. (The experimenters’ own records would not give them the information because clinical trials are generally double blind and only the industrial partner has the key.)

The more Blumsohn looked into the matter, the less confidence he had in the original paper. He points out that the graphs in the published paper [8] had been truncated in a way that omitted about 40 per cent of the data, and argues, after consulting with experienced statisticians, that the statistical analyses used were inappropriate. As is usual in such experiments, instead of measuring bone resorption directly, the experimenter measures the level of one or more marker substances, which are assumed to reflect the resorption rate. Eastell and his colleagues had used two, CTX (carboxyterminal cross-linking telopeptide of bone collagen) and NTX (aminoterminal cross-linking telopeptide of bone collagen), but even they had found a threshold in only one of them, CTX, which, Blumsohn explains, is widely accepted as less significant.

Above all, the refusal of P&G to allow access to the original data violates one of the basic principles of science, which is that you must be able to back up any claim you make. Here, not only were the data not made available to anyone who doubted the conclusion, they were not provided even to the authors of the paper, apart from the one who was an employee of P&G. Yet all the authors signed the journal’s required declaration that they had had access to the data; this is clearly stated at the end of the paper.

When Blumsohn tried to raise the matter, Sheffield University refused to support him, and when the Times Higher Educational Supplement (THES) reported his case, the University began disciplinary proceedings against him for not following proper internal procedures. In the end, he left Sheffield University on terms that have not been made public.

New developments

The incident continued to receive publicity and the controversy refused to die down. Eventually, and it seems largely because of pressure from the media, Eastell agreed to write a response in the same Journal of Bone and Material Research (JBMR). P&G finally provided Eastell with the data in May 2006, and he asked a statistician, also at Sheffield University but not previously involved in the project, to assist in a re-analysis. A second statistician, not at Sheffield, was then sent the new analysis and the data for comment. Given the nature of the controversy it would seem unsatisfactory that Eastell chose the statisticians himself and that the one who carried out the re-analysis was from Sheffield University, which Blumsohn had left on far from friendly terms.

The results of the re-analysis were published in October this year [9], but even if we take them as definitive (there is as yet unpublished work by two leading statisticians who argue strongly that they are not  [10-11]) they do very little to improve the situation from the point of view of P&G. Writing in the same issue of JBMR, the editor and the chair of the Publications Committee of the American Society for Bone and Mineral Research conceded the first two conclusions of the 2003 paper, which relate to the efficacy of anti-resorptives and are consistent with earlier studies [12]. As for the third conclusion, which concerns the crucial issue of the threshold, they point out that while the re-analysis also found that there was a level below which further reduction in CTX was not associated with greater reduction in fracture incidence, this was not the case for the other marker, NTX. How this difference can have arisen when the two are supposed to measure the same phenomenon is, they add, yet to be explained. This contrasts sharply with the confident last sentence of Eastell’s response: “Thus, the third conclusion can still be supported based on the new analysis” or his statement to the THES that his letter was “the final word on the topic.” [13]

In the light of this, why did the editor of JBMR allow Eastell and his co-workers to include that last sentence in their response? He clearly does not accept that they have established there is a threshold; he should have insisted on a much more cautious statement of what they have achieved. As it is, despite the editor’s publicly stated reservations, the statement that there is a threshold has been allowed to become part of the literature, i.e. science that later workers and regulators will rely on as having been established.

What can we learn from this?

What do these papers tell us about Actonel? Even if the threshold exists, and there is still considerable doubt about that, then it is no better than Fosamax at preventing fractures. If there is no threshold, then it is not as effective. Unless there are other factors, such as different side effects, to consider – and there don’t appear to be (see Box) - that must make Fosamax or some other bisphosphonate better at reducing resorption a preferable choice. Assuming, that is, we decide to use any anti-resorptive at all.

There are broader issues here. There have been far too many cases where industries have interfered with the normal process of science and science publishing and often with the journals playing a far from professional role [14, 15] (see Science and Scientist Abused  and Biotech Canada SLAPP Scandal, SiS 36). The industries have sought to prevent researchers from publishing results that were contrary to what the sponsor wanted to hear, they have asked academics to put their names to papers they hadn’t written, they refuse to make data available to the regulators or the scientific community or even, as here, to the investigators. They have failed to register drug trials, and this prevents others from discovering exactly what was done and allows them to publish only the favourable results [16] (Post Mortem on the TGN1412 Disaster, SiS 30).

Some journals are now – you may think belatedly – trying to put a stop to such practices. In particular, since 2005, the International Committee of Medical Journal Editors will not consider for publication papers that come from unregistered clinical trials. JBMR now requires that authors confirm they are not aware of any disagreements concerning the paper from anyone involved in the work that led to it, though whether that will have more effect than their long-standing requirement that all authors must have had access to the data remains to be seen. That the editor agreed to publish the response [9] when he himself does not accept that the conclusions follow from the rest of the paper is hardly an encouraging sign.

We should also be concerned about what all this says about the scientific establishment. The authors pointedly write in their letter of response that articles raising concern about their results had appeared “in the popular press”. If by that they mean that only journalists had felt an obligation to complain about what they had done, then this is a criticism not of journalists but of scientists. There are a number of organisations that are very quick to attack what they claim is not “sound science”, but they have been conspicuously silent about this case, as they have about many others.

In the short story Silver Blaze, Sherlock Holmes comments on the curious incident of the dog in the night. As everyone knows, the curious incident was that the dog did nothing; in particular he did not bark. What not everyone remembers is why he didn’t: it was because he knew the person he saw.

This very aptly describes what is happening here. The people that have set themselves up as watchdogs do not bark when their friends in the biotech and pharmaceutical industries are involved. They are very quick to bark when they see a threat to their friends.

The Institute of Science in Society, The Old House 39-41 North Road, London N7 9DP
telephone:   [44 20 7700 5948]   [44 20 8452 2729]

Contact the Institute of Science in Society