This is the first paragraph from yesterday's press release from Hyperion:
Hyperion Therapeutics, Inc. (HPTX) today announced it is terminating development of DiaPep277 for newly diagnosed Type 1 diabetes. The company has uncovered evidence that certain employees of Andromeda Biotech, Ltd., which Hyperion acquired in June 2014, engaged in serious misconduct, including collusion with a third-party biostatistics firm in Israel to improperly receive un-blinded DIA-AID 1 trial data and to use such data in order to manipulate the analyses to obtain a favorable result. Additional evidence indicates that the biostatistics firm and certain Andromeda employees continued the improper practice of sharing and examining un-blinded data from the ongoing DIA-AID 2 trial. All of these acts were concealed from Hyperion and others. The Company has suspended the Andromeda employees known to be involved, is notifying relevant regulatory authorities, and continues to investigate in order to explore its legal options. Hyperion employees were not involved in any of the improper conduct.
A note on terminology: Hyperion did not use the term "fraud" in describing what happened, although Globe News did, and another news service said "falsified data". Instead, in their conference call and press release, Hyperion used terms like "serious misconduct and deceit", "collusion", "extensive measures to conceal their wrong doing", "actively and consistently lied", "dishonesty and deceit", "deception was extraordinarily serious", and so on. Based on all that, I do think that "fraud" is the right term, but it is important to remember that I mean this word in the English dictionary meaning [d1], not the legal meaning. No one has been convicted of any crime, not even charged, and I doubt anyone ever will be.
The sound track for this posting is here:
http://grooveshark.com/#!/s/Saturday+Night+s+Alright+for+Fighting/2UTbqy
(Note this is The Who's version, because I can hear the words better in it than in Elton John's.)
As usual [d] notes are at the bottom of the posting, and provide more details.
Background on DiaPep277
DiaPep277 is a peptide (a part of a protein). It is a small part of a naturally occurring protein called "heat shock protein 60". The hope was that it would cause the immune system to stop attacking beta cells. Development was done by Andromeda (either as a separate company or a division within another company), and no other company is doing work in this area. It is one of the potential cures for type-1 diabetes that I have followed from the very beginning of my research. It had already finished phase-II trials when my daughter was diagnosed in 2003. I have made more postings on DiaPep277 than any other potential cure, except for Diamyd. You can read them here:
http://cureresearch4type1diabetes.blogspot.com/search/label/DiaPep%20277
In 2008 I published a blog based on DiaPep277's earliest data from a phase-III trial and I felt the results were so small it was unlikely to be successful. You can read that here:
http://cureresearch4type1diabetes.blogspot.com/2008/08/disappointing-news-on-diapep-227.html
However, in 2011 I published this slightly more upbeat blog:
http://cureresearch4type1diabetes.blogspot.com/2011/11/andromedas-diapep277-succeeds-in-phase.html
I continued to follow it until 2013 when I "threw in the towel" stating that the results seen so far were so small that they could not lead to a cure (although I still held out hope they could lead to a new treatment).
http://cureresearch4type1diabetes.blogspot.com/2013/06/possible-cures-for-type-1-in-news-june.html
What Happened?
As you read my description of what happened, it is important to remember that all my information comes from Hyperion (except for a tiny bit from Evotech), and none of it comes from Andromeda, or any of the specific employees who are alleged to have participated in the dishonesty. If Andromeda or the people involved publicize their side of the story, I will likely need to update this, based on that new information.
It is normal practice to run two large clinical trials to get the data required by the FDA and the EMEA, and Andromeda had started two: DIA-AID-1 and DIA-AID-2. They were designed to be twin studies and have 450 people each [d2]. Just last June, DiaPep277 was sold to Hyperion. This sale included all rights to the new drug, and the transfer of some Andromeda employees who were working on it. At that point DIA-AID-1 was complete and had been published, but DIA-AID-2 was not quite finished. The completion date is early 2015. So the Hyperion statistics team were evaluating the entire DIA-AID-1 data set, as a sort of practice run to get ready to analyse the DIA-AID-2 data, when it was ready.
You can read the DIA-AID-1 results in this paper:
http://care.diabetesjournals.org/content/37/5/1392.full.pdf+html
Thanks very much to Diabetes Care, published by the ADA, for making the whole paper available on line.
When the Hyperion statisticians looked at the full data set for DIA-AID-1, they noticed something very odd. If they analyzed the entire data set, then DiaPep277 did not have a statistically significant good effect in the primary outcome measurement. The clinical trial had failed. However, Andromeda had excluded 30+ patients from the analysis because they had violated the rules about who should be signed up [d3]. With those exclusions, the data showed a statistically significant good effect in the primary outcome [d4]. The study had succeeded. That's unusual, because the exclusions are supposed to be made "blind" (not knowing if the drug worked for those people, or even if they got the drug), and excluding people randomly from a trial, should not change the outcome. These exclusions were done by an outside company which was involved in the clinical trial, and that company was not supposed to know who got the drug and who got the placebo.
Except Hyperion's investigation found (according to Hyperion) that some of the Andromeda employees passed data to the outside company, and people at that company used that data to selectively remove patients from the study to bias the results. Also, Andromeda employees changed the primary end point of the study [d5], so that it would be successful. In both cases, the decision was made "unblinded" (i.e. knowing who received DiaPep277, and who received placebo), when the decision should have been made "blind". This completely undercuts the results of the clinical trial. Hyperion said that this did happen in the DIA-AID-1 trials results, and was also in process of happening in the DIA-AID-2 trial [d6]
Although Hyperion was careful not to name the company that did the statistical analysis for Andromeda (they always referred to it as an Israeli Biostatistics company), nor did they name any of the people involved. However, on page 1399 of the DIA-AID-1 paper, there is discussion of what companies did statistical analysis as well as describing what each author did in running the study and writing the paper. (In the future, I'll be checking to see if this company or these researchers are involved in any research I report on.)
Impact
Hyperion has said that there is no way to move forward with regulatory approval for DiaPep277, and they will not attempt it. So DiaPep277 is dead. That's the short term impact. I suppose they could try to sell it to someone else, but who would want to buy it now?
The medium-term impact has three questions:
1. Will the paper describing the results from the DIA-AID-1 trial be retracted? It was published in Diabetes Care (a journal of the American Diabetes Association), so it will be interesting if the authors retract it, or if the editors/publishers retract it [d7].
2. Will there be a civil lawsuit? Will anyone face a criminal charge? Remember that Hyperion paid tens of millions of dollars for DiaPep277 based largely on results which were invalid. Andromeda and the nameless Biostatistics company are Israeli, while Hyperion is American, and I'm sure that will complicate both civil and criminal legal matters.
3. In addition to the results paper, Andromeda employees also published a research paper comparing the primary end points (new and old) of their study. If Hyperion's can show that this data was manipulated, then this study should be retracted as well.
The abstract of the paper is here:
http://www.ncbi.nlm.nih.gov/pubmed/24408401
and says specifically that the findings were "unexpected", which Hyperion claims is untrue.
The long term impact is less predictable, but could be much wider. How many other studies used the same biostatistics company? Some of the researchers involved in this study are very big names, and have published many other papers, and worked with other companies, including some companies doing clinical trials.
Some Personal Notes
I gave up on DiaPep277 long ago, so in that sense, it was dead to me even before this came up. But it is still deeply shocking. (A statistician that looked at this situation described it as "staggering".) At the end of the day, new drug safety and effectiveness are supposed to be shown via scientific testing. There is a lot of good statistics used to show that results are meaningful, and not due to chance, accident or mistake. But all those statistics assume that the people running the study are not liars or cheats. Detecting people who are willing to commit serious misconduct in their scientific studies is not easy. Implementing the procedures necessary to detect active deceit in all scientific studies would be horribly expensive. Currently, the FDA uses statistical methods, to find "honest" mistakes, rather than do the sort of investigations and surveillance required to find premeditated fraud.
That's part of the reason why I think it's important to the scientific process to see what consequences the people and corporations face in this case. Because if other people and other corporations see that they don't face huge consequences, then they will be more willing to risk the same kind of misconduct that is alleged here.
In general, this blog does not report on financial transactions. That is specifically because of Andromeda and DiaPep277. Whenever a company buys a new drug, there are always very positive press releases, and early on I thought about reporting those in the blog. But I noticed that DiaPep277, in particular, was getting passed around to a lot of different companies, and for no good reason that I could see. I don't remember now all the moves (this was years ago), but I'm pretty sure that Andromeda sold it, got it back, sold it to someone else, and got it back again. All the companies were Israeli, and as I remember it, some of them were partial owners of each other [d8]. I did not understand it. That's a lot of moving around in a small market. It convinced me not to report on company-to-company movement in my blog; that it didn't mean anything, or at least I didn't know the meaning. But now, in retrospect, I wonder if it was a sign of trouble.
Finally, I want to personally urge Hyperion to make public the researchers involved, and the evidence about each one's involvement in this alleged misconduct. There are 21 named authors of the DIA-AID-1 study, and 6 named authors of the change-of-primary-endpoint paper (with much overlap). Right now, all of these researchers are "under a cloud" but it is likely that many of them did nothing wrong. Maybe none of them did anything wrong, and the misconduct was done by others, or never happened at all. But in any case, the whole type-1 world deserves to know who did what, and the supporting evidence.
Extra Discussion
[d1] For example (from dictionary.com): "deceit, trickery, sharp practice, or breach of confidence, perpetrated for profit or to gain some unfair or dishonest advantage."
[d2] This is a quirk of the American approval process. It is law that there must be two studies to confirm the results. Therefore, you can not run one 600 person study, you must run two 300 people studies, even if they are otherwise identical.
[d3] It is common for some patients enrolled in a clinical trial to be excluded from the results for a variety of reasons. The most common is that someone drops out, and complete data for that person is not available. However, people are sometimes enrolled by mistake in violation of the study's rules. For example, a trial might require first treatment within 100 days of diagnosis, but a review of paperwork, after the study completes might determine that the patient signed the paperwork within 100 days, but didn't actually get the drug until later. Data for that patient would be (properly) dropped from the study results.
[d4] If you look at the patient flow diagram on page 1394 of the paper, you can see that a total of 34 patients were excluded from analysis. It is the two boxes just below the "allocated" boxes, and these changes impacted both the "mITT" data and the "PP" data. The claim Hyperion made was that these exclusions were made "unblinded" and were specifically tailored to bias the results.
[d5] The primary end point is the most important result of a clinical trial. Usually, there is one primary end point, and several (less important) secondary end points. If the primary end point shows a statistically significant good effect, then the trial is successful. If this is not seen, then the trial is unsuccessful. The FDA generally determines effectiveness of new drugs based on primary end points. So therefore, changing the primary end point of a study, in the middle of the study is very unusual, and if it was done "unblinded" (ie. with knowledge that the current primary end point is failing, or the new one succeeding) that is scientific fraud (in my opinion). Its the moral equivalent of moving the goal posts to the ball's location, rather than putting the ball through the (stationary) goal posts.
In the case of DIA-AID-1, both the original primary end point, and the new primary end point involved measuring C-peptide, but the two end points measured it in different ways. The details are described in the comparison paper. Abstract is here:
http://www.ncbi.nlm.nih.gov/pubmed/24408401
Remember that although Hyperion has said that this happened in the DIA-AID-1 study, I have not seen their supporting evidence, and so have no way of knowing if this is correct or not.
[d6] As part of this research the DIA-AID-2 data collected to date was unblinded, and Hyperion said that there is almost no chance that it would end up showing a good effect in the primary end point. (Remember that most of the DIA-AID-2 data has already been collected, even as they are still waiting for the last few patients to get the last few data points.)
[d7] To be blunt, the paper contains the following two quotes (pages 1393 and 1394), which Hyperion now claims are untrue: "Participants, investigator site staff, persons performing the assessments, and data analysts were blinded to patient allocation from the time of randomization until database lock" and "The study protocol was amended, and the statistical analysis plan was planned and finalized before the study was unblinded, with the GST clearly defined as the primary end point."
[d8] Even now, I don't have a complete list, but here is a partial list of companies involved in DiaPep277 development: Andromeda, Clal Biotechnologies, Teva, Peptor, DeveloGen, and Evotek.
Joshua Levy
http://cureresearch4type1diabetes.blogspot.com
publicjoshualevy at gmail dot com
All the views expressed here are those of Joshua Levy, and nothing here is official JDRF, JDCA, or Tidepool news, views, policies or opinions. My daughter has type-1 diabetes and participates in clinical trials, which might be discussed here. My blog contains a more complete non-conflict of interest statement. Thanks to everyone who helps with the blog.
