Tuesday, August 28, 2012

Details of Dr. Faustman's Phase-I Results for BCG

Dr. Faustman has published the peer reviewed results of her phase-I clinical trial.
The trial was complex, and the results are complex, and I know there is a lot of interest in this particular research, so this is a very long post describing her results in detail, along with the issues they bring up.   I've blogged extensively on her research in the past, which you can read here:

Musical accompaniment for this blog posting is from the best band ever to come out of Palo Alto High School (in California, USA).  The Donna's It's On The Rocks:

Because this is a long, complex posting, I've put in three different types of footnotes:
    [dNN] refer to extra discussion, extra details, etc.
    [rNN] are references to papers, web pages, etc.
    [rpNN] are references to specific parts of the Faustman paper.
All of these are at the end of the posting.

If you have already read my posting "Summary of Dr. Faustman's Phase-I Results for BCG", then you can skip the next two sections (Background and Summary), as they are identical, except for the footnotes.

Background to Dr. Faustman's Research

The essence of Dr. Faustman's theory on how to cure type-1 diabetes is [r2]:
  • Part 1: BCG causes the body to generate TNF
  • Part 2: TNF causes fewer autoreactive T-cells  [r3]
  • Part 3: Fewer autoreactive T-cells results in beta cell regrowth and more insulin generation
  • Part 4: More insulin generation is the path to curing type-1
BCG (Bacillus Calmette–Guérin) is a biologic which has been given to over a billion people (in low dose) as a tuberculosis vaccine, and is also approved (in much higher doses) as a bladder cancer treatment.  It is a generic drug with a very long record of safety.  More information [r4]. This trial focuses on parts 1-3 of the theory.  Part 4 is not controversial at all, and part 1 is widely believed as well, so it is parts 2 and 3 that really need testing.

TNF ("Tumor necrosis factor" or TNF-alpha) is a naturally occurring protein that can cause cells to die.  It is involved in the natural regulation of immune cells.

"Autoreactive" refers to immune cells which mistakenly attack the body's own beta cells.  The destruction of these beta cells leads to type-1 diabetes.  This is sometimes referred to as an "autoimmune attack" because the body's own immune system attacks the body itself.

Summary of Results

In this day of 10 second sound bytes and 140 character messages, everyone wants a short summary.  But this is a complex trial, and just about any quick summary will be an over simplification.   This is the best that I can do:

This trial was supposed to provide support for three parts of Dr. Faustman's theory, but:
1. It provided no data to support part 1 of Dr. Faustman's theory.
2. It provided data that part 2 was not happening (ie. it contradicted part 2 of Dr. Faustman's theory).  In particular, no change in live autoreactive T cells were reported.
3. The study shows a very small improvement to C-peptide numbers (but that was dependent on using a particular control group).  This improvement to C-peptide production supports part 3 of Dr. Faustman's theory.

Result 2 above, suggests that Dr. Faustman's theory about how BCG could help cure type one diabetes, was wrong.  But result 3 holds out a sliver of hope, that maybe in some different way BCG will be part of a cure.  Therefore, all discussions about the success or failure of this experiment are going to quickly boil down to the importance (and even existence) of the vary small increase in C-peptide production in item 3 above.  This very small increase in C-peptide production was "brittle", meaning it was only seen with a particular control group.

Other important points:
4. There were no safety issues, although this was expected considering BCG's long safety history, and the fact that it was chosen specifically because it was known safe.
For item 5 please refer to the SummaryOfFaustmanCPeptides.pdf file attached to this email.
5. The improvement to C-peptide numbers, is much smaller than that seen in established type-1 diabetics enrolled in Dr. Zhao's or LCT's previously published phase-I clinical trials.
6. In this very small study, A1c numbers got worse for people given BCG.  For me, this was a worrisome effect, although the paper does not treat it as such, and the study size may be too small to be significant.
7. Part of the paper described one patient who came down with Epstein Bar Virus (EBV) during the trial.  I consider that a single patient case study, not a clinical trial, so I'm not going over it in detail. If you're interested in that one patient, you should read those parts of the paper.

Results In Detail

For each result, I present the data and then discuss it.  I've put them in order of most important to least important [d1].  I also discuss data that was supposed to be collected, but was not reported.


C-peptide measures are critical for two reasons.  From a scientific point of view, they are the gold standard that the body is producing it's own insulin.  (Measuring insulin will confuse injected insulin and produced insulin, but C-peptide is a side effect of the production process.  If you see that, you know the body is generating it's own insulin.)  Also read [d6].  From a regulatory point of view, the US FDA has issued guidance that C-peptide numbers will be used to approve any future cure or treatment for type-1 diabetes.  So these numbers will be central to the approval of any new treatment.

Of the three patients who got BCG, two of them had a short term, but statistically significant [d18], rise in insulin production (as measured by C-peptide) when compared to the reference subjects. The reference subjects averaged 1.65 pmol/L and the two BCG patients were 3.49 and 2.57, so averaged 3.03. So the net average benefit (for the two patients who responded) was about 1.38 pmol/L.

How do these numbers compare to a cure?

According to the Mayo clinic an average person (non-diabetic) generates a minimum of 365 pmol/L.
Dr. Zhao used a lower reference number of about 199 pmol/L.  [d2]

If we use the Mayo numbers, then these patients were about 1 / 264 th of the way to a cure.
If we use the Zhao numbers, then they are about 1 / 144 th of the way to a cure.

How do these number compare to other researchers?
Details on the source of this data and the calculations to compare them are in [d3].
I think it is reasonable to compare these results to other clinical trials in established type-1 diabetes, that use a safe treatment.   Comparing them to improvements seen in honeymoon type-1 diabetics would be unfair, as would comparing them to risky treatments [d4].  The closest comparison is Dr. Zhao's research [r5], and LCT is another good comparison.

For Dr. Faustman (see [d3] for difference between "all treated" and "responders only"):
Average improvement (all treated patients) was .37 pmol/L
Average improvement (responders only) was 1.38 pmol/L

For Dr. Zhao:
Average improvement (all treated patients) was 157 pmol/L

To put it bluntly, if you compare all treated by Dr. Zhao to all treated by Dr. Faustman, Dr. Zhao has about 425 times bigger effect than Dr. Faustman (157 vs .37).  If you compare all treated by Dr. Zhao to those treated by Dr. Faustman who responded well, then Zhao's effect is "only" about 114 times bigger (157 vs 1.38).  Also, Dr. Zhao's saw those results in a sample size four times larger than Dr. Faustman's, and the improvement in C-peptide numbers was seen for a longer time period in Dr. Zhao's trials as compared to Dr. Faustman's.  [d5]

Unfortunately LCT reports on insulin dose, rather than C-peptide [d6].  In their most recent clinical trials, they saw an average dose of 41 units drop to 36 and then 30 [r6].  Taken at face value and very roughly, this would imply a C-peptide number of about 91 pmol/L (using the Mayo reference), or  50 pmol/L (using the Zhao reference) [d7]. Both numbers vastly larger than Dr. Faustmans 1.38 or .37.

HbA1c (commonly called "A1c")

The results: The paper said "There were no significant changes in any of the clinical trial patients in ... HbA1c levels" [rp14].  The HbA1c data below was included in figures [rp15]:

The three BCG treated patients had these A1c changes:
5 weeks after first treatment the changes were +0.6, +0.1, and 0.  Average +0.2.
20 weeks after first treatment the changes were +0.7, +0.5, and +0.2.  Average +0.5.

The two untreated patients who did not get EBV had these A1c numbers:
5 weeks after first treatment the changes were 0 and -0.2.  Average -0.1.
20 weeks after first treatment the changes were  +0.4 and -0.5.  Average 0 (no change).

What this means: In the placebo patients, some went up, some down.  In the BCG group all went up.  After 5 weeks one BCG patient has a A1c change of 0.5 or greater, and after 20 weeks, 2 had.  Since I consider an A1c change of 0.5 or more to be important, that is a worrisome result [r8].  But with only three treated patients, these differences may not be statistically significant [d19].   Discussion of 5 vs. 20 week importance is here [d8].

Insulin-autoreactive T cells

When the trial was first designed this was the primary outcome [r7].  This outcome is critical to  part 2 of Dr. Faustman's theory of how to cure type-1 diabetes.  It is important that the number of autoreactive cells drop [r5].  Even if there are no clinical improvements, if there were a drop in Insulin autoreactive T cells, then there would be hope for the research theory.

The results:

1. The total number of Insulin-autoreactive T cells was much higher in the BCG treated patients. [rp2]
2. However, the extra Insulin-autoreactive T cells were generally dead. [rp3]
3. The paper did not report on any statistically significant change in the number of "live" Insulin autoreactive T cells in the patients treated with BCG as compared those who were not.
4. The paper did report a statistically significant increase in dead autoreactive T cells, but no drop in live ones.

What this means: So in terms of showing that BCG resulted is fewer living Insulin-autoreactive T cells, the trial was a failure.  This means it failed it's primary end point, as specified by Dr. Faustman when the trial started [r7][d9].  More importantly it means that her basic theory (that BCG would lead to TNF would lead to fewer "bad"/autoreactive T-cells) was wrong [d10].

If BCG works at all, someone will need to come up with a different underlying theory of mechanism that explains it.  This isn't an insurmountable problem however, plenty of treatments are used successful without being fully understood, especially early on.  However, it does make it hard to predict dosing, side effects, effective populations, etc.

Autoantibody Levels

The paper reported on autoantibodies, which are another form of autoimmune cells.  If BCG did not work by lowering autoimmune T cells, the hope here is that it worked by lowering autoimmune antibody cells.

The results:  For GAD "There was a statistically significant and sustained change in GAD autoantibody levels in two of the three BCG-treated subjects after injections, with one diabetic showing a decrease and the other an increase relative to self-baseline" [rp4].  Basically one BCG treated patient went up, another went down, and a third stayed the same.  For IA-A2 and ZnT8A "only ZnT8A had statistically significant decreases in one BCG treatment subject." [rp4].  So with six combinations (two autoantibodies and three people) they got one statistically significant decrease.

If you look at the graphs for 17 weeks, there doesn't seem to be a difference between the starting point and ending point. Nor is there any obvious up or down trends right after the BCG is given.

What this means: Overall, I would describe this as a whole lot of nothing.

Regulatory T Cells

Originally, this was not part of the trial [r7], but it was reported on.  These are immune cells which police other cells, so more of them are expected to help type-1 diabetes.  But these cells are not type-1 specific, they don't just regulate the bad T cells that cause type-1.  They regulate all T cells. Extra regulatory T cells can be expected to slow down the whole immune system, not just the bad autoimmune part [d11].

If there were no drop in Insulin autoreactive T cells and no drop in autantibodies, then perhaps there is an increase in regulator T cells?

The results: There were clearly more T-reg cells in the BCG treated patients than in paired healthy controls, or in the placebo group.

What this means: I'm not sure that it means anything [d12].  The whole point of vaccines is to stimulate an immune response.  (And viruses do this as well.)   It makes sense that the response would include regulator T cells and all other types of T cells, in equal proportion.   A therapeutic effect would only be expected if regulator T cells went up and other T cells stayed the same, or went down (or at least did not go up as much).  No data on T cells in general is included, so there is no way to know if reg T cells went up in sync with all T cells or not.

No TNF Results

The results: The original experimental design was going to measure TNF, TNF-receptors, [and] other cytokines [r7].  None of that data was reported in the paper, although the text did say that BCG was chosen specifically because it induced TNF generation [rp5].

What this means: By not reporting this data, the clinical trial could not show that part 1 of Dr. Faustman's theory on how to cure type-1 diabetes was correct.  Now, part 1 is less controversial than parts 2 and 3, but still: the original experimental design would have shown conclusively that part 1 was correct and did happen in people, and would have given valueable dosing information for future trials.  It's the scientific equivalent of "low hanging fruit" and it's puzzling that it was dropped.

Weaknesses in Data Analysis

In the discussion above, I have taken all the results reported in the paper as correct without question.  However, there are two important weaknesses in how the data analysis was done, which call into question these results.  These weaknesses especially effect the C-peptide numbers.

As background remember how you were taught to do scientific experiments back in school:  Take a group of similar people, divide them into two groups.  Give one group the drug you are studying and give the other a "placebo" (do nothing) treatment.  Compare results between the two groups.  This is the "gold standard" of experimental design.  However, this paper doesn't use this treated-vs-placebo design [rp6].

But there are other ways to do comparisons in clinical trials.  For example, you might compare data from before a person was treated to data from after they were treated.  This can be thought of as a sort of "silver standard": still very good.   In some cases this is the only comparison available [d13].  Or, you might compare data from your treated group, to data from a similar reference group.  This technique can be done well or poorly, depending on how the reference group is created [d14].   This paper uses both a before-vs-after design and high quality versions of compare-to-reference designs.

Obviously, researchers prefer to use the gold standard, but sometimes they can not, especially in phase-I trials [d13].  However, this trial is unique in my experience, because they did have a placebo group, and collected all the data needed for the "gold standard" comparison, but then chose not to do that, and to use a "silver standard" comparison instead.  And here's the punchline: if they had done the "gold standard" comparison of treated vs. placebo group the results would likely not have been statistically significant, and the entire trial would have been a failure.

To understand how this works, consider the placebo group first (the two who did not get EBV). [rp7].    Notice that the first patient's C-peptide numbers ranged from about 180 to about 280, and the second from about 20 to 140.  So that means these guys naturally varied by over 100 pmol/L.   Now take a look at the three patients who actually got BCG.  The three of them varied from about 2.2 to 4.4, from 1.8 to 6.2 and from 1.8 to about 6 [d16].  (All numbers very rough since I'm reading off of graphs.)

Here is the bottom line: people who were not given BCG naturally varied by 100 points during the trial.  People who were given BCG varied by less than 5 points.  The BCG changes were much less than the random fluxuations in the untreated people.  It's the very definition of "not statistically significant".  But remember, this is all based on the gold standard of comparing treated to placebo.  In fact, the results reported on in the paper are comparing the BCG treated patients to the "reference group" described in part C of the figure [d15].

The second weakness is an unusual technique of multiple control groups.  Obviously, most clinical trials report on several different results, and this trial is no exception, reporting on five different types of results.  The common thing to do is to use the same comparison group for all the results.  If you are comparing the treated group to a reference group (for example), you do that for C-peptide and A1c and insulin usage and whatever else you are testing for.  But this study uses a different comparison group for each different result [rp8].  Although the paper is very clear that this was done, there is no discussion of why it was done.  Here are the comparison groups, as I understand them:

The C-peptide results compared the treated group to the reference patients.
The Autoantibody results compared before and after data in the treated group.
The T-reg results compared the treated group and paired controls or to the placebo group.
The auto-reactive T cells  results compared treated group to healthy controls, to reference diabetics and to reference healthy controls.
The A1c comparison group was not specified.

Obviously, this brings up a whole host of questions, the most important is: why not use one control group for everything?  Are the results so fragile that they are dependent on careful selection of the control group?  See above for why that might be true for the C-peptide data, and it may well be true for the other data also.

Based on all of this, I think it is a mistake to say, as many have, that the study shows a very small improvement to C-peptide numbers.   I think it is more accurate to say that the study shows a very small improvement to C-peptide number, but this is dependent on the control group used.

The Future of This Research

In one sense, the future of this research is easy to predict.  Dr. Faustman has already gathered $8 million for the follow on phase-II trial, so I have every expectation that a follow on clinical trial will be done.

But scientifically, the future is much more murky.  In a nutshell, here are the current problems:
  1. Dr. Faustman's theory that BCG would lower the number of autoreactive T cells (or any other immune cells specific to type-1 diabetes) is not supported by this data.  No improvement is seen in living autoreactive T cells.
  2. There was no clinically significant good outcomes.  The C-peptide improvements, even if real, are too small to impact a person [d18]. 
  3. There was a worrisome outcome in A1c results.
Of course, research is always about the future: what can this grow into?  But if you are betting on future improvements, then it makes sense to start with a stronger base, and Dr. Zhao had much better results 6 months ago, and LCT for longer than that.

Moving this research forward is not going to be easy.  Because the current results are 100s of times too small for a cure, I don't think it is reasonable to just give "linear" higher doses or more frequent doses.  Certainly, not a dose 100s of times higher.  Often, it is the theory about why the treatment works that gives you clues about how to change doses between phase-I and phase-II.  If there is an important threshold, for example.  But in this case the theory about why the treatment works has not been supported by the phase-I, so there is no help there.

My opinion: Everyone wants to know if this clinical trial succeeded or failed.  This trial did not succeed, but did it fail?  Maybe, but I would phrase it a little differently.  One way to phrase it is this: This trial succeeded in telling us that the (BCG to TNF to less autoreactrive T cells to more beta cells to more insulin) theory was wrong.  Another way to phrase it is this: This trial is more indicative of a basic research result, than a path-to-product research result.  There were anomalies that might turn out to be interesting: the dead autoreactive T cells and also the change in GAD/TnT8A antibodies and regulatory T cells, but these represent basic research and there is no obvious path from these results to a cure.

Extra Discussion and References


[d1] I view clinical results (ie. effects that directly help a person with type-1) as much more important that research results (ie. effects that might lead to further research, or are interesting and novel).  So C-peptide is the most important clinical result because it means less insulin is used, and is the official FDA marker for cures of type-1 diabetes.  A1c is second most important clinical result because it is highly cooralted with fewer long term complications of type-1.  For the research results, autoreactive T-cells are the most important because they are central to Dr. Faustman's theory (and I"m sure that's why this was the primary outcome listed).  The autoreactive antibody are next because they are specific to type-1 diabetes, while the reglatory  T cells are near the bottom because they are not type-1 specific, and TNF is  sort of an internal check of Dr. Faustman's theory, so I put it last.

[d2] According to the Mayo clinic [r9], their testing shows reference numbers (for fasting) of:
1.1-4.4 ng/mL which converts to: 365 - 1456 pmol/L.  Zhao's uses a lower reference number of about 0.6 ng/ml which is 199 pmol/L.  Zhao's research was done in China, and Zhao believes that the Chinese have, on average, lower fasting C-peptide numbers than the Americans who make up the baseline of the Mayo clinic's numbers.  Web poster "Celsus" used a range of 260 pmol/L - 1320 pmol/L here: http://islet.org/forum/messages/55197.htm

[d3] Details on C-peptide Calculations

Faustman's C-peptides (paper in pmol/L):
Two responding BCG treated patients: 3.49+2.57 = 6.06 / 2 = 3.03 [rp1]
All BCG treated paitents: 3.49+2.57+0 = 6.06 / 3 = 2.0.2
Baseline average: 1.65

Average improvement (responders only): 3.03 - 1.65 = 1.38 pmol/L
Average improvement (all treated patients): 2.02 - 1.65 = .37 pmol/L

Zhao's C-peptides (paper in ng/ml):
Average improvement for Group B: 0.5 ng/ml x 331 = 165.5 pmol/L
Average improvement for Group A: 0.45 ng/ml x 331 = 149 pmol/L
Average improvement (all treated patients): 165.5+149 / 2 = 157 pmol/L
These numbers are approximate, because I read them off of Zhao's graph at 24 weeks.[r9]

For Dr. Zhao:
Average improvement (all treated patients): 165.5+149 / 2 = 157 pmol/L

[d4] I'm specifically excluding Burt research (including both the Snarski and Li follow on work) because of the risk involved.

[d5] Remember that I have not done any analysis of measurement error here.

[d6]. I consider measuring injected insulin to be an inferior measurement as compared to measuring C-peptide.  If a person eats less carbs, or lets their A1c number rise, they will need less insulin, but that is not an improvement.  On the other hand, C-peptide measures how much insulin their body is naturally producing.

[d7]  I'm assuming that if a person's insulin needs go down by 10%, and the minium C-peptide a non-type-1 diabetic needs is about 200 pmol/L, then that is equivelent to generating 10 pmol/L.  It's a very rough estimate.

[d8] A1c numbers were recorded at "Baseline", "Week 5", and "Week 20".  Officially, A1c numbers are supposed to be an average of the last 13 weeks, although the most recent 2-4 weeks seem to have more impact than the 9 earlier weeks.  BCG was dosed at week 0 and 4, the best time to measure A1c is dependant on how long it takes for the BCG to become maximally effective and how long it would be effective for.  For example, if the BCG took 1 week to reach maximum effectiveness, and then was effective for 1 more week, then measuring A1c on week 7 would give the strongest results.  On the other hand, if BCG was effective for 4 weeks and took effect immediately, then the best A1c would be measured at 9 weeks.  Unfortunately, if you look at Figure 8 part A, there is no clear trend on either how long it takes to take effect or how long effectriveness lasts.

[d9] People using phrases like "success" and "failure" with respect to a clinical trial may have different ideas about what exactly they mean.  However, I use the following definition which is pretty standard: "A statistically significant positive result in the primary outcome."

[d10] As you might expect, Dr. Faustman does not share this view.  The paper is very specific: 
The rapid release of dead insulin-autoreactive T cells supports the hypothesis, first demonstrated in the NOD-mouse model of autoimmune diabetes, that BCG ameliorates the advanced autoimmune process underlying type 1 diabetes by stimulating TNF, which selectively kills only disease-causing cells and, further, permits pancreas regeneration as evidenced by the transient increase in C-peptide secretion we observed using an ultrasensitive C-peptide assay [rp16].
I can not agree that excess dead autoreactive T cells (with no drop in live ones) supports the hypothesis.  Not without a long stretch and a lot of wishful thinking.

[d11] Dr. Faustman has always made a big point of the fact that BCG was supposed to target the specific immune cells that were misbehaving, and not a whole class of immune cells.  For example:
unlike other immunosuppressive therapies for autoimmune diseases that harm both healthy and disease-causing T-cells, this treatment appears to provide a way to achieve “targeted  removal” of only autoimmune disease-causing cells.
which is from: http://www.faustmanlab.org/docs/newsletters/Faustman_updates_f09.pdf

[d12] For a recent spectacular example of why regulatory T cell drops may be nothing, consider this research:
These guys were testing something that prevented type-1 in NOD mice, but found that the same treatment in people made type-1 worse, but also increased regulatory T cells.

[d13]  For example, in a phase-I trial it might be too expensive or too much hassle to recruit twice as many people.  Or, the treatment may be hard to placebo (maybe it has a unique taste, or is some form of exercise/training where everyone involves knows if you did it or not).  Or the disease may be so rare that getting enough people for the treated group is hard enough, and getting more for a placebo group is not reasonable.  Obviously, none of these things applied to this trial.  In fact, they had a placebo group.  They just didn't use it for the important analysis.

[d14] For example, comparing people from two different countries often makes for awful science, because the groups are often not fundamentally similar.  In fact, they are often quite different.  On the other hand, a reference group specifically recruited from the same population as the treated group can be much better, and that's what was done for this trial.

[d15] This reference group was not mentioned in the original experimental design [r7], but was included in the published paper [r1] showing that it was added later.  There was a few sentences about AUC measurements of C-peptide levels being higher, but no mention of if it were statistically significant [rp17]. 

[d16]  As you read the graph in the paper, notice the different labeling on the Y access.  The BCG and EBV patients are labeled from 0-8, but the placebo patients are labeled from 0 to 400 or 600: that's a huge difference.  Imagine what these graphs would look like, if all the patients were graphed on the same Y access!

[d100] This is the data reported in Faustman's abstract.  She implies that the third patient did not see improvement in C-peptide, but does not provide exact data.

[d18] It is important to understand the difference between "statistically significant" and "clinically significant".  Statistically significant means the result is unlikely to be an accident or due to random fluctuations.  In a sense, it means there really is a difference between the treated and untreated patients.  Clinically significant is a difference that matters to the patient.  It is a difference that makes them healthier, or makes the management of their disease easier,  in a way they can experience.

[d19] Of course, the C-peptide numbers only came from three people as well, so (in theory) they should have the same problems with small sample size as the A1c numbers.  However, for the C-peptide number the researchers used a larger "reference group" for comparison, rather than the very small placebo group.  They didn't publish A1c numbers for the reference group, so I can not compare those patients to the BCG group.


[r1] This is the URL for Dr. Faustman's paper, which is available for free on the web.
(Thanks PLoS One!)

[r2] Dr. Faustman's Newsletter here:
Especially the article "Why use bacillus Calmette-Guérin (BCG)?" which says:
BCG causes the body to make a natural substance called TNF. TNF helps to regulate the immune system and can kill a portion of the “bad” T cells that cause diabetes.
Another newsletter, with the same info is here: http://www.faustmanlab.org/docs/newsletters/Faustman_updates_f09.pdf
Especially the article "BCG Therapy At Work" on page 5.

[r3] Secondary source is here:
Primary source is here:
As far as I can tell this was first suggested by Alleva in 1993:

[r7] Original experimental design is summarized here:

[r5] You can find extensive discussion of Dr. Zhao's work here:

[r4] General information on BCG is here:


[r8] For example, from this document: http://www.nice.org.uk/nicemedia/live/12165/44318/44318.pdf
In general, 0.5% HbA1c is considered a clinically significant change (e.g. treatment guidelines from ADA/EASD and NICE)
Another example is this study:
reported that lowering A1c by 0.4 was a good result, so  raising it more than that would be a bad one.

[r9]  The 331 constant for the conversion and the 1.1-4.4 ng/mL baseline comes from the Mayo clinic's web page: http://www.mayomedicallaboratories.com/test-catalog/Clinical+and+Interpretive/8804

References from the Paper

[rp1] Information from the paper's abstract.

[rp11] Figure 6 A. on page 11.

[rp2] Figure 4 A on page 8.

[rp3] Figure 4 B on page 8 and also Figure 5 on page 9 for one specific example.

[rp5] Third paragraph of the Introduction.  She uses this paper as a reference:
but I don't see any references to BCG in there.

[rp6] From the first paragraph of "Statistical Analysis" on page 9:
None of the analyses compared the BCG-treated to placebo-treated clinical trial subjects.
[rp7] Look in the lower left have side of this diagram:

[rp8] Figure 8 on page 13 has the C-peptide data.

[rp6] This is the exact quote, from the section "Statistical Analysis" (1st paragraph):
"None of the analyses compared the BCG-treated to placebo-treated clinical trial subjects."

[rp35] All this data came from ...

[rp4]  The paragraph just after figure 4 on page 8.

[rp14] Page 10, the paragraph "Other Clinical and Safety Monitoring"

[rp15] Figure 2, page 7, last few rows.

[rp16] This is from the first paragraph under "Discussion" on page 10.  I've removed the footnotes.

[rp17] The last paragraph in the "Fasting Insulin Secretion Temporarily Increased..." section on page 10.

Joshua Levy
All the views expressed here are those of Joshua Levy, and nothing here is official JDRF or JDCA news, views, policies or opinions. My blog contains a more complete non-conflict of interest statement.
Clinical Trials Blog: http://cureresearch4type1diabetes.blogspot.com
Cured in Mice Blog: http://t1dcuredinmice.blogspot.com/


pdx_dc said...

Thanks for all the work, Joshua. I feel even the Doctor herself is taking a step back and saying I dont have a cure but...

I am wondering if "we" should all pitch in and see what we can do to help Dr. Zhao's research.

How can the T1D community organize a grassroots movement to advance Dr. Zhao's work?

Bennet said...

Thanks for the detailed review of this paper. also for the link you posted on YDMV.net to Karmel's. I agree reading both is valuable.

Most significantly I love the precedent of specific musical accompaniment to detailed blog post and your particular selection. I think you set a new gold standard for adding value to long and detailed blog post and I thing we should all follow in step. My only suggestion would be to like a spottily play list long enough to get through the whole post.

Kudos my friend for the musical innovation and the post.


Unknown said...

Fantastic post, detailed and critical, just the way I like it. Thanks!

Joshua Levy said...

pdx_dc: I don't know how to organize that kind of direct support. So I don't have any specific advice. However, if you (or anyone) does want to set something up, please tell me about it! I think that Dr. Zhao's work is very promising and deserves a lot of support. His timely results have earned it.

Joshua Levy

Shaun Tonstad said...

Joshua, I thought the purpose of this study's phase I is to measure safety. How can you evaluate efficacy when dosing has not been established?

Joshua Levy said...

This study was not about safety at the start.

The ultimate proof that this trial is not primarily aimed at safety is the Clinical Trial record for it. Here is the section where Dr. Faustman describes what will be measured, and the safety implications:

Primary Outcome Measures:
* concentration of autoreactive t-cells [ Time Frame: Measured weekly in first 8 weeks, then every other week for weeks 8-12 ] [ Designated as safety issue: No ]

Secondary Outcome Measures:
* Concentration of TNF, TNF-receptors, other cytokines, and c-peptide levels [ Time Frame: Weekly for first 8 weeks, then every other week for weeks 8-12 ] [ Designated as safety issue: No ]

That's the entire section. Notice that none of the measurements are listed as a safety issue. The official trial record number is NCT00607230, and you can see it here: http://www.clinicaltrials.gov/ct2/show/NCT00607230

As for your evaluation of efficancy vs. establishing dosing, there is not difference. Think about it: establishing efficancy means establishing dosing, and visa-versa. Of course a study that fails it's efficency measures will also not know what dosing is needed. The reverse is also true: proving efficency means you will have established dosing.

Joshua Levy said...

Also: remember that the idea that phase-I trials are only to measure safety, is wrong in two ways.

First, phase-I trials almost always try to measure both safety and efficiency. Every now and then you'll see one which really is safety only, but it's rare. Most are both.

Second, and more importantly, the phase-I as safety trial is only for new drugs. Drugs that are being tested for the first time. Obviously, BCG is the exact opposite of that. BCG is one of the most used treatments in the world (over 1 billion people dosed to date!) Last I checked, there were dozens of clinical trials using BCG active right now, and off all phases. So safety is clearly a non-issue. Even for type-1 diabetes specfically, I know of at least 6 studies that have already given BCG to people with type-1.

No matter how you slice it: safety is just an excuse not to talk about effectiveness, because effectiveness failed.