Understanding the Research Funnel

I know that a lot of people are frustrated by the progress being made to cure type-1 diabetes. This posting is my attempt to explain the research process. Hopefully this will lead to less frustration with it, or at least more informed criticism of it.

The mental picture to have in your mind is a large funnel, with the big end to the left, and the small end to the right. The goal is to have a cure for type-1 diabetes come out of the right side, and the funnel shape represents the number of possible cures active at each stage in the process. In order to be approved for widespread human use, a medicine needs to go through five levels of testing:

• animal testing ("pre-clinical")
• safety testing on a very small group of people ("phase I clinical trials")
• effectiveness testing on a slightly bigger group ("phase II clinical trials")
• wide spread testing on a larger group of people ("phase III clinical trials")
  
• approval to mass-produce and market

This is the system used for new drugs, devices and implants in the USA.

The key thing to remember, is that no cure for type-1 diabetes is going to be a surprise. Because every cure is going to slowly go through all five steps listed above. Nothing comes out of left field. They slowly progress through each step. So if you pay attention to what research is completing each step, you will know what is going on now, and what will be happening in the future.

Now the first question about this funnel is, how long does it take? How long does it take a possible cure to go through all the required research and testing, and get an actual cure out the other end. Obviously, we don't know for sure, because no cure has come out the end. But the length has got to be more than 10 years, because the Phase III research that we see now, is all a little less than 10 years old already.

So that is one valuable thing learned already. When someone says "I just cured type-1 in mice, so a cure in humans is 3-5 years away" we know that person is full of crap. The cure is at least 10 years away, even after mice have been cured. Remember that. I'll speak about it more, later. My basic rule of thumb is that the first step above (animal testing), can take a very long time. Many years, even decades. The three "phase" steps (human or clinical trials) each take 1-3 years if the research moves forward smoothly. Mass production and marketing is a little quicker, maybe 1/2 to 2 years.

So a second valuable thing learned is that if someone says "I will have a cure ready for you in 2 years", you just ask if it has completed phase III trials now. If the answer is no, then it surely will not be ready for the mass market in two years.

I'm not exactly sure how big the wide end of the funnel is. Basically, how many times per year researchers find a cure for type-1 diabetes in mice. My gut feeling is that it is happening at a rate of about 4 times per year, right now. (Although in the past I think it might have been only 2 or 3 per year.) Those possible cures are the "raw material" that the funnel works on.

Another thing to remember is this: don't get excited about cures in mice, because they hardly ever lead to cures in people. That's the whole funnel idea. The mice side is big, but the human side is small. Based on my estimates, it appears that only about half of the mice cures even enter human testing.

I do have a pretty good idea of how many possible cures are at different points within the funnel:
(As of Nov 2008)

• There are 15 possible cures preparing for Phase I trials now.
• There are 10 possible cures in Phase I trials now.
• There are 5 possible cures in Phase II trials now.
• There are 3 possible cures in Phase III trials now.
• There are none waiting for marketing approval now.
  

See the funnel shape? From 15 down to 9 down to 5 down to 3 to 0.

Why are newspaper articles, bloggers, and researchers always saying that a cure is 3-5 years away? Why are they always wrong?

This is easy to explain, but also very galling. It boils down to over-optimism coupled with inexperience. So, some researchers have been working for years, maybe decades, to find a cure for type-1 diabetes in mice, and they finally do it! They are ecstatically happy, and they publish a press release.

So the first thing to notice, is that most reporters just rephrase the press release. They don't have a deep enough science background to understand it or know about similar research. Most bloggers don't even do that: they just cut-n-paste the press release. So the press release is the news. (Except for a few science reporters who really do know what is going on.)

Back to the researchers, who are writing their press release. First off, it is totally optimistic. It doesn't mention any possible problems or complexities. Also, they know the research needs to go through three phases of testing before it is generally available, and they know that each phase might only take a year, if all goes perfectly. (They always forget about the FDA production and marketing approvals.) So therefore, they say "It will be ready in as soon as 3 years" or "it will be ready in 3-5 years" or something very much like that.

But this is never true! For one thing, almost all phases take more than a year. Some take three years. For another, it assumes they start human trials the next day, and they never do. For a third, it ignores the time to get approval for manufacturing and marketing. And finally, it assumes that Phase II starts the day after Phase I ends, and Phase III the day after Phase II ends, and so on. And this is never true either. Often after the end of one phase, there is a delay while investors are found, the FDA is consulted, results are analyzed, and new trial subjects are found. So the result of all this is that these predictions of 3 years are never even close to right. The real number is 10 to 15 years.

So, why is this exact same mistake made with every new mouse cure? There are three reasons. First, it is usually the researcher's first time in curing diabetes in mice. It's not like they have done it before, so they don't realize the trap they have fallen into. Second, there is no "downside" to being wrong. Let's say they tell the reporters that the cure will be available in 3 years, and they are wrong. What bad thing will happen to them? Nothing. There is no reason for them to try to be accurate, because it just doesn't matter. No one is going to come back to them years later and stop a grant because they were wrong about this! Third, and most important, there are good reasons for the researchers to be over optimistic. At this point they are either trying to get grant money to do their human trials, or they are trying to get venture capital money to start a company to do their human trials. In either case, it is much better for them to say "we can cure it in 3-5 years" rather than "we can cure it in 10-15 years", even if the first is true and the second is not.

Some extra Information on the Phases

Phase-I
The goal of this phase is to prove safety; showing that the treatment works is a secondary goal. Therefore, phase-I trials usually involve only a few people (usually around 10), and usually use the lowest dosage possible of the treatment.

One thing to remember about these trials, is that they are rarely successful in terms of the treatment they are trying. Because the smallest possible dosage is used (for safety reasons), the results are often partial. So for type-1 cures, in phase-I you might temporarily cure type-1 diabetes, but have it come back later. Or maybe you use less insulin, but you still need to use some. Or your A1Cs get better, but still not good enough. These are all partial successes, which should be expected in phase-I trials.

Phase-II
The goal of this phase is to prove effectiveness: that the treatment actually works. Another goal is to show that it is safe in a larger segment of people. The doses used are often higher than phase-I. Sometimes different doses are used in different groups of people. These trials usually involve more people (often between 50 and 100).

At this point, you really should be seeing something close to what you want. The doses are higher and more people are involved.

Phase-III
The goal of this phase is to make sure that the treatment works in a larger group of people, and as sort of a "dress rehearsal" for actual use. The doses used here are generally the same as the doses that will be used in the general market. These tests usually involve hundreds of people.

At this point, the results seen in trial are pretty much the same as the results you should expect in the mass market. So no more wishful thinking: what the study shows is likely what will be available when the treatment is mass marketed.

Phase-IV
I have sometimes seen this term used for clinical trials that use a treatment that is already approved for one purpose, to test to see if it will work for other purposes. For example, you might have a drug that curred one form of cancer, and test it to see if it will cure diabetes, also. I think of these more like phase-II trials.