Sanjay Kaul wrotes:
By statute (“the least burdensome” pathway), the approval standard for devices by the US FDA is lower than for drugs. Before a new drug can be marketed, the sponsor must show “substantial evidence of effectiveness” as based on two or more well-controlled clinical studies (which literally means 2 trials, each with a p value of <0.05, or 1 large trial with a robust p value <0.00125). In contrast, the sponsor of a new device, especially those that are designated as high-risk (Class III) device, need only demonstrate "substantial equivalence" to an FDA-approved device via the 510(k) exemption or a "reasonable assurance of safety and effectiveness", evaluated through a pre-market approval and typically based on a single study. What does "reasonable assurance" or "substantial equivalence" imply to you as a Bayesian? These are obviously qualitative constructs, but if one were to quantify them, how would you go about addressing it?
The regulatory definitions for “reasonable assurance of safety and effectiveness” are provided below.
Evidence for Safety
Are there reasonable assurances, based on valid scientific evidence that the probable benefits to health from use of the device outweigh any probable risks?
Evidence for Effectiveness
Is there reasonable assurance based on valid scientific evidence that the use of the device in the target population will provide clinically significant results?
My reply:
I’d prefer to set this up in a decision-theoretic framework. What are the costs and benefits of approving or not approving a new drug or device? I’m not really sure the best way of thinking about this, though. I’d be interested in the opinions of some real experts, such as: John Carlin, Chris Schmid. Don Berry, and Don Rubin.
Andrew: I didn't make your list of "real experts" (rightly so), but since I've worked a lot with the Berrys I'll take a crack at this one ;)
Device and drug trial regulation are just fundamentally different. Drugs has 3 phases
(really 4, if you count the post-market phase, which few companies take all that seriously
since it comes *after* approval for sale). Drug trials (at least after Phase I) are strongly
focused on efficacy, and have been ever since the patron saint of biostatisticians, Sen. Estes Kefauver, got this one through:
http://en.wikipedia.org/wiki/Kefauver_Harris_Amen…
The Kefauver amendment has been interpreted by FDA to mean the sorts of rigorous (frequentist)
testing that Sanjay talks about. But on the device side, there are no "phases", and
the testing that is required (by the CDRH branch of of FDA, not the CDER which
regulates drugs) tends to focus on safety and showing that a new treatment (say,
a new cardiac pacemaker) is as good as what has come before — that is, it need not
be shown to be significantly *better*. Again, Sanjay mentions this.
So, as is often the case in biopharm, it's regulatory requirements that are driving
this. We can talk about decision theory but it might be better to talk about game theory:
what can Player 1 (the company) do to maximize the chance that Player 2 (FDA) approves the device? ;)
Those requiring more on Bayesian methods for drug and device approval might benefit
from checking out my new book on the subject,
http://www.amazon.com/Bayesian-Adaptive-Methods-C…
This book was the #1 seller at the CRC table this past August in Vancouver, even outselling
the company's all-time sales leader, Gelman et al!
Hi, Brad. Thanks for the comments. Also, send me a copy of your new book and I'll review it here!
One difference between devices and drugs is that the former are much better understood.
If I say that this version of a pacemaker is the same as the previous version except for some mechanical variation, it's likely the new device acts a lot like the previous one.
But when someone makes an analogous argument for chemical compounds, there's more uncertainty. If you say this molecule is just like this other molecule with a couple more atoms tacked on, the properties may be radically different.