WHY NO ANTIVIRALS: A CASE HISTORY OF FAILED TRIAL DESIGN

AZT provides limited benefits to persons with AIDS or HIV, and many people cannot use it at all. Many promising new antivirals have long been in the research and regulatory pipe-line: for example, DDI, AZDU (CS-87), D4T, DDC, hypericin, and trichosanthin (compound Q). None has become available since AZT was released almost three years ago. And at the Montreal AIDS conference earlier this month, we learned why none will become available for years unless certain current practices in the design of clinical trials can be changed. This article will illustrate some of the problems, and suggest solutions.

The basic problem lies not in any single agency, company, or other institution, but instead in a conventional wisdom which cuts across institutional boundaries. A professional consensus guides the design and conduct of clinical trials, and the shepherding of experimental drugs through the testing system. This consensus today includes certain assumptions which make it impossible for the existing system of clinical trials and drug approval to respond successfully to AIDS as a public-health emergency.

Note and disclaimer: Readers may notice that this issue of AIDS TREATMENT NEWS has a call for volunteers (above) for the same trial analyzed below as an illustration of a failure of the clinical-trial system. This is not an oversight or contradiction.

This trial is no worse than other AIDS studies. It seems to be ethical in its treatment of volunteers. The problem is that it will not produce results for years. But for now it is the trial we have, so we must support it.

For the same reason, this article is not intended as a criticism of persons conducting this trial, nor of it's sponsor. They have done well within the system of shared assumptions which controls all mainstream AIDS research. It is this system which needs reform.


A Case History: New DDC Trial

DDC (dideoxycytidine), an antiviral like AZT but with different toxicities, is not the most important new drug. But it is farthest ahead in the drug-approval pipeline among major antivirals. Because it is ahead of the others, and plans for a major new study have been revealed, it provides an excellent case study of the problems which will impede the approval of all important new antivirals, not only DDC but also more interesting drugs such as DDI.


DDC Background

DDC, like most of the new AIDS antivirals, was discovered to have anti-HIV activity by U.S. Government scientists. The United States then asserted exclusive worldwide rights, and assigned these rights to a pharmaceutical company (in this case, Hoffmann-La Roche, Inc. of Nutley, New Jersey).

Several trials have already been conducted. In early studies, some patients developed severe peripheral neuropathy, causing numbness or pain in the feet. Later human studies found that lower doses could reduce P24 antigen levels, a sign of antiviral activity, with manageable toxicity.

On June 5, 1989, Hoffmann-La Roche announced new trials, designed in cooperation with the FDA (U.S. Food and Drug Administration). A major phase II trial, which could lead to marketing approval for the drug, will compare low-dose DDC head-to-head with AZT "in persons with AIDS or advanced ARC."

The problem with this trial is that because of the design chosen, it is unlikely to produce any conclusion for two and a half years.

And since DDC is ahead of all other major antivirals in the drug- approval pipeline, and the delays in this study design are generic to AIDS antivirals and not specific to DDC, it is likely that all major new AIDS drugs will face a similar delay. This fact alone strongly suggests that no major new treatment for AIDS will come out of the drug-approval pipeline for years, unless the assumptions currently guiding clinical trials can be changed.

An analysis of the design of the new DDC/AZT comparison trial, and the assumptions behind this design, will show exactly how this intolerable situation came about, and how it can be changed.


DDC Rumor: A Treatment IND?

Rumors have circulated that DDC may become more available through a "treatment IND" before the end of 1989. We hope these rumors are true.

But we are skeptical. The FDA has interpreted the treatment IND very conservatively, using it only near the end of efficacy trials, when the drug is almost sure to get full marketing approval after the final paperwork is complete. If this procedure is followed for DDC, a treatment IND will probably be more than two years away, as we will show below.

The record is full of comforting but broken promises that things have changed and therefore AIDS research will move faster in the future. When the future arrives, the public has forgotten the promises.

Why will the trial take so long?

This new phase II trial will compare DDC with AZT, using a randomized, double-blind design. No placebo will be used; every patient will get one of the drugs. The trial is scheduled to last two years; recruiting the subjects is expected to take about six months in addition.

In theory, the study could end earlier. A team of experts will periodically monitor the results, secretly breaking the code to see if there is statistical proof that patients getting DDC are doing much better or much worse than those getting AZT. In practice, however, for reasons explained below, it is almost impossible that this study will end this way. The researchers expect it to take the whole two years.

The reason that the study will take so long must be explained in several steps:

(1) The FDA will not approve a drug based only on "surrogate markers", meaning improvement in blood work such as reduction in P24 antigen, or T-cell rises. The FDA also wants statistical proof that the drug is helping people.

(2) After rejecting surrogate markers, the FDA has insisted on the slowest measure of clinical improvement "clinical end-points", meaning OIs (opportunistic infections) or deaths. This means that the drug being tested is not measured by improvements in the patients who receive it, but OIs or deaths in those who do not.

The DDC trial will compare that drug with AZT. Since AZT works fairly well for the first year, the number of deaths and

OIs in the control (AZT) group will be low. Therefore, even if the drug being tested were perfect and everybody taking it were cured instantly, the clinical trial design would not recognize that fact until enough deaths and OIs had accumulated in the control group to provide statistical proof that DDC was no worse than AZT.

(3) This study, like some others, will use a team of experts (sometimes called a "data safety monitoring board") to meet periodically and secretly break the code and examine the results so far, to see if the study should be ended early. The public is told that such reviews can end studies as soon as statistical proof of effectiveness is obtained.

But in practice it is unlikely that this or any similar study will be ended early. The reason why not involves an esoteric problem in statistical interpretation. If researchers take an early look at their data to decide whether to stop the study early and call the drug a success, but then decide that the data does not justify stopping, meaning that the study will run to its normal conclusion, then the very fact that they looked early means that they must tighten their interpretation of the final results. A drug which otherwise could have been considered a success might now need to be counted a failure just because the researchers looked at the data and might have acted on that information even though in fact they did nothing different as a result of the look.

This seemingly preposterous conclusion is hard to explain even to scientists, let alone to readers with no statistical background. We will try to do so; those who are not interested in the details can skip the next four paragraphs.

[When researchers claim statistical proof that their drug works, they are usually claiming that the drug passed a test which only a small percentage of worthless drugs could have passed by chance; the smaller the percentage, the better. For example, if a journal article claims that a result is "statistically significant at the p<0.01 level," this means that the probability (p) that a worthless drug could have done as well or better by chance alone is less than one percent (0.01).

What happens, then, if you look at the data early? Suppose that the researchers did not know about the problem that we are describing here, and they decided to take an early look at their data, and end the trial immediately if the drug was good enough to have reached the p<0.01 level already. If not, they would continue the study and see if they achieved that level later.

Clearly then the chance of accepting a worthless drug at some time in their trial would now be greater than one percent. This is because there is a full one percent chance to accept such a drug at the early look and if the worthless drug did not pass the test at that time, there is some additional chance that it could pass later. Since the overall probability of accepting a worthless drug is now greater than one percent, the researchers cannot correctly claim that their trial showed efficacy at the p<0.01 level. To honestly make that claim, the researcher must use a higher standard both for the early look, and also at the normal end of the study (if the early look did not result in the trial's termination).

This means that if researchers take an early look at their data but decide not to end the study as a result, they then must tighten their standard for judging a drug successful later. Drugs which would otherwise have been judged effective will therefore now be rejected. Clinical trial design can minimize this problem by making the early look be as conservative as possible.]

The practical effect of this statistical oddity is that researchers have a strong incentive to use an extremely conservative criterion for ending a study early. As a result, a "data safety monitoring board" provides less protection to the volunteers in a study than they may be led to believe. And the assurance to the public that experts are monitoring the trial and will end it as soon as the data justifies, thereby speeding final approval of the drug, is largely empty.

(Note: The AZT trial was stopped early in September, 1986, when there were 16 deaths in the placebo group, vs. only one death in the AZT group. No one knows why this extreme difference occurred, as later experience does not support a 16 to one difference in death rate with AZT. And despite this great difference in deaths, the decision to stop the study then has been controversial.)

During the Montreal conference, Hoffmann-La Roche conducted a press conference on DDC. The speakers were Thomas Merigan, M.D., principal investigator at the AIDS Clinical Trial Group at Stanford University, and Whaijen Soo, M.D., Ph.D., director of clinical virology at Roche. Few reporters came to this meeting, which was a mile away from the main conference. Our impression from the discussions at that press conference is that nobody expected the study to end before two years.

The important question is not whether to end studies early. It is whether the best way to prove a drug is to wait for deaths and OIs in those who do not receive it. This trial design makes studies inherently slow, whether they are ended early or not.

No one at the press conference raised the issue of whether a study design which will take more than two years to get results is an acceptable public health response to the epidemic. We are concerned that all the important AIDS antivirals are behind DDC in the pipeline. If they suffer the same delay as DDC, then we can almost guarantee that no major new AIDS antiviral will be generally available for at least two years.


Recruiting Problems Likely?

One of the problems with many AIDS clinical trials is that entry criteria are designed purely for scientific reasons, without thought as to whether there will be patients available to fit them. As a result, many studies take much longer than
intended, or even fail altogether, because of recruiting difficulties.

The DDC study may have this problem. Volunteers must have less than 200 T-helper cells, and also have had pneumocystis in the last four months or have certain ARC symptoms. And yet they must have never taken AZT. Most people will have already tried AZT before they have severe symptoms and under 200 T- helper cells.

Some may have never taken AZT because they chose not to. But they would be unlikely to volunteer for this study because 50 percent of the people enrolled, chosen at random, will go into a control group and receive AZT instead of DDC.

It seems that the only volunteers left would be those who never took AZT because they could not afford it; in the study, the drug is free. But these people face another problem. The study also requires use of aerosol pentamidine, but will not pay for it. If persons could not obtain AZT in the past, how will they obtain aerosol pentamidine for the next two years into the future?

It would seem that these conditions, taken together, systematically exclude almost everybody from the trial. A few might get through, such as those whose first contact with the medical system is pneumocystis.

Notice how much of the problem with this study, including recruitment, stems from the decision to prove DDC by counting "clinical events" (deaths and OIs) in the control group. To get clinical events, the patients must be seriously ill although never treated with AZT. But once on the study, for ethical reasons they must receive an antiviral and pneumocystis prophylaxis, reducing the clinical events and therefore requiring more volunteers (therefore a multicenter trial) and a two-year dura- tion. All this to get enough deaths and OIs to allow the drugs to be compared.

An alternative would be randomized, double-blind trials designed to use patients' overall clinical condition as the outcome measure, not deaths and OIs. The problem seems to be that academic researchers do not trust physicians' evaluations in outcome measures in their experiments even within a double-blind trial because such evaluations involve some subjective element.

A body-count outcome sounds more scientific.


THE IDEOLOGY AND PUBLIC RELATIONS OF CLINICAL TRIALS

Having looked at the reality of the modern phase II clinical trial for AIDS antivirals, we will now look at the image. The image is important, because it is used to calm the public, justify the existing system, and impede calls for reform.

The DDC press packet from Hoffmann-La Roche provides a convenient look at this image. Any other public relations from a mainstream clinical trial would be similar, however, as government and other controls have imposed a research monoculture. Even the public front is uniform.

From a June 5 press release we learn that "Everyone collaborating on this project at Roche, the FDA and the National Institutes of Health is intensely aware of the urgency for developing safe and effective treatments for AIDS. Awareness of that urgency constantly compels us to work together as expeditiously as possible toward definitive results." We also learn that "Initial studies suggest that DDC may have an antiviral effect at the low doses that result in manageable toxicity. The studies now being planned are essential if we are to turn suggestions into medically useful conclusions."

An undated Dideoxycytidine (DDC) Fact Sheet includes a question and answer section on the availability of DDC. We quote it at length because it illustrates several aspects of the currently prevailing ideology of clinical trials.

"Q: When will DDC be available?

"A: That depends largely on the results of the new trials. When dealing with human life, the adverse effects profile and optimum dosage of a drug must be carefully studied no matter how urgent the need. As soon as the clinical data warrant, Roche will file a New Drug Application (NDA).

"Meanwhile, each of the new trials has entry criteria specific to its design, and some of the studies already have their full complement of volunteer patients. People who would like to participate in, or simply learn more about, the trials should call FDA (sic) at 800/874-2572 (800/TRIALS-A) or Roche (collect) at 201/235-2355.

"Q: Will Roche provide DDC on a compassionate plea basis?

"A: The urgent need for more effective weapons against HIV weighs heavily on everyone associated with this project at Roche, the FDA, and the NIH. However, at present, we are agreed that the clinical data now available are insufficient to justify distribu- tion or use of DDC against AIDS outside of carefully controlled clinical trials. Only new data can change this situation. Consequently, we are working closely together to expedite the next round of therapeutic trials, from which the medically necessary data will flow.

"Q: When will Roche submit an NDA for DDC?

"A: Roche will submit an NDA as soon as the data from the pivotal studies allow. A special review board will continually evaluate data from all of the trials and make appropriate recommendations to FDA."

Some points to note about the world of AIDS treatment research according to press releases:

(1) Everyone involved feels urgency, and is working well with everyone else. (During the press conference, however, this reporter could find no shred of evidence of urgency.)

(2) More studies are, of course, essential. (300 people have already been given DDC in clinical trials,.)

(3) The phrase "no matter how urgent the need", in the context of justifying withholding a drug until more studies collect still more information about "the adverse effects profile and optimum dosage", clearly illustrates the fact that no weighing of costs and benefits (of the extra studies and their associated delays) will be considered. Instead, persons with AIDS can simply get lost until the researchers are finished. In theory they might join the study, but in practice less than one percent of persons with AIDS or related conditions will be able to do so.

Incidentally, the dose has already been determined well enough to bet this entire phase II study on it, as only one dose will be used in this study.

(4) Unless they qualify for a trial, patients and their physicians have no role in the decision of whether or not to use a drug, until someone is ready to sell it to them. This decision is made for them, by agreement between government officials and potential vendors. For AIDS, the answer is almost always no. Other diseases have been treated more liberally.

(5) The public is not told that the reason the trial will take so long is that deaths and OIs must be accumulated. Instead, the public is told that the trial might not take two years but could end any time, because experts will watch over it and pull the plug as soon as medically possible, moving the drug to the next step in the approval pipeline. As we have seen, this study will almost certainly take more than two years.

(6) The Dideoxycytidine (DDC )Fact Sheet also said that the first comparison trial was expected to "begin" in July and last up to two years, "depending on results". Readers might assume that the maximum delay for this trial is therefore two years and one month. This assumption would be wrong.

The six months for recruiting subjects was an informal estimate mentioned by one of the researchers at the press conference. Past experience suggests that it is probably optimistic.

Note that a trial which "begins" in one month and lasts "up to" two years may take far longer than 25 months to be finished. This is because the trial "begins" with the recruitment of the first subject, but the two-year clock starts only with the recruitment of the last. In addition, multicenter trials often have recruitment quotas for different centers, meaning that the clock starts only when the slowest center is ready.

The difference recruitment can make is illustrated by a study of Imuthiol (DTC). Over two years ago, on April 10, 1987, AIDS TREATMENT NEWS reported that this six-month study was underway. Most centers recruited patients promptly and completed their phase of the study. But because of stragglers, this sixmonth study was still running two years later, and the data from those subjects who completed the trial long ago has not been released.

The point is that press releases about clinical trials are designed to provide a comforting picture of reality. Since clinical research is a forbidding area complex, esoteric, and involving risk to human life few in the media or elsewhere have looked behind the image. It is much easier to trust the experts.

Because of lack of understanding of what is really going on, people are repeatedly surprised at the lack of new drugs for AIDS. Those who do look will realize that the current system of clinical trials could not possibly meet the needs of the AIDS emergency, and is very unlikely to release even a single important new AIDS drug for years even though the drugs are there. The drugs which will provide the important treatment advances of 1992 and 1993 are already available and quite well known we named some of them above. But DDC will take over two years for the upcoming trial alone, and all the other important antivirals are behind it in the pipeline, so they will take longer still.


THE CENTRAL ISSUE

The key reason no new antivirals are available is that clinical trials have waited for deaths and OIs, instead of looking directly at clinical benefit, which with some drugs is dramatic. Conservative trial designers have used deaths and OIs because the numbers seem more "scientific" than clinical ratings, which depend in part on judgments of physicians, and may not be identical from one researcher to the next; by contrast, everyone can agree precisely on the number of deaths in the treatment and control groups. However, "softer" kinds of data such as average ratings by panels of experts have been handled successfully in many fields of science. When we are looking for dramatic effects, as with DDI or Compound Q, these methods have more than enough precision to do the job.

The AIDS community must continue to raise the issue of whether counting deaths and OIs is truly the only legitimate way to tell whether an AIDS antiviral is working. Using "surrogate markers", such as T-helper cell count or P24 antigen level, is one approach to faster study design.

But we fear that surrogate markers alone are not enough that to try to use blood work as the sole basis for approving new drugs would lead to a long and unproductive argument. A middle ground, which we believe will be most productive, is to use sur- rogate markers and also direct measurements of clinical benefit in persons taking the drug, such as numerical ratings based on examinations by physicians. The variability in the course of the disease, sometimes cited in arguing against this approach, can be controlled for by double-blind designs, and by well-known statistical methods. The purpose here is to find the dramatically effective drugs, the home runs, and to test them quickly; slower study designs are acceptable when researchers are looking for minor or marginal differences.

The issue of surrogate markers is already receiving serious professional attention. But the related issue of the reluctance of trial designers to use direct measures of clinical benefit as proof of efficacy of AIDS antivirals, instead of insisting on deaths and OIs in the control group as the only important measure, has been largely overlooked. The AIDS community must insist that this issue be considered on it merits, as this key reform will allow the most important new AIDS antivirals to be tested many times faster than by the methods now in use.


STATEMENT OF PURPOSE

AIDS Treatment News reports on experimental and complementary treatments, especially those available now. It collects information from medical journals, and from interviews with scientists physicians, and other health practitioners, and per- sons with AIDS or ARC.

Long-term survivors have usually tried many different treatments, and found combinations which work for them. AIDS Treatment News does not recommend particular therapies, but seeks to increase the options available. We also examine the ethical and public-policy issues around AIDS treatment research.