Genetic tests devised at Duke University by researchers using the new methodology have turned out to be worthless, though they were once hailed as breakthroughs. Two new blood tests for ovarian cancer have also been abandoned.
Despite the setbacks, researchers say they cannot give up on their quest for cancer signatures. Dr. Lajos Pusztai, a breast cancer researcher at the University of Texas MD Anderson Cancer Center, is one of them.
How many new drugs, he asks, were approved for breast cancer treatment in the past decade? His answer: seven. None was much different from drugs already on the market.
Yet in the same decade, he said, there were 8,000 publications in medical and scientific journals on breast cancer and more than 3,000 clinical trials at a cost of over $1 billion. “What came out of this is seven ‘me too’ drugs,” Pusztai said.
Yes, there have been studies showing single genes can go awry and fuel certain cancers, he and other scientists say. And yes, those studies have led to new drugs, so-called targeted therapies, that block the genes, extending the lives of some patients with some types of cancer.
But for a major advance in the way cancer is diagnosed and treated, Pusztai and other researchers believe that work must continue on genomic signatures. It’s a very different sort of science, an elaborate enterprise that involves complicated analyses of patterns of genes or other components of cells.
The hunt for cancer signatures also is a type of work that requires a leap of faith. It is impossible for scientists to use their intuition to know whether a signature has any biological meaning – it is just a pattern, and the meaning comes from its statistical association with a result.
By looking for these signatures, scientists are looking for a sort of next generation of biomarkers, and biomarkers have a troubled history in cancer research.
About 15 years ago, when the world was simpler, the American Society for Clinical Oncology called together a group of experts and asked them to develop some guidelines. Which cancer biomarkers are useful for making clinical decisions?
Dr. Daniel Hayes, a breast cancer researcher at the University of Michigan, was part of the group.
“We all kind of sat around and looked at each other and said, ‘We have no idea how to do this,”’ Hayes recalled. The field of tumour biomarkers, he said, “had been so chaotic.”
All too often, researchers claimed to have discovered reliable ways to identify a particular cancer, but studies confirming they were valid had never been done.
The group ended up writing a paper with what they called a “level of evidence scale” outlining the results they needed before they would believe a claim. Only a handful of tumor biomarkers met their criteria.
Yet, Hayes said, “during that time there had been hundreds of putative markers reported for breast cancer alone, let alone other diseases.”
With genomic signatures, the situation is more involved than it was 15 years ago, but the many of the same problems remain, Hayes and others say.
Signatures can be used to identify cancer cells in the way a tartan can identify a Scottish clan, Hayes said. “Each tartan is composed of threads of several different colors, but when woven into a single cloth, presents a distinctive pattern or signature that distinguishes one clan from another,” he said.
The tools used to find signatures are so complex they can be misleading if the results are not tested properly. Investigators look for patterns in huge arrays of genes or proteins or RNA molecules, and they constantly find spurious associations with cancer that look for all the world like true ones.
“The question is, what is noise and what is real?” said Dr. Steven Goodman, a biostatistician at Johns Hopkins University. In these studies, he added, “you are guaranteed to find things,” and almost always what is found is nonsense.
Gene or protein patterns, said Donald Berry, a statistician at MD Anderson Cancer Center, “are very difficult to get right.” Finding them, he said, “is like looking for a needle in a haystack when you can’t tell the needle from the hay.”
Adding to the confusion is that the research requires a group of experts, each of whom has a different, highly technical skill. Each person on a team may be so specialized that no one is qualified to know exactly what his or her colleagues are doing.
Pusztai’s team, for instance, includes pathologists, molecular biologists and biostatisticians. “No one person on the team sees all the pieces together,” he said.
For example, he said, just analysing cancer tissue for a genomics study involves 200 to 3,000 steps and takes several days to complete. “Any one of these steps can go wrong, and a good researcher should know what can influence the success of each step and control for the quality,” he said.
What comes out of this analysis is “reams of numbers,” Pusztai said. “If one were to print these out it would amount to thousands of pages. The interpretation of these numbers is purely statistical and mathematical.”
Even when researchers find a real association, it may not be clinically useful. A genomic signature test that correctly identifies most tumors that will respond to a drug but incorrectly identifies others may not be of net benefit to patients.
What is needed, and what rarely has been done, is research to see if a test based on a new biomarker does more good than harm.
It’s expensive and time consuming, but it is the sort of evidence-gathering that is almost always done to see if a new drug is effective, Hayes noted. Yet there is little incentive to evaluate biomarkers, because the Food and Drug Administration does not require it and companies are not reimbursed as much for tests based on biomarkers as they are for new cancer drugs.
“There is a whole series of things in this cycle that need to be broken,” Hayes said.
One step that could make a big difference, scientists said, would be for researchers who think they have found a new genomic signature for cancer to publish enough of their data and analysis for others to verify their work. Surprisingly few have done so.
The only reason the Duke research was discovered to be flawed, in fact, was that it relied on publicly available data sets and algorithms. Even so, unraveling the details of the work was so complicated that it took Keith Baggerly and Kevin Coombes, two statisticians at MD Anderson, 2,000 hours to find all the errors.
Perhaps illustrating the perils of the needle-in-a-haystack approach, the only two genomic signatures for cancer that have been validated, used in the tests Oncotype DX and MammaPrint, were discovered in a very different ways.
In both cases, researchers started with hundreds of genes that they had some reason to believe were important. They winnowed the collection down to those that seemed to be clinically useful. The Oncotype DX assay relies on 21 genes, and MammaPrint on 70. Both companies then rigorously tested their signatures to be certain that they were accurate in women with breast cancer. Oncotype DX showed its signature could predict prognosis and whether women would benefit from chemotherapy.
Yet the two signatures used to make these tests have just one gene in common. “What it means, as I suppose everybody is beginning to know, is that cancer is a very complicated thing,” Berry said.
No comments:
Post a Comment