Why 23andMe’s Sample Swap is Actually an Argument In Favor of DTC Genetic Testing
Direct-to-consumer (DTC) genetic testing company 23andMe made news this week thanks to a lab mix-up that left up to 96 customers reviewing genetic data that was not their own. Full details of the mix-up, and analysis of 23andMe’s response, are available from Daniel MacArthur at Genetic Future and Turna Ray at Pharmacogenomics Reporter.
23andMe’s sample swap follows close on the heels of the FDA announcing an investigation into Pathway Genomics and Congress launching an even broader investigation of the three leading DTC genetic testing providers. Not surprisingly, many commentators have pointed to 23andMe’s mix-up as just the latest example of the dangers of DTC genetic testing and further evidence of the need for greater federal regulatory scrutiny.
There are a number of reasons why DTC genetic testing may soon find itself subject to increased federal regulatory oversight. However, 23andMe’s widely publicized data error should not be one of those reasons. In fact, the sample swap, while unfortunately timed, actually presents a compelling argument in favor of the direct-to-consumer model for genetic testing.
Reviewing CLIA and Analytical Validity. To understand this counterintuitive notion, we begin with a review of the regulatory landscape surrounding genetic testing. Recall that DTC genetic tests are, for the most part, simply a subset of what are known as laboratory developed tests (LDTs). The FDA currently exercises “enforcement discretion” in regulating all but a select subset of LDTs, although some commentators – and perhaps the agency itself – believe that certain types of LDTs, including those that are provided directly to consumers, might warrant additional oversight.
The FDA is not, however, tasked with overseeing the analytical validity – in other words, the accuracy and reliability – of genetic tests. Ensuring analytical validity is the responsibility of the Centers for Medicare & Medicaid Services (CMS), which enforces the Clinical Laboratory Improvement Amendments of 1988, or CLIA. Under CLIA, labs that perform human testing, including genetic testing, are regulated through a certification process. For a lab to obtain CLIA certification, it must satisfy CMS requirements relating to quality control, personnel qualifications, records maintenance, and proficiency testing.1 CLIA certification also requires the laboratory to demonstrate the analytical validity of its tests.
In its 2008 report on “The U.S. System of Oversight of Genetic Testing” (pdf), the Secretary’s Advisory Committee on Genetics, Health, and Society (SACGHS) expressed concern that CLIA’s proficiency testing (PT) requirements may not adequately ensure the analytical validity of genetic tests:
There are also questions about the sufficiency of CLIA’s requirements for assessing the performance of genetic testing laboratories. While CLIA requires laboratories to have quality assurance programs in place, most genetic testing laboratories are not required by CLIA to perform PT unless they are testing a small subset of established analytes regulated under CLIA, none of which are genetic tests per se. PT serves as an assessment of laboratory competence by comparing a laboratory’s test performance and results to an established external standard, and it is considered to be the most rigorous form of performance assessment currently available. In principle, all genetic tests and other high-complexity tests should be required to undergo PT. Thus, gaps in oversight still exist regarding the regulation, breadth, costs, and availability of testing materials for existing PT programs.
Plugging the Right Regulatory Gap. Would establishing PT guidelines for genetic testing have helped 23andMe and LabCorp (the CLIA-certified genetic testing laboratory used by 23andMe) spot the error before returning erroneous results to customers? Possibly. There is no way to say for sure since those PT guidelines do not exist. Even if they did, many CLIA-certified laboratories also participate in voluntary accreditation programs that already go beyond the likely PT requirements for genetic testing (and, as Daniel MacArthur writes, this particular mistake is one that likely should have been prevented under current industry best practices). [Added in edit: It is important to clarify, as a careful reader pointed out, that this particular mistake was more appropriately characterized as one of quality control, not analtyical validity.]Ultimately, however, even the most robust regulations, whether CLIA or otherwise, are incapable of eliminating all mistakes.
What we can say, however, is that preventing this type of mistake from happening in the future means addressing deficiencies in regulations that apply to all genetic tests, not just genetic tests of the DTC variety. The most commonly discussed regulatory changes for DTC genetic testing – in particular, a requirement that test results be ordered and/or delivered only under the supervision of a medical professional – would have done nothing to reduce the likelihood that the tests results themselves were accurate.
To put it simply, the mistake made by 23andMe had nothing to do with the DTC element of its genetic test. It was a mistake that could – and does – occur in both the clinical and consumer setting. It may be that additional regulation – specifically a new specialty area and PT standards under CLIA – is needed, but such regulation should clearly apply to all genetic tests, not just DTC genetic tests.
An Argument in Favor of DTC Genetic Testing? In his analysis of the 23andMe sample mix-up, Daniel MacArthur reprises one of his previous columns to make an important point: consumers must not take their genetic test results for granted. MacArthur urges genetic test recipients to check for obvious errors and to engage with their data, “rather than being a passive recipient of genetic forecasts.”
In fact, as set out in Ray’s and MacArthur’s coverage, 23andMe’s active user community did just that – engaged with their data – and helped to rapidly identify the sample swap, enabling the company to quickly notify its customers and retest the samples. One particularly active 23andMe community member, Nora Probasco, described her own experience:
Yes, I was initially shocked and upset, when I received the results. I also knew they had to be wrong. I contacted others on the ISOGG (Int’l Society of Genetic Genealogists) mailing list to find out who else got bad results. I also monitored the 23 & Me Community Forum and also found others. Many of us worked together to contact those we knew at 23 & Me to find out what the problem was. Later we were sent messages explaining there was a problem and the bad results were pulled. Some of us received phone calls to discuss it with 23 & Me as well, and asked if we had any questions.
The last time that a DTC genetic testing provider committed a high-profile error with one of its user’s data it was a software error by deCODEme that caused New Scientist writer Peter Aldhous to wonder, at least temporarily, if he was even human. Similar to Probasco, Aldhous took matters into his own hands to verify that he was, indeed, a member of the species Homo sapiens (see Leveraging the Crowd to Understand Your Genome).
Yet as the availability of genetic testing expands, it is unreasonable to expect every individual to be as diligent in vetting their genetic results as MacArthur suggests, and as Probasco and Aldhous have demonstrated. Indeed, the number of individuals who prefer to be “passive recipients of genetic forecasts” will only increase as genetic testing moves from early adopters into mainstream society. Individuals who are either unwilling or unable to validate their own genetic test results place the onus on regulators and genetic test providers to ensure that mistakes, when they inevitably occur, are detected and corrected as quickly as possible.
Improved regulation, particularly expanded CLIA oversight across all genetic tests, not just DTC genetic tests, is one way to reduce the incidence of genetic testing errors. But it is unrealistic to expect regulation to eliminate such mistakes entirely, which means that complementary strategies must also be pursued. Judging from the response to 23andMe’s sample swap, the presence of an engaged community of genetic testing consumers appears to be one such strategy.
It is worth considering, for instance, whether the same sample swap would have been as quickly identified and addressed – particularly in the case of a “passive recipient” – had the genetic results been delivered to 96 separate doctor-patient pairs (with the patients possibly unable to access the underlying genetic data at all). At least in this particular example, the inevitability of genetic testing errors argues in favor of more consumer access – not less – to allow individuals to continue to play an active role in finding and correcting such mistakes.
There is no question that 23andMe made a mistake, and must take appropriate measures to help prevent similar errors from occurring in the future. But while the timing of the incident is unfortunate, it seems that the response to this mistake – particularly from 23andMe’s community of users – is more indicative of DTC genetic testing’s promise than its perils.
1 While most states use CLIA to certify laboratories, a few states, including New York, have state regulations and certification programs that are more stringent than CLIA.