Discrepancies in interpretation – when can exomes speak from themselves?

Interpretation. There is huge promise in discovering the genetic basis of neurodevelopmental disorders using exome sequencing, but it is often not clear how ambiguous results are communicated to families. In a recent publication in Clinical Genetics, the authors try to understand what happens to exome results as they land on the clinician’s desk – and leave us with the conclusion that diagnostic exome sequencing when reviewed in a clinical setting may have a false positive rate of up to 20% with 5% of false negatives.

Main results by Shashi and collaborators. In 25-30%, exome sequencing results in positive findings. Of those, 80% are considered diagnostic by clinicians. Of the 70-75% of negative exomes, 5% are considered positive upon closer review by the clinician. These discrepancies are mainly due to additional genetic and phenotypic data that is available to the clinicians.

Main results by Shashi and collaborators. In 25-30%, exome sequencing results in positive findings. Of those, 80% are considered diagnostic by clinicians. Of the 70-75% of negative exomes, 5% are considered positive upon closer review by the clinician. These discrepancies are mainly due to additional genetic and phenotypic data that is available to the clinicians.

The exome. Last week, we sent off quite a few exomes. When I try to summarize the overall expectations of this technology for students during rounds, I usually refer to what I would call the “exome mantra”: we expect that we find a diagnosis in unclear neurodevelopmental disorders in 25-30%. Of those patients with a diagnosis by exome, we would expect that the exome result significantly changes management in 10%. Presenting exomes that way somehow assumes that results come back as a clear and definite answer, be it positive or negative. What I leave out in such a discussion is the fact that there might be some ambiguity when it comes to exome results: do I actually believe the official diagnostic report, and is my final diagnosis the diagnosis that the exome lab suggested? In their recent publication in Clinical Genetics, Shashi and collaborators try to address this very question.

The gold standard exome. Exome interpretation is difficult on a conceptual level – there is no gold standard. Most genetic diagnoses made by exome do not have a clear biomarker and no clear diagnostic standard that we can refer to to test the precision of our prediction. Not having a gold standard is a major problem – there is basically no way to know how good we are, as it often comes down to interpretation and indirect evidence. There are some findings that are easy to interpret – the obvious truncating mutation in SCN1A or the previously described pathogenic homozygous mutation in EPM2A. But in many cases, the situation is not as clear-cut and the decision to call a given exome positive or negative relies on unvalidated assumptions. In the absence of a gold standard, one way of finding out how good we are is to assess how well we agree with each other when we analyze exomes.

Lab vs. clinician. Shashi and collaborators tried to follow up the results of 93 patients who underwent exome sequencing. They investigated how the clinical interpretation of the lab results changed the diagnosis or was consistent with it. Overall, the results are quite encouraging. In patients 24/93 (~25%), exome sequencing showed a positive result and in 19/24 (80%) of cases, the clinicians agreed with the molecular diagnosis of the lab. However, in 20% of patients reported to be positive by the diagnostic lab, the clinicians thought that the suggested molecular diagnosis was not correct. Also 3/60 patients (5%) that were considered negative by the exome lab or had a lower confidence diagnosis, were eventually found to be positive at the review of the exome data by the clinicians. In summary: 20% false positives, 5% false negatives.

Quality criteria. Is 20% false positives and 5% false negatives good or bad? The truth is that this is difficult to tell as there is no gold standard, which suggests that using the phrases “false positive” and “false negative” is actually incorrect – these numbers are simply comparing the agreement and disagreement between interpretations in different clinical settings. Interestingly, the extent of disagreement also depends on the type of genetic variation seen. While there is significance concordance in the evaluation of de novo mutations, the evaluation becomes more difficult when recessive disorders are taken into account. This hints at two different scenarios that an exome result can have: the de novo result that relies more on big data than on post-test clinical judgment and the recessive result that needs significant clinical interpretation. For many de novo mutations, exomes may be able to speak for themselves; for recessive conditions, we can expect much work and follow-up testing in the clinical setting to test whether the diagnosis is correct. It will be interesting to watch quality studies for exome interpretation emerge in the near future. From the perspective of the laboratory, it is crucial to receive feedback from clinicians to improve the analytic pipeline in the case of discrepancies. In the end, exome reporting remains a dynamic process.

Ingo Helbig is a child neurologist and epilepsy genetics researcher working at the Children’s Hospital of Philadelphia (CHOP), USA.

Twitter