PCR tests and false positives

[MikeWM]

This seems very important. Our current strategy of widespread testing appears to be ignoring the fact that the PCR test has a non-zero false positive rate, which has major implications. In particular, at the moment it appears that *even with a positive test* it is more likely than not that you *don't* have the virus! This is really not good when serious decisions are being made based on the results of these tests.

Article in today's Telegraph (paywalled, sadly)
https://www.telegraph.co.uk/news/20...ns-coronavirus-pandemic-may-never-officially/

...even at the most optimistic levels, false positives could be hugely skewing the current testing figures.

For example, take the current testing rates. Roughly 310,000 tests are being carried out each day at the moment and the Office for National Statistics (ONS) suggests there is a current community infection rate of 0.05 per cent.

That means that if testing was perfect, we should be picking up 155 positive cases a day and 309,854 negative cases.

Yet with the current false negative rates, we would miss 20 per cent of the positive cases, picking up just 124. More alarmingly, we would pick up 310 false positives, giving a total of 434 reported cases, nearly three times the real positive figure.

It also means that the probability of an individual with a positive test actually being infected would be around 28.5 per cent.

The anomaly also means that even if there was no virus in the community at all, we would never fall below 310 cases a day if we were testing at current levels.

And this quote from Carl Heneghan from the Oxford CEBM, who as I've mentioned before has been one of the few people worth listening to throughout (his paper exposed the issue with the death figures last month, for example)

“It does matter when your prevalence is very low. At this point if you have a positive test you are more likely not to be infected than to actually have the virus. Your chance of being infected is less than 30 per cent.

“The ONS currently cannot estimate prevalence because it does not now know what the false positive rate of PCR testing is.

“It looks like we’ll struggle to get out of this. We’re now in a spiral of bad data.”

(Bold mine, in both quotes)

 

[NorthOxonian]

Ah yes, the dreaded Bayes' Theorem strikes again. One of my favourite statistical paradoxes, and one that comes up all the time when dealing with "rare" events.

The best way to deal with such a problem is to try and find a further independent test - hopefully we have good medical minds on the case. Your "prior" probability after one positive test is quite high (30% is the example figure given), so a further independent positive result, even with a pretty unreliable test, would be a much clearer sign.

 

[Mojo]

Is the bigger risk that not that a test returns a false negative? This is something I wonder about especially given that most of the tests are now being self-administered.

I’ve had two tests and have never been able to do it myself (my tonsils were removed as a baby) without first touching the roof of my mouth or my tongue. My partner did a test and inserting the rod in his mouth caused him to projectile vomit all over the mirror and the hallway cabinet.

 

[AdamWW]

Ah yes, the dreaded Bayes' Theorem strikes again. One of my favourite statistical paradoxes, and one that comes up all the time when dealing with "rare" events.

And it's something that can be frustratingly difficult to explain.

"You said the test only gives a false positive 1 time in 100 so why is mine so likely to be false"?

Even without allowing for this the numbers being used for random sampling are pretty small - I presume the reason not to use more people is cost, not lack of tests (because it's a small fraction of the overall number of tests), so maybe a large increase in sample size would be worth throwing some money at so we have a better idea of what we're doing?

 

[ComUtoR]

"PCR" ?

 

[AdamWW]

"PCR" ?

From Wikipedia

Polymerase chain reaction (PCR) is a method widely used to rapidly make millions to billions of copies of a specific DNA sample, allowing scientists to take a very small sample of DNA and amplify it to a large enough amount to study in detail. PCR was invented in 1984 by the American biochemist Kary Mullis at Cetus Corporation. It is fundamental to much of genetic testing including analysis of ancient samples of DNA and identification of infectious agents. Using PCR, copies of very small amounts of DNA sequences are exponentially amplified in a series of cycles of temperature changes. PCR is now a common and often indispensable technique used in medical laboratory and clinical laboratory research for a broad variety of applications including biomedical research and criminal forensics.[1][2]

I don't think it was developed as a diagnostic tool in the way it's being used now.

Talking of 'false positives', I believe it lets you find tiny amounts of DNA (well, RNA in this case) on surfaces and jump to conclusions about how long coronavirus lasts, when you don't get infected from a bit of RNA - it has to be in an intact virus particle.

 

[MikeWM]

Talking of 'false positives', I believe it lets you find tiny amounts of DNA (well, RNA in this case) on surfaces and jump to conclusions about how long coronavirus lasts, when you don't get infected from a bit of RNA - it has to be in an intact virus particle.

Here's a rather technical paper on that

https://www.cebm.net/covid-19/infectious-positive-pcr-test-result-covid-19/

Part of the upshot there is that it seems you can have a 'true' positive test many weeks, even months, after you've recovered from the disease and stop being infectious, as you're still shedding bits of viral RNA even though the virus is defeated.

Add that to the 'false positive' problem, and you do have to question making massive decisions on lockdowns etc. based on these tests alone.

 

[Bletchleyite]

That article has one major flaw. With present policy, testing will find a much higher percentage of positives than the general prevalence, because people are only supposed to have a test if they think they are infected.

 

[MikeWM]

The best way to deal with such a problem is to try and find a further independent test - hopefully we have good medical minds on the case. Your "prior" probability after one positive test is quite high (30% is the example figure given), so a further independent positive result, even with a pretty unreliable test, would be a much clearer sign.

Yes, assuming that the reasons for 'false positives' are independent of the person giving the sample, just having a second test would be a fairly good indicator. (If they're not independent - if the tests are more likely to give a false positive on some people than others - then that doesn't help as much!)

The article actually mentions that

It might seem purely academic but there are a growing number of reports of false positives from real world scenarios. On July 24 it emerged that only one of seven staff members who tested positive at St Mirren football club actually had the virus. A Motherwell player also was retested and found to not be infected.

 

[MikeWM]

That article has one major flaw. With present policy, testing will find a much higher percentage of positives than the general prevalence, because people are only supposed to have a test if they think they are infected.

They've taken that into account I believe, hence no explanation given of why they say they would expect 434 reported cases when the daily figure is currently higher.

I think the point is more that it could never go below a certain number at current testing levels (around 300 a day) even if the virus has completely disappeared.

 

[Mojo]

That article has one major flaw. With present policy, testing will find a much higher percentage of positives than the general prevalence, because people are only supposed to have a test if they think they are infected.

Whilst these may be the minority, a number of people without symptoms may have a government test funded by the taxpayer:

- care home residents and staff
- essential workers
- those being admitted to hospital, where the hospital has requested they have a test
- people who live, work, or study, in one of the local authority areas that are problem areas

 

[Bletchleyite]

True, but that (plus the people who are lying about symptoms to get a test) isn't going to result in the expected positive rate on the tests carried out being anywhere near the community prevalence. It will be, and is, much higher, and that, not the Torygraph's theorising, is why.

 

[MikeWM]

Whilst these may be the minority, a number of people without symptoms may have a government test funded by the taxpayer:

- care home residents and staff
- essential workers
- those being admitted to hospital, where the hospital has requested they have a test
- people who live, work, or study, in one of the local authority areas that are problem areas

I know of two people who had tests because they drove other people to a testing centre and were offered one 'while they were there', even though they had no symptoms and no need for a test.

Anecdotal I know, but I wonder how much that is happening?

(They both tested negative, as it happens

 

[Bantamzen]

That article has one major flaw. With present policy, testing will find a much higher percentage of positives than the general prevalence, because people are only supposed to have a test if they think they are infected.

True, but with a growing list of symptoms associated with covid, more & more people who don't have it might well be having tests.

 

[Mojo]

I know of two people who had tests because they drove other people to a testing centre and were offered one 'while they were there', even though they had no symptoms and no need for a test.

Anecdotal I know, but I wonder how much that is happening?

(They both tested negative, as it happens

On a similar note when I went to get a test online it offered that I got one for people who live with me, even though I told the website they had no symptoms.

It seems anyone can get a test online without symptoms, even if they tell the website that they have no symptoms. I’d say there are such a small number of people who are actually ill the government has got to get as many people being tested as possible to keep their figures looking rosy.

 

[Bletchleyite]

Yep, it does look like you can pick "other" and "no" and it still offers a test.

 

[AdamWW]

Whilst these may be the minority, a number of people without symptoms may have a government test funded by the taxpayer:

- care home residents and staff
- essential workers
- those being admitted to hospital, where the hospital has requested they have a test
- people who live, work, or study, in one of the local authority areas that are problem areas

Also the ones in the random panel of people being used to calculate prevalance of coronavirus in the population, where this level of false positive is, I would have thought, a bit of a problem.

 

[Bletchleyite]

Also the ones in the random panel of people being used to calculate prevalance of coronavirus in the population, where this level of false positive is, I would have thought, a bit of a problem.

If you're doing statistical work like that you can calculate to take it into account.

 

[Yew]

If you're doing statistical work like that you can calculate to take it into account.

You can, but the question is if they are, and what effect that has on the accuraccy of the measurements.

 

[AdamWW]

If you're doing statistical work like that you can calculate to take it into account.

Yes, and I hope they have.

But even if you know exactly what the false positive rate is (and presumably we don't), it will add a lot of uncertainty I think as they are getting a very small number of positive results as it is.

 

[AdamWW]

You can, but the question is if they are, and what effect that has on the accuraccy of the measurements.

I'm afraid I can't find the SAGE document now (will add a link if I find it) but the answer is that yes they are, and they think it isn't at present having a significant effect even on the ONS random survey (which would be the most affected).

You can get a good idea from whether you keep seeing people in one household with a positive test when the other members test negative.

 

[Yew]

You can get a good idea from whether you keep seeing people in one household with a positive test when the other members test negative.

Thats certainly one form of failure case, but I don't think it's the only (or perhaps even no the mean) manfestation of false positives.

 

[AdamWW]

Thats certainly one form of failure case, but I don't think it's the only (or perhaps even no the mean) manfestation of false positives.

No but you can get some idea of the false positive rate from this.

 

[Yew]

No but you can get some idea of the false positive rate from this.

I disagree, I feel it will be more like background noise in random sampling, as opposed to a consistent false signal.

 

[talldave]

No but you can get some idea of the false positive rate from this.

My partner and her friends sent 6 tests back unopened. All were reported as positive. Not very confidence inspiring, almost conspiracy theory supporting!

 

[Bantamzen]

My partner and her friends sent 6 tests back unopened. All were reported as positive. Not very confidence inspiring, almost conspiracy theory supporting!

In normal times I'd suggest taking this to the press.

 

[talldave]

In normal times I'd suggest taking this to the press.

One of them flagged it up on Instagram - her account was shut down. You're not allowed to challenge Project Fear.

 

[Bantamzen]

One of them flagged it up on Instagram - her account was shut down. You're not allowed to challenge Project Fear.

That is seriously worrying.

 

[talldave]

That is seriously worrying.

Yes, hence my total scorn for anything the morons in charge ask us to do without justification "for the greater good".

I don't go for conspiracy theories and believed in the reasoning behind lockdown. But the lies, fiddling, misinformation and irrational requirements without any stated objectives leave me doubting the competence of those in charge and wondering what this is really all about. That said, the morons in charge are no cleverer than you or I, so I doubt their ability to have a master plan that we're unaware of!

 

[AdamWW]

I disagree, I feel it will be more like background noise in random sampling, as opposed to a consistent false signal.

The point is, I think, that if when you sample households you tend to get several positive tests per household, that is suggestive of the majority of positive tests being real, rather than false positives which (unless the false positive rate is stupidly high) should mostly be one per household.

Anyway my point (and it would be stronger if I could find the document it was in) was that false positives are being considered and thought not to be a significant problem at the moment.