Saturday, April 25, 2020

About Those Studies

Peer review on those Kung Flu studies is in...


You may recall we've had a bit to say, and none of it good, about the surveys that allege that Kung Flu is vastly more widespread than was supposed.
We pointed out no small number of major flaws in all those studies.
Lest you think I'm alone in this, let me share a few links and pull quotes, courtesy of MD Ed Grouch:
"(The Guardian/UK) Both studies [at Stanford and USC/LA County] used an antibody test made by Premier Biotech company that has not been approved by the FDA and comes with an acknowledgment that it can record false positives. 
Hundreds of antibody tests have emerged on the world market in recent weeks, including some that promise a result from a finger prick in just hours, an executive from the diagnostics and pharmaceutical company Roche told Reuters on Tuesday. None of them currently have FDA approval and some of them are “a disaster”, the Roche CEO, Severin Schwan, said. 
Then there are concerns about the Stanford study’s sample and statistical analysis. The biggest criticism was that it estimated cases for the whole county’s population based on detecting only 50 positives out of 3,300 people sampled. And since the tests had a false positive rate in one assessment of two out of 371, critics argued all the Covid-19 cases detected by the tests in Santa Clara could conceivably have been false positives.

“I think the authors of the above-linked paper owe us all an apology,” wrote Andrew Gelman, director of the applied statistics center at Columbia University, who has written numerous books on teaching statistical methods. “We wasted time and effort discussing this paper whose main selling point was some numbers that were essentially the product of a statistical error.”  
The prominent Washington state genetics researcher Trevor Bedford said on Twitter he was glad to see antibody studies emerging but was “skeptical” of the high results. The author and biotech investor Peter Kolchinsky tweeted that the “flaws with this study could trick you into thinking that getting shot in the head has a low chance of killing you”.  
The study was also criticized for recruiting its volunteers on Facebook, a method some critics charged could have induced some to participate in the study because they had had symptoms but were unable to get tested. Researchers say they attempted to screen for this by collecting information from participants on any recent symptoms, such as coughing or fever. 
Both the Stanford University team and the researchers at USC declined to respond to a request for comment.
We're just getting started.
From a peer review of the Stanford study posted on Medium
  1. First, the false positive rate may be high enough to generate many of the reported 50 positives out of 3330 samples. Or put another way, we don’t have high confidence in a very low false positive rate, as the 95% confidence interval for the false positive rate is roughly [0%, >1.2%] and the reported positive rate is ~1.5%.
From Columbia University's Statistical Modeling page, quotes from an e-mail sent to Andrew Gelman, Director of Columbia's Applied Statistics Center, and posted on their site:
It’s perfectly plausible that the shocking prevalence rates published in the study are mostly, or even entirely, due to false positives. 
Recruitment was done via facebook ads with basic demographic targeting. Since we’re looking for a feature that affects something like 2% of the population (or much, much less), we really have to worry about self selection. They may have discussed this in the portions of the paper I didn’t read, but I can’t imagine how researchers would defeat the desire to get a test if you had reason to believe that you, or someone near you, had the virus (and wouldn’t some people hide those reasons to avoid being disqualified from getting the test?).
So all 50 positives could be statistical error, and they had people self-selecting with an ulterior motive, either of which, by itself, could have represented the entire positive sample size. That's about as crapola as you can get.

But wait! There's more, this time about the tests being used in those studies:
(NBC News) But some COVID-19 antibody tests, including those being used by public health departments in Denver and Los Angeles and provided to urgent care centers in Maryland and North Carolina, were supplied by Chinese manufacturers that are not approved by China's Center for Medical Device Evaluation, a unit of the National Medical Product Administration, or NMPA, the country's equivalent of the U.S. Food and Drug Administration, NBC News has found.
Two U.S. companies — Premier Biotech of Minneapolis and Aytu Bioscience of Colorado — have been distributing the tests from unapproved Chinese manufacturers, according to health officials, FDA filings and a spokesman for one of the Chinese manufacturers. Many of the unapproved tests appear to have been shipped to the U.S. after the FDA relaxed its guidelines for tests in mid-March and before the Chinese government banned their export just over two weeks later. 
If COVID-19 antibody tests are unreliable, they can produce false results, either negative or positive, health officials said.

Sketchy tests, with accuracy so dubious it even worries the Chinese, with false results both positively and negatively, being used to do surveys on which public health decisions might be made. What could possibly go wrong?

I repeat, all the US surveys to date have HUUUUUGE problems with sample size, selection bias, accuracy of the tests, and are unpeer-reviewed hokum and horsie pooh that should be laughed out of any serious consideration.

You're being bullshitted by people who should know better, doing shoddy work, using subpar tests, and passing it off as research.


FredLewers said...

Garbage in = garbage out

Knightsofnee said...

Aesop, I think you would like this book.

ThatWouldBeTelling said...

used an antibody test made by Premier Biotech company that has not been approved by the FDA

Not required unless it's marketed for sole use in clinical decisions.

and comes with an acknowledgment that it can record false positives

From what I've heard, the floor for false positives in antibody tests is 4% due to the inherent messiness of the immune system. I think the Columbia prof fisking looked harder at how much we can in theory trust this test, not that it matters if you screw up the rest of a study.

We already went over the Stanford Santa Clara study, it says nothing real, which was no surprise given that one of the primary co-authors (last few in the list of authors), John Ioannidis being very invested in his "it's just the flu, bro" conceit, something you were the first to ping on in the sources I follow.

But wait, it gets worse, the wife of one of the other primary co-authors screwed up the selection more than we knew, "[She] solicited parents via email in Los Altos to join her husband's coronavirus antibody study, falsely claiming that the tests were "FDA approved" and would tell participants whether they were immune, potentially comprising the study's findings..." Strange, some cultures just don't care much whether the "science" they do is done correctly.

Haven't heard in detail about the LA one, but the NYC one has obvious selection issues, seeing as how it was of people out and about shopping.

Night driver said...

I'm unable to decide what the DESIRED result was for either of those "studies" (a term used under advisement).

This is especially true in that we are VERY RAPIDLY approaching Fauci's minimum number of US deaths. We should surpass his 60K WELL before Wednesday this coming week. We're running between 2 and 3 K per 24 hour period, per the Johns Hopkins ARC-GIS Dashboard.

Night Driver

ASM826 said...

"You're being bullshitted by people who should know better, doing shoddy work, using subpar tests, and passing it off as research."

On this we are in total agreement.

Knightsofnee said...

Goal seeking the numbers. Shot ton of people are going to die in the next 6 month's. FUBAR + SNAFU * Normalcy Bias/2 = It's just flu bro.

Women and children only.

LSP said...

Goat Rodeo.

Robin Datta said...

The highfalutin' academic emmeffs have to maintain a certain minimum of decorum in normal times: their audience is other academics just as eager to cut their throats as they are of their academic audience.

When panic is rife in the populace, it constitutes a sudden expansion in the audience who, unlike academic mutual cutthroats, are available to be massaged and titillated. Scientific method and peer review be damned. So we're gonna have many a freshet of male bovine fecal material. However this particular variety grows not roses, but exclusively thorns. Spread it at one's own hazard!

gwalchmai munn said...

Good post.

I'd be interested in your comments about this, Oh Teller of Tales -

Charlie said...

Any thoughts on this?

ThatWouldBeTelling said...

gwalchmai munn: In your link, in the first "Lies" section, the author says "We know it mutates and evolves just like the flu."

This is so false it's hard to know where to begin. Coronaviruses have a proofreading mechanism unique among all RNA viruses. Influenza's RNA is split into segments, and these mix and match very easily in infected animals, especially pigs and people; this is also used to make seed strains for egg vaccine production. Plus the flu proteins that are obvious to the immune system, the parts of the proteins they latch onto called epitopes, are not "conserved", they can change quite a bit while still allowing "the virus to virus."

Not worth reading more after such a clanger.

Night driver said...

First off, anyone discussing studies and such should have read at LEAST one of the earlier (like first or second edition) At the VERY LEAST of this
My bride brought home the first or second edition a couple or 3 decades ago and it's been pretty close to the top of the circulating stacks here at Chaos Manor MidWest ever since. Without that they are HIGHLY unlikely to see what you are so happy about in these "studies" (used under advisement).

In ref CVTruths via godaddysites is bound to raise yer BP by more than a sublingual spray or 5 of nitro can help.

As one of my more fav posters in my normal I-web stomping grounds put it:

Seriously , the guys an idiot. He claims it's not very infectious. But everyone has had it already. Even though it only appeared a couple months ago . He claimed that everyone who went to the ER has been tested in March. Something I know for a FACT isn't true. He's just repeating half baked conspiracy theories.

But if you like hypocritical liars who can't keep Their fact straight. He's a great read.

Can't find credentials and it's got TONNES of issues.

Night Driver

Aesop said...

@gwalchmai munn and Charlie,

He's full of shit.
By the fourth paragraph, it's pretty obvious he doesn't know WTF he's talking about.
His side:
"LIE: COVID-19 is more infectious than influenza.COVID-19 was first identified late December/early January. A little over three and a half months in, we have slightly over 2 million confirmed cases globally. By comparison, according to the National Center for Biotechnology Information (Google Influenza Update- NCBI), influenza is responsible for up to a billion infections annually. Flu season is basically from the start of October until April. At the flu’s three and a half month mark, that works out to 583 million cases globally. Even accounting for the fact COVID-19 is underreported that is a HUGE difference."
Um. just no. The number of <><> SARS-C0v-19 cases is a function of testing, not the actual number of people infected.
1) China flat-out lied and continues to lie about the true number of infected there. With a population three times larger than the US, it's flatly recockulous to think that the US and half a dozen other countries have more cases in 2-3 months than China did in 5-6.
2) japan, where it's blowing up now, refuses 90% of requests from doctors to test people suspected there of having Kung Flu. No tests, no confirmed cases, Easy peasey.
3) The US had no tests for months, while it exploded here. We still undertest.
In fact, the only reason we knew it was here was because a doctor in Seattle, running the Seattle Flu Project studying annual flu tested people for COVID-19 against the express orders of the CDC and discovered clusters, including the convo home where thirty people died, and half the 150+ staff members were symptomatic for the virus, and let go. One cluster she found included six people who all got Kung Flu from a dinner party. Not a swinger's party. Not a mosh pit. Just a bunch of people sitting together at dinner for a couple of hours, and six of them got it in one night.
And closing schools for the annual flu? WTAF?? I've never heard of that nor seen it in my entire life, and the flu has been an annual thing since before living memory. Must be a Bumpkin County phenomenon. It never happens in Califrutopia, and I've never even heard of anyone doing it. BTW, all the schools everywhere are closed for Kung Flu. But we don't even need to know that to prove Doc Bumpkin is F.O.S.

Aesop said...

The unfortunate reality for his line of utter bullshit: The r-naught is how you calculate what is more contagious.
It is an actual expression of how many people any Patient Zero can be expected to transmit a given disease onward to.
An R-naught of 1 means you'll infect 1 other person.
An R-naught of 10 means you'll infect 10 other persons.
So it isn't hard to see what's what.
And it's not a particularly obscure concept, either in epidemiology, general medicine, or disease science.

The r-naught for seasonal flu is about 1.3.

The r-naught for SARS-CoV-19 is now assessed to be 5.7.
Earlier estimates put it at 2.2-2.6, but they've been revised upwards.

See if you, without any medical training whatsoever, can assess which one is therefore more contagious.
I'll wait right here.

So maybe Dr. Bumpkin was sick the day they covered R-naughts in medical school?

If so, he probably shouldn't be posting pages on the internet where he pulls his pants down in front of God and everyone on the 'net, and spanks himself in public.
It's embarrassing (literally), unprofessional, and unseemly.
Yet there he stands, pants around his ankles, and @$$cheeks redder that a sunburned pig.
After that inauspicious beginning, I'm not up to fisking the entire article, and frankly, when I can spot such an obvious whopper in 30 seconds, I've already wasted too much time on what is likely a total pile of horseshit, without even a pony at the bottom.

If a doctor is talking about how contagious something is, and hasn't heard of r-naughts, he's a moron not worth further discussion, and I'd only dredge through and fisk the rest of that load of codswallop kicking and screaming.

Marina said...

Aesop, thoughts ? California ER Doctors Dan Erickson and Artin Massihi on 12 min video all for reopening. "We're used to the flu, we'll get used to the C-19 too":

ThatWouldBeTelling said...

3) The US had no tests for months, while it exploded here.

To put hard numbers and dates on this, from January 11th to February 28th, the FDA prevented every test except the CDC's flawed one from being used, and only 4,000 people were tested. At best, Trump's half-hearted travel ban's advantage was totally consumed by this act of what's looking to me like criminal malice.

And closing schools for the annual flu? WTAF?? I've never heard of that nor seen it in my entire life, and the flu has been an annual thing since before living memory. Must be a Bumpkin County phenomenon. It never happens in Califrutopia....

OK, I'm from Bumpkin County flyover territory. Some schools in my area did close down for short periods of time due to the especially nasty at least for us 2019-20 flu season. The predominant in the early part B strain, which is more prone to harm the young, seems to have been a part of it. Killed an early 20s school worker....

Marina said...

Oops re Calif ER: Make sure to listen to the end as they talk of lack of ammo, loss of freedom ... from Californians !

Ray - SoCal said...

R of 5.7? Wow.

Outside of closed environments, nursing homes, prisons, etc. where are you seeing the infection spread coming from in CA, if we are in lockdown?

Aesop - Thanks for opening up the comments again. Your quick fisking of the article was very useful, lots of good information in that.

ADS said...

We have a real problem with junk science and garbage papers being queefed out by biased researchers with a conclusion already in mind, unquestioningly reported by an unfree press, and accepted as gospel by adherents of Science The One True Faith. This first came to light for me reading junk studies about gun violence.
Lying governments, junk science, deep state media, clueless public... if we stumble upon good data it will be accidental.

Bee Ess said...

Lol. That deconstruction of ass-hattery is half the reason i came back to this blog after my first visit. Hey! At least this time it's not aimed at a wayward commentor, thus no butthurt salve necessery.

OvergrownHobbit said...

All the various experts have excellent reasons to lie to us, now.

Why single out the Stanford Study jokers? These are the scientists who brought us the Climate Stasis cult, and the reproducibility crisis.

Two side questions: Do your nursing staff have enough free time to choreograph tik-toc dances? Where is this coming from?

And: What if the news that nursing staff in places like New York are deliberately killing the Chi-com Herpes patients who end up in their IF wards?

Ray - SoCal said...


I don’t think it’s deliberate, but just don’t care / over whelmed / bad management/ bad culture. Probably similar to what happened at Killer King in LA.

Video I thin you are referring to by a nurse practitioner:

Crew said...

- Morbidity/mortality has decreased the past 24 years, most visible in the 80+ age group (13% yearly mortality in 1995 vs 10.8% in 2019)
- 2019 was a mild year across age groups
- Winter of 2017/2018 had a nasty flu season around week 10
- Winter of 1999/2000 had a nasty flu season around week 1, comparable to the peak in 2020
- Deaths in 0-65 age group was 3% less than compared to past 3 year average, and 14% less - than the 24 year average
- Deaths in 65-80 age group was 10% higher than past 3-year average, but still 14% less than 24-year average
- Deaths in 80+ age group was 6% higher than past 3-year average and 2% higher than the 24-year average
- Peak in 2020 is outside of the regular flu season

From Norway said...

At crew: vox is comparing apples to oranges. He is comparing a free range flu with an everyone home corona. If we were not all home maybe it would make sense comparing those numbers.

ThatWouldBeTelling said...

Ray - SoCal and Unknown:

This is consistent with for instance the 90% fatality rate of people on ventilators in NYC, which is as far as we know worse than the already very bad US standard. And I previously linked to this article about the terrible level of nursing in NYC.

Ray - SoCal said...

Thanks ThatWouldBeTelling!

I forgot about that article. The article makes a lot more sense on re-reading it.

And the hospital mentioned, Elmhurst, is in Queens, and has 1/3rd of the NYC Coronavirus cases.

Yelp Reviews, 1 1/2 Stars:

And Hospital is run by the city.

Elmhurst was made into a 100% Coronavirus Facility:

Lovely - Head of the Hospital March 31, was saying Masks don't do much:


Zeroh Tollrants said...

I'm not in disagreement with anything stated, I would keep in mind that while the numbers are ludicrous and I'm sure a majority of people tested already had symptoms they felt might be the Wu Flu, never forget perfectly asymptomatic Karens who have insisted to see your manager demanding to know why they haven't been allowed testing on demand, despite being asymptomatic.
You guys always forget to factor in dumb things women do because you wouldn't do those things yourself.