Should We Be Universally Screening to Prevent Death by Bolt of Lightning?
Taking a “crack” at public health understanding, together with a dose of humor.
I’m so excited. In this article, I'm going to teach you how to understand what a screening test is, what it’s good for, and how to determine if it's worth it for you, your company, or your family!
Usually I’m just mumbling about “screening” to myself as one season bleeds into another and the toll of time erodes what’s left of my youth. There have been some exciting updates in the world of screening.
Colon cancer. Prostate cancer. Anxiety. Depression. Suicidal children. Oh my God. Protect us from all the things!!! Sweet lord, tell me if I’m gonna die early or not already?!?!??!
Screening. It offers the promise of saving our life. It’s like careful, the word, but in billable health care service format.
We even have a source of news for screening.
Say it with me now. Our favorite source of information about screening? The United States Protective Services Task Force (USPSTF). They are the Justice League of preventative medicine. The USPSTF— that’s a great acronym. Okay, it’s a terrible acronym.
What is screening?
A screening test is done to detect potential health disorders or diseases in people who do not have any symptoms of disease.
It’s testing to detect a problem before one shows up at your door. Before you get sick.
Screening is the fishing expedition of testing in healthcare. You don’t go fishing because you already have a fish on the hook. You go fishing because you might catch one.
Getting tested for something because you think you might have the thing you’re getting tested for, or because you have symptoms, that’s different. That’s testing to confirm, rule in, or rule out a diagnosis in someone with some (and this is a science term coming up here) pre-test probability of having the illness. You’re worried because there is a reason to be worried—a symptom! Screening is testing done, by definition, when you have no reason to be worried. “Just in case.”
Screening tests can have one of 4 outcomes:
True positive: It's like finding a needle in a haystack and actually recognizing it as a needle.
True negative: It's like correctly identifying everybody who doesn't have a dangerous highly infectious respiratory infection prior to your birthday party.
False positive: It's like getting a message on LinkedIn from anybody at all who says they're interested in your business, but they are a bot trying to sell you some SaaS product, not because you are special.
False negative: It's like using a random number generator instead of a diagnostic test. You'll get a lot of reassurance, but it won't do a lot of good to protect you from the actual bad thing. I am looking at you, owlet dream sock.
Screening recommendations, therefore, only apply to people we don’t have reason to suspect are sick. If we were selectively choosing people we thought had a problem, and then testing them, it’s not screening anymore. It’s confirmatory testing. There’s some assessment that comes before the test. And that makes it not screening, which is a broad recommendation to everybody (in a certain population) that they should go get this test come hell or high water because it’s going to pick up something, maybe, that could save their life.
And again, by definition, recommended screenings are applied to everybody in a certain population. “Everybody over 50”, “all women under 25”, and “all children above 10”: that’s a population. Those are the people who get screened.
Screening is also ripe for healthcare profiteering. Screening exposes people to risk. It doesn’t just reduce risk—it creates it. Isn’t it good to ask a question about whether someone might have a problem, and then get a test for it?
And the reason we have the USPSTF is that the answer to that question is sometimes “no”. And the answer to that question is also sometimes “it’s not worth it”.
The best screening test is paired with an illness that can be improved:
Detects problems early AND
Early treatment is beneficial
And the test has low false positives
And the early intervention or confirmation isn’t dangerous
The risks outweigh the benefits of even asking the question in the first place!
If you got it, as of now, you can stop right here. If you want to understand Bayesian statistics in more detail with a very dumb example, read on:
In order to make this clear, I am going to use an example of a terrible screening test. It sounds like a good idea, but it's catastrophically expensive and stupid because the underlying problem is extremely rare, and even a very accurate test doesn't change outcomes meaningfully.
Let’s consider screening for getting hit by lightning. The test for getting hit by lightning would be a predictive one. Let’s say we had a test (which, by the way, we don’t) and it had some percent chance of predicting that you would get hit by lightning…ever. And also, like every other test, it’s not perfect.
The adventure will benefit from some definitions:
Sensitivity and specificity are my least favorite terms from medical school, cause I couldn’t tell them apart.
Sensitivity is like having a lightning detector that goes off every time there's even a hint of lightning in the area. It's sensitive, but it might also be prone to false alarms.
Specificity, on the other hand, is like having a lightning detector that only goes off when there's actually lightning striking nearby. It's specific, but it might miss some lightning strikes.
Our Statistics MasterClass continues with Positive Predictive Value (PPV) and Negative Predictive Value (NPV):
PPV is like a weatherman predicting that there will be lightning tomorrow, and is great at his job—it’s going to storm so reliably that you will justly run for cover. A high NPV is like a weatherman predicting that there won't be any lightning tomorrow. You can safely plan your day when NPV Weather says it’s a good day for a picnic.
My NoZapAssure test has a 99% sensitivity and a 99% specificity. Let’s imagine it is a screening test for the whole US population. For reference, the Rapid Covid-19 tests:
The overall sensitivity of the rapid antigen test was 65.3% (95% confidence interval [CI] 56.8-73.1), the specificity was 99.9% (95% CI 99.5-100.0)
Let’s assume it’s cheap to administer— $20.
And then let’s do the thought experiment to see what the adverse effects look like if you don’t just decide to ignore it as total bulls&*$:
True positives: 330
True negatives: 329,999,670
False positives: 3.3 million
False negatives: 0ish
The cost of the test: If the cost of the screening test is $20, then the estimated cost of screening 330 million people would be $6.6 billion.
Now, Imagine you got a positive result. And let’s assume you don’t really care or understand the tiny absolute risk increase incurred…what would it cost to act on that information?1 Short version, too much.
Screening: it’s complicated. And expensive to get wrong…
A special thank you to
for his screening education!It is worth drawing attention to: Quizzify, helps humans and their employers navigate healthcare via education. The “Prevent Consent” language for every ER visit is a particular favorite: (you write this in pen on your ER financial consent forms):
"Superseding other consents, I consent to responsibility (including insurance) for up to 2 times Medicare following receipt of an itemized bill for appropriate treatment coded at the correct Level."
—Owen Scott Muir, M.D.
Maybe you wanted to move to a place with less risk or lightning? Not cheap:
The median US home price is about $467,000. So as long as you don’t have to move to somewhere other than Portland you are fine?
San Francisco, California - San Francisco has a very low incidence of lightning strikes due to its location on the coast and mild climate. Median home: $985,929
Seattle, Washington - Seattle is another coastal city…less lightning. Median home: $794,100
Portland, Oregon - Portland has a moderate climate low lighting risk and a median home: $460,000
Vancouver, British Columbia - It’s Expensive, but in Canadian dollars…with the median home price in the city being around CAD 1.1 million
And maybe add a lighting-safe room to spend all your time in? And a Special Car? Or upgrade the whole house?
Lightning-safe room: A small, basic lightning-safe room can cost around $2,000-$10,000 to construct, depending on the size, materials, and features. This cost may include materials such as lumber, drywall, insulation, and grounding rods.
A metal vehicle with a hard top: As an example, a new mid-size car with a metal roof and frame can cost around $20,000-$30,000, while a used vehicle with similar specifications could cost significantly less. No rag tops. Bummer. You only buy cars based on misleading screening once.
And how about just a giant cage for the whole house?
A small, basic metal building (e.g., 20' x 20') can cost around $10,000-$20,000 to construct.
A larger metal building (e.g., 40' x 60') with additional features such as insulation, windows, and doors can cost around $30,000-$50,000.
A large, complex metal building (e.g., 100' x 100') with multiple levels, custom features, and high-end materials can cost hundreds of thousands or even millions of dollars.
Regardless, let’s say you blow 100,000 K on moving and protection expenses, that is an additional $330,000,000,000 spent on the 3.3 million people who will be worried. And only 330 of them might have been actually struck.
The sad thing is the above math is actually cheaper and safer than many medical interventions can be, and if you don’t believe me, ask anyone who got screened with a PSA test for prostate cancer and had DaVinci Surgical prostate surgery that left them impotent at great cost and no benefit.
I have questioned screening for years - everyone should read this article.