Seth's Blog : Anecdotes are not science
Anecdotes are not science
Phrenology was discredited a long time ago. People who should have known better were sure that by studying the bumps on someone’s head, a trained expert could divine insights about their personality. It ended up being used to advance racist and class-based agendas, and was completely debunked. It faded away for decades. And it’s back.
New technology creates the appearance (and sometimes the actual fact) of new insights, new resolution, new certainty.
We might not know what an oscillation overthruster is, or why single photon imaging is better, but it sounds well studied and precise. A chart from Excel seems a lot more certain than one that’s hand drawn.
In our search for anecdotes, particularly about health, behavior or the economy, this apparent increase in accuracy opens the door for more hope, even if it’s not based on widespread results.
The charts used to describe the behavior of stocks and tokens keep getting more complex and refined, but they’re still unable to accurately predict what will happen next week.
The fancy readouts of horoscopes or biorhythms glow with many insignificant digits, but they still tell us nothing about someone’s future, any more than palm reading does.
And an x-ray can tell us with great certainty if your appendix has burst, but a SPECT scan is useless in determining someone’s personality without the aid of an in-person consultation, which is all we’ve ever needed. In fact, that’s precisely how phrenology used to work: meet with someone first, then find validation in the mysterious reading of their bumps.
The standard worth checking for is easy: From the chart or the bumps or the scan alone, without meeting the patient, tell me what you see and what’s going to happen next.
They put Einstein’s brain in a jar, but learned nothing from it.
The folks who ate green coffee beans or swallowed colloidal silver have plenty of anecdotes to support their placebos. And when they move on from pyramids to magnets, the anecdotes will follow them. But anecdotes aren’t science. Like coincidences, they’re by-products of our story-seeking minds, connections we make as we search for solace in a confusing world. And sometimes marketers use the anecdotes to make a sale and hurt the customer.
Very few interventions that involve humans are simple. We need more than a double-blind study, because humans aren’t double-blind. We know what’s on offer, and the story we tell ourselves changes how we behave.
Science is often not the right answer to every question–it often fails to deliver what we need. But hustles pretending to be science are almost always a bad idea.
In fact, stories are too important and worthwhile to need a babble of pseudoscience that some would like put on them.
Placebos are powerful, and if they’re cheap and benign, I’m all for them. My day is filled with placebos of all kinds, because they work. The problems happen when they stop being benign, when they keep us from appropriate treatment and when they’re used against us…
Somehow, we’ve persuaded ourselves that we need to pretend that our anecdotal interventions are actual scientific breakthroughs instead of embracing the fact that we’re humans, and that stories work on us. By wearing the mantle of science, hypesters are not only able to charge more, but they also degrade the reputation of the very methods they purport to use–when we see firsthand that pretend science doesn’t work, we’re tempted to imagine that the same is true for interventions that are actually studied and tested.
We wouldn’t fly on a plane or cross a bridge that was built with the same doublespeak that many folk medicines and soothsayers use. They have their place, they make us who we are, but anecdotes aren’t science.
…
You're getting this note because you subscribed to Seth Godin's blog. Don't want to get this email anymore? Click the link below to unsubscribe.
Email subscriptions powered by FeedBlitz, LLC • 1800 Camden Road, Suite 107-258 • Charlotte, NC 28203, USA
Post a Comment