Narre af statistisk signifikans

Don’t let poets lie to you

Behold the world’s shortest lecture on #Statistikker and everything that is wrong with how people approach it:

42.

Or, rather: p= 0.042

Contrary to popular belief, the term “statistisk signifikant” does not mean that something vigtigt, betydningsfuld eller overbevisende took place. If you think that we’re using the word signifikant here in a way that would make your thesaurus proud, you’re falling victim to a cunning bit of sleight of hand. Don’t let poets lie to you.

“You shouldn’t let poets lie to you.” — Björk

For those who prefer to keep their exposure to statistiske nitty gritty to a minimum, here’s all you need to know about the term statistisk signifikant:

• It doesn’t mean that anything significant has happened.
• It doesn’t mean the results are “big” or noteworthy.
• It doesn’t mean you will find the data interessant.
• It means that someone is claiming to be surprised by something.
• It doesn’t tell you anything useful if you don’t know much about the person og noget i spørgsmålet.

To everyone other than the decision-maker in question, statistically significant results are rarely signifikant in the sense of “vigtigt” — they’re occasionally great for raising interesting questions, but often they’re irrelevant.

Be on extra high alert when non-experts use this term, especially when it is accompanied by breathless exuberance. Sometimes especially cheeky charlataner go one step further and drop the “statistically” bit, tapping into the full power of the poetry. “Hey look,” they tell you, “what we’re talking about is SIGNIFICANT in the eyes of the universe.”

Nej, det er det ikke.

The worst possible offenders are those who pronounce “statistically significant” like it’s a synonym for “konkret"Eller"vis"Eller"flawless knowledge” — there’s some irony getting lost here. The term comes from a field that deals with usikkerhed and thus (by definition!) only belongs in settings where our knowledge is ikke fejlfri.

For those who prefer to fight jargon with jargon, I’ll help myself to more formal language in the next section. Feel free to nope out of that bit, but if you’re simultaneously curious , new around here, take a little detour to zoom through all the biggest ideas in statistics in just 8 minutes:

Most of the links in my articles take you to blog posts where I’ve given you a deeper overview of highlighted topics, so you can also use this article as a launchpad for a Choose Your Own Adventure minicourse on datalogi.

“Statistical significance” merely means that a p-værdi* was low enough to change a decision-maker’s mind. In other words, it’s a term we use to indicate that a nulhypotesen var afvist.** What var the null hypothesis, though? And how strict was the test? ¯_(ツ)_/¯

Welcome to statistics, where The Answer is p = 0.042 but you don’t know what the question was.

Teknisk set beslutningstager who set up the conditions of the hypothesis test is the kun person for whom that test’s results can be statistically significant.

Statistics gives you a set of tools for decision-making, but how you use them is up to you — it’ll be as individual as any other decision.

The process involves phrasing your decision question very carefully, picking the antagelser you’re willing to live with, making some risk tradeoffs about the different ways your answer might be wrong*** (because randomness is a jerk), and then using mathematics to get a risk-controlled answer to your particular question.

There’s something perverse and comical in its popularity as a prop for rhetorical bullying.

That’s why real experts would never use statistics like a hammer for beating Truth into one’s enemies. Two decision-makers can use the same tools on the same data and come to two different — and completely valid — conclusions… which means that there’s something both perverse and comical in its popularity as a prop for rhetorical bullying.

Statistical significance is personal. Just because I am surprised enough by the data to change my mind doesn’t mean you should be.

Så snart jeg forstod how statistics works, I couldn’t help but marvel at how remarkably arrogant — almost rude — it is to declare something to be statistically significant in the presence of people who aren’t fluent in the limitations of statistical decision-making. The term sounds much too universal for anyone’s good; it plays like a “shut up and trust me because my methods are fancy” rhetorical device. I hope you’ll join me in giving that brand of rhetoric the “pfft” it deserves.

Hang on, is there nothing at all we can learn from someone else’s statistically significant result?

Here’s where it gets somewhat philosophical, so I’ll need a separate article for my take on that question:

In a nutshell, my advice is that it’s fine to delegate some of your decision-making to other people as long as you trust them to be competent and have your best interests at heart. When they’re convinced, you’ll borrow their opinion so you don’t have to redo all their work yourself.

By using someone else’s statistical conclusions, you’re not basing your decision on data but rather on your trust in an individual human being.

Just be aware that by using someone else’s results, you’re not basing your decision on data but rather on your trust in an individual human being. There’s no problem with choosing to trust others so you don’t need to build your whole worldview empirically from scratch — knowledge sharing is part of what makes the human species so successful — but it’s worth being aware that you might be a few rounds of broken telephone downstream of whatever “knowledge” you think you’re tuning into.

If you let someone step up to make decisions on your behalf — that’s what it means to consume someone else’s p-værdi and conclusions for decision-making — then be sure it’s someone you consider sufficiently competent and trustworthy.

What if the person shoveling statistical jargon at you is someone you ikke trust? Run for the hills!

Whenever there’s a whiff of persuasion clinging to declarations of statistical significance, be extra cautious of whatever wares the utterer is peddling. If you trust the person you’re talking to, you don’t need their appeals to statistical significance. All you need to know is that they’re convinced. If you don’t trust them, you ikke kan stole på their stats jargon any more than you’d trust their jazz hands.

What good is an answer if you haven’t bothered to understand what the question was?

If there’s one thing I’d like you to take away from this blog post, it’s this: If you don’t know much about the decision-maker and how they set about figuring out whether they should skift sind (and precisely about what), then their claims related to statistical significance are utterly meaningless to you. What good is an answer if you haven’t bothered to understand what the question was?

If you had fun here and you’re looking for an applied AI course designed to be fun for beginners and experts alike, here’s one I made for your amusement:

Let’s be friends! You can find me on Twitter, YouTube, understakog LinkedIn. Interested in having me speak at your event? Use denne formular At komme i kontakt.

Here are some of my favorite 10 minute walkthroughs:

*If you’re keen to learn what a p-value is, here’s a video I made to help you out:

**For an explanation of hypothesis testing, head over to my blogindlæg on the topic or check out this pair of videos:

Fooled by statistical significance Republished from Source https://towardsdatascience.com/fooled-by-statistical-significance-7fed1bc2caf9?source=rss—-7f60cf5620c9—4 via https://towardsdatascience.com/feed

<!–

->