Megtévesztette a statisztikai szignifikancia

Don’t let poets lie to you

Behold the world’s shortest lecture on #statistics and everything that is wrong with how people approach it:

42.

Or, rather: p= 0.042

Contrary to popular belief, the term “statisztikailag szignifikáns” does not mean that something fontos, fontosvagy meggyőző took place. If you think that we’re using the word jelentős here in a way that would make your thesaurus proud, you’re falling victim to a cunning bit of sleight of hand. Don’t let poets lie to you.

“You shouldn’t let poets lie to you.” — Björk

For those who prefer to keep their exposure to statisztikai nitty gritty to a minimum, here’s all you need to know about the term statisztikailag szignifikáns:

• It doesn’t mean that anything significant has happened.
• It doesn’t mean the results are “big” or noteworthy.
• It doesn’t mean you will find the dátum érdekes.
• It means that someone is claiming to be surprised by something.
• It doesn’t tell you anything useful if you don’t know much about the valaki és a valami kérdéses.

To everyone other than the decision-maker in question, statistically significant results are rarely jelentős in the sense of “fontos” — they’re occasionally great for raising interesting questions, but often they’re irrelevant.

Be on extra high alert when non-experts use this term, especially when it is accompanied by breathless exuberance. Sometimes especially cheeky charlatans go one step further and drop the “statistically” bit, tapping into the full power of the poetry. “Hey look,” they tell you, “what we’re talking about is SIGNIFICANT in the eyes of the universe.”

Nem, nem az.

The worst possible offenders are those who pronounce “statistically significant” like it’s a synonym for “határozott"Vagy"bizonyos"Vagy"flawless knowledge” — there’s some irony getting lost here. The term comes from a field that deals with bizonytalanság and thus (by definition!) only belongs in settings where our knowledge is nem flawless.

For those who prefer to fight jargon with jargon, I’ll help myself to more formal language in the next section. Feel free to nope out of that bit, but if you’re simultaneously curious és a new around here, take a little detour to zoom through all the biggest ideas in statistics in just 8 minutes:

Most of the links in my articles take you to blog posts where I’ve given you a deeper overview of highlighted topics, so you can also use this article as a launchpad for a Choose Your Own Adventure minicourse on adat-tudomány.

“Statistical significance” merely means that a p-érték* was low enough to change a decision-maker’s mind. In other words, it’s a term we use to indicate that a null hipotézist volt elutasított.** What volt the null hypothesis, though? And how strict was the test? ¯_(ツ)_/¯

Welcome to statistics, where The Answer is p = 0.042 but you don’t know what the question was.

Technikailag a döntéshozó who set up the conditions of the hypothesis test is the csak person for whom that test’s results can be statistically significant.

Statistics gives you a set of tools for decision-making, but how you use them is up to you — it’ll be as individual as any other decision.

The process involves phrasing your decision question very carefully, picking the feltételezések you’re willing to live with, making some risk tradeoffs about the different ways your answer might be wrong*** (because randomness is a jerk), and then using mathematics to get a risk-controlled answer to your particular question.

There’s something perverse and comical in its popularity as a prop for rhetorical bullying.

That’s why real experts would never use statistics like a hammer for beating Truth into one’s enemies. Two decision-makers can use the same tools on the same data and come to two different — and completely valid — conclusions… which means that there’s something both perverse and comical in its popularity as a prop for rhetorical bullying.

Statistical significance is personal. Just because I am surprised enough by the data to change my mind doesn’t mean you should be.

Amint megértettem how statistics works, I couldn’t help but marvel at how remarkably arrogant — almost rude — it is to declare something to be statistically significant in the presence of people who aren’t fluent in the limitations of statistical decision-making. The term sounds much too universal for anyone’s good; it plays like a “shut up and trust me because my methods are fancy” rhetorical device. I hope you’ll join me in giving that brand of rhetoric the “pfft” it deserves.

Hang on, is there nothing at all we can learn from someone else’s statistically significant result?

Here’s where it gets somewhat philosophical, so I’ll need a separate article for my take on that question:

In a nutshell, my advice is that it’s fine to delegate some of your decision-making to other people as long as you trust them to be competent and have your best interests at heart. When they’re convinced, you’ll borrow their opinion so you don’t have to redo all their work yourself.

By using someone else’s statistical conclusions, you’re not basing your decision on data but rather on your trust in an individual human being.

Just be aware that by using someone else’s results, you’re not basing your decision on data but rather on your trust in an individual human being. There’s no problem with choosing to trust others so you don’t need to build your whole worldview empirically from scratch — knowledge sharing is part of what makes the human species so successful — but it’s worth being aware that you might be a few rounds of broken telephone downstream of whatever “knowledge” you think you’re tuning into.

If you let someone step up to make decisions on your behalf — that’s what it means to consume someone else’s p-érték and conclusions for decision-making — then be sure it’s someone you consider sufficiently competent and trustworthy.

What if the person shoveling statistical jargon at you is someone you nem trust? Run for the hills!

Whenever there’s a whiff of persuasion clinging to declarations of statistical significance, be extra cautious of whatever wares the utterer is peddling. If you trust the person you’re talking to, you don’t need their appeals to statistical significance. All you need to know is that they’re convinced. If you don’t trust them, you can’t trust their stats jargon any more than you’d trust their jazz hands.

What good is an answer if you haven’t bothered to understand what the question was?

If there’s one thing I’d like you to take away from this blog post, it’s this: If you don’t know much about the decision-maker and how they set about figuring out whether they should meggondolják magukat (and precisely about what), then their claims related to statistical significance are utterly meaningless to you. What good is an answer if you haven’t bothered to understand what the question was?

If you had fun here and you’re looking for an applied AI course designed to be fun for beginners and experts alike, here’s one I made for your amusement:

Let’s be friends! You can find me on Twitter, Youtube, Alsó raklapés LinkedIn. Interested in having me speak at your event? Use ebben a formában kapcsolatba lép.

Here are some of my favorite 10 minute walkthroughs:

*If you’re keen to learn what a p-value is, here’s a video I made to help you out:

**For an explanation of hypothesis testing, head over to my blogbejegyzés on the topic or check out this pair of videos:

Fooled by statistical significance Republished from Source https://towardsdatascience.com/fooled-by-statistical-significance-7fed1bc2caf9?source=rss—-7f60cf5620c9—4 via https://towardsdatascience.com/feed

<!–

->