Post: #1

We are bombarded by numbers and stats at all times and new ones get thrown at us daily. "1 in 5 adults in the US suffer from 'major depression' at some points in their lives". "1 out of every 4 boys is a bully". "Only 2% of all rape accusations are false". "Countries other than the US deliver better healthcare at half the cost."

It can be hard even for very intelligent men to make sense of this deluge of statistical claims, and to separate the wheat from the chaff. As a result there is a temptation to give up trying, and resort to the extremes of either blind trust ("this study said so, therefore it must be true"), or blind cynicism ("all these studies are garbage, there is nothing to learn from them"). Neither of these extremes is warranted. We can do better. And as "red pill" men who try to think for themselves and make their own way in the world instead of accepting the conventional wisdom on any given subject, we need to do better.

This thread is meant to be the RVF guide and resource for making sense of the numbers we encounter in real life, and developing a few rough-and-ready techniques that will help guys in evaluating statistical and numerical claims about subjects that might be of interest to them. It is meant to be a resource for everyone -- not just for the analytically and quantitatively inclined. I'm not going to burden anyone with advanced math or statistics. My goal is to show over a number of posts that a little straight thinking and a little common sense will take you a very long way towards getting a much better handle on these matters than you might have thought possible. And I hope that every member or lurker who follows this thread will leave it better equipped to deal with these subjects in his day-to-day life.

The way I'm going to structure this thread will follow the template set by Dusty's fantastic and highly recommended thread about Albert Ellis and REBT. I'm going to take an excellent book called "The Numbers Game" (available from Amazon on Kindle, hardcover and paperback), and review it chapter by chapter in subsequent posts. It is a brief book consisting of 12 short chapters in addition to an Introduction, and it's the best everyday stats and numbers guide for laymen that I know of. Of course I encourage guys who are interested to buy the book, but those who don't feel like it should be able to get most of the value by following the thread over time.

After I'm done with the book review part of the thread, which will probably take some weeks, I'm going to open it to an ongoing discussion of real life examples of numerical and statistical claims that guys encounter and would like to evaluate and make some sense of. I'm going to encourage posters to post examples of particular studies or claims they are interested in, and I will do my best to help them analyze these claims and try to come to some conclusions, or at least to a better understanding. I may not be able to get back to every question right away, but I will try to answer all of them over time. Of course others are always welcome to come in and help with the evaluation process, using the techniques outlined in the book review phase.

In case someone wonders about my qualifications for running such a thread -- I hold a terminal degree in a highly quantitative field from one of the handful of the world's best university departments in that field. But the more important message is that while knowing the math can help here and there, you don't need to be a rocket scientist to get a good rough and ready understanding of this stuff. Learning a few basic tricks and approaches, and using a lot of common sense, will take you a very long way.

Stay tuned for the next post, in which I will describe the book I will be reviewing in somewhat more detail, and review its Introduction.

Post: #2

I'll just throw a really simple concept in here that seemingly very few people at large know (I didn't know about it until my late 20s/early 30s): The Rule of 72. It's amazing how useful this concept (that can be calculated in your head) is for calling BS on all sorts of financial or economic claims.

Post: #3

"The Numbers Game": Introduction

The book I'll be reviewing, "The Numbers Game", was written by two Brits: Michael Blastland, a journalist, and Andrew Dilnot, an economist (note that neither of them is a professional mathematician or statistician). It was born of an extremely successful BBC radio program called "More or Less" which the two of them co-anchored between 2001-2007 (the program still exists but is run by someone else now). In this program they examined numerical and statistical claims that came up in the news or were made by politicians or other public figures. They showed how to think through these claims with clarity, and in the process taught the listeners techniques that they could then use themselves. The book is a distillation of these techniques, and uses some of the examples that were examined on the radio program.

In the brief Introduction to the book, the authors make one key point that will be worth remembering in later posts as we get into the meat of the book:

RELAX. You already know just about everything you need to know.

This is the most important and unusual piece of instruction given in the book:

Quote:We know more than we think we do. We have been beautifully conditioned to make sense of numbers, believe it or not, by our own experience. This is the radical premise of the book -- that readers have no need to throw up their hands in fear and contempt, if only they see how much they know already.

In other words, people have so much trouble understanding and evaluating numerical claims mostly because they are mesmerized by these claims and forget how much they actually know about the world. If, instead, they relax and actively use the knowledge they already have, making sense of the numbers gets a lot easier.

True to this approach, the authors try to bring every example they look at in the book "back to earth", as they say. If there is a single thread running throughout the book, it is how you can use things you already know well to make sense of the new stats or numerical claims that you encounter.

The authors make a few other points in the Introduction:

1. Even though we seem to be flooded with often contradictory statistical claims at all times, just turning out back on all of them is not a good option. There is plenty of useful and important information that can be extracted from all the noise, and your life will be better if you learn how to extract it.

2. Absolute certainty is not an appropriate standard to apply to statistical claims or their evaluation. No matter what, some of the claims will be wrong and so will some of your conclusions. The goal is to do as well as possible and get some useful information -- not to achieve certainty or perfection. More than that, one of the most important things to understand about any given claim is the degree of certainty to which it can be known.

3. People like to disdain statisticians, but (good) statisticians actually understand the scope and limitations of their discipline better than anyone else. They are usually not to blame for the way statistics are misused and abused by others.

Finally, the authors note that the book is not structured anything like a standard textbook in either its choice of topics or their ordering (which I think is a great virtue):

Quote:This is a book from the point of view of the consumer of numbers. It is short and to the point. Each chapter starts with what we see as the nub of the matter: a principle, or a vivid image. Wipe the mental slate clean of anxiety or fuzziness, and inscribe instead these ideas, keep each motif in mind while reading, see how they work in practice from the stories we tell. In this way we hope to light the path toward clarity and confidence.

In the next post, I will start getting to some actual substance as I review Chapter 1 of the book.

Post: #4

Simpson's paradox show how counterintuitive seemingly simple statistics can be.

There are alot of examples to demonstrate it - but this one is one of my favourites:

Quote:The low birth-weight paradox is an apparently paradoxical observation relating to the birth weights and mortality rate of children born to tobacco smoking mothers.

Low birth-weight children born to smoking mothers have a lower infant mortality rate than the low birth weight children of non-smokers.

It is an example of Simpson's paradox.

http://en.wikipedia.org/wiki/Simpson's_paradox

Post: #5

cardguy, yes, that's exactly the sort of thing I'm going to talk about. It seems counterintuitive at first blush but is actually pretty easy to understand if you think through it.

Post: #6

The Numbers Game Chapter 1: "Counting: Use Strawberry Jam"

In this chapter, the authors make a very simple point. Whenever you see a number being reported, it means that something, somewhere, has been counted in some way. The number was not just handed down from on high -- there was some activity involved in coming up with the number. In particular, someone had to come up with a definition of what was being counted, and with a method of how the counting would be done. And there can be a great deal of complexity, messiness, and arbitrariness that is hidden in those details of "what" and "how".

While this seems blindingly obvious, it is often forgotten, because we associate numbers with perfectly regular patterns that allow no room for confusion or ambiguity, such as the actual abstract activity of counting that we learn in kindergarten: one, two, three, four, and on it goes in perfect order. Lulled by this illusion of perfect regularity and orderliness, it can be easy to forget that the real world usually doesn't work like that. Rather than be mesmerized by a reported number as if it was God-given, it is important to try to understand the "what" and the "how" of the counting activity that has been performed, in all its often opaque complexity. The authors suggest thinking of "strawberry jam" as shorthand for what counting in real life resembles: something soft, mushy and amorphous, rather than something hard as a diamond.

To illustrate this point, the authors consider a number of examples, all of them quite relevant to the kinds of numbers we see reported today:

1. US Centenarians: how many centenarians (people 100 or older) are there in the US? Well, how would you find out? One way would be to ask people how old they are and take down their answers. This is what the US Census Bureau does. Using this method, the answer came back that there were about 106,000 centenarians in the US in 1970.

Job done? Hardly. The actual number of centenarians in the US at the time is believed to be closer to 5,000, or about one-twentieth of the self-reported number. Basically, many very old people have no idea of how old they are. Others know but are forgetful. Still many others lie for this or that reason. The Census Bureau tries to come up with a "preferred estimate" like 5,000 by doing extensive cross-checking with the Social Security Administration, Medicare and other agencies. Still, 5,000 is no better than an educated guess.

2. "Yob Britain": a survey that made a lot of noise in the UK in 2005 suggested that "1 in 4 teen boys claims they have done a robbery, burglary, assault or sold drugs". The press had a field day with this, with headlines screaming that the UK was raising a whole generation of "yobs" (thugs or bullies).

The thing to do is look at the definition. What was actually counted? While the quote above mentions "robbery, burglary or assault", the vast majority of the self-reported "crimes" were instances of assault. Next, how was "assault" defined in the survey? Here is what the survey question actually asked:

Quote:Have you ever used force or violence on someone on purpose, for example by scratching, hitting, kicking, or throwing things, which you think injured them in some way?

Please include your family and people you know, as well as strangers.

58% of assaults turned out to be instances of "pushing or grabbing". And 36% were against siblings. So as the authors say, "anyone who pushed a brother or sister six times leaving them no worse for the wear is counted as a 'prolific offender'". Anyone who pushed a little brother or sister even once and left a bruise would be counted as a "serious offender" since the offense led to injury, and be bracketed together with drug dealers, burglars, juvenile murderers and so on.

In other words, the definition used in this survey (the "what") was so ludicrously broad that the "1 in 4" number loses all meaning. In fact, what is more stunning about this number is that 3 of 4 teen boys self-reported as being lifelong little angels who never pushed or shoved a friend or a sibling. And that brings us to the "how" of this number -- self-reported answers to a survey, which have no necessary relation to reality.

3. Bullying in US schools: this example is similar to the previous one. A US study reported the alarming finding that 30% of children in the US in grades 6-10 had "moderate to frequent involvement in bullying". This seems like a huge number but again we need to look at the definition.

First, "involvement" encompassed both the supposed perpetrators and victims, both the putative bullies and those being "bullied". Second, "moderate bullying" was defined as happening "sometimes". Finally, and most importantly, here is how "bullying" was actually defined in the survey:

Quote:Bullying can take three forms: physical (hitting, kicking, spitting, pushing, taking personal belongings); verbal (taunting, malicious teasing, name calling, making threats); and psychological (spreading rumors, manipulating social relationships, or engaging in social exclusion, extortion, or intimidation).

As Joel Best, who studied this example in his book "More Damned Lies and Statistics", remarked: "does anyone remember high school"? Again, with this kind of fantastically broad definition, the real thing to marvel at is that the figure arrived at the survey is 30%, rather than closer to 100%.

There are more examples in the chapter but these suffice to illustrate the point being made here. Don't imagine a number, just because it is a number, as possessing some sort of special hardness and brilliance. Think strawberry jam, not diamonds. Think of how the number was arrived at -- what was being counted (the definition), how was this done (the method). Many claims that appear to be implausible or sensational on their face dissolve under closer inspection because the actual definition of what was being counted does not match the common sense expectation of what the correct definition would be; or because the method used to arrive at the number was flawed or error-prone; or some combination of both.

The lesson of this chapter is:

Be alert to the "what" and the "how" of counting; remember that it is usually a complex human activity; don't imagine that real-life numbers have a special hardness or brilliance; don't think diamonds, think strawberry jam.

Post: #7

Just found this thread Lizard. Looks interesting!

Post: #8

(02-18-2014 08:36 AM)Feisbook Control Wrote:  I'll just throw a really simple concept in here that seemingly very few people at large know (I didn't know about it until my late 20s/early 30s): The Rule of 72. It's amazing how useful this concept (that can be calculated in your head) is for calling BS on all sorts of financial or economic claims.

Very true. Knowing Pareto's law is even more helpful in life.

http://en.wikipedia.org/wiki/Pareto_principle


Another helpful principle (in terms of protecting yourself from it), is the "stupid argument" doctrine (for lack of a better name). Life has become so complex that very few people are knowledgeable about everything anymore. As a result, all someone needs to do to neutralize the argument of someone who is 100% right is to make a stupid argument.

Virtually any stupid argument will do, although emotional arguments are best. About half of the population, usually the lazy non-thinkers, will simply gravitate towards the stupid argument. How is this doctrine pertinent to this thread? Aside from stupid emotional arguments, stupid statistical arguments typically work just as well.

Post: #9

Great thread, keep up the good work!

Post: #10

This is a great topic, because it shines a light on how we are being manipulated by the so called "elites." They'll have a pre-determined conclusion in mind, conduct a study to get the results they want (by conflating relatively benign occurrences which happen with more frequency with the malignant occurrence in the headline). Then the "surprising" stat gets repeated by the media, academia, lobby/activist groups, and politicians. Then laws and rules are written to address the "problem."

It's a way to create hysteria to manipulate the public to give you want you want.

Example from The Women's Center Inc:

Quote:FACTS:

Sexual violence remains the most dramatically under reported crime, with an estimated two-thirds of attacks unreported.

Every minute in America there are 1.3 rapes of women.

Sexual assault remains the most dramatically underreported crime, with two-thirds of attacks unreported. Friends and acquaintances commit more than half of reported sexual assaults and rapes.

http://www.thewomenscenterinc.org/sexual-assault-info/

I can only imaging how diluted the definition of rape had to be to get the stats they needed above.

Post: #11

(03-04-2014 02:19 PM)Dusty Wrote:  This is a great topic, because it shines a light on how we are being manipulated by the so called "elites." They'll have a pre-determined conclusion in mind, conduct a study to get the results they want (by conflating relatively benign occurrences which happen with more frequency with the malignant occurrence in the headline). Then the "surprising" stat gets repeated by the media, academia, lobby/activist groups, and politicians. Then laws and rules are written to address the "problem."

It's a way to create hysteria to manipulate the public to give you want you want.

Example from The Women's Center Inc:

Quote:FACTS:

Sexual violence remains the most dramatically under reported crime, with an estimated two-thirds of attacks unreported.

Every minute in America there are 1.3 rapes of women.

Sexual assault remains the most dramatically underreported crime, with two-thirds of attacks unreported. Friends and acquaintances commit more than half of reported sexual assaults and rapes.

http://www.thewomenscenterinc.org/sexual-assault-info/

I can only imaging how diluted the definition of rape had to be to get the stats they needed above.

Another classic example: "We must ban guns because of the number of children dying from gun shots."

What they purposefully fail to disclose is that the vast majority of those "children" are gang members (under age 18) killing each other.

Post: #12

Here are some interesting curiosities...

Gotta' mention Benford's Law - http://en.wikipedia.org/wiki/Benford's_law - the tax authorities y actually use this mathematical law to see if you are fiddling the numbers in your books.

Also - my interest in magic and sleight of hand intorduced me to nontransitive dice. They are pretty interesting. Warren Buffet finds them fascinating as well.

http://en.wikipedia.org/wiki/Nontransiti...en_Buffett

Running with the idea - you get this interesting application:

http://en.wikipedia.org/wiki/Penney's_game

Lastly - don't forget the legendary Monty Hall Problem:

Post: #13

I am a big fan of the books of John Allen Paulos.

Quote:With the same user-friendly, quirky, and perceptive approach that made Innumeracy a bestseller, John Allen Paulos travels though the pages of the daily newspaper showing how math and numbers are a key element in many of the articles we read every day.

From the Senate, SATs, and sex, to crime, celebrities, and cults, he takes stories that may not seem to involve mathematics at all and demonstrates how a lack of mathematical knowledge can hinder our understanding of them.

http://www.amazon.com/Mathematician-Read...038548254X

In the book above - he includes a nice observation about the power of 'self selecting groups'. The example he gives is of a saturday night walking along a beach and seeing all the happy, young couples walking hand in hand and being in love.

After spending an hour walking past one happy couple after another - it is easy for a melancholy soul to imagine that he is the only one unhappy and alone that night.

But of course - all the unhappy and alone people tend not to spend their Saturday nights going for a sunset walk by the beach. It is only those who are happy and in a relationship who tend to head to the beach in the evening to advertise that fact.

Statistical and mathematical thinking help us see the bigger picture. It helps me in my daily life as well. If you do anything enough times you will eventually get a bell curve type effect. Most days will be average, some days will be great. And some days will be shit.

I saw this the other day in my local shop. I go in there most days - and have never had a problem. Yesterday - I went in and their was nearly a fight between the customer and the store clerk who asked him for ID for alcohol.

If I rarely went in the shop - I might be annoyed at having to deal with that kind of stress straight after work. But since I go on there most days - I expect to (eventually) see every type of extreme.

And the same is true for every other aspect of your life. If you do something enough times - eventually the shit will hit the fan. I even expect it when I eat out. I eat out alot - and sad to say - it is only a matter of time before I find a giant hair or a fly in my food. Just the nature of these things - when you do the same thing again and again. You can expect just about anything to eventually happen.

His book 'A Mathematician Plays The Stock Market' is brilliant as well. One of the best books I have ever read.