Stephen Jacob (mavjop) wrote,
Stephen Jacob
mavjop

  • Location:
  • Mood:

Statistical Mythology - TRUE!

We're watching the movie "21". It seems like it has the potential to be an interesting story.

But then ... the story starts out with the main character's induction into a group of blackjack card counters at his university, MIT, because he wows his professor by first spouting history about Newton-Rhapson method (which apparently people in the US refer to only as Newton's Method! Huh!), which is fine, but then by spouting a STUPID logical fallacy, which the professor agrees with him on.

That fallacy?

Well, the question is:
    Host shows you 3 doors. Behind 1, a prize. Behind the other 2, nothing.

    You choose a door.

    The host opens one of the other 2 doors, showing nothing.

    He offers you the option of changing your guess.

    Should you change it?
The stupid, stupid, stupid answer is:
    You should change your guess!

    *hand-waving*

    *statistical bullshit*

    ... so the one you chose first was 33.3%, which means the other remaining one has become 66.7%!
This is crap, plain and simple. The fact of the matter is that YES, the odds have changed since you originally picked (where it was 33.3% for each), but they have changed to 50% each. There is no magic causing ONE of the cards to inherit the probability of the one that was eliminated.

I've heard this fallacy often enough, back in college at least.

It makes me really angry that a movie which a lot of people who don't know any better will see depicts a supposedly top-flight MIT student and an MIT professor proclaiming this as statistical fact. I hope I can enjoy the movie in spite of this early huge screw-up.


So, yeah, I was wrong. Argh.

I take it back. Nobody has ever succeeded in explaining this to my satisfaction. It's always sounded like hand-waving (hm, how zen). The statistical explanations have always sounded like voodoo.

I just worked it out on paper, and it actually ... does work that way. WEIRD. But true. I shall explain. Perhaps my reasoning (since it's the only reasoning which has persuaded me) will help persuade others

I've heard people talk of the fact that the host is an intelligent agent, and therefore not subject to statistical analysis and so on and so forth, but none of this has ever pointed out the real reason why switching is best:

In 2 out of 3 cases, you have forced the intelligent agent to take one, with no option. He only has a choice 1 out of 3 times!

When you have forced the agent's hand, the remaining card is it. When he had a choice of which card, the one you picked first was it.

Weird? Yes, but true. I came to this realisation after I jotted down the possibilities.

I = your initial pick
E = choice eliminated by host
S = the switch option


Door LayoutDealer Option 1Dealer Option 2Initial OutcomeSwitch Outcome
1 0 01(I) 0(E) 0(S)1(I) 0(S) 0(E)always winalways lose
0 1 00(I) 1(S) 0(E)--always losealways win
0 0 10(I) 0(E) 1(S)--always losealways win

(You can re-order things so that you don't always choose the first door, but that is statistically irrelevant, assuming the prize door is properly randomized).

So, ur, wow. 2/3 of the time, the agent was forced to point out the right door, effectively.

Weird, but no longer pure voodoo. If someone had shown me what I worked out, I would have believed them. Other people's explanations now make some sense, in hindsight, but really didn't in the absence of this realisation.

I guess I owe the makers of the movie an apology!

[Edited to fix my mismatched legend and use of symbols! Oops!]
Tags: argh, geeky, media, stupid
Subscribe

  • Anyone[*] want some pots and pans? :)

    Anyone[*] want any pots and pans? We have a whole lot of pots and pans that are perfectly usable, except that they don't work with induction stoves…

  • Amusing Air Travel

    Last week, on our way home from Milwaukee, WI, we were on an American Airlines puddle-hopper (Embraer regional jet) from MKE to DFW and the captain…

  • Travel day 2

    Elapsed time: 30 hours. Back in the airport (ORD) again, waiting for AA 1341 to SFO (on time thus far). I'm looking forward so much to seeing my…

  • Post a new comment

    Error

    Anonymous comments are disabled in this journal

    default userpic

    Your reply will be screened

    Your IP address will be recorded 

  • 10 comments

  • Anyone[*] want some pots and pans? :)

    Anyone[*] want any pots and pans? We have a whole lot of pots and pans that are perfectly usable, except that they don't work with induction stoves…

  • Amusing Air Travel

    Last week, on our way home from Milwaukee, WI, we were on an American Airlines puddle-hopper (Embraer regional jet) from MKE to DFW and the captain…

  • Travel day 2

    Elapsed time: 30 hours. Back in the airport (ORD) again, waiting for AA 1341 to SFO (on time thus far). I'm looking forward so much to seeing my…