Friday 27 February 2015

The Objections of chrysics - Part 1

Please note that since I wrote this article, I have been persuaded that the argument it relates to is wrong (meaning that chrysics and Mathematician and irishsultan and ChalkboardCowboy were all right from the start and I should have listened to them rather than arguing with them).  Fortunately, I didn't because, for me at least, this little intellectual journey has been far more interesting than it would have otherwise been.

The correct answer for the scenario as it is worded is not 1/2 but rather 1/3 (meaning that the likelihood of winning as a consequence of staying is 2/3).

---



Over on Reddit, I’ve been engaged in discussions with many “dumb repeaters” and a small cadre of intelligent, highly educated people who have provided useful feedback in the way of objections to what I’ve been writing about recently.

Because I find Reddit particularly unwieldy for more in-depth discussions, I will try to respond to a longer comment here.  It’s not intended that these be comprehensible for all readers, in fact it is pretty much directed at one person, but the challenge was made publically, so the response is presented publically.  If you have not already been engaged in the furore, I suggest looking back at earlier articles here and traipsing over to reddit.com to check things out.  Try r/badmathematics where I am in the running for a Golden Goat Award (I’d like to thank my family …), while you are there, you can soak up the arguments against my position, which I strongly recommend that you do.

The comment is from u/chrysics but this person has also commented on this blog:

neopolitan – 100% of the time when a Red door has been opened to reveal Mary will the Red door have been opened to reveal Mary

chrysics – Nobody is disputing that. But this can occur in two ways:

1.    Red Mary can be the only choice the host was allowed to make

2.    The host had a choice between Red Mary and Green Ava

Now, the probability of either arrangement of goats is equal. But the arrangement of goats required for case two only gives you a 50% chance of a Red Mary game, while case 1. gives you a 100% chance. Two thirds of the time that Red Mary shows up, it's because you were in case 1.

See Q10 of Marilyn Gets My Goat, this just says the same thing.  Also, I address this issue at Monty Two Face.
           
neopolitan – It's a single iteration, one-shot scenario,

chrysics – That has nothing to do with anything.

Naturally I disagree.  I very specifically wrote The Reverse Monty Hall Problem to make it a real situation in which there is a single iteration of the game.  I peopled the scenario with real people (and tried to make that even clearer in Marilyn Gets My Goat).

neopolitan – Personally, I don't really care if the goats are interchangeable or not,

chrysics – Well you should, because - unless you're using the term 'interchangeable' to mean something different from its established meaning in probability - it changes the outcome of the game.

I’m probably not using the term 'interchangeable' in a strict sense.  All I mean is that in a real version of the game, there will be two goats and these goats are not the same goat and thus these real goats are not interchangeable.  (Equally, there is not 2/3 of a goat behind each door and 1/3 of a car.)

neopolitan – I can build my argument on them being the same goat hidden behind two doors, if that is what you prefer.

chrysics – It's not. I don't care about the goats. There is one door with a car and two losing doors. If both losing doors are selected, the host decides which to open randomly; otherwise the host opens the only losing door selected. That is all that is important. The losing doors can hold separate goats, the same goat, nothing, a guitar in one and a performance art troupe behind the other, it's completely and utterly irrelevant.

I agree.  But some people arguing wanted to stick with hypothetical goats, strangely divisible goats, even when I was trying to talk about a notionally real situation.

neopolitan – Which do you actually want me to argue? That the goats are interchangeable or not interchangeable?

chrysics – The interchangeability of the goats is a function of the rules of the game. You have consistently described a set of rules in which the goats are interchangeable. But you keep saying they're interchangeable, and on that one occasion you also described a version of the game in which the rules differ in a subtle-but-important way from your other descriptions, which just so happens to make the goats non-interchangeable.

I don't want you to argue that the goats are interchangeable, nor do I want you to argue that the goats are not interchangeable. I want you to clarify which set of rules you want to play by. Is the host required to open Red Mary whenever that is a possibility, or is the host permitted to open Green Ava even when Mary is behind the red door? If the latter, then I also want you to explain why you insist that the goats are not interchangeable.

Actually, this simply isn’t true.  When I first mentioned the concept of “interchangeability”, I wrote (emphasis added):

(as wotpolitan) It is easy enough to overcome logically. I say my scenario is a single instance, one-shot game. Then someone says, aha, but he could have opened a different door. But my point is that he didn't. He opened a specific door revealing a specific goat and it is this specific scenario that the contestant has to deal with. Some say that the goats are interchangeable, and I say that might be the case with a purely imaginary game, but I put the contestant in a shopping centre with me (the dishevelled amateur philosopher), a relatively real person, conducting the experiment with goats that I keep trying to tell people are real, and patently not interchangeable. They say, if you run it multiple times you'll get 2/3, I agree and say that it's different if you run it just once. I show how it works with conditional probability, they say that that would be fine, but conditional probability doesn't apply - perhaps they are right about that, but it seems to apply fine from what I can see.

I am fine using whichever you prefer, but if you have no preference, let’s go with non-interchangeable goats called Mary and Ava, because it prevents the problem mentioned above.

In the scenario I describe at both The Reverse Monty Hall Problem and Marilyn Gets My Goat, there is no obligation on the part of the host to open the Red Door if Mary is behind it, unless the car is behind the other selected door in a Red-White or Red-Green scenario.  However, if the host has opened the Red Door to reveal Mary, then there is an obligation to consider only scenarios in which the host could open the Red Door to reveal Mary for that particular instance of the game (which is the only instance as I formulated the scenario).

neopolitan – I think your comment there is simply irrelevant because the host makes one decision and one decision only to open one door

chrysics – In potential Red Mary games, the host only make a decision half of the time - the other half of the time, the host has no choice at all. And half of those decisions result in the outcome of the game not being Red Mary. Of the potential Red Mary games, only 3/4 are actual Red Mary games. In 2/3 of those 3/4, you win the car by sticking with your initial choice. The other 1/4 are irrelevant, as they're not Red Mary games at all.

This looks to be based on a misunderstanding of what a Red Mary game is.  I take responsibility if I was insufficiently clear, but a Red Mary game is a game in which Mary is behind the Red door, irrespective of whether she is revealed by the host as being behind the Red Door.

However, if we call these Revealed Red Mary games, you are right in that the host makes the decision whether to open the Red Door or nor half the time and is obliged to open the other half of the time.  I address the problem highlighted here at Monty Two Face.

chrysics – Let me break it into stages, so you can tell me where you think I go wrong. (We'll assume for the sake of consistency and simplicity that the contestant's chosen doors are Red, the leftmost door, and Green, the rightmost door (neopolitan comment – agreed, this is consistent with the scenario as laid out in Marilyn Gets My Goat)

1.    A Red Mary game denotes a game in which the host opens the Red door, revealing the goat Mary to be behind it (neopolitan comment – not agreed, this is a subset of the set of games I call Red Mary games, we could call this a Revealed Red Mary game, I will update your text below to reflect this, updates in brackets)

2.    The goats and the car are arranged behind the doors at random

3.    A (Revealed) Red Mary game may occur when the goats and car are arranged MAC or MCA

4.    Both of those arrangements are equally likely:
P(MAC) = P(MCA)

5.    No other arrangement can produce a (Revealed) Red Mary game:
P((
Revealed)RedMary| (!MCA && !MAC) ) = 0

6.    The contestant has no knowledge of the arrangement until the host opens the door

7.    At that point, the contestant has no knowledge beyond what is revealed (Mary is behind the Red door) and anything that can be deduced from that

8.    If the arrangement is MAC, there is a 100% chance that the host will open the Red Door, producing a (Revealed) Red Mary game

9.    If the arrangement is MAC, there is no possible result other than the one given above:
P((
Revealed)RedMary|MAC) = 1

10. If the arrangement is MCA, there is a 50% chance that the host will open the Red Door, producing a (Revealed) Red Mary game

11. If the arrangement is MCA, there is a 50% chance that the host will open the Green Door, producing a (Revealed) Green Ava game

12. If the arrangement is MCA, there is no possible result other than the two given above:
P((
Revealed)RedMary|MCA) + P((Revealed)GreenAva|MCA) = 1, and they are equally likely:
P((
Revealed) RedMary|MCA) = P((Revealed)GreenAva|MCA)

13. As MAC and MCA are equally likely arrangements,
P((
Revealed)RedMary|MAC) = P((Revealed)RedMary|MCA) + P((Revealed)GreenAva|MCA)

14. Using:
P((
Revealed)RedMary|MCA) = P((Revealed)GreenAva|MCA), and
P((
Revealed)RedMary|MAC) = P((Revealed)RedMary|MCA) + P((Revealed)GreenAva|MCA), we arrive at:
P((
Revealed)RedMary|MAC) = P((Revealed)RedMary|MCA) + P((Revealed)RedMary|MCA)

15. Using:
P((
Revealed)RedMary| (!MCA && !MAC) ) = 0, it can be deduced that:
P((
Revealed)RedMary) = P((Revealed)RedMary|MCA) + P((Revealed)RedMary|MAC)

16. Using:
P((
Revealed)RedMary|MAC) = P((Revealed)RedMary|MCA) + P((Revealed)RedMary|MCA), this may be rewritten as:
P((
Revealed)RedMary) = P((Revealed)RedMary|MCA) + [P((Revealed)RedMary|MCA) + P((Revealed)RedMary|MCA)] (the square brackets denote nothing other than where the equation has been re-written)

17. The re-written equation P((Revealed)RedMary) = P((Revealed)RedMary|MCA) + P((Revealed)RedMary|MCA) + P((Revealed)RedMary|MCA) accounts for all (Revealed) Red Mary games, and consists of three equal contributions.

18. The first of these contributions accounts for the (Revealed) Red Mary games arising with the arrangement MCA.

19. The second and third contributions - each of which is equal individually to the first - account for all remaining (Revealed) Red Mary games, in which the arrangement is MAC. Collectively, this MAC. Collectively, this accounts for two thirds of (Revealed) Red Mary games

20. For any given (Revealed) Red Mary game, there is a 2/3 probability that the arrangement is MAC.

I am not well versed enough in this notation to confidently identify the precise step in which the error (as I see it) creeps in.  However, the issue is that we are talking about a situation in which there is a Revealed Red Mary, meaning that the only options are that we are playing an MAC game or an MCA game.  I agree that we will only see the Revealed Red Mary half the time when we are playing a Red Mary game of the form MCA (where Red and Green were selected).  But I disagree that this means that we can eliminate the MCA games that we would not see.  This is the point I am trying to make at Monty Two Face.  I think that we need to compare Red Mary games, not Revealed Red Mary games.


By rejecting Red Mary games that are also Revealed Green Ava games (ie 50% of the MCA games), which is effectively what you are doing, we skew the result.

---

Update:

As mentioned in the comment below, I was not seeing the wood for the trees when reviewing chrysics post.  I was convinced that his core argument was that I was wrong, so I completely overlooked that his conclusion was that the probability of a Red Mary being revealed as a result of a MAC distribution of goats and car is 2/3 - a point I made in Marilyn Gets My Goat (see Q10 and also Q16).  Therefore, I actually agree with him.  No wonder that I could not find any error (a fact that was a little disconcerting, since it all seemed pretty straightforward).

That said, my last few comments above still stand, I just wasn't properly addressing what chrysics was saying.

8 comments:

  1. So, we'll accept that "Red Mary" refers just to games where Mary is behind the Red door, independent of whether or not that's revealed to the contestant.

    First,a question: If you consider my argument with the addition of the "Revealed" proviso, do you disagree with the conclusion? You said in the post that you're not familiar enough with the notation. I struggle to understand that, you've used it yourself ( http://neophilosophical.blogspot.de/search?updated-max=2015-02-22T22:12:00-08:00&max-results=1&start=3&by-date=false ). The only notation I added to that was using C-style logical operators in (!MCA && !MAC), but I feel the text explained that clearly enough even if you're not familiar. There's no trick to the notation, P(event|condition) is the probability of event, given that condition. Beyond that everything I did was basic algebraic rearrangement and substitution. If you need anything clarified, I'll be happy to do so.

    But ok, now on to a response. The problem is that when you define Red Mary as just "any game in which Mary is behind the Red Door", it's no longer indicative of the information the contestant has. The contestant can't deduce probabilities on the basis of whether or not they're in a Red Mary game, because the contestant regularly does not know whether or not they're in a Red Mary game. There's still validity to picking a specific scenario for illustration, although I must admit I'm confused as to why you'd pick this one in particular. In any case that's not the point. Red Mary is isomorphic to Green Ava, White Mary, and all other similarly-defined labels.
    Previously, while discussing what we shall now call a Revealed Red Mary, I was happy to go along with the assumption that the contestant's chosen doors are Red and Green. This is because I was discussing the probabilities deduced by the contestant, on the basis that Mary has been revealed behind the Red Door. The contestant choosing Red and Green or Red and White doesn't affect anything, in that scenario.

    In a plain Red Mary, the contestant is no longer deducing on that basis. Essentially, the label is irrelevant to the contestant. Still, we can go ahead and work out the probabilities, using our position outside the game to have perfect knowledge of the situation. But we can no longer reduce the game to the situations where the contestant picked "Red and Green", as there are other types of Red Mary game which do not have the same set of probabilities. The contestant's choice of doors is based on no information relating to what's behind the doors, essentially they choose at random. Any choice (RG, RW, WG) is equally likely.

    If we say that there's a Red Mary game where the contestant has picked Red and Green:
    your argument is correct, there are two possibilities (MAC and MCA), each of which is equally likely. The contestant has a 50% chance of winning by switching.

    And if we say that there's a Red Mary game where the contestant has picked Red and White:
    your argument is again correct, there are two equally likely possibilities (MAC and MCA). The contestant has a 50% chance of winning by switching.

    But there's a final option. This was never discussed previously, as I stated earlier, due to the fact that it can never produce a (Revealed) Red Mary. But without the requirement that the Red Mary be revealed, this scenario must also be considered:

    A Red Mary game where the contestant has picked White and Green: your argument is not valid. There are two equally likely possibilities (MAC and MCA), but both arrangements have the car behind one of the contestant's chosen doors. The contestant has a 0% chance of winning by switching.

    As each selection of doors is equally likely, you can thus determine that the overall chance of winning by switching - given that Mary is behind the Red Door - corresponds to the mean of the three cases: (0.5+0.5+0)/3 = 1/3

    ReplyDelete
  2. Sometimes you can't see the wood for the trees and you have to just walk away from a problem for a while to get some perspective. I was focussing close on your notation there, chrysics, and I looked and I looked and I looked and I could not see an error.

    Then I went off to do something else for a while, during which time you replied and gave away the fact that you might be doing this all in a second language - which makes my complaint that I am not familiar with the notation sound a little weak. So, I thought about it again.

    Guess what! You're not wrong. Your conclusion is that for any Revealed Red Mary, the likelihood is 2/3 that it is due to there being an MCA configuration of goats and car. I agree. I said as much in Marilyn Gets My Goat (see Q10). I was working on the assumption that what you were attempting to prove here was something that I had not already said was true - which would sort of explain why I couldn't find any error.

    Just as I have accused others of doing, I was trying to solve the wrong problem when reviewing your reddit post.

    Now, in your reply, you say:

    "If we say that there's a Red Mary game where the contestant has picked Red and Green:
    your argument is correct, there are two possibilities (MAC and MCA), each of which is equally likely. The contestant has a 50% chance of winning by switching."

    That's basically been my argument all along. Everything aside from that has been a diversion, perhaps due to my not being entirely crystal clear about what I meant. (And when I say that everything aside from that has been a diversion, I include some of what you wrote after that comment, because it's not actually part of my argument - and again, I am not blaming you, it's likely due to me not making myself absolutely crystal clear.) Thanks,I really do appreciate you staying with the discussion this far.

    Now, how do I get myself redeemed given that I have been pilloried so long and so hard over at /r/badmathematics? :)

    ReplyDelete
    Replies
    1. "and gave away the fact that you might be doing this all in a second language"

      not really the point, but I'm kinda curious as to how I've done that?

      "Your conclusion is that for any Revealed Red Mary, the likelihood is 2/3 that it is due to there being an MCA configuration of goats and car. "

      Just to clarify... for any Revealed Red Mary, there is a 2/3 chance that it happened because the arrangement MAC (green door on the right). I guess that's just a typo or a mixup in the order of the doors, you're agreeing that it's a 2/3 chance that some given Revealed Red Mary arises because the host had no other option? In any case it's clear that you're not claiming a Revealed Red Mary is equally likely to be a result of MCA as MAC.

      Further clarification: You are asking for the probability that the contestant will win by switching. Specifically, the probability deduced by the contestant, on the basis of the information available to them. I've no real doubt about this but just want to make it absolutely clear.

      Delete
    2. I don't think that's been your argument all along, neopolitan. In particular, there's this comment of yours from reddit: "I do realise (as suggested at the comments) that if I am right about the Reverse Monty Hall Problem, then the Monty Hall Problem, as originally phrased by Whitaker was incorrectly solved by Marilyn vos Savant. Meaning that ONCE THE HOST OPENS THE DOOR, the probability of benefiting from a switch is actually 0.5, just as almost everyone believed at the time."

      (Caps mine, it's the only emphasis I know how to produce here.) This clearly corresponds to a REVEALED Red Mary game. The statement chrysics made is about a plain Red Mary game. It also doesn't disagree with the accepted solution to the Monty Hall Problem, as you have consistently claimed to do.

      If you want to be "redeemed" (and does it matter, really?), you're going to have to admit that you were wrong, not try to change your mind about what you've been saying.

      Delete
    3. Can I suggest you read my original article - http://neophilosophical.blogspot.com/2015/02/the-reverse-monty-hall-problem.html - if you want to work out what I was originally saying. As I have said elsewhere, I am not really responsible for what others think I wrote, only for what I wrote. I agree completely that what I wrote corresponds to a Revealed Red Mary Game. I am glad that you consider that clear. It was intended to be clear. (I also agree that it doesn't agree with the accepted solution to the Monty Hall Problem, although I am at a bit of a loss as to where you think I said that I agree totally with the accepted solution to the Monty Hall Problem. My acceptance of the accepted solution is caveated. If I ever left that caveat off accidentally, I apologise and ask that you point out where I did so.)

      The difference between a Revealed Red Mary game and a Red Mary game is that the contestant only knows that she is playing a Red Mary game AFTER the door is opened. And once the door is opened, the contestant not only knows that she is playing a Red Mary game but also that she is not playing any of the games in which the car or Ava are behind the Red Door.

      Redemption doesn't really matter. The people who get off on r/badmathematics/ (and now, due to the attentions of the thoroughly delightful /u/CompletelyIneffible, also on /r/badphilosophy) are not likely to be the kindly, thoughtful souls who would, when they find that they were wrong about the subject of their circle-jerking, try to make amends - they'll just move on to the next target. It's rather sad indictment on the mob, but the mob never had a particularly good reputation.

      Fortunately, as I get sufficient validation from the real people I interact with, I don't need any from reddit.

      Delete
  3. Yes, there is a typo, it should read MAC, rather than MCA (MCA is a record label, so it rolls of the fingers more easily).

    Yes, I was asking for the probability that the contestant would win (the car) by switching, as deduced by the contestant, on the basis of the evidence available to the contestant.

    As to the language thing, Blogger is a Google product and they divide us up so as to more efficiently advertise to us. When you linked to one of my comments, there was a .de country code indicating that you are either German or living in/visiting Germany. There's a fair chance that you are German, so you probably speak German as a first language :) It's just a guess, so I did say "might".

    ReplyDelete
    Replies
    1. Ah, the logic makes sense on the language thing, but you've been misled. I'm actually from the UK originally, just moved to Germany a few years ago. Anyway...

      So yes, the question relates to the probability, as deduced by the contestant, that switching will win the car.

      Let's consider, then, a Revealed Red Mary game. The contestant - once the host has opened the Red Door, revealing Mary - knows that there are two possibilities for the arrangement of goats & car: MAC or MCA. From the information available to them, and following the process I outlined, the contestant can deduce that there is a 2/3 chance they have found themselves in a Revealed Red Mary game due to the arrangement being MAC, and only a 1/3 chance they are in this situation because the arrangement was MCA. Indeed, not only do you not disagree with this, but you even point out that you've actually made an equivalent observation in your previous post "Marilyn Gets My Goat". So it seems that this is beyond question, if the contestant finds themselves in a Revealed Red Mary game, they can deduce that there is a 2/3 likelihood that this is due to the arrangement being MAC.

      So this is where I'm incredibly confused. Why do you believe the contestant would deduce that they have a 1/2 chance of winning by switching, if you also believe they would deduce that they have a 2/3 chance of being in a game where they lose by switching?

      Delete
    2. I'm glad that I availed myself of the word "might" :)

      Now, onto the incredible confusion. I don't think that they have an imbalanced likelihood of winning and losing. It's 1/2 both ways, a likelihood of winning of 1/2 from switching and a likelihood of losing of 1/2 from switching. You seem to agree with this for a single iteration, one shot instance of the game (from where you write "If we say that there's a Red Mary game where the contestant has picked Red and Green:
      your argument is correct, there are two possibilities (MAC and MCA), each of which is equally likely. The contestant has a 50% chance of winning by switching.")

      The thing is that we know that in a Red Mary game in which the contestant selected two goats, it is LESS likely that the Red Door will be selected than it would in a Red Mary game in which the contestant selected a goat and a car. However, this would be exactly the same if, for example, in a Red Mary game, the host opened the Green Door to reveal Ava (it would have been LESS likely that the Green Door would have been opened than it would have in a Green Ava game in which the contestant selected a goat and a car). We expect to see Red Mary opened less frequently if Ava is behind the other door. Therefore, in a single iteration, one shot instance of the game, we need to factor that in.

      I believe that this is handled quite adequately by conditional probability.

      I'm going to respond to your longer response in a separate article, because it gives me more flexibility than responding in comments.

      Delete

Feel free to comment, but play nicely!

Sadly, the unremitting attention of a spambot means you may have to verify your humanity.