Saturday, January 16, 2010

Hey, Your Game Cheats!

So I have debated for a while now about writing this post, as it can be perceived negatively by some. However, over the last few days we have seen a lot of downloads of the Cribbage Pro game on the Android phones, which is really awesome, and at the same time there has been a rise in the number of comments and emails we have been fielding from players who seem to insist the computer is cheating on the various levels of play. I think the vast majority of players realize this is not the case, as many of them we can see in the statistics are winning against the computer an incredibly large number of times. Similarly, we get emails and comments from others that say the computer is simply too easy to beat and they never lose on any level. I think something inside of us (and yes, I mean "us" as some on our staff have honestly asked the same question) want to accuse a computer of cheating when it beats us soundly in a game. This I suppose is one of the draw backs of creating a game that is indeed challenging to win at.

Initially, I found the accusations funny, because I wrote the game and I know every line of code and piece of logic that is in it inside and out. The funny part is that it would have taken me longer and more code to write a game where the computer is cheating then what we actually did in spending our time researching game play strategy and writing a game that simply knows the rules of cribbage very well and how to win. So let's start with a clear and straightforward statement - I stake my personal reputation and that of the entire Fuller Systems Inc. staff, brand and name on everything we make, and this game is no different. With that in mind, I can say with extreme and utter certainty that the game in no way shape or form is cheating. The computer play is playing a completely "fair game" in every sense of the phrase.

With that said, let me explain what is going on in the game a little, and how it works. If you have received an email from me on this topic, this will look familiar.

First, the hands dealt are done so exactly the same way for each and every difficulty level, with the only difference in the levels of play being which cards the computer may choose to hold or discard. This is why on advanced, the hands the computer ends up with tend to score significantly higher then on the other skill levels. The code for the game runs in such a way as to do the deal of cards exactly the same way each time and the shuffle is as natural as possible for each hand. I fully understand that sometimes the computer can/will achieve hands that are consistently higher in the advanced mode, but this is due to the level of computations it goes through in determining what the best possible holding of cards could be. It is in no way doing anything other then just that - working with the cards it is dealt. This is not to say that the computer can not go "on a run", just like a human player might, and get a series of good hands dealt to it - that is just as likely for the computer as it would be for a human player if that human player were to make the same discard decisions. Just as we don't manipulate the deal or play to give the computer an advantage, we don't handicap it either.

To go a step further, here is the exact code used during the dealing of the cards from the deck which is a simple object that holds an array of cards that have been shuffled prior to this step.


// 6 cards for each player - 12 total
for (int i = 0; i < 6; i++) {
  //Deal to pone player first, then to dealer
  if (isP0Deal) {
    player1Cards[i] = deck.nextCard();
    player0Cards[i] = deck.nextCard();
  } else {
    player0Cards[i] = deck.nextCard();
    player1Cards[i] = deck.nextCard();
  }
}

If you know code structure, or can read through and derive the basic thought from this, you can see that it has no consideration for the computer player (player0) then the other player and that it deals out the cards as per cribbage rules based on who is the dealer. Unfortunately there is not much more I can add here, short of sharing the entire game code with everyone, which we can not do for obvious reasons. So one could claim that this code is not the real code, or that later in the game code changes are made or something. Please understand, there would be absolutely no value for Fuller Systems to have a game that we give away for free, cheat in any way. In the future releases of features we add, you will see more on why that is (yes, think multi-player). In those future versions it becomes even more critical that we ensure no player has any advantage in the deal over any other player.

Lastly, we are working on finding an independent authority to audit and review the game to prove it is fair and accurate in it's play, which we hope will remove some of these concerns. As of yet, we have not found an independent organization that is capable of doing this, so if you have suggestions, please let us know by email or in the comments here.

Thank you for letting me take the time on this blog to respond to some of the accusations, and if you have any thoughts, suggestions or questions, please don't hesitate in letting us know.

-- Josh

21 comments:

  1. Josh,

    While I've seen some pretty awesome play from the CPU player, it's not a million miles away from a very skilled human (IMHO). And that's on normal too.

    Sure - I've been disappointed by losing so heavily some times, but I've also come to realise that my play in those games was sub-par to say the least.

    I'm happy that this is one of the better Cribbage games that I've played on a computer. All you need to do now is fix the recording of double-skunks - yes, I double skunked the CPU. :D

    ReplyDelete
  2. Thanks for the kind words and detailed comment.

    We do track skunks and double skunks in your game statistics and announce them when you do too. Make sure you have the latest version if you don't see it. If that is not what you are referring to, please let me know.

    Thanks again,
    Josh

    ReplyDelete
  3. This comment has been removed by the author.

    ReplyDelete
  4. Hi Josh,
    Awesome game. I love it.
    It doesn't bother me one way or the other whether the computer cheats on advance or not as there is no cash involved.
    I work in auditing within the gambling sector and certify code for it's 'fairness'. The snippet of code you have posted would not be enough to should the code is fair; you need to show the deck being created and being shuffled too before people believe you, then you have to show there is sufficient card distribution. Between me and you, I wouldn't stress about it if people are suspicious; when money comes into however, you'll have to go further to 'prove' it.
    Again, awesome game. Thanks.

    Actually, there is one thing that bugs me. When I win, I get a couple of points, even if I thrash him. If I lose however, even by 3 points - which is why I read this thread in the first place - I drop 40 (four-zero) points! Wtf?

    ReplyDelete
  5. Hi "Third Man", thanks for your post. I agree that more code could be shared, and I am more then willing to share it with the right person/institution when the time comes, but as there is some proprietary logic in what some of it does, posting it online publicly would pose a disadvantage right now. Also, as I said, anyone can always say I did not post something or removed the part that was "cheating", etc. Either way, no matter what I share personally, it is just me sharing it and so it becomes less credible to some.

    In regards to the points, it is a simple, but technical, answer - the system is designed for ranking each player as accurately as possible and so we had to give virtual "points" to the computer players you are playing against (the advanced player is a "2500" point player). So the points is your "Rank" against all others, and just as others win and lose, their points will follow the same algorithm. Those computer players only have so many points themselves to give away, and as your level goes beyond their level, you get fewer and fewer points up to the minimum amount of 2 points for a win. Similarly, if you lose then the system works in reverse and so the penalties are larger as your points are larger then theirs (up to the maximum of 41). The system we are using is very close to this one:

    http://www.cxrchess.com/CXRStatsExplained.php

    It is intended for player to player ranking more then computer ranking, which is why it seems odd right now. When we add the multi-player functions, it will make a lot more sense. However, we may change the algorithm as well as we are still in beta with it, and I completely understand your point about "WTF". We will likely keep this one for multi-player either way, as we believe it proves out to be the fairest we have found.

    Thanks again,
    Josh

    ReplyDelete
  6. Ah, so it's based on an simplified ELO-like algorithm; I get ya. If the advanced player is 3500, what are the ratings for the other levels - just out of curiousity.

    As far as code goes, only share what you feel comfortable sharing, afterall, it's your IP and you shouldn't feel like you need to justify anything.

    I haven't played cribbage since I was young, so any implementation is sweet. Your implementation is rocking. I find it odd that we have had so many advances in technology, yet we always go back to the basic game-play games, chess, cribbage, patience et al.

    ReplyDelete
  7. Thats right, sounds like you get the idea and understand it quite well.

    The three point levels for the computer players are (and apologies, this all should have been in the help file and is not):

    Advanced: 2500
    Intermediate: 2000
    Novice: 1500

    Thanks again, and if you have any suggestions at all, or see anything strange, do let me know.

    Thanks,
    Josh

    ReplyDelete
  8. Great game.
    My only comment is about scoring for unfinished games. It looks like lot of players in top 50 have an inordinately large number of unfinished games. This leads me to think some people simply abandon games instead of taking a loss. In fairness there should probably be some pretty heavy penalty for unfinished games over a certain percentage of total games, especially since it is so easy to save and restart. After all if we were playing in some kind of competition,I couldn't simply abandon losing games.

    ReplyDelete
  9. @Mike - Thanks for your comments. I agree that it appears there are at least some that are "gaming the system" by abandoning games instead of taking a hit for the loss. This is why we added that level of transparency to the "Top 50" - in hopes that the community would regulate itself without direct penalty. However, it has become apparent that some will continue that behavior anyway, and we are now very close to being ready to make a change to the system to handle that.

    We will soon roll out to our servers that track the statistics, a change that will block players from being eligible for the "Top 50" if their ratio of unfinished to completed games is over a certain percentage (more games unfinished then finished). The exact percentage is being debated internally, but once settled I will post here the details.

    -- Josh

    ReplyDelete
  10. Thank you for all the hard work.

    I have not played since I was a child... about 30 years ago.
    I was a little sceptical about how well the computer played at first. I soon remembered better how to play and realized the game was spot on..

    Cheers mate.

    I look forward to the multiplayer version..

    ReplyDelete
  11. Thanks "DanceDemon", I appreciate your comment and feedback. We try very hard to make the game competitive and fair and it is great to hear.

    We are really excited about multiplayer as well, and it is getting really close. We ran into a few hiccups along the way and had to switch our hosting strategy, etc, but we are hoping to have it ready in the next month or so.

    -- Josh

    ReplyDelete
  12. OK, I could get behind the game being really smart in the actual play of the game, i.e. pegging.  However, the computer seems to always have (before the cut)  at least 8 points or more- usually 12+.  Again, BEFORE the cut. I know its futile to argue, but it sure seems like the pro level always has superior holding hands.

    ReplyDelete
  13. Josh, your bias is clear...of course you are going to defend your baby. But, I agree with others who have actually taken the time to post after experiencing the "unreal" aspects of playing your game. If, I suspend reality and accept for a moment that all is fair in your game... How is it - the computer deals itself 3 - 12 plus point hands in the first 3 hands of a game? While i have an average of 6 points per hand... If I count the last 2 hands of the computer in the game just previous to this one I am referring to - the computer had 5 hands in a row with 12 plus points. That - Josh is just NOT realistic.

    ReplyDelete
  14. I am convinced this game was designed by Josh to drive people insane. I am going to un-install the app for Cribbage Pro... It sucks.

    ReplyDelete
  15. There are a few aspects of your comment that relate to how the game works that I can clarify for you.  The first is about the how the "computer deals itself" the cards.  As we have given in this post and shown in the others (like the one on shuffling and the deck that you have also commented on) the computer is using random data to populate a deck of cards and dealing them out in the correct pone then dealer sequencing.  It is not adjusting the position of cards in the deck to favor any side and it is not manipulating the random data.  It is is simply dealing out the cards and each has an equal chance of getting those cards.  This is not me defending a bias, it is simple fact.  Not only have we shown it in these blog posts, but I am willing to let anyone willing to sign a simple waiver actually view the code itself to prove it.  If you want to take us up on that offer, just let me know.

    Your other comment about it being "NOT realistic" is something that we have never tried to prove to anyone.  I am not saying the game will match what you expect games to be like in the real physical world, because nobody can say that about a computer generated game.  A physical world game simply has too many other factors from how fully randomized a deck of cards is (most people do not shuffle enough or correctly to achieve that between hands) to the obvious fact that most players make calculation mistakes that a computer simply does not.  However, just because the game may not be what you expect would happen in a physical world game, it in no way means it is not being fair to every player.

    ReplyDelete
  16. LOL, you found us out.  We are trying to make a game nobody enjoys playing :-)

    We are of course trying to make a game that people of many different skill levels will find fun and challenging.  Some can't, and some can.  If you can not and need to move on, that is fine. I'm not trying to frustrate anyone, and a game of cribbage is not worth that.

    ReplyDelete
  17. Great for beginners. Frustrating nightmare for those who know the game.

    ReplyDelete
  18. Josh, the one thing that validates the pitiful reality of your game is the fact that there are people that have a 75+ percent winning stat. Anyone who plays here,and I only have 4000+games! knows fully well that that is a statistical impossibility.

    ReplyDelete
  19. Josh, as an ex-IT Programmer, I agree your code as shown appears to be perfectly unbiased. However, there is definitely something that skews the hands in favour of the computer. The number of times that the computer is dealt a hand that consistently scores 12-16 for 2-4 consecutive hands makes truly random deals unlikely so the issue has to be with the Random Number Generator. On two occasions now, I have got within 1 game of winning the same number of games as the computer and on each occasion, I then lost 8 &10 games in a row, some of them because the computer had first count and scored 20+ points to get home when I only needed about 6.

    ReplyDelete
  20. Hi Ian, as a programmer you will also know that a computer can not, and will not cheat to get ahead like that without it being written to do so. A bug in a random number generator could not yield that result. However, don't take my word for it. If you want to take us up on the offer to actually review the very code itself for yourself, just let me know by emailing support@FullerSystems.com with your request. We have nothing to hide, and can absolutely guarantee it is not cheating you.

    ReplyDelete
  21. I appreciate your observations and questions, but we will probably have to agree to disagree on this one. We have shown in other blog posts that the random shuffle used results in a statistically random deck and even used real world games as the source of the data analysis. I hope you can continue to enjoy the game anyway.

    ReplyDelete