Android Market Takes A Page from Amazon - Fails

For our regular readers, this post is quite different from normal, but it is something we have decided we can not ignore and that needs more attention from everyone who is a fan of Android, or Google in general.

Just recently (7/12/2011), Google started to release an update to the Android Market for devices 2.2 and above (a vast majority of devices).  This update includes a complete redesign and a lot of great new features that could help everyone participating in the market.  However, there is one change that is very negative all around for consumers and developers/publishers that will offset any of those positives in my opinion, but has not been discussed much.  This change is in how they are now choosing to try and "rank" user reviews of apps, and in many cases only show 20 of such reviews that they are deeming "worthy".

In theory, trying to find the most relevant and meaningful reviews on something is a great idea.  This is in fact something Google does really well at with it's PageRank and searching of the web on Google.com  It is an idea that is very much needed to help filter out the "spam" and make reviews meaningful in any App Store (sorry Apple, that is a generic term).  Amazon does this quite well with their regular products (to be determined on their own "Appstore"), and now Google is trying it out on their Android Market on devices and on the web version of the market as well (it has been on the web version for a little while now).

So, what is the problem then?  It is a great idea, they are trying to do it, and that is all good, right?  The problem is that their attempt is currently a clear failure and I can demonstrate this not only with the free version of our game Cribbage Pro, but more pointedly with Google's own self published apps as well.  They are making the reviews system irrelevant, unreliable and at best completely useless to the consumer and at worst a huge negative hit for many developers.

Let's take Google's "Gmail" app available on Android.  This app is clearly one of the best implementations of Gmail on any mobile device, and arguable the best email app for mobile devices period.  Every app has some haters, so it is would be expected there are a few negative reviews as well.  To be fair, if you have the full list of reviews on your older Android Market still and take a look, you will indeed find some bad reviews and people complaining about features they want and giving a lower rating as a result.  However, the app still has an overall 4.5 star rating with nearly 113,000 reviews as of this writing.  It is also one of the top rated in it's category (number 3 in the "Communication" free category).  It has a very good ratio of positive star ratings to negative star ratings as well, which is all to be expected for a great app.  So what do the reviews say?  Well, that tells a very different story.  The first three reviews are always what is shown by default for each app without clicking "More" or "Read all reviews" (which does not let you read all reviews, more on that later), so let's start in.

The first review shown for Gmail as of this writing is 3 out of 5 stars by "Adam" on June 21, 2011 (nearly a month ago).  As that star rating would indicate, it is a mediocre review, but actually it is not a review at all.  It is instead instructions on how to fix a problem that he was having with the app to help other users who may have a similar problem.  Good on you "Adam" for trying to help your fellow man, but this is not a review that will help you decide if this is a quality app or not in nearly all cases, and it really can't be argued to be the most relevant review in that regard either.  Let's go on to the second review shown and see if that will help us out.

The second review for Gmail is a 1 star out of 5 by "mark" on Jun 4, 2011 (getting even older now).  He seems to have had a problem with an update released around that time and decided to ding the app with a poor rating in response.  This kind of thing happens quite often in the Android Market, and is a fair review, but does the app still have this issue?  Did anyone else around the same time complain of similar issues?  Sorry, there is absolutely no way to know here.  It is a 1 star negative review, with no context and no way of knowing if it is still a problem or if more recent reviews may have stated it as resolved.   Again, a useless and likely outdated review that is very negative for what as stated earlier is nearly universally accepted as a great app.  We move on to the third and final review shown on the main app page.

The third review for Gmail is also a 1 star out of 5 review, this time by "Erion" on March 28, 2011.  Yes, you read that correctly.  March 28th and today it is July 14th.  Much older and more dated, and also a negative review of a great app.  He is complaining of a missing feature to download attachments, which seems to be a reasonable complaint.  So, is that still a problem nearly 4 months later?  Who knows, it might be, it might not be.  There is again no context of other reviews, no recent reviews showing this and no revision history by date that one could even try and use to divine that understanding.  In fact, this could just be a user error and there may have been a response from another user just moments later saying so but you have no way of knowing that.

We will stop there, as things only get worse and you can view what Google says is all their reviews by clicking the "Read all reviews" for yourself.  The next review as shown today is from way back in October of 2010 believe it or not and also negative, and the trend continues on for a while.  Interestingly, Google Gmail gives itself 4 pages of 10 reviews each that you can look at for it's app, where as nearly every other app will max out at 20 (two pages of 10; like their own published Google Search app is limited to).

So without even using my own app as an example (feel free to take a look, you will see the same trends and some very old reviews), or the many others who are impacted, we see that this new "ranking" of reviews has a few critical issues.  I'll outline them here for clarity.

• Reviews Are Old & Not Tied to a Release
• The most glaring issue that will first hit you is that the reviews being shown are almost all very old reviews in the context of how many reviews this app is getting every single day.  All apps we looked at that are impacted by this problem had this very same issue.  Since these reviews are not tied to a release of the app in any way (say like Apple does), and have no other reviews around them from the same time, we have no context in which to decide if this is helpful or not.
• Reviews Are Heavily Negative
• This is a great app, with many great reviews.  I can't link to those here because the web version now only shows these old reviews, but if you have an older Market on your device you can easily go there and see for yourself.  How can an app with such high star ratings overall, and such quantity of great reviews have such an overwhelming amount of negative reviews being selected for showing in the market?
• "Read all reviews" Does Not Let You Read All Reviews
• The option to "Read all reviews" is misleading at best.  In fact, it just shows more of a filtered and ranked list that Google has chosen to show you using it's magical black box algorithm (Google seems to like this idea of ranking things using complex formulas).  There is no way to see all the current reviews, no list showing the most recent or tied to this version, and no way to rank up or down as helpful or not helpful those same reviews that have been filtered by Google.  So if you don't find the current list helpful and you "down vote" them as "Unhelpful", you will have a very hard time finding a positive one to "up vote" as "Helpful", and good luck finding a recent one at all.

In the old market these things were much less of an issue as all reviews were always shown from the dawn of time and sorted by most recent first.  You had a natural context from other reviewers around the same time.  If someone mentioned an update causing a problem for them, you could find others possibly later saying the problem was fixed, or responding to the comment with clarifications or help.  People could leave reviews with helpful hints, or responses to other reviewers who claimed to be having issues, but in reality possibly failed to follow instructions correctly (say for an app that needs root, or a widget that can't be moved to SD card, etc.)

Please Google, let's admit there is a problem here and fix it.  I realize some apps don't have this problem for whatever reason the secret black box ranking is choosing better reviews (or at least not yet), but for those where it exists it really needs to be fixed.  In fact, some apps that have very poor overall ratings, now are showing only positive reviews.  So this problem is cutting both ways.  Your changes here are hurting us, your developers and your customers, and now that you are rolling this out to devices world wide this problem is about to explode.

If you have been hit by this same issue, or you disagree and want to tell us why we are just a bunch of whiny developers, let us know in the comments.  We would love to hear from you.

Achievements Bug & Top 50 Single Player

Today is a short post, but an important one.  After releasing our new achievements system on Android, we found a bug in the points system for achievements when playing against the Novice computer player.  In that mode, you should have been limited to the number of points you can earn towards the Top 50 (should be limited to earning each achievement about 50 times).  Unfortunately, that was not happening correctly and as a result many who just played only the Novice computer a huge number of times were incorrectly placing in the Top 50.

We have now corrected this, but I'm quite sure this will upset some of you who will now find your not in the Top 50 and your total points are now lower then they were as well.  I sincerely apologize for that, as I know how frustrating that can feel and hope that you will understand why we had to fix this bug before things got too out of hand and the Top 50 became just a bunch of people who can beat a really simple Novice as it would make the list meaningless.

At the same time, we have cleaned up the Top 50 to remove all the "Anonymous" accounts as well, which should make the list much more realistic and give other more serious players a chance to get on the list.

Again, apologies to all, and a reminder as well that if you ever find anything in the game that does not look right, chances are that it is not right, and we would really appreciate hearing from you at support@FullerSystems.com

-- Josh

Release 1.7 - Some Love For Single Player Cribbage

After quite a few releases that had a heavy focus on the multiplayer side of the game in Cribbage Pro, although we always give some single player improvements too, we are really excited to finally release version 1.7 which is almost entirely focused on just the single player side of the game (coming first on Android; iOS users it is coming soon hopefully with your iPad support too).

So without further delay, here is a run down of these new features in our now traditional bullet list style.

• New Single Player Achievements System
• You can now earn over 80 different achievements while playing Cribbage Pro on single player, and each achievement you earn gives you more points towards appearing in the "Top 50" leader boards.  That's right, you can even lose a game but have an achievement earned which off-sets that loss and still move up.  We think it's more fair then just earning for a win.  You showed skill in earning the achievement (say a nice 24 point hand) and that skill should be represented in the rankings.
• Not all achievements are for showing your greatness, some of them just mark a milestone in your game play.  For instance, you get one for playing 100 games, then 1000 and then 2000+ at each level of difficulty.  You may also get a "19 Point Crib" achievement if you find yourself unlucky enough to have 0 points in your own crib. This is of course not a good thing, but just a common slang used in cribbage in general that we thought would be fun to add in to soften that harsh blow.
• Right now there are over 80, and we have ideas for even more, but would like your suggestions as well.  So email us what you think might make a good achievement in the game and we will see what we can do to add it in for a future release.
• Real World Rewards from Kiip
• OK, so achievements are great and all, but what if they were not just points you could earn to get to the leader board but included other real-world rewards for significant game achievements.  Once we heard of Kiip and how it worked out with the timing of our achievements release here, we got really excited as we think you all will too.  If you have been lucky enough to earn one already or gift it to a friend, you know what we are talking about. For those of you who have no idea what Kiip is and why we are excited to bring it to you, check out their video on earning real rewards in games you are already playing.
• "Top 50" with new "Recently Played" and "All Time"
• We heard you all on this one a few times, and we are glad to be able to finally start to do something about the Top 50 leader-board for single player.  One of the big glaring issues was that someone who played the game over a year ago and did really well may be sitting on that list, but only now playing the game just intermittently enough to keep their spot.  This first set of changes in this release are starting to address this.  With the new "Recently Played" list being the default, players who have not played the game in a while will drop off the list and the list can change each and every day that ticks past giving the newer players who just found us a chance to make the list too.  Of course, there is still the "All Time" list to try and get to as well.  Note that we changed this for single player and multiplayer Top 50.
• Some Graphical Tweaks
• We made the "Count" box bigger, bolder, styled it and generally made it easier to read and we also made a few other tweaks to the overall layout which I'm betting nobody notices very much, but they were bothering us so we fixed them.
• Improvements to Using Random.org
• On Android we have been using Random.org for our random seed data in shuffling/etc. in the game for a while, but it was not as efficient as it should be and it would run out too fast at times and not be able to get new good random data (not a good enough internet connection, etc.) and be forced to fall back to using the devices random source (which we have to say almost universally terrible).  So in this release we added a few new technical adjustments to that system and how we utilize it to make the cards you get (and the cards the computer is dealt) as random as we possibly can as often as we possibly can.

We have already had a few requests for a list of all the possible achievements you can earn in the game, and when.  Right now we will not be releasing the full list (remember, that's 80+ so it would be a long list), but in a future blog post we will list out some of our favorites and/or ones that we are getting questions.  So more to come there.

As always, if you have suggestions, comments or feedback of any kind please let us know in the comments here on the blog or (particularly if you want a quick response) by emailing support@fullersystems.com