Goodsol Solitaire Blog: Our Descent into App Store Hell

On September 15th Apple released iOS 9, the new version of the iPad/iPhone operating system. Later that day, we started to get reports that our iPad games were crashing on iOS 9.

This was odd and it was working fine on my iPads. It is common for apps to break when Apple releases a new iOS version. This should not happen, but Apple simply doesn't care about backwards compatibility - it is simply not a priority for them. If anything gets broken, they just expect the developer to fix it. This adds a huge burden to app developers on the iOS platform, but that's a topic for another blog post.

In the past we've been pretty lucky, we've never had a major issue with our apps and new OS versions before. Anyway, it quickly became clear that this time was different.

After collecting some crash reports from users who had the problem (Apple's own crash reporting system turned out to be useless), a pattern emerged: all the of the crashes were happening on 4th generation iPad models. We managed to get access to an iPad of this model and was able to duplicate the crash.

All of this took a few days and then within a couple of days we'd figured out the problem - a line of code that should be fine was crashing the 4th generation iPads and the 4th generation iPads only. What made these iPads different, we didn't know. But it didn't matter, because it was a simple thing to just change the line of code and stop the crashes.

So all we had to do was change the one line of code, rebuild the apps, resubmit, and wait a week for Apple to approve them (because it takes Apple a week on average to review submitted apps). A pain in the butt to be sure, but pretty routine.

So all 5 of our apps got the fix, got submitted, and we wait.

Then the first app got reviewed and rejected. The rejection was bizarre - it said the app wouldn't load on either an iPad running iOS 9 or an iPhone. The app is iPad only, so why would they even be testing it on an iPhone? And the app loaded and ran fine on every iPad we tested it on (which by this point had grown to quite a few as we had added more iPads to our testing). As the week went on, the other 4 apps got rejected for the same reason.

We asked Apple why they were testing an iPad only app on iPhones, but we got no reply to that question. Our first thought is that something had gone wrong in XCode (the development platform for iOS apps). So first we tried rebuilding in a different version of XCode. No luck. In all, we built on 3 different versions of XCode and made 10 submission attempts, all getting the same result (with just enough difference in the results to make everything confusing - sometimes the app reviewer would say it froze on startup, sometimes the splash screen would show), but always with the bizarre statement that it was tested on iPhones as well as iPads. After the first rejections I found out there is a way to get apps reviewed faster, I applied for that and it got approved, so after the first week wait, the next submissions got reviewed within a day or so.

We tried everything we could think of, nothing worked. We tried making a Tech Support Incident with Apple to try to get someone not the app reviewer to look at the problem, but we had no luck there. At this point, we were getting into all kinds of details about the testing process, about looking at build UUIDs to make sure they matched what Apple was testing, possibilities of XCode issues with plist files, all sorts of stuff that got us exactly nowhere.

Throughout it all, it just seemed like this iPhone thing was the key to it all, unless it was just boilerplate text that a lazy reviewer put in (most of the app rejection text was simply boilerplate text, so it was hard to tell if they were really trying to test on iPhones or if it was just part of their form letter).

Finally, after more than a month (by now it was October 24th), after the latest rejection, we asked again, why are you saying you are testing an iPad only app on iPhones? Why is a one line of code fix causing so much trouble? Perhaps I worded it differently this time, or maybe it went to a different person at Apple who actually knew something.

Anyway, this time, suddenly, the reply was that they looked at it again and there is no problem, it is being put into review again, best regards from Apple. And it went into review and was immediately approved.

So after more than a month of late nights, endless testing and submissions, we have no idea why the approval was delayed for more than a month. We were just caught in some sort of Apple review trap, and somehow got free. The build that got approved was essentially the same as the first one submitted over a month earlier.

Best regards from Apple.

The full as-it-happened saga is detailed here.