I’ve been digging into eventing data for some time now on the Coursera App using Amplitude to gauge usage, follow user flows and investigate potential production issues. With that experience as a baseline, I’m pretty happy with Apple’s App Analytics. Combining the two systems (internal eventing + Apple’s analytics) I’ve finally been able to create a complete funnel from referrals and app views to app launch and feature usage. However, there are a few peculiar things about the system which tries so hard to be beautiful, but sometimes ends up convoluted.
The platform has all the base functionality I want as an engineer wanting an analytic overview: App views, installations and sessions over a custom time period with the ability to segment the population by country, platform, etc. A few things I liked beyond this are the ability to compare multiple metrics on a single graph such as number of App Store Views over Installations. You can find this feature on the Metrics tab under the third section of the left sidebar with title ‘COMPARE TO’.
Another nice surprise was the ability to view website referrals to the app’s page. Through these data I found the App Store still contributed the lion-share of our referrals which made me much more appreciative of it in spite of all the recent verbal frustration around discovery. It was also a great insight to see which websites outside of the App Store provided the most referrals to gauge marking and referral partner effectiveness.
First off, its entirely unclear what an App Unit is and how it differs from an Installation. Reading the iTunes Connect FAQ for App Analytics I found that App Units count downloads, excluding re-downloads, and Installations count installs, including re-installs. Given that downloads and installations are essentially the same concept (I don’t understand the difference) I would expect Installation metrics to always be higher than App Units, because they include re-installations however the opposite is true (App Units are always higher). As a result, I don’t know how to use App Units.
* Update: I’ve had a thread open with Apple since writing this post. They have confirmed that “Metrics under the ‘Usage’ section fall under ‘Engagement Data’ since this data requires customers to opt-in to see usage.” App Units represent the number of first-time App purchases and as a purchase, don’t fall under ‘Engagement Data’. This can be semi-confirmed with: App Units * Opt-In % =< Installations.
Apple’s attempt at displaying retention really looks like they made a choice of beauty over usability here. I’m not quite sure what extra insights I’ll gain from this fine grained retention matrix, but a matrix of percentages isn’t a great way to recognize patterns. The number matrix also takes up so much space it was only after several times on the page that I noticed the much more useful graph of average retention for each N days after download and the filter buttons. I would have much preferred the ability to compare retention over time across filters to answer questions like, “Do iPad users retain better than iPhone users?” without having to switch between graphs and do manual calculations.
The beta label worries me about how much I can trust this data. I’m definitely not a data scientist, but I worry how representative the data is of our user base since it only makes up around 18% of our users. I’d love to see some data that indicates whether there are regional or socioeconomic preferences towards opting in vs. opting out. The main reason I’m a little wary of the numbers here is that I’m seeing a single country with a much higher percentage of session metrics compared to our Amplitude ApplicationDidLaunch event. Additionally, I’m seeing much higher session numbers in Apple’s Analytics when compared to this same amplitude event.
If I was to put something on my wish list for the analytics it would be more information around referrals. Features like tracking them over time on a graph and segmenting by territory, platform, etc. Also, more information about where the App Store referrals are coming from (search terms / browse flows) and how we can improve the discovery of our app.