May 12 2010
Measuring Design Changes
A week ago, I replaced the Flickr photos in my blog’s header with iPhone OS apps: Retrospect Touch and Dollar Clock. Since I hadn’t updated Flickr since Christmas, the iPhone/iPad apps seemed more appropriate, and I was curious to see if the switch affected the download numbers. Here’s the before and after:
Today, I checked the numbers in iTunes Connect for Dollar Clock. At first, I only looked at the total downloads, in the left graph, where the hash mark notes the header change. Clearly, the header change increased the downloads. Then I looked at the purchases and updates separately, in the right graph, and the pattern was less clear. During the week after the header change (the bar after the hash mark), more of the downloads were updates, not purchases.
During the same week I changed the header, Dollar Clock 2.0 was posted to the App Store, and many people downloaded the update that week. The total downloads indicates my header change had a significant effect, but the breakdown reveals confounding variables. Perhaps if I had more variables, I would find that the header change had absolutely no effect.
Or maybe I just wanted an excuse to use Tufte’s small multiples. :)
