I'm a bit excited about this post. It's a rare chance for us to share some real customer data for a large mobile service we run (call it Service X). I can't identify the customer at this stage, but hope to be able to in future.

We, and Service X, have been quite disturbed by the wastage is customers which has been an open secret in the mobile industry for some time now; MobHappy had an excellent post about it a while back. Basically: people are requesting content, being sent a WAP Push message linking to it, and then never collecting the content by managing to click on this link, go online, and download it.

When we first saw this, it rather scared us. We were pretty certain that it wasn't anything that we were doing, and thought that it was down to a combination of factors: handsets being sold without WAP settings installed, customers losing WAP push messages hidden deep in their phone menus ("service inbox" anyone?) or being intimidated by them - let's face it, anything called "Service message" which shows you a URL is a bit scary.

Service X did some follow-up research on this; they rang up a sample of customers who'd failed to download their content and established that some of them realised they didn't have the right settings installed. Most people told them "it just didn't work", which is a fair enough response for a Real Person from outside the mobile telecomms industry, but didn't help us definitively diagnose the issue.

We recently took Service X onto a large operator portal. Previously they'd been running an off-portal operation on their own shortcode, and between August and December 2005 they saw 46% customer wastage - nowhere near the 75% Russell observed for his ad campaign, but still worrying. In the first 8 months of 2006 this had decreased slightly to 41%, which we thought reflected the growing education of consumers and ability for them to download mobile content.

In going on-portal, we reasoned, we'd be putting Service X in front of a user-base which by definition had WAP settings on their phone (otherwise they'd not be able to access the portal in the first place) and might be more inclined to locate and click on those WAP Push messages. The initial on-portal activity saw us integrate with the operator by providing a WAP page linked from the portal showcasing Service X. We then actually delivered the content via a billed WAP push message; to get it up and running ASAP we had opted to do all billing via premium SMS instead of by integrating tightly into the operator. This resulted in significantly lower wastage of 33.7%, but was still higher than we expected to see.

So we changed approach and tried linking directly from the operator WAP page to the chunks of content, and using SMS for billing only. So customers click on the content they want to download from the WAP page there and then, and get it, instead of waiting for a WAP Push to turn up. Suddenly we saw wastage drop massively, down to 7.37%.

We still don't quite understand what this 7.37% involves: perhaps it's people double-clicking links (resulting in 2 purchase requests, where only the second actually gets downloaded), we need to spend more time on this. But my conclusions are:

1. We experienced a wastage of roughly 26% (33.7% - 7.37%) because of something intrinsically nasty about WAP Push: either customers don't like the messages, the messages get blocked, or lost. Even when handsets were set up and capable of getting online, something bad about WAP Push stopped customers getting their content. WAP Push is a problem.

2. We experienced a wastage of roughly 7.3% (41% - 33.7%) from customers who wanted to purchase mobile content, but couldn't get online to collect it... or from customers who didn't have supported handsets. (The operator portal pages only get shown to visitors who have a handset we can support)

3. Overall, wastage is dropping slowly (46% last year to 41% this year).
   

Anyhow, there you go: anyone got any other thoughts?

Comments

[Raddedas]

Very very interesting, thanks for concrete stats. Can you break down the 26% further, by handset type? I presume you managed to get a user agent out of them on the wap page which triggered the wap push?

7% failure to click on a link really is also very impressive. Maybe a few of them were in moving vehicles or had bad signals, and a few more had no space left on the phone for the content, but really 7%? The mind boggles...

[Tom Hume]

Interesting idea... for historical reasons (rather than anything else) we don't actually gather user agents from these pages in such a way that they can be matched up to individual purchases, but maybe we should look into it...

[gustaf]

Interesting!

So, would you say that it makes more sense sending plain SMS's with embedded URLs even though lots of handsets wont recognize them?

Let's say you dont have WAP-portal access and want to push a download sold from an online site, wap-push or SMS?

[Jordy Mont-Reynaud]

I don't know if I totally understand the improved (7%) use case... you say you "tried linking directly from the operator WAP page to the chunks of content, and using SMS for billing only" -- does that mean that they got billed by SMS after downloading the content? What were the concrete steps that the end user had to go through to get the content in this case? Your stats sound really interesting, I'm just trying to figure out how the actual user experience compares to the WAP Push case.

[Tom Hume]

Jordy

We used to have this process:

1. User visits WAP portal
2. User clicks link to buy puzzle
3. User sees "thank you page" on WAP portal
4. User receives premium rate WAP Push message, linking them to their purchased puzzle and billing them

We moved to this process:

1. User visits WAP portal
2. User clicks link to buy puzzle
3. User sees "thank you page" on WAP portal, and clicks on a link on this page to download their purchased puzzle
4. User receives premium rate SMS message billing them

The latter process gave us the 7% wastage.

[Laney]

Interesting stats indeed, but surely your revised approach leaves the content provider exposed to actually not redeeming any payment? Which is actual a double whammy, as suppliers will expect payment for all items delivered. Or am i missing something here? :)

[Tom Hume]

Laney: I think so. In both cases the customer is billed, but with the revised approach more customers are getting what they paid for: which can't be a bad thing, no?