Apple's big mistake in its online surveys
Posted: Tue Jan 07, 2025 5:26 am
About two years ago Apple launched the iPhone 5c, a low-end mobile device that would serve to take advantage of its competitors in the price of its devices. Some time later, like any good global company, it decided to do online surveys to receive feedback from its customers. However, they did not do so well because they did not design the questions correctly.
In this post we will specifically analyze a question from the online surveys received by those who purchased an iPhone 5c two years ago.
Analyzing Apple's online surveys:
Many times consumers don't know why they made a particular purchase. First and most importantly, this question assumes that respondents are aware of the reasons why they made a purchase. In reality, any purchase is as much an emotional decision as a rational one. The reasons we think we made a decision are often not the reasons we made it.
The wording is not designed from the consumer’s point of view. The phrases “iOS 7,” “A6 chip,” “1080p HD video recording,” and “LTE wireless” are not common things you would hear in casual conversation. A good survey researcher speaks the consumer’s language, not the jargon of the engineering team. Consumers are interested in benefits, not features: they want to be able to use new apps (the iOS 7 benefit), they want fast response time (the A6 chip benefit), high-resolution video (1080p recording), etc. “I considered buying the iPhone 5c last month but didn’t because it didn’t feel very durable” is the kind of language consumers would use to think about their reasons. Apple should have run a test survey asking 100 consumers the open-ended question, “Why did you buy the iPhone 5c instead of another phone?” and then developed a list of the most common reasons in the words consumers actually used.
Matrix questions produce less accurate results. A matrix question shows less variation between items, since respondents think less about each row than when an item is in a stand-alone question. Also, respondents hate macedonia phone number them, so presenting them to your own consumers isn't the best. Because of the low variation between the results of each item, whenever a client has me use a matrix like this in their online surveys, I try to follow up with, "What was the most important feature or attribute to you in making your purchasing decision?"
The 5-point bi-polar scale is not appropriate. The most reliable bi-polar scale (a scale with opposites, in this case, importance and NOT importance) has 7 fully labeled points, but bipolar scales are cognitively more difficult for respondents. A 5-point single-polar scale would be more reliable: “Not at all important, somewhat important, important, very important, extremely important.”
In this post we will specifically analyze a question from the online surveys received by those who purchased an iPhone 5c two years ago.
Analyzing Apple's online surveys:
Many times consumers don't know why they made a particular purchase. First and most importantly, this question assumes that respondents are aware of the reasons why they made a purchase. In reality, any purchase is as much an emotional decision as a rational one. The reasons we think we made a decision are often not the reasons we made it.
The wording is not designed from the consumer’s point of view. The phrases “iOS 7,” “A6 chip,” “1080p HD video recording,” and “LTE wireless” are not common things you would hear in casual conversation. A good survey researcher speaks the consumer’s language, not the jargon of the engineering team. Consumers are interested in benefits, not features: they want to be able to use new apps (the iOS 7 benefit), they want fast response time (the A6 chip benefit), high-resolution video (1080p recording), etc. “I considered buying the iPhone 5c last month but didn’t because it didn’t feel very durable” is the kind of language consumers would use to think about their reasons. Apple should have run a test survey asking 100 consumers the open-ended question, “Why did you buy the iPhone 5c instead of another phone?” and then developed a list of the most common reasons in the words consumers actually used.
Matrix questions produce less accurate results. A matrix question shows less variation between items, since respondents think less about each row than when an item is in a stand-alone question. Also, respondents hate macedonia phone number them, so presenting them to your own consumers isn't the best. Because of the low variation between the results of each item, whenever a client has me use a matrix like this in their online surveys, I try to follow up with, "What was the most important feature or attribute to you in making your purchasing decision?"
The 5-point bi-polar scale is not appropriate. The most reliable bi-polar scale (a scale with opposites, in this case, importance and NOT importance) has 7 fully labeled points, but bipolar scales are cognitively more difficult for respondents. A 5-point single-polar scale would be more reliable: “Not at all important, somewhat important, important, very important, extremely important.”