Customer Surveys: Open-Ended vs. Multiple-Choice Questions

Customer Experience

As a business owner, you simply can’t overestimate the importance of what your customers think, feel, and say about your product or service. Most likely, you already know the key to discovering this invaluable information: A customer survey.

Customer surveys allow you to reach out to your customer base, probe them for their thoughts and opinions (in a friendly way, of course 😊), and filter their feedback for common trends and themes. The resulting insight can help you improve your product, make better business decisions, and convert more of your customers into vocal, enthusiastic advocates for your brand. Sounds good to us!

The trouble is, how do you decide on what kind of survey to use? With a seemingly endless variety of formats and question types, it’s difficult to know where to begin for the best results.

In this article, we’re eliminating some of the guesswork for you by giving you foundational advice to creating an effective customer survey, as well as a good old-fashioned compare-and-contrast between two of the most common survey question: open-ended vs. multiple-choice.

Your Starting Point: Net Promoter Score (NPS)

Before deciding how you want to write your customer survey, you need to decide what sort of information you want to collect. If you’re focusing on customer satisfaction, there are several metrics that you can gather. These include CSAT (customer satisfaction score), CES (customer effort score), and finally, NPS (net promoter score).

While CSAT focuses on measuring general rates of customer satisfaction, CES seeks to gather information about how challenging it is for customers to interact with your product. Both of these metrics can be valuable in their own right, but what you’ll really want to focus on is NPS.

Why?

NPS effectively captures a spectrum of customer satisfaction-related insights by asking your customers how likely they are to recommend you to a friend, relative, or coworker. In doing so, they let you know how happy they are and how usable your product is. Maybe most importantly, you also gain insight into your success in encouraging word-of-mouth marketing.

NPS is generally collected by asking, “How likely are you to recommend Product X to a friend, relative, or coworker?”

Survey-takers then have the opportunity to respond on a scale of 0-10. Customers who give you a 9 or 10 are promoters; customers who give you a 7 or 8 are passives, or neutral, and customers who give you a 6 or below are detractors.

In a perfect world, all your customers would be promoters, eagerly sharing the good news of your product with others. But that’s not usually the case. In all likelihood, you’ll find more of a combination of customer types. Your goal is to discover why promoters love you, and why detractors are less than satisfied. And then, you’ll want to turn that feedback into actionable advice.

The Follow-Up Question: Multiple-Choice or Open-Ended?

Your follow-up question is the key to turning insight into action.

If someone gives you an enthusiastic “10,” you can take a guess about why they love your product…but in reality, you need to hear from your audience to get an authentic answer. On the flip side, if a survey-taker gives you a “4,” you’ll most likely be desperate to understand the logic behind their less-than-thrilled answer.

In any case, you need to give them an opportunity to explain.

You can do so in one of two ways: a multiple-choice question, giving survey-takers a pre-set range of options to choose; or an open-ended question, providing them the chance to use their own words to explain their rating choice.

Multiple-Choice: Pre-set Answers for Easy Analysis

A multiple-choice question for collecting NPS feedback for a meal delivery service might read as follows:

What can we improve on?

a) Customer service
b) Quality of ingredients
c) Package design
d) Recipe instructions

The core advantage of asking a multiple-choice question is that you generate a highly controllable outcome: You receive specific information with very little need for analysis. It’s fast and easy, and can give you some actionable insight.

The disadvantage, however, of asking a multiple-choice question is that you don’t really learn why your customers rated you the way that they did. For example, by asking your customer base the question above, you learn that a number of them don’t like your package design, and very few have complaints about customer service. But did you really learn about their true concerns? You don’t understand your customer’s true sentiment.

Let’s imagine that your detractors’ primary complaints are a slow delivery time or unclear messaging. You haven’t given them an opportunity to share that information. Instead, you’ve made assumptions about the logic behind their rating.

Not only that, but you haven’t created a space to allow your customer base to make suggestions or recommendations. Now you’re missing out on a relationship-building opportunity that shows you value their opinion.

Open-Ended: Detailed, Honest Feedback

On the other hand, an open-ended question for following up on an NPS rating question might simply read:

Please explain your answer.

Pretty simple, right?

Giving your customers free reign to give you feedback produces some powerful insight. You may discover common threads and trends in your feedback analysis that surprises you, and help you to effectively turn more of your passives into promoters–and your promoters into more active advocates for your brand.

For example, you may learn that many of your customers are unhappy with a particular ingredient that’s present in many of your meal delivery boxes. This ingredient could be killing off your retention rate, and turning your satisfied customers sour.

But without an open-ended question, you would have remained blindly ignorant of your meal delivery faux pas and left to wonder about dropping subscription rates.

Not only that, but an open-ended question gives your customers space to voice their complaints, and shows that you want to listen.

This can help you to develop effective strategies for improving customer loyalty and advocacy.

The primary challenge of using an open-ended question is, of course, analysis.

Open-ended feedback leaves you with a vast amount of information to read, organize, filter, and most importantly, understand. The larger your customer base, the more amplified your challenge. Take into consideration the fact that many of your customers may misspell certain words and use different phrasing for similar sentiments, and you have hours of data sorting on your hands.

Don’t Sweat It: There’s a Solution

Open-ended feedback produces the not-insignificant challenge of analysis. But relying on a multiple-choice question to gain authentic insight into your customer base leaves a lot to be desired…and may ultimately result in wasted time, money, and resources.

The solution looks to augmented intelligence to help you effectively categorize, filter, and analyze open-ended feedback. Caplena helps you deliver NPS surveys and analyze the feedback, getting you detailed, accurate insight without the massive challenge of trying to perform the analysis yourself. Instead, artificial intelligence does the heavy lifting for you, by using NLP (natural language processing) to evaluate language. At the same time, we help create the codebooks ourselves, using human creativity and insight to establish an effective system of categorization.