We wrote about it in 2010 and we still stand by this: At CheckMarket we advise our clients to be sparing with open-ended questions. Nevertheless, this does not mean you can’t use open questions in your survey. In this article we will help you deal with open-ended questions.
What are open-ended questions?
Open-ended questions are questions where you give the respondent the possibility to give their own answer instead of providing them with options to choose from.
The benefit of using open-ended questions is, that you can more specifically capture how a respondent is feeling about a certain subject or they can give you more insights on why they gave a specific answer or score. It gives the opportunity for a respondent to provide more context to an answer and as an organization, you can really focus on what areas specifically need improvement in your business.
Additionally, if you have the consent of your respondents, open-ended questions can serve as great testimonials that you can put on your website or use on your social media to increase the social proof of your business.
How to use open questions? Some examples
There are different scenarios where it is very useful to use open-ended questions when for example a list of imposed answers is too restrictive in your view or as we mentioned before, you want to determine the why behind a certain score.
One of the most common examples where we definitely advise everyone to use open-ended questions is with Net Promoter Score surveys. One of the pitfalls of the NPS is that with only a score as an answer, you don’t know the underlying reasons behind your NPS score. And without knowing the underlying reasons, you can’t make the appropriate improvements to increase your NPS in the future.
Additionally, you can also use text answers that have been given to determine if you show a certain question or not, using branching. For example if you conduct NPS research for your hotel and someone uses the word breakfast as an open answer, you can only show an additional question to these respondents to get more details about what they didn’t like about the food in particular. This keeps your survey short and relevant by only showing questions to respondents that have something to say about a specific topic.
With a customer satisfaction survey, you can ask respondents to rate certain features of your business and use additional open-ended questions to ask for specific feedback if the rating is below a certain score or you can end the survey with asking if they have any more suggestions or recommendations to improve your service or product. This will encourage customer empowerment because in general, people like giving their opinion.
Another example is with a (work from home) pulse survey. A great first question is to ask your employees to rate how they are feeling today. Depending on this score, you can use an open-ended question to follow-up on why they might not feel well and what can be done to improve this rating.
Make sure to adjust the phrasing of your open question to the result of the question it relates to. It makes no sense to ask someone ‘Why are you feeling good today?’ when this person only gives a one star rating. You can use page display logic to show different follow-up questions based on the score of the source question, or you can use CheckMarket Scripting Language (CSL) to adapt the formulation of your open question.
Another tip: don’t be too general, but specify what you really want to know. This will result in more valid answers. E.g. the question ‘What needs to be our top priority as an employer to improve your feeling?’ will give more useful results compared to the question ‘Why are you feeling this way?’.
How to analyze open-ended questions?
One of the reasons people avoid using open-ended questions is because they take much more time to analyze. Word clouds can give you a first impression of the main categories, but at CheckMarket we go a step further. Our tool includes a text analysis feature that will make it a lot easier and faster to analyze & report the open answers. With tags, you can group certain answers efficiently together and create clear charts to show how the categories stack up. This makes reporting open questions easy and understandable for everyone.
Additionally, it is also possible to apply sentiment to your keywords. With sentiments you can discover the key indicators of satisfaction & dissatisfaction. For each keyword, a chart shows the percentages of positive, neutral and negative comments.
You can also use the automatic tagging to categorize open-ended questions as they come in. This is especially useful for long running surveys. Auto-tagging is a process of automatically applying tags to open answers based on (a combination of) keywords, in near real-time. Auto-tagging can do the heavy lifting for you, so you can spend more time turning those insights into action.
The Net Sentiment Score
Now that we have setup auto-tagging to code and count the sentiment for the different categories, you might be wondering how can you best report and track them over time? We willl simplify this by using the Net Sentiment Score (NSS). The NSS is the percent of positive sentiments minus the percent of negative sentiments.
% positive – % negative = Net Sentiment Score
Now that we have a single number, it is easy to plot it on a timeline and spot trends. This makes it possible to react quickly and adequately when you detect a sudden change in NSS for one of your keywords. For instance, a sudden drop in NSS for ‘Quality’ is an indicator you might have an issue with your product or service.
Conclusion
Using open-ended questions can help you better understand your respondents. They can give you more insights and a clear guidance in which action you need to take. Nevertheless, we believe that the power of open-ended questions lays with how easily you can analyze them. The more complicated and time-consuming the process, the less likely you will be able to capture the details and get the most out of your responses. A survey software with text analysis and auto-tagging included, like CheckMarket, will help you make the most out of your open answers. The less time you have to spend on analyzing, the more time you can spend on taking action.
Can’t wait to find out how easy it is to analyze open-ended questions? Register today & test it yourself.
Leave a Reply