CheckMarket offers fifteen different types of answers when you draw up your questionnaire. A frequent application in this list is in particular the use of open questions. This gives the respondent full freedom, albeit limited to a certain number of characters, to react to a given question or position.

The results in an open text field, as the sole response possibility or as “Other, please specify” in a multiple-choice question, are not so simple to analyse, as a multiple choice question, however.

Open questions or not?

CheckMarket advises its clients to be sparing with open questions. The rule of thumb in designing surveys is to ask only what is really important for your survey. Avoid questions that are less important about things that you would like to know, “because we are asking questions anyway.” This applies both to questions with an imposed answer and to questions with an open answer.

• Opt only for an open answer possibility if a list of imposed answers is too restrictive in your view.
• Do not opt for an open answer possibility because you absolutely wish to introduce variation to your survey.
• Do not opt for an open answer possibility as the easiest way to find answer possibilities that you cannot think up yourself .
• If you have enough time to read attentively and to analyse open answers, then open questions can certainly be useful.
• Because open questions are often skipped and thus usually answered only by respondents who really want to give their opinion on the question posed, these answers can be very useful.
• Open questions can be a valuable instrument. For instance, after a product or service has been purchased, you ask clients to take part in your satisfaction survey. In that survey, you ask your clients whether you may publish their answers (anonymously or by name). You can then post their opinions or judgements on your website or in a newsletter, etc.

Apply certain guidelines yourself when you process open answers. It can prove very useful to give a certain code or mark to the different answers. Properly identified guidelines are needed to be sure that related results get the code you have attributed.

How to analyse open questions

Open questions can be analysed in different ways, depending on the time you want to devote and the precision with which you wish to proceed. You can opt to select answers at your own discretion. A few well chosen citations from open answers can reveal important trends in your results. This manner of processing is rapid and efficient, but often very subjective.

Another approach is process answers by grouping them. Answers often have the same content worded different. Putting similar open answers together can simplify substantially the counting of the results and thus reflect striking trends faster. To group similar answers, you can first rely on the most recurrent words in the “answer bubble” at the top of the page right, with “textual answers.” When you click on a word in this answer bubble, a filter is applied automatically, and only answers that contain this word are shown. In addition, you can use our search function to see how frequently certain words are used in open answers. By using this search function, you can filter the answers in which your search word occurs.

Open answers can be grouped in a structured manner by attributing codes. You devise categories for the answers and also a category for detached answers, the “remainder.”
Consider the cases in which such a coding system can be applied:

• “I have no idea what kind of answers I will get and which will be most applicable to me.”
  Coding often provides an overview in the broad stream of answers you get from your respondents.
• “I would like to know how the answers tally with previous observations.”
  You can place the answers in pre-defined categories. You can work with colour codes corresponding to a caption (see example below).
• “I select the reactions that meet our standards.”
  You can thus group answers that meet preset criteria.

A simple example by means of colour codes demonstrates how this method works in practice. By attributing colour codes, the analyst can see at a glance which answers recur the most for a given question. A large number of answers can be counted in Word file with a format search command. To this end, click “Edit” in your Word file, and then ‘Find.” In the dropdown menu Format, choose Font. Finally, in ‘Font colour’ you select the colour by which you want to filter.

Codes can also be useful when you want to analyse the results at a later point on the basis of other criteria. Link the necessary instructions with the attribution of a certain code, both for yourself and the principal account holder with CheckMarket as well as for your aliases that are involved in the processing of the survey results. Good instructions are a must, certainly for comparable answers, but with each, a specific aspect that can influence the survey.
Doubt situations often arise that require proper delineation:

• Incomplete or meaningless answers
• Ambiguous answers
• The answer is usable, but not compatible with other answers
• The answer is usable, but can be placed under more than one code

A qualitative survey in any event requires a manual effort to read over all open answers and to place them in the right category. You can make notes in each category group to familiarise others with your classification.

For a quantitative survey, you can naturally also use the practical search function and answer bubble offered by CheckMarket through the on-line reporting of your answers. In such a case, you will naturally get only the results that meet the specific search term(s).