In this new blog article we are looking into the use of socio-demographics in surveys. More specifically, we will try to answer three questions. First of all, should you ask about them and why (not)? Second, if you decide to ask about them, which ones should you ask? Third, if you ask about them, where do you put these questions in your survey?

However, first things first. What are socio-demographics? Socio-demographics are nothing more than characteristics of a population. Generally, characteristics such as age, gender, ethnicity, education level, income, type of client, years of experience, location, etc. are being considered as socio-demographics and are being asked in all kinds of surveys.

1) Should you ask about socio-demographics and why (not)?

Yes, you should

There are in fact various reasons to do so. First of all, in some cases it might be imperative to know who is filling in your survey. For instance, if your survey targets a specific audience, it allows you to determine whether you are actually reaching your target audience and whether or not you are gathering the information you are effectively seeking. Furthermore, if you aim for a representative sample of a population, knowing the distribution of the demographic characteristics of your respondents will help you in determining how close the sample replicates the population.

Second, if sample sizes are large enough, it enables you to differentiate between different sub-groups. This segmentation might offer you insights that you would have missed by only looking at the aggregate data. For instance, you might conclude that your employees are generally satisfied with their career opportunities in your organisation. Nevertheless, the aggregate data might hide the fact that the employees of the IT department are not at all satisfied with their career opportunities.

No, you shouldn’t

There are also various reasons not to ask about socio-demographics. First of all, you need a substantial sample to draw any statistically meaningful conclusions. If you have a sample at your disposal, differentiating between certain socio-demographic sub-groups might trim down your sample to a collection of less relevant sample(s). These samples could then be too small to draw any meaningful conclusions from. For instance, if you find that a particular product is very popular with 15 to 25 years olds but you have only 50 respondents in that particular age category, the sub-sample is simply too small to draw any conclusions.    

Second, adding socio-demographic questions lengthens your survey significantly. As many researchers have already shown, the dropout rate of a survey correlates positively with the length of the survey. The longer a survey, the higher the dropout rate. In short, adding (too many) socio-demographic questions might make it more difficult to gather a substantial sample size.

Especially if, thirdly, respondents might become concerned by having to answer a large number of (identifying) demographic questions. They might feel that these questions compromise the anonymity of the survey or experience these questions as an invasion of their privacy.

So, what do I do?

So, to summarize, it is imperative that you strike the right balance. We believe that you should ask your respondents about their socio-demographic profile. However, asking about socio-demographic characteristics is not without its pitfalls. That is why we advise you to determine the most important socio-demographics to your research in the start-up phase of the survey and only ask about them. Furthermore, you should use them wisely while analysing the survey results.

2) What are the right demographic questions?

Asking the right demographic questions will allow you to discover meaningful and actionable insights to assist you in making better business decisions.

Below we discuss two commonly asked socio-demographics:

1. Age:

It has been shown in various scientific disciplines that opinions on a vast number of topics differ between different age groups. When asking respondents about their age, ideally, you should ask them to fill in their age (expressed as a number of years) or their birthday. As such you have (or you can calculate) for each respondent a value (in number of years) which you can use as a continuous variable in the analysis. In other words, a variable which you can express differences in magnitude with. For instance, “respondent A is forty years old and is twice as old as respondent B”. Furthermore, it allows you to ‘recode’ the age data infinitely in many different age categories.

Alternatively, as age might be a sensitive topic for some people, you could use age categories. Depending on how specific you would like to be, you can use more/less narrow/wide categories. However, the use of non-overlapping equal categories is recommended. As such, categories are equally wide and each data point can only fall in one category.

At CheckMarket we generally use the following categorisation: ‘18-24’, ‘25-34’, ‘35-44’, etc.

2. Education:

More often than not there are clear differences in opinion between respondents with a different educational level. Moreover, educational level – generally asked as ‘the highest level of education completed’ – is also quite often used as a proxy for income.

In certain countries – such as Belgium – respondents are not too keen on ‘openly’ discussing their income, even if it is an anonymous survey. Therefore, to keep as many respondents as possible on board, researchers often opt for excluding questions about the income. It is then argued that the educational level gives an impression of the respondent’s income, or more generally its socio-economic status (SES). When polling respondents about their educational level, it is important to keep in mind that the school system is not the same in each country. In practice, it is even hardly the same in two countries! So, depending on which countr(y)(ies) you are surveying in and how specific you would like to be, you can use a more/less specific categorisation.

At CheckMarket we usually use the following general categorisation:

• None
• High/Secondary school
• Bachelor
• Master
• PhD

Besides these two socio-demographics also gender, marital status, number of children, employment status (with categories ‘student’, ‘unemployed’, ‘retired’, etc.), etc. are often being asked.

3) Where to put socio-demographic questions in the survey?

As we have established that you should only ask the socio-demographics that are the most important to your research, does it matter where you put them in your survey? Yes, it does. Conventional wisdom was to put these questions at the end of the survey. An often cited reason for doing so is, amongst others, that respondents are hesitant to answer – or become annoyed with answering – demographic questions and, as a result, might drop out if you put these questions at the beginning of the survey.

Another argument says that it might make a difference in respondents’ answers if they know that their socio-demographics (e.g. race, gender, etc.) are being taken into account. This refers to the so-called ‘stereotype threat’. The ‘stereotype threat’ is defined as the “experience of anxiety or concern in a situation where a person has the potential to confirm a negative stereotype about their social group”[1]. In other words, if a respondent belongs to a negatively stereotyped social group they might perform less than at their maximum level, if they know their social group is taken into consideration. So, if you put the socio-demographic questions at the end of the survey, the reasoning goes, this will not affect the results.

However, recent research shows that placing socio-demographic questions at the beginning of the survey in fact increases the response rate to these questions. This should not come as a huge surprise as it seems rather intuitive that respondents are more likely to lose interest after a while in the survey, drop out and, as a consequence, do not answer the socio-demographic questions at the end of the survey. If these questions were placed at the beginning of the survey, respondents would already have answered them. As a result, if demographics play an important part in your survey (which they do in most research projects) or if you want to branch on particular demographics, these socio-demographic questions should be put up front.

That is why at CheckMarket we always recommend to put the socio-demographic questions at the beginning of the survey.

Related articles:

• Sample size calculator
• How to calculate the optimal sample size?
• How to estimate your population and survey sample size?