Surveys are standardised instruments used to collect information from a group of people, also called respondents. The purpose of a survey is to obtain structured data on opinions, attitudes, behaviours, or demographic information. Surveys can be conducted in different forms. You can think of interviews, written questionnaires, or online surveys.

Surveys are often used in different fields, including scientific research, marketing research, and UX research. Surveys collect data that is consequently used to identify trends and patterns.

What are questionnaires?

A questionnaire is a specific form of a survey. It is an instrument consisting of a series of questions presented to the respondent to gather information. Questionnaires can be either standardised or non-standardised. They can contain open questions where respondents are free to formulate their answers, or closed questions where the choices are predetermined.

The difference between surveys and questionnaires

The difference between surveys and questionnaires lies mainly in the broader use of the term "survey". A survey can consist of several methods, including interviews and observations, while a questionnaire refers specifically to an instrument with a set of questions. Key differences are:

Scope

Survey: A broader term that encompasses various methods of data collection.
Questionnaire: A specific instrument with a set of questions

Possible methods

Survey: May include a variety of methods, such as interviews, observations, and questionnaires.
Questionnaire: Specifically aimed at asking respondents questions.

Flexibility

Survey: Flexible in terms of methods and goals used.
Questionnaire: More specific and structured.

In short, the terms are often used interchangeably. Even though there is a subtle difference between the two, in most cases (also this article) you can assume that when using both terms, reference is made to questionnaires.

Advantages and Disadvantages of Surveys

Advantages of Online Surveys

Efficiency. Surveys allow researchers to quickly collect data from a large group of respondents. This makes it an efficient method for obtaining quantitative information.
Standardized. By using standardized questions, the same questions can be asked to all respondents. This ensures consistency and comparability of data.
Objectivity. In online surveys, there is no direct interaction between interviewer and respondent, which promotes objectivity and reduces the likelihood of being influenced.
Cost-effective. Online surveys reduce costs compared to traditional methods such as face-to-face interviews or phone surveys.
Anonymity. Respondents can often respond anonymously, which can lead to more honest answers, especially on sensitive topics.
Range. By using online platforms, surveys can reach a wide audience, regardless of geographical location.

Disadvantages of online surveys

Limited insights. Surveys are limited to the questions that are included. They offer less in-depth understanding compared to interviews and observations.
Bias. The wording of questions can lead to a bias. Respondents can be influenced by the way questions are asked, which can affect the accuracy of the answers.
Low response rate. Not all selected respondents respond to surveys, which can lead to biased sampling and results.
Limited context. Surveys often lack the context behind the answers. The "why" behind certain opinions can remain unclear.
Less suitable for complex matters. For complex topics, surveys may be inadequate, as respondents may struggle to express their thoughts at length.
Technological barriers. With online surveys, technological barriers, such as lack of access to the internet, can be challenging.
Over- or under-reporting. Respondents may tend to over- or under-report, depending on the topic and the social desirability of the answers.

It is important to consider the disadvantages mentioned above. To overcome them, it is advisable to use various research techniques. This includes conducting interviews, focus groups, and usability research to enrich the results.

When do you send out surveys or questionnaires?

Questionnaires are a valuable tool in both UX research and market research. Here are some specific situations in which questionnaires are conducted in these contexts.

In UX research

Questionnaires and surveys are an important part of UX research.

Before and after user research. To spot trends and patterns in the market, it can be valuable to conduct pre- or post-launch surveys.
Usability testing. After conducting usability testing to gain in-depth insight into specific aspects of the user experience, as well as gather additional feedback.
Prototype testing. During the prototype development phase, to gather feedback on the design, usability, and overall user experience.
Competitor tests and UX benchmarks. To collect standardized data on how users experience the various (digital) products.
At fixed times. Regularly throughout a product's lifecycle, to continue to monitor the user experience and adjust as needed. By combining questionnaires with a baseline and follow-up measurement, progress can be tracked.
With new features. When introducing new features or updates, to measure user responses and quickly identify any issues.

In market research

Surveys and questionnaires are also used in market research.

Before product launch. Even before a new product is developed or brought to market, it can be of value to conduct surveys. Providing insight into the perceptions, expectations, and needs of the target group helps to develop a matching product.
Customer satisfaction survey. For performing baseline and follow-up measurements in customer satisfaction surveys. In this way, progress over a certain period can be measured.
In pricing strategies. In determining the right pricing strategy, by measuring customers' perceptions of price and value.
Competitor analysis. As part of competitor analysis to understand how a product or brand is positioned against competitors.
Brand evaluation. Periodically, to measure the perception of the brand among the target audience and identify any changes in brand preference.
In advertising campaigns. After running marketing campaigns, to measure the effectiveness of advertising and understand how customers respond to marketing messages.
Product improvement. When considering product enhancements or new features to understand what customers desire and which features are most valuable.

In conclusion, surveys play an important role in both UX and market research. Although the moment at which these are deployed depends on the product and the available budget, it is always advisable to deploy them to a large group. Subsequently, the results can be interpreted using other research methods, as explained below.

Combining surveys with other research methods

Questionnaires can be effectively combined with several other research methods to gain a more comprehensive understanding of the research area. Here are some common combinations and the benefits they offer:

In-depth interviews

More in-depth insight. Interviews provide an opportunity to delve deeper into respondents' answers. They help in understanding the reasons behind specific responses and provide context.
Clarification. If there are any ambiguities or inconsistencies in survey results, interviews can be used to clarify them.

Focus groups

Diverse perspectives. Focus groups bring different opinions and perspectives to light. They can help identify broader themes that may be overlooked in survey results.
Interaction between participants. Interaction between participants can give rise to new ideas, which can be valuable for refining survey questions.

Usability testing

Insight into bottlenecks. Usability testing can assess the user-friendliness of a product, with surveys providing additional feedback on overall user satisfaction and perceptions.
Deeper understanding of user experience. Surveys can provide quantitative data on the user experience, while usability testing can provide in-depth insights into how users interact with the product.

Recruiting respondents to take surveys

Recruiting respondents to fill out surveys can be a challenging task. You can use your own network, or 'buy respondents' via a research platform such as User Sense. Want to know more? Then take a look at our recruitment service.

Of course, when recruiting respondents, it is important to take into account the final sample and the population about which you want to make a statement. To ensure that you recruit respondents that align with your goals, you can use both random and non-random sampling – or a combination of the two.

Sampling methods

The non-random sample

A non-random sample is a sample in which researchers specifically select certain participants based on certain characteristics or criteria. This could be to ensure that the sample is representative of a specific subpopulation or to investigate how certain characteristics affect the study outcome.

Characteristics of a non-random sample:

Targeted. Participants will be selected with a specific goal in mind, such as age, gender, occupation, etc.
Representativeness. The sample is designed to be representative of a particular population trait.
Targeted information. Researchers have predetermined characteristics or criteria in mind to study.

The random sample

A random sample is a sample in which every respondent has an equal chance of participating in the study.

Characteristics of a random sample:

Randomness. All members of the population have an equal chance of being included in the sample.
Random selection. The selection is done in a random way, for example, by using random number generators or random draws.
Representativeness: The goal is to obtain a broad representation of the population without influencing predetermined characteristics.

When do you choose a random or non-random sample?

A non-random sample is often drawn when:

Researchers want to approach a specific group to participate in the study
In-depth understanding of a specific group required
The results do not have to be generalizable for the entire population

The random sample

A random sample is often used:

When a random but representative sample of the population is required
When it is not important to study a specific group
When it is important that the results can be generalized

Combining both sampling techniques

The choice between a random and non-random sample depends on the research aims, the nature of the population, and the available resources. Sometimes, a combination of both methods is used to achieve a balanced approach.

Types of survey questions

Open-ended questions:

Open-ended questions ask respondents to answer in free text. They provide space for comprehensive, unrestricted responses, allowing for in-depth insights. An example of an open-ended question might be: What are your thoughts on [topic]? followed by an open text field where information can be entered.

Closed-ended questions:

Closed-ended questions limit responses to predetermined options. They can include yes/no questions, multiple-choice questions, or scale questions. This type of question helps with getting standardized answers.

An example of a closed-ended question is: Are you satisfied with our service?

1) Yes

2) No

Multiple-choice questions:

Multiple-choice questions provide respondents with multiple options to choose from. They are useful for standardized answers. An example of a multiple-choice question is: Which of the following factors influences your purchase decision the most?

(a) Price

b) Quality

Likert scale questions:

Likert scale questions ask respondents to rate their level of agreement or satisfaction on a scale. They are often used to measure attitudes. An example of a Likert scale question is: How satisfied are you with our customer service?

1) Very dissatisfied

2) Dissatisfied

3) Neutral

4) Satisfied

5) Very satisfied

Ranking order questions:

Ranking order questions ask respondents to rank options based on their preference or importance. For example: Rank the following features of our product according to their importance to you.

Dichotomous questions:

Dichotomous questions ask respondents to make a choice between two opposing options. A common example is: Have you received our newsletter?

1) Yes

2) No

Matrix questions:

Matrix questions organize multiple related questions into a matrix format. They make it easier for respondents to provide an answer. An example is: Give your opinion on the following aspects of our product on a scale of 1 to 5.

Image choice questions:

Image choice questions let respondents choose from images instead of textual options. They are helpful in collecting visual preferences. For example: Show images of different product packaging and ask: Which packaging appeals to you the most?

(a) Figure 1

(b) Figure 2

Value judgement statements:

Value judgement statements ask respondents to give their opinion on specific statements. They measure attitudes and beliefs. An example is: I agree with the statement: [statement].

a) Strongly agree

b) Agree

d) Disagree

e) Strongly disagree

8 Tips for Creating Effective Survey Questions

Define clear objectives. Set clear objectives for your survey. What exactly do you want to know? These goals form the basis for formulating your questions.
Choose the right type of questions. Select the most appropriate type of questions for your research. An example of all types of questions can be found above.
Keep questions simple and understandable. Formulate questions in clear and simple language. Avoid ambiguity or complex sentence structures. Test comprehensibility with a small group of respondents before distributing the survey.
Avoid duplicate questions. Each survey item should contain only one specific question. Avoid duplicate or compound questions.
Use neutral language. Formulate questions in a neutral and impartial manner to maintain objectivity. Avoid suggestive or directing language.
Keep the survey short and relevant. Limit the length of the survey so as not to overwhelm respondents. Make sure each question is relevant to your research goals.
Follow a logical sequence. Organize questions logically, starting with simple questions to get respondents comfortable.
Use measurable terms. If you want quantitative data, use measurable terms. For example, "How often?" instead of "Often?". Use closed-ended questions in combination with a range (e.g. 1 - 5 times a month, 6 - 10 times a month) to easily analyse the results afterwards.

Step-by-step-guide for creating online surveys

Define clear objectives. Set clear objectives for your survey. What exactly do you want to know? These goals form the basis for formulating your questions.
Choose the right type of questions. Select the most appropriate type of questions for your research. An example of all types of questions can be found above.
Keep questions simple and understandable. Formulate questions in clear and simple language. Avoid ambiguity or complex sentence structures. Test comprehensibility with a small group of respondents before distributing the survey.
Avoid duplicate questions. Each survey item should contain only one specific question. Avoid duplicate or compound questions.
Use neutral language. Formulate questions in a neutral and impartial manner to maintain objectivity. Avoid suggestive or directing language.
Keep the survey short and relevant. Limit the length of the survey so as not to overwhelm respondents. Make sure each question is relevant to your research goals.
Follow a logical sequence. Organize questions logically, starting with simple questions to get respondents comfortable.
Use measurable terms. If you want quantitative data, use measurable terms. For example, "How often?" instead of "Often?". Use closed-ended questions in combination with a range (e.g. 1 - 5 times a month, 6 - 10 times a month) to easily analyse the results afterwards.

Research techniques