8 keys to increase response rates
8 keys to increase survey response rates
When it comes to having results that are representative of your research population (i.e. best reflect the truth) an important feature is to have as high a response rate as possible. For example, 80 responses out of a sample of 100 people, is far more likely to represent the truth than 200 responses out of 1,000 as there will be a much lower ‘non-response bias’.1 But what are the best ways to ensure we have a high response rate and ensure that the data is representative? Here are 8 keys to consider before circulating your survey.
1. Shorter is good, but not vital
Whilst most polling and market research practitioners encourage a less than 10 minute survey length, research often shows that the number of questions isn’t as influential on response rates as other factors. Some meta-studies have shown that it does make an impact,2,3 whilst others find that it does not.4,5 In this regard, we recommend keeping your survey as short as you can without compromising on quality of questions. Instead, when thinking about response rate, focus more on the following factors in this page.
2. Ask closed and easy to understand questions
If there is debate about the number of questions in a survey being influential, a much more important factor is the type of questions being asked. Here long and difficult worded open questions significantly reduce response rates compared to closed, short, and simple questions. Likewise, placement of the questions is also influential. According to several studies, when facing a web survey request, respondents tended to glance over the survey questions quickly and make a preliminary assessment of the amount of effort required. If difficult questions were placed upfront in the questionnaire, respondents deemed the effort of completing the survey to be too high and were less likely to start or complete the survey.6
3. Send out pre-notifications
The more contact there is between the respondent and you, the higher the participation will be. Studies using samples of the general population found that pre-notification letters or emails increased response rates by 10 to 29 percentage points.2 By having this ‘warning’ notification, respondents are able to process and understand the requirements of the forthcoming survey, without having to commit to responding immediately.
4. Send reminders
Likewise, sending reminders is also a proven way to increase response rates.2,5 Whilst there is little research in the frequency and amount, we would recommend 3-5 reminders, spaced at least 4 days apart from one another.
5. Don’t include a progress bar
Another popular perception is that progress bars increases response rate. Whilst again counter intuitive, a meta-analysis conducted in 2013 on web-based surveys found that it had no impact on response rates, and other more recent studies have shown it to have a negative impact.7
6. Intrinsic rewards: Give purpose and meaning
People do things if they believe it is interesting, personally satisfying, and will result in personal growth. This is what is known as an ‘intrinsic reward’. By sharing the vision behind your survey and aligning it with personal gains of the respondents, this intrinsic reward has been shown to be more motivating than financial rewards to completing a survey. Whilst there are a variety of ways to achieve this, we recommend sharing the reasons and heart behind your research, allowing respondents to see their survey results immediately after completion, as well as how their results compare with the rest of the community you are surveying.8
7. Extrinsic rewards: Offer individual financial rewards up-front, not sweepstakes
Offering an extrinsic financial or voucher-based reward also can increase response rates, but not in all the ways you might expect. Decades of research shows that guaranteed small rewards increase the response rate by 10-15%.9, 10 If budget permits, pre-paying these incentives (i.e. incentives that are circulated before they complete the survey) increase the response rate even further (as supposed to post completion incentives). Interestingly however, almost all studies conclude sweepstakes or prize draws have no impact on response rates.10
8. Make it personal
Finally, where possible, make the invite to the survey personalised. By personalizing correspondence, you increase the degree to which recipients perceive the importance and value attached to their response. As a result, response rates have been shown to increase by up to 10%.11 This can be done through email subject lines that use the name of the individual you are emailing, as well as having the email come from your personal account. Both of these features are possible by using platforms such as mailchimp.
Whilst there are many additional ways to increase your response rates, start by considering these when circulating your survey.
REFERENCES
1. American Association for Public Opinion Research. 2008. ‘Response Rates – An Overview’. Available online at: http://www.aapor.org/Education-Resources/For-Researchers/Poll-Survey-FAQ/Response-Rates-An-Overview.aspx (last accessed: 16/11/19)
2. Edwards, P. Roberts, I. Clarke, M. Diguiseppi, C. Wentz, R. Kwan, I. Cooper, R. Felix, M. Pratap, S. 2009. ‘Methods to increase response to postal and electronic questionnaires’.Cochrane Database Syst Rev, 8:(3).
3. Fan, W. and Yan, Z.2010. ‘Factors affecting response rates of the web survey: A systematic review’. Computers in Human Behavior. 26(2): 132–139.
4. Rolstad,S. Adler, J. Rydén, A. 2011. ‘Response burden and questionnaire length: is shorter bettter? A review and meta-analysis’. Value health, 14(8):1101-8
5. Blumenberg, C. Menezes, B. Gonçalves, H. Assunção, M. Wehrmeister, F. Barros, F. and Barros, I. 2019. ‘The role of questionnaire length and reminders frequency on response rates to a web-based epidemiologic study: a randomised trial’. International Journal of Social Research Methodology, 22(6):625-635
6. Ganassali, Stephane. (2008). ‘The Influence of the Design of Web Survey Questionnaires on the Quality of Responses’. Survey Research Methods. 2. 21-32. 10.18148/srm/2008.v2i1.598.
7. Villar, A. Callegaro, M. Yang, Y. 2013. ‘Where am I? A meta-analysis of experiments on the effects of progress indicators for web surveys’. Social Science Computer Review, 31, 744–762.
8. 2019. For the Love of Money? Motivation and Engagement in Online Research Communities. Available online at: https://www.ipsos.com/en-nz/love-money-motivation-and-engagement-online-research-communities (last accessed: 16/11/19)
9. Murdoch, M. Simon, A. Polusny, M. Bangerter, A. Grill, J. Noorbaloochi, S. and Partin, M. 2014. ‘Impact of different privacy conditions and incentives on survey response rate, participant representativeness, and disclosure of sensitive information: a randomized controlled trial’. BMC Med Res Methodol, 14: 90.
10. Robb, K. Gatting, L. Wardle, L. 2017. ‘What impact do questionnaire length and monetary incentives have on mailed health psychology survey response?’. Br J Health Psychol, 22(4):671-685.
11. Dirk, H. Vanhove, T. Matthijs, K. and Loosveldt, G. 2005. ‘The effect of personalization on response rates and data quality in web surveys’. International Journal of Social Research Methodology, 8(2): 85-99.
