This summer we are taking our readers On the Road to a Great Survey. At this point, after writing and testing the survey questions, and once your survey is in what you think is likely it’s final form, it’s time to formally pilot test your survey.
A pilot test, or field test, as with the final survey, draws on the skills, expertise, and resources of a number of sources, so this part of the survey process is complex, resource-intensive, and collaborative. Pilot testing is designed to test the quality of the data you get from your questions, as well as to test the viability of the survey as a whole, and the coordination of your team’s procedures and systems.
Datassist has worked with a number of professional survey firms, in both developed and emerging economies, in developing data collection tools and training. Our experience consistently demonstrates that this point in the survey process is critical to the successful collection and analysis of the data.
Conduct a pilot test exactly as you will the final survey. To conduct a pilot test, it’s best to use your actual field enumeration team and a sample from your research population.
Provide the enumerators with a script that they can follow, which they use to introduce themselves to the respondent, explain what the survey is for, and ask for consent. The surveyors also need to be trained in how to behave in safe and neutral ways in order not to influence the answers and to encourage most participants to complete the entire survey.
Choose respondents from your actual research population to complete the pilot test. It is not essential, but is recommended, that you choose these pilot test respondents using your sample selection methodology, since it is a good time to test our your sampling methods.
Implement all the steps in your survey process from start to finish. Distribute the survey exactly as you will in your final run. Then, clean the pilot survey data and analyze the data. Many times in our consulting experience, it is while attempting to analyze pilot survey data that we realize that the survey is not giving us the data we really need. This can be a frustrating time to make this discovery. However, it is much cheaper and much more feasible to make the necessary changes now than to redo an entire field survey.
Things to look for in your pilot test:
- Do the questions seem to provide consistent answers when compared to each other?
- Is the entire range of choices to most questions actually used by respondents, or are almost all answers clustering around one or two choices in the range?
- Are there strange or unexplainable patterns in your data? Could it be caused by confusion around your questions?
- Are you able to answer your core research questions with the data?
- Is the length of time it takes to complete the survey okay?
- What is the feedback from the pilot field team?
The final question about pilot tests is whether that data can be used as part of the final database. Whether or not this is a good idea depends a lot on how similar the pilot respondents are to the final sample selection, as well as to the amount of time between the pilot survey and the final survey and the number of changes made to the final survey as a result of the pilot. If any substantial change has been more or it has been more than a few weeks between the pilot and the final survey it’s safest to exclude the pilot data from the main database.
…So take time right now to:
- Sign up for monthly data science and visualization resources.
- Check out this recently published book that features Datassist work,
- Download this Datassist Road Map to a Great Survey
- Take a look at some of the work we have done.
- Friend us on Facebook for a personal connection with what we do.
- Follow us on Twitter for the latest news.
- Get inspired with us on Pinterest.