Guides
How to check survey data while the team is still in the field. Keep your collection platform; add the quality checks.
Start here
- The Field Data Quality Standard. The Field Data Quality Standard: an open checklist of the daily checks field survey data should pass, with default thresholds. Version 1.0, CC BY 4.0.
- High-frequency checks for survey data. High frequency checks catch survey errors while the team is still in the field: every data quality check, what it catches, and how to act on each flag.
For clients of survey firms
- How to monitor a survey firm's data while it's in the field. How to monitor a survey firm during fieldwork: read access to the data, checks agreed in the contract, daily checks by enumerator and weekly back-checks.
- How to choose a data collection firm. How to choose a data collection company for a field survey: the questions to ask, a scoring table for proposals, and the red flags to watch for.
- Data quality requirements for a survey firm's terms of reference. Data quality clauses for a terms of reference for data collection: data access, daily checks, back-checks, audit trail, deliverables and personal data.
Your collection platform
- SurveyCTO data quality monitoring: the checks to run on every sync. SurveyCTO data quality: what constraints, automated quality checks and the review workflow cover, and the checks by enumerator to run on every sync.
- KoboToolbox data quality checks and cleaning. KoboToolbox data quality: what validation criteria and validation status do, the checks to run on each export, and how to clean Kobo data safely.
- ODK data quality: monitoring ODK Central fieldwork every day. ODK data quality: what ODK Central's review states, edits and exports give you, and the daily checks by enumerator to run on every new batch.
The checks
- Back-checks: how to run a back check survey. How to run a back check survey: which questions to re-ask, how to sample by enumerator, how to compare answers, and what to do when they disagree.
- Outliers in survey data: how to find and handle them. How to find outliers in survey data with the IQR or standard deviations, within groups, with Stata code, and what to do with each value you flag.
- Duplicate survey responses: finding repeated IDs and interviews. How to find duplicate survey responses: repeated IDs, households interviewed twice, forms sent twice and copied interviews, and what to do with each.
- Interview duration checks: interviews that are too short or too long. How to check survey duration: where interview length comes from, setting limits from the median, reading it by enumerator, and handling speeders.
Running fieldwork
- Enumerator monitoring: each interviewer's data quality, every day. Enumerator monitoring for field surveys: what to track for each interviewer, how to read it fairly, and how to act, from a conversation to reassignment.
Using DataFlow
- Moving DataFlow and your projects to a new computer. Move DataFlow to a new computer: sign in and your plan follows you. Projects stay behind with their keys, so download each one and set it up again.
Personal data, tools and comparisons
- Data cleaning software for field surveys. Data cleaning software for field surveys: what it must do that generic data cleaning tools don't, a fair comparison of approaches, and what DataFlow does.
- How to de-identify survey data before sharing it. De-identification of data from field surveys: find direct and indirect identifiers, move them to an encrypted key file, and check before you share.