Data quality assurance plan for a field survey: a template

A data quality assurance plan (QA plan) is the survey team's own operating document for data quality. It is written before training, agreed with the client or principal investigator, and kept beside the questionnaire for the whole of fieldwork. A terms of reference (ToR) says what a data collection firm is contracted to deliver; the QA plan says how the team will do it, day by day. For the contract side, use the data quality clauses for a terms of reference. This page is the plan that meets them.

It is also narrower than a data management plan (DMP). The Digital Curation Centre's guidance on data management plans describes a DMP as stating what data will be created and how, and the plans for sharing and preservation, and notes that funders increasingly require one. A QA plan covers what a DMP usually leaves out: how the data is checked and corrected while it is being collected. Let each refer to the other.

How to use the template

Each numbered block is text to copy into your plan. Parts in square brackets are yours to fill in. References such as “Standard 9.4” point to items in the Field Data Quality Standard, an openly licensed checklist with default thresholds: attach it as an annex and you don't have to define each check again. Delete blocks that don't apply and say so in the plan, so a reader can tell a skipped check from a forgotten one. Give the plan a version number and a date, and record every change in block 11.

1. Scope

1.1 This plan covers [survey name], round [number], collected by [organisation] for [client] on [SurveyLoopr / SurveyCTO / ODK Central / KoboToolbox]. Fieldwork runs from [date] to [date] in [areas], with a target of [number] completed interviews from [number] enumerators in [number] teams. It applies to the main survey, the back-check survey and any pilot data that reaches the server.

2. Roles

2.1 [Name, role] is responsible for this plan and approves every change to it. [Name] runs the daily checks and sends the flag list. [Name] holds access to personal data. [Names] supervise the field teams and answer flags. [Name] leads the back-check team, which reports to [name] and not to the field supervisors. [Name] at [client] receives the weekly report and may run their own checks.

2.2 Only [name] may change the form on the server. Every change carries a new form version and is recorded in block 11.

Name people, not job titles: “the data team” can't be asked why a flag is three days old. The back-check line matters most. A supervisor who back-checks their own team has a reason to find nothing (Standard 9.2).

3. Form design checks

Most errors are cheapest to stop in the form, before any check runs. Review the XLSForm against this block before training and again after the pilot.

3.1 Before training, [name] will confirm that the form: (a) has a constraint on every number with a hard limit, and a confirmation question for answers that are possible but unlikely; (b) asks for the respondent ID twice, or picks it from the sample list; (c) uses the same codes throughout for “don't know” ([-98]), “refused” ([-99]) and “other” ([-96]); (d) records the start and end times, the form version, the enumerator ID and, where the ethics approval allows, a GPS point; (e) ends with a comments field; and (f) marks as required every question that must be answered.

Two XLSForm rows show the difference between a hard limit and a soft one:

The first row blocks an impossible answer. The second asks the enumerator to confirm a large household with the respondent instead of blocking it, so a real household of 18 can still be recorded. The range checks guide covers choosing the limits.

4. Training and pilot

4.1 Enumerators and supervisors will be trained for [number] days, including [number] days of practice in the field. Each enumerator will pass a test on the questionnaire and the field protocols, with a pass mark of [score], before starting work.

4.2 The programmed form will be piloted in [place], similar to the survey areas but outside the sample, on [dates]. The pilot data will be exported and every check in block 5 run on it before fieldwork starts. Pilot and training interviews will be set aside with a reason, and fieldwork data counted from [date] (Standard 1.5).

J-PAL's guide to questionnaire piloting describes three phases, ending with a full-scale practice of the programmed form, the field protocols, GPS and data transfer. It also recommends testing how data moves from the device export through cleaning, and examining the dataset in Stata or R. Running your daily checks on the pilot data tests both: a check that flags every pilot interview has a wrong threshold, or a variable that was renamed.

5. Daily high-frequency checks

High-frequency checks are the checks run on every new batch of interviews during fieldwork; the check guide explains each one. List them in the plan with their thresholds, so nobody loosens a limit quietly to make flags disappear.

5.1 From the first day of fieldwork, [name] will run the checks below on all data received, at least [once] a day, by [time]. A threshold may change only with [name]'s approval, recorded in block 11.

5.2 Each day [name] will send each supervisor one list of new flags, giving the interview ID, the question, the recorded value and what is needed. Within [number] working days the supervisor will answer each flag in one of three ways: confirm the value, saying how it was checked; correct it, with the reason; or revisit or call the respondent and report the result.

5.3 Results will be read by enumerator, as rates per [100] interviews, against enumerators working in the same area. An enumerator who stands out on several checks at once will be accompanied by a supervisor within [number] days, and their respondents back-checked first.

The interquartile range (IQR) is the spread of the middle half of the answers, so “3 IQRs beyond the quartiles” is far outside the usual values; see outliers. Reading results fairly by person is covered in enumerator monitoring, and the signs of invented interviews in fabricated interviews.

6. Back-checks and spot-checks

6.1 The back-check team will re-interview [10]% of each enumerator's completed interviews, drawn at random, plus the interviews named under block 5.3, starting in the first week and within [number] days of the original interview. Each back-check question has a type: 1, shouldn't change (roof material); 2, tests the enumerator's skill; 3, may change, compared within [10]% (Standard 9.3).

6.2 An enumerator whose type 1 answers differ in [10]% or more of comparisons, once at least [10] are compared, will be reviewed under block 5.3. Rates will also be reported for each back-checker and each question (Standard 9.4 and 9.5).

6.3 Supervisors will observe [15]% of interviews, unannounced and more often in the first week, and record each visit on a form enumerators don't see. [Where respondents consent and the ethics approval allows, [share]% of interviews will be audio-recorded and reviewed.]

These defaults have a source. J-PAL's guide to data quality checks encourages back-checking at least 10% of respondents, with at least one per enumerator, treats an error rate above 10% on type 1 questions as a red flag, and suggests spot-checking 15% of surveys, more at the start. For phone surveys it advises a much higher back-check share. The back-checks guide covers choosing the questions.

7. Corrections and the log

7.1 Raw submissions will never be edited, on the server or in the raw data files. Every correction will be made in [Stata / R] code applied to a copy of the raw data, or in a correction log that the code applies.

7.2 The correction log records, for every change, the interview ID, the variable, the old value, the new value, the reason, who made it and the date. Interviews left out of the clean data (duplicates, test interviews, interviews that failed back-checks) are set aside with a reason, never deleted. A change without a reason will not be made (Standard 11.2).

In Stata, the simplest way to keep to both blocks is one do-file that is the only way the clean data gets made:

* clean.do: run from the top; the only way clean/survey.dta is made
use "raw/survey_raw.dta", clear      // as downloaded, never saved over

* Corrections, one per line, each with its reason
replace hhsize = 6 if hhid == "H0217"           // typed 60; supervisor confirmed 6
replace respondent_age = 34 if hhid == "H0333"  // digits swapped; ID card seen on revisit

* Set aside, with the reason (they stay in the raw file)
drop if inlist(hhid, "T001", "T002")            // pilot interviews before fieldwork began

save "clean/survey.dta", replace

Anyone can rerun it, read why each value changed and undo a line that was wrong. The duplicates guide shows how to decide which copy of a repeated interview to keep.

8. Personal data

8.1 Names, phone numbers, addresses, ID numbers, GPS points and other identifying fields will be stored apart from the rest of the data, encrypted, with access limited to [names]. Free-text answers will be checked for names and numbers before any data is shared (Standard 10.1 to 10.3).

8.2 Data shared with the analysis team or the client will be de-identified unless [name] approves otherwise in writing, and each such release recorded (Standard 10.4). Interview devices will be encrypted and locked with a passcode, and data moved only by [secure method].

Your ethics approval sets the rules; the plan says who carries them out. See how to de-identify survey data.

9. Reporting to the client

9.1 Every [week], [name] will send [client] a field report showing: completed interviews against the target, by area and enumerator; flags raised, answered and still open, by check and enumerator; how long flags took to answer; back-check agreement rates; actions taken on enumerators; and any change to a threshold or the form. [The client will have read access to the server, or a daily export,] and may run its own checks (Standard 11.5).

A client who commissions the survey can use this block to agree what they will see. How to monitor a survey firm covers the client's side.

10. Closing the dataset

10.1 When fieldwork ends, [name] will run every check on the full dataset once more. Each flag will be closed as confirmed, corrected or set aside, and any left open will be listed with the reason.

10.2 The final delivery will contain the raw data as exported; the clean data; the cleaning code that produces the clean data from the raw data with no manual steps; the correction log; the back-check data and comparison; the codebook; every version of the form; and this plan with its change log. [Client contact] will accept it once running the code on the raw data reproduces the clean data (Standard 11.3).

10.3 Identifying data will be archived in [encrypted location], with access held by [name], and removed from working copies and devices by [date].

11. Changes to this plan

11.1 Every change to this plan, the form or a threshold is recorded below with the date, what changed, the old and new values, the reason and who approved it.

How DataFlow fits the plan

The plan works with any tool. DataFlow connects to SurveyLoopr, SurveyCTO, ODK Central and KoboToolbox, or reads CSV and Excel exports, and can sync and run the checks on a timetable you set, such as every 2 hours between 07:00 and 22:00. On import it reads the form and the data and adds most of block 5 itself: duplicates on ID-like variables, interview length (under 40% of the median or 5 minutes, whichever is longer, or over 3 times the median), ranges from the form's constraints, outliers beyond 3 IQRs, more than 20% “don't know” or refused answers, the form's skip logic, older form versions and GPS. A suggested check that would flag more than 5% of interviews starts switched off, unless most of its flags come from one enumerator. Pilot interviews can be set aside by a condition, with a reason.

For blocks 5.3 and 6, it computes six signs of made-up interviews per enumerator against others in the same area (at least 10 interviews each, 4 enumerators to compare), picks a random share of each enumerator's interviews to back-check (10% to start) plus those the signs point to, and saves them as a CSV the back-check form can preload. It flags an enumerator when 10% or more of type 1 answers differ, once at least 10 are compared.

For blocks 5.2 and 7, flags reach supervisors as a note to copy, which comes back after a set number of days, or as an Excel sheet where they write the corrected value and a reason on each row. Every correction needs a reason and can be undone, the raw data is never modified, and checks run, corrections, downloads and reports go into an activity log where each entry carries a hash of the one before.

For blocks 8 to 10, DataFlow suggests which variables hold personal data, from the form and from free-text answers, and the ones you protect move to a separate encrypted store, left out of checks, reports and downloads unless someone unlocks it, which is logged. The Field Report is one HTML file, at a public, management or team level, with interviews against the target and flags raised and still open by check, area and enumerator. A download holds the clean data, a codebook, every correction with who, when and why, and by default the raw data with a Stata do-file that replays each correction.

Questions