Third-party monitoring in development: checking a survey firm's work independently
Third-party monitoring (TPM) in development is when a donor or agency hires an independent firm to verify that a programme was delivered, or that a survey was done properly, usually where its own staff can't travel. The monitor checks the survey firm's data as it arrives, runs its own back-checks and site visits, and reports to the client rather than to the firm. Write the monitor's access, methods, independence and data protection into its terms of reference, and share reports and de-identified data rather than respondents' personal details wherever possible.
By Mehrab Ali, founder of DataFlow · Updated 7 November 2026
What third-party monitoring is
Third-party monitoring (TPM) means contracting an organisation outside the programme to collect and verify monitoring data. The SAVE research programme's report The use of third-party monitoring in insecure contexts (Sagmeister and Steets, 2016), based on research in Afghanistan, Somalia and Syria, describes aid agencies using it mainly to monitor their partner organisations' activities where their own staff face access restrictions. For donors it is a way to verify what partners report; for agencies, a source of first-hand field data for programming.
Most TPM contracts in development cover one or both of two jobs:
- Verifying delivery: visiting sites and talking to the people a programme was meant to reach, to confirm that distributions, trainings or construction happened as reported. The SAVE report found TPM most useful for this kind of quantitative and physical check: the quantities distributed, how many people were reached and who they were, and assets or infrastructure built.
- Verifying a survey: checking that a baseline, midline, endline or monitoring survey run by a survey firm was carried out as its contract says, with real interviews with the sampled households.
The report treats TPM as a complement to an agency's own monitoring: most useful as a last resort in areas with constrained access, reassessed regularly, and backed by community feedback systems and open communication with communities. It also found that agencies underestimated the time involved; contracting monitors under internal procurement rules typically took two to four months.
Who does what
| Party | Its job | Answers to |
|---|---|---|
| The client (donor, UN agency, government or NGO) | Sets the questions, contracts both firms, decides what happens on the findings | Its own management and funders |
| The survey firm or implementer | Collects the data or delivers the programme, and runs its own quality checks | The client, under its own contract |
| The monitor | Checks the firm's data and process, and gathers its own evidence from respondents and sites | The client, with no contract or reporting line to the firm |
The day-to-day routine of reading a firm's data (read access, daily checks, one list of flags, weekly back-check results) is the same whether the client does it or a monitor does it for them, and is set out in how to monitor a survey firm. What a monitor adds is independence and its own contact with respondents.
What a monitor checks in a survey firm's data
A monitor answers the client's questions from two sources: the firm's data and process, and evidence it collects itself. For each question, both matter.
| The client's question | In the firm's data | The monitor's own evidence |
|---|---|---|
| Were the sampled households interviewed? | Interview IDs against the sample list, progress by area and day, replacements | Visits to a sample of villages; checks that replacement households followed the protocol |
| Did the interviews take place? | Interview length, start times, overlapping interviews, GPS, and signs of fabricated interviews by enumerator | Back-checks on a random sample and on interviews the signs point to |
| Are the answers right? | Ranges, logic, outliers, don't-know shares | Back-check answers to questions that shouldn't change between visits |
| Does the firm's own quality control work? | Whether flags are answered, how quickly, and with what explanation | Whether the firm's back-checkers are independent of the enumerators' supervisors |
| Were corrections honest? | The raw data against the clean data, and the correction log | Running the firm's cleaning code on the raw data and getting the same clean data |
| Is personal data protected? | Names, phone numbers or GPS points in files that were shared | How the firm stores data, who has access, and how it is transferred |
The monitor's back-checks are the core of the job, because they are the only evidence that doesn't come from the firm. The AAPOR and American Statistical Association task force report Falsification in Surveys (2020) describes survey organisations recontacting roughly 5 to 15 percent of the sample to verify interviews, often combining a random sample with cases picked because something in the data or the interviewer's performance looked wrong. A monitor can do the same against the firm: draw within each enumerator's interviews, add the interviews the daily checks flag, call or visit within days, and report the targeted cases apart from the random ones. The back-checks guide covers the questions to re-ask and how to compare answers.
Reconcile the firm's reports with its data as well. The completes, refusals and replacements in a weekly progress report should match what the server holds, and a field report's statement that a region was covered should match the interviews and GPS points from that region.
Checking the monitor's own data
A monitor's data can be made up for the same reasons as a survey firm's. In the SAVE research, almost all agencies in Afghanistan reported irregularities in TPM data, such as data from areas monitors had not visited. Agencies responded with GPS-stamped photos, tracking of field staff, a consultant to check and clean the data, and triangulation between teams; all of them said they had to invest in cross-checking what monitors sent, with systems to monitor the monitors.
In practice, the client runs the same checks on the monitor's submissions as on the firm's: interview length, timing, GPS where it is safe to collect, the signs of fabricated interviews, and a small call-back sample of the monitor's own interviews. The SAVE report also warns that using technology to verify data carries risks in many conflict settings, and recommends low-visibility tools, such as ordinary smartphones rather than GPS devices and cameras, discussed openly with the field monitors.
What to put in the monitor's terms of reference
The survey firm's terms of reference (ToR) set what the firm owes the client: data access, daily checks, back-checks, a correction log and the deliverables. The clauses are in data quality requirements for a survey firm's terms of reference. Two additions connect it to the monitor: name the monitor in the firm's contract as a party that gets read access to the data and may back-check any interview, and attach the same checks and thresholds to both contracts, so the firm and the monitor discuss findings rather than definitions. The monitor's own ToR then covers the following.
| Topic | What to write | Why |
|---|---|---|
| Scope | The questions the monitor answers, the few indicators it verifies, and which surveys, sites and rounds | The SAVE report found that a focus on a few key indicators or areas, with data that is valid, works better than asking for too much. |
| Independence | No financial, family or subcontracting link with the survey firm or implementer; field monitors rotated between sites | The report found monitors embedded in local networks may be less likely to report wrongdoing, and monitors sent repeatedly to the same sites can lose their independence. |
| Access | Evidence of access to the sites in the bid, and what happens when access changes | Bidders have a reason to overstate access, and many agencies found it lower than expected. |
| Methods | The share of interviews back-checked per enumerator, the targeted sample, phone or visit, how soon after the interview, and whether visits are announced | The report advises against unannounced visits unless all parties have agreed to them. |
| Evidence | What counts as verified (a call record, a GPS point or photo where safe), and how the client will check the monitor's data | The monitor's findings carry weight only if its own data holds up. |
| Personal data | What the monitor may receive, where it is kept, how it moves, who sees it, and when it is deleted | See the next section. |
| Duty of care | A security plan, and insurance or equivalent compensation for field monitors, costed into the contract | Risk moves to the monitors. The report found most providers it studied had no proper security procedures, and recommends insurance as a selection criterion. |
| Reporting | How often and in what format, oral debriefs alongside written reports, and a route for urgent findings such as suspected fraud | The report found written TPM reports often unsatisfactory and oral debriefs a useful complement. |
| Deliverables | The monitor's raw data, cleaning code, back-check comparisons and a log of every correction | So the client can check the monitor's work the way it checks the firm's. |
Plan how the findings will be used before the first visit. The SAVE report found that agencies had to adjust their information systems to absorb what monitors sent, and advises making sure the information collected can inform decisions.
Sharing data safely between the firm, the monitor and the client
With a monitor involved, three organisations see the same respondents' data, and each copy is another place it can leak. Share the least that does each job, and decide in advance who gets what:
| What | Who needs it | How to share it |
|---|---|---|
| Submissions with names, phone numbers and GPS points | The survey firm | They stay where they were collected; the monitor reads them through a read-only account or receives de-identified exports |
| Contact details for the back-check sample | The monitor's back-checkers | Only the sampled households, and only what is needed to find them (name, village, phone); deleted once the back-checks are done |
| De-identified data, the correction log and the cleaning code | The monitor | Through the server account or an agreed encrypted transfer, never as unencrypted email attachments |
| Back-check results and flags by enumerator | The client and the firm's field manager | Interview IDs, enumerator codes and the questions that differ |
| Summary reports | The client and the donor | Counts and rates only, with enumerators as codes and small areas grouped |
- Put it in writing: a data sharing agreement that says who holds which data, for how long and how it is deleted, consistent with the consent respondents gave and the ethics approval.
- Make the monitor's account read-only, so it can't change submissions or the form.
- Keep names and contact details apart from the answers, linked by an ID, as the guide to de-identifying survey data describes.
- In conflict-affected areas, treat GPS points, photos and lists of who was interviewed as dangerous for respondents and monitors alike, and collect them only where the risk assessment allows.
- Keep a record of every correction and of every time someone opens personal data, so the client can later see who changed or saw what.
How DataFlow supports third-party monitoring
A monitor can connect DataFlow to the survey firm's SurveyLoopr, SurveyCTO, ODK Central or KoboToolbox server with the account the firm provides, or import CSV and Excel exports. DataFlow downloads the submissions without changing them on the server, keeps them in an encrypted project file on the monitor's computer, runs the checks on every sync and works out the six signs of made-up interviews for each enumerator. The monitor's back-check form can be linked to the survey it checks: the back-check sample is a random share of each enumerator's interviews not yet back-checked, plus the interviews the made-up interview check points to, and the results are compared by enumerator, back-checker and question.
Findings travel as a Field Report: a single HTML file, saved from the app, that opens in a browser and makes no network requests. It carries counts and rates, never the answers, at one of three levels:
- Public: totals and rates, enumerators shown as codes (E01, E02 and so on), areas with fewer than 5 interviews grouped as “Other areas”, and no text written by the team, so correction reasons are left out.
- Management: adds enumerators' names, every area, the reasons values were corrected, and changes made to many interviews at once.
- Team: adds the list of items still open, with each one's enumerator, area and how long it has waited, for the field team.
Every import, check, confirmation and correction goes into DataFlow's activity log, an audit log in which each entry is sealed with a SHA-256 hash of the one before, so changing an entry breaks the chain; the app shows whether the chain is intact. Each Field Report's Integrity tab shows whether the log behind it was intact and how many entries it had, and checks in the browser that the file itself is unchanged since it was made, against a fingerprint of its contents. Making a report is itself logged, with its level and fingerprint.
Variables you protect as personal data move to a separate store with its own encryption key, and the working dataset keeps blanks in their place. Checks, Field Reports and downloads are de-identified; putting personal data back into a download needs an explicit unlock, which is recorded in the activity log. A download includes every correction with who made it, when and why, including corrections later undone, and can include the raw data as received with a Stata do-file that replays each correction, so a client can check the monitor's cleaning the same way it checks the firm's.
Questions
What is third-party monitoring in development?
An arrangement in which a donor or agency hires an independent organisation to collect and verify monitoring data, such as whether a programme reached the people it was meant to or whether a survey firm's interviews really took place. It is used most where the client's own staff can't travel.
What does a third-party monitor check in a survey?
Whether the sampled households were interviewed, whether the interviews took place, whether the answers hold up, whether the firm's own quality control works, whether corrections were honest, and whether personal data is protected. It uses the firm's data and its own back-checks and site visits.
Can third-party monitoring replace an agency's own monitoring?
Research on TPM in Afghanistan, Somalia and Syria recommends using it as a complement to an agency's own monitoring, as a last resort in areas with constrained access, and reassessing it regularly. Agencies should do as much of their own monitoring as they can.
Who checks the third-party monitor?
The client. Run the same checks on the monitor's data as on the survey firm's (interview length, timing, GPS where safe, signs of made-up interviews) and call back a small sample of the monitor's own interviews.
How should a survey firm share data with a monitor?
Through a read-only account on the firm's server or de-identified exports, with contact details only for the households in the monitor's back-check sample, under a written agreement on who holds which data, for how long and how it is deleted.