ODK data quality: monitoring ODK Central fieldwork every day

ODK Central stores your submissions and gives you the tools to review them one at a time. It doesn't compare them with each other, so a household interviewed twice or an enumerator finishing every interview in eight minutes stays invisible until someone checks. Daily ODK data monitoring, read enumerator by enumerator, catches those problems while the team can still go back.

What ODK gives you

In the form

• Constraints, required questions and relevance. A constraint rejects an impossible answer and shows your message; required questions can't be skipped; relevance hides questions that don't apply, so a blank means “not asked”.
• Start and end times. start and end record when the form was opened and finished. They come from the device clock, which ODK's documentation warns can be wrong.
• The audit log. Add an audit question and ODK Collect records events as the form is filled, with timestamps: how long each question took, how the enumerator moved through the form and, if you turn it on, which answers changed. Elapsed times within one session are reliable even when the device clock is not. See the form audit log documentation.

In ODK Central

• The submissions table. Newest first, with a choice of columns, filters by submitter, date and review state, and a map of the first location question.
• Review states. Received is the state of a new submission, and Edited is set automatically when someone edits one. Project managers and administrators set the other three: Has issues, Approved and Rejected, each with an optional comment. Rejected submissions are still exported unless you filter them out.
• Edits. A submission can be edited in the browser. Each edit makes a new version, and the submission's activity feed shows what changed between versions. ODK's documentation suggests leaving a comment that explains each edit.
• Exports. A zip with one CSV for the main table and one for each repeat group, joined by key columns, plus media files and the audit log if you collect one. Options split select-multiple answers into one column per choice, remove group names from headers and include deleted fields. Filters on the table, review state included, apply to the export.
• OData. A live feed of submissions that Excel and Power BI read directly, and that R and Python read through ruODK and pyODK.
• Entities. A registration form can create a list of households or sites that follow-up forms select from, with values saved from earlier visits. For back-checks, an entity list of the households to revisit keeps IDs consistent between the main survey and the back-check.

ODK's guide to managing submissions in Central covers review states, edits and exports in full.

What ODK Central leaves to you

• Checks across submissions. Duplicate IDs, outliers against the rest of the data, interview length compared with the median.
• A view by enumerator. Filtering by submitter lists one person's submissions, but not their flag rate or how they compare with the rest of the team.
• Deciding the review state. Central sets Received and Edited; Has issues, Approved and Rejected are judgements someone has to make, ideally from the checks below.
• Back-check comparison. Matching back-check answers to the original interviews and counting differences per enumerator.
• Cleaning code. Central's edit history shows what changed on its server. Corrections made after export need their own record.

A daily monitoring routine by enumerator

1. Pull what arrived

After the day's uploads, download new submissions: through OData, filtered on submission date, or as a CSV export with a date filter. Keep a copy of each download unchanged; it is the raw data. Then add it to everything received so far, because most checks compare the new interviews with the old ones.

2. Run the checks

For interview length, remember that end minus start includes time the form sat saved and unfinished, and depends on the device clock. Flag interviews far below the median length rather than below a fixed number, and use the audit log to see where the time went. The high-frequency checks guide covers each check, and the outliers guide explains the two usual ways to measure “far from the rest”.

3. Make one table per enumerator

One row per enumerator: interviews today and in total, median interview length, flags per 100 interviews for each check, and the share of don't-know answers. Compare each person with the others working in the same area, since a remote village can make honest interviews look unusual. Use an enumerator question in the form; the submitter name in Central only identifies the enumerator if each one has their own app user.

Patterns matter more than single flags. One short interview is a question; a third of someone's interviews running short, with answers that barely vary and no “don't know” anywhere, is a reason to back-check that person's work first.

4. Mark what needs a look in Central

Use review states to track the follow-up. Set each flagged submission to Has issues with a comment saying what to check, send the list to the supervisor, and set it to Approved once it's explained or fixed, or Rejected if the interview should be dropped. Agree with the team what each state means before fieldwork starts, so Approved means the same thing to everyone.

Cleaning ODK Central data

You can correct in Central or in your cleaning code; pick one. Central's edits keep a version history on the server, and exports show the latest version. Cleaning after export keeps the downloaded data untouched, with each correction written as code or as a row in a log: the submission's instanceID, the question, old value, new value, who decided and why. Doing both leaves corrections in two places that no single record explains.

Either way, the final cleaning should rebuild the clean dataset from the raw data in one run. If your study has personal data, de-identify before sharing; the guide to de-identifying survey data covers what to remove.

Running the checks with DataFlow

DataFlow is a desktop app that runs this routine on every sync. It connects to ODK Central with the server's https address and the email and password of a Central account. It exchanges them for a session token, keeps only the token in your computer's keychain, and asks you to sign in again when the token expires. You choose the project and form from lists. If the form was uploaded as an XLSForm, DataFlow downloads it and sets up checks from its questions, choices and constraints.

Each sync reads new submissions through Central's OData feed, oldest first, starting after the newest submission date it has already seen. Answers inside groups keep their question names, and repeat groups become numbered columns that checks can still read row by row. The submitter comes in with each submission. DataFlow only reads from Central: it never changes a submission or its review state. Syncs can run on a timetable, such as every two hours through the working day.

SurveyLoopr, managed ODK Central hosting from the makers of DataFlow, connects the same way through a SurveyLoopr account, so no ODK password is typed into DataFlow. The integrations page lists every connection.

Step by step

1. Prepare the form. Add constraints, required questions, an enumerator question and the start and end metadata. Add the audit log if you want time per question.
2. Agree how review states are used. Decide what Has issues, Approved and Rejected mean for this survey, and who sets them.
3. Pull new submissions every day. Use OData or a CSV export filtered by date after the day's uploads, and keep each download unchanged.
4. Run the checks over all submissions. Duplicate IDs, interview length, outliers, ranges, missing answers, don't-know share, "other, specify", logic rules and GPS.
5. Build the enumerator table. Interviews, median length and flags per 100 interviews for each enumerator, compared with others in the same area.
6. Mark flagged submissions Has issues. Add a comment saying what to check and send the list to the supervisor the same day.
7. Record corrections and close the loop. Log each correction with who and why, set resolved submissions to Approved or Rejected, and rebuild the clean data from the raw download.
8. Compare back-checks each week. Match back-check answers to the original interviews and count differences per enumerator and per question.

Questions