KoboToolbox data quality checks and cleaning

KoboToolbox is built to get forms into the field quickly, and its quality tools act on one submission at a time: the form's rules at entry, then the data table, where you can edit a submission and set its validation status. Checks that compare submissions with each other, or enumerators with each other, happen after you export, and how well they work depends a good deal on the export settings you choose.

What KoboToolbox gives you

In the form

• Validation criteria. What the Formbuilder calls a constraint in XLSForm: a rule an answer must meet, with a message shown when it doesn't. Use them for anything impossible, such as an age over 120 or a negative price.
• Required questions and skip logic. Required questions can't be left blank; skip logic hides questions that don't apply, so a blank means “not asked” rather than “forgotten”.
• Metadata. start and end record when the form was opened and finished, and today the date. They come from the device clock.
• An enumerator question. Not a Kobo feature, but the most useful field you can add: a list of enumerator IDs, so every check can be read person by person. Kobo's _submitted_by records the account that sent the form, which only identifies the enumerator if each one has their own account.

On the server

• The data table. Every submission, newest first, with search, filters for each question, sorting, and a switch between labels and the underlying values.
• Editing. Users with permission to edit can change a submission from the table, or set one answer across several selected submissions at once.
• Validation status. Each submission can be marked Approved, Not approved or On hold, one at a time or in bulk, by users with the Validate submissions permission. The status is exported in the _validation_status column.
• Reports and maps. A chart and table for each question, which can be grouped by a select-one question, and a map of GPS points. Kobo's own documentation says reports are for a quick review and don't replace cleaning.
• Exports. XLS, CSV, GeoJSON, SPSS labels, KML and a zip of media, with the options covered below.

Kobo's help centre covers viewing and validating data and exporting it in detail.

What Kobo leaves to you

• Duplicates. The same household ID in two submissions. Each submission has its own _uuid, so only your own ID shows a household interviewed twice.
• Interview length. There is no ready-made check, though start and end give you what you need.
• Outliers. A value can pass the form's rules and still sit far outside everything else collected.
• Logic across questions, after the fact. Rules you think of mid-survey have to be run over what has already arrived.
• Comparisons between enumerators. Flag rates, interview lengths and answer patterns per person.
• Back-check comparison. Matching the back-check survey to the original interviews and counting the differences.
• A record of corrections. An edit in Kobo changes the answer you export, so the export no longer shows what the enumerator first recorded. Unless you note the change somewhere, nobody can see later what was corrected or why.

Export Kobo data so checks can read it

Many problems with cleaning Kobo data start with the export settings. Set these once under Data, Downloads, Advanced options, and save them with Save selection as, so every export comes out the same:

• XLS format. Kobo recommends it for forms with repeat groups: each repeat group goes on its own sheet. CSV exports leave repeat groups out.
• XML values and headers. Column names are question names and answers are choice names, which match your XLSForm and don't change when someone edits a label or switches language.
• Include groups in headers: off. Headers stay as the question name (hh_size, not household/hh_size), so they match the form.
• Store date and number responses as text: on. Otherwise the XLS stores dates and times in Excel's own format, which drops the time zone.
• Include data from all versions: on (the default). If you changed the form mid-survey, older submissions keep the answers to questions you later removed.

Every export also carries system columns. _uuid identifies the submission and is the one to match on across exports. _submission_time is when the server received it, _submitted_by is the account that sent it, and _validation_status is the status set in the table. _index is only a row number made at export time, so never use it as an ID. Kobo can also give you a link that refreshes a saved export in Excel or Power BI, which suits dashboards; for cleaning, keep a dated copy of each download.

The checks to run on each export

Run them over the whole export each time, not only the newest rows: a duplicate or an outlier often only shows up against what came before. The high-frequency checks guide explains each one.

Interview length from start and end

Kobo forms have no field like SurveyCTO's duration, so length is end minus start. That includes any time the form sat saved and unfinished, and both times come from the device clock, so a wrong clock gives nonsense. Treat very long interviews as a question, not an error, and look hard at very short ones: once a few dozen interviews are in, flag those far below the median length and see whose they are.

Use validation status as your to-do list

Validation status works well as a tracker for checks you run outside Kobo. Put each flagged submission On hold, send the list to the field team, then mark it Approved once the problem is explained or fixed, or Not approved if the interview should be dropped. Because the status is exported, the next export shows what is still open, and analysis can leave out the Not approved submissions.

Cleaning Kobo data without losing the raw data

1. Save every download as it came, with the date in the file name, and don't edit it. That file is your raw data.
2. Write each correction as code (Stata or R) or as a row in a correction log: _uuid, question, old value, new value, who decided and why.
3. Build the clean dataset by applying the corrections to the raw export, so anyone can run it again and get the same result.
4. Decide whether corrections happen in Kobo or in your log, not both. If some are edits in Kobo and others are lines in a script, no single record explains the final data.

Personal data needs the same care: names and phone numbers in a Kobo export travel with every copy of the file. The guide to de-identifying survey data covers what to remove before sharing.

Using DataFlow with KoboToolbox

DataFlow has no KoboToolbox connection yet. It connects directly to SurveyLoopr, SurveyCTO and ODK Central; Kobo data comes in by importing an export. The workflow:

1. Create a project and add your XLSForm, the same file you deployed in Kobo. DataFlow reads its questions, choices and constraints, finds start and end, and sets up checks before any data arrives.
2. Download the XLS export with the settings above and import it. DataFlow uses _uuid as the submission key, and you pick the column that names the enumerator.
3. Each day, download a new export and import it into the same project, or save it over the file you imported and sync. Submissions already there are matched by _uuid, new ones are added, and the checks run again over everything.
4. Work through the flags and approve corrections. They sit on top of the imported data, which is never changed, and the download includes a Stata do-file that rebuilds the clean data from it.

Two limits to know. DataFlow reads the first sheet of an Excel file, so rosters that Kobo puts on their own sheets aren't checked yet. And because imports are by hand, daily checking depends on someone downloading the export each day. The integrations page shows what connects today.

Step by step

1. Build checks into the form. Add validation criteria for impossible answers, required questions, an enumerator ID question, and the start and end metadata.
2. Save one export setting. XLS, XML values and headers, groups in headers off, dates and numbers stored as text. Save it so every download matches.
3. Download every day and keep each file. Save each export with the date in its name and never edit it. It is the raw data.
4. Run the checks over all submissions. Duplicate IDs, interview length, outliers, ranges, missing answers, don't-know share, "other, specify", logic rules and GPS.
5. Read the flags by enumerator. Count flags per enumerator per 100 interviews and compare each person with others in the same area.
6. Put flagged submissions On hold. Set their validation status to On hold and send the list, with each _uuid and question, to the field team.
7. Log corrections, then approve. Record each correction outside the raw export, rebuild the clean data, and mark resolved submissions Approved or Not approved.

Questions