All docs

Response quality

Flag speeders, straight-liners, repeats and gibberish with a reason, and decide what each report counts.

Updated . 3 min read.

Response quality review points at answers that may not be reliable. It never deletes or edits a response. A flag is advice with a reason, and only a person can take a response out of a report.

Open Surveys, your survey, Analysis, Quality.

What is flagged

Check What it looks for Default
Speeding Finished in less than 30 percent of the median completion time, once at least 10 responses have timing On
Straight-lining The same answer on 5 or more rating questions, or down one matrix column On
Repeats A later response that repeats an earlier one answer for answer, with at least 3 answers. The first is never flagged On
Gibberish Open text such as keyboard mashing or one character repeated On

Every flag shows its reason in plain words, for example "Finished in 10 s, under 30% of the median completion time (60 s, from 12 responses)". Under Thresholds you can switch each check off and change its limits for this survey. Saving thresholds does not change existing flags; run a scan to apply them.

When flags are updated

A scan reads all responses and rebuilds the flags. It runs in the background:

  • a few minutes after new responses arrive, once for a burst of responses,
  • every night for surveys with recent responses,
  • when you press Run quality scan.

Manual decisions are never touched by a scan.

Decide

Open a flagged response and choose Exclude or Keep. Undo clears the decision. A legitimate low score stays in your reports unless someone deliberately excludes it. Each decision records who made it, when, and an optional note, can be reversed, and survives the next scan.

Choose what a report counts

The inclusion policy is available on crosstabs, trends and exports:

Policy Counts
All responses except spam and manual exclusions The default
Quality-checked Leaves out flagged responses and manual exclusions, unless you chose to keep them
Only flagged or excluded The responses the quality-checked policy leaves out, for review
Every non-spam response Ignores flags and decisions

Exports name the policy and add the flags and the decision to each row. This holds for the survey export, the unified Responses export and the agent export tool.

In the mobile app

The mobile app counts responses the same way. Its Results screen has a Counting menu with the four choices above, and the totals, the finished share and every question are built on the responses the choice counts, so a figure in the app matches the same figure on the web. Trends take the same choice.

Without a choice, the app and the mobile export use All responses except spam and manual exclusions, the same default as the web results. A file exported from the app names the policy it used and says how many survey responses it holds and how many it left out; the export sheet lets you write the file again counting differently. A hand-picked selection or an export of the spam folder is written exactly as chosen unless you pick a policy.

API and agents

With surveys:read and surveys:write:

  • GET /surveys/{survey}/quality reads thresholds, counts and the review list.
  • POST /surveys/{survey}/quality/scan queues a scan, PUT .../quality/settings saves thresholds, POST .../quality/responses/{response} records a decision.
  • Agent tools: get_survey_quality, scan_survey_quality, set_survey_quality_settings, decide_survey_response_quality.

Next