Survey Data Cleaner
Clean a survey export and get a list of every value that changed.
What it does
Give it a survey export (CSV, a Qualtrics CSV, or an SPSS .sav file) and it does the routine clean-up before analysis. It drops the two extra header rows Qualtrics adds under the column names. It empties missing-value codes: by default NA, NULL, None, -99 and a few similar ones, so a written answer of just "none" is emptied too unless you take it off the list. It trims stray spaces. It also removes repeat submissions: when two rows have the same respondent ID (a column such as ResponseId, or one you name), it keeps the first row and removes the later ones. Some responses are flagged for you to review, never deleted: speeders (people who finished in under a third of the median time), straight-liners (the same answer to every item of a question grid), unfinished responses and answers outside ranges you set. You get the cleaned file, a codebook and a change report. The report has the row, column, old value, new value and rule for every change, so you can check or undo any of it.
What you get
- The cleaned data as CSV. If you give it an SPSS file, you also get a cleaned .sav with its variable and value labels kept.
- A change report: one line per changed value, with the row number in your original file, respondent ID, column, old value, new value and the rule that changed it. Each removed repeat submission has its own line.
- A flag list, plus a qc_flags column in the cleaned file, for speeders, straight-lining across question grids, unfinished Qualtrics responses and values outside your ranges.
- A codebook for every variable: label, type, counts of answered and missing, range, value labels or the answers seen, and which missing codes were found.
- A readable report of all of the above. Every change except the Qualtrics header-row removal can be switched off. Flags only mark rows; they never change your data.
Example: the change report
| Row | Respondent | Column | Old | New | Rule | Note |
|---|---|---|---|---|---|---|
| 1 | 1 | abany | -100 | spss-user-missing | iap | |
| 1 | 1 | abdefect | -100 | spss-user-missing | iap | |
| 1 | 1 | abhlth | -100 | spss-user-missing | iap |
Best for
- The routine clean-up between downloading an export and starting your analysis, with a record you can keep with your data.
- Spotting low-quality responses to review, without anything being deleted for you.
Not for
- Decisions that depend on your study design. The rules are mechanical: check the change report before analysis.
- Keeping different kinds of missing apart. Codes such as "refused" and "don't know" all become empty cells; the change report and codebook keep the original codes, or you can switch the rule off.
- Open-text answers where a word like "none" is a real answer. "None" is on the default missing-code list, so remove it from the list for those surveys.
- Excel or Stata files: save them as CSV first. Dates written day/month or month/day are left unchanged.
Cost and how to start
- Open it on Apify Store and sign in, or create a free Apify account.
- Give it one survey file: a direct download link or a file uploaded to your Apify storage and start the run.
- Download the cleaned file, the change report, the flags and the codebook, or open the report.
Free from us. Apify bills its own small platform usage to your Apify account. Apify charges its own small usage fee to your account: about US$0.001 for a small export; about US$0.03 for a 1,624-response survey with 33,000 changes; about US$0.05 for the 31 MB General Social Survey file when we tested it.
Questions
- My file is on my computer. How do I clean it?
- In the Apify Console, open Storage, then Key-value stores, open a store and upload your file as a record. Then pick that store in the actor's input form. Or give a link that downloads the file without a login. Files up to 100 MB are accepted by default; you can raise the limit to 500 MB in the input. Large files need more run memory: the 31 MB General Social Survey file needed about 2.3 GB.
- Where is my data processed and kept?
- The file is read inside your own run on Apify, and the cleaned file and reports are saved in that run's storage in your Apify account. The actor doesn't send them anywhere else. Apify deletes run storage automatically under your plan's data-retention rules, or you can delete it sooner: open the store under Storage in the Apify Console and use its Actions menu. A file you upload to a key-value store stays there until you delete it or the store's retention period ends, so delete it when you're done. Survey data often holds personal details, so consider removing names and contact details first.
- Can I try it without a file?
- Yes. With no file given, it cleans a small made-up Qualtrics export so you can see the output.
How we tested it
- Every change is on the list. We ran four public survey files from Australian government open-data portals, 20,744 rows in all, and compared each cleaned file with the original cell by cell. 122,174 cells had changed, and every one was in the change report with the right old and new value. Most were placeholders standing in for no answer: the word NULL in 71,982 cells of one survey, NA in 17,248 of another, a single space in 32,777 of a third.
- Qualtrics exports come out ready to analyse. On eight Qualtrics exports shared in public GitHub projects, it removed the question-text and ImportId rows under the header each time. It kept the question text as the variable labels in the codebook.
- Large SPSS files work. The US General Social Survey 2022 (4,149 respondents, 1,297 variables) was cleaned in under 90 seconds on Apify, with the run given 4 GB of memory. It emptied the 3,002,874 answers that the file itself declares as missing, such as "iap" (inapplicable: the question wasn't asked of that person) and "no answer". The cleaned .sav kept its variable and value labels.