De-identification Tips for Quantitative and Qualitative Data

Generative AI tools are now part of how many doctoral students write code and analyze data, and removing identifiers on your own computer before any of the data go into a tool seems to be the step that is easiest to skip. I am not a lawyer or an IRB, and your approved protocol, consent form, and data use agreement decide what is allowed for a given study. This post is what I would check first, for quantitative data and for qualitative data, so please take the parts that fit your project and leave the rest.

Re-identification research

Re-identification means putting a name back on a record after the name was removed, usually by combining a few ordinary details that together fit only one person. Studies have shown this for decades:

  • ZIP code, sex, and birth date. Latanya Sweeney estimated from 1990 census data that these three were enough to make 87% of the US population likely unique (Sweeney, 2000); Golle later put the figure at 63% with 2000 census data.
  • A governor’s health record. Sweeney found the Massachusetts governor’s record in state employee health data by matching those three fields to a Cambridge voter list she bought for $20 (Sweeney, 2015).
  • Fifteen attributes. Rocher and colleagues (2019) estimated that 15 demographic attributes would correctly re-identify 99.98% of Americans.
  • Quoted text. Ayers and colleagues (2018) looked at health studies of Twitter and found that searching the quoted tweets led to the person who wrote them 84% of the time.

Generative AI makes this much faster and cheaper, because a model can read someone’s posts and guess where they live, how old they are, or what they earn:

  • Reddit comments (2023). Staab and colleagues reported “up to 85% top-1 and 95% top-3 accuracy at a fraction of the cost (100Ă—) and time (240Ă—) required by humans.”
  • Published interview transcripts (2026). Tianshi Li gave the scientist interviews that Anthropic had published from its AI interviewer to widely available models with web search. For six of the 24 interviews that mentioned a publication, the models found the paper behind the interview, in some cases identifying the interviewee, at under 50 cents per transcript and using only what the interviewees said about their own work.
  • Pseudonymous accounts (2026). Lermen and colleagues built an agent that linked pseudonymous online accounts to real people with “up to 68% recall at 90% precision compared to near 0% for the best non-LLM method.”
  • Clinical notes (2024). After a de-identification model had masked every PII span it found, an LLM-based attack still re-identified 9% of the notes (Morris and colleagues, a preprint).

For a social work study, this means that removing names is where de-identification begins. A participant who describes their job title and the small agency they work for may be recognizable to anyone who knows the field, and now to a model with a search engine.

Definitions

These three terms are used loosely, and the plain meanings below follow the University of Michigan’s IRB data security guidelines.

TermPlain meaningExample
AnonymousNobody, including you, can tell who gave the data.An online survey with no names, emails, IP addresses, or location data collected at all
CodedNames are replaced with study IDs, and a separate key links each ID back to a person.A panel survey where you keep emails in a key file to send the follow-up wave
De-identifiedIdentifiers are removed and the key is destroyed, so nobody can link the data back.The coded panel file after the study ends and the key file is deleted

The guidelines add: “Note that coding the data does not make that data anonymous.” Most of the data doctoral students work with seem to be coded data, since the key is needed for follow-up waves, incentive payments, or withdrawals. That is fine; it means the key and the analysis file need separate storage, and the analysis file is the only one that should ever go near an AI tool.

HIPAA Safe Harbor and Expert Determination

HIPAA matters only when your data come from health care. It applies to covered entities (health plans, clearinghouses, and providers who transmit health information electronically for billing and other standard transactions) and their business associates. A survey you design and field yourself is usually governed by the Common Rule and your IRB, while records that come from a clinic or health system usually bring HIPAA with them. Even when HIPAA does not apply, its list of identifiers is a handy checklist. The Privacy Rule gives two ways to de-identify protected health information.

Under Safe Harbor, you remove 18 kinds of identifiers for the person and for their relatives, employers, and household members, and you must not have actual knowledge that what remains could identify someone. The 18 include:

  • Names, phone and fax numbers, email addresses, URLs, and IP addresses
  • Geographic units smaller than a state, and all elements of dates except the year
  • Social Security, medical record, health plan, and account numbers
  • Biometric identifiers and full-face photos
  • “any other unique identifying number, characteristic, or code”

Three details are easy to miss:

  • ZIP codes. A three-digit ZIP code is allowed only when that area “contains more than 20,000 people.”
  • Ages over 89. They are collapsed into “a single category of age 90 or older.”
  • Partial identifiers. According to the HHS guidance, initials or the last four digits of a Social Security number do not meet Safe Harbor either.

Under Expert Determination, a person with statistical expertise determines that “the risk is very small” that the information could identify someone, and documents the methods. HHS states that “there is no explicit numerical level of identification risk” that meets that standard, so the decision rests on the expert’s documented analysis.

The actual knowledge clause deserves a close read. The HHS example is a record listing the patient’s occupation as “former president of the State University.” Nothing in that field is one of the 18 identifiers, and it still identifies the person once it is combined with age or state. Social work data often include fields like that, such as a rare job title or the name of a small agency.

Under the Common Rule, private information is identifiable when “the identity of the subject is or may readily be ascertained by the investigator” (45 CFR 46.102). A file can meet Safe Harbor and still be identifiable to someone who knows the population.

Local processing before AI tools

Anything you paste into a chatbot leaves your computer, and on consumer plans it may also be used to train future models. So raw data should not go into a generative AI tool, including the ones the university licenses. De-identification happens on your own computer, with code you can rerun, and only the de-identified file goes anywhere else. The tools and the university say much the same:

  • OpenAI consumer plans. Its help page on model training says that for Free, Plus, and Pro “we may use your content to train our models” unless you opt out, and that if you give feedback, “the entire conversation associated with that feedback may be used to train our models.” Business, Enterprise, Edu, and API use are excluded by default.
  • U-M Safe Computing. Its page on AI and U-M data says “Do not place sensitive data in an AI service unless there is a U-M contract or data agreement in place that permits it.” The ITS AI pages point to the Sensitive Data Guide for the data types each university service permits.
  • U-M School of Nursing. Its faculty AI guide notes that even with an approved tool, a consent form or data use agreement “may prohibit AI processing even when direct identifiers are removed,” and advises: “Do not ask an AI model to perform the initial de-identification unless the model is approved to receive the original identifiable data.”
  • Your IRB protocol. If AI-assisted analysis was not in the protocol your IRB approved, an amendment may be needed before you start.
Diagram of a de-identification workflow in which the raw file and the key remain on the researcher's computer
A de-identification workflow that keeps the raw data on your own computer.

Quantitative data

In a survey file or an administrative record, identifiers come in two kinds. Direct identifiers point to a person on their own: a name, an email address, a phone number. Indirect identifiers are harmless one at a time and identifying together, such as age, job title, county, or the year someone started a program. Dropping direct identifiers is straightforward, and indirect identifiers take judgment.

Survey exports

A Qualtrics export includes columns you may not have asked for, and its dataset documentation lists them:

  • IPAddress, the respondent’s IP address.
  • LocationLatitude and LocationLongitude, a location estimated from the IP address that is “typically accurate to the city level” inside the United States.
  • Recipient name and email, included when you send the survey to a contact list.
  • External reference, a field that often holds a student or employee ID.
  • ResponseId, which links the row to the response stored in your Qualtrics account.

If you do not need IP addresses or locations, turn on Anonymize responses before launch. It applies only to responses collected after you switch it on.

Indirect identifiers and small cells

Take a survey of child welfare workers in one state. Age, job title, and county are each common, but a 64-year-old supervisor in a rural county may be the only one. ICPSR’s data preparation guide names detailed geography, exact occupations, exact dates of events, and detailed income as variables that “make unique cases visible.” The usual fixes:

  • Group values into broader categories, such as age ranges instead of exact age.
  • Cap extreme values, so every income above a threshold is reported as one top category (top-coding).
  • Blank out the rare combination (local suppression).

If you plan to share the dataset, the R package sdcMicro counts how many people share each combination and applies these fixes, and the World Bank’s practice guide walks through it.

Small cells in a paper’s tables raise the same problem. A cell with three people in it can point to those three people, which is why the All of Us dissemination policy bars any participant count “of 1 to 20” and CMS bars cells of 1 to 10.

Study IDs and key files

Each person gets a new study ID that has nothing to do with who they are, and the list that links IDs to people goes in a separate file, the key. Three kinds of IDs do not work:

  • Initials plus birth date. Anyone who knows those two facts can rebuild it.
  • A hashed email address. The FTC’s Office of Technology wrote in a 2024 post, No, hashing still doesn’t make your data anonymous, that a hash “still creates a unique signature that can track a person or device over time.”
  • The Qualtrics ResponseId. It points back to the record in your Qualtrics account that still has the identifiers.

U-M’s IRB guidance says “identifiers, data, and keys should be placed in separate, password protected/encrypted files” and that “each file should be stored in a different secure location.” The three scripts below do the same job in Stata, R, and Python:

  1. Read the raw export from a restricted folder.
  2. Shuffle the rows with a fixed seed and number them, so the IDs are random but come out the same every time the script runs.
  3. Save the key (new ID plus the identifying columns) in the restricted folder.
  4. Drop the identifying columns, check that every ID is unique, and save the analysis file somewhere else.

The column names are the Qualtrics defaults; change them to match your export.

Stata
import delimited "restricted/survey_raw.csv", clear varnames(1) case(preserve)

* New study ID in random order
set seed 20261117
generate double u = runiform()
isid u
sort u
generate study_id = _n

* Key file: the only link between study_id and a person
preserve
keep study_id ResponseId RecipientEmail
save "restricted/key.dta", replace
restore

* Analysis file without direct identifiers
drop u ResponseId IPAddress LocationLatitude LocationLongitude ///
    RecipientFirstName RecipientLastName RecipientEmail ExternalReference
isid study_id
save "analysis/survey_deid.dta", replace
R
library(dplyr)
library(readr)

raw <- read_csv("restricted/survey_raw.csv", show_col_types = FALSE)

# New study ID in random order
set.seed(20261117)
raw <- raw |> mutate(study_id = sample(n()))

# Key file: the only link between study_id and a person
raw |>
  select(study_id, ResponseId, RecipientEmail) |>
  write_csv("restricted/key.csv")

# Analysis file without direct identifiers
deid <- raw |>
  select(-ResponseId, -IPAddress, -LocationLatitude, -LocationLongitude,
         -RecipientFirstName, -RecipientLastName, -RecipientEmail,
         -ExternalReference) |>
  arrange(study_id)
stopifnot(n_distinct(deid$study_id) == nrow(deid))
write_csv(deid, "analysis/survey_deid.csv")
Python
import numpy as np
import pandas as pd

raw = pd.read_csv("restricted/survey_raw.csv")

# New study ID in random order
rng = np.random.default_rng(20261117)
raw["study_id"] = rng.permutation(len(raw)) + 1

# Key file: the only link between study_id and a person
raw[["study_id", "ResponseId", "RecipientEmail"]].to_csv(
    "restricted/key.csv", index=False)

# Analysis file without direct identifiers
deid = raw.drop(columns=[
    "ResponseId", "IPAddress", "LocationLatitude", "LocationLongitude",
    "RecipientFirstName", "RecipientLastName", "RecipientEmail",
    "ExternalReference",
]).sort_values("study_id")
assert deid["study_id"].is_unique
deid.to_csv("analysis/survey_deid.csv", index=False)

A few details in these scripts are worth checking:

  • Set the seed yourself in Stata. Stata resets its seed to the same value “every time Stata is launched” (set seed manual), so without set seed the IDs depend on whatever else used random numbers in that session.
  • Sort the analysis file by the new ID. A file left in the original row order still lines up row by row with the raw export.
  • The seed does not need to be secret. Rebuilding the key from it requires the raw file, which already has the names.

Qualitative data

In transcripts, field notes, case notes, and open-ended survey answers, identifiers hide inside sentences. A participant who says she was “the only Korean-speaking case manager in the county” has given no name, and a colleague would still know who she is. The work is closer to editing than to dropping columns. Open-ended answers count here even when they come in a survey file; the ICPSR guide says they “must be reviewed for disclosure risk.”

Replacement rules

The UK Data Service guide on anonymising text and the Qualitative Data Repository’s page on de-identification agree on a few basics:

  • Decide the rules before you start, and apply them the same way to every file.
  • Replace instead of deleting, and show that you did. “[case manager]” or “[clinic]” keeps the sentence readable, while a blank leaves the reader guessing.
  • Use the same replacement for the same person in every transcript, field note, and memo.
  • Keep a log of every replacement, stored with the key and away from the de-identified files.
  • Leave what the analysis needs. QDR warns against “unnecessary de-identification,” since a transcript with every place and organization removed can be hard to interpret.

A common setup is for participants to become their study codes, staff to become their roles, and every other person a participant mentions to become [name]. The replacements come from a rules file written while reading the raw transcript and kept with the key. A short script then applies it the same way to every file:

Python
import re
import pandas as pd

# The rules file lists every name and place found in the transcript.
# It stays in the restricted folder with the raw text.
rules = pd.read_csv("restricted/name_rules.csv")  # columns: find, replace
rules = rules.sort_values("find", key=lambda s: s.str.len(), ascending=False)

with open("restricted/interview_07_raw.txt", encoding="utf-8") as f:
    text = f.read()

# Longest strings first, so "Maria Lopez" is replaced before "Maria"
for find, repl in rules.itertuples(index=False):
    text = re.sub(r"\b%s\b" % re.escape(find), repl, text, flags=re.IGNORECASE)

with open("analysis/interview_07_deid.txt", "w", encoding="utf-8") as f:
    f.write(text)

A list only replaces what is on it. If a participant named Maria mentions her sister Mariana, “Maria” becomes P07 and “Mariana” is left as is until the sister gets her own rule.

Automated detection

Software can catch names and places that a tired reader misses, and it misses things a careful reader would catch, so it works best as a second check on the list:

  • Presidio. Presidio, originally from Microsoft, runs on your own computer in Python and flags names, places, phone numbers, and similar details. Its own documentation says “there is no guarantee that Presidio will find all sensitive information.”
  • Philter. Philter, built at UCSF for clinical notes, caught more than 99% of identifiers in its tests, and its authors still wrote that recall “was not perfect.”
  • A local language model. A model run on your own computer through Ollama or LM Studio can flag candidates too. Ollama’s FAQ says “We don’t see your prompts or data when you run locally,” and it has a setting to turn off its cloud features, which is worth turning on. Research on local models for this job is still mostly on clinical notes and mostly in preprints.

In a shared clinical de-identification task, job titles and locations were the hardest categories overall, and interview participants mention both all the time. So a person still reads every transcript, whatever the tool; the UK Data Service says “automated anonymisation should always be supplemented with careful review.”

Social media posts

Reddit posts and other social media text need one more step, whether you code them by hand or classify thousands with a model. A quoted post can be pasted into a search engine, and in the Ayers study above that led back to the author 84% of the time. Quotes from Reddit posts should therefore be paraphrased in anything you publish, and usernames should never appear.

Cultural context and vulnerable populations

What counts as identifying depends on the community. A detail that is common in one group can be unique in another, and being identified costs some participants far more than others.

  • Small communities. In a county with one Korean church or a handful of Spanish-speaking providers, language, ethnicity, religion, or country of origin can point to a single person. Kaiser calls this deductive disclosure, which “occurs when the traits of individuals or groups make them identifiable in research reports” (Kaiser, 2009). The paper’s example is a study that names its school district, where anyone who knows the district could identify teachers from age, gender, and years on the job.
  • Readers from inside the community. The person most likely to recognize a participant is often a family member, a pastor, or a coworker who reads the paper or attends the talk. Kaiser notes that deductive disclosure is also called internal confidentiality, and a quote that looks anonymous to a reviewer can be obvious to the participant’s own network.
  • Names and kinship terms. Naming conventions differ: Korean names put the family name first, many Latin American names include two surnames, a single name can have several English spellings, and people are often called “auntie” or by a church role instead of a name. A rules file needs every variant. Automated tools lag here too. A study of English named entity recognition systems found that they “perform better at identifying names from specific demographic groups” (Mishra and colleagues, 2020), and Presidio’s default configuration covers English only, so names from minoritized groups and text in other languages are more likely to slip through.
  • Multilingual transcripts. Code-switching, dialect, and regional expressions can point to a hometown, and they often disappear in translation, so the original-language transcript needs its own review. Interpreters and translators have seen the raw data too, which puts them under the same confidentiality procedures as the research team.
  • Higher stakes. For undocumented immigrants, people in recovery, or survivors of violence, re-identification can bring legal or physical danger. NIH Certificates of Confidentiality “protect the privacy of research participants by prohibiting disclosure of identifiable, sensitive research information to anyone not connected to the research,” and they are worth knowing about before a study with these groups begins.
  • Harm to a whole group. A finding can stigmatize a small, named community even when no individual is identified. The CARE Principles for Indigenous Data Governance note that open data principles focus on sharing “while ignoring power differentials and historical contexts,” and ask researchers to weigh collective benefit and community control alongside individual privacy.
  • Disaggregated categories. Reporting Hmong or Marshallese respondents separately, instead of folding them into “Asian,” is often the point of an equity study, and it also produces the small cells that suppression rules like All of Us’s exist for. The reporting categories are best decided with the community before the tables are made.

Checklist before analysis

  • Turn on Anonymize responses in Qualtrics before launch if you do not need IP or location data.
  • Store raw files and the key in a restricted folder that is not synced to a personal cloud account, and the analysis files somewhere else.
  • Assign study IDs at random and save the key as its own file.
  • Drop direct identifiers with a script so the step can be rerun when new responses arrive.
  • Check indirect identifiers: geography, rare occupations, exact dates, small cells, open text.
  • Check community-specific identifiers such as language, religion, name variants, and kinship terms.
  • Read every transcript after the automated step.
  • Check the protocol, consent form, and data use agreement for language about AI tools before any data go into one, de-identified or not.
  • November 17, 2026