Every call a healthcare voice agent handles produces a transcript, and nearly every transcript contains protected health information. Patients say their name, date of birth, phone number, address, and often why they’re calling. That’s the point of the call.
Those same transcripts are what you want for everything that improves the product: analytics, search, LLM-generated summaries, and evaluation datasets. The question is how to use them without spreading PHI to every system that touches them.
Our answer was to put a de-identification step between raw transcripts and everything downstream, based on the HIPAA Safe Harbor method.
Safe Harbor in one paragraph
HIPAA gives two ways to de-identify health data. Expert Determination has a qualified expert certify that re-identification risk is very small. Safe Harbor is prescriptive: remove 18 categories of identifiers (names, geographic units smaller than a state, all date elements except year, phone and fax numbers, email addresses, SSNs, medical record numbers, account numbers, device identifiers, URLs, IP addresses, biometric identifiers, full-face photos, and any other unique identifying number or code), and have no actual knowledge that what’s left could identify someone.
Safe Harbor suits engineering because it’s a checklist you can test against. It’s also strict: dates become years, and an address becomes, at most, a state.
Three passes, because no single one is enough
Pass 1: patterns
Structured identifiers have shapes: phone numbers, emails, SSNs, MRN formats, URLs, dates. Regular expressions catch most of them cheaply and predictably.
The catch is that transcripts are speech converted to text, and people don’t speak in formats. “My number is four one five, five five five, oh one nine two” contains no digits at all. So before pattern matching, a normalisation step converts spoken numbers to digits on a copy of the text, while keeping a character map back to the original so redactions land in the right place.
// "four one five, five five five, oh one nine two" -> "415 555 0192"
// Returns normalised text plus a map from normalised offsets to original offsets,
// so a match found in the normalised copy can be redacted in the original.
export function normaliseSpokenNumbers(text: string): { text: string; map: number[] } {
/* word-to-digit table ("oh" -> 0, "double five" -> 55, "twenty twenty" -> 2020),
applied over token windows; offsets tracked per token */
}Pass 2: entities
Names and addresses don’t have shapes. A named-entity recognition model finds people, locations and organisations. On its own it’s noisy in this domain: it misses unusual names and flags medication names as people. So its output is combined with context from the conversation itself: the word after “my name is” or “I live on” is a strong signal.
Pass 3: known values
This is the pass that made the biggest difference. During the call, the agent looked the patient up in the EHR. That means the system already knows this caller’s name, date of birth, MRN and phone number. The scrubber takes those values and searches for them directly in the transcript, including spoken and slightly misheard variants.
Detecting a value you already know is far more reliable than detecting “something that looks like a name”. It catches the cases the generic passes miss, which are exactly the ones that matter.
Replace, don’t delete
Detected spans are replaced with typed placeholders: [NAME], [DOB], [PHONE], [ADDRESS], [MRN]. Deleting them would destroy the structure that makes transcripts useful. “Patient [NAME] calling to reschedule their appointment on [DATE]” is still a perfectly good input for a summary, an analytics rule or an eval.
Where a downstream use genuinely needs consistency within one transcript (so that the same person isn’t mistaken for two people) we use numbered placeholders scoped to that transcript only: [NAME_1], [NAME_2]. The numbering is never stable across transcripts, because that would make it a linking key.
Audit counts, never values
Each scrubbed transcript gets an audit record: which detector fired, how many spans of each type were removed, and the scrubber version. It never stores the removed values. That lets us answer “is the scrubber working?” and “what changed when we upgraded it?” without creating a second copy of the PHI we just removed.
Testing it like a security control
A scrubber is a control, so it’s tested like one:
- A labelled test set of synthetic transcripts, written to include the hard cases: spoken numbers, spelled-out names, addresses split across turns, a date of birth said as “the third of March, eighty-four”.
- Recall is the headline metric. A missed identifier is a leak; an extra redaction is an inconvenience. We tune for recall and accept some over-redaction.
- Every miss found in review becomes a test case.
Where the raw data lives
De-identification only helps if the raw data is contained. Raw transcripts sit in one restricted store with short retention and access limited to the services that need it for the live call and for scrubbing. Analytics, summaries and eval datasets read only from the redacted store. That boundary is what turns “we scrub transcripts” into something you can show an auditor.
The honest caveat
Safe Harbor is a floor, not a guarantee. Free-text conversation can identify someone through a combination of details that aren’t on any list: a rare condition plus a small town plus an unusual job. That’s why the scrubber is one control among several (access limits, retention, and keeping redacted data internal) and not the only thing standing between a transcript and the outside world.