OpenAI and HIPAA: what a BAA covers and what your application still has to do

Note: this article explains the general framework HIPAA applies to AI vendor relationships. It is not legal advice and does not assess whether your specific application or configuration is compliant. Any determination about your organization’s HIPAA obligations should be made by your compliance and legal counsel.

If you’re building a healthcare application that sends any patient data to OpenAI, you’ve probably asked whether OpenAI is “HIPAA compliant.” That’s the wrong question — compliance isn’t a property a vendor has, it’s a property of how PHI is handled end to end, across your application and every system it touches. What OpenAI can offer is a contractual instrument (a BAA) covering their side of that chain. Whether your application is HIPAA compliant depends on what you build around it.

What OpenAI’s documentation says about HIPAA

According to OpenAI’s published documentation, a Business Associate Agreement (BAA) is available for the API without requiring an enterprise agreement — it is requested directly from the provider. This agreement, when signed, establishes OpenAI as a “business associate” under HIPAA, meaning they accept contractual responsibility for handling Protected Health Information (PHI) in accordance with the regulation.

One detail that is easy to miss: signing the BAA is not the only condition. That same documentation states HIPAA eligibility for the API is contingent on the account being provisioned with Modified Retention. A signed BAA on an account without that provisioning does not put you where you think you are — verify both, not just the agreement.

Note also that BAA availability differs by product. ChatGPT Enterprise and Edu require a sales-managed account; ChatGPT Business is not covered at all. If your team is pasting clinical text into a ChatGPT plan rather than calling the API, check which plan you are actually on.

This sounds reassuring. It isn’t — not by itself.

A BAA covers the provider’s systems and infrastructure as a contractual matter. It says nothing about how your application sends data. It does not prevent your prompt from containing a patient’s name, diagnosis, medication, or insurance number before it reaches the provider’s servers. The moment that data leaves your system, it has already moved — and whether it should have moved at all is entirely your problem.

What a BAA does (and doesn’t) cover

A Business Associate Agreement is a legal instrument. It establishes obligations, liability, and breach notification procedures. What it does NOT do:

• Prevent data from being transmitted — PHI in your prompt travels over the network to OpenAI regardless of any paperwork
• Guarantee technical controls — BAAs don’t specify encryption at rest, access controls within OpenAI, or how long inference data is retained
• Cover your application’s logic — if your code logs the prompt to a database, sends it to a webhook, or stores it in a browser session, the BAA is irrelevant to those flows

Under HIPAA’s Security Rule technical safeguard requirements, covered entities and their business associates must implement:

1. Access controls — unique user identification, automatic logoff, encryption
2. Audit controls — hardware, software, and procedural mechanisms for activity in systems containing ePHI
3. Integrity controls — protection against improper alteration or destruction of ePHI
4. Transmission security — protection against unauthorized access to ePHI during transmission

A BAA addresses none of this in your code.

Where PHI actually leaks

Here’s what developers often miss. The places PHI escapes your control aren’t always obvious:

Prompt content: The most direct path. A support chatbot that receives “My patient, John Smith (DOB 04/15/1962), is showing symptoms of…” sends that PHI verbatim to the provider the moment the request fires. Your BAA is somewhere in a PDF. John Smith’s data is already on the provider’s servers.

Fine-tuning datasets: If you fine-tune a model on clinical notes — even with a BAA in place — you need extraordinary guarantees about data segregation, retention, and deletion. Standard fine-tuning pipelines from most providers are not designed with medical data segregation in mind by default; verify explicitly with the provider before sending clinical text into a fine-tuning job.

Embeddings and vector stores: When you embed patient documents for RAG applications, the underlying text is sent to the embedding model. The resulting vectors are opaque, but the process exposes the raw content.

Application logs: Your own infrastructure often logs everything. The HIPAA risk isn’t just the model provider — it’s your own request/response logging middleware, your CDN’s access logs, your load balancer, your error tracking service.

The gap: business associate vs. technical prevention

Here’s the critical distinction that most compliance guides gloss over: signing a BAA with a provider makes that provider a business associate. It does not prevent PHI from being in the data you send them.

HIPAA’s Privacy Rule requires minimum necessary use. You’re supposed to send only the PHI necessary to accomplish the purpose. In practice, with LLM applications, the “purpose” is answering complex clinical questions — which often does require context. But does it require the patient’s name? Their address? Their Social Security Number? Almost never.

The minimum necessary standard is not about what the model needs. It’s about what PHI you have legal standing to transmit.

---

This is exactly the problem Privedge was built to solve. Privedge is an AI inference proxy that sits between your application and OpenAI (or any LLM provider). Before a prompt reaches the API, Privedge intercepts it, tokenizes every piece of PII and PHI using reversible anonymization, and forwards only the anonymized version. The response comes back with the same tokens, which Privedge replaces with the original values — in your environment, never theirs.

Switching is a single line of code:

// Before — PHI reaches OpenAI verbatim
const openai = new OpenAI({ apiKey: process.env.AI_KEY })

// After — PHI is intercepted before transmission
const openai = new OpenAI({
  apiKey: process.env.AI_KEY,
  baseURL: 'https://api.privedge.io/v1',
  defaultHeaders: { 'X-Privedge-Key': process.env.PRIVEDGE_KEY },
})

Your existing code doesn’t change. Your SDK calls don’t change. What changes is that “Analyze patient John Smith’s CBC results” becomes “Analyze patient [PERSON_1]‘s CBC results” before it ever leaves your network.

---

Practical compliance: what you actually need

A genuinely HIPAA-compliant AI application in 2026 requires:

1. A BAA with every service that touches ePHI — yes, including OpenAI
2. Technical controls that enforce minimum necessary — anonymization or strict data filtering before transmission
3. Audit logging on your side — who sent what, when, with what intent
4. A data flow map — every system that touches the data, with documented controls for each hop
5. Incident response procedures — what happens when something goes wrong

The BAA is item one on a five-item list. Without items two through five, item one provides legal cover that doesn’t match your technical reality — and that mismatch is where HIPAA violations live.

Conclusion

OpenAI can be a viable partner for building healthcare applications, and a signed BAA is a necessary prerequisite when PHI is involved. But calling your application “HIPAA compliant” because you have a BAA is like calling a car crash-safe because you have liability insurance.

The technical controls are your responsibility. The data minimization is your responsibility. The architectural decision about what PHI actually reaches the LLM provider is yours to make — and the safer default is: as little as possible, ideally none.

If you’re building clinical AI and want the BAA question off the critical path, Privedge provides proxy infrastructure that keeps PHI out of what reaches the provider in the first place, plus documentation to support your own compliance file.