HIPAA Compliant LLMs in 2026: Which AI Can Touch Patient Data, Which Can't Along With What Your Team Needs to Know

reviewed: July 2026. Reflects AWS Bedrocks February 2026 HIPAA addition, inflation-adjusted OCR penalty tiers along with the AI Provider Trust Registry July 5 BAA status update.

Someone on your team has already pasted data into an AI tool.

That's not an accusation. It's a statistical certainty. Surveys of staff across multiple health systems found a majority using consumer AI tools for work tasks without IT approval. A significant share of those tasks touched information. The tools worked. The compliance exposure was real and invisible to everyone involved.

Here's what makes this particularly hard to catch. ChatGPT Enterprise looks like ChatGPT Plus. Claude through AWS Bedrock looks like Claude.ai Pro. Same interface, same outputs, completely different legal standing. The gap isn't in what you see on screen. It's in whether a Business Associate Agreement exists along with whether the infrastructure behind it's configured correctly.

OCR imposed $4.18 million in HIPAA penalties across 13 enforcement actions in the recent enforcement year. Nearly double the prior years total. Than 375 million individuals were impacted by healthcare breaches in 2025. 978 Large healthcare data breaches are currently under investigation. This isn't a slowing trend.

What a BAA Does Along With What It Doesn't

A Business Associate Agreement transfers obligations to the AI vendor. It doesn't make your deployment compliant by itself.

Many U.S. Developers mistakenly believe that signing a BAA with OpenAI along with AWS makes their application compliant. It doesn't. AWS along with Google guarantee the security of the server along with the encryption of the connection. The developer is responsible for identity management prompt logging along with ensuring no PHI leaks through the system prompt along with user inputs.

A BAA is the starting point for building a deployment. Everything after it. Encryption configuration, access controls, audit logging, staff tool policy. Belongs to your organization.

What Violations Actually Cost in 2026

The inflation-adjusted penalty tiers as of January 28 2026: Tier 1 (Lack of Knowledge) runs $141 to $36,298 per violation. Tier 2 (Cause) runs $1,452 to $72,596 per violation. Tier 3 (Willful Neglect, Corrected) runs $14,522 to $72,596 per violation.

The per-violation structure is what makes AI-related exposure particularly severe. Each conversation containing PHI sent to a -BAA endpoint is a potential separate violation. A coordinator using Claude.ai Pro for work across three months could represent hundreds of individual violations before anyone identifies the pattern. The healthcare industrys average breach cost hit $7.42 million in 2025. The highest of any sector for 14 years. And OCR fines are typically the smallest piece of that total.

Which LLMs Are HIPAA Eligible in 2026

As of July 5 2026 the AI Provider Trust Registry tracked 16 AI model offerings: 6 with public BAA evidence 6 conditional along with sales-gated along with 4 with no evidence at all.

Platforms With Clear Public BAA Coverage

AWS Bedrock is the practical starting point for most healthcare teams. You sign the AWS Business Associate Addendum self-serve via AWS Artifact. No sales call required no cost. That agreement extends to Bedrocks model sub-processors including Anthropic, Meta along with Mistral when you're running a HIPAA- model in a HIPAA-eligible configuration.

The part that catches teams: not every model along with endpoint on Bedrock qualifies. AWS added Bedrock along with Bedrock AgentCore to its HIPAA Eligible Services Reference in February 2026. Verification against their published list is required before PHI flows through. Skip that step along with you may believe you're covered when you're not. The signature is easy. The verification is what matters.

Azure OpenAI Service sits under Microsofts healthcare BAA, which's typically part of an existing enterprise agreement for organizations already standardized on Microsoft infrastructure. Your data isn't used to train OpenAIs models by default along with the data residency controls give you contractual clarity that holds up in an audit.

Google Cloud Vertex AI covers Vertex AI, the Gemini API on Vertex along with the Healthcare API under Google Clouds BAA. Two things have to be true for this to work: the BAA must be in place at the Google Cloud organization level along with the regulated-data flag must be enabled at the project level for PHI-bearing workloads. Vertex AI FedRAMP High authorization wasn't generally available as of May 2026 which matters for healthcare contractors with those requirements.

Claude via AWS Bedrock, Gemini via Vertex AI Cohere via AWS Bedrock along with Llama via AWS Bedrock are all covered under the BAAs above. Meta offers no BAA. Llama can only touch PHI through AWS Bedrock.

Platforms Where a BAA Exists But Requires Legwork

OpenAI signs BAAs for the API platform through request along with for ChatGPT Enterprise along with ChatGPT for Healthcare through sales-managed accounts. Consumer ChatGPT in every tier is ineligible.

Anthropic offers HIPAA- Claude through enterprise plans on direct contracts along with more commonly through AWS Bedrock along with Vertex AI along with Azure where the cloud BAA covers the model. Consumer Claude.ai has no BAA path under any plan.

Mistral via Azure AI along with Llama via Azure AI are conditional on Azure enterprise BAA coverage. Neither Meta along with Mistral offers their direct BAA.

Which LLMs Can't Touch Patient Data

No BAA Path at All

xAI (Grok) API: No BAA as of July 2026. That's the picture.

Mistral La Plateforme: No BAA on Mistrals own hosted platform. Mistral through Azure is a situation. The hosted service can't touch PHI.

Cohere API (standalone). No BAA evidence separate from the AWS Bedrock deployment.

DeepSeeks hosted API along with web application send conversations to servers in China. No BAA exists. No contractual data handling guarantee exists. This isn't a call. It's an absolute prohibition for any workflow touching patient data. The open-weight DeepSeek model self-hosted on your HIPAA-eligible infrastructure is an entirely different situation because the model provider never touches your data in that setup.. The hosted service is off the table.

Consumer Tiers

ChatGPT Free, Along with Team. Claude.ai Free, Pro along with Team. Gemini.google.com. None of these are HIPAA eligible. No enterprise contract your organization holds changes that.

This is the widespread compliance gap in healthcare right now. The enterprise agreement is signed. Staff use the consumer tier because its faster. The organization thinks its covered. It's not along with the two things aren't related to each other.

Five Architectures for HIPAA-Compliant LLM Deployment

AWS Bedrock with BAA

Self-serve BAA, VPC isolation, multiple models under an agreement. The practical option for most healthcare teams building PHI-touching features. Works well for: healthcare software products, clinical decision support, revenue cycle AI along with any workload needing Claude along with Llama along with Cohere under one agreement. Verify each model against AWSs HIPAA- list before PHI flows through it. That check is not optional.

Azure OpenAI with BAA

The path if you're already on Azure infrastructure. Contractual data handling clarity along with other OpenAI models, auditable guarantees. Works well for: teams in the Microsoft ecosystem along with use cases requiring OpenAI models specifically.

Google Vertex AI with BAA

Good fit for teams already using Google Cloud. Geminis long context window handles clinical document work well. FedRAMP High authorization wasn't generally available as of May 2026. Rules Vertex AI out for healthcare contractors with those requirements.

Private Cloud, with Open-Weight Models

By the half of 2026 Meta Llama 4 Alibaba Qwen 3.5 Mistral Large 3 along with Google Gemma 4 had all shipped with permissive licenses along with production-grade quality. When you serve these on your GPU infrastructure no model provider ever touches PHI. No model BAA needed because theres no model vendor in the chain.

You own the Security Rule story: TLS termination, authentication, audit logging along with patching. This is the call for organizations with strong internal ML engineering capacity along with high data sensitivity requirements. It's the call for teams without dedicated infrastructure expertise because the ongoing maintenance burden is real.

Air-Gapped Enclave

The model runs in an isolated environment with no internet connectivity. Nothing goes out. This is typically reserved for data, behavioral health records along with PHI subject to state-level protections that exceed HIPAAs federal floor. It's the expensive along with operationally complex option on this list.

Technical Safeguards That Are Your Responsibility of Vendor

Signing a BAA with AWS along with Azure along with Google doesn't transfer your obligations to them.

Encryption means AES-256 for PHI at rest along with TLS 1.2 minimum for PHI in transit. That applies to data going into the model outputs coming along with any intermediate storage of prompts along with responses.

Prompt engineering controls matter more than most teams realize. PHI shouldn't appear in system prompts where it can be avoided. System prompts often get logged by default in development tooling along with can surface through error messages. Patient-specific data should enter through controlled user message channels not hardcoded into system configuration.

Zero data retention is a configuration state, not a default. Eligible API endpoints support it. Verify your specific endpoint is configured correctly along with document that configuration for audit purposes.

Audit logging is non-negotiable. Who queried the model, when what category of prompt what the response was. Tamper-evident. Retained for HIPAAs six-year minimum.

Observability tools along with analytics platforms need their BAAs when healthcare application telemetry contains PHI.

Role-based access controls should be built in from the first sprint not added later. Tie access levels to clinical along with administrative roles. Review grants quarterly. A billing coordinator doesn't need access to a documentation AI.

No-training guarantee needs to be written into the BAA. Every major enterprise provider offers this at enterprise tier. Get it in the contract. It's a required element of compliance documentation.

On de-identification: HIPAA defines two methods. Safe Harbor removes all 18 specified identifiers along with Expert Determination uses risk analysis. In a RAG setup your compliance is only as strong as your vector database. Text clinical notes routinely leak identifiers that automated scrubbers miss. Embeddings generated from PHI are themselves PHI under HIPAA. Formal validation of your -identification process is required, not assumed.

The Four Patterns That Actually Generate Violations

Shadow AI is an ergonomics problem that shows up as a compliance problem. A physician whose organizations AI platform doesn't work properly on mobile will use Claude.ai Pro on their phone. That's not malicious. Fix the UX on approved tools so staff aren't motivated to find alternatives.

The subcontractor gap catches teams that got the contract right along with missed everything downstream. If a logging tool, analytics platform along with error monitoring service handles healthcare application telemetry containing PHI each one needs its BAA. The gap is almost never the model provider. It's usually three layers into the stack.

Tier confusion is the common audit failure mode. A team assumes BAA coverage when actual coverage is feature-specific along with configuration-dependent. Document which product tier along with which endpoints are covered under each BAA. Review it when vendors update their lineup. They update it often.

New features without review is the slow-accumulation risk. AI vendors ship capabilities continuously. A feature not in scope when the BAA was signed may now handle PHI. Assign someone to monitor vendor feature releases against your BAA scope. Thirty minutes a month is significantly easier than explaining the gap to OCR.

HIPAA BAA Status: Quick Reference

Where Fornex Fits In

We build healthcare software. HIPAA compliance architecture is part of the engineering from the sprint not a review that happens before launch.

The teams we talk to aren't being reckless. They're moving fast on features while the compliance infrastructure lags. A developer picked AWS Bedrock because it was familiar but didn't verify the model was on the HIPAA-eligible list. A product manager enabled an AI summary feature without knowing the logging configuration captured PHI in plaintext. Neither of these is unusual. Both are fixable before a product ships.

When we build AI features for healthcare clients the work covers platform selection, BAA verification, encryption configuration prompt engineering controls, audit logging along with staff tool access policy. Not as a compliance track. Baked into how the product gets built.

If your organization is building AI features touching data along with you're not confident the current setup would hold up in an audit that conversation is worth having now.

Our Healthcare Software Development team builds HIPAA- AI infrastructure from the start. Reach out through our contact page for a look at where your current architecture stands.

For the governance framework covering AI vendor accountability read: AI Governance in Healthcare Is No Longer Optional

About the Author

Written by the Fornex Health technical team. Fornex builds HIPAA- software along with AI infrastructure for hospitals, health systems along with digital health companies. Connect on LinkedIn along with follow the Fornex Health blog.

Frequently Asked Questions

What is a HIPAA compliant LLM?

No LLM carries HIPAA certification on its own. What exists is a HIPAA- deployment: a model running through a BAA-covered endpoint wrapped in access controls audit logging along with no-training guarantees. Compliance is a property of the deployment, not the model.

Is ChatGPT HIPAA compliant?

Consumer ChatGPT. Free, Plus along with Team. Is not HIPAA eligible along with can't be used with PHI. ChatGPT Enterprise along with ChatGPT for Healthcare are eligible through a sales-managed BAA process. The same GPT-4o model also runs on Azure OpenAI under Microsofts enterprise BAA. Using any consumer tier with data is a violation regardless of what an enterprise agreement covers elsewhere.

Is AWS Bedrock HIPAA compliant?

Yes, since February 2026 when AWS added Bedrock to its HIPAA Eligible Services list. The BAA is self-serve through AWS Artifact at no cost. Not every model along with endpoint configuration on Bedrock qualifies. Verify your setup against the published list before PHI flows through it.

Can I use DeepSeek for healthcare AI?

DeepSeeks hosted API along with web interface route data to servers in China with no BAA. That makes it off-limits for PHI. The open-weight DeepSeek model self-hosted on your HIPAA-eligible infrastructure is a different situation. In that setup the model provider never touches your data.. The hosted service can't handle patient data.

Does signing a BAA make my deployment automatically compliant?

No. The BAA transfers obligations to the vendor. Your organization still owns encryption, access controls, audit logging along with staff access policy. A BAA is where the compliance work starts, not where it ends.

What are the 2026 HIPAA penalty amounts?

Tier 1 (Lack of Knowledge): $141 to $36,298 per violation. Tier 2 (Cause): $1,452 to $72,596 per violation. Tier 3 (Willful Neglect, Corrected): $14,522 to $72,596 per violation. Each conversation containing PHI sent to a -BAA endpoint is a potential separate violation. OCR along with state AGs can both impose penalties for the breach.

Can open-source LLMs handle HIPAA workloads without a BAA?

Yes, when self-hosted on infrastructure you control. When the model runs on your servers under a cloud provider BAA, the model vendor never touches PHI. So there's no model BAA to sign. You take responsibility for encryption access controls along with audit logging.

What's the difference between HIPAA along with HIPAA compliant?

HIPAA eligible means the platform will sign a BAA along with meets the contractual requirements to handle PHI. HIPAA compliant describes a deployment thats correctly configured with all required safeguards in place. A platform can be eligible while a configured deployment on that same platform creates violations.