HIPAA-Compliant AI Tools
Managing HIPAA BAAs across multiple AI vendors
By Mat Steinlin, Head of Information Security
Last updated: July 2026
Getting a BAA with one AI vendor is a procurement task. Staying compliant across four of them is an ongoing operational problem.
Most healthcare teams don't end up with a single AI vendor. The more realistic process is: they start with one, add another for a specific use case, adopt a cloud provider's native model for convenience, and eventually find themselves with a matrix of agreements, data handling commitments, audit logging implementations, and key management configurations that all need to be kept current and consistent.
This is the normal state of a growing health tech company in 2026. But how do you manage it without it becoming a liability?
The multi-vendor reality
A hypothetical healthcare technology company's AI stack in 2026:
Claude for primary reasoning tasks and complex analysis
GPT-4o via Azure OpenAI for specific workflows or fallback
AWS Bedrock for model flexibility and cost optimization
Google Workspace Gemini embedded in the organization's productivity stack
GitHub Copilot in the engineering team's workflow
A clinical documentation tool used by clinicians
Each of these has its own BAA process, its own data handling commitments, its own audit logging approach, and its own key management requirements. None of them coordinate with each other.
This creates several categories of operational risk that compound as the stack grows.
The contract management problem
Every BAA has a counterparty, a set of covered services, and renewal and notification obligations. Keeping track of them manually is manageable for one or two vendors. At four or five, gaps appear.
Common failure modes:
Scope drift. A vendor adds new AI features to a product. The existing BAA was scoped to the original product. The new features are used by clinical staff before anyone checks whether they're covered. This is already happening with Slack AI, Notion AI, Zoom AI Companion, and similar features that get added to platforms with existing BAAs.
The same pattern applies to third-party services outside the AI stack. Sending an appointment reminder through a default Twilio configuration with the reason for the visit in the message body is a breach for every single text that went out. The default configuration wasn't designed for PHI. The gap between what a service can do and what it's configured to do safely is where exposure happens.
Renewal gaps. Enterprise BAAs typically have annual review cycles, and vendor terms can change at renewal. A vendor's terms change; the organization doesn't notice; practices that were covered under old terms continue under new terms that don't cover them the same way. This is especially common when vendors update subprocessor lists or narrow the scope of covered services in routine boilerplate updates that don't trigger a formal re-negotiation.
Employee-initiated tools. An engineer or clinician adopts a new AI tool without going through procurement. The tool may handle PHI. It has no BAA. See Shadow AI in healthcare.
Acquisition and service changes. AI vendors are being acquired, merged, and pivoted at a high rate. What happens to your BAA when the legal entity changes? What happens to your data?
The model-switching risk
A common mistake teams make is swapping models during testing or performance optimization and accidentally routing PHI through a path without BAA coverage.
Example: an engineer is optimizing costs and swaps a Claude call for a Llama 3 model accessed directly through a Hugging Face API. The implementation looks identical. The model is cheaper. The Hugging Face endpoint has no BAA. PHI flows through an uncovered path. No alert fires.
Or: a developer tests a new OpenAI model using their personal API key before the enterprise agreement is updated to cover that model. The test uses a sanitized dataset. But it runs in a shared environment where PHI-handling code also runs. The boundaries blur.
The risk is gap between the speed of AI development and the lag of compliance processes. When BAA coverage requires separate negotiations per vendor, per product version, and per new model, the compliance boundary trails the development velocity.
Strategies for managing multi-vendor compliance
Path 1: Consolidate on a cloud provider's model marketplace
The simplest compliance posture for teams already on a major cloud provider is to route all model traffic through that provider's managed model service. One cloud provider BAA covers access to multiple model providers through a single API.
AWS Bedrock: One AWS BAA covers most Bedrock-accessible models: Claude, Llama 3, Titan, Mistral, and others. Note that not every model on Bedrock is HIPAA-eligible: Fable and Mythos models are explicitly excluded from the AWS HIPAA eligible services list. Verify model-level eligibility before routing PHI. For teams already in an AWS enterprise agreement, this is typically part of standard onboarding. See AWS Bedrock for the compliance setup.
Azure OpenAI: One Microsoft enterprise agreement covers Azure OpenAI, which includes GPT-4o, o-series models, and other models hosted on Azure. For teams already in a Microsoft enterprise agreement, the same consolidation advantage applies. See Azure OpenAI.
Trade-offs: model availability lags direct provider releases by weeks to months, coverage is limited to what each marketplace supports, and you're tied to that cloud provider's infrastructure and pricing for model access.
If your model requirements are covered by what Bedrock or Azure OpenAI offer and your infrastructure already lives on one of those clouds, this is the most operationally straightforward path.
Path 2: Self-host models
Running inference on your own infrastructure (using frameworks like vLLM on your own AWS environment, for example) eliminates the external business associate question for the model layer entirely. If the model runs inside your own infrastructure, there's no third-party vendor receiving PHI at inference time.
If you have strict data residency requirements, this may be a good option for you.
Trade-offs: significant operational overhead (infrastructure management, model updates, availability, scaling), model quality and currency limitations relative to frontier models, and the application-layer compliance controls (audit logging, key management) remain entirely your responsibility. Self-hosting the model doesn't reduce that surface. It just removes one BAA from the stack.
Path 3: Use an LLM Gateway
An LLM Gateway is a compliance proxy that sits between your application and all LLM providers. Instead of your application calling each provider's API directly, all model traffic routes through the gateway, which handles compliance uniformly across vendors.
Aptible LLM Gateway provides this for healthcare teams:
One BAA covers all supported models. Your Aptible BAA covers all LLM Gateway usage, across Claude, OpenAI, and AWS Bedrock. You don't negotiate separate BAAs with each provider for traffic that runs through the gateway.
Audit logging is automatic and centralized. Every prompt and response, regardless of which model, is logged with timestamps, user attribution, model identity, and request metadata. Without a gateway, audit log formats differ across vendors. Each has different log access paths, different retention defaults, and different field schemas. When an incident occurs, identifying which vendor's logs are relevant requires understanding each vendor's logging architecture separately before reconstruction can begin. Encrypted at rest. Retained for six years per HIPAA documentation retention requirements (45 CFR § 164.530(j)).
PHI de-identification is built in. Identifiers are scrubbed before requests reach the model and restored after responses return. This is handled at the gateway layer, not in application code.
Key management is handled. API credentials for each underlying model provider are managed by LLM Gateway. Your application authenticates to the gateway, not directly to each vendor. Rotation, scoping, and revocation are centralized.
Model switching doesn't restart compliance. Moving from Claude to GPT-4o to Llama 3 is a configuration change, not a compliance event. Engineers can test new models without a compliance review for each model change.
For teams building agentic workflows
Multi-vendor BAA complexity compounds significantly with agentic AI.
An agent doesn't just make a single LLM call. It calls tools, queries data sources, takes actions, and may invoke sub-agents. Each of those operations is potentially in scope if PHI is present in the context. The compliance boundary expands with each capability the agent has.
Specific challenges in agentic architectures:
Tool calls. When an agent calls an external tool (a database query, an API, a code interpreter) that tool may receive PHI from the agent's context. Each external tool that processes PHI may need its own BAA.
MCP (Model Context Protocol). Teams building with MCP introduce new surfaces where PHI can flow through tool-calling architectures. The compliance requirements don't change, but the surface area does. The MCP security guide covers the specific security and compliance requirements for MCP deployments in regulated environments.
Multi-agent systems. When agents call other agents, each hop in the chain is potentially in scope. Audit logging for a multi-agent workflow needs to capture the full interaction chain, not just the top-level call.
Context persistence. Agent frameworks often maintain conversation history or memory across turns. If PHI enters the context window, it may persist across interactions. Audit logging needs to account for context, not just individual requests.
What to do this week
If your team is currently using more than one AI vendor with PHI:
Audit your BAA portfolio. List every AI vendor or tool that may receive PHI. Confirm you have a BAA with each. Note any gaps.
Check scope. For each BAA, verify that the specific product surface in use (API, chat, coding assistant, meeting tool) is covered by the agreement. BAAs scoped to one product don't automatically extend to new features.
Review your audit logging. For each vendor, confirm you have application-level logs of PHI interactions, not just infrastructure logs. Verify encryption, access controls, and retention.
Document your model inventory. Which models are in use in production? Which are in testing? Are all of them covered?
Establish a policy for new tool adoption. Before a new AI tool touches PHI, the BAA review happens. Not after.
Evaluate new vendors on two questions before anything else. First: can they show you exactly where PHI lives at every step of the pipeline? If they can't walk you through that, the answer is no. Second: have they done a BAA before, or are they googling what that means? These are the questions experienced healthcare IT practitioners use to screen vendors quickly. On audit logging specifically: audit logs and access controls should be table stakes, not a feature add. If a vendor treats that as an upsell, that's a red flag.
FAQs
Do I need a separate BAA for each AI vendor I use?
Yes, unless you're using an LLM Gateway that consolidates coverage. Every vendor that receives, accesses, or processes PHI on your behalf requires a BAA. Using four models through four vendors means four BAAs.
How do I track which AI vendors have BAAs?
A simple register works: vendor, product/service covered, BAA counterparty, effective date, renewal date, and covered services scope. The operational discipline is reviewing it when you add new tools and when vendors add new features to existing products.
What happens if I switch AI models mid-project?
If you're routing directly to vendors, switching models means checking whether the new model is covered under an existing BAA, which may require contacting the vendor. If you're using an LLM Gateway, switching models doesn't affect your compliance posture. Coverage is at the gateway layer, not the model layer.
Can one BAA cover multiple AI vendors?
Through an LLM Gateway: yes. Your BAA is with the gateway provider, and the gateway covers your model traffic across supported vendors. Without a gateway, each vendor requires a separate agreement.
What is an LLM Gateway and how does it help with HIPAA compliance?
An LLM Gateway is a proxy layer that sits between your application and LLM providers. It handles compliance uniformly: audit logging, PHI de-identification, key management, and BAA coverage across all traffic that flows through it. Aptible LLM Gateway is designed specifically for this use case in regulated environments.
What should I do if I discover we're using an AI tool with PHI and no BAA in place?
Stop sending PHI to that tool. Assess whether a breach notification may be required under HIPAA's Breach Notification Rule. Whether the exposure was a "breach" under HIPAA is not automatic. An impermissible use or disclosure is presumed to be a breach unless a risk assessment demonstrates there is a low probability the PHI was compromised. Factors include whether the PHI was actually acquired or viewed, who the unauthorized person was, and the nature of the information involved. (HHS Breach Notification Rule, 45 CFR §§ 164.400-414) Engage your compliance counsel. Then remediate: either execute a BAA with that vendor (if they offer one), or migrate those workflows to a covered tool.
What's next
For the full breakdown of what a HIPAA BAA covers and what it doesn't, see What an AI BAA actually covers.
For teams building agentic AI workflows where PHI flows through model-driven tool calls, see the MCP security guide for the compliance and security requirements specific to those architectures.
For platform-level compliance covering the infrastructure running your healthcare AI application, see Aptible for HIPAA-compliant infrastructure.