Using AI with protected health information is not simply a matter of getting a Business Associate Agreement signed. HIPAA compliance for AI requires architectural decisions: where data is processed, whether the model trains on PHI, how audit logs are maintained, and how breach notifications would work if something goes wrong. This guide gives healthcare technology leaders the framework to evaluate any AI vendor — and the questions they must ask before signing.
HIPAA's requirements for AI systems are not explicitly spelled out in the statute — it was written before modern AI existed. But the existing Privacy, Security, and Breach Notification Rules apply fully to AI systems that process, store, or transmit protected health information. Understanding what each rule requires architecturally is the foundation for evaluating any AI vendor's HIPAA claims.
The HIPAA Privacy Rule defines what counts as PHI: any individually identifiable health information created, received, maintained, or transmitted by a covered entity or business associate. In an AI context, this is broader than most organisations realise. A patient query to an AI chatbot ("What is my appointment tomorrow?") is PHI. A clinical note fed to an AI for coding assistance is PHI. A scheduling record that includes a diagnosis code is PHI. Even a patient name combined with a treatment date is PHI. The Privacy Rule limits how PHI can be used and disclosed — and "disclosing" PHI to a cloud AI vendor for processing is a disclosure that must be covered by a BAA and satisfy the minimum necessary standard.
The HIPAA Security Rule sets out what technical safeguards must protect electronic PHI. For AI systems, the material requirements are: access controls (unique user identification, automatic logoff, encryption and decryption), audit controls (hardware, software, and procedural mechanisms that record and examine activity in systems that contain PHI), integrity controls (ensuring PHI is not improperly altered or destroyed), and transmission security (encryption of PHI in transit). For an AI system, this means the model must run in an environment where access is gated by user authentication, every access to PHI by the AI is logged, and any transmission of PHI to or from the AI is encrypted end-to-end. Shared cloud AI infrastructure typically cannot satisfy the audit control requirement with the granularity HIPAA requires.
The Breach Notification Rule creates notification obligations when PHI is improperly accessed, used, or disclosed. If a cloud AI vendor experiences a security breach affecting your patient data, you — the covered entity — have mandatory notification obligations to affected individuals, HHS, and potentially media. This is why the question of where PHI is processed is not merely a technical preference but a financial risk management decision. A breach involving PHI processed in a vendor's shared cloud infrastructure triggers the same notification requirements as a breach of your own systems — but with the added complexity that the vendor controls the forensic investigation timeline. The Business Associate Agreement requirement means any third party that processes PHI on your behalf must sign a BAA. What a BAA covers: it legally obligates the business associate to safeguard PHI, use it only for authorised purposes, and report breaches. What a BAA critically does not cover unless explicitly contracted: it does not prevent the AI model from training on your PHI. OpenAI's standard enterprise BAA does not include a no-training commitment unless you are on a specific enterprise arrangement with that commitment explicitly contracted. Many healthcare organisations discover this gap only after an audit or incident reveals that patient data was used for model improvement.
The majority of AI tools available in 2025 — including leading consumer and SMB platforms — fail one or more of the four core HIPAA architectural requirements. Healthcare organisations that have adopted these tools based on marketing claims of "HIPAA compliance" (which often mean only that a BAA is available) are carrying compliance risk they may not have fully assessed.
Consumer and SMB tiers of GPT-4, Gemini, and Microsoft Copilot process data in shared cloud infrastructure. PHI in a prompt — a patient question, a clinical note, a scheduling record — is PHI in the vendor's cloud the moment it is submitted. Regardless of a BAA, the data has left your controlled environment and is subject to the security controls of the vendor's shared infrastructure. These controls may be strong, but they are the vendor's controls, not yours, and you cannot independently audit them to the depth HIPAA's audit requirements assume.
Unless you are on a no-training enterprise contract with an explicit contractual commitment, your data may be used to improve model performance. This applies to many AI platforms by default. PHI used for model training without patient authorisation is a HIPAA violation — it constitutes a use of PHI for a purpose not covered by the Privacy Rule's treatment, payment, and operations exception. The model improvement purpose is not a covered HIPAA use. Healthcare organisations must verify, in writing and in contract, whether their AI vendor's model trains on customer data — and this must be confirmed for the specific tier they are using, not the enterprise tier they have not subscribed to.
HIPAA's Security Rule requires audit controls that record and examine activity in systems containing PHI. The required granularity: who accessed which PHI, when, and in what context. Most AI tools provide aggregate usage logs — number of queries, tokens consumed, API call volumes. They do not provide patient-level audit logs that show which patient's data was accessed in which AI interaction, by which user. This is not a configuration issue — it is an architectural limitation of how shared cloud AI systems work. HIPAA compliance requires patient-level PHI access logging, and most AI platforms simply do not produce it.
A HIPAA-compliant AI architecture is not more complex than a non-compliant one — it is different. The four requirements below are non-negotiable for any AI system that processes PHI. Each one eliminates a distinct failure mode that creates regulatory exposure.
The AI model runs inside your perimeter — in your data centre or in a private cloud environment (a dedicated VPC) that your organisation governs. PHI never leaves your infrastructure to reach an external AI service. This single architectural decision eliminates cloud processing exposure and puts your organisation in control of the security environment the AI operates within. It also makes your compliance posture independently auditable — you can demonstrate to HHS, to a HITRUST assessor, or to your own compliance team exactly where PHI is processed and under what controls.
The AI agent uses a locally deployed model — no API calls to OpenAI, Anthropic, Google, or any external AI service. The model weights are deployed on your infrastructure and run entirely within your environment. When a clinician asks the AI to draft a discharge summary, that query and the patient data it references never leave the hospital network. This requirement eliminates both the cloud processing exposure and the model training risk in a single architectural decision. The locally deployed model is fine-tuned on your clinical workflows, your documentation standards, and your terminology — producing better outputs for your specific context than a generic cloud model would provide.
Who can invoke the AI agent, what patient data it can access, and what actions it can take are all gated by the same RBAC system your EHR uses. A clinical documentation agent accessed by a physician should be able to access that physician's patient panel's records. A billing agent should be able to access claims data but not clinical notes beyond diagnosis codes. A patient-facing appointment agent should access scheduling data but not full clinical history. These access boundaries are enforced at the agent level, not just the application level, and every access decision is logged for audit. This satisfies HIPAA's minimum necessary standard and its access control requirements simultaneously.
Every agent query, every data access event, every action taken is logged with four mandatory attributes: timestamp, user identity (the human who invoked the agent), data accessed (which patient record, which PHI fields), and action taken (what the agent read, wrote, or transmitted). Logs are immutable — they cannot be altered or deleted by any system user, only by a logged administrative action. These logs satisfy HIPAA's audit control requirements and provide the forensic record needed for breach investigation and OCR inquiries. They are also the foundation for your own AI governance programme — the ability to audit what the AI is doing with PHI on an ongoing basis, not just after an incident.
The following use cases are fully deployable within a HIPAA-compliant architecture. Each one involves PHI processing — clinical notes, patient records, insurance data, scheduling records — and each is designed with on-premise deployment, no-training guarantees, RBAC, and immutable audit logging as standard.
Physician note drafting, ICD-10 coding assistance, and discharge summary generation — all processing PHI entirely on-premise. AI dramatically reduces the documentation burden that consumes 30–40% of physician time, while producing structured, compliant documentation. The agent works within your EHR environment, accessed through your existing authentication, with every clinical note access logged at the patient level.
Insurance pre-authorization requests — extracting clinical data from patient records, mapping to payer criteria, and drafting submission packages — with a complete compliance audit trail. Prior auth is one of the most time-intensive administrative burdens in US healthcare; AI reduces the time per request from hours to minutes while maintaining the documentation standard payers require. Every record access and submission is logged for compliance review.
Appointment reminders, care plan follow-up messages, no-show re-booking, and post-discharge check-ins — using HIPAA-compliant messaging channels (encrypted SMS, secure patient portal). The agent personalises communications based on the patient's care plan and history, improving engagement while maintaining the access controls and audit logging that HIPAA requires for PHI-containing communications.
Claim preparation, denial triage, appeals letter generation, and payer follow-up — integrated with your billing system and governed by the same RBAC and audit framework as clinical AI. Revenue cycle AI can reduce denial rates by 20–30% through more accurate initial claim preparation and faster, more comprehensive appeals. All billing data access is logged at the patient and claim level for HIPAA audit purposes.
Patient routing, referral management, and specialist handoff coordination — across care team members and sites of care. Care coordination AI maintains a shared view of the patient's care pathway that updates in real time as new encounters occur. All care team members with appropriate access can query the agent for current patient status, pending referrals, and outstanding care plan items. Every access is tied to a user identity and logged.
HEDIS measure calculation, CMS quality reporting programme submissions, and patient safety event documentation — AI agents that query clinical records, calculate measure performance, and produce formatted submissions. Quality reporting is one of the most resource-intensive compliance obligations in healthcare; AI reduces the analyst time required per reporting cycle while improving accuracy. All PHI accessed for quality reporting is logged under the healthcare operations exception to HIPAA's minimum necessary standard.
Healthcare technology leaders have a fiduciary obligation to evaluate AI vendors against HIPAA requirements, not just marketing claims. The following eight questions are the minimum due diligence for any AI vendor that will process PHI. A vendor that cannot answer these questions clearly and in writing should not be given access to patient data.
Questions healthcare technology leaders ask most often about deploying AI in compliance with HIPAA requirements.
ChatGPT in its standard consumer and SMB tiers is not HIPAA compliant. OpenAI does offer a Business Associate Agreement for enterprise customers on specific tiers, but a BAA is only one component of HIPAA compliance. HIPAA also requires that PHI not be used for model training without patient authorisation, that audit logs track every PHI access event, and that data is processed in an environment with appropriate access controls.
Even on enterprise tiers with a BAA and a no-training commitment, the data transits OpenAI's cloud infrastructure — which may not satisfy data localisation requirements for healthcare organisations subject to state health data privacy laws more stringent than HIPAA. Healthcare organisations that use ChatGPT for PHI-involving tasks should not assume BAA availability equals compliance.
An AI tool is HIPAA compliant when it satisfies four architectural requirements: PHI is processed in an environment under your organisation's control; the AI model does not train on your PHI (explicitly contracted, not assumed); all PHI access events are logged at patient level with user identity and timestamp; and access to the AI system is governed by role-based access controls consistent with HIPAA's minimum necessary standard.
Meeting all four requirements simultaneously is the test. Many AI tools satisfy one or two but fail others — typically on audit trail granularity or model training defaults. A vendor claiming HIPAA compliance should be able to demonstrate each of these four requirements with technical documentation, not just contractual assertions.
No. A BAA is necessary but not sufficient. A BAA establishes legal accountability and documents the business associate's obligations — it does not by itself ensure compliant architecture. Specifically, a BAA does not prevent model training on your data unless that is explicitly contracted. It does not ensure data localisation. It does not generate patient-level audit logs.
Healthcare organisations that treat a signed BAA as the endpoint of HIPAA compliance analysis create significant exposure. The BAA is one layer of a multi-layer compliance requirement — the architectural requirements (on-premise processing, no-training, RBAC, audit logs) are equally mandatory and must be verified independently of the BAA.
Yes — AI can be used in healthcare without HIPAA concerns when deployed for use cases that do not involve PHI. Scheduling optimisation using aggregate data, supply chain management, administrative workflow automation without patient data, and staff training applications that use synthetic data all fall outside HIPAA's scope.
For any use case that involves patient data — clinical documentation, prior authorisation, patient communication, care coordination, revenue cycle management — HIPAA requirements apply fully. The practical solution for most healthcare organisations is to deploy all AI in a HIPAA-compliant architecture from the outset, rather than attempting to segregate PHI and non-PHI workloads across different tools with different compliance postures.
HIPAA is the law — it sets the minimum legal requirements for protecting PHI. HITRUST CSF (Common Security Framework) is a certification framework — it provides a structured, independently audited approach to demonstrating HIPAA compliance and broader information security controls against over 150 control categories.
For AI vendors in healthcare, HITRUST CSF certification is a significantly stronger signal than self-certified HIPAA compliance, because it requires third-party validation. HITRUST Type I is based on a self-assessment against the framework; HITRUST Type II requires an independent assessor to validate the controls are in place and operating effectively. When evaluating AI vendors, ask for their HITRUST CSF status and their SOC 2 Type II report — both provide independently verified evidence of security controls.
Yes — AI can process clinical notes under HIPAA when deployed in a compliant architecture. Clinical notes are among the most sensitive PHI categories, containing diagnosis details, treatment descriptions, and clinical observations. Processing them with AI requires the model to run in your controlled environment, to have no training on patient notes, for every access to be logged at the patient level, and for access to be governed by RBAC consistent with the treating clinician's role.
When these conditions are met, AI for clinical documentation — physician note drafting, ICD-10 coding assistance, discharge summary generation — is legally compliant and highly valuable. Physicians typically spend 1–2 hours per day on documentation that AI can handle in minutes, reclaiming time for direct patient care.
If an AI vendor as a business associate experiences a breach involving your patient PHI, your organisation has mandatory HIPAA Breach Notification Rule obligations. The vendor must notify you within 60 days of discovering the breach. You then must assess the breach, notify affected individuals (within 60 days), report to HHS, and if 500 or more individuals in a state are affected, notify local media.
HHS OCR may investigate and issue civil monetary penalties up to $1.9 million per violation category per year. This is why the architectural decision — whether PHI ever leaves your perimeter — is a material financial risk management decision, not just a compliance checkbox. A breach involving PHI in a vendor's cloud triggers the same notification and enforcement exposure as a breach of your own systems, but with less control over the investigation timeline.
On-premise deployment is not explicitly required by HIPAA — the regulation is technology-neutral. However, on-premise or private cloud deployment is the most reliable way to satisfy the full set of HIPAA architectural requirements, because it gives your organisation complete control over where PHI is processed, who can access it, and how it is logged.
Private cloud deployment in a dedicated VPC governed by your organisation can also satisfy HIPAA requirements when properly configured. What HIPAA does require — appropriate administrative, physical, and technical safeguards — is most clearly demonstrable with on-premise or private cloud architectures. Shared cloud AI infrastructure is the deployment model least likely to satisfy the full audit and access control requirements, and the most difficult to defend to regulators if something goes wrong.
Use cases, ROI benchmarks, and deployment patterns for AI agents across clinical, administrative, and revenue cycle workflows.
→The architecture that keeps PHI inside your perimeter — how Upcore deploys AI within your data centre or private cloud.
→Upcore's full suite of HIPAA-compliant AI solutions for health systems, hospital networks, and ambulatory care organisations.
→The five decisions every healthcare enterprise must make before deploying AI — from build vs. buy to governance frameworks.
→Upcore builds and deploys AI agents for healthcare organisations with PHI staying 100% within your perimeter. Our on-premise deployment model satisfies HIPAA's architectural requirements from day one — no retrofitting, no compliance gaps.