Synthetic health data and federated learning for medical AI: when synthetic data helps, when it misleads, HIPAA status, FDA expectations and the federated alternative

The short answer

Synthetic health data is good for building and testing software, for demos and training, and as a supplement for rare classes. It is weak evidence that a model works on real patients. Both FDA and NIST point the same way: develop with synthetic data if it helps, but validate on real, independent data, and document how any synthetic data was made and why it is fit for purpose.

Synthetic data is not automatically private. Generators trained on patient records can memorize and leak them, and HIPAA has no separate synthetic-data category, so data derived from PHI is judged by the same de-identification standard as any other data. When the real goal is to learn from data that cannot leave a hospital, federated learning moves the model instead of the data. It reduces, but does not remove, privacy risk.

This guide covers synthetic and federated approaches only. For where to get real datasets (MIMIC-IV, All of Us, UK Biobank and others), see Health data sources for medical AI. For how many real test cases you need, see How big should a medical AI test set be?.

Sources: FDA draft guidance, AI-enabled device software functions (Jan 2025); NIST SP 800-188 (Sept 2023); 45 CFR 164.514 (eCFR).

What counts as synthetic health data

FDA's draft AI guidance defines synthetic data as data created artificially, for example through statistical modeling or computer simulation, intended to represent the structure, properties and relationships of actual patient data without containing real information about individuals. NIST SP 800-188 separates four kinds, and the difference matters for privacy:

Simulation-based generators sit in their own place. Synthea, an open-source generator from MITRE, simulates patient lifespans from models of disease progression and standards of care rather than copying records. Its paper describes modules for the 10 most frequent reasons for primary care encounters and the 10 chronic conditions with the highest morbidity in the United States, with output in HL7 FHIR, C-CDA and CSV.

Sources: FDA draft guidance, AI-enabled device software functions (Jan 2025); NIST SP 800-188 (Sept 2023); Walonoski et al., JAMIA 2018;25(3):230–238; Synthea (GitHub).

When synthetic data helps

• Test fixtures and development. Build and test ETL, FHIR interfaces, dashboards and labeling tools before any PHI is involved. Label synthetic values in the data itself, as NIST recommends (for example "SYNTHETIC PERSON"), so files are never mistaken for real records once separated from their documentation.
• Augmentation and rare classes. FDA's device research program notes that representative data can be burdensome to get because of acquisition cost, safety limits, privacy restrictions or low disease prevalence, and is studying synthetic (in silico) data to supplement patient data. Its T-SYNTH breast imaging dataset provides 9,000 synthetic digital breast tomosynthesis examples. Adding T-SYNTH to patient data matched or exceeded patient-data-only performance in some cases, not all.
• Stress tests and subgroup probes. Simulated cases can probe a model on rare presentations or controlled variations. FDA's generative AI discussion paper says synthetic inputs modeled on real patient data might, with justification, supplement evaluation where real inputs are limited. That paper is a request for feedback, not guidance.
• Hypothesis generation. Analysts can explore synthetic data and then confirm findings on real data. NIST SP 800-188 describes a validation service that runs queries built on synthetic data against the actual data.

Sources: NIST SP 800-188 (Sept 2023); FDA OSEL: Addressing the limitations of medical data in AI; FDA regulatory science tool T-SYNTH (2026); FDA generative AI discussion paper (2026).

When synthetic data misleads

• It only knows what the generator knows. NIST notes fully synthetic data cannot provide more information than the model that produced it, and that faithfully representing all properties of the original data while enforcing strong privacy guarantees is impossible.
• Outcomes and care variation are hard to simulate. A validation of a 1.2-million-patient Synthea Massachusetts cohort found colorectal cancer screening compliance of 63% vs. 77.3% reported for Massachusetts and 69.8% nationally, and COPD 30-day mortality of 0.7% vs. 7% in Massachusetts and 8% nationally. The authors found Synthea reliable for demographics and service probabilities but limited for heterogeneous outcomes after care.
• Domain gap hides real failures. FDA's T-SYNTH documentation warns that synthetic data may not represent the true patient population, risking missed variability or misjudged model performance, and says it cannot fully replace real patient data for evaluating breast imaging AI.
• Same-family generators repeat the same blind spots. FDA's generative AI discussion paper asks how to handle the risk that synthetic data made by models of the same class as the device reproduces the very performance gaps the evaluation should detect.
• No method wins everywhere. A benchmark of synthetic EHR generators on data from two academic medical centers found a utility-privacy tradeoff and no method best on all criteria in every use case.
• Training on generated data compounds. Shumailov et al. showed that indiscriminate training on model-generated content causes "model collapse," where the tails of the original distribution disappear. In medicine, the tails are often the rare patients you most need.

Sources: NIST SP 800-188 (Sept 2023); Chen et al., BMC Med Inform Decis Mak 2019;19:44; FDA regulatory science tool T-SYNTH (2026); FDA generative AI discussion paper (2026); Yan et al., Nat Commun 2022;13:7609; Shumailov et al., Nature 2024;631:755.

Re-identification and memorization risk

"Synthetic" describes how data was produced, not how private it is. NIST SP 800-188 says both high-fidelity models and the synthetic data they generate may leak personal information that is potentially re-identifiable, that formal privacy models such as differential privacy can control the leakage, and that for deep learning models it may be difficult to quantify how much training data the model memorizes and can regenerate.

• Memorization. Carlini et al. extracted over a thousand training examples from image diffusion models and found them much less private than earlier GANs. In language models, they extracted hundreds of verbatim training sequences, including personal information that appeared in just one document, and larger models were more vulnerable.
• Membership inference. Shokri et al. showed attackers can tell whether a record was in a model's training set, including on a hospital discharge dataset. Being in a disease-specific training set can itself be sensitive.
• Privacy gain is hard to predict. Stadler et al. evaluated synthetic data publishing and found it either did not prevent inference attacks or did not keep data utility, with a privacy-utility tradeoff that is hard to predict.

Practical controls: measure nearest-neighbor distance and duplicate rates between synthetic and training records, run membership inference tests against the generator, consider differential privacy when training it, treat the generator itself as sensitive, and restrict who can query it.

Sources: NIST SP 800-188 (Sept 2023); Carlini et al., arXiv:2301.13188; Carlini et al., arXiv:2012.07805; Shokri et al., arXiv:1610.05820; Stadler et al., arXiv:2011.07018.

HIPAA status of synthetic data

Guidance only, not legal advice. The HIPAA Privacy Rule's de-identification standard (45 CFR 164.514) does not mention synthetic data, and neither does HHS's de-identification guidance. Information is de-identified if it does not identify an individual and there is no reasonable basis to believe it can be used to identify one, shown by one of two methods:

• Expert Determination: a qualified expert determines the risk is very small that the information could identify an individual, alone or with other reasonably available information, and documents the methods and results.
• Safe Harbor: the 18 listed identifiers are removed and the covered entity has no actual knowledge that the rest could identify someone.

What that means in practice:

• A generator trained on PHI is built by using PHI, so training it is a use of PHI under the normal rules. HHS says a covered entity may have a business associate de-identify PHI on its behalf only to the extent the business associate agreement authorizes it.
• Synthetic output derived from PHI should not be assumed to be de-identified. Removing the 18 Safe Harbor identifiers does not address memorized records, and Safe Harbor also requires no actual knowledge that the rest could identify someone, so plan on an Expert Determination to show the risk is very small. HHS notes there is no explicit numerical risk level that universally meets that bar, and that properly de-identified data still carries some risk.
• Data simulated without patient records, such as Synthea output, contains no real individuals.
• State privacy laws and contracts such as data use agreements may impose terms beyond HIPAA. For consumer health apps outside HIPAA, see Health apps HIPAA doesn't cover.

Sources: 45 CFR 164.514 (eCFR); HHS de-identification guidance; Walonoski et al., JAMIA 2018;25(3):230–238.

What FDA expects

• Describe synthetic data. FDA's January 2025 draft guidance on AI-enabled device software functions asks submissions to describe any synthetic data used, with a comprehensive explanation of how it was generated and why it is fit for purpose. The guidance was still a draft when we checked on Oct 27, 2026.
• Validate on real, independent data. The same draft asks for test data independent of training data (for example, from completely different clinical sites), sequestered from developers, and warns that leakage between development and validation data creates uncertainty about true performance. It says reliance on a single site is generally not appropriate and that at least three geographically diverse US clinical sites or health systems may be appropriate to clinically validate an AI-enabled device.
• Representative data and independent tests. The Good Machine Learning Practice principles (FDA, Health Canada and MHRA, 2021; an IMDRF final version followed in January 2025) call for data representative of the intended patient population, training sets independent of test sets with patient, acquisition and site dependence considered, and testing under clinically relevant conditions.
• Generative AI is still open. FDA's 2026 discussion paper on generative AI-enabled devices floats synthetic data as a supplement for evaluation when it is representative, and asks how to account for differences between synthetic and real distributions. It is a request for feedback, not draft or final guidance.

To plan the real test set, see test set sample size and validating a clinical prediction model. For whether your product is a device at all, see Is my app a medical device?.

Sources: FDA draft guidance, AI-enabled device software functions (Jan 2025); FDA: Good Machine Learning Practice guiding principles; FDA generative AI discussion paper (2026).

Federated learning: move the model, not the data

McMahan et al. named federated learning in 2016: training data stays where it is, each site trains locally, and a shared model is learned by aggregating the updates. In health, the appeal is that hospitals keep control of their data. Rieke et al. describe FL as training collaboratively without exchanging the data itself.

• It has worked at scale. The EXAM study used data from 20 institutes to predict oxygen needs of patients with COVID-19 and reported an average AUC above 0.92 at 24 and 72 hours, a 16% improvement in average AUC and a 38% average increase in generalizability over single-site models, with validation at three hospitals not used for training. The FeTS glioblastoma study used 71 sites across 6 continents (n = 6,314) and reported 33% and 23% delineation improvements over a publicly trained model. EXAM was organized and coordinated by NVIDIA, and FeTS included Intel-affiliated authors; both papers disclose this.
• It does not fix data quality. Rieke et al. state FL does not solve issues inherent to medical data: training still depends on data quality, bias and standardization, and heterogeneous data can mean the global model is not optimal for each site.
• Updates can leak. Zhu et al. recovered private training data from shared gradients, pixel-wise accurate for images. Rieke et al. note models can memorize information under certain conditions. Proposed defenses include differential privacy, learning from encrypted data and trusted execution environments; FeTS ran its aggregator in trusted hardware. Rieke et al. note privacy techniques trade off against accuracy.
• It needs infrastructure and agreements. Sites need on-premise or private-cloud compute, standardized data formats, and agreements on governance and how optimality is defined before training.

Sources: McMahan et al., arXiv:1602.05629; Rieke et al., npj Digit Med 2020;3:119; Dayan et al., Nat Med 2021;27:1735; Pati et al., Nat Commun 2022;13:7346; Zhu et al., arXiv:1906.08935.

Synthetic, federated or real: a quick guide

Checklist

• Write down why you need synthetic data and which type (test, realistic, partially or fully synthetic).
• If a generator is trained on PHI, confirm the use is permitted and covered by your BAA or data use agreement.
• Document how the data was generated, its source data, and why it is fit for purpose, as FDA's draft guidance asks.
• Test the generator for memorization and membership inference before releasing output; consider differential privacy.
• Do not call synthetic output de-identified without an Expert Determination or Safe Harbor analysis.
• Label synthetic records in the data itself.
• Never evaluate only on synthetic data. Keep a sequestered, real, multi-site test set.
• Report results on real data separately from any synthetic results, including by subgroup.
• For federated projects, agree on governance, data formats and privacy defenses before training.

Key references

• 45 CFR 164.514 (eCFR) and HHS de-identification guidance: the HIPAA de-identification standard and its two methods.
• NIST SP 800-188, De-Identifying Government Datasets (Sept 2023): synthetic data types, leakage and validation.
• FDA draft guidance, AI-enabled device software functions (Jan 2025): synthetic data description and independent test data (draft).
• FDA: Good Machine Learning Practice guiding principles: GMLP principles on representative and independent data.
• FDA regulatory science tool T-SYNTH (2026): FDA's synthetic breast imaging dataset and its stated limits.
• Chen et al., BMC Med Inform Decis Mak 2019;19:44: validating Synthea against clinical quality measures.
• Stadler et al., arXiv:2011.07018: privacy evaluation of synthetic data publishing.
• Rieke et al., npj Digit Med 2020;3:119: federated learning for digital health, benefits and limits.

Related resources

• Healthcare AI consulting
• HIPAA compliance consulting
• Medical data labeling
• Technical due diligence
• Health data sources for medical AI
• How big should a medical AI test set be?
• Medical imaging data: DICOM de-identification and annotation
• LLM APIs, PHI and HIPAA BAAs
• Local validation of vendor AI models
• Digital health due diligence checklist