Whether you build a wrist-worn PPG device, an ECG patch, a sleep wearable or an EEG headset, investors, pharma sponsors, health systems and regulators all want the same answer: does your device measure what you say it measures, in the people you say it works for? This guide walks through the standard framework for answering that question, plus the study-design choices that decide whether your evidence holds up.
1. The framework: V3 and V3+
The Digital Medicine Society (DiMe) and collaborators defined V3 as the foundation for showing that a sensor-based digital health technology is fit for purpose (Goldsack et al., npj Digital Medicine, 2020):
| Step | Question it answers | Where it happens |
|---|---|---|
| Verification | Does the sensor capture raw data accurately and reliably? | Bench testing, in silico and in vitro, at the level of sensor output |
| Analytical validation | Does the algorithm turn raw data into an accurate physiological or behavioral metric? | In humans, against a reference standard |
| Clinical validation | Does the metric identify, measure or predict a meaningful clinical state in your intended population and context of use? | In the target population |
In 2024, DiMe added a fourth component, usability validation, creating V3+. It asks whether intended users can use the device, app, instructions and workflow correctly in real conditions (npj Digital Medicine, 2024; DiMe). DiMe recommends starting with a use specification and use-related risk analysis, iterating with formative testing, and finishing with a summative human-factors validation (DiMe sDHT Navigator).
The common mistake is treating "validated" as a single claim. A sensor can be well verified while its algorithm fails analytically in a new population, or the metric can be accurate yet clinically meaningless for your use case. Plan evidence for each layer.
If your wearable will collect data in a clinical trial, FDA's final guidance Digital Health Technologies for Remote Data Acquisition in Clinical Investigations (December 2023) is the reference document sponsors will hold you to (FDA).
2. Choosing a reference standard
Analytical validation compares your device against a reference, and that choice decides how credible your results are.
- Match the reference to the measurand. Common choices are an ECG for heart rate, arterial blood gas (SaO2) for pulse oximetry, polysomnography or a clinically validated actigraph for sleep, and a standardized clinical assessment for neurological function. Mike's own peer-reviewed work compared Apple Watch sleep monitoring with a clinically validated actigraph (Roomkham, Hittle et al., F1000Research, 2019).
- Use recognized standards where they exist.
- Consumer heart-rate wearables: ANSI/CTA-2065-A sets definitions and performance criteria under controlled lab conditions (CTA), and ANSI/CTA-2065.1-2023 covers real-world conditions (CTA).
- Automated blood pressure devices: ISO 81060-2 governs clinical investigation of non-invasive sphygmomanometers (ISO).
- Pulse oximeters for medical purposes: follow FDA's draft guidance (below).
Reference standards by modality
ECG patch (heart rate and rhythm). For heart-rate and arrhythmia claims, an ambulatory ECG (patch or Holter) gives continuous beat-level ground truth during free-living wear. Apple's AFib History Feature study used the Cardea SOLO Wireless ECG Patch (K123217) as its reference device (FDA MDDT SEBQ). Plan for lead placement, skin preparation, wear time, and expert or adjudicated rhythm annotation of the reference.
Polysomnography (sleep). In-lab PSG is the clinical reference for sleep staging. Plan for scorers blinded to device output, a documented scoring manual and version, and epoch-by-epoch alignment. Actigraphy is a lighter but lower-fidelity comparator: Mike co-authored a peer-reviewed comparison of Apple Watch sleep monitoring against a clinically validated actigraph (Roomkham, Hittle et al., F1000Research, 2019).
Clinical EEG (neurotech). For wearable or dry-electrode EEG, simultaneous recording against a conventional clinical EEG system is the usual comparator. Pre-specify what "agreement" means for your claim: signal-level similarity, event detection (for example seizures or sleep spindles) against expert review, or a derived index. For event-detection claims, document the number and qualifications of the expert reviewers and how disagreements are resolved.
General practices
- Synchronize time carefully. Small clock offsets between the device and the reference can make a good device look bad.
- Pre-specify your agreement metrics (for example bias and limits of agreement, mean absolute error, or root-mean-square error) and your acceptance thresholds before collecting data.
- Be honest about imperfect references. When no gold standard exists, as with many neurological and behavioral measures, document why you chose your comparator and what its limits are.
Precedent: a wearable feature qualified as an FDA development tool
On May 1, 2024, FDA qualified Apple's Atrial Fibrillation History Feature as a Medical Device Development Tool (MDDT). It was the first digital health technology qualified under the MDDT program (FDA bulletin). The qualified context of use is narrow: it is a biomarker test for estimating AFib burden as a secondary effectiveness endpoint in clinical studies of cardiac ablation devices (FDA SEBQ). The supporting study used 280 subjects and an ECG patch reference. Limits of agreement for weekly AFib burden were −11.4% to 12.8%, with a mean difference of 0.67% (95% CI −0.05% to 1.38%) (FDA SEBQ). The lesson for wearable companies: a tightly defined context of use, a recognized reference and Bland-Altman-style agreement evidence are what regulators can qualify.
What agreement and detection results look like (illustrative examples)
These figures are illustrative examples built from open research data. They don't show any company's device, and the simple algorithms behind them aren't validated products.
3. Picking the study population
Report who you studied. DiMe's DATAcc systematic review of analytical validation studies of sensor-based digital health technologies found that "only 9% of the studies included reported race and/or ethnicity" (DiMe Analytical Validation Library). Report demographics, and stratify results by them.
A device validated on healthy young volunteers at rest says little about older patients or people moving through daily life. Build diversity into the design from the start.
Skin tone
Optical sensors (PPG, SpO2) are the clearest example. FDA states that "current scientific evidence suggests there are some accuracy differences in pulse oximeter performance between individuals with lighter and darker skin pigmentation" (FDA, Jan 6, 2025). Its January 2025 draft guidance for medical pulse oximeters proposes (FDA; draft guidance):
- clinical data across the full range of skin pigmentation
- more study participants
- both a subjective scale (Monk Skin Tone) and an objective measure (Individual Typology Angle, ITA)
- reporting MST and ITA values for each participant (draft PDF)
As of October 2026, FDA's guidance page still lists this document as a draft (FDA). Even when the rules don't strictly apply, capturing skin tone objectively is a sensible design default for any optical sensor.
Age
Age affects skin, vasculature, gait and wear behavior. Enroll across the age range of your intended users and prespecify subgroup analyses. If your market is Medicare-age patients, a cohort of mostly younger volunteers is a weak foundation.
Activity and context
Report accuracy separately for rest, everyday activity and vigorous movement, since motion is a major source of error for optical sensors. CTA's separate lab and real-world heart-rate standards reflect the same distinction (CTA-2065-A; CTA-2065.1).
Clinical population
Clinical validation belongs in the population you will claim, such as people with a specific neurological condition, not just healthy proxies. Also plan for the realities of remote, wearable-enabled studies described in FDA's decentralized-trials guidance (Federal Register, Sept 2024).
4. Study-plan template (outline)
Download the full template: Study Plan + Statistical Analysis Plan template (.md) · (.docx)
The outline at a glance:
- Intended use and context of use. What the metric is, who it's for, and what decision it supports.
- Claim and evidence tier. Wellness, digital endpoint, or medical/SaMD claim, and which V3+ layers this study covers.
- Device and algorithm description. Hardware, firmware and algorithm versions, which should be locked for the study.
- Verification summary. Bench results this study builds on.
- Reference standard. Choice, justification, applicable standard (CTA, ISO, FDA guidance) and synchronization method.
- Study population. Inclusion and exclusion criteria, plus diversity targets for skin tone (MST + ITA), age, sex and clinical status.
- Protocol conditions. Rest, activity and free-living segments; sensor placement; duration.
- Primary and secondary endpoints and acceptance criteria. Prespecified agreement metrics and thresholds.
- Sample size rationale. Driven by the precision needed overall and within key subgroups.
- Statistical analysis plan. Agreement (Bland-Altman, MAE), detection (sensitivity/specificity, ROC/AUC), subgroup analyses, and handling of missing data and non-wear. The SAP is detailed in Part B of the template.
- Usability validation. Use-related risks, formative findings, summative test plan.
- Data management and integrity. Electronic records, audit trails, de-identification.
- Ethics. IRB, informed consent or eConsent.
- Reporting plan. What gets published and what goes into regulatory or partner submissions.
Data sources and attribution
Figures 1 and 2 are illustrative examples computed by Palo Alto Health Labs from open PhysioNet databases, made available under the Open Data Commons Attribution License v1.0. The analyses are our own; the dataset authors and PhysioNet do not endorse them.
- Pimentel MAF, et al. Towards a robust estimation of respiratory rate from pulse oximeters. IEEE Transactions on Biomedical Engineering 2016;64(8):1914–1923. doi:10.1109/TBME.2016.2613124. Data: BIDMC PPG and Respiration Dataset v1.0.0, PhysioNet.
- Moody GB, Mark RG. A new method for detecting atrial fibrillation using R-R intervals. Computers in Cardiology 1983;10:227–230. Data: MIT-BIH Atrial Fibrillation Database v1.0.0, PhysioNet.
- Goldberger AL, Amaral LAN, Glass L, et al. PhysioBank, PhysioToolkit, and PhysioNet: components of a new research resource for complex physiologic signals. Circulation 2000;101(23):e215–e220. doi:10.1161/01.CIR.101.23.e215
Related
Why Palo Alto Health Labs
Palo Alto Health Labs was founded by Mike Hittle, PhD, who earned his PhD in Epidemiology and Clinical Research at Stanford University School of Medicine. He works across wearables, mobile health, neurology and health AI. His work includes:
- founding CardinalKit, an open-source platform for regulated mobile health apps (JAMIA Open, 2023)
- creating the PHENOMS eCOA platform for neurology
- co-founding and co-directing Stanford's CS342 "Building for Digital Health"
- current postdoctoral research in the Ramayya Lab at Stanford Neurosurgery, developing a wearable-based neurological exam for the ICU
- peer-reviewed wearable research, including Apple Watch sleep validation (F1000Research, 2019) and wearable physiology before sudden cardiac death (JACC: Case Reports, 2026)
Planning a validation study, or not sure which evidence your claim needs? Book a free 30-minute scoping call.
This guide is educational and isn't regulatory or legal advice. Regulatory status reflects sources checked in October 2026.