Single-arm study design
DD4 · All, Diagnostic, Implantable, Digital Health / SaMD
MSAC consistently rejects clinical utility claims based on single-arm studies. Comparative evidence against current Australian standard of care is required. Single-arm studies are only accepted for safety and feasibility claims, not for clinical utility or cost-effectiveness.
Signal: Single-arm design proposed or completed with no comparator arm.
Risk of BiasIndirectness
No active comparator or wrong comparator
DD3 · All
The comparator must reflect current Australian clinical practice — not placebo, historical control, or international standard of care unless explicitly justified. MSAC has deferred multiple applications where the comparator was not the Australian standard of care.
Signal: Planned study uses placebo, no comparator, or a non-Australian standard of care.
Indirectness
Surrogate endpoints only
DD4 · Diagnostic, IVD, Digital Health / SaMD
MSAC requires patient-relevant outcomes — survival, quality of life, functional status, or validated intermediate endpoints with demonstrated linkage to patient outcomes. Diagnostic accuracy alone is insufficient without evidence of downstream patient benefit.
Signal: Only sensitivity/specificity or other surrogate reported, no linkage to patient outcome.
IndirectnessImprecision
Population not generalisable to Australia
DD5 · All
Studies conducted entirely in non-Australian populations require explicit bridging justification. MSAC has rejected or deferred applications where prevalence, treatment pathway, or clinical setting differed materially from Australian practice.
Signal: Evidence base is entirely international (particularly US/European) with no bridging argument.
Indirectness
No economic model or inadequate cost-effectiveness analysis
DD7 · All
MSAC requires a full health economic model for all MBS listing applications, using Australian costs and epidemiology, presenting cost per QALY or cost per clinical outcome. Absence of an economic model is an automatic rejection driver.
Signal: No economic model, or model not yet started/scoped.
Imprecision
Insufficient follow-up
DD6 · Implantable, All
MSAC has deferred applications where follow-up was insufficient to capture clinically meaningful outcomes. Implantable devices require a minimum 12 months; chronic condition devices require follow-up commensurate with the disease course.
Signal: Follow-up period under 12 months for an implantable, or short relative to the natural disease course.
Imprecision
Equity evidence absent
DD10 · All
MSAC requires explicit consideration of access for Aboriginal and Torres Strait Islander peoples and rural/remote populations. Applications that do not address equity of access are flagged, weighted heavily where the condition has disproportionate burden in these populations.
Signal: No equity evidence or access consideration included in the submission.
Clinical utility not established independently from safety
DD1 · Diagnostic, IVD
MSAC separates assessment of safety, effectiveness, and cost-effectiveness. A device shown to be safe and to perform as claimed has not thereby shown clinical utility — it must show it changes clinical management in a way that improves patient outcomes.
Signal: Evidence plan conflates device performance/safety data with a clinical utility claim.
Indirectness
Small sample size / underpowered studies
DD4 · All
MSAC has rejected applications where sample sizes were insufficient to detect a clinically meaningful difference. Studies powered for regulatory clearance are frequently underpowered for MSAC's comparative-effectiveness purposes.
Signal: Sample size set for regulatory clearance only, not powered for comparative effectiveness.
Imprecision
No pre-specified primary endpoint
DD4 · All
Post-hoc endpoint selection or composite endpoints without pre-specification are a recurring MSAC concern. The primary endpoint must be pre-specified, clinically relevant, and powered accordingly.
Signal: Primary endpoint not yet pre-specified, or endpoint selection deferred to analysis stage.
Risk of Bias
Evidence does not meet ESF tier requirement
DD1 · Digital Health / SaMD
NICE's Evidence Standards Framework (ESF) sets tier-specific requirements for digital health technologies. Tier A (self-care) needs acceptability/usability evidence; Tier B (guided self-care) needs effectiveness evidence; Tier C (treatment/clinical pathway) needs RCT-level or equivalent evidence. Claiming a higher tier than the evidence supports is a primary rejection driver.
Signal: Claimed ESF tier is higher than the evidence base supports.
Imprecision
No QALY-based economic model
DD7 · All
NICE requires a cost-utility analysis using QALYs, reference case threshold £20,000–£30,000/QALY (higher for end-of-life criteria: survival under 24 months, QALY gain ≥3 months). Applications without a QALY model are rejected at assessment stage.
Signal: No QALY model present, or economic model status is not started/scoped.
Imprecision
Non-UK population without bridging evidence
DD5 · All
Evidence from non-UK populations must be explicitly bridged to UK NHS clinical practice, addressing differences in care pathways, disease prevalence, and treatment standards. US evidence in particular frequently requires substantial bridging.
Signal: Evidence base is non-UK (especially US) with no bridging argument to NHS practice.
Indirectness
Surrogate endpoints not validated against patient outcomes
DD4 · Diagnostic, IVD, Digital Health / SaMD
NICE requires validated surrogate endpoints — evidence that the surrogate reliably predicts the patient-relevant outcome. Unvalidated surrogates lead to high uncertainty ratings and typically a not-recommended or research-only outcome.
Signal: Surrogate/biomarker endpoint used with no validation evidence linking it to patient outcomes.
Indirectness
No equality impact assessment
DD10 · All
Under the Equality Act 2010, NICE requires applicants to demonstrate consideration of impact on protected characteristics (age, sex, disability, ethnicity, etc). Missing equality impact assessments are a compliance gap that delays or blocks appraisal.
Signal: No equality impact assessment included in the submission.
Inadequate long-term safety data for implantables
DD2 · Implantable
For implantable devices, NICE and the MHRA require long-term safety evidence — short-term studies are insufficient. Implant registries, post-market surveillance data, and adverse event reporting are expected.
Signal: Safety data limited to pre-clinical or early clinical only, no long-term/registry data.
Imprecision
No real-world evidence for digital health post-deployment
DD6 · Digital Health / SaMD
For SaMD and digital health technologies, NICE increasingly requires real-world evidence of effectiveness in NHS settings following initial deployment. Bench performance and trial data alone are insufficient for sustained commissioning.
Signal: No real-world/post-deployment evidence plan for a digital health or SaMD claim.
Indirectness
Health economic model does not meet NICE reference case
DD7 · All
NICE's reference case specifies perspective (NHS and personal social services), time horizon (lifetime), discount rate (3.5% costs and outcomes), and utility measurement (EQ-5D). Deviations without justification result in model rejection.
Signal: Economic model deviates from NICE reference case parameters without justification.
Risk of Bias
No Patient Access Scheme when cost-effectiveness uncertain
DD9 · All
Where cost-effectiveness is uncertain or the ICER exceeds threshold, NICE expects a Patient Access Scheme proposal. Absence of a PAS when the base case ICER is above £30,000 is a negative signal.
Signal: Base case ICER likely to exceed £30,000/QALY with no PAS proposed.
Imprecision
Single-arm studies for treatment pathway devices
DD3 · Therapeutic / Surgical, Implantable
For devices in the treatment pathway (ESF Tier C), NICE requires comparative evidence. Single-arm studies demonstrating device performance are insufficient to establish comparative effectiveness against current NHS standard of care.
Signal: Single-arm design for a Tier C / treatment-pathway device claim.
Risk of BiasIndirectness
No comparator against current Hospital/Community Device Schedule item
DD3 · All
PHARMAC's Factors for Consideration require comparison against the current funded alternative (an existing Schedule device or standard hospital practice), not an unfunded or international comparator, unless a case is made for a first-in-class listing.
Signal: No head-to-head or cost comparison against the currently funded device/practice.
Indirectness
No cost-utility / budget impact analysis
DD7 · All
PHARMAC's investment approach weighs health need, health benefit, cost, and suitability within a fixed device/pharmaceutical budget. Applications without a budget impact analysis and cost-utility case are difficult to prioritise against competing investments.
Signal: No budget impact or cost-utility analysis prepared for the funding application.
Imprecision
Equity evidence absent for Māori and Pacific populations
DD10 · All
Under Te Tiriti o Waitangi obligations, PHARMAC requires explicit consideration of equity impact for Māori and Pacific peoples. Absent equity evidence can block funded access or generate binding conditions, consistent with the pattern seen across PHARMAC's pharmaceutical decisions.
Signal: No equity evidence or impact assessment for Māori/Pacific populations.
Population not generalisable to New Zealand clinical practice
DD5 · All
Evidence generated entirely offshore (commonly US, European, or Australian trials) requires bridging to the New Zealand health system, clinical pathway, and population profile before PHARMAC will treat it as directly applicable.
Signal: Evidence base is entirely non-NZ with no bridging rationale to NZ practice.
Indirectness
Insufficient evidence of health need or clinical benefit
DD1 · All
PHARMAC's Factors for Consideration weigh the health need of the population against the strength and directness of clinical benefit evidence. Surrogate-only or indirect evidence weakens the case relative to devices with direct, patient-relevant outcome data.
Signal: Clinical benefit case rests on surrogate or indirect endpoints rather than direct patient outcomes.
Imprecision
No suitability/implementation evidence for the NZ health system
DD8 · Implantable, Therapeutic / Surgical, Digital Health / SaMD
PHARMAC's suitability criterion considers practical implementation — training burden, supply chain, compatibility with existing hospital equipment/workflow. Devices with no NZ implementation or training plan face additional funding risk.
Signal: No implementation, training, or supply plan addressing NZ health system suitability.
Imprecision
No Coverage with Evidence Development (CED) pathway proposed
DD9 · Implantable, All
For novel or high-uncertainty devices, CMS frequently covers only under Coverage with Evidence Development — a national registry tracking safety/effectiveness outcomes for a defined period — rather than issuing an unconditional yes or no. Devices with no post-market data-collection plan face a higher risk of CED-only or non-coverage, as seen with TAVR and LAAC.
Signal: No registry, post-market surveillance, or data-collection plan proposed alongside the funding request.
Imprecision
No site-of-service, operator-volume, or team-based requirement addressed
DD8 · Implantable
CMS has repeatedly conditioned coverage for higher-risk implantables on multidisciplinary team evaluation (e.g. TAVR's heart-team requirement), facility accreditation, and operator case-volume minimums. Applications silent on these operational conditions are less likely to secure unconditional coverage.
Signal: No proposed site-of-service, credentialing, or team-based-review requirement for a higher-risk implantable.
Risk of Bias
FDA clearance assumed to establish CMS 'reasonable and necessary' evidence
DD1 · All
CMS's coverage standard is independent of FDA clearance/approval — a device can be legally marketable and still be found not reasonable and necessary for Medicare if outcome evidence for the specific population is lacking. CMS declined coverage for vagus nerve stimulation in treatment-resistant depression on exactly this basis, despite FDA approval for that indication.
Signal: Evidence plan treats FDA clearance as sufficient justification for Medicare coverage without population-specific outcome data.
Indirectness
No national coverage determination — relies on fragmented Local Coverage Determinations
DD9 · All
Many device categories have no National Coverage Determination at all, leaving coverage to Local Coverage Determinations set independently by each Medicare Administrative Contractor. A device can be funded in one region and denied in another with no single national resolution — a fragmentation risk that a funding strategy should explicitly address.
Signal: Funding strategy assumes a single national US coverage outcome without accounting for MAC-level LCD variation.
Inconsistency
No evidence positioning against the drug-based standard of care
DD3 · Implantable, Therapeutic / Surgical
Where a device proposes to replace or supplement a pharmaceutical standard of care (e.g. an anticoagulation alternative), CMS coverage has required evidence on appropriateness for patients who can and cannot tolerate the drug-based comparator long-term — not device performance data alone.
Signal: No evidence addressing patient suitability for, or intolerance of, the existing drug-based standard of care.
Indirectness
No defined population threshold for a borderline/intermediate subgroup
DD5 · All
CMS coverage often applies a specific clinical threshold (a risk score, symptom severity band, or audiometric cutoff) with a narrower, trial-only pathway for patients just outside it — as seen in cochlear implantation's borderline benefit-score carve-out. Evidence plans with no defined threshold or borderline-population strategy face ambiguous coverage at the margins.
Signal: No defined clinical threshold or subgroup criteria distinguishing clearly-covered from borderline patients.
Imprecision
No shared decision-making documentation for a preference-sensitive device
DD9 · Implantable
For elective, preference-sensitive procedures with a viable non-device alternative, CMS has made a documented shared decision-making conversation between clinician and patient a coverage precondition, not just a best-practice recommendation.
Signal: No shared decision-making documentation requirement built into the proposed care pathway.
Risk of Bias
No early CMS engagement for a Breakthrough Device under the TCET pathway
DD9 · All
CMS's 2024 Transitional Coverage for Emerging Technologies (TCET) pathway offers expedited Medicare coverage for FDA-designated Breakthrough Devices, but requires manufacturer self-nomination roughly 12 months ahead of expected FDA action. Breakthrough-designated devices with no CMS engagement plan risk a coverage gap after FDA clearance.
Signal: Device holds or is pursuing FDA Breakthrough Device designation with no corresponding CMS/TCET engagement plan.
Risk of Bias