Clinical Trial Design and Feasibility

Question
How many patients could a trial actually enroll, and when, once eligibility over time, new patients and the age of the data are taken into account?
Status
Proposed research: Proposal submitted
Approach
Eligibility tracked over time in EHR data, incident patients and data freshness in the estimate, calibration against observed trial accrual
Updated

A UF-FSU CTSA pilot proposal, From Population to Participant: Calibrating Real-World Data for Trial Feasibility and Accrual, has been submitted. It would turn one-time database counts into time-varying estimates of recruitment opportunity and check them against real accrual.

A one-time count versus recruitment opportunity over timeSynthetic curves over 24 months. A one-time database count is flat. Patients already in the record who remain eligible decline over time, while newly eligible patients accumulate, so the true opportunity first falls below and later rises above the one-time count. A database that is refreshed with a delay shows fewer patients than are really available; the shaded band is the difference.0612182401×Months after the trial opensEligible patients, relative to the countRecruitment opportunityOne-time query countWhat the data showExisting patients still eligibleA one-time count versus recruitment opportunity over time0612182401×Months after the trial opensRecruitment opportunityOne-time query countWhat the data showExisting patientsNot yet visible in the data
6 months

The shaded band is the part of the opportunity a delayed data refresh cannot yet see.

Synthetic example

From a one-time count to recruitment opportunity over time

Curves come from a simple model and show shape only; they are not estimates for any trial or site. The control changes how far the data lag behind real time.

Figure description

Over 24 months a one-time count is flat at 1. Existing patients who remain eligible decline steadily, newly eligible patients accumulate, and their sum, the recruitment opportunity, dips below the count in the first year and rises above it in the second. A dashed line shows what a database refreshed six months late would report; it falls below the true opportunity, and the gap widens as the delay grows.

Work in this direction

  1. Published

    Eligibility criteria and generalizability

    Established work

    Studies of how eligibility criteria shape who can join a trial, using real-world data from a large clinical research network, for colorectal cancer and Alzheimer's disease trials.

  2. Proposal submitted

    From a one-time count to recruitment opportunity

    Proposed research

    Aim 1 of the pilot proposal. Convert a static count into an estimate over time that follows eligibility as it changes, adds newly eligible patients and accounts for how current the data are.

  3. Proposal submitted

    Calibration against real accrual

    Proposed research

    Aim 2 of the pilot proposal. Compare estimated opportunity with observed accrual to measure the error, describe its uncertainty and show where limited observability constrains the estimate.

Why this question

Feasibility is often judged by running a trial’s eligibility criteria against a clinical database once and reporting how many patients match. That number answers a narrower question than the one a study team has. It counts people who matched at one moment, using only the criteria that structured data can express, in data that may be months old. It does not say how many people will still be eligible when screening happens, how many new patients will appear while the trial is open, or how many of them will be seen where the trial is running.

From a database count to an enrolled participantSteps from a database query to enrollment, with the reason people leave at each step.Counted by a querystructured criteriacriteria the datacannot expressEligible at screeningall criteria, that dayno longer, ornot yet, eligibleSeen at a sitewhere the trial runscare receivedelsewhereApproachedidentified in timemissed ornot contactedConsenteddeclinedEnrolledFrom a database count to an enrolled participantCounted by a querystructured criteriacriteria the datacannot expressEligible at screeningall criteria, that dayno longer, ornot yet, eligibleSeen at a sitewhere the trial runscare receivedelsewhereApproachedidentified in timemissed ornot contactedConsenteddeclinedEnrolled

Conceptual illustration

From a database count to an enrolled participant

Each step loses people for a different reason, and only the first step is visible in a one-time query. No counts are shown because none are implied.

Figure description

Six boxes run left to right: counted by a query, eligible at screening, seen at a participating site, approached, consented and enrolled. Below each of the first five, a dotted line leads to the reason people leave at that step: criteria the data cannot express, no longer or not yet eligible, care received elsewhere, missed or not contacted, and declined.

The proposal

From Population to Participant: Calibrating Real-World Data for Trial Feasibility and Accrual is a pilot proposal submitted to the UF-FSU Clinical and Translational Science Award program. It is under review; nothing on this page is a result.

The first aim treats eligibility as something that changes. A patient who matches today may not match in three months, and a patient who does not exist in the data today may be diagnosed next month. The estimate therefore follows eligibility over time, adds incident patients as they appear, and accounts for how far the data lag behind the clinic. The second aim tests that estimate against what actually happened, comparing estimated opportunity with observed accrual to measure the error and its uncertainty, and to show where the data cannot see what recruitment depends on.

Methods

  • Eligibility as a state over time. Each criterion is evaluated on a calendar, with its own lookback window, rather than once at query time.
  • Incident patients. Newly diagnosed or newly eligible patients enter the estimate as the trial runs.
  • Data freshness. The estimate is shifted by the delay between care and its appearance in the research data, as in the figure above.
  • Calibration and uncertainty. Estimated opportunity is compared with observed accrual, and the remaining error is reported with its uncertainty rather than folded into a single number.

Earlier work this builds on

My doctoral research asked how eligibility criteria shape who can join a trial. One study assessed an a priori patient-trial generalizability score with real-world data from a large clinical data research network, using a colorectal cancer trial as the case; it placed second in the AMIA 2019 student paper competition. Another used real-world data to examine eligibility criteria design in Alzheimer’s disease trials, followed by studies of how criteria affect participants’ age and the severity and adverse events of eligible patients. Later work on power and sample size for longitudinal and clustered designs (GLIMMPSE 3) addressed the planning side of the same problem.

Next steps

If the pilot goes ahead, the first products will be a written specification of the time-varying estimate and a calibration report that states where it works and where it does not. Either way, the methods and their limitations will be described here as they are developed.

Published work from my record in the areas this direction draws on. None of it reports results of the work above.

Sources