How sample sizes versus scan times affect data quality and radiology research overhead costs
Contrary to popular belief, completing longer scans on fewer patients may be more beneficial for data collection than conducting shorter exams on more patients.
The advancement of artificial intelligence largely depends on the data it is trained on, as it requires massive amounts to develop algorithms with generalizability. This has forced researchers to make a choice on whether to prioritize scan speed (to gather data from more patients) or sample size (to gather more quality data from fewer patients) for their studies.
New work published in Nature may help address the issue at hand.
“There is a fundamental asymmetry between sample size and scan time per participant owing to inherent overhead cost associated with each participant that can be quite substantial, for example, when recruiting from a rare population,” Nico U. F. Dosenbach, MD, PhD, with the Mallinckrodt Institute of Radiology at Washington University School of Medicine, St Louis, and co-authors noted. “Notably, the exact trade-off between sample size and scan time per participant has not been comprehensively characterized. This trade-off is not only relevant for small-scale studies, but also important for large-scale data collection, given competing interests among investigators and limited participant availability.”
The group developed a mathematical model to determine whether individual-level phenotypic prediction accuracy of AI models increases with sample size and scan times. The model can determine AI’s accuracy for predicting 76 phenotypes from nine resting-fMRI and task-fMRI datasets. What’s more, it integrates data across a diverse set of patients, scanners, sequences and disorders.
Initially, the model helped the team determine that, when scanning patients for less than 20 minutes, accuracy increases in line with the logarithm of the total scan duration; this suggests that scan time and sample size are interchangeable, but that sample size ultimately takes precedent. However, when overhead costs—a critical figure in research settings—are taken into consideration, the team concluded that longer scans on fewer patients provided ample data but at a substantially lower cost. In other words, scanning the same patients for longer provides the same predictive value as scanning more patients for less time, only with a less intimidating price tag.
“On average, 30 min scans are the most cost-effective, yielding 22% savings over 10 min scans,” the authors explained. “Overshooting the optimal scan time is cheaper than undershooting it, so we recommend a scan time of at least 30 min.”
The group suggested that this new knowledge has the potential to significantly benefit the field of neuroscience and could guide researchers in designing cost-effective, fruitful studies of the brain.
In addition to her background in journalism, Hannah also has patient-facing experience in clinical settings, having spent more than 12 years working as a registered rad tech. She began covering the medical imaging industry for Innovate Healthcare in 2021.