TY - JOUR
T1 - Sociotechnical feasibility of natural language processing-driven tools in clinical trial eligibility prescreening for Alzheimer’s disease and related dementias
AU - Idnay, Betina
AU - Liu, Jianfang
AU - Fang, Yilu
AU - Hernandez, Alex
AU - Kaw, Shivani
AU - Etwaru, Alicia
AU - Padilla, Janeth Juarez
AU - Ramírez, Sergio Ozoria
AU - Marder, Karen
AU - Weng, Chunhua
AU - Schnall, Rebecca
N1 - Publisher Copyright:
# The Author(s) 2024. Published by Oxford University Press on behalf of the American Medical Informatics Association.
PY - 2024/5/1
Y1 - 2024/5/1
N2 - Background: Alzheimer’s disease and related dementias (ADRD) affect over 55 million globally. Current clinical trials suffer from low recruitment rates, a challenge potentially addressable via natural language processing (NLP) technologies for researchers to effectively identify eligible clinical trial participants.Objective: This study investigates the sociotechnical feasibility of NLP-driven tools for ADRD research prescreening and analyzes the tools’ cognitive complexity's effect on usability to identify cognitive support strategies.Methods: A randomized experiment was conducted with 60 clinical research staff using three prescreening tools (Criteria2Query, Informatics for Integrating Biology and the Bedside [i2b2], and Leaf). Cognitive task analysis was employed to analyze the usability of each tool using the Health Information Technology Usability Evaluation Scale. Data analysis involved calculating descriptive statistics, interrater agreement via intraclass correlation coefficient, cognitive complexity, and Generalized Estimating Equations models.Results: Leaf scored highest for usability followed by Criteria2Query and i2b2. Cognitive complexity was found to be affected by age, computer literacy, and number of criteria, but was not significantly associated with usability.Discussion: Adopting NLP for ADRD prescreening demands careful task delegation, comprehensive training, precise translation of eligibility criteria, and increased research accessibility. The study highlights the relevance of these factors in enhancing NLP-driven tools’ usability and efficacy in clinical research prescreening.Conclusion: User-modifiable NLP-driven prescreening tools were favorably received, with system type, evaluation sequence, and user's computer literacy influencing usability more than cognitive complexity. The study emphasizes NLP's potential in improving recruitment for clinical trials, endorsing a mixed-methods approach for future system evaluation and enhancements.
AB - Background: Alzheimer’s disease and related dementias (ADRD) affect over 55 million globally. Current clinical trials suffer from low recruitment rates, a challenge potentially addressable via natural language processing (NLP) technologies for researchers to effectively identify eligible clinical trial participants.Objective: This study investigates the sociotechnical feasibility of NLP-driven tools for ADRD research prescreening and analyzes the tools’ cognitive complexity's effect on usability to identify cognitive support strategies.Methods: A randomized experiment was conducted with 60 clinical research staff using three prescreening tools (Criteria2Query, Informatics for Integrating Biology and the Bedside [i2b2], and Leaf). Cognitive task analysis was employed to analyze the usability of each tool using the Health Information Technology Usability Evaluation Scale. Data analysis involved calculating descriptive statistics, interrater agreement via intraclass correlation coefficient, cognitive complexity, and Generalized Estimating Equations models.Results: Leaf scored highest for usability followed by Criteria2Query and i2b2. Cognitive complexity was found to be affected by age, computer literacy, and number of criteria, but was not significantly associated with usability.Discussion: Adopting NLP for ADRD prescreening demands careful task delegation, comprehensive training, precise translation of eligibility criteria, and increased research accessibility. The study highlights the relevance of these factors in enhancing NLP-driven tools’ usability and efficacy in clinical research prescreening.Conclusion: User-modifiable NLP-driven prescreening tools were favorably received, with system type, evaluation sequence, and user's computer literacy influencing usability more than cognitive complexity. The study emphasizes NLP's potential in improving recruitment for clinical trials, endorsing a mixed-methods approach for future system evaluation and enhancements.
UR - http://www.scopus.com/inward/record.url?scp=85190882464&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85190882464&partnerID=8YFLogxK
U2 - 10.1093/jamia/ocae032
DO - 10.1093/jamia/ocae032
M3 - Article
C2 - 38447587
AN - SCOPUS:85190882464
SN - 1067-5027
VL - 31
SP - 1062
EP - 1073
JO - Journal of the American Medical Informatics Association : JAMIA
JF - Journal of the American Medical Informatics Association : JAMIA
IS - 5
ER -