Process Data for Modern Educational Assessment and Learning

This research project will use process data to develop tools for educational assessment and adaptive design for testing and learning. With the increasing use of computer-based testing, a variety of behavioral data have been collected. This project will focus on developing methods to conduct accurate assessments and deliver effective personalized learning materials/interventions. These methods will be based on process data collected in log files of computer-based tests. Specific topics to be addressed include the analysis of process data via statistical learning methods, development of process-data-based assessments, and adaptive learning through process data. The results of this research will provide a deeper understanding of students' behaviors and cognitive processes in an environment increasingly defined by technology-based interaction and communication. Guidelines to improve item quality will be provided, with a focus on more innovative item types such as those in scenario-based and simulation-based environments. The results of this research will benefit instruction and intervention programs designed to help students in academic environments. In addition, open-source software will be developed, and graduate students will be involved in the conduct of the research.

Recent large-scale computer-based assessments have developed a number of interactive problem-solving items. This research project will develop tools for the analysis of these new items. The investigators will concentrate on several aspects that are very challenging in modern computer-based assessment and online learning. Specifically, they will focus on the following topics: 1) understanding students' cognitive processes by means of statistical learning techniques, extracting information from process data; 2) improving current assessment tools by means of process data; and 3) incorporating information in process data to online adaptive/personalized learning. The analysis will combine techniques and concepts from education research and statistical learning. The proposed models will combine latent variable modeling and deep learning techniques for process data analysis. The investigators will employ recent advances in modeling and segmenting techniques for natural language processing. Adaptive learning will be studied through a reinforcement learning framework. In addition, optimization algorithms will be developed by means of recent advances in numerical methods. This award is supported by the MMS Program and a consortium of Federal statistical agencies.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.