Informing Educational Assessment with Test-Taking Process Data (Susu Zhang)

Computerized testing affords the collection of additional behavioral data beyond scores on test questions. One such instance is process data, the computer-recorded log files generated from test-taker interactions with a computerized assessment, e.g., keystrokes and clickstreams, in pursuit of solving a problem. This type of data offers rich insights into the cognitive processes underlying problem-solving, opening new avenues to address existing measurement and educational questions and explore novel ones. However, like constructed responses on open-ended questions with infinitely possible answers of different lengths, process data are highly unstructured and often noisy. This precludes the direct application of many well-established tools and psychometric models for structured test response data. In this talk, I discuss how sequential features extracted from test-taking process data can be incorporated into latent variable modeling frameworks commonly used in educational assessment, to address unique needs in measurement and educational research, such as improving score reliability and testing/generating hypotheses.