COIL is proud to support the Exploratory Development of a Tool to Measure Learners’ Knowledge Structure in Online Learning Courses: Formative Structural Assessment project through our Research Initiation Grant Program.
There are several existing automatic writing evaluation (AWE) software tools for supporting free-text responses in online courses. However, the existing AWE tools are typically used for summative purposes, based on linguistic assessment, and/or implemented in English language. This project will develop a formative structural AWE tool applicable to any language by integrating two standalone offline software tools from different research traditions, and will pilot how it is received by students and instructors, and whether this software positively influences both online students’ learning and instructors’ practice. The fully developed software will be able to immediately convert students’ writings into network graphs, Pathfinder Networks, which are hypothesized to represent the salient “knowledge structure (KS)” related to the course content. Based on our empirical evidence carried out in various kinds of learning environments across several languages, this “integration” approach is a useful and usable way to capture and visually represent the content KS of learners. This project proposes to (1) develop an integrated online version of the offline that automatically provide individualized KS feedback, (2) implement the online as a formative structural assessment tool to empower online learners – namely, through reflection on their KS, and (3) evaluate on a small scale the effectiveness and appropriateness of its automated evaluation and KS feedback for online learners’ academic success to provide a foundation for a larger scale externally funded trial.