Scalable Assessment in MOOC Courses

COIL is proud to have supported the Scalable Assessment in MOOC Courses project through our Research Initiation Grant Program.

Abstract:

Massive open online courses (MOOCs) have arisen prominently since 2011. Many of these courses

have greater than 50,000 students enrolled, and thus assessment has critical challenges, including 1) scalability of assessment for so many students, and 2) course engagement and completion. Using our Fall 2013 MOOC in “Creativity, Innovation, and Change” to take data, we aim to compare two models of peer-to-peer scalable assessment (Calibrated Peer-to-Peer (CPTP) Review, and an “Amazon.com Model”) with a Control Model similar to typical faculty/teaching assistant grading. We will seek to answer the following questions:

1. Comparison to Control Model. How do the two Peer Review models compare with the Control Model, in terms of scoring, quantity of comments, and quality of comments for student work? 

2.    Reviewer engagement. Do more students respond when using the Amazon Model, or the CPTP? What differences in assessment occurs when reviewers rate high or low-performing students?

3.    Student persistence. How is the quantity of feedback a student receives correlated with course completion? Only correlation, not causation, will be examined in the present research. We will use this research to guide future offerings of our course, and we aim to help other MOOC faculty by disseminating our work broadly.

Team:

• Darrell Velegol, Distinguished Professor of Chemical Engineering, Penn State University Park
• Jack V. Matson, Emeritus Professor of Civil Engineering, Penn State University Park
• Kathryn Jablokow, Associate Professor of Mechanical Engineering, Penn State Great Valley
• Kyle Peck, Professor of Education, Penn State University Park
• Kate Miffitt, Instructional Designer, Education Technology Services (ETS) at Penn State
• John Bellanti, Psychologist
• Charles Camarda, NASA Engineer

Contact:

Darrell Velegol
Department of Chemical Engineering
108 Fenske Lab
University Park, PA 16802
velegol@psu.edu
(814) 865-8739