Poughkeepsie, NY
March 16–20, 2015
Conference Website
The theme for this year’s conference, Scaling Up: Big Data to Big Impact, reflects the success of the growing community of researchers, practitioners and learners in leveraging the power of “big data” to create substantial impact within higher education and learning at increasingly larger scale.
Thursday, March 19
We are presenting two papers in the same session.
Tracking student progress in a game-like learning environment with a Monte Carlo Bayesian knowledge tracing model
Gey-Hong Gweon, Hee-Sun Lee, Chad Dorsey, Robert Tinker, William Finzer & Daniel Damelin
10:15 AM–12:00 PM, Nelly Goletti Theatre
The Bayesian Knowledge Tracing (BKT) model is a popular model used for tracking student progress in learning systems such as an intelligent tutoring system. However, the model is not free of problems. Well-recognized problems include the identifiability problem and the empirical degeneracy problem. Unfortunately these problems are still poorly understood and how they should be dealt with in practice is unclear. Here, we analyze the mathematical structure of the BKT model, identify a source of the difficulty, and construct a simple Monte Carlo BKT model to analyze the problem in real data. Using the student activity data obtained from the ramp task module at the Concord Consortium, we find that the Monte Carlo BKT analysis is capable of detecting the identifiability problem and the empirical degeneracy problem, and, more generally, gives an excellent summary of the student learning data. In particular, the student activity monitoring parameter M emerges as the central parameter.
How does Bayesian Knowledge Tracing model emergence of knowledge about a mechanical system?
Hee-Sun Lee, Gey-Hong Gweon, Chad Dorsey, Robert Tinker, William Finzer, Daniel Damelin, Nathan Kimball, Amy Pallant & Trudi Lord
10:15 AM–12:00 PM, Nelly Goletti Theatre
An interactive learning task was designed in a game format to help high school students acquire knowledge about a simple mechanical system involving a car moving on a ramp. This ramp game consisted of five challenges that addressed individual knowledge components with increasing difficulty. In order to investigate patterns of knowledge emergence during the ramp game, we applied the Monte Carlo Bayesian Knowledge Tracing (BKT) algorithm to 447 game segments produced by 64 student groups in two physics teachers’ classrooms. Results indicate that, in the ramp game context, (1) the initial knowledge and guessing parameters were significantly highly correlated, (2) the slip parameter was interpretable monotonically, (3) low guessing parameter values were associated with knowledge emergence while high guessing parameter values were associated with knowledge maintenance, and (4) the transition parameter showed the speed of knowledge emergence. By applying the k-means clustering to ramp game segments represented in the guessing, slip, and transition parameter spaces, we identified seven clusters of knowledge emergence. We characterize these clusters and discuss implications for future research as well as for instructional game design.