A Dashing New Look into Dragon Genetics

Our popular dragon genetics games have been used by tens of thousands of high school biology students to learn about heredity and genetics. In hundreds of classrooms we have watched students light up when they solve a virtual breeding challenge and create an offspring drake with the right traits. However, it isn’t always obvious—to us or to the teacher—how well or poorly a student is progressing in the game and understanding the genetics concepts.

The GeniGUIDE project, funded by the National Science Foundation, seeks to improve student learning using game data in two ways: by supporting the student with just-in-time help and by sending valuable information about student progress to the teacher. We partnered with North Carolina State University to design an Intelligent Tutoring System (ITS) and integrate it with our newest dragon genetics game, Geniventure. While students try to solve the challenges in Geniventure, the ITS works in the background, monitoring student performance and building a model of student knowledge. If students struggle with Geniventure challenges, the ITS provides hints and extra practice for genetics concepts, and passes information to the teacher via a real-time dashboard system.

To build the ITS, we identified specific game challenges with traceable on-screen actions that could be used to show evidence of learning. At the same time, we used an Evidence-Centered Design (ECD) approach to simplify broad genetics learning goals, generating a comprehensive list of individual concepts. We then mapped these bite-sized concepts to the game challenges. This process led to an initial version of the ITS—a rules-based system that evaluated student understanding of a handful of basic concepts, such as the need for two recessive alleles to produce a recessive trait and the ability to map from a genotype to phenotype and vice versa. Using this system, the Geniventure game was able to provide students with three levels of progressively more directed hints.

After piloting this system in multiple classrooms, observing students, and gathering teacher feedback, we upgraded the rules-based ITS to utilize a more advanced student model based on Bayesian Knowledge Tracing (BKT). While using BKT is common in software addressing more algorithmic topics (think mathematics and computer programming), it is a novel approach in an open-ended genetics game.

A model of student understanding

The data-driven, BKT-based system creates an initial student model using data collected from previous classroom implementations. As a student works through the six levels and over 60 challenges of Geniventure, the BKT-based ITS tracks their in-game actions (for instance, changing an allele or choosing particular parent drakes to breed) and modifies the initial model accordingly, making it specific to the student and continuing to refine it as the student plays Geniventure. The ITS calculates a probability that the student understands the genetics concept identified by the ECD process. The probability represents the likelihood that the student has learned the concept that is being addressed by their behavior as they set alleles, sort drakes by predicted phenotype, or attempt to breed offspring that match a target. Using machine learning, the BKT-based ITS takes into account all observations of the student’s performance to constantly model a student’s knowledge of genetics concepts. This information is used to estimate the probability that the student already knows the concept, is guessing correctly, or is making a mistake in applying a concept that he or she actually does understand.

The resulting student model yields a more complete representation of student knowledge, which results in improved tutoring actions, such as delivering contextualized hints in a timely fashion. In addition, the BKT-based ITS also provides remediation for students who are truly struggling. When the ITS detects that a student is struggling, the game presents the student with a simplified “bonus challenge” that focuses on one specific concept. After completing this challenge, students can continue with the regular challenge sequence. The ITS also evaluates unnecessary actions made by the student while attempting to complete Geniventure challenges—a type of “gaming the system” that can be caught automatically.

Hints help students

With the integrated ITS, students who get stuck aren’t stuck for long. The hints enable students to master simple concepts quickly and with less repetition. In fact, the speed with which students can now work through the challenges has allowed us to introduce more complex concepts and more difficult challenges earlier in the game, reducing the overall length of time needed to complete all game challenges from two or three weeks to just over one week. When the ITS determines that a student is struggling with a concept, it provides a hint. Early challenges provide hints that include both a textual explanation as well as a visual cue (Figure 1). As the game progresses, some of the hint scaffolding is removed. The hints and bonus challenges are designed to look like the regular game screens so the screens of students who are getting several hints or remediation do not alert curious peers.

Following completion of each challenge, students are rewarded with different colored crystals based on their skill and efficiency. Each student is charged with earning crystals; the color of the crystal determines its in-game value. Students can track their progress in the game by viewing the crystals they have earned for each level. The game narrative encourages repeating challenges when low-level crystals are earned. For some students, achieving blue crystals—the highest level in the mythical dragon kingdom—has been a great motivator.

Teacher dashboard

In a small class, it’s easy to catch a student who is disengaged. With a class of 20 or 30, however, these students can easily escape a teacher’s attention. The Geniventure teacher dashboard displays real-time student progress and performance, functioning as an extra set of eyes in the classroom (Figure 2). To help teachers determine where students might be struggling, the dashboard offers two views. The game progress view displays the crystals students have earned for each attempted challenge, whether they entered remediation for a challenge, and the challenge that students are currently working on. The genetics concept view shows their ITS-estimated level of understanding of each concept.

In both views, the dashboard displays individual student reports and class-level summaries. By sorting the data, teachers can quickly identify students who are behind or having trouble and intervene. As one teacher said, “I saw a student get a black crystal several times in a row, and before he got too frustrated and gave up, I was able to help him.” After all, the teacher is the most intelligent tutor in the room!

Next steps

We have piloted Geniventure in a small number of classrooms. In the next phase, we will collect pre- and post-tests of content knowledge, surveys of students’ attitudes of the game, and logs of teachers’ experiences. This spring, we plan to field test with teachers who want to use Geniventure with their students and are willing to give us their feedback on all aspects of the game, including our teacher support materials. Geniventure, teacher guides, and other classroom support materials will be freely available in summer 2019.

Trudi Lord (tlord@concord.org) is a project manager.
Frieda Reichsman (freichsman@concord.org) is a senior research scientist.

This material is based upon work supported by the National Science Foundation under grant DRL-1503311. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.