Our understanding of what K-12 students learn from engineering design is limited. The National Research Council's Committee on Standards for K-12 Engineering Education found "very little research by cognitive scientists that could inform the development of standards for engineering education in K-12." This project responds to these problems by developing cutting-edge process analytics for K-12 engineering design research.
In the context of K-12 science education, engineering design is a complex cognitive process in which students learn and apply science concepts to solve open-ended problems with constraints to meet specified criteria. The complexity, open-endedness, and length of an engineering design process often create a large quantity of student data that makes learning difficult to discern using traditional assessment methods. For example, a pattern that looks like "gaming the system" in an inquiry activity may be a legitimate search in a vast problem space for meaningful alternatives in a design project. An idea that sounds ridiculous initially may lead to the most creative design at the end. Students may learn more from failed designs than from successful ones because failure promotes the need to explain and revise. The focus of engineering design assessment is not simply on whether or not students "get the right answer," but on how they acquire science and engineering knowledge and skills in the quest for optimal design solutions. Engineering design assessment thus requires innovative solutions that can track and analyze student learning trajectories over a significant period of time. Sophisticated data mining and machine learning techniques originally developed for scientific and business applications provide such solutions.
In this research project, approximately 3,000 secondary students will be involved in 6-8 hours of intense design projects using our Energy3D software, producing about 60 GB of process data that will become the "gold mine" of educational research on engineering learning. Process analytics based on mathematical and computational models such as time series analysis, graph theory, correlation analysis, Bayesian performance tracing, and so on will be developed to decipher learning patterns in these "big data."
Xie, C., Zhang, H., Nourian, S., Pallant, A. & Bailey, S. (In press). A study of the instructional sensitivity of CAD logs based on time series analysis, International Journal of Engineering Education, 30.
Xie, C., Zhang, H., Nourian, S., Pallant, A. & Hazzard, E. (2014). A time series analysis method for assessing engineering design processes using a CAD tool, International Journal of Engineering Education, 30(1), 218-230.
Xie, C. (2013, December 6). Large-scale research on engineering design based on big learner data logged by a CAD tool. NSF ECR inaugural meeting, Arlington, VA.
Zhang, H., Xie, C. & Nourian, S. (2014, June 23-27). Detecting iterative cycles of engineering design from student digital footprints in computer-aided design software. The 11th International Conference of the Learning Sciences, Boulder, CO.