In the ever-evolving landscape of educational technology, the true measure of success extends beyond mere engagement metrics. It is essential to focus on actual learning outcomes. However, quantifying learning can often be a complex task. A platform designed around the principles of mastery learning encourages students to immerse themselves in their studies until they achieve a high level of proficiency before progressing to new concepts. Recent studies indicate that dedicating over 30 minutes weekly to this platform correlates with improved learning results. Furthermore, our latest findings reveal that prioritizing the attainment of proficiency in skills significantly enhances performance on external assessments.
To accurately gauge mathematical learning, it is crucial to utilize metrics that align with established, reliable assessments of academic achievement. One such assessment is the nationally recognized MAP Growth mathematics standardized test. We conducted an analysis to explore the relationship between MAP Growth scores and various indicators of mastery progress within the platform.
The mastery learning framework employed by the platform categorizes skills into levels of mastery: from ‘attempted’ to ‘familiar,’ ‘proficient,’ and ultimately ‘mastered.’ Students progress through these stages by correctly answering questions on exercises, quizzes, unit tests, and course challenges. Research in the field of mastery learning suggests that students who persist in mastering a skill until they achieve a high score are more likely to retain the information over the long term.
Our research findings indicate that the percentage of math skills completed at a proficient or mastered level is positively correlated with MAP Growth scores, regardless of the total number of skills attempted or the time spent on the platform. This trend is consistent across various grade levels, reinforcing the idea that achieving higher mastery levels before advancing to new skills is linked to better retention of knowledge.
We now assess the effectiveness of our platform based on both the time users spend engaging with it and the proportion of skills they complete at a proficient or mastered level. This proficiency metric is particularly significant due to its strong correlation with external measures of mathematical achievement, demonstrating what researchers refer to as ‘convergent validity.’ This approach aligns with the principles of mastery learning and the educational design strategies we implement.
Each exercise on the platform typically consists of multiple items targeting the same skill. Students progress to levels such as ‘familiar’ or ‘proficient’ based on their performance on these items. If a student reaches the ‘familiar’ level, they have the option to retake the exercise to achieve ‘proficient’ status or to move on to a new skill.
Choosing to stop at the ‘familiar’ level rather than striving for ‘proficient’ or higher can signify a difference between superficial and deep learning, with the latter necessitating more practice. Our analyses suggest that focusing on a smaller number of skills while achieving proficiency is more beneficial than attempting to tackle multiple skills without reaching a solid understanding. In a typical 30-minute session, a learner can effectively work on two exercises and achieve proficiency in two skills. If a learner does not succeed in reaching proficiency on their first attempt, we encourage them to retry rather than shift to a new skill.
Our ambitious five-year objective is to enhance the learning outcomes of 10 percent of U.S. students by 50 percent in mathematics and/or science. We utilize the metric of ‘skills to proficient+’—the count of skills a learner has advanced to proficient or mastered—as our internal benchmark for learning. This metric is straightforward and is associated with significant improvements in external mathematics assessments. We are committed to making a measurable impact on student learning outcomes as we work towards achieving our five-year goal.