Self-paced asynchronous online courses offer students significant flexibility in when and where learning occurs. Recent research by the Michigan Virtual Research Institute examined how student pacing, particularly the order in which they submit assignments, is related to online STEM and World Language course performance. Understanding students’ pacing behavior and its relationship to course performance can help inform the strategies educators and mentors use when working with students in self-paced online courses.  

In the following interview from our “Interview with a Researcher” blog series, the lead researchers behind Michigan Virtual Learning Research Institute’s (MVLRI) STEM and World Languages reports synthesize some take-home messages about students’ assignment submission patterns. 

Why is it important to consider online learners’ assignment submission patterns? 

Assignment submission patterns are a part of a set of student behaviors called pacing—how students progress through a course. Pacing has traditionally been thought of as the timing of students’ assignment submissions. When conceptualizing pacing in this way, we often ask questions like, Are students waiting until the last minute to submit assignments? Are they submitting assignments early or late? Are they submitting a lot of assignments close together? It’s well-established that the timing of student assignment submissions is related to course outcomes. However, our team wanted to know more about the possible impact of out-of-order assignment submissions because, anecdotally, this was a pattern Michigan Virtual (MV) instructors were noticing within these asynchronous courses. Assignment sequencing is the term we gave to describe the order in which students submit their assignments. When we looked at this behavior across two domains (STEM and World Languages), we found evidence that it is related to course outcomes. Specifically, as students submit more assignments out of their intended course order, final course scores tend to decline.  

Why did you feel like it was important to look at assignment sequencing in World Language courses?

Great question! The original hypothesis that prompted this research was that submitting assignments out of their intended order would be detrimental to student performance because it would undermine the scaffolding built into the courses. So, based on this hypothesis, our first study on assignment sequencing used a sample of students enrolled in Michigan Virtual STEM courses since they are highly scaffolded. Of course, scaffolding is likely to vary by subject area and course, so we felt it was important to expand our research. After preliminary analyses of assignment sequencing in several other subject areas, World Language had a high percentage of students who moved out of alignment with course pacing guides and is a distinct subject area from STEM, making it an ideal choice for expanding our previous research. Looking across these two subject areas also allows us to understand the generalizability of our findings, compare and contrast key differences, and provide data-backed recommendations to instructors and mentors of students in these subject areas. 

What did students’ assignment submission patterns look like in World Language courses? Could you explain the relationship between students’ assignment sequencing and their final course scores?

We found that it was really common for students to deviate from course pacing guides! 97% of students submitted at least one assignment out of alignment with their course pacing guide. Among these students, approximately 45% of completed course assignments were submitted out of order. While the volume of assignments submitted out of order was fairly high, students were about three assignments “off” from the intended pacing guide order.

Looking across the spectrum of student performance, we observed that students’ final course scores steadily declined as their assignment submissions became increasingly out of order, both in terms of the number of assignments submitted out of order and how far “off” students were from the pacing guide expectation. To put this in perspective, students who submitted the fewest assignments out of order had average final course scores as much as a full letter grade higher than students who had the greatest number of assignments submitted out of order. 

You mentioned that the first study in this series looked at assignment sequencing in online STEM courses. Were there any notable differences between that study and this one? Did you see any patterns across these two studies?

The general pattern of results was similar across the two studies in that students’ assignment submission patterns had a relationship with final course scores. The biggest difference, however, was in the percentage of students who went out of order in each subject area. While both studies showed high rates of out-of-sequence behavior, 93% of students went out of order in the STEM study compared to 97% in World Languages. Across both studies, course scores steadily declined as students submitted a greater percentage of assignments out of order and strayed further from the intended assignment sequence. In STEM courses, there was a 9.5 point difference in average final course scores between students with the fewest and greatest number of assignments submitted out of order, whereas in World Language courses, there was a 9.6 point difference. The relationship between assignment sequencing and final course scores was really similar across the two studies, which suggests that monitoring and encouraging proper pacing is important for student performance in both subject areas.

Based on your findings across these two studies, what recommendations do you have for online instructors and mentors?

Our findings indicate that it is common for students to deviate from course pacing guides at least once during their time enrolled in MV online asynchronous courses. Some deviation is to be expected and is unlikely to negatively impact student performance, especially if that deviation is infrequent or small (e.g., within a unit). However, if students are consistently moving between units or submitting a high volume (more than 25%) of their assignments out of order, online instructors may want to flag these behaviors (and students) and monitor for performance declines. 

In general, adhering to best practices for online teaching and mentoring is recommended to help online learners be as successful as possible. Communicating course expectations early on (informing students of the structure, workload, pacing, and demands of self-paced online learning) may help students adjust their expectations and approach to their course(s). Regularly checking the gradebook and benchmarking student progress against course pacing guides can help teachers and mentors identify students who may be struggling with course pacing. Mentors and instructors should also communicate regularly about students’ progress and work collaboratively to address pacing issues.

It is also possible that submitting assignments out of order may have a greater impact on some students’ performance than others. For example, students with less content knowledge may miss key benefits of built-in scaffolding when submitting assignments out of order, which may negatively impact course performance. Further, because the design of these studies limits our ability to make cause-and-effect statements, it is likely that other factors interact with pacing to affect student performance. In particular, encouraging the development of metacognitive, time management, and self-regulated learning skills may help students reflect and make adjustments to their own learning behaviors. In this regard, providing students with personalized feedback may be useful.

Final Thoughts

Across two reports, the relationship between pacing and final course scores has consistently shown that final course scores decline as students become increasingly out of alignment with their course pacing guides. Instructors and mentors can help students succeed by paying particular attention to students’ pacing within their online courses. 

You can check out the full reports below: 

Out of Order, Out of Reach: Navigating Assignment Sequences for STEM Success

Out of Order, Still Out of Reach: Navigating Assignment Sequences for Michigan Virtual World Language Courses

In addition, this blog is part of a blog series exploring the impact of student assignment submission patterns.

Introduction

Michigan’s Top Ten Strategic Education Plan was adopted in August 2020 to guide K-12 education stakeholders in working toward a common set of goals. The second goal on the list is to “Improve early literacy achievement” (Michigan Department of Education, n.d.). In recent years, small declines in Michigan students’ performance on the reading portion of the National Assessment of Educational Progress (NAEP) (The Nation’s Report Card, n.d.) have emphasized the need for this goal, as well as the need for new interventions that will boost students’ literacy skills. In Fall 2024, Michigan Virtual, a key stakeholder in the education success of K-12 students in Michigan, partnered with Shoelace Learning to run a pilot aimed directly at tackling this goal.

Shoelace Learning is an educational technology company focused on building students’ reading comprehension skills through video games. Michigan Virtual worked with Shoelace Learning to implement their games in 13 elementary classrooms across Michigan to study whether these games would improve students’ confidence and literacy skills.

Literacy in the US

Literacy is an important skill with wide-reaching implications for all ages. A study by the Annie E. Casey Foundation noted that children who were not proficient in reading by the end of third grade were four times more likely to drop out of high school than reading-proficient peers (Hernandez, 2011). Those who continue to have poor literacy levels in secondary school often experience difficulty throughout and beyond school (Hakkarainen et al., 2016; The Children’s Reading Foundation, n.d.). Indeed, insufficient literacy skills in adulthood can have financial impacts, as adults with low literacy skills are more likely to be unemployed than adults with high literacy skills (The National School Boards Association, 2014). Educators and policymakers have recognized literacy’s importance and the need to help students develop strong skills early (Michigan Department of Education, 2024).

Over the years, many reading interventions have been implemented with the hopes of improving K-12 literacy with varied success. Interventions, such as small-group and motivational reading have been shown to positively impact students’ reading abilities (Hall & Burns, 2018; McBreen & Savage, 2021). Unrau et al. (2018) and Moon et al. (2017) have also highlighted that increasing reading enjoyment and self-efficacy may also help students to engage and persist in reading. But, most importantly, early interventions seem to be crucial for struggling readers (Wanzek et al., 2018).

Gamification

Gamification in education, creating game-like experiences to engage learners with content and help them progress toward a goal, has become a popular intervention strategy in recent years (Dehghanzadeh et al., 2024). Gamification has shown promise in improving behavioral, affective, and cognitive/learning outcomes in K-12 settings, and this may, in part, be attributed to its ability to engage and motivate students (Dehghanzadeh et al., 2024; Huang et al., 2020; Prados Sánchez et al., 2021; Sailer & Homer, 2020). While gamification has been shown to engage students in the task at hand, its ability to improve student outcomes has been mixed and can hinge on various factors related to game design, the context in which it is delivered, and learner characteristics (Dehghanzadeh et al., 2024). For this reason, this pilot was conducted to examine Shoelace Learning’s reading comprehension games to evaluate them both on their ability to engage and motivate students and, even more importantly, on their ability to impact student learning.

Study

Given the promise of gamification for engaging students, the goal was to examine the impact that providing teachers with games designed to engage students in reading comprehension could have on literacy rates. The current study was designed to assess teachers’ perceptions of Shoelace (both in terms of the impact on their students and the ease of use in their classrooms), student engagement with the platform, and the efficacy of the games in improving literacy skills. The pilot lasted for eight weeks and consisted of 13 teachers and their classes, all from Michigan.

Design

The main component of the pilot was the eight-week period of play for the students. However, the teachers started their participation a couple of weeks in advance by first participating in a 60-minute orientation and introduction to Shoelace Learning. During this orientation, teachers were provided with an overview of the Shoelace platform and teacher dashboard, including examples of gameplay, assessments, reporting, and assignments.

While teachers had considerable autonomy over how they implemented Shoelace in their classrooms, they were asked to have their students play for a minimum of 30 minutes each week for the eight weeks. During this play period, Michigan Virtual researchers conducted roundtable discussions with many of the teachers and students, and Shoelace employees also did in-class visits with a number of the classrooms. 

Following the eight weeks of play, gameplay data was collected, and the teachers received a survey that assessed their perceptions of students’ reading comprehension, enjoyment, and confidence, and the usability of Shoelace in the classroom.

Participants

Thirteen participants were recruited through the Michigan Elementary and Middle School Principals Association (MEMSPA). They came from six school districts with which Michigan Virtual had existing relationships. In return for participating, Michigan Virtual provided teachers with Shoelace access for two years, from September 2024 to June 2026, and a $150 stipend upon completing the pilot.

Table 1 provides an overview of the 13 classes that participated. 

Table 1 – Grade level and number of students for each participating class.

Shoelace

Shoelace is an online platform that provides reading comprehension practice through game-based delivery and is intended for students in grades 3-8. In order to progress in the games, students must correctly answer reading comprehension questions. The reading comprehension questions cover over 100 different reading skills and are each assigned a grade and difficulty level. The questions may be delivered in conjunction with a short reading passage. These bundles (questions + passage) are used to evaluate students’ overall Reading Comprehension Level (RCL). Students also encounter standalone questions, which are questions that do not have a corresponding passage and instead focus on specific skill development.

There are two games that students can choose to play: Dreamscape or Dreamseeker Drift. Dreamscape is a strategy game that is similar to Clash of Clans and is built around a central “vision core,” a diamond-like structure that players must protect and level up to progress. Dreamseeker Drift is an endless runner in a similar vein to Subway Surfers, where players aim to achieve the longest “run” possible by avoiding obstacles. In both games, in order to engage with the game elements (for example, buy new avatars and skins, compete in challenges, or start new runs) students must correctly answer reading comprehension questions.

Gameplay data reported by Shoelace:

• Learning Moments Delivered (LMD): The number of questions a player answered (regardless of correctness). 
• Reading Comprehension Level (RCL): A leveling system based on passage (e.g., sentence structure, vocabulary) and question difficulty. The scale ranges from 1.0 to 8.9, with the first number representing grade level (e.g., 3.4 is grade 3) and the second indicating progress towards the next grade level (e.g., 0.4 indicates approximately halfway through the level).

Results

Participating classrooms were expected to have students use Shoelace for 30 minutes per week for eight weeks. As time data was not available, prior Shoelace data showing that 30 minutes of quality play was equivalent to approximately 25 LMD was used instead. For the purpose of our evaluation, student fidelity is defined as students who played a minimum of seven weeks (to provide flexibility for absences) with an average of 25 LMD per week. As Table 2 shows, fidelity varied across participating classrooms. For classroom fidelity, we modified this definition to be that 80% of the students in the class played with fidelity. As shown in the next section, a class or student who did not meet the fidelity benchmarks should not be interpreted as the class or student did not play (or even that they played very little). While 58.7% of students played with fidelity, 91% of students had a minimum of five weeks of participation.

Table 2 – Participation and fidelity levels by class. Classes marked with an * achieved class level fidelity.

Student and Class Participation

A student was considered to have participated in a given week if they answered a minimum of one LMD. Total class participation (active # of students / total students in class * 100) provided insight into what percentage of a class played. Students were free to play either game during the eight-week pilot period so the following discussion will be agnostic as to which game they played. Of the 300 students across the 13 classes, 299 of them (or 99.7%) played at least once. 

Figure 1 shows the average percentage of active students per week. From week to week, participation ranged from about 71% to 91%. There was no clear trend, with participation fluctuating from week to week but overall staying relatively high between approximately 80 and 90%. The largest gain was seen from weeks 2 to 3 of 9.7%, while the largest loss happened between weeks 5 and 6 of about 6.7%.

Figure 1 – Percent of students who played by week.

At the class level, as seen in Table 3, participation fluctuated more widely from week to week, as most weeks’ classes either had most everyone play or no one play. Class 3 was the main outlier, as max participation never surpassed 60% in a given week.

Table 3 – Min, max, and average participation levels by class.

Learning Moments Delivered (LMD), Accuracy and Guessing

Learning Moments Delivered (LMD) represents a single question attempted by a student (regardless of whether they got it right or wrong). On average, students attempted 721 LMD over the eight-week period. Excluding the one student who did not participate, the number of LMD earned over the period showed large variation, as they ranged from a low of 3 to a max of 6,858. Similarly, when looked at by weekly averages, students ranged from a low of 1 to a high of 859. Figure 2 highlights the variation in student data.

Figure 2 – Percent of students grouped by their average weekly LMD counts. Target was ≥25 LMD/week.

In addition to looking at the total number of LMD, student accuracy was also examined. Accuracy was calculated by dividing the number of correctly answered questions by the total number of questions students attempted. Table 4 shows the min, max, and average accuracy by class and overall. Students averaged an accuracy rate of 50.2%, with a minimum of 26.7% and a maximum of 94.7%.

Table 4 – Min, max, average accuracy by class over the eight-week period. 

Prior Shoelace data shows that maintaining an accuracy rate of 50% or higher from their learning engine is suggestive of positive use and learning results. Because the learning engine will periodically introduce incrementally more difficult content to determine if students are ready to engage with it, student accuracy rates will not be akin to, nor should they be compared to, those produced from summative assessments. Accuracy rates between 40-50% usually indicate students are struggling more with the content and may be turning to guessing (something often seen with students whose reading comprehension is too low for the program). Accuracy rates that fall below 40% are generally indicative of students who have primarily turned to guessing.

Reading Comprehension Levels (RCL)

Reading Comprehension Level (RCL) is Shoelace’s leveling system based on passage and question difficulty. RCL, thus, provides important information about students’ progress and reading abilities over the course of their engagement with Shoelace. A student’s RCL can range from 1.0 to 8.9. The first digit represents the grade, and the value after the decimal represents progress towards the next grade. An RCL of 3.4 would mean the student is reading at a grade 3 level and approximately 40% of the way to grade 4. Most students in a class had RCL data available (97.9%, SD = 0.04). 

The first RCL value that students receive is set by the placement test that is initiated upon a student’s first play session. The placement test is a quick assessment designed to adjust the starting point for the learning engine. The placement test results in a value between 1.0 and 8.5 in increments of 0.5. At the start of the pilot, the average RCL after the placement test was 2.0, with a range from 1.0 to 6.5. By the end of the 8 weeks, the average RCL value was 2.1, with a range of 1.0 to 6.6. 

Figure 3 shows the average change by class. On average, classes showed an increase of 0.13 RCL, with a range of -0.1 to 0.4.

Figure 3 – Average start vs end RCL by class.

Table 5 shows how many students fall into specific RCL Change groupings based on whether they played with fidelity. RCL Change data was available for 295 out of 299 (98.7%) students. One hundred and seventy-five students played with fidelity, while 120 did not. Of the 175 students (59.3%) who played with fidelity, 97 (55.4%) saw an RCL Change of greater than 0.0, and 75.4% of those had an RCL Change value greater than 0.2. Taking a look at the 120 students who did not play with fidelity (40.7%), 40 (33.3%) saw an RCL Change of greater than 0.0 while 20 (50.0%) saw RCL Change values of greater than 0.2.

Table 5 – RCL changes by student based on fidelity of play.

Teacher Perceptions

At the end of the eight-week play period, the 13 teachers filled out a survey about their perceptions of how playing Shoelace games impacted students’ reading confidence, comprehension, fluency, vocabulary, and enjoyment. The vast majority of the teachers reported that they believed using Shoelace increased or significantly increased their students’ confidence in reading (n=10), reading comprehension (n=10), fluency (n=10), vocabulary development (n=12), and enjoyment of reading (n=12). Of those teachers who did not report an increase, they all reported no changes (n=1-3). Table 6 provides a closer look at teachers’ ratings of student outcomes.

Table 6 – Teacher ratings of student growth.

Teachers were also asked about how easy or difficult they found it to include Shoelace in their classroom and how well it aligned to their curriculum and their daily teaching. Table 7 shows that the vast majority of teachers were satisfied or very satisfied with all aspects of Shoelace usability (n=11-12).

Table 7 – Teacher perceptions of Shoelace usability

Discussion

Class Participation and Engagement

Class-level engagement patterns appear consistent with teachers’ self-reported data about the usability and impact of Dreamseeker Drift and Dreamscape. Survey data collected from teachers indicated a positive experience overall, with 11 of the 13 teachers satisfied or very satisfied with integrating Shoelace into the curriculum, their daily teaching, and its use as a teaching tool. There was, however, considerable variation in engagement levels across classes and fluctuations in both the percentage of active students and average LMD over the eight-week pilot period. While only five of the classes were identified as having reached the benchmark for class-level fidelity, and while a few of the classes had very few students who hit the individual student benchmark for fidelity, overall, 58.7% of students played with fidelity. This percentage of player fidelity can be deemed as highly positive, as research by Stanhope and Rectanus (2015) has shown that for most products, on average only 5.2% of student licenses reach full dosage amounts.

The LMD values across the pilot at both the individual student and class level revealed substantial variability, which also likely points to both between- and within-class differences in-game use. From the survey results and the roundtable discussions with the teachers, this may also be a result of the learning curve the teachers identified in implementing the games in their classrooms. Given this, along with the autonomy they were given over their individual implementations, these results are not surprising. Similarly, the differences in weekly LMD averages and accuracy are also likely impacted by factors such as class time, other commitments, how teachers chose to implement Shoelace, ease of implementation, student perceptions, or students’ reading abilities. 

The overall activation rate (students who played at least once) of 99.7% significantly exceeded the industry standard of an average student license activation rate of 63.4% across school and district sites (Stanhope & Rectanus, 2015). This exceptionally high activation rate may reflect positive teacher perceptions of the platform’s curricular alignment and/or teacher commitment to implementation fidelity, given that they were taking part in a paid research pilot period. While outside motivators may have influenced activation and the continual engagement with the platform, the volume of students who answered large numbers of LMD (71.3% attempted ≥200 LMD) show that the games did engage the vast majority of the classes’ and students’ interest over the entire pilot period.

Impact on Literacy Skills

Shoelace assigns students their first Reading Comprehension Level (RCL) at the end of the initial placement test. After that, their RCL value is updated each time a student completes a passage and question bundle. How it changes (increasing, decreasing, or staying the same) depends on how the student performs (accuracy over the entire bundle), the difficulty of the passage, and the students’ current RCL. RCL values range from 1.0 to 8.9. For this study, changes to students’ RCL over the eight-week period were used as a proxy for the improvement of their literacy skills. 

While the classes in the study ranged from grades 2 through 5, the students’ initial placement test results revealed that the students’ abilities covered a much larger range from a 1.0 (equivalent to the start of grade 1) to 6.5 (equivalent to about halfway through grade 6) with an average value of 2.0. This distribution of grades reflects the challenge of trying to teach students with a wide variation of skill level, which teachers frequently encounter in their classrooms.

By the end of the eight weeks, the range had widened slightly (1.0 to 6.6) but more interesting was the increase to the average RCL to 2.1. While a 0.1 increase may sound minor, each 0.1 step on the RCL scale can be viewed as roughly equivalent to a month’s progress (given a 10-month school year). When the data was dug into further, of the students who played with fidelity, 55.4% had at least a 0.1 increase and 42% had an increase of at least 0.3, or three or more months’ worth of growth in 8 weeks. Of the students who didn’t reach fidelity with their play, 33.3% had an increase of at least 0.1 and 16.7% of them showed 3 or more months of growth. These values demonstrated the relationship between consistent gameplay and positive change in RCL. 

In addition to these results, the survey and conversations with teachers showed that the teachers were also seeing positive literacy growth. From the survey results, a minimum of 76.9% of the teachers (or 10 of the 13) saw increases in students’ confidence, comprehension, fluency, vocabulary and enjoyment.

Limitations of the Study

Classrooms were expected to play either of the Shoelace games for a minimum of 30 minutes a week for each of the eight weeks of the pilot. At the class level, a class was considered to have met this goal and to have played with fidelity if a minimum of 80% of their students played with fidelity (reduced to provide flexibility). Even with this reduced benchmark, only 38.5% (or 5 of the 13) classes that participated met the fidelity threshold. At the individual level, students did slightly better, with 58.7% of them meeting their fidelity definition of a minimum of seven weeks played and answering a minimum average of 25 LMD per week. 

There are many reasons for students and classes not to have hit the benchmark for fidelity, including (but not limited to) time limitations, unknown barriers to implementation (i.e. confusion around gameplay), and student (or teacher) absences. Overall, while the number of classes that achieved fidelity was low, as the data above has shown, there is a good reason to set the bar for fidelity high and to push for this level of usage. 

The irregular usage may also imply barriers to consistent implementation (e.g., time) or perceptions about Shoelace’s usability or impact that were not captured by survey data. Indeed, whole-class conversations revealed that many teachers perceived Shoelace as having somewhat of a learning curve and that it required them to be “hands-on” while their students were engaged with the games. Students had overall positive perceptions of the games, with many talking enthusiastically about them. However, they also noted that more detailed in-game help would be beneficial. While a training session was provided to teachers at the start of the pilot, and additional training material made available, conversations from the teachers who received in-person visits by the Shoelace team revealed that additional face-to-face (vs asynchronous) training was extremely useful, especially once their class had started to play and the teachers had a clearer understanding of where they wanted and needed further information and training. The time and effort required to implement Dreamseeker Drift and Dreamscape may have impacted some teachers’ classroom use.

Conclusion

Students with low literacy skills struggle throughout their entire K-12 educational journey, but the impact does not end there. It will follow them throughout their adult life, affecting everything from the jobs they have to their financial health. Addressing this problem is the second goal of Michigan’s Top Ten Strategic Education Plan. Michigan Virtual and Shoelace Learning partnered together to run an eight-week pilot to examine whether using Shoelace games in Michigan classrooms would improve students’ confidence and literacy skills. 

The results of this study strongly indicate that Shoelace games result in gains in student literacy and confidence, with four important notes:

1. Fidelity of usage is important. The students who played most consistently across the eight weeks showed the greatest gains, with 42% of them seeing the equivalent of at least 3 months of reading growth. 
2. There is a learning curve for teachers, and they need training and support to overcome it. An initial overview and introduction to the platform is not enough, teachers need follow up once their students have started playing to address any questions and concerns that come up in the classroom. 
3. Engagement was high. While the educators who chose to participate did receive a stipend and two-year Shoelace licenses for doing so, that doesn’t discount just how high participation levels remained throughout the pilot and beyond. More than half the students answered over 300 LMD over the eight-week pilots, and more than 80% were still playing more than two months after the conclusion of the pilot.
4. Teachers’ positive perceptions of Shoelace and its impact on students’ skills were high. The vast majority of the teachers indicated that their students improved their confidence, comprehension, fluency, vocabulary, and overall enjoyment. A similar majority found that it met their curriculum and daily teaching needs. 

The combination of the pilot data and the teachers’ positive perceptions points to the overall utility of Shoelace as a tool for teachers looking to improve their students’ reading comprehension.

References

Hakkarainen, A. M., Holopainen, L. K., & Savolainen, H. K. (2016). The impact of learning difficulties and
socioemotional and behavioural problems on transition to postsecondary education or work life in Finland: a five-year follow-up study. European Journal of Special Needs Education, 31(2), 171, 186.https://eric.ed.gov/?id=EJ1095058

Hall, M. S., & Burns, M. K. (2018). Meta-analysis of targeted small-group reading interventions. Journal of School Psychology, 66, 54-66. https://doi.org/10.1016/j.jsp.2017.11.002

Hernandez, D. J. (2011). Double jeopardy: How third-grade reading skills and poverty influence high school graduation. Annie E. Casey Foundation. https://www.aecf.org/resources/double-jeopardy

Huang, R., Ritzhaupt, A. D., Sommer, M., & et al. (2020). The impact of gamification in educational settings on student learning outcomes: A meta-analysis. Educational Technology Research and Development, 68, 1875–1901. https://doi.org/10.1007/s11423-020-09807-z

McBreen, M., & Savage, R. (2021). The impact of motivational reading instruction on the reading achievement and motivation of students: A systematic review and meta-analysis. Educational Psychology Review, 33(3), 1125-1163. https://doi.org/10.1007/s10648-020-09584-4

Michigan Department of Education. (2024, September 26). Child Literacy Would Improve Under Bills Passed by Michigan Lawmakers. https://www.michigan.gov/mde/news-and-information/press-releases/2024/09/26/child-literacy-would-improve-under-bills-passed-by-michigan-lawmakers

Michigan Department of Education. (n.d.). Michigan’s Top 10 Strategic Education Plan.
https://www.michigan.gov/mde/-/media/Project/Websites/mde/top10/top_10_mi_strategic_ed_plan_promising_practices_1_pager.pdf?rev=13e8d60cd2be4ab4aa6f3cd86bbcf532&hash=5002C7DD79CA084D872FC3C1053603C2

Moon, A.L., Wold, C.M. & Francom, G.M. Enhancing Reading Comprehension with Student-Centered iPad Applications. TechTrends 61, 187–194 (2017). https://doi.org/10.1007/s11528-016-0153-1

National School Boards Association: Center for Public Education. (2014, October). Beyond fiction: The importance of reading for information. https://cdn-files.nsba.org/s3fs-public/Beyond-Fiction-Full-Report-PDF.pdf

Prados Sánchez, G., Cózar-Gutiérrez, R., del Olmo-Muñoz, J., & González-Calero, J. A. (2021). Impact of a gamified platform in the promotion of reading comprehension and attitudes towards reading in primary education. Computer Assisted Language Learning, 36(4), 669–693.https://doi.org/10.1080/09588221.2021.1939388

Sailer, M., & Homner, L. (2020). The gamification of learning: A meta-analysis. Educational Psychology
Review, 32, 77–112. https://doi.org/10.1007/s10648-019-09498-w

Stanhope, D. & Rectanus, K. (2015). Current realities of EdTech use: Research brief. Lea(R)n, Inc.
https://www.ikzadvisors.com/wp-content/uploads/CurrentRealitiesOfEdTechUse_Infographic_ResearchBrief.pdf

The Children’s Reading Foundation. What’s The Impact? (n.d.). https://readingfoundation.org/the-impact

The Nation’s Report Card. (n.d.). National achievement-level results.
https://www.nationsreportcard.gov/reading/nation/achievement/?grade=8

Unrau, N. J., Rueda, R., Son, E., Polanin, J. R., Lundeen, R. J., & Muraszewski, A. K. (2018). Can reading self-efficacy be modified? A meta-analysis of the impact of interventions on reading self-efficacy. Review of Educational Research, 88(2), 167-204.

Wanzek, J., Stevens, E. A., Williams, K. J., Scammacca, N., Vaughn, S., & Sargent, K. (2018). Current Evidence on the Effects of Intensive Early Reading Interventions. Journal of Learning Disabilities, 51(6), 612–624. https://doi.org/10.1177/0022219418775110

The K-12 education landscape is evolving, offering more options for students and their families seeking alternatives to the traditional model. Many desire flexible learning environments that better align with their needs and interests. At the same time, educators and school leaders are rethinking how to make learning more engaging and centered around essential skills or competencies that students must master.

Two instructional models driving this shift are project-based learning (PBL)—where students gain knowledge by tackling real-world and personally meaningful projects—and competency-based education (CBE)—where students advance by demonstrating mastery of competencies or key skills rather than completing a set number of classroom hours.

At FlexTech High School, these two approaches come together in a flexible model that enables students to incorporate their interests as they progress at their own pace toward graduation. To explore how FlexTech blends PBL and CBE, Michigan Virtual Learning Research Institute (MVLRI) researchers spoke with Dr. Sarah Pazur, director of school leadership at CS Partners, who oversees the FlexTech High School network in Michigan. The transcript of our conversation was edited for clarity and brevity. 

Can you tell us a little bit about FlexTech and why it was important for you to implement competency-based education?

FlexTech is a project-based, competency-based, blended-learning school offering face-to-face instruction Monday through Thursday with optional in-person student support offered on Fridays. All of our courses are accessible online through Google Classroom, allowing students to choose a learning model that works best for them—whether fully in-person, hybrid, or entirely online. Our advisory program, which is really focused on helping students find their purpose and make a plan for post-secondary that suits their passions and strengths, is the cornerstone of FlexTech. Beginning in the ninth grade, every student is paired with an advisor who follows them through their senior year and serves as an advocate for that student. 

There are three FlexTech campuses, all located in Michigan: Brighton, Oakland, and Shepherd. Each campus is small by design, with around 175 students at both Brighton and Oakland and 80 at Shepherd. Our small size enables us to offer students many options, flexibility, and one-on-one instructional support. FlexTech was actually designed as a competency-based school from the start. It was born out of a desire to provide options for students to finish high school—whether they weren’t going to finish because they needed to work during the day, life circumstances made school difficult, or they left their previous school because they weren’t successful, we wanted to provide these students with an option. Our flexibility and personalized support are designed to serve a wide range of students, including those who may not have been successful in traditional models.

What challenges did FlexTech face in implementing competency-based education?

One of our biggest challenges has been finding an LMS (learning management system) and a gradebook that work well in a competency-based/project-based learning environment to account for all of the competency-based nuances (e.g., indicating the “rigor rating” of an assignment, coordinating with a student’s transcript). Many learning management systems are designed around the student’s schedule; however, we need one that focuses, instead, on the project. This allows us to more accurately track competencies and allow for flexibility in terms of mastery. 

Another challenge has been tracking student progress. Because we’re competency-based, students can progress at their own pace, so we need detailed tracking systems to monitor which competencies each student has mastered and which still need work. Because of seat time (instructional time) requirements and expectations that students finish a course in a semester or in a year, we’ve had to create many of our own internal systems using spreadsheets and Google Docs, which allow teachers to make notes from semester to semester and year to year.

One area where we’ve experienced some pushback is around our grading scale. In our system, a 4.0 indicates a student can apply a skill in a new and novel way. However, the ceiling for an assignment that only requires mastery of a lower-level skill might be a 2.0. We’ve had to work hard to help parents understand that these numbers don’t directly translate to traditional letter grades.

Tell me a little bit about what competency-based education looks like in practice at FlexTech.

At FlexTech, competencies are primarily subject-based. While some competencies are multidisciplinary, most are tied to specific courses to ensure students meet credit requirements. To develop our competencies, we worked with curriculum designers and a consultant to create a crosswalk that aligns our competencies with state standards (e.g., MMC, NextGen, CCSS). This alignment process is crucial because it ensures that our competencies truly reflect the big ideas we want students to take away from each course. We continue to diligently revisit and revise these competencies to ensure they are still reflective of the right big ideas and standards. 

PBL and CBE complement each other at FlexTech. Most courses are designed to be project-based; however, some are more authentically project-based than others, as we do offer some off-shoot traditional courses for students who need them (e.g., a basic math class). Typically, students either a) start a project based on a personal interest or current event, and then, with help, competencies are wrapped into the project; or b) teachers create the “frame,” outlining specific competencies for the project, and then students create the “focus” based on their interests. Projects run for 3-10 weeks and typically focus on 1-3 specific competencies. Milestones, formative feedback, and self-reflection are key components of every project. Our project-based approach helps avoid treating CBE as a checklist, keeping the focus on authentic learning experiences. While there is a general class pace, FlexTech offers flexibility through asynchronous learning for students who move faster or slower. Daily project support time and one-on-one appointments on Fridays provide additional individualized assistance to ensure students stay on track.

Grades at FlexTech focus purely on competency mastery, unlike traditional systems where behavior or extra credit might influence grades. Teachers provide ongoing formative feedback to help students gauge their progress. Each campus functions a little differently when it comes to its transcript. When I was the principal at the Oakland campus, we used a conversion scale because parents wanted a more traditional transcript. However, the Brighton and Shepherd campuses offer a fully competency-based transcript, which is accompanied by a one-page explanatory document for universities (and parents).

How do you measure success with regard to competency-based education? 

We measure success in multiple ways. Because we’re project-based, students have the flexibility to demonstrate mastery of academic competencies in creative ways, which isn’t restricted by a rigid curriculum. Our smaller size allows us to track student progress very closely. Beyond academic competencies, we focus on students’ personal growth through the competencies outlined in our Portrait of a Graduate (e.g., passionate, problem-solver, growth mindset). Our Senior Chronicle serves as a culminating portfolio where students reflect on how they’ve grown throughout their time with us, including their ability to meet our Portrait of a Graduate competencies. 

We also measure success by hearing stories of FlexTech students who are relieved of the traditional school “pressures” (e.g., lack of flexibility, social anxiety, inability to catch up) and are now finding their way. A fundamental part of FlexTech is taking students who are not doing well and helping them, whether that means helping them for five or maybe even six years. However, maintaining our identity and inclusivity within an education system with a strict four-year finish policy is a challenge.

What advice would you give to other school leaders interested in implementing competency-based education?

Start by staying true to your values and beliefs about teaching and learning. Implementing CBE is not just about helping students finish faster—it’s about embracing a philosophy that requires a complete system-wide shift. This means rethinking everything from your instructional framework to your scheduling, course offerings, and even your LMS. Developing a shared vision is crucial, and so is being prepared to make some concessions along the way. 

It’s also essential to get everyone on board from the beginning, especially with the “why” behind CBE. We have a very personalized intake process, where we interview every family and explain how FlexTech functions. We also offer sessions during curriculum nights to educate parents and students about competency-based education and what it means for their learning journey.

Lastly, consider piloting the program on a small scale first to see how it fits within your school or district before going full-scale.

Final thoughts

FlexTech’s project-based and competency-based model is designed to give students more autonomy in how they meet academic standards and develop essential skills. While this approach provides opportunities for personalized support, it also presents challenges, such as navigating external expectations for a four-year graduation timeline and finding internal systems that can effectively track student progress in a non-traditional format. FlexTech continues to adapt its model in response to these challenges while maintaining a focus on individualized student growth.

As computer science (CS) continues to grow in importance in K-12 education, understanding what motivates students to pursue this field is becoming increasingly vital. In a study, Dr. Aman Yadav from Michigan State University and Dr. Kristen DeBruler from Michigan Virtual studied how students’ motivation – beliefs about their abilities (self-efficacy), the perceived challenges of learning CS (cost), and the perceived value of the subject (value) –  shape their intentions to continue studying CS. 

Connecting Motivation to Computer Science

When applying these concepts to computer science, it becomes clear why motivation is crucial. CS is often seen as challenging, requiring complex problem-solving skills and a significant time investment. This perception can either motivate students who see the effort as worthwhile or discourage those who find the challenge too daunting. Moreover, understanding how CS is applied in real-world careers, like data analysis, can enhance students’ appreciation for its utility and relevance.

The Study: High School Students’ Experiences in Online CS Courses

The researchers focused on 44 high school students enrolled in online AP Computer Science courses, examining how self-efficacy, cost, and utility influenced their intention to continue studying CS. Here’s what they found:

Self-efficacy initially appeared to be a significant factor in predicting students’ intent to pursue CS. This means those who felt more capable in their CS courses were more inclined to continue. However, when other factors (cost and utility) were included in the analysis, self-efficacy’s impact diminished.

Perceived cost had a surprising effect. Students who believed that studying CS would require significant effort were actually more likely to want to continue! This finding challenges the assumption that high perceived cost always discourages engagement. It suggests that students might associate CS with a meaningful challenge worth their time and effort.

Utility value showed an unexpected negative relationship with intent to pursue CS. Students who saw a higher utility in studying CS were less likely to want to continue. One possible explanation is that students may feel the subject’s relevance but find the commitment to learning it too demanding, especially in an online setting where support and guidance might be limited.

What Does This Mean for Teaching Computer Science?

The findings highlight the complex ways in which students’ perceptions influence their motivation to study computer science. The idea that high perceived cost can increase motivation suggests that students who view CS as a challenge are willing to tackle it if they see the effort as rewarding. However, the negative relationship between utility value and intent to pursue suggests that even if students understand the importance of CS, they might need more support to overcome perceived difficulties.

For educators, these insights are essential. As more high schools introduce CS courses, especially online options, it’s crucial to:

• Provide support and resources to help students overcome the challenges of studying CS, ensuring they feel capable and confident.
• Highlight the real-world applications of CS, clarifying the subject’s utility and emphasizing how students can succeed despite the challenges.

To learn more and explore related research, you can read the following papers:

Lishinki, A. & Yadav, A. (2021). Self-evaluation interventions: Impact on self-efficacy and performance in introductory programming. ACM Transactions on Computing Education. DOI: 10.1145/3447378  

Lishinski, A., Yadav, A., Good, J., & Enbody, R. (2016). Learning to program: Gender differences and interactive effects of students’ motivation, goals, and self-efficacy on performance. In Proceedings of International Computing Educational Research (pp. 211-220). Melbourne, Australia: Association for Computing Machinery. DOI: 10.1145/2960310.2960328.

Do you remember the first time you picked up a hobby just for the fun of it? Maybe it was painting, coding, or playing the guitar. For me, it was writing stories in the margins of my school notebooks. That quiet joy of creating something, just because I loved it, stayed with me.

That’s the thing about hobbies. They’re more than just something to pass the time. They spark curiosity, build skills, and help shape who we become. And when we give kids the chance to explore their interests—especially outside of the classroom—we’re helping them discover parts of themselves they might not find otherwise.

Why Hobbies Matter

We’ve all seen the moment when a child lights up after trying something new and realizing they love it. Whether it’s cooking, sketching, building robots, or playing a team sport, these experiences help kids connect with their creativity and build confidence.

In the rush of school, homework, and screen time, hobbies can get pushed to the side. But carving out space for them is more important than ever. Hobbies offer kids something school can’t always provide: time to explore freely, take healthy risks, and express themselves without a grade attached.

They also build real-world skills. Through hobbies, kids learn how to problem-solve, stick with something challenging, manage their time, and work toward a goal. These are the kinds of skills that carry them through school and into adulthood.

In fact, research from the National Institutes of Health shows that leisure activities can reduce stress, boost brain function, and improve creativity. So yes, letting kids play, build, and explore can actually help them succeed in school, too.

Helping Kids Find a Hobby They Love

Not every child knows right away what excites them, and that’s okay. Exploration is part of the process. Here are a few ways to help them find something that sticks:

1. Offer a range of experiences

Let them try different things. Community centers and schools often offer low-cost classes or clubs that introduce kids to everything from music to robotics.

2. Notice what catches their attention

Do they draw in every notebook? Are they always building with blocks or tinkering with tech? Follow those clues and offer opportunities to go deeper.

3. Encourage group activities

Social hobbies like sports, theater, or coding clubs help kids connect with peers, build teamwork skills, and stay motivated.

4. Be flexible

It’s normal for kids to bounce between interests. Support the journey, not just the outcome. Over time, they’ll land on something that clicks.

5. Join in when you can

Some hobbies are more fun when they’re shared. Cook together. Build something as a family. In classrooms, invite students to showcase their interests and share their skills with others.

When Hobbies Become Pathways

Here’s something else to consider: hobbies can sometimes lead to future career paths. While it might be hard to imagine a child’s fascination with video games turning into something more, the reality is that many hobbies can translate into valuable career skills. For example, a love of gaming could lead to a career in game design or esports. Similarly, a passion for photography could evolve into a professional photography career.

With the rise of the gig economy and digital platforms, what starts as a hobby can often become a side hustle or even a full-time job. By supporting a child’s hobbies, we’re not just helping them develop as individuals—we’re also opening up future possibilities they might not have considered otherwise.

Resources to Explore

If you’re looking for resources to help your child discover new hobbies, there are plenty of places to start. Here are a few recommendations:

• Common Sense Media: Offers reviews and recommendations for educational tools, apps, and games to help kids discover new interests.
• Local Community Centers and Libraries: Many offer free or low-cost classes and events, especially after school or during school breaks.
• Youtube & Online Platforms: From guitar tutorials to animation guides, there are thousands of kid-friendly resources available. Educators can help curate safe, high-quality options for independent learning.
• Enrichment Programs: Courses like Wiz Kid Learning’s Roblox Game Design or other structured micro-courses offer students a chance to dive into their interests in a more guided way.

Supporting Hobbies Through Flexible Learning

Today’s learning environment gives us more flexibility than ever. With options like online enrichment courses, after-school clubs, and independent projects, students can explore their hobbies in ways that work with their schedule and learning style.

These flexible experiences help students build confidence and take ownership of their learning. And when a child sees their passion taken seriously, they’re more likely to stick with it and grow.

Final Thoughts

At the heart of it, hobbies are about more than just having fun. They’re tools for growth, connection, and self-discovery. Whether your child is learning an instrument, building games, or exploring nature, those experiences can shape who they are—and who they’ll become.

So keep encouraging their curiosity. Let them try new things. Celebrate the process, even when it’s messy or short-lived. Because those moments might be the start of something truly meaningful.

Want to help your child take the next step?

Explore enrichment courses designed to support student interests, build skills, and make learning feel like play.
Register Now

In an era where virtual learning is becoming increasingly prevalent, understanding the best practices for engaging students online is crucial. Researchers at the Michigan Virtual Learning Research Institute (MVLRI) have conducted a comprehensive study to uncover effective strategies used by virtual educators, particularly those that help disengaged and struggling students succeed. 

The following interview, part of our “Interview with a Researcher” blog series, shares some highlights from this research.

Why was it important to examine effective practices in virtual learning environments, especially for struggling students?

The shift to emergency remote instruction during the COVID-19 pandemic highlighted a significant disparity in the success of virtual learning implementations. Schools with pre-existing, well-established virtual teaching practices fared much better. We wanted to identify what made these programs successful and how new virtual teachers and administrators could adopt these practices to better engage all students, particularly those who are disengaged or struggling.

What were some of the key strategies identified for engaging disengaged or struggling students in virtual environments?

One of the most frequently used and effective strategies was providing frequent and specific feedback, which was reported by nearly 79% of educators. This type of feedback not only supports academic progress but also helps in building strong teacher-student relationships. Additionally, involving other adults, such as onsite mentors and parents, was idenfied as being crucial. Around 69% of educators communicated with the student’s onsite mentor, and 61% encouraged parental involvement. These strategies help bridge the gap created by the lack of physical presence in virtual learning.

Communication seems to be a recurring theme. Can you elaborate on the importance of communication in virtual learning environments?

Absolutely. Communication is the backbone of virtual education. Effective communication strategies include maintaining regular contact through various channels like LMS messaging, phone calls, and web conferencing tools. Many educators also emphasized the importance of being available for students through scheduled office hours or drop-in times. Establishing clear communication channels helps ensure that students, parents, and educators are on the same page, which is vital for student engagement and success.

The study also looked at professional development for virtual educators. What sources of professional development were found to be most effective?

Our findings showed that optional opportunities provided by the virtual school or program were considered the most effective, with over 50% of educators endorsing them. Conferences and informal peer mentoring were also highly valued. These professional development sources are preferred because they are immediately applicable and foster a sense of community among educators, which is essential for sharing best practices and support.

What challenges did educators face in virtual teaching, particularly in connecting with disengaged students?

One of the biggest challenges is the lack of face-to-face interaction, which makes it difficult to read body language and establish personal connections. This physical separation also complicates identifying the specific reasons behind a student’s disengagement. Additionally, educators mentioned difficulties in effectively communicating with parents and guardians, who are crucial allies in supporting student engagement and progress.

Based on your research, what recommendations would you give to new virtual teachers working with disengaged or struggling students?

Focus on building strong relationships with your students from the beginning. Use frequent, specific feedback to show students that you care about their progress. Keep open channels of communication and be flexible with your teaching methods to accommodate diverse learning needs. Also, involve parents and onsite mentors whenever possible to create a supportive network around the student. Flexibility, patience, and a personalized approach are key.

The insights from this study underscore the importance of tailored strategies, consistent communication, and community support in virtual learning environments. By focusing on relationship-building, providing specific feedback, and involving parents and mentors, educators can significantly improve engagement and success for all students, especially those who struggle. As virtual learning continues to evolve, these findings offer a valuable roadmap for educators seeking to enhance their practices and better support their students in a digital age.

Introduction

Helping students succeed in their online courses is a team effort involving leveraging course design principles and technology to facilitate learning. Instructional designers use technology to design courses that optimize students’ and teachers’ experiences. Technology operations staff help implement and adapt systems to meet users’ needs. Pacing, or how students move through a course, is important to student success. Cramming and submitting assignments out of their intended order are associated with poor course performance (DeBruler, 2021; Cuccolo & DeBruler, 2024; Michigan Virtual Learning Research Institute, 2019). Because online courses are dynamic environments facilitating learning “anytime, anyplace,” it is crucial to leverage technology and course design principles to support students’ pacing to optimize their experience. In this blog, we’ll explore the pivotal roles of instructional designers and technology operations in students’ experiences with pacing in online courses.

Expert Interviews

Michigan Virtual Learning Research Institute researchers talked with Kim Garvison and Megan Riggers from Michigan Virtual’s Instructional Product Development (iPD) team, and Kristen Crain Senior Director of Technology Operations about the interplay between technology and course design principles in addressing pacing. This blog highlights central themes that cut across our conversations. The transcript has been edited for clarity and brevity.

Functionality: Supporting Students, Teachers, and Guardians through LMS Features

Centering Students and Teachers

How is course pacing important within the context of course design?

We try to ensure the workload is evenly distributed. We consider when students might be doing these assignments and how we’re balancing intensity in terms of cognitive load for the student and the teacher. Another way we consider pacing is through how we distribute auto and teacher-graded assignments. Even though we have a short turnaround time for teachers to grade assignments, balancing the type of assignment does help move students through their course without having roadblocks.

What design choices have you found helpful for supporting students struggling with pacing in their online courses?

As designers, we have to think about our end users and how much is realistic for them to handle. What grade are they in? What’s their age? What’s their reading level? What can they handle? What experience might they have in an online setting? We try to create courses with predictable structures: approximately the same number of lessons and each unit follows the same general structure. That way, both teachers and students know what to expect. Predictability also helps mentors and parents support students more effectively during their courses.

We also ensure our content is accessible for students with different learning needs. For example, our LMS helps ensure we have appropriate alt text (descriptive text concisely conveying the meaning of an image). Having hurdles for students with different learning needs is a big deal for their pacing, and therefore is a big deal to us when designing a course.

Organizational Tools Aid in User Experience

Organizational tools provided by the LMS are critical for helping students navigate the course efficiently. For instructors, organization allows them to serve students more effectively by creating a centralized location where they can assess students’ progress.

How can technology or the LMS be leveraged to address pacing?

We are very intentional with setting up course navigation. We don’t want students thinking “What do I do next? How do I find the quiz? Where do I click?” That’s the background instructional design part that helps students work through their content. Having a good LMS helps us have students get friction from content and learning, and not from navigating their course. I think that the technology is what allows us to not think about the technology. The more none of us have to think about the technology, that’s a good sign that it’s working well for us.

Our LMS is heavily invested in simplifying the instructor and teacher experience. Brightspace centralizes everything so an instructor can see who is accessing content, who submitted content, and student grades in one view. LMSs also do a great job creating buckets of content, nesting units together, and keeping students moving through all the content linearly.

Reminders

Reminder tools within the LMS were viewed positively as a way to provide guide rails for students as they move through content at their own pace.

Is there anything that helps students adhere more closely to the pacing guides?

Instead of thinking, “Okay, on this date, we have to blast out this reminder,” for our courses, location is more important than time because we don’t know where the students will be within a course at a specific date.

In addition to strategically placed reminders within the LMS, data from the LMS can be used to create reminders that are pushed outward to students, and guardians.

LMSs can compile extensive amounts of data in a consumable way for a school, an instructor, and even parents. They can leverage the data into tools, widgets, calendars, emails, and other things that make the students’ experience more streamlined. For example, certain LMSs let parents opt into features such as automated daily emails telling them what’s due for their child.

Leveraging LMS to Promote Sequential Course Progression

How do you encourage students to follow the logical progression of the course content? Do you use any specific tools?

We use LMS features that should help students get back on track. For example, we have a checklist of assignments at the beginning of each unit. There are also LMS features at the end of lessons and units pointing out incomplete assignments and reminding students to go back. Our LMS also provides the option to put a password on an assignment. We do this for all final exams as another way to say, “Hey, this is important.”

We make sure everything is scaffolded so it’s clear to students that they have to go through certain assignments before they can do the next project. If students skip content, the teacher can point out that they don’t meet the rubric requirements because we develop rubrics that emphasize the lesson’s content.

We also have an internally designed pacing guide application that’s accessible by the student and the mentor so they can see a week-by-week breakdown of what students should accomplish including graded and non-graded activities.

Addressing Assignment Cherry-Picking

We’ve conducted research showing students tend to favor auto-graded and higher-point assignments. What are some potential workarounds for addressing this cherry-picking?

We try to anticipate it and consider the point spread and the ratio of auto-graded to teacher-graded assignments. We don’t want a student to be able to only take the auto-graded quizzes and pass the course. Usually, we go for approximately 40% auto-graded to 60% teacher-graded.

One way to address this is to incorporate conditional releases which can function based on completion and unlock specific content, units, or modules. We don’t use this tool often, but it can create different learning methods for students. One student may come into a course and say, “I want to learn today via video.” Then they go into the content, choose the method they want, and it unlocks content that’s all video-based instruction.

We leave everything open though, and that’s a transparency piece so the student can see the whole scope of the class and every graded object upfront. Sometimes waiting for a teacher to grade an assignment before being able to move on frustrates students. In addition, only allowing students to see one unit at a time (rather than the whole course) can add another layer of frustration. From an instruction design standpoint, it seems easy – just lock it until they pass. However, from the user side, that’s not always how it goes.

Thinking Ahead

What would an LMS that perfectly addresses students’ pacing needs look like?

One that automatically sets a student’s pace as they start the course based on its duration, and is flexible enough to empower teachers or administrators to override, re-pace, or modify students’ pace in bulk since they’re dealing with large course loads of students.

It would be neat if courses had their own AI bot that could provide students with reminders like, “Hey, you should be working on this assignment,” or “Hey, it looks like you are behind here. Your new due date is.”

Course Pacing Blog Series

In our Course Pacing blog series, we discuss pacing and how it impacts student success with input from several different subject matter experts. Our hope with this series is to bring to light how different organizations and experts approach course pacing, share their insights and struggles, provide relevant research and resources, and determine areas for future research. Stay up to date on future blogs in this series by signing up for email notifications!

One of the benefits of online learning is that students can work at their own pace. However, not all students have developed the time management skills to work through a course consistently. Perhaps unsurprisingly, research has shown that consistent course pacing results in higher student achievement (DeBruler, 2021). 

To explore course pacing through the lens of an online program administrator, researchers from the Michigan Virtual Learning Research Institute (MVLRI) interviewed Andrea McKay, Director of Instruction for Michigan Virtual in April 2024. The transcript from our interview has been edited for clarity and brevity. During our conversation, we explored how through intentional decisions made by online program administrators and support from both online teachers and on-site mentors, students are guided and supported to stay on pace in their online courses. 

Understanding Course Pacing and Developing Pacing Guides

How is course pacing currently addressed in Michigan Virtual courses? For example, how do you ensure students keep up without feeling rushed?

We provide pacing guides within our courses that show students a week-by-week breakdown of assignments to complete. Our teachers can access and adjust the pacing guide if students get behind and need a new plan to help them catch up. Once students have enrolled, their start date remains the same, and the length of the term remains the same; however, we can adjust the pacing guide so students see what assignments they need to complete within the shorter working time they have as a result of falling behind. 

How do you develop pacing guides for your courses? 

After a course is designed, the pacing guide is developed by splitting up the content and assignments over the number of weeks in the length of the term. Subject matter experts who understand the content and assignment expectations evaluate the pacing guide to ensure the pace is realistic. In addition, our courses are regularly updated which includes re-evaluating the pacing guide and making necessary adjustments. 

Tools for Pacing, Progress Tracking, and Data Utilization

Are there any tools or platforms that Michigan Virtual uses to help students keep pace and track their progress?

Teachers send monthly progress reports to the student, mentor, and parent so everyone understands the student’s course progress. Progress reports are personalized to each student by pulling data from our LMS (Brightspace) and student information system (SIS) such as the number of complete and incomplete assignments, current grade, and total assignments in the course. Some teachers add information such as personalized comments or a reminder of the course end date. For example, “Are you on track to complete the course by [course end date]? With a few adjustments on the back end, this tool within our LMS allows teachers to send these personalized progress reports to their entire course roster. In addition, teachers use our SIS to sort students in a course by start date and, based on that, send timely communication to help keep students on pace. Being able to sort students within a course by start date is very important in terms of accurately tracking student progress as we offer many different course start dates within each semester to better meet school districts’ needs for course start dates to align with their school calendar.

Online Program Course Pacing: Challenges and Solutions

What major challenges do you run into regarding course pacing in an online setting?

One major challenge is grade reporting. A common request from schools is to know a student’s current grade to determine whether that student is eligible to play sports. Unfortunately, the way our systems report students’ scores does not give them that information. In our courses, students start with zero points and build up their total course points with every assessment submitted. We provide an accurate display of their overall score at all times, but it’s not the same as a weekly grade that schools may be more used to. However, adjusting our grade reporting process would result in far fewer start and end date options and reduced flexibility in submission deadlines.

What do you think are the biggest pacing hurdles for students learning online? 

While the flexibility to work at your own pace is a common reason why many students take courses with us, some students need help to learn how to manage their time effectively. If a student gets behind, assignments add up quickly, resulting in a poor learning experience and a mountain of assignments to submit. 

What challenges do teachers face with pacing in their online courses? 

If instructors are overloaded with a flurry of assignment submissions at the end of a course, they cannot provide the same feedback quality while also meeting grading turnaround expectations. The resulting assignment feedback isn’t as effective because those students procrastinating and turning in numerous assignments during the last few weeks are probably not as concerned about whether they didn’t quite meet a learning target, what content they might need to revisit, or how to use and grow from instructor feedback—they’re just trying to get through it. An additional problem when students leave so many assignments for the end of the course is that they might be tempted to take some shortcuts and plagiarize, which turns into another huge headache at the end of a term. We have found the number of plagiarism incidents increases drastically in the last few weeks of a course. As a result, teachers communicate progress regularly and try to keep students on pace. 

Given our recent finding that the extent to which students submit assignments out of order is associated with lower grades, what’s your take on getting students to stick to the order and pace you set out?

Teachers understand that students may choose to complete assignments based on which ones will earn them a higher score more quickly, otherwise known as cherry-picking. Unfortunately, this means that rather than working sequentially through courses designed to scaffold and build skills as students progress, students are sometimes more focused on points than learning. Our teachers recognize this and will reach out to students to get them back on track. 

Pacing: Mentor and Parent Roles

Do mentors and parents play a part in monitoring students’ pacing? What does that collaboration look like?

There is a triangle of student support between the teacher, mentor, and (ideally) the parent as they are a critical component in overall student success (Borup & Stimson, 2017; Borup et al., 2017). Both mentors and parents are copied on progress reports to remain aware of student pace. In addition, specific mentor reports and monitoring tools are available through our LMS, Brightspace, to help mentors support students most effectively. Teachers understand that parents and mentors may know students in ways that they do not, so we all value the roles and support they provide and work hard to establish and maintain open lines of communication. 

Looking Down the Road

Are there any big changes you would like to see or pacing challenges you are preparing for in K-12 online learning?

In addition to continuing to meet districts’ needs for different course start/end dates and providing the flexibility of an adjustable pacing guide, we would love to offer a different view of students’ scores based on current progress and performance as school districts so frequently request it. This is a current limitation of our flexible scheduling options and how they interact with the LMS; however, we are working to find solutions. Despite students having access to a pacing guide and inevitably tallying up points, my other hope is to get to a point where students think more about their learning rather than the points. 

Course Pacing Blog Series

In this Course Pacing Blog Series, we discuss pacing and how it impacts student success with input from several different subject matter experts. Our hope with this series is to bring to light how different organizations and experts approach course pacing, share their insights and struggles, provide relevant research and resources, and determine areas for future research. Stay up to date on future blogs in this series by signing up for email notifications!

References

Borup, J. & Stimson, R. (2017). Helping students be successful: Mentor responsibilities. Michigan Virtual University. https://michiganvirtual.org/research/publications/helping-online-students-be-successful-mentor-responsibilities/  

Borup, J., Chambers, C. B., Stimson, R. (2017). Helping online students be successful: Parental engagement. Michigan Virtual University. https://michiganvirtual.org/research/publications/helping-online-students-be-successful-parental-engagement/ 

DeBruler, K. (2021). Research On K-12 Online Best Practices. Michigan Virtual. https://michiganvirtual.org/blog/research-on-k-12-online-best-practices

Acknowledgments

The author would like to thank Andrea McKay and Dr. Shannon Smith from Michigan Virtual’s Learning Services team as well as Dr. Kelly Cuccolo and Dr. Kristen DeBruler from the Michigan Virtual Learning Research Institute for their contributions and advice in developing this blog post.

In the ever-evolving landscape of education, online learning has become increasingly prevalent, offering students flexibility and accessibility to course materials. Recent research has delved into the dynamics of online coursework completion, particularly focusing on the sequencing of assignments and its impact on student success. Understanding how students navigate through their coursework, whether adhering to prescribed pacing guides or deviating from them, provides valuable insights for educators aiming to enhance student outcomes. 

The following interview from our “Interview with a Researcher” blog series shares some highlights from this research.

Is it common for students to go out of sequence when completing online course assignments?

Absolutely. Our study revealed that a vast majority of students, around 93%, tended to go out of sequence at least once when completing assignments in online STEM courses. This suggests that flexibility in pacing is a common practice among online learners.

How does the average number of assignments submitted out of sequence affect students’ performance?

We found that, on average, students submitted approximately 17.5 assignments out of order, accounting for about 38.15% of all course assignments. While some deviation from the prescribed order is expected, this trend indicates that a significant portion of students are not strictly following the intended sequencing of assignments. We observed that as students became increasingly out of sequence (submitted a higher proportion of assignments out of order) their grades dropped such that students with the highest number of assignments out of order had the lowest final course grades.

Could you explain the relationship between students’ course progression and their final grades?

Certainly. Our research uncovered a significant negative relationship between the proportion of assignments completed out of order, as well as the magnitude of these deviations, and students’ final grades. Essentially, as students submitted more assignments out of sequence, their final grades tended to decrease accordingly.

Are there noticeable differences in final grades between students who go out of sequence and those who stay in sequence?

Yes, indeed. We observed a clear distinction in final grades between students who adhered to the intended sequencing of assignments and those who did not. On average, students who stayed in sequence achieved a final grade of 89.2, whereas those who went out of sequence averaged 79.7. When we grouped students by how often they submitted assignments out of sequence the discrepancies in final grades were even more pronounced. Students in the bottom 25% for being out of sequence (the least proportion of “out of order” assignment submissions) consistently had the highest grades on average, and those in the top 25% (the highest proportion of “out of order” assignments) had the lowest, 86.8 compared to 74.1–a difference of 12.7 points. This sizable difference underscores the impact of following the prescribed order of assignments on student success.

What recommendations do you have for educators to support students in navigating online course assignments?

Educators play a pivotal role in guiding students toward effective online learning strategies. It is essential to emphasize the significance of following pacing guides and completing assignments in the intended order. Providing clear expectations, scaffolding course content, and explaining the purpose of assignments can help students grasp the value of pacing and organization. Moreover, supporting students in developing self-regulatory skills and effective time management practices empowers them to succeed in online learning environments.

The insights gleaned from this study illuminate the intricate relationship between assignment sequencing and student success in online STEM courses. As online learning continues to evolve, understanding how students navigate their coursework, the impact of pacing deviations, and the role of educators in guiding them toward success becomes increasingly vital. By embracing these findings, educators can empower students to effectively manage their coursework, enhance engagement, and ultimately achieve academic success in the digital realm of education.

Be sure to check out the full research report for more information! In addition, this blog is part of a blog series exploring the impact of student assignment submission patterns.

In recent years, the landscape of education has undergone a significant transformation, with online learning becoming increasingly prevalent. Like many other states, Michigan has witnessed a surge in virtual education, with approximately 14% of K-12 students engaging in at least one virtual course during the 2021-2022 academic year, boasting a pass rate of 69% (Freidhoff, 2023). However, while the accessibility and flexibility of online learning are undeniable, ensuring student success in this domain presents its own set of challenges.

The Crucial Role of Pacing

Research has shed light on one key aspect that significantly influences student outcomes in online courses—pacing. Pacing refers to how students progress through course material over time, and it has emerged as a crucial determinant of success (DeBruler, 2021; Michigan Virtual Learning Research Institute, 2019). Studies indicate that students’ early engagement, such as submitting assignments within the first week, correlates positively with final grades, suggesting a strong link between pacing and engagement (Zweig, 2023).

Consistency in pacing throughout the course is also pivotal. Students who maintain a steady pace are more likely to succeed compared to those who exhibit erratic pacing behaviors, such as cramming assignments toward the end of the course (DeBruler, 2021). The significance of pacing becomes even more pronounced in online courses that lack firm deadlines, where students have the flexibility to progress at their own pace (Martin & Whitmer, 2016; Wakeling & Robertson, 2017).

Guidance and Structure: Pacing Guides

To assist students in navigating the challenges of pacing, some online course providers, like Michigan Virtual, provide pacing guides. These guides offer a structured roadmap of assignments and activities for each week or sequence, serving as a benchmark for students to evaluate their progress. While not mandatory, adhering to these pacing guides can significantly aid students in managing their workload, particularly in the absence of strict deadlines (DeBruler, 2021).

Understanding Engagement: Learning Trajectories and Sequencing

Delving deeper into the realm of online learning, growth mixture modeling (GMM) has revealed various learning trajectory profiles in K-12 mathematics courses. These profiles offer valuable insights into how students engage with course material and how their engagement patterns are intertwined with pacing and overall performance (Kwon, 2018; Kwon & DeBruler, 2019). 

The first profile illustrates nearly linear, on-pace progression, prevalent in advanced placement (AP) courses characterized by a cohort model. This underscores the importance of structured pacing and collective progress (Kwon, 2018).

Conversely, the second profile depicts a steep increase in learner scores near the end of the semester, often unattainable for many learners, particularly those in foundation courses like Algebra and Geometry, or courses for credit recovery (Kwon & DeBruler, 2019).

The third profile portrays minimal progress over the semester, indicative of disengagement leading to eventual withdrawal from courses (Kwon, 2017).

Lastly, strong early achievement signifies a group with a linear progression from the beginning to the end of the semester, often characterized by a surge in time investment towards the final weeks (Kwon & DeBruler, 2019).

In essence, these learning trajectory profiles serve as a testament to the intricate dance between engagement patterns and course pacing in online learning environments. By recognizing the pivotal role of pacing in shaping students’ online learning journeys, educators can implement strategies to scaffold pacing, provide timely interventions, and foster a supportive learning environment conducive to student success.

MVLRI also investigated student course sequencing and its relationship to student achievement. This study sheds light on the impact of assignment sequencing on student performance. It reveals a significant negative correlation between submitting assignments out of order and final grades. Specifically, students who adhered to the prescribed sequence achieved final grades averaging 9.5 points higher than their counterparts who did not (Cuccolo & DeBruler, 2024). Students were also grouped into one of four groups based on the proportionality of out-of-sequence submissions. 

Students who stayed the most in sequence had final grades averaging approximately 13 points higher than the most out-of-sequence student group. The findings underscore the importance of emphasizing the significance of pacing guides and adherence to sequential assignment submissions to enhance student academic success. Educators should consider incorporating strategies to promote adherence to course pacing and assignment sequencing to optimize student outcomes.

Implications for Success

Cumulatively, these findings bring to light the nuanced relationship between engagement patterns, pacing, and student success in online learning. While linear progression and early achievement are indicative of positive outcomes, they may not be representative of the entire student population. The prevalence of late surges in engagement suggests varied approaches to pacing, with no one-size-fits-all solution (Kwon, 2017b).

Navigating the Path to Success

In light of these insights, it’s evident that fostering success in online learning requires a multifaceted approach. Institutions must prioritize providing students with the necessary guidance and structure, such as pacing guides, to facilitate steady progress through course material while still offering the flexibility afforded by online learning. Moreover, educators and policymakers must recognize the diverse learning trajectories exhibited by students and tailor support mechanisms accordingly.

Conclusion

As online learning continues to shape the educational landscape, understanding the intricacies of pacing and engagement is paramount. By leveraging insights from research and best practices, we can empower students to navigate the complexities of virtual education successfully. Ultimately, by fostering a supportive and adaptive learning environment, we can ensure that all students have the opportunity to thrive in online learning.

Course Pacing Blog Series

In our Course Pacing Blog Series, we discuss pacing and how it impacts student success with input from several different subject matter experts. Our hope with this series is to bring to light how different organizations and experts approach course pacing, share their insights and struggles, provide relevant research and resources, and determine areas for future research. Stay up to date on future blogs in this series by signing up for email notifications!

References

Cuccolo, K. & DeBruler, K. (not published). Charting the course: Understanding student sequencing and achievement. 

DeBruler, K. (2021). Research On K-12 Online Best Practices. Michigan Virtual. https://michiganvirtual.org/blog/research-on-k-12-online-best-practices/

Freidhoff, J. R. (2023). Michigan’s k-12 virtual learning effectiveness report 2021-22. Michigan Virtual. https://michiganvirtual.org/research/publications/michigans-k-12-virtual-learning-effectiveness-report-2021-22/

Kwon, J. B. (2017a). Growth modeling with LMS data: Data preparation, plotting, and screening. Michigan Virtual University. https://michiganvirtual.org/research/publications/growth-modeling-with-lms-data-data-preparation-plotting-and-screening/

Kwon, J. B. (2017b). Course engagement patterns in mathematics and non-mathematics courses. Michigan Virtual University. https://michiganvirtual.org/research/publications/course-engagement-patterns-in-mathematics-and-non-mathematics-courses/

Kwon, J. B. (2018). Learning trajectories in online mathematics courses. Michigan Virtual University. https://michiganvirtual.org/research/publications/learning-trajectories-in-online-mathematics-courses/

Kwon, J. B. & DeBruler, K. (2019, September 26). Pacing Guide for Success in Online Mathematics Courses. Michigan Virtual. https://michiganvirtual.org/blog/pacing-guide-for-success-in-online-mathematics-courses/ 

Martin, F., & Whitmer, J. C. (2016). Applying learning analytics to investigate timed release in online learning. Technology, Knowledge and Learning, 21, 59-74. https://doi.org/10.1007/s10758-015-9261-9

Michigan Virtual Learning Research Institute. (2019). Pacing Guide For Success In Online Mathematics Courses. https://michiganvirtual.org/blog/pacing-guide-for-success-in-online-mathematics-courses/ 

Wakeling, V., & Robertson, P. R. (2017). A comparison of student behavior and performance between an instructor-regulated versus student-regulated online undergraduate finance course. American Journal of Educational Research, 5(8), 863. https://digitalcommons.kennesaw.edu/cgi/viewcontent.cgi?article=5075&context=facpubs

Zweig. J. (2023). The first week in an online course: Differences across schools. Michigan Virtual. https://michiganvirtual.org/research/publications/first-weeks-in-an-online-course/