Impact Evaluation of
Imagine MyPath® in a
Northeastern School
District
2021–2022
Imagine MyPath® in a
Northeastern School
District
2021–2022
IntroductionWhile it is clear from the National Assessment of Educational Progress that the majority of
students across the nation are behind in reading and math (NAEP, 2022), the academic diversity
within classrooms makes it incredibly difficult for teachers to address individual student needs
(Smale-Jacobse et al, 2019). Imagine Learning’s adaptive intervention program, Imagine MyPath,
is designed to address this problem. Imagine MyPath’s technology prioritizes essential skills and
creates individualized learning paths, which continuously adapt to address the varying needs of
academically diverse learners. Ultimately, use of the program is anticipated to close achievement
gaps and maximize academic growth for K–12 students in reading and mathematics.
Imagine Learning commissioned research after the 2021–2022 school year to measure the
impact of Imagine MyPath on student learning outcomes. To accomplish this goal, Imagine
Learning partnered with a northeastern public school district that implemented Imagine MyPath
for supplemental math and reading instruction with students in Grades 2–5. The following report
describes this implementation as it relates to the study, details the results of the analyses, and
discusses the implications of the findings.
Methods
POPULATION
Imagine MyPath was implemented with students in Grades 2–5 at a public school district during
the 2021–2022 school year. The partnering district is located in the northeastern region of the
United States and serves approximately 8,000 students. The program was used at teachers’
discretion in the classroom. A total of 1,877 students used Imagine MyPath for supplemental
math instruction, and 1,913 students used Imagine MyPath for supplemental reading instruction.
Conversely, there were 92 students who did not use Imagine MyPath for math, and 93 students
who did not use Imagine MyPath for reading. Because use of Imagine MyPath in the schools
was determined for individual students (rather than for entire classrooms or schools), statistical
corrections for clustering were not required. Appendix A details the distribution of Imagine
MyPath users and non-users across schools and grades.
RESEARCH DESIGN
This study followed a quasi-experimental design and was conducted retrospectively using data
from the 2021–2022 school year. Because Imagine MyPath offers distinct instruction in math
and reading, and it is possible for students to have used Imagine MyPath for one subject and
not the other, the two subjects were analyzed separately. For the math analysis, the treatment
group was comprised of all students who used Imagine MyPath for at least one minute of
math instruction, while the control group was comprised of all students who did not log any
usage in the math program. Similarly, the treatment group for reading was comprised of all
1Impact Evaluation of Imagine MyPath in a Northeastern School District 2021–2022
students across the nation are behind in reading and math (NAEP, 2022), the academic diversity
within classrooms makes it incredibly difficult for teachers to address individual student needs
(Smale-Jacobse et al, 2019). Imagine Learning’s adaptive intervention program, Imagine MyPath,
is designed to address this problem. Imagine MyPath’s technology prioritizes essential skills and
creates individualized learning paths, which continuously adapt to address the varying needs of
academically diverse learners. Ultimately, use of the program is anticipated to close achievement
gaps and maximize academic growth for K–12 students in reading and mathematics.
Imagine Learning commissioned research after the 2021–2022 school year to measure the
impact of Imagine MyPath on student learning outcomes. To accomplish this goal, Imagine
Learning partnered with a northeastern public school district that implemented Imagine MyPath
for supplemental math and reading instruction with students in Grades 2–5. The following report
describes this implementation as it relates to the study, details the results of the analyses, and
discusses the implications of the findings.
Methods
POPULATION
Imagine MyPath was implemented with students in Grades 2–5 at a public school district during
the 2021–2022 school year. The partnering district is located in the northeastern region of the
United States and serves approximately 8,000 students. The program was used at teachers’
discretion in the classroom. A total of 1,877 students used Imagine MyPath for supplemental
math instruction, and 1,913 students used Imagine MyPath for supplemental reading instruction.
Conversely, there were 92 students who did not use Imagine MyPath for math, and 93 students
who did not use Imagine MyPath for reading. Because use of Imagine MyPath in the schools
was determined for individual students (rather than for entire classrooms or schools), statistical
corrections for clustering were not required. Appendix A details the distribution of Imagine
MyPath users and non-users across schools and grades.
RESEARCH DESIGN
This study followed a quasi-experimental design and was conducted retrospectively using data
from the 2021–2022 school year. Because Imagine MyPath offers distinct instruction in math
and reading, and it is possible for students to have used Imagine MyPath for one subject and
not the other, the two subjects were analyzed separately. For the math analysis, the treatment
group was comprised of all students who used Imagine MyPath for at least one minute of
math instruction, while the control group was comprised of all students who did not log any
usage in the math program. Similarly, the treatment group for reading was comprised of all
1Impact Evaluation of Imagine MyPath in a Northeastern School District 2021–2022
students who used Imagine MyPath for at least one minute of reading instruction, while the
control group was comprised of all students who did not log any usage in the reading program.
Since the identified study sample included a proportionally smaller number of students with zero
exposure to the Imagine MyPath program for reading or mathematics, statistical procedures
were employed to ensure baseline equivalence of the analytical treatment and control samples.
Specifically, for every student that did not use the Imagine MyPath program, a statistically
similar (based on demographics and baseline achievement) student was identified who used
Imagine MyPath for more than one minute during the school year. This strategy significantly
reduced the analytical sample to 70–75 students who used Imagine MyPath and 70–75 students
who did not use Imagine MyPath. However, in utilizing this statistical methodology to identify
study groups, we could be confident that the study groups were highly similar and that no
external factors could explain detected differences between study groups.
MEASURES
The data used in this study included student-level demographic information, NWEA MAP
Growth RIT scores, and Imagine MyPath program data. These data sources are reviewed in
more detail below.
Math Proficiency. Students completed the NWEA MAP Growth Math assessment in Fall 2021 and
Spring 2022. The average number of days between the Fall 2021 and Spring 2022 assessments
was 233 (229 days for students in the control group and 233 days for students in the treatment
group). Fall 2021 scores were used to establish baseline equivalence between study groups, and
Spring 2022 scores were used to estimate the effect of Imagine MyPath on math proficiency.
Reading Proficiency. Students completed the NWEA MAP Growth Reading assessment in Fall
2021 and Spring 2022. The average number of days between the Fall 2021 and Spring 2022
assessments was 227 (225 days for students in the control group and 227 days for students in
the treatment group). Fall 2021 scores were used to establish baseline equivalence between
study groups, and Spring 2022 scores were used to estimate the effect of Imagine MyPath on
reading proficiency.
Student Demographics. Student-level demographic data were collected for grade level, race/
ethnicity, English language classification, special education classification, and free/reduced
price lunch status for the 2021–2022 school year.
Imagine MyPath Usage. Imagine MyPath program usage data was collected to determine study
groups and better understand treatment students’ engagement and progress in the program.
These data included the total minutes students spent in math and reading instruction and the
number of lessons students passed (defined as lessons that students completed and achieved
at least 80% accuracy on the post-lesson quiz).
2Impact Evaluation of Imagine MyPath in a Northeastern School District 2021–2022
control group was comprised of all students who did not log any usage in the reading program.
Since the identified study sample included a proportionally smaller number of students with zero
exposure to the Imagine MyPath program for reading or mathematics, statistical procedures
were employed to ensure baseline equivalence of the analytical treatment and control samples.
Specifically, for every student that did not use the Imagine MyPath program, a statistically
similar (based on demographics and baseline achievement) student was identified who used
Imagine MyPath for more than one minute during the school year. This strategy significantly
reduced the analytical sample to 70–75 students who used Imagine MyPath and 70–75 students
who did not use Imagine MyPath. However, in utilizing this statistical methodology to identify
study groups, we could be confident that the study groups were highly similar and that no
external factors could explain detected differences between study groups.
MEASURES
The data used in this study included student-level demographic information, NWEA MAP
Growth RIT scores, and Imagine MyPath program data. These data sources are reviewed in
more detail below.
Math Proficiency. Students completed the NWEA MAP Growth Math assessment in Fall 2021 and
Spring 2022. The average number of days between the Fall 2021 and Spring 2022 assessments
was 233 (229 days for students in the control group and 233 days for students in the treatment
group). Fall 2021 scores were used to establish baseline equivalence between study groups, and
Spring 2022 scores were used to estimate the effect of Imagine MyPath on math proficiency.
Reading Proficiency. Students completed the NWEA MAP Growth Reading assessment in Fall
2021 and Spring 2022. The average number of days between the Fall 2021 and Spring 2022
assessments was 227 (225 days for students in the control group and 227 days for students in
the treatment group). Fall 2021 scores were used to establish baseline equivalence between
study groups, and Spring 2022 scores were used to estimate the effect of Imagine MyPath on
reading proficiency.
Student Demographics. Student-level demographic data were collected for grade level, race/
ethnicity, English language classification, special education classification, and free/reduced
price lunch status for the 2021–2022 school year.
Imagine MyPath Usage. Imagine MyPath program usage data was collected to determine study
groups and better understand treatment students’ engagement and progress in the program.
These data included the total minutes students spent in math and reading instruction and the
number of lessons students passed (defined as lessons that students completed and achieved
at least 80% accuracy on the post-lesson quiz).
2Impact Evaluation of Imagine MyPath in a Northeastern School District 2021–2022
ANALYTICAL SAMPLES
Separate analytical samples were created for the math and reading analyses. To ensure that the
baseline characteristics of treatment and control students used in each analysis were comparable,
1:1 propensity score matching without replacement was used to create a statistically equivalent
analytical sample.1 Control students were matched to treatment students based on their Fall
2021 NWEA Map Growth RIT score and all demographic information available: race/ethnicity,
English language classification, special education classification, and free/reduced price lunch
status. This matching process was conducted separately for math and reading, to create a math
analytical sample and reading analytical sample. It was also completed on each grade individually
before combining the matched grade level samples to create the total analytical samples.
For math, the resulting analytical sample included 75 users of Imagine MyPath and 75 non-users.
For reading, the resulting analytical sample included 70 users of Imagine MyPath and 70
non-users. Table 1 describes the characteristics of the analytical samples.
ANALYTICAL APPROACH
Multiple linear regression was used to evaluate the differences in Spring 2022 NWEA MAP
Growth achievement between Imagine MyPath users and non-users, controlling for Fall 2022
MAP Growth achievement and other covariates (including grade level, race/ethnicity, special
education classification, and free/reduced price lunch status). An indicator of whether a student
was a control or treatment student was included in the regression as the primary predictor
variable. Using multiple linear regression after propensity score matching ensured that any
remaining differences in the underlying treatment and control samples were controlled for by
the regression model, effectively isolating the impact of Imagine MyPath. Separate analyses
were conducted for reading and math outcomes.
1 Propensity score matching was executed using the matchit function in R’s MatchIt package. For the Imagine
MyPath Math sample, nearest neighbor matching with a caliper of 0.1 and exact matching on race/ethnicity was
used for Grade 2; optimal pair matching was used for Grades 3-5. For the Imagine MyPath Reading sample, op-
timal pair matching was used for Grade 2; nearest neighbor matching with a caliper of 0.09 and exact matching
on race/ethnicity was used for Grades 3-5.
3Impact Evaluation of Imagine MyPath in a Northeastern School District 2021–2022
Separate analytical samples were created for the math and reading analyses. To ensure that the
baseline characteristics of treatment and control students used in each analysis were comparable,
1:1 propensity score matching without replacement was used to create a statistically equivalent
analytical sample.1 Control students were matched to treatment students based on their Fall
2021 NWEA Map Growth RIT score and all demographic information available: race/ethnicity,
English language classification, special education classification, and free/reduced price lunch
status. This matching process was conducted separately for math and reading, to create a math
analytical sample and reading analytical sample. It was also completed on each grade individually
before combining the matched grade level samples to create the total analytical samples.
For math, the resulting analytical sample included 75 users of Imagine MyPath and 75 non-users.
For reading, the resulting analytical sample included 70 users of Imagine MyPath and 70
non-users. Table 1 describes the characteristics of the analytical samples.
ANALYTICAL APPROACH
Multiple linear regression was used to evaluate the differences in Spring 2022 NWEA MAP
Growth achievement between Imagine MyPath users and non-users, controlling for Fall 2022
MAP Growth achievement and other covariates (including grade level, race/ethnicity, special
education classification, and free/reduced price lunch status). An indicator of whether a student
was a control or treatment student was included in the regression as the primary predictor
variable. Using multiple linear regression after propensity score matching ensured that any
remaining differences in the underlying treatment and control samples were controlled for by
the regression model, effectively isolating the impact of Imagine MyPath. Separate analyses
were conducted for reading and math outcomes.
1 Propensity score matching was executed using the matchit function in R’s MatchIt package. For the Imagine
MyPath Math sample, nearest neighbor matching with a caliper of 0.1 and exact matching on race/ethnicity was
used for Grade 2; optimal pair matching was used for Grades 3-5. For the Imagine MyPath Reading sample, op-
timal pair matching was used for Grade 2; nearest neighbor matching with a caliper of 0.09 and exact matching
on race/ethnicity was used for Grades 3-5.
3Impact Evaluation of Imagine MyPath in a Northeastern School District 2021–2022
Table 1. Student Characteristics of the Analytical Samples.
Math
Comparison
Students (n = 75)
Imagine MyPath
Students (n = 75) p-value
Standardized Mean
Difference (SMD)
Average (SD) Fall 2021 NWEA
MAP Growth RIT Score 169.67 (15.36) 169.60 (14.28) .978 0.004
Grade Level >.999 <0.001
Grade 2 17 17
Grade 3 34 34
Grade 4 14 14
Grade 5 10 10
Race/Ethnicity .993 .081
White 6 6
Black or African American 4 3
Hispanic or Latino 56 58
Multi-ethnic 1 1
Not Specified or Other 8 7
English Language
classification >.999 <0.001
Yes 19 19
No 56 56
Special education
classification >.999 <0.001
Yes 29 29
No 46 46
Free/reduced price lunch >.999 <0.001
Yes 41 41
No 34 34
4Impact Evaluation of Imagine MyPath in a Northeastern School District 2021–2022
Math
Comparison
Students (n = 75)
Imagine MyPath
Students (n = 75) p-value
Standardized Mean
Difference (SMD)
Average (SD) Fall 2021 NWEA
MAP Growth RIT Score 169.67 (15.36) 169.60 (14.28) .978 0.004
Grade Level >.999 <0.001
Grade 2 17 17
Grade 3 34 34
Grade 4 14 14
Grade 5 10 10
Race/Ethnicity .993 .081
White 6 6
Black or African American 4 3
Hispanic or Latino 56 58
Multi-ethnic 1 1
Not Specified or Other 8 7
English Language
classification >.999 <0.001
Yes 19 19
No 56 56
Special education
classification >.999 <0.001
Yes 29 29
No 46 46
Free/reduced price lunch >.999 <0.001
Yes 41 41
No 34 34
4Impact Evaluation of Imagine MyPath in a Northeastern School District 2021–2022
Table 1. Student Characteristics of the Analytical Samples cont.
Reading
Comparison
Students (n = 70)
Imagine MyPath
Students (n = 70) p-value
Standardized Mean
Difference (SMD)
Average (SD) Fall 2021 NWEA
MAP Growth RIT Score 175.94 (16.91) 177.09 (18.62) .704 0.064
Grade Level >.999 <0.001
Grade 2 29 29
Grade 3 25 25
Grade 4 9 9
Grade 5 7 7
Race/Ethnicity >.999 <0.001
White 5 5
Black or African American 8 8
Hispanic or Latino 42 42
Multi-ethnic 3 3
Not Specified or Other 12 12
English Language
classification >.999 <0.001
Yes 11 11
No 59 59
Special education
classification >.999 0.034
Yes 17 16
No 53 54
Free/reduced price lunch >.999 .029
Yes 27 26
No 43 44
5Impact Evaluation of Imagine MyPath in a Northeastern School District 2021–2022
Reading
Comparison
Students (n = 70)
Imagine MyPath
Students (n = 70) p-value
Standardized Mean
Difference (SMD)
Average (SD) Fall 2021 NWEA
MAP Growth RIT Score 175.94 (16.91) 177.09 (18.62) .704 0.064
Grade Level >.999 <0.001
Grade 2 29 29
Grade 3 25 25
Grade 4 9 9
Grade 5 7 7
Race/Ethnicity >.999 <0.001
White 5 5
Black or African American 8 8
Hispanic or Latino 42 42
Multi-ethnic 3 3
Not Specified or Other 12 12
English Language
classification >.999 <0.001
Yes 11 11
No 59 59
Special education
classification >.999 0.034
Yes 17 16
No 53 54
Free/reduced price lunch >.999 .029
Yes 27 26
No 43 44
5Impact Evaluation of Imagine MyPath in a Northeastern School District 2021–2022
Results
IMAGINE MYPATH USAGE
Treatment students in the math analysis spent an average of 5.37 hours in Imagine MyPath for
math and passed an average of 10.88 lessons. Similarly, within the reading analytical sample,
treatment students spent an average of 4.76 hours in Imagine MyPath for reading and passed
and average of 7.64 lessons. See Figure 1 for a distribution of hours and lessons passed by grade.
Figure 1. Boxplot of Time Spent and Lessons Passed in Imagine MyPath by Subject and Grade
40
35
30
25
10
53
20
15
5
2 4
40
35
30
25
10
53
20
15
5
2 4
Lessons Passed in Imagine MyPath
Grade
Note: Within each box, thick horizontal black lines denote median values; boxes extend from the 25th to the 75th
percentile of each group’s distribution of values; vertical extending lines denote adjacent values (i.e., the most extreme
values within 1.5 interquartile range of the 25th and 75th percentile of each group); dots denote observations outside
the range of adjacent values.
Hours Spent in Imagine MyPath
Math Reading
Grade
30
25
10
53
20
15
5
2 4 532 4
30
25
10
20
15
5
Grade Grade
6Impact Evaluation of Imagine MyPath in a Northeastern School District 2021–2022
IMAGINE MYPATH USAGE
Treatment students in the math analysis spent an average of 5.37 hours in Imagine MyPath for
math and passed an average of 10.88 lessons. Similarly, within the reading analytical sample,
treatment students spent an average of 4.76 hours in Imagine MyPath for reading and passed
and average of 7.64 lessons. See Figure 1 for a distribution of hours and lessons passed by grade.
Figure 1. Boxplot of Time Spent and Lessons Passed in Imagine MyPath by Subject and Grade
40
35
30
25
10
53
20
15
5
2 4
40
35
30
25
10
53
20
15
5
2 4
Lessons Passed in Imagine MyPath
Grade
Note: Within each box, thick horizontal black lines denote median values; boxes extend from the 25th to the 75th
percentile of each group’s distribution of values; vertical extending lines denote adjacent values (i.e., the most extreme
values within 1.5 interquartile range of the 25th and 75th percentile of each group); dots denote observations outside
the range of adjacent values.
Hours Spent in Imagine MyPath
Math Reading
Grade
30
25
10
53
20
15
5
2 4 532 4
30
25
10
20
15
5
Grade Grade
6Impact Evaluation of Imagine MyPath in a Northeastern School District 2021–2022
PROGRAM IMPACT ON STUDENT ACHIEVEMENT
Overall, use of Imagine MyPath was found to generate a positive and statistically significant
impact on students’ mathematics performance. Specifically, students who used Imagine MyPath
for supplemental math instruction (n = 75) scored an average of 3.43 points higher on the Spring
2022 NWEA MAP Growth Math assessment than otherwise similar non-user students (n = 75),
B = 3.43, t(137) = 2.710, p = .008. Program usage and the other covariates in the model accounted
for 79% of the variance found in Spring 2022 scores, R2 = .792, F(12,137) = 43.41, p < .001. The
Hedges’ g effect size of Imagine MyPath Math program usage was .21.2 Table 2 summarizes
the results of the multiple linear regression. The covariate-adjusted mean Spring 2022 score
was 184.72 for Imagine MyPath Math users and 181.29 for non-users.
Overall, use of Imagine MyPath was found to generate a positive impact on students’ reading
performance, which was marginally significant. Specifically, students who used Imagine MyPath
for supplemental reading instruction (n = 70) scored an average of 2.90 points higher on the
Spring 2022 NWEA MAP Growth Reading assessment compared to other similar non-user
students (n = 70), B = 2.90, t(127) = 1.744, p = .084. Program usage and other covariates in the
model accounted for 69% of the variance found in Spring 2022 scores, R2 = .691, F(12,127) = 23.7,
p < .001. The Hedges’ g effect size of Imagine MyPath Reading program usage was .17. Table 2
summarizes the results of the multiple linear regression. The covariate-adjusted mean Spring
2022 score was 185.83 for Imagine MyPath Reading users and 182.94 for non-users.
2 The effect size is calculated using Hedges’ g computation following What Works Clearinghouse’s Procedures and
Standards Handbook, Version 5.0. The unadjusted standard deviations of the Spring 2022 scores can be found in
Appendix B.
7Impact Evaluation of Imagine MyPath in a Northeastern School District 2021–2022
Overall, use of Imagine MyPath was found to generate a positive and statistically significant
impact on students’ mathematics performance. Specifically, students who used Imagine MyPath
for supplemental math instruction (n = 75) scored an average of 3.43 points higher on the Spring
2022 NWEA MAP Growth Math assessment than otherwise similar non-user students (n = 75),
B = 3.43, t(137) = 2.710, p = .008. Program usage and the other covariates in the model accounted
for 79% of the variance found in Spring 2022 scores, R2 = .792, F(12,137) = 43.41, p < .001. The
Hedges’ g effect size of Imagine MyPath Math program usage was .21.2 Table 2 summarizes
the results of the multiple linear regression. The covariate-adjusted mean Spring 2022 score
was 184.72 for Imagine MyPath Math users and 181.29 for non-users.
Overall, use of Imagine MyPath was found to generate a positive impact on students’ reading
performance, which was marginally significant. Specifically, students who used Imagine MyPath
for supplemental reading instruction (n = 70) scored an average of 2.90 points higher on the
Spring 2022 NWEA MAP Growth Reading assessment compared to other similar non-user
students (n = 70), B = 2.90, t(127) = 1.744, p = .084. Program usage and other covariates in the
model accounted for 69% of the variance found in Spring 2022 scores, R2 = .691, F(12,127) = 23.7,
p < .001. The Hedges’ g effect size of Imagine MyPath Reading program usage was .17. Table 2
summarizes the results of the multiple linear regression. The covariate-adjusted mean Spring
2022 score was 185.83 for Imagine MyPath Reading users and 182.94 for non-users.
2 The effect size is calculated using Hedges’ g computation following What Works Clearinghouse’s Procedures and
Standards Handbook, Version 5.0. The unadjusted standard deviations of the Spring 2022 scores can be found in
Appendix B.
7Impact Evaluation of Imagine MyPath in a Northeastern School District 2021–2022
Table 2. Overall Impact of Imagine MyPath Math on Spring 2022 MAP Growth Math RIT Scores.
Math Reading
Coefficients Estimate
Standard
Error p-value Estimate
Standard
Error p-value
Intercept 22.69 8.70 .010 52.99 9.59 <.001
Imagine MyPath User Indicator 3.43 1.26 .008 2.90 1.66 .084
Grade-Level Indicator
Grade 3 -2.43 1.83 .187 -0.16 2.22 .943
Grade 4 -5.05 2.29 .029 -1.89 3.27 .564
Grade 5 -5.30 2.71 .053 2.86 3.56 .424
Fall 2021 MAP Growth RIT Score 0.93 0.05 <.001 0.75 0.06 <.001
Race/Ethnicity Indicator
Hispanic or Latino 1.52 3.08 .621 4.26 2.87 .140
Multi-ethnic 4.96 6.37 .437 0.44 4.90 .929
Not Specified or Other 5.47 3.69 .141 1.33 3.55 .710
White 7.12 3.83 .065 1.53 4.32 .725
English language indicator 2.18 1.82 .235 -3.26 2.81 .250
Special education indicator 1.41 1.70 .409 0.29 2.23 .898
Free/reduced price lunch indicator -2.30 1.44 .113 -4.68 1.91 .016
DIFFERENTIAL IMPACT BY GRADE BAND
A series of analyses were further conducted to examine whether the effects of Imagine MyPath
varied across grade band.3 Descriptive tables of unadjusted average NWEA MAP Growth Math
and Reading RIT scores by grade band can be found in Appendix B and tables demonstrating
baseline equivalence by grade can be found in Appendix C.
Grade 3–5 Imagine MyPath users demonstrated statistically significantly higher Spring 2022
NWEA MAP Growth RIT scores than comparable non-users in both math and reading. Specifically,
Grade 3–5 students who used Imagine MyPath scored an average of 3.34 points higher on the
Spring 2022 NWEA MAP Growth Math assessment than otherwise similar non-user students
(covariate-adjusted means = 185.00 and 181.66, respectively), B = 3.34, t(104) = 2.177, p = .032.
Program usage and the other covariates in the model accounted for 80% of the variance
found in Spring 2022 scores, R2 = .801, F(11,104) = 38.06, p < .001. The Hedges’ g effect size of
Grade 3–5 Imagine MyPath program usage was .19. Additionally, Grade 3–5 students who
used Imagine MyPath scored an average of 5.42 points higher on the Spring 2022 NWEA MAP
3 Grade bands follow standards outlined by Evidence for ESSA: Grade 2 and Grades 3-5 are analyzed separately.
8Impact Evaluation of Imagine MyPath in a Northeastern School District 2021–2022
Math Reading
Coefficients Estimate
Standard
Error p-value Estimate
Standard
Error p-value
Intercept 22.69 8.70 .010 52.99 9.59 <.001
Imagine MyPath User Indicator 3.43 1.26 .008 2.90 1.66 .084
Grade-Level Indicator
Grade 3 -2.43 1.83 .187 -0.16 2.22 .943
Grade 4 -5.05 2.29 .029 -1.89 3.27 .564
Grade 5 -5.30 2.71 .053 2.86 3.56 .424
Fall 2021 MAP Growth RIT Score 0.93 0.05 <.001 0.75 0.06 <.001
Race/Ethnicity Indicator
Hispanic or Latino 1.52 3.08 .621 4.26 2.87 .140
Multi-ethnic 4.96 6.37 .437 0.44 4.90 .929
Not Specified or Other 5.47 3.69 .141 1.33 3.55 .710
White 7.12 3.83 .065 1.53 4.32 .725
English language indicator 2.18 1.82 .235 -3.26 2.81 .250
Special education indicator 1.41 1.70 .409 0.29 2.23 .898
Free/reduced price lunch indicator -2.30 1.44 .113 -4.68 1.91 .016
DIFFERENTIAL IMPACT BY GRADE BAND
A series of analyses were further conducted to examine whether the effects of Imagine MyPath
varied across grade band.3 Descriptive tables of unadjusted average NWEA MAP Growth Math
and Reading RIT scores by grade band can be found in Appendix B and tables demonstrating
baseline equivalence by grade can be found in Appendix C.
Grade 3–5 Imagine MyPath users demonstrated statistically significantly higher Spring 2022
NWEA MAP Growth RIT scores than comparable non-users in both math and reading. Specifically,
Grade 3–5 students who used Imagine MyPath scored an average of 3.34 points higher on the
Spring 2022 NWEA MAP Growth Math assessment than otherwise similar non-user students
(covariate-adjusted means = 185.00 and 181.66, respectively), B = 3.34, t(104) = 2.177, p = .032.
Program usage and the other covariates in the model accounted for 80% of the variance
found in Spring 2022 scores, R2 = .801, F(11,104) = 38.06, p < .001. The Hedges’ g effect size of
Grade 3–5 Imagine MyPath program usage was .19. Additionally, Grade 3–5 students who
used Imagine MyPath scored an average of 5.42 points higher on the Spring 2022 NWEA MAP
3 Grade bands follow standards outlined by Evidence for ESSA: Grade 2 and Grades 3-5 are analyzed separately.
8Impact Evaluation of Imagine MyPath in a Northeastern School District 2021–2022
Growth Reading assessment than otherwise similar non-user students (covariate-adjusted
means = 193.11 and 187.70, respectively), B = 5.42, t(70) = 2.754, p = .007. Program usage and the
other covariates in the model accounted for 78% of the variance found in Spring 2022 scores,
R2 = .783, F(11,70) = 22.93, p < .001. The Hedges’ g effect size of Imagine MyPath program usage
for students in Grades 3-5 was .31.
Results for Grade 2 were not statistically significant. Table 3 summarizes the results of the multiple
linear regressions for each grade band. Complete regression results can be found in Appendix D.
Table 3. Impact of Imagine MyPath on Spring 2022 MAP Growth Math RIT Scores by Grade Band
Math Reading
Grade Band
Estimate on
Imagine MyPath
Indicator Variable
Standard
Error p-value
Estimate on
Imagine MyPath
Indicator Variable
Standard
Error p-value
Grade 2 3.59 2.09 .097 -2.32 2.73 .399
Grades 3-5 3.34 1.53 .032 5.42 1.97 .007
9Impact Evaluation of Imagine MyPath in a Northeastern School District 2021–2022
means = 193.11 and 187.70, respectively), B = 5.42, t(70) = 2.754, p = .007. Program usage and the
other covariates in the model accounted for 78% of the variance found in Spring 2022 scores,
R2 = .783, F(11,70) = 22.93, p < .001. The Hedges’ g effect size of Imagine MyPath program usage
for students in Grades 3-5 was .31.
Results for Grade 2 were not statistically significant. Table 3 summarizes the results of the multiple
linear regressions for each grade band. Complete regression results can be found in Appendix D.
Table 3. Impact of Imagine MyPath on Spring 2022 MAP Growth Math RIT Scores by Grade Band
Math Reading
Grade Band
Estimate on
Imagine MyPath
Indicator Variable
Standard
Error p-value
Estimate on
Imagine MyPath
Indicator Variable
Standard
Error p-value
Grade 2 3.59 2.09 .097 -2.32 2.73 .399
Grades 3-5 3.34 1.53 .032 5.42 1.97 .007
9Impact Evaluation of Imagine MyPath in a Northeastern School District 2021–2022
ConclusionEducational technology provides a means for improving student achievement through
supplemental instruction that is individualized to meet the needs of each student. Moreover,
digital learning solutions have a demonstrated impact on student outcomes and are a critical
component of the future of education (Haleem et al., 2022; Li & Ma., 2010; Stacy et al., 2017).
Imagine MyPath offers such a solution. The program prioritizes critical foundational skills in
reading and math, with the goal of accelerating student growth to grade-level success.
This study set out to examine the impact of Imagine MyPath on the math and reading
achievement of students in Grades 2–5. Overall, findings revealed a statistically significant
impact for math and a marginally significant impact for reading. Students who used Imagine
MyPath scored 3.43 points higher on the Spring 2022 administration of the NWEA MAP Growth
Math assessment than did similar comparison students (p < .01). Imagine MyPath program
users scored 2.90 points higher on the NWEA MAP Growth Reading assessment compared to
similar non-users (p = .08).
A limitation of this study includes small sample sizes for some analyses. In particular, the grade
band separation resulted in a small sample size for the Grade 2 analyses (nmath = 34; nreading = 58).
This, in addition to lower usage in Grade 2 (Figure 1), likely contributed toward marginally
significant math and non-significant reading results. The sample size for the Grade 3–5 analysis
was slightly larger (nmath = 116; nreading = 82), and usage was greater; results revealed a significant
impact of Imagine MyPath in both math and reading. In the future, a larger sample size and
increased usage would allow for an even more robust evaluation of the program.
In summary, this study provides evidence of effectiveness of Imagine MyPath on math and
reading outcomes. Specifically, it demonstrates Imagine MyPath’s impact on the math and
reading achievement of students in Grades 3–5, and the math achievement of students in
Grade 2, by comparing the outcomes of students who participated in the program to those
who did not.
10Impact Evaluation of Imagine MyPath in a Northeastern School District 2021–2022
supplemental instruction that is individualized to meet the needs of each student. Moreover,
digital learning solutions have a demonstrated impact on student outcomes and are a critical
component of the future of education (Haleem et al., 2022; Li & Ma., 2010; Stacy et al., 2017).
Imagine MyPath offers such a solution. The program prioritizes critical foundational skills in
reading and math, with the goal of accelerating student growth to grade-level success.
This study set out to examine the impact of Imagine MyPath on the math and reading
achievement of students in Grades 2–5. Overall, findings revealed a statistically significant
impact for math and a marginally significant impact for reading. Students who used Imagine
MyPath scored 3.43 points higher on the Spring 2022 administration of the NWEA MAP Growth
Math assessment than did similar comparison students (p < .01). Imagine MyPath program
users scored 2.90 points higher on the NWEA MAP Growth Reading assessment compared to
similar non-users (p = .08).
A limitation of this study includes small sample sizes for some analyses. In particular, the grade
band separation resulted in a small sample size for the Grade 2 analyses (nmath = 34; nreading = 58).
This, in addition to lower usage in Grade 2 (Figure 1), likely contributed toward marginally
significant math and non-significant reading results. The sample size for the Grade 3–5 analysis
was slightly larger (nmath = 116; nreading = 82), and usage was greater; results revealed a significant
impact of Imagine MyPath in both math and reading. In the future, a larger sample size and
increased usage would allow for an even more robust evaluation of the program.
In summary, this study provides evidence of effectiveness of Imagine MyPath on math and
reading outcomes. Specifically, it demonstrates Imagine MyPath’s impact on the math and
reading achievement of students in Grades 3–5, and the math achievement of students in
Grade 2, by comparing the outcomes of students who participated in the program to those
who did not.
10Impact Evaluation of Imagine MyPath in a Northeastern School District 2021–2022
References
Haleem, A., Javaid, M., Qadri, M. A., & Suman, R. (2022). Understanding the role of digital technologies in education:
A review. Sustainable Operations and Computers, 3, 275-285.
Li, Q., & Ma, X. (2010). A meta-analysis of the effects of computer technology on school students’ mathematics
learning. Educational Psychology Review, 22(3), 215-243.
National Research Council. (2012). Education for life and work: Developing transferable knowledge and skills in
the 21st century. Committee on Defining Deeper Learning and 21st Century Skills, James W. Pellegrino and
Margaret L. Hilton (Eds.), Board on Testing and Assessment and Board on Science Education, Division of
Behavioral and Social Sciences and Education. The National Academies Press.
Smale-Jacobse, A. E., Meijer, A., Helms-Lorenz, M., & Maulana, R. (2019). Differentiated instruction in secondary
education: A systematic review of research evidence. Frontiers in psychology, 10, 2366.
Stacy, S. T., Cartwright, M., Arwood, Z., Canfield, J. P., & Kloos, H. (2017). Addressing the math-practice gap in
elementary school: Are tablets a feasible tool for informal math practice?. Frontiers in psychology, 8, 214487.
What Works Clearinghouse. (2022). What Works Clearinghouse procedures and standards handbook, version 5.0.
U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and
Regional Assistance (NCEE).
11Impact Evaluation of Imagine MyPath in a Northeastern School District 2021–2022
Haleem, A., Javaid, M., Qadri, M. A., & Suman, R. (2022). Understanding the role of digital technologies in education:
A review. Sustainable Operations and Computers, 3, 275-285.
Li, Q., & Ma, X. (2010). A meta-analysis of the effects of computer technology on school students’ mathematics
learning. Educational Psychology Review, 22(3), 215-243.
National Research Council. (2012). Education for life and work: Developing transferable knowledge and skills in
the 21st century. Committee on Defining Deeper Learning and 21st Century Skills, James W. Pellegrino and
Margaret L. Hilton (Eds.), Board on Testing and Assessment and Board on Science Education, Division of
Behavioral and Social Sciences and Education. The National Academies Press.
Smale-Jacobse, A. E., Meijer, A., Helms-Lorenz, M., & Maulana, R. (2019). Differentiated instruction in secondary
education: A systematic review of research evidence. Frontiers in psychology, 10, 2366.
Stacy, S. T., Cartwright, M., Arwood, Z., Canfield, J. P., & Kloos, H. (2017). Addressing the math-practice gap in
elementary school: Are tablets a feasible tool for informal math practice?. Frontiers in psychology, 8, 214487.
What Works Clearinghouse. (2022). What Works Clearinghouse procedures and standards handbook, version 5.0.
U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and
Regional Assistance (NCEE).
11Impact Evaluation of Imagine MyPath in a Northeastern School District 2021–2022
Appendix A
Table A1. Imagine MyPath Users and Non-Users by School and Grade Level for Math
Full Sample Analytical Sample
School Name
Grade
Level
Imagine MyPath
Math User Non-User
Imagine MyPath
Math User Non-User
Elementary School # 1 4 136 7 1 7
5 148 9 3 9
Elementary School # 2 4 146 6 7 6
5 153 0 1 0
Elementary School # 3 2 13 7 2 2
3 16 3 0 3
4 28 1 1 1
5 34 1 1 1
Elementary School # 4 2 111 16 6 9
3 130 28 6 28
Elementary School # 5 2 142 9 2 5
3 167 0 16 0
Elementary School # 6 2 155 2 7 1
3 157 3 12 3
Elementary School # 7 4 178 0 5 0
5 163 0 5 0
12Impact Evaluation of Imagine MyPath in a Northeastern School District 2021–2022
Table A1. Imagine MyPath Users and Non-Users by School and Grade Level for Math
Full Sample Analytical Sample
School Name
Grade
Level
Imagine MyPath
Math User Non-User
Imagine MyPath
Math User Non-User
Elementary School # 1 4 136 7 1 7
5 148 9 3 9
Elementary School # 2 4 146 6 7 6
5 153 0 1 0
Elementary School # 3 2 13 7 2 2
3 16 3 0 3
4 28 1 1 1
5 34 1 1 1
Elementary School # 4 2 111 16 6 9
3 130 28 6 28
Elementary School # 5 2 142 9 2 5
3 167 0 16 0
Elementary School # 6 2 155 2 7 1
3 157 3 12 3
Elementary School # 7 4 178 0 5 0
5 163 0 5 0
12Impact Evaluation of Imagine MyPath in a Northeastern School District 2021–2022
Table A2. Imagine MyPath Users and Non-Users by School and Grade Level for Reading
Full Sample Analytical Sample
School Name
Grade
Level
Imagine MyPath
Reading User Non-User
Imagine MyPath
Reading User Non-User
Elementary School # 1 4 138 10 3 5
5 143 12 1 4
Elementary School # 2 4 147 13 2 4
5 152 3 2 3
Elementary School # 3 2 18 2 0 2
3 18 1 0 1
4 30 0 0 0
5 36 0 0 0
Elementary School # 4 2 113 16 6 16
3 136 23 3 22
Elementary School # 5 2 152 1 11 1
3 168 2 14 2
Elementary School # 6 2 158 10 12 10
3 161 0 8 0
Elementary School # 7 4 180 0 4 0
5 163 0 4 0
13Impact Evaluation of Imagine MyPath in a Northeastern School District 2021–2022
Full Sample Analytical Sample
School Name
Grade
Level
Imagine MyPath
Reading User Non-User
Imagine MyPath
Reading User Non-User
Elementary School # 1 4 138 10 3 5
5 143 12 1 4
Elementary School # 2 4 147 13 2 4
5 152 3 2 3
Elementary School # 3 2 18 2 0 2
3 18 1 0 1
4 30 0 0 0
5 36 0 0 0
Elementary School # 4 2 113 16 6 16
3 136 23 3 22
Elementary School # 5 2 152 1 11 1
3 168 2 14 2
Elementary School # 6 2 158 10 12 10
3 161 0 8 0
Elementary School # 7 4 180 0 4 0
5 163 0 4 0
13Impact Evaluation of Imagine MyPath in a Northeastern School District 2021–2022
Appendix B
Table B1. Imagine MyPath Sample Unadjusted Mean MAP Growth Math RIT Scores by Grade Band
Fall 2021 (SD) Spring 2022 (SD) Mean Change
Grade 2
Comparison (n = 17) 165.94 (9.02) 177.65 (11.28) 11.71
Imagine MyPath Math (n = 17) 164.47 (10.41) 180.12 (9.73) 15.65
Grades 3-5
Comparison (n = 58) 170.76 (16.67) 180.05 (18.19) 9.29
Imagine MyPath Math (n = 58) 171.10 (14.98) 183.84 (16.92) 12.53
All Grades
Comparison (n = 75) 169.67 (15.36) 179.51 (16.83) 9.84
Imagine MyPath Math (n = 75) 169.60 (14.28) 182.84 (15.60) 13.24
Table B2. Imagine MyPath Reading Sample Unadjusted Mean MAP Growth Math RIT Scores by
Grade Band
Fall 2021 (SD) Spring 2022 (SD) Mean Change
Grade 2
Comparison (n = 29) 169.31 (16.06) 183.21 (12.58) 13.90
Imagine MyPath Reading (n = 29) 166.34 (15.96) 179.86 (15.69) 13.52
Grades 3-5
Comparison (n = 41) 180.63 (16.06) 186.73 (18.71) 6.10
Imagine MyPath Reading (n = 41) 184.68 (16.63) 195.61 (15.31) 10.93
All Grades
Comparison (n = 70) 175.94 (16.91) 185.27 (16.44) 9.33
Imagine MyPath Reading (n = 70) 177.09 (18.62) 189.09 (17.23) 12.00
14Impact Evaluation of Imagine MyPath in a Northeastern School District 2021–2022
Table B1. Imagine MyPath Sample Unadjusted Mean MAP Growth Math RIT Scores by Grade Band
Fall 2021 (SD) Spring 2022 (SD) Mean Change
Grade 2
Comparison (n = 17) 165.94 (9.02) 177.65 (11.28) 11.71
Imagine MyPath Math (n = 17) 164.47 (10.41) 180.12 (9.73) 15.65
Grades 3-5
Comparison (n = 58) 170.76 (16.67) 180.05 (18.19) 9.29
Imagine MyPath Math (n = 58) 171.10 (14.98) 183.84 (16.92) 12.53
All Grades
Comparison (n = 75) 169.67 (15.36) 179.51 (16.83) 9.84
Imagine MyPath Math (n = 75) 169.60 (14.28) 182.84 (15.60) 13.24
Table B2. Imagine MyPath Reading Sample Unadjusted Mean MAP Growth Math RIT Scores by
Grade Band
Fall 2021 (SD) Spring 2022 (SD) Mean Change
Grade 2
Comparison (n = 29) 169.31 (16.06) 183.21 (12.58) 13.90
Imagine MyPath Reading (n = 29) 166.34 (15.96) 179.86 (15.69) 13.52
Grades 3-5
Comparison (n = 41) 180.63 (16.06) 186.73 (18.71) 6.10
Imagine MyPath Reading (n = 41) 184.68 (16.63) 195.61 (15.31) 10.93
All Grades
Comparison (n = 70) 175.94 (16.91) 185.27 (16.44) 9.33
Imagine MyPath Reading (n = 70) 177.09 (18.62) 189.09 (17.23) 12.00
14Impact Evaluation of Imagine MyPath in a Northeastern School District 2021–2022
Table C1. Grade 2 Band Baseline Equivalence
Math
Comparison
Students (n = 17)
Imagine MyPath
Students (n = 17) p-value
Standardized Mean
Difference (SMD)
Average (SD) Fall 2021 NWEA
MAP Growth RIT Score 165.94 (9.02) 164.47 (10.41) .663 0.151
Race/Ethnicity >.999 <0.001
White 1 1
Black or African American 0 0
Hispanic or Latino 15 15
Multi-ethnic 0 0
Not Specified or Other 1 1
English Language
classification >.999 <0.001
Yes 3 3
No 14 14
Special education
classification >.999 .209
Yes 1 2
No 16 15
Free/reduced price lunch >.999 <0.001
Yes 10 10
No 7 7
Appendix C
15Impact Evaluation of Imagine MyPath in a Northeastern School District 2021–2022
Math
Comparison
Students (n = 17)
Imagine MyPath
Students (n = 17) p-value
Standardized Mean
Difference (SMD)
Average (SD) Fall 2021 NWEA
MAP Growth RIT Score 165.94 (9.02) 164.47 (10.41) .663 0.151
Race/Ethnicity >.999 <0.001
White 1 1
Black or African American 0 0
Hispanic or Latino 15 15
Multi-ethnic 0 0
Not Specified or Other 1 1
English Language
classification >.999 <0.001
Yes 3 3
No 14 14
Special education
classification >.999 .209
Yes 1 2
No 16 15
Free/reduced price lunch >.999 <0.001
Yes 10 10
No 7 7
Appendix C
15Impact Evaluation of Imagine MyPath in a Northeastern School District 2021–2022
Table C1. Grade 2 Band Baseline Equivalence cont.
Reading
Comparison
Students (n = 29)
Imagine MyPath
Students (n = 29) p-value
Standardized Mean
Difference (SMD)
Average (SD) Fall 2021 NWEA
MAP Growth RIT Score 169.31 (16.06) 166.34 (15.96) .484 0.185
Race/Ethnicity >.999 <0.001
White 1 1
Black or African American 4 4
Hispanic or Latino 19 19
Multi-ethnic 2 2
Not Specified or Other 3 3
English Language
classification >.999 <0.001
Yes 2 2
No 27 27
Special education
classification >.999 <0.001
Yes 4 4
No 25 25
Free/reduced price lunch .769 0.155
Yes 9 7
No 20 22
16Impact Evaluation of Imagine MyPath in a Northeastern School District 2021–2022
Reading
Comparison
Students (n = 29)
Imagine MyPath
Students (n = 29) p-value
Standardized Mean
Difference (SMD)
Average (SD) Fall 2021 NWEA
MAP Growth RIT Score 169.31 (16.06) 166.34 (15.96) .484 0.185
Race/Ethnicity >.999 <0.001
White 1 1
Black or African American 4 4
Hispanic or Latino 19 19
Multi-ethnic 2 2
Not Specified or Other 3 3
English Language
classification >.999 <0.001
Yes 2 2
No 27 27
Special education
classification >.999 <0.001
Yes 4 4
No 25 25
Free/reduced price lunch .769 0.155
Yes 9 7
No 20 22
16Impact Evaluation of Imagine MyPath in a Northeastern School District 2021–2022