Impact Evaluation of
Imagine MyPath in
Moline-Coal Valley School
District
Michael A. Cook, PhD
Nathan Storey, PhD
Jane Eisinger, MS
Maria Jose Barros, PhD
Steven M. Ross, PhD
November 2023
ii
© Johns Hopkins University, 2023
Contents
EXECUTIVE SUMMARY: ............................................................................................ iii
Impact Evaluation of Imagine MyPath for Kindergarten Students ................................ iii
Program Description ............................................................................................. iii
Research Design ................................................................................................... iii
Study Sample ....................................................................................................... iv
Program Impact on Reading and Mathematics Achievement .................................... iv
Teacher Perceptions .............................................................................................. v
Impact Evaluation of Imagine MyPath for Kindergarten Students ................................. 1
Method ..................................................................................................................... 2
Research Design .................................................................................................... 2
Participants ........................................................................................................... 2
Student sample. ................................................................................................. 2
Teacher sample. ................................................................................................. 3
Measures .............................................................................................................. 3
NWEA MAP Growth. ............................................................................................ 3
Program usage. .................................................................................................. 3
Teacher questionnaire. ....................................................................................... 4
Analytical Approach ............................................................................................... 4
Achievement Results ................................................................................................. 4
Descriptive Analyses .............................................................................................. 4
Usage data. ....................................................................................................... 5
Impact analyses. ................................................................................................ 6
Associations between usage and achievement. .................................................... 7
Teacher Questionnaire Results ................................................................................... 7
Appendix A: Imagine MyPath Teacher Survey ........................................................... 20
Appendix B: Baseline Equivalence Tables .................................................................. 26
© Johns Hopkins University, 2023
Contents
EXECUTIVE SUMMARY: ............................................................................................ iii
Impact Evaluation of Imagine MyPath for Kindergarten Students ................................ iii
Program Description ............................................................................................. iii
Research Design ................................................................................................... iii
Study Sample ....................................................................................................... iv
Program Impact on Reading and Mathematics Achievement .................................... iv
Teacher Perceptions .............................................................................................. v
Impact Evaluation of Imagine MyPath for Kindergarten Students ................................. 1
Method ..................................................................................................................... 2
Research Design .................................................................................................... 2
Participants ........................................................................................................... 2
Student sample. ................................................................................................. 2
Teacher sample. ................................................................................................. 3
Measures .............................................................................................................. 3
NWEA MAP Growth. ............................................................................................ 3
Program usage. .................................................................................................. 3
Teacher questionnaire. ....................................................................................... 4
Analytical Approach ............................................................................................... 4
Achievement Results ................................................................................................. 4
Descriptive Analyses .............................................................................................. 4
Usage data. ....................................................................................................... 5
Impact analyses. ................................................................................................ 6
Associations between usage and achievement. .................................................... 7
Teacher Questionnaire Results ................................................................................... 7
Appendix A: Imagine MyPath Teacher Survey ........................................................... 20
Appendix B: Baseline Equivalence Tables .................................................................. 26
EXECUTIVE SUMMARY iii
© Johns Hopkins University, 2023
EXECUTIVE SUMMARY:
Impact Evaluation of Imagine MyPath for Kindergarten Students
Program Description
Imagine Learning contracted with the Center for Research and Reform in
Education (CRRE) at Johns Hopkins University to conduct a quasi-experimental design
(QED) in Grades K–5 in two school districts. The current district is Moline-Coal Valley
School District No. 40, which serves approximately 7,200 students in 15 schools, 12 of
which are elementary schools. This report focuses on efficacy impacts and teacher
perceptions of Imagine MyPath in the Moline-Coal Valley School District.
Imagine Learning’s Imagine MyPath is a K–12 reading and mathematics
intervention program designed to close achievement gaps and maximize academic
growth for students. Per the developer, “Imagine MyPath is a supplemental curriculum
that utilizes Smart Sequencer™ technology to prioritize essential skills and create
individual learning paths (ILPs) in reading and mathematics. ILPs are grounded in
research, and continuously adapt to ensure success among academically diverse
learners. All Imagine MyPath lessons are age appropriate and intentionally designed to
align with each student’s grade and skill level. A student’s chronological grade
determines their experience in the program and the presentation of information, but
their skill level determines the types of questions presented. The curriculum uniquely
leverages a cycle of assessment, assignment, adaptivity, analysis, and action to create
an ILP that delivers an adaptive sequence of lessons so students efficiently catch up,
keep up, and get ahead.”
The evaluation’s design addressed the following research questions:
1) How does participation in Imagine MyPath impact student achievement in
mathematics and reading?
a) Does level of program usage relate to student achievement effects?
b) To what degree do effects vary across:
i) Schools?
ii) Grade levels?
2) What are teachers’ perceptions of the program with regard to:
a) Benefits for students?
b) Student engagement?
c) Implementation requirements and experiences?
d) Strengths and weaknesses?
e) Recommendation for implementation improvement?
Research Design
© Johns Hopkins University, 2023
EXECUTIVE SUMMARY:
Impact Evaluation of Imagine MyPath for Kindergarten Students
Program Description
Imagine Learning contracted with the Center for Research and Reform in
Education (CRRE) at Johns Hopkins University to conduct a quasi-experimental design
(QED) in Grades K–5 in two school districts. The current district is Moline-Coal Valley
School District No. 40, which serves approximately 7,200 students in 15 schools, 12 of
which are elementary schools. This report focuses on efficacy impacts and teacher
perceptions of Imagine MyPath in the Moline-Coal Valley School District.
Imagine Learning’s Imagine MyPath is a K–12 reading and mathematics
intervention program designed to close achievement gaps and maximize academic
growth for students. Per the developer, “Imagine MyPath is a supplemental curriculum
that utilizes Smart Sequencer™ technology to prioritize essential skills and create
individual learning paths (ILPs) in reading and mathematics. ILPs are grounded in
research, and continuously adapt to ensure success among academically diverse
learners. All Imagine MyPath lessons are age appropriate and intentionally designed to
align with each student’s grade and skill level. A student’s chronological grade
determines their experience in the program and the presentation of information, but
their skill level determines the types of questions presented. The curriculum uniquely
leverages a cycle of assessment, assignment, adaptivity, analysis, and action to create
an ILP that delivers an adaptive sequence of lessons so students efficiently catch up,
keep up, and get ahead.”
The evaluation’s design addressed the following research questions:
1) How does participation in Imagine MyPath impact student achievement in
mathematics and reading?
a) Does level of program usage relate to student achievement effects?
b) To what degree do effects vary across:
i) Schools?
ii) Grade levels?
2) What are teachers’ perceptions of the program with regard to:
a) Benefits for students?
b) Student engagement?
c) Implementation requirements and experiences?
d) Strengths and weaknesses?
e) Recommendation for implementation improvement?
Research Design
EXECUTIVE SUMMARY iv
© Johns Hopkins University, 2023
A quasi-experimental design (QED) was used to compare mathematics and
reading achievement of Imagine MyPath students to matched comparison students
obtained from a Similar Schools Report (SSR) provided to the district by NWEA. An SSR
was used because all elementary schools in the district used Imagine MyPath. A Similar
Schools Report contains data from students who, relative to the intervention sample,
come from schools in a similar area (urban, suburban, rural), with similar percentages
of free and reduced meals students (FARMS), creating a “virtual comparison group” of
students, and allowing for direct comparisons of NWEA MAP Growth score growth
between Moline-Coal Valley students who used Imagine MyPath and otherwise similar
students who did not use the program.
The evaluation also examined teachers’ perceptions of Imagine MyPath through
an online teacher questionnaire. Teachers were asked about topics including
experience with digital learning, program implementation, professional development,
and student impact. Likert-scale items were used to collect data relating to teachers’
perceptions of Imagine MyPath. Additionally, four open-ended queries provided
teachers with the opportunity to indicate which program resources were most
effective, what students liked most about the program, and to offer suggestions for
improving the Imagine MyPath platforms for educators and students alike.
Study Sample
The present study sample consisted of 466 Grade K students from across 10
elementary schools in Moline-Coal Valley. The student population among Moline-Coal
Valley students was mostly White (slightly under 50%), followed by Black and Hispanic
students (approximately 20% each). The teacher survey sample consisted of 26
teachers.
Program Impact on Reading and Mathematics Achievement
Observed impacts of Imagine MyPath on student achievement were mixed. A
statistically significant positive impact of Imagine MyPath on mathematics achievement
was observed in Moline-Coal Valley, with treatment students outgaining virtual
comparison students by more than 2 points. While treatment students also outgained
virtual comparison students in reading achievement, the difference was minimal and not
statistically significant.
Rates of Imagine MyPath usage varied somewhat across subjects. Moline-Coal
Valley students used the program approximately 11 hours for reading and eight hours
for mathematics. On average, more reading lessons were completed than mathematics
lessons, although lesson pass counts were very similar across subjects with students
averaging approximately 27 passed lessons in each subject. Program usage metrics
were generally positively related to mathematics and reading achievement gains, with
© Johns Hopkins University, 2023
A quasi-experimental design (QED) was used to compare mathematics and
reading achievement of Imagine MyPath students to matched comparison students
obtained from a Similar Schools Report (SSR) provided to the district by NWEA. An SSR
was used because all elementary schools in the district used Imagine MyPath. A Similar
Schools Report contains data from students who, relative to the intervention sample,
come from schools in a similar area (urban, suburban, rural), with similar percentages
of free and reduced meals students (FARMS), creating a “virtual comparison group” of
students, and allowing for direct comparisons of NWEA MAP Growth score growth
between Moline-Coal Valley students who used Imagine MyPath and otherwise similar
students who did not use the program.
The evaluation also examined teachers’ perceptions of Imagine MyPath through
an online teacher questionnaire. Teachers were asked about topics including
experience with digital learning, program implementation, professional development,
and student impact. Likert-scale items were used to collect data relating to teachers’
perceptions of Imagine MyPath. Additionally, four open-ended queries provided
teachers with the opportunity to indicate which program resources were most
effective, what students liked most about the program, and to offer suggestions for
improving the Imagine MyPath platforms for educators and students alike.
Study Sample
The present study sample consisted of 466 Grade K students from across 10
elementary schools in Moline-Coal Valley. The student population among Moline-Coal
Valley students was mostly White (slightly under 50%), followed by Black and Hispanic
students (approximately 20% each). The teacher survey sample consisted of 26
teachers.
Program Impact on Reading and Mathematics Achievement
Observed impacts of Imagine MyPath on student achievement were mixed. A
statistically significant positive impact of Imagine MyPath on mathematics achievement
was observed in Moline-Coal Valley, with treatment students outgaining virtual
comparison students by more than 2 points. While treatment students also outgained
virtual comparison students in reading achievement, the difference was minimal and not
statistically significant.
Rates of Imagine MyPath usage varied somewhat across subjects. Moline-Coal
Valley students used the program approximately 11 hours for reading and eight hours
for mathematics. On average, more reading lessons were completed than mathematics
lessons, although lesson pass counts were very similar across subjects with students
averaging approximately 27 passed lessons in each subject. Program usage metrics
were generally positively related to mathematics and reading achievement gains, with
EXECUTIVE SUMMARY v
© Johns Hopkins University, 2023
counts of lessons passed and lessons completed generally having the strongest
magnitude associations with student achievement gains.
Teacher Perceptions
Teacher perceptions of Imagine MyPath were largely positive, especially
regarding perceptions of the program meeting student needs with approximately 80%
of program teachers agreeing that the curriculum:
• appropriately adopted instruction for students based on their needs, and
• initially placed students at a level appropriate for their skills and presented
students with content appropriate for their skill level throughout the school
year.
In general, implementing teachers also reported positive perceptions of the
program’s impact on student learning with over 80% of teachers agreeing that Imagine
MyPath improved students’ math and reading skills. Teacher perceptions were
somewhat lower regarding the program’s ability to accelerate students who were
behind in grade-level content, and in motivating students to persist through difficult
content. Overall, teacher perceptions of professional development (PD) were positive
although some teachers stated that additional or more focused PD, especially in relation
to technical components of the program, would improve their implementation,
particularly in their use of the online teacher platform.
© Johns Hopkins University, 2023
counts of lessons passed and lessons completed generally having the strongest
magnitude associations with student achievement gains.
Teacher Perceptions
Teacher perceptions of Imagine MyPath were largely positive, especially
regarding perceptions of the program meeting student needs with approximately 80%
of program teachers agreeing that the curriculum:
• appropriately adopted instruction for students based on their needs, and
• initially placed students at a level appropriate for their skills and presented
students with content appropriate for their skill level throughout the school
year.
In general, implementing teachers also reported positive perceptions of the
program’s impact on student learning with over 80% of teachers agreeing that Imagine
MyPath improved students’ math and reading skills. Teacher perceptions were
somewhat lower regarding the program’s ability to accelerate students who were
behind in grade-level content, and in motivating students to persist through difficult
content. Overall, teacher perceptions of professional development (PD) were positive
although some teachers stated that additional or more focused PD, especially in relation
to technical components of the program, would improve their implementation,
particularly in their use of the online teacher platform.
Imagine MyPath in Moline-Coal Valley 1
© Johns Hopkins University, 2023
Impact Evaluation of Imagine MyPath for Kindergarten Students
Imagine Learning contracted with the Center for Research and Reform in
Education (CRRE) at Johns Hopkins University to conduct a quasi-experimental design
(QED) study in Grades K–5 in two school districts. The current district is Moline-Coal
Valley School District No. 40, which serves approximately 7,200 students in 15 schools,
12 of which are elementary schools. This report focuses on efficacy impacts and teacher
perceptions of Imagine MyPath in the Moline-Coal Valley School District.
In brief, as described on the program website:
Imagine MyPath is designed to maximize student learning through personalized
learning paths that prioritize the essential skills students will need for grade-level
success. As students move from intervention to supplemental grade-level
practice and beyond, Imagine MyPath continues to deliver adaptive lessons
propelling them forward...To help meet demands in today’s classrooms, Imagine
Learning built a next-generation learning environment for K–12 that uses
Imagine MyPath’s built-in assessment data or can integrate with NWEA® MAP®
Growth and Renaissance Star® to design individualized learning paths that: (a)
Prioritize critical foundational skills and adapt learning based on student
performance, (b) Provide accessible, explicit, and age-appropriate scaffolded
instruction to ensure success among diverse learners, (c) Has the ability to
translate words, screens, or entire passages into more than 60 available
languages with read aloud support in over 30 languages, (d) Include Spanish
language math lessons for grades K–5, (e) Incorporate evidence-based practices
for teaching reading and mathematics, (f) Deliver actionable, real-time data to
inform instructional decision-making, and (g) Optimize student motivation,
engagement, and conceptual understanding.
Imagine MyPath was implemented three days per week in a period designated
for digital learning. This period served as planning time for regular classroom teachers
and was therefore led by a digital learning support assistant (DLSA). While both
classroom teachers and DLSAs received Imagine MyPath professional development,
DLSAs provided hands-on implementation and support to students in the program.
Classroom teachers also had access to Imagine MyPath reporting and resources to
utilize for individualized support to students who required additional instruction.
The evaluation’s design addressed the following research questions:
1) How does participation in Imagine MyPath impact student achievement in
mathematics and reading?
a) Does level of program usage relate to student achievement effects?
b) To what degree do effects vary across:
i) Schools?
© Johns Hopkins University, 2023
Impact Evaluation of Imagine MyPath for Kindergarten Students
Imagine Learning contracted with the Center for Research and Reform in
Education (CRRE) at Johns Hopkins University to conduct a quasi-experimental design
(QED) study in Grades K–5 in two school districts. The current district is Moline-Coal
Valley School District No. 40, which serves approximately 7,200 students in 15 schools,
12 of which are elementary schools. This report focuses on efficacy impacts and teacher
perceptions of Imagine MyPath in the Moline-Coal Valley School District.
In brief, as described on the program website:
Imagine MyPath is designed to maximize student learning through personalized
learning paths that prioritize the essential skills students will need for grade-level
success. As students move from intervention to supplemental grade-level
practice and beyond, Imagine MyPath continues to deliver adaptive lessons
propelling them forward...To help meet demands in today’s classrooms, Imagine
Learning built a next-generation learning environment for K–12 that uses
Imagine MyPath’s built-in assessment data or can integrate with NWEA® MAP®
Growth and Renaissance Star® to design individualized learning paths that: (a)
Prioritize critical foundational skills and adapt learning based on student
performance, (b) Provide accessible, explicit, and age-appropriate scaffolded
instruction to ensure success among diverse learners, (c) Has the ability to
translate words, screens, or entire passages into more than 60 available
languages with read aloud support in over 30 languages, (d) Include Spanish
language math lessons for grades K–5, (e) Incorporate evidence-based practices
for teaching reading and mathematics, (f) Deliver actionable, real-time data to
inform instructional decision-making, and (g) Optimize student motivation,
engagement, and conceptual understanding.
Imagine MyPath was implemented three days per week in a period designated
for digital learning. This period served as planning time for regular classroom teachers
and was therefore led by a digital learning support assistant (DLSA). While both
classroom teachers and DLSAs received Imagine MyPath professional development,
DLSAs provided hands-on implementation and support to students in the program.
Classroom teachers also had access to Imagine MyPath reporting and resources to
utilize for individualized support to students who required additional instruction.
The evaluation’s design addressed the following research questions:
1) How does participation in Imagine MyPath impact student achievement in
mathematics and reading?
a) Does level of program usage relate to student achievement effects?
b) To what degree do effects vary across:
i) Schools?
Imagine MyPath Moline-Coal Valley 2
© Johns Hopkins University, 2023
ii) Grade levels?
2) What are teachers’ perceptions of the program with regard to:
a) Benefits for students?
b) Student engagement?
c) Implementation requirements and experiences?
d) Strengths and weaknesses?
e) Recommendations for implementation improvement?
Method
Research Design
A quasi-experimental design (QED) was used to compare mathematics and
reading achievement of Imagine MyPath students to that of matched comparison
student data obtained through a Similar Schools Report provided by NWEA to the
intervention district. Similar Schools Reports contain data from students who, relative to
the intervention students at the district, come from schools in a similar area (urban,
suburban, rural), with similar percentages of free and reduced meals students (FARMS).
Additionally, students as a group are matched on the basis of grade level and prior
NWEA MAP Growth (MAP) achievement, as well as demographic variables including
gender and ethnicity. This creates “virtual control” groups of students, allowing for
direct comparisons of MAP score growth between intervention students and otherwise
similar students who did not use Imagine MyPath. In addition, student-level Imagine
MyPath program usage data from Imagine Learning were also obtained. Originally,
implementation of Imagine MyPath was planned by the district for multiple elementary
grades but was subsequently restricted to Grade K only. Qualitative data were collected
through an online teacher questionnaire that was administered to kindergarten
intervention teachers in Moline-Coal Valley School District (as kindergarten was the only
grade level in which Imagine MyPath was used for the entire 2022–23 school year).
Participants
Student sample. Moline-Coal Valley School District No. 40 serves
approximately 7,200 students in 15 schools, 12 of which are elementary schools. Data
from a total of 466 kindergarten students in Moline-Coal Valley were used in the
present analyses. Available demographic data for these students is shown in Table 1.
Table 1
Student Characteristics of Moline-Coal Valley Analytic Sample
Grade K
Group Percentages
© Johns Hopkins University, 2023
ii) Grade levels?
2) What are teachers’ perceptions of the program with regard to:
a) Benefits for students?
b) Student engagement?
c) Implementation requirements and experiences?
d) Strengths and weaknesses?
e) Recommendations for implementation improvement?
Method
Research Design
A quasi-experimental design (QED) was used to compare mathematics and
reading achievement of Imagine MyPath students to that of matched comparison
student data obtained through a Similar Schools Report provided by NWEA to the
intervention district. Similar Schools Reports contain data from students who, relative to
the intervention students at the district, come from schools in a similar area (urban,
suburban, rural), with similar percentages of free and reduced meals students (FARMS).
Additionally, students as a group are matched on the basis of grade level and prior
NWEA MAP Growth (MAP) achievement, as well as demographic variables including
gender and ethnicity. This creates “virtual control” groups of students, allowing for
direct comparisons of MAP score growth between intervention students and otherwise
similar students who did not use Imagine MyPath. In addition, student-level Imagine
MyPath program usage data from Imagine Learning were also obtained. Originally,
implementation of Imagine MyPath was planned by the district for multiple elementary
grades but was subsequently restricted to Grade K only. Qualitative data were collected
through an online teacher questionnaire that was administered to kindergarten
intervention teachers in Moline-Coal Valley School District (as kindergarten was the only
grade level in which Imagine MyPath was used for the entire 2022–23 school year).
Participants
Student sample. Moline-Coal Valley School District No. 40 serves
approximately 7,200 students in 15 schools, 12 of which are elementary schools. Data
from a total of 466 kindergarten students in Moline-Coal Valley were used in the
present analyses. Available demographic data for these students is shown in Table 1.
Table 1
Student Characteristics of Moline-Coal Valley Analytic Sample
Grade K
Group Percentages
Imagine MyPath Moline-Coal Valley 3
© Johns Hopkins University, 2023
% Black 16.77
% White 44.00
% Hispanic 21.75
% Other Race 18.07
% Female 48.27
N 466
Nearly half of students in the Moline-Coal Valley sample were White, followed by
approximately 21% Hispanic, and slightly smaller percentages of Black and Other Race
students, consistent with overall district demographic trends.
Teacher sample. A total of 38 kindergarten treatment teachers across 12
elementary schools were invited to complete the Imagine MyPath questionnaire and
were offered an incentive in the form of a $15 gift card for their participation. A total of
26 participants completed the questionnaire, yielding a 68.4% response rate. Most
participants were classroom teachers, with smaller numbers of digital learning support
assistants, instructional aides or paraprofessionals providing feedback.
Measures
Data sources for the current study include student mathematics and reading
achievement scores, as measured by progress monitoring assessments, along with
program usage data and a teacher questionnaire.
NWEA MAP Growth. Moline-Coal Valley provided 2022–23 fall and spring
NWEA MAP Growth Mathematics and Reading assessment scores for all kindergarten
students in district elementary schools. MAP Growth RIT scores are vertically scaled so
that scores can be directly compared across grade levels. Table 2 shows the ranges of
MAP Growth RIT Mathematics and Reading scores for kindergarten students at the end
of the 2022–23 school year.
Table 2
MAP Growth RIT Score Ranges, by Grade and Subject, Moline-Coal Valley
Subject MAP RIT score range
Reading (n = 466) 117 - 191
Mathematics (n = 458) 120 - 201
Program usage. Imagine Learning provided student-level Imagine MyPath
program usage data. Imagine MyPath usage metrics include total active minutes, as
well as counts of lessons completed, and lessons passed. Percentages of passed lessons
were also included in these metrics. In order to optimize student learning outcomes
when using Imagine MyPath, Imagine Learning recommends that students use the
© Johns Hopkins University, 2023
% Black 16.77
% White 44.00
% Hispanic 21.75
% Other Race 18.07
% Female 48.27
N 466
Nearly half of students in the Moline-Coal Valley sample were White, followed by
approximately 21% Hispanic, and slightly smaller percentages of Black and Other Race
students, consistent with overall district demographic trends.
Teacher sample. A total of 38 kindergarten treatment teachers across 12
elementary schools were invited to complete the Imagine MyPath questionnaire and
were offered an incentive in the form of a $15 gift card for their participation. A total of
26 participants completed the questionnaire, yielding a 68.4% response rate. Most
participants were classroom teachers, with smaller numbers of digital learning support
assistants, instructional aides or paraprofessionals providing feedback.
Measures
Data sources for the current study include student mathematics and reading
achievement scores, as measured by progress monitoring assessments, along with
program usage data and a teacher questionnaire.
NWEA MAP Growth. Moline-Coal Valley provided 2022–23 fall and spring
NWEA MAP Growth Mathematics and Reading assessment scores for all kindergarten
students in district elementary schools. MAP Growth RIT scores are vertically scaled so
that scores can be directly compared across grade levels. Table 2 shows the ranges of
MAP Growth RIT Mathematics and Reading scores for kindergarten students at the end
of the 2022–23 school year.
Table 2
MAP Growth RIT Score Ranges, by Grade and Subject, Moline-Coal Valley
Subject MAP RIT score range
Reading (n = 466) 117 - 191
Mathematics (n = 458) 120 - 201
Program usage. Imagine Learning provided student-level Imagine MyPath
program usage data. Imagine MyPath usage metrics include total active minutes, as
well as counts of lessons completed, and lessons passed. Percentages of passed lessons
were also included in these metrics. In order to optimize student learning outcomes
when using Imagine MyPath, Imagine Learning recommends that students use the
Imagine MyPath Moline-Coal Valley 4
© Johns Hopkins University, 2023
program for a minimum of 30 minutes per week per subject. Students below grade
level should spend 60–90 minutes per subject per week. During this time in the
program, students should complete at least one lesson per subject per week.
Teacher questionnaire. The teacher questionnaire was administered to
teachers of Imagine MyPath in the 2022–23 school year. The questionnaire included
curriculum-specific questions relating to classroom practices, student motivation and
achievement, program implementation and usage, experience with digital learning,
professional development and training, and overall program perceptions. The
questionnaire contained Likert-scale and yes/no questions, along with four open-ended
items. Likert-scale questionnaire responses were analyzed using descriptive statistics
(e.g., percentages and counts), while open-ended questionnaire responses were
analyzed qualitatively. A copy of the teacher questionnaire can be found in Appendix A.
Analytical Approach
Achievement data for kindergarten students were analyzed descriptively by
examining patterns in MAP mathematics and reading scores for the intervention and
virtual control group in each district. Fall MAP mathematics and reading scores were
defined as pretest measures, while spring MAP mathematics and reading scores were
defined as posttest measures. As students were matched with their virtual comparison
counterparts and are otherwise similar in terms of prior achievement and
demographics, dependent t-tests were conducted by CRRE to examine differences in
MAP mathematics and reading growth between Imagine MyPath students and virtual
comparison students.
Likert-scale questionnaire items were analyzed descriptively, while open-ended
responses were analyzed using qualitative analytic techniques (Miles et al., 2002).
Achievement Results
In this section, we describe the results of the QED comparing student MAP
mathematics and reading score growth from fall 2022 to spring 2023 of Imagine
MyPath students to that of similar virtual comparison students who did not use Imagine
MyPath. Baseline equivalence on MAP mathematics and reading scores is shown in
Appendix B; as students were matched by NWEA on prior achievement, this
requirement is essentially routine, and baseline differences did not exceed 0.01
standard deviations on any grade-level comparisons.
Descriptive Analyses
We first descriptively compare MAP mathematics and reading score trends for
kindergarten students across the 2022–23 school year. Only students with non-missing
© Johns Hopkins University, 2023
program for a minimum of 30 minutes per week per subject. Students below grade
level should spend 60–90 minutes per subject per week. During this time in the
program, students should complete at least one lesson per subject per week.
Teacher questionnaire. The teacher questionnaire was administered to
teachers of Imagine MyPath in the 2022–23 school year. The questionnaire included
curriculum-specific questions relating to classroom practices, student motivation and
achievement, program implementation and usage, experience with digital learning,
professional development and training, and overall program perceptions. The
questionnaire contained Likert-scale and yes/no questions, along with four open-ended
items. Likert-scale questionnaire responses were analyzed using descriptive statistics
(e.g., percentages and counts), while open-ended questionnaire responses were
analyzed qualitatively. A copy of the teacher questionnaire can be found in Appendix A.
Analytical Approach
Achievement data for kindergarten students were analyzed descriptively by
examining patterns in MAP mathematics and reading scores for the intervention and
virtual control group in each district. Fall MAP mathematics and reading scores were
defined as pretest measures, while spring MAP mathematics and reading scores were
defined as posttest measures. As students were matched with their virtual comparison
counterparts and are otherwise similar in terms of prior achievement and
demographics, dependent t-tests were conducted by CRRE to examine differences in
MAP mathematics and reading growth between Imagine MyPath students and virtual
comparison students.
Likert-scale questionnaire items were analyzed descriptively, while open-ended
responses were analyzed using qualitative analytic techniques (Miles et al., 2002).
Achievement Results
In this section, we describe the results of the QED comparing student MAP
mathematics and reading score growth from fall 2022 to spring 2023 of Imagine
MyPath students to that of similar virtual comparison students who did not use Imagine
MyPath. Baseline equivalence on MAP mathematics and reading scores is shown in
Appendix B; as students were matched by NWEA on prior achievement, this
requirement is essentially routine, and baseline differences did not exceed 0.01
standard deviations on any grade-level comparisons.
Descriptive Analyses
We first descriptively compare MAP mathematics and reading score trends for
kindergarten students across the 2022–23 school year. Only students with non-missing
Imagine MyPath Moline-Coal Valley 5
© Johns Hopkins University, 2023
fall 2022 and spring 2023 scores are included in these analyses. Table 3 shows average
MAP math and reading scores at both time points.
Table 3
Average MAP Growth Math and Reading Scores, Fall 2022 to Spring 2023, Moline-Coal
Valley (Grade K)
Condition Fall 2022 Spring 2023 Change
Reading (n = 470)
Imagine MyPath 133.83 153.32 19.49
Virtual Comparison 133.88 153.02 19.14
Mathematics (n = 458)
Imagine MyPath 137.61 160.70 23.09
Virtual Comparison 137.65 158.62 20.97
Imagine MyPath students outgained virtual comparison students on the
mathematics assessment by slightly more than 2 points, while MAP reading gains for
Imagine MyPath students and virtual comparison students were similar, with Imagine
MyPath students slightly outgaining virtual comparison students.
Usage data. Imagine MyPath usage data were analyzed descriptively. Table 4
shows counts of provided Imagine MyPath usage metrics from across the 2022–23
school year for students with at least one non-missing outcome measure.
Table 4
Average Imagine MyPath Program Usage by Subject
Usage Type Mean SD Minimum Maximum
Mathematics (n = 458)
Active Minutes 855.97 291.08 20.00 1,979
Lessons Completed 37.35 13.46 1 94
Lessons Passed 27.36 14.62 0 89
Lessons Passed Percentage 69.76 22.10 0 100
Reading (n = 466)
Active Minutes 1,055.72 331.66 190.00 1,855
Lessons Completed 46.11 17.20 4 127
Lessons Passed 27.80 18.68 0 111
Lessons Passed Percentage 56.10 24.68 0 100
In Moline-Coal Valley, average Imagine MyPath usage metrics were generally
higher for reading than they were for mathematics. Students averaged nearly 11 hours
© Johns Hopkins University, 2023
fall 2022 and spring 2023 scores are included in these analyses. Table 3 shows average
MAP math and reading scores at both time points.
Table 3
Average MAP Growth Math and Reading Scores, Fall 2022 to Spring 2023, Moline-Coal
Valley (Grade K)
Condition Fall 2022 Spring 2023 Change
Reading (n = 470)
Imagine MyPath 133.83 153.32 19.49
Virtual Comparison 133.88 153.02 19.14
Mathematics (n = 458)
Imagine MyPath 137.61 160.70 23.09
Virtual Comparison 137.65 158.62 20.97
Imagine MyPath students outgained virtual comparison students on the
mathematics assessment by slightly more than 2 points, while MAP reading gains for
Imagine MyPath students and virtual comparison students were similar, with Imagine
MyPath students slightly outgaining virtual comparison students.
Usage data. Imagine MyPath usage data were analyzed descriptively. Table 4
shows counts of provided Imagine MyPath usage metrics from across the 2022–23
school year for students with at least one non-missing outcome measure.
Table 4
Average Imagine MyPath Program Usage by Subject
Usage Type Mean SD Minimum Maximum
Mathematics (n = 458)
Active Minutes 855.97 291.08 20.00 1,979
Lessons Completed 37.35 13.46 1 94
Lessons Passed 27.36 14.62 0 89
Lessons Passed Percentage 69.76 22.10 0 100
Reading (n = 466)
Active Minutes 1,055.72 331.66 190.00 1,855
Lessons Completed 46.11 17.20 4 127
Lessons Passed 27.80 18.68 0 111
Lessons Passed Percentage 56.10 24.68 0 100
In Moline-Coal Valley, average Imagine MyPath usage metrics were generally
higher for reading than they were for mathematics. Students averaged nearly 11 hours
Imagine MyPath Moline-Coal Valley 6
© Johns Hopkins University, 2023
of reading usage, as compared to slightly more than eight hours of mathematics usage.
Students also averaged approximately nine more reading lessons completed than
mathematics lessons completed, although counts of lessons passed were similar in both
subjects.
Impact analyses. Next, we examine the impacts of Imagine MyPath on MAP
mathematics and reading gain scores, in relation to virtual comparison students, by
conducting dependent (matched) t-tests on mathematics and reading gain scores from
fall 2022 to spring 2023. Table 5 shows estimated Imagine MyPath impacts on MAP
mathematics and reading gains for kindergarten, while Figure 1 shows adjusted fall to
spring gains for Imagine MyPath and virtual comparison students. Students included in
these analyses had non-missing fall 2022 and spring 2023 MAP mathematics and/or
reading scores.
Table 5
Imagine MyPath Impacts on MAP Gain Scores, by Subject for Kindergarten, Moline-Coal
Valley
Subject Estimate Standard Error p value
Mathematics (n = 458) 2.129*** 0.449 <.001
Reading (n = 469) 0.298 0.484 .538
Note. *** p < .001.
Figure 1
MAP Gain Scores, by Subject and Condition
Note. *** p < .001.
23.09***
19.49
20.97
19.14
0
5
10
15
20
25
Mathematics Reading
MAP Growth RIT Score Gains
MyPath Comparison
© Johns Hopkins University, 2023
of reading usage, as compared to slightly more than eight hours of mathematics usage.
Students also averaged approximately nine more reading lessons completed than
mathematics lessons completed, although counts of lessons passed were similar in both
subjects.
Impact analyses. Next, we examine the impacts of Imagine MyPath on MAP
mathematics and reading gain scores, in relation to virtual comparison students, by
conducting dependent (matched) t-tests on mathematics and reading gain scores from
fall 2022 to spring 2023. Table 5 shows estimated Imagine MyPath impacts on MAP
mathematics and reading gains for kindergarten, while Figure 1 shows adjusted fall to
spring gains for Imagine MyPath and virtual comparison students. Students included in
these analyses had non-missing fall 2022 and spring 2023 MAP mathematics and/or
reading scores.
Table 5
Imagine MyPath Impacts on MAP Gain Scores, by Subject for Kindergarten, Moline-Coal
Valley
Subject Estimate Standard Error p value
Mathematics (n = 458) 2.129*** 0.449 <.001
Reading (n = 469) 0.298 0.484 .538
Note. *** p < .001.
Figure 1
MAP Gain Scores, by Subject and Condition
Note. *** p < .001.
23.09***
19.49
20.97
19.14
0
5
10
15
20
25
Mathematics Reading
MAP Growth RIT Score Gains
MyPath Comparison
Imagine MyPath Moline-Coal Valley 7
© Johns Hopkins University, 2023
A statistically significant positive impact of Imagine MyPath on mathematics gain
scores was evidenced for kindergarten students in Moline-Coal Valley, with treatment
students averaging more than 2-point larger gains in relation to virtual comparison
students. No statistically significant impact of Imagine MyPath on reading gain scores
was evidenced, with treatment students slightly outgaining virtual comparison students
by approximately 0.3 points.
Associations between usage and achievement. As Imagine Learning
provided student-level program usage for Imagine MyPath, Pearson correlations were
computed between usage metrics and MAP mathematics and reading gain scores to
probe potential associations between program usage and achievement. Table 6 shows
these correlations for Imagine MyPath usage.
Table 6
Associations Between Imagine MyPath Curriculum Usage and MAP Growth Score Gains,
by Subject, Moline-Coal Valley
Pearson’s r
Subject Active
Minutes
Lessons
Completed
Lessons
Passed
Lessons
Passed
Percentage
Mathematics (n = 458) +.25*** +.37*** +.42*** +.38***
Reading (n = 466) +.14*** +.45*** +.52*** +.44***
Note. *** p < .001.
Associations between Imagine MyPath usage metrics and MAP gain scores were
generally statistically significant and positive, with magnitudes of these associations
ranging between .25 to .52. The strongest associations were found between measures
of lessons passed (counts and percentages) and achievement gains, while the weakest
associations were found in relation to total active minutes.
Teacher Questionnaire Results
Major takeaways from the Moline-Coal Valley School District teacher
questionnaire responses are presented in the section below. We begin with findings
pertaining to teacher backgrounds and implementation of the Imagine MyPath program.
These sections are followed by teacher perceptions of program impact on student
engagement and achievement. Response rates varied by question and item throughout
the survey, with item sample sizes reported within each figure.
Background
© Johns Hopkins University, 2023
A statistically significant positive impact of Imagine MyPath on mathematics gain
scores was evidenced for kindergarten students in Moline-Coal Valley, with treatment
students averaging more than 2-point larger gains in relation to virtual comparison
students. No statistically significant impact of Imagine MyPath on reading gain scores
was evidenced, with treatment students slightly outgaining virtual comparison students
by approximately 0.3 points.
Associations between usage and achievement. As Imagine Learning
provided student-level program usage for Imagine MyPath, Pearson correlations were
computed between usage metrics and MAP mathematics and reading gain scores to
probe potential associations between program usage and achievement. Table 6 shows
these correlations for Imagine MyPath usage.
Table 6
Associations Between Imagine MyPath Curriculum Usage and MAP Growth Score Gains,
by Subject, Moline-Coal Valley
Pearson’s r
Subject Active
Minutes
Lessons
Completed
Lessons
Passed
Lessons
Passed
Percentage
Mathematics (n = 458) +.25*** +.37*** +.42*** +.38***
Reading (n = 466) +.14*** +.45*** +.52*** +.44***
Note. *** p < .001.
Associations between Imagine MyPath usage metrics and MAP gain scores were
generally statistically significant and positive, with magnitudes of these associations
ranging between .25 to .52. The strongest associations were found between measures
of lessons passed (counts and percentages) and achievement gains, while the weakest
associations were found in relation to total active minutes.
Teacher Questionnaire Results
Major takeaways from the Moline-Coal Valley School District teacher
questionnaire responses are presented in the section below. We begin with findings
pertaining to teacher backgrounds and implementation of the Imagine MyPath program.
These sections are followed by teacher perceptions of program impact on student
engagement and achievement. Response rates varied by question and item throughout
the survey, with item sample sizes reported within each figure.
Background
Imagine MyPath Moline-Coal Valley 8
© Johns Hopkins University, 2023
Respondents (n = 26) represented 10 different elementary schools within the
Moline-Coal Valley School District; the majority (57.7%, n = 15) identified primarily as
classroom teachers who teach in a variety of grade levels, ranging from kindergarten to
Grade 5. All respondents reported teaching kindergarten, while also reporting teaching
first, second, third, fourth, and fifth grades. In addition to teachers, 11 respondents
(42.3%) identified as digital learning support assistants (DLSAs). Respondents
estimated that the majority of students in their classrooms were either below-grade
level (31.7%) or on-grade level (35.7%) readers.
Teachers reported on whether they had prior exposure to the Imagine MyPath
program. The majority of respondents (69.2%, n = 18) had no previous exposure to
Imagine MyPath during the previous 2021–22 school year. Most respondents to this
question (85.7%, n = 12) also identified themselves as being confident or somewhat
confident implementing technology in their classroom. Respondents reported prior
experience using math programs including ST Math, Prodigy Math, and Imagine
MyPath, and reading programs such as Imagine Language & Literacy, Kidz A to Z, and
RazKidz.
Respondents were first asked about their usage of Imagine MyPath in the
classroom with students. Every respondent reported using Imagine MyPath on a weekly
or daily basis, evenly split between seven respondents reporting daily use and seven
respondents reporting weekly use. They were also asked to specify their usage
frequency for specific practices and aspects of the program (Figure 2). In this set of
items, the percentage reported reflects the number of individuals who reported using
certain Imagine MyPath instructional practices.
Figure 2
Teachers’ Reported Use of Imagine MyPath Instructional Practices with Students
© Johns Hopkins University, 2023
Respondents (n = 26) represented 10 different elementary schools within the
Moline-Coal Valley School District; the majority (57.7%, n = 15) identified primarily as
classroom teachers who teach in a variety of grade levels, ranging from kindergarten to
Grade 5. All respondents reported teaching kindergarten, while also reporting teaching
first, second, third, fourth, and fifth grades. In addition to teachers, 11 respondents
(42.3%) identified as digital learning support assistants (DLSAs). Respondents
estimated that the majority of students in their classrooms were either below-grade
level (31.7%) or on-grade level (35.7%) readers.
Teachers reported on whether they had prior exposure to the Imagine MyPath
program. The majority of respondents (69.2%, n = 18) had no previous exposure to
Imagine MyPath during the previous 2021–22 school year. Most respondents to this
question (85.7%, n = 12) also identified themselves as being confident or somewhat
confident implementing technology in their classroom. Respondents reported prior
experience using math programs including ST Math, Prodigy Math, and Imagine
MyPath, and reading programs such as Imagine Language & Literacy, Kidz A to Z, and
RazKidz.
Respondents were first asked about their usage of Imagine MyPath in the
classroom with students. Every respondent reported using Imagine MyPath on a weekly
or daily basis, evenly split between seven respondents reporting daily use and seven
respondents reporting weekly use. They were also asked to specify their usage
frequency for specific practices and aspects of the program (Figure 2). In this set of
items, the percentage reported reflects the number of individuals who reported using
certain Imagine MyPath instructional practices.
Figure 2
Teachers’ Reported Use of Imagine MyPath Instructional Practices with Students
Imagine MyPath Moline-Coal Valley 9
© Johns Hopkins University, 2023
Respondents reported also on the frequency with which they conducted certain
activities while implementing Imagine MyPath with students. The majority of
respondents reported conducting various student support activities on a daily or weekly
basis. Most (64.3%, n = 9) respondents worked with individuals who were struggling
on a daily or weekly basis, with 42.9% (n = 6) doing so on a daily basis. However,
78.6% (n = 11) of respondents reported never working with groups of students who
were not struggling and 64.3% (n = 9) of respondents reported never using offline
resources with students. Meanwhile, over one-quarter (28.6%, n = 4) reported never
working with students who were struggling.
As noted in Figure 2, teachers implementing Imagine MyPath had the option to
include additional incentives to encourage student engagement in learning and
classroom activities. More than half (64.3%, n = 9) of respondents said they motivated
students with extra incentives outside of the program at some point, with 35.7% (n =
5) doing so on a weekly basis and 14.3% (n = 2) doing so on a daily basis.
Incentives used varied from respondent to respondent, with two respondents
mentioning points, such as Dojo points, “claw cards”, or as part of a campus or school
award system. Prizes were another common incentive used, such as stickers or candy,
extra recess, or time on fun computer websites. Celebrations of various sorts were also
popular ideas, announced as an opportunity for students who reached certain learning
thresholds, such as the number of lessons completed. These included Fun Fridays or
78.6%
64.3%
42.9%
35.7%
28.6%
7.1%
14.3%
14.3%
7.1%
7.1%
21.4%
35.7%
21.4%
14.3%
28.6%
21.4%
14.3%
42.9%
Worked with groups of
students. (n = 14)
Used offline resources
with students. (n = 14)
Worked with individual
students who were not
struggling. (n = 14)
Motivated students with
extra incentives
(outside the...
Worked with individual
students who were
struggling. (n = 14)
Never Monthly Weekly Daily
Frequency of Instructional Practices Use
© Johns Hopkins University, 2023
Respondents reported also on the frequency with which they conducted certain
activities while implementing Imagine MyPath with students. The majority of
respondents reported conducting various student support activities on a daily or weekly
basis. Most (64.3%, n = 9) respondents worked with individuals who were struggling
on a daily or weekly basis, with 42.9% (n = 6) doing so on a daily basis. However,
78.6% (n = 11) of respondents reported never working with groups of students who
were not struggling and 64.3% (n = 9) of respondents reported never using offline
resources with students. Meanwhile, over one-quarter (28.6%, n = 4) reported never
working with students who were struggling.
As noted in Figure 2, teachers implementing Imagine MyPath had the option to
include additional incentives to encourage student engagement in learning and
classroom activities. More than half (64.3%, n = 9) of respondents said they motivated
students with extra incentives outside of the program at some point, with 35.7% (n =
5) doing so on a weekly basis and 14.3% (n = 2) doing so on a daily basis.
Incentives used varied from respondent to respondent, with two respondents
mentioning points, such as Dojo points, “claw cards”, or as part of a campus or school
award system. Prizes were another common incentive used, such as stickers or candy,
extra recess, or time on fun computer websites. Celebrations of various sorts were also
popular ideas, announced as an opportunity for students who reached certain learning
thresholds, such as the number of lessons completed. These included Fun Fridays or
78.6%
64.3%
42.9%
35.7%
28.6%
7.1%
14.3%
14.3%
7.1%
7.1%
21.4%
35.7%
21.4%
14.3%
28.6%
21.4%
14.3%
42.9%
Worked with groups of
students. (n = 14)
Used offline resources
with students. (n = 14)
Worked with individual
students who were not
struggling. (n = 14)
Motivated students with
extra incentives
(outside the...
Worked with individual
students who were
struggling. (n = 14)
Never Monthly Weekly Daily
Frequency of Instructional Practices Use
Imagine MyPath Moline-Coal Valley 10
© Johns Hopkins University, 2023
class parties. One respondent also mentioned growth charts as a means of
documenting and following along with progress.
Many respondents (78.6%, n = 11) said that these additional incentives were
effective in motivating their students to pass lessons. Respondents also commented on
whether incentives were effective in different ways for different groups of students.
Overall, the majority felt that there were no differently influential incentives or that they
had not noticed any differences. One respondent noted “It is effective for most
students. I don’t see a specific type of student that it wouldn’t work for. I could see that
it wouldn’t work as well with older students.”
Respondents were also asked about the equipment used when implementing the
program. Every respondent for this item (100%, n = 24) reported that most or all of
their students wore headphones while using Imagine MyPath.
Program Implementation
Respondents were asked to describe the frequency with which they used certain
teacher support components of the Imagine MyPath program (see Figure 3), including
student and class reports, assignment builders, and a teacher toolkit. In this set of
items, the percentage reported reflects the number of individuals who reported using
certain Imagine MyPath programmatic features.
It is important to note that DLSAs (42.3%, n = 11) were the only respondents
who were responsible for the implementation of Imagine MyPath. While classroom
teachers (57.7%, n = 15) received some professional development on the program and
were encouraged to use program resources to support their instruction, they were not
directly responsible or held accountable for their student’s use of Imagine MyPath.
Figure 3
Teacher Responses for Frequency of Usage for Specific Imagine MyPath Features and
Resources
© Johns Hopkins University, 2023
class parties. One respondent also mentioned growth charts as a means of
documenting and following along with progress.
Many respondents (78.6%, n = 11) said that these additional incentives were
effective in motivating their students to pass lessons. Respondents also commented on
whether incentives were effective in different ways for different groups of students.
Overall, the majority felt that there were no differently influential incentives or that they
had not noticed any differences. One respondent noted “It is effective for most
students. I don’t see a specific type of student that it wouldn’t work for. I could see that
it wouldn’t work as well with older students.”
Respondents were also asked about the equipment used when implementing the
program. Every respondent for this item (100%, n = 24) reported that most or all of
their students wore headphones while using Imagine MyPath.
Program Implementation
Respondents were asked to describe the frequency with which they used certain
teacher support components of the Imagine MyPath program (see Figure 3), including
student and class reports, assignment builders, and a teacher toolkit. In this set of
items, the percentage reported reflects the number of individuals who reported using
certain Imagine MyPath programmatic features.
It is important to note that DLSAs (42.3%, n = 11) were the only respondents
who were responsible for the implementation of Imagine MyPath. While classroom
teachers (57.7%, n = 15) received some professional development on the program and
were encouraged to use program resources to support their instruction, they were not
directly responsible or held accountable for their student’s use of Imagine MyPath.
Figure 3
Teacher Responses for Frequency of Usage for Specific Imagine MyPath Features and
Resources
Imagine MyPath Moline-Coal Valley 11
© Johns Hopkins University, 2023
Note. + <5%.
Most teachers reported not using the tools on a daily basis, with just 13% of
teachers (n = 3) reporting using student reports or class reports on a daily basis to
monitor individuals’ usage, growth, and progress. Tools were more commonly used on a
weekly or monthly basis. A total of 60.9% of respondents (n = 14) used the student
reports either weekly or monthly, with 34.8% (n = 8) saying they used them on a
weekly basis. Just over half (56.5%, n = 13) of respondents said they used class
reports to monitor usage, growth, and progress on a weekly or monthly basis, including
34.8% of whom reported doing so on a weekly basis (n = 8).
Some tools did not seem to be utilized as frequently. While 15% of respondents
(n = 3) used the Imagine MyPath reports to individualize instruction on a weekly basis,
60.0% (n = 12) of teachers claimed to have never used the reports. No respondents
reported doing so on a daily basis. In addition, no respondents reported using the
assignment builder on a daily or weekly basis. Meanwhile, 70% (n = 14) said they
never used the assignment builder and 75% (n = 15) reported never having used the
teacher toolkit to reteach lessons to students needing that support.
Teachers were also asked about their experience using the Imagine MyPath
platform and reports (Figure 4). In this set of items (and those moving forward),
percent agreement is defined as the percentage of teachers that somewhat agree or
70.0%
75.0%
60.0%
30.4%
26.1%
30.0%
15.0%
25.0%
21.7%
26.1%
+
+
15.0%
34.8%
34.8%
+
+
13.0%
13.0%
Used the assignment
builder. (n = 20)
Used the teacher toolkit
to reteach lessons to
students who needed it.
(n = 20)
Used Imagine MyPath
reports to individualize
instruction. (n = 20)
Used class reports to
monitor usage, growth,
and progress. (n = 23)
Used student reports to
monitor individuals’
usage, growth, and
progress. (n = 23)
Never Monthly Weekly Daily
Frequency of Resource Use
© Johns Hopkins University, 2023
Note. + <5%.
Most teachers reported not using the tools on a daily basis, with just 13% of
teachers (n = 3) reporting using student reports or class reports on a daily basis to
monitor individuals’ usage, growth, and progress. Tools were more commonly used on a
weekly or monthly basis. A total of 60.9% of respondents (n = 14) used the student
reports either weekly or monthly, with 34.8% (n = 8) saying they used them on a
weekly basis. Just over half (56.5%, n = 13) of respondents said they used class
reports to monitor usage, growth, and progress on a weekly or monthly basis, including
34.8% of whom reported doing so on a weekly basis (n = 8).
Some tools did not seem to be utilized as frequently. While 15% of respondents
(n = 3) used the Imagine MyPath reports to individualize instruction on a weekly basis,
60.0% (n = 12) of teachers claimed to have never used the reports. No respondents
reported doing so on a daily basis. In addition, no respondents reported using the
assignment builder on a daily or weekly basis. Meanwhile, 70% (n = 14) said they
never used the assignment builder and 75% (n = 15) reported never having used the
teacher toolkit to reteach lessons to students needing that support.
Teachers were also asked about their experience using the Imagine MyPath
platform and reports (Figure 4). In this set of items (and those moving forward),
percent agreement is defined as the percentage of teachers that somewhat agree or
70.0%
75.0%
60.0%
30.4%
26.1%
30.0%
15.0%
25.0%
21.7%
26.1%
+
+
15.0%
34.8%
34.8%
+
+
13.0%
13.0%
Used the assignment
builder. (n = 20)
Used the teacher toolkit
to reteach lessons to
students who needed it.
(n = 20)
Used Imagine MyPath
reports to individualize
instruction. (n = 20)
Used class reports to
monitor usage, growth,
and progress. (n = 23)
Used student reports to
monitor individuals’
usage, growth, and
progress. (n = 23)
Never Monthly Weekly Daily
Frequency of Resource Use
Imagine MyPath Moline-Coal Valley 12
© Johns Hopkins University, 2023
agree with an item, while percent disagreement is defined as the percentage of
teachers who disagree or somewhat disagree with an item.
Figure 4
Teachers’ Reported Experience Using Different Imagine MyPath Resources
Responses here were largely positive, with over three-quarters of respondents
agreeing or somewhat agreeing with each prompt, finding it easy to access and read
reports, as well as navigating the platform. In addition, 82.4% of respondents (n = 14)
found professional development to be essential in helping them navigate the platform
with little frustration. These findings suggest that the platform itself, as well as the
professional development preparing them to use Imagine MyPath, was well-designed
for the target audience.
Professional Development
Teachers received professional development (PD) related to the Imagine MyPath
program, and the related questionnaire item sought to evaluate the helpfulness of this
training. In particular, teachers were asked to indicate their level of agreement with the
following statements (see Figure 5).
Figure 5
16.7%
15.8%
15.8%
11.8%
5.6%
5.3%
5.3%
5.9%
33.3%
36.8%
31.6%
47.1%
44.4%
42.1%
47.4%
35.3%
I found it easy to navigate
the Imagine MyPath
platform. (n = 18)
I found it easy to read
reports that displayed
students’ usage, growth, and
progress. (n = 19)
I found it easy to access
reports that displayed
students’ usage, growth, and
progress. (n = 19)
The PD was essential for me
to navigate the Imagine
MyPath platform with little
frustration. (n = 17)
Disagree Somewhat Disagree Somewhat Agree Agree
Experience Using Imagine MyPath Teacher Resources
© Johns Hopkins University, 2023
agree with an item, while percent disagreement is defined as the percentage of
teachers who disagree or somewhat disagree with an item.
Figure 4
Teachers’ Reported Experience Using Different Imagine MyPath Resources
Responses here were largely positive, with over three-quarters of respondents
agreeing or somewhat agreeing with each prompt, finding it easy to access and read
reports, as well as navigating the platform. In addition, 82.4% of respondents (n = 14)
found professional development to be essential in helping them navigate the platform
with little frustration. These findings suggest that the platform itself, as well as the
professional development preparing them to use Imagine MyPath, was well-designed
for the target audience.
Professional Development
Teachers received professional development (PD) related to the Imagine MyPath
program, and the related questionnaire item sought to evaluate the helpfulness of this
training. In particular, teachers were asked to indicate their level of agreement with the
following statements (see Figure 5).
Figure 5
16.7%
15.8%
15.8%
11.8%
5.6%
5.3%
5.3%
5.9%
33.3%
36.8%
31.6%
47.1%
44.4%
42.1%
47.4%
35.3%
I found it easy to navigate
the Imagine MyPath
platform. (n = 18)
I found it easy to read
reports that displayed
students’ usage, growth, and
progress. (n = 19)
I found it easy to access
reports that displayed
students’ usage, growth, and
progress. (n = 19)
The PD was essential for me
to navigate the Imagine
MyPath platform with little
frustration. (n = 17)
Disagree Somewhat Disagree Somewhat Agree Agree
Experience Using Imagine MyPath Teacher Resources
Imagine MyPath Moline-Coal Valley 13
© Johns Hopkins University, 2023
Teachers’ Perceptions of the Imagine MyPath Professional Development
Teachers were generally positive regarding the impact of the Imagine MyPath
professional development. The majority of respondents agreed or somewhat agreed
that the PD enhanced their understanding of using data (77.0%, n = 10) and Teacher
Resources (63.7%, n = 7) to individualize instruction and/or reteach concepts, while
73.3% (n = 11) felt the PD enhanced their understanding of reviewing students’
Portfolios to inform instruction. However, teachers were less positive about the impact
of the PD on their understanding of grouping and regrouping students based on their
growth and progress, as 53.9% (n = 7) of teachers disagreed or somewhat disagreed
that it had improved their understanding.
Perceived Impact
Teachers were asked to provide their overall perceptions of the Imagine MyPath
program in terms of how the program met the needs of diverse learners, and how it
improved student engagement and learning.
In order to better understand the effectiveness of the program’s assessment,
structure, and materials, respondents answered questions (see Figure 6) related to how
well Imagine MyPath placed students based on their skill level, adopted instruction and
provided content based on student needs, moved students through the material, and
addressed the needs of diverse learners and emergent multilingual learners. In
particular, teachers were asked to indicate their level of agreement with the following
statements.
15.4%
9.1%
6.7%
7.7%
38.5%
27.3%
20.0%
15.4%
30.8%
36.4%
53.3%
46.2%
15.4%
27.3%
20.0%
30.8%
Grouping and regrouping
students based on growth
and progress. (n = 13)
Using Teacher Resources
to individualize and/or
reteach concepts. (n = 11)
Reviewing students’
Portfolios to inform
instruction. (n = 15)
Using data to individualize
instruction. (n = 13)
Disagree Somewhat Disagree Somewhat Agree Agree
Teachers' Peceptions of MyPath Professional Development Impact
© Johns Hopkins University, 2023
Teachers’ Perceptions of the Imagine MyPath Professional Development
Teachers were generally positive regarding the impact of the Imagine MyPath
professional development. The majority of respondents agreed or somewhat agreed
that the PD enhanced their understanding of using data (77.0%, n = 10) and Teacher
Resources (63.7%, n = 7) to individualize instruction and/or reteach concepts, while
73.3% (n = 11) felt the PD enhanced their understanding of reviewing students’
Portfolios to inform instruction. However, teachers were less positive about the impact
of the PD on their understanding of grouping and regrouping students based on their
growth and progress, as 53.9% (n = 7) of teachers disagreed or somewhat disagreed
that it had improved their understanding.
Perceived Impact
Teachers were asked to provide their overall perceptions of the Imagine MyPath
program in terms of how the program met the needs of diverse learners, and how it
improved student engagement and learning.
In order to better understand the effectiveness of the program’s assessment,
structure, and materials, respondents answered questions (see Figure 6) related to how
well Imagine MyPath placed students based on their skill level, adopted instruction and
provided content based on student needs, moved students through the material, and
addressed the needs of diverse learners and emergent multilingual learners. In
particular, teachers were asked to indicate their level of agreement with the following
statements.
15.4%
9.1%
6.7%
7.7%
38.5%
27.3%
20.0%
15.4%
30.8%
36.4%
53.3%
46.2%
15.4%
27.3%
20.0%
30.8%
Grouping and regrouping
students based on growth
and progress. (n = 13)
Using Teacher Resources
to individualize and/or
reteach concepts. (n = 11)
Reviewing students’
Portfolios to inform
instruction. (n = 15)
Using data to individualize
instruction. (n = 13)
Disagree Somewhat Disagree Somewhat Agree Agree
Teachers' Peceptions of MyPath Professional Development Impact
Imagine MyPath Moline-Coal Valley 14
© Johns Hopkins University, 2023
Figure 6
Teachers’ Perceived Impact of Imagine MyPath on Student Needs
Note. + <5%.
Respondents were largely positive regarding the program’s structure and
adaptability. Most teachers (86.9%, n = 20) agreed or somewhat agreed that the
program presented students with content appropriate to their skill level throughout the
year, and 82.6% (n = 19) that the program initially placed students at a level
appropriate for their skills. Similarly, teachers felt that the program worked for students
with few interruptions (82.6%, n = 19). Almost as many (78.2%, n = 18) felt that the
program appropriately adopted instruction for students based on their needs (78.3%,
n = 18), moved students through material efficiently (78.2%, n = 18), and met the
needs of diverse learners (78.3%, n = 18). Two-thirds (75.0%, n = 15) also agreed
that the program supported emergent multilingual learners (or “English learners”).
10.0%
+
8.7%
4.3%
15.0%
21.7%
21.7%
17.4%
8.7%
13.0%
13.0%
55.0%
39.1%
43.5%
39.1%
34.8%
39.1%
39.1%
20.0%
39.1%
34.8%
39.1%
47.8%
43.5%
47.8%
Effectively supported
emergent multilingual
learners. (n = 20)
Moved students through
material efficiently. (n =
23)
Met the needs of diverse
learners. (n = 23)
Appropriately adopted
instruction for students
based on their needs. (n...
Initially placed students at
a level appropriate for
their skills. (n = 23)
Worked for students with
few technical
interruptions. (n = 23)
Presented students with
content appropriate for
their skill level. (n = 23)
Disagree Somewhat disagree Somewhat agree Agree
The program...
© Johns Hopkins University, 2023
Figure 6
Teachers’ Perceived Impact of Imagine MyPath on Student Needs
Note. + <5%.
Respondents were largely positive regarding the program’s structure and
adaptability. Most teachers (86.9%, n = 20) agreed or somewhat agreed that the
program presented students with content appropriate to their skill level throughout the
year, and 82.6% (n = 19) that the program initially placed students at a level
appropriate for their skills. Similarly, teachers felt that the program worked for students
with few interruptions (82.6%, n = 19). Almost as many (78.2%, n = 18) felt that the
program appropriately adopted instruction for students based on their needs (78.3%,
n = 18), moved students through material efficiently (78.2%, n = 18), and met the
needs of diverse learners (78.3%, n = 18). Two-thirds (75.0%, n = 15) also agreed
that the program supported emergent multilingual learners (or “English learners”).
10.0%
+
8.7%
4.3%
15.0%
21.7%
21.7%
17.4%
8.7%
13.0%
13.0%
55.0%
39.1%
43.5%
39.1%
34.8%
39.1%
39.1%
20.0%
39.1%
34.8%
39.1%
47.8%
43.5%
47.8%
Effectively supported
emergent multilingual
learners. (n = 20)
Moved students through
material efficiently. (n =
23)
Met the needs of diverse
learners. (n = 23)
Appropriately adopted
instruction for students
based on their needs. (n...
Initially placed students at
a level appropriate for
their skills. (n = 23)
Worked for students with
few technical
interruptions. (n = 23)
Presented students with
content appropriate for
their skill level. (n = 23)
Disagree Somewhat disagree Somewhat agree Agree
The program...
Imagine MyPath Moline-Coal Valley 15
© Johns Hopkins University, 2023
Teachers were asked specifically about how participating students engaged with
the Imagine MyPath program, with a focus on motivation and whether students were
appropriately challenged by the program. Specifically, teachers were asked to indicate
their level of agreement with the following statements. The majority of respondents felt
that the program appropriately challenged students (73.9%, n = 17), while more than
half (65.2%, n = 15) of respondents agreed or somewhat agreed that the program
content motivated students to persist through difficult content.
Respondents were also asked to share their perceptions on students’ growth in
math and reading skills, and the ability of the program to address students’ knowledge
gaps (see Figure 7).
Figure 7
Teachers’ Perceived Impact of Imagine MyPath on Student Growth
Note. +<5%.
37.5%
29.2%
33.3%
29.2%
12.5%
12.5%
16.7%
20.8%
+
+
+
+
25.0%
25.0%
37.5%
37.5%
45.8%
50.0%
20.8%
25.0%
25.0%
29.2%
37.5%
33.3%
Accelerated students who
were behind in reading to
grade-level content by the end
of the school year. (n = 24)
Accelerated students who
were behind in math to grade-
level content by the end of the
school year. (n = 24)
Addressed students’
knowledge gaps in reading. (n
= 24)
Addressed students’
knowledge gaps in math. (n =
24)
Improved students’ math
skills. (n = 24)
Improved students’ reading
skills. (n = 24)
Disagree Somewhat Disagree Somewhat Agree Agree
The program...
© Johns Hopkins University, 2023
Teachers were asked specifically about how participating students engaged with
the Imagine MyPath program, with a focus on motivation and whether students were
appropriately challenged by the program. Specifically, teachers were asked to indicate
their level of agreement with the following statements. The majority of respondents felt
that the program appropriately challenged students (73.9%, n = 17), while more than
half (65.2%, n = 15) of respondents agreed or somewhat agreed that the program
content motivated students to persist through difficult content.
Respondents were also asked to share their perceptions on students’ growth in
math and reading skills, and the ability of the program to address students’ knowledge
gaps (see Figure 7).
Figure 7
Teachers’ Perceived Impact of Imagine MyPath on Student Growth
Note. +<5%.
37.5%
29.2%
33.3%
29.2%
12.5%
12.5%
16.7%
20.8%
+
+
+
+
25.0%
25.0%
37.5%
37.5%
45.8%
50.0%
20.8%
25.0%
25.0%
29.2%
37.5%
33.3%
Accelerated students who
were behind in reading to
grade-level content by the end
of the school year. (n = 24)
Accelerated students who
were behind in math to grade-
level content by the end of the
school year. (n = 24)
Addressed students’
knowledge gaps in reading. (n
= 24)
Addressed students’
knowledge gaps in math. (n =
24)
Improved students’ math
skills. (n = 24)
Improved students’ reading
skills. (n = 24)
Disagree Somewhat Disagree Somewhat Agree Agree
The program...
Imagine MyPath Moline-Coal Valley 16
© Johns Hopkins University, 2023
A large majority of respondents felt that students improved in math (83.3%,
n = 20) and reading (83.3%, n = 20) due to use of the Imagine MyPath program. A
majority of respondents also agreed or somewhat agreed with the program’s ability to
address students’ knowledge gaps in math (66.7%, n = 16) and reading (62.5%,
n = 15). Responses were less positive regarding the impact of the program on
accelerating the learning for students who were behind in grade-level content, as half
of teachers disagreed it accelerated students in math (50.0%, n = 12) and more than
half disagreed to some extent with the statement regarding reading acceleration
(54.2%, n = 13) by the end of the school year.
Overall Perceptions
Teachers were asked specifically about their satisfaction with the program and
whether they would use or recommend the program in the future using the following
statements. Most respondents were positive about the program and would like to use
Imagine MyPath again in the future, with 60.9% (n = 14) agreeing or somewhat
agreeing with the statement. Fewer (52.2%, n = 12) said they agreed or somewhat
agreed that they would recommend Imagine MyPath to other teachers.
Open-Ended Responses
Teacher impressions of the Imagine MyPath online educator platform
resources supportive of effective classroom instruction.
Educators provided open-ended responses to several questions related to their
satisfaction with the program and areas for improvement. They were first asked to
provide comments on the Imagine MyPath resources that they found to be most helpful
in enabling them to be effective in the classroom. Roughly one-quarter of the
participants responded (26.9%, n = 7,) to the query. Only two participants provided
information on helpful resources, namely the program worksheets and the student
progress page which the participant liked because it “showed at a glance how many
lessons were being passed/failed.” From the remaining five responses to the query, one
participant replied, “N/A,” another stated that they “didn't like [the resources] and went
out on my own to find better material,” and three explained that someone else from
their school “does” or “oversees” the utilization of Imagine MyPath resources. In
explanation, one stated, “Our computer specialist is the one who looks at the reports
and does all the online stuff. Teachers in our building do not alter the programs or look
at reports usually.”
Teacher recommendations for improving Imagine MyPath online educator
platform to better support teachers in the classroom.
© Johns Hopkins University, 2023
A large majority of respondents felt that students improved in math (83.3%,
n = 20) and reading (83.3%, n = 20) due to use of the Imagine MyPath program. A
majority of respondents also agreed or somewhat agreed with the program’s ability to
address students’ knowledge gaps in math (66.7%, n = 16) and reading (62.5%,
n = 15). Responses were less positive regarding the impact of the program on
accelerating the learning for students who were behind in grade-level content, as half
of teachers disagreed it accelerated students in math (50.0%, n = 12) and more than
half disagreed to some extent with the statement regarding reading acceleration
(54.2%, n = 13) by the end of the school year.
Overall Perceptions
Teachers were asked specifically about their satisfaction with the program and
whether they would use or recommend the program in the future using the following
statements. Most respondents were positive about the program and would like to use
Imagine MyPath again in the future, with 60.9% (n = 14) agreeing or somewhat
agreeing with the statement. Fewer (52.2%, n = 12) said they agreed or somewhat
agreed that they would recommend Imagine MyPath to other teachers.
Open-Ended Responses
Teacher impressions of the Imagine MyPath online educator platform
resources supportive of effective classroom instruction.
Educators provided open-ended responses to several questions related to their
satisfaction with the program and areas for improvement. They were first asked to
provide comments on the Imagine MyPath resources that they found to be most helpful
in enabling them to be effective in the classroom. Roughly one-quarter of the
participants responded (26.9%, n = 7,) to the query. Only two participants provided
information on helpful resources, namely the program worksheets and the student
progress page which the participant liked because it “showed at a glance how many
lessons were being passed/failed.” From the remaining five responses to the query, one
participant replied, “N/A,” another stated that they “didn't like [the resources] and went
out on my own to find better material,” and three explained that someone else from
their school “does” or “oversees” the utilization of Imagine MyPath resources. In
explanation, one stated, “Our computer specialist is the one who looks at the reports
and does all the online stuff. Teachers in our building do not alter the programs or look
at reports usually.”
Teacher recommendations for improving Imagine MyPath online educator
platform to better support teachers in the classroom.
Imagine MyPath Moline-Coal Valley 17
© Johns Hopkins University, 2023
Respondents were asked about recommendations to make the Imagine MyPath online
educator platform more effective for helping teachers in the classroom. Four participants
replied to this query (15.4%) with two answering, “N/A.” The remaining two participants
provided fairly detailed suggestions for how the platform might provide more options for
working with lessons and reports. Their comments were as follows:
Have the capability to reassign more than one lesson at a time. When
running reports, be able to sort on lessons passed or not passed or active
time. Have the capability to print multiple lesson resources at once
identified with student names on the papers as well as a teacher copy.
The program placed all students in their grade level track. While I think
this has the benefit of ensuring that high achieving students showed
mastery of all of these skills, my most advanced students then spent quite
a bit of time doing the mastery checks and it took much longer for them
to get to the content that fit their ability level.
Teacher impressions of student satisfaction with the Imagine MyPath
student platform.
Teachers were asked to comment on their perceptions of what components of
Imagine MyPath their students most enjoyed. Eleven participants (42.3%) responded to
the query with one recording “N/A” and another stating that “students don’t enjoy the
program.” The remaining participants’ responses fell equally into the following two
categories:
1. Program-generated rewards and feelings of accomplishment obtained
by completing program problems/lessons (n = 7). Teachers stated that
their students enjoyed receiving “stars” or “tickets” when they
completed work correctly, and one teacher recounted that students
liked “showing me the screen when they passed a lesson.”
2. Engaging program features including the songs, stories, and avatars
(n = 7).
Teacher suggestions for improving the Imagine MyPath student platform to
increase student learning.
Finally, teachers provided suggestions that they felt would help the Imagine
MyPath student platform to be more effective in increasing student learning. With nine
participants responding (34.6%), two replied, “N/A,” and two more responded either
“Nothing,” or “Not sure.” Of the others, several participants suggested that the program
difficulty level should be adjusted, noting that program content was “difficult, confusing
or difficult to understand,” particularly for students at the kindergarten level, who also
© Johns Hopkins University, 2023
Respondents were asked about recommendations to make the Imagine MyPath online
educator platform more effective for helping teachers in the classroom. Four participants
replied to this query (15.4%) with two answering, “N/A.” The remaining two participants
provided fairly detailed suggestions for how the platform might provide more options for
working with lessons and reports. Their comments were as follows:
Have the capability to reassign more than one lesson at a time. When
running reports, be able to sort on lessons passed or not passed or active
time. Have the capability to print multiple lesson resources at once
identified with student names on the papers as well as a teacher copy.
The program placed all students in their grade level track. While I think
this has the benefit of ensuring that high achieving students showed
mastery of all of these skills, my most advanced students then spent quite
a bit of time doing the mastery checks and it took much longer for them
to get to the content that fit their ability level.
Teacher impressions of student satisfaction with the Imagine MyPath
student platform.
Teachers were asked to comment on their perceptions of what components of
Imagine MyPath their students most enjoyed. Eleven participants (42.3%) responded to
the query with one recording “N/A” and another stating that “students don’t enjoy the
program.” The remaining participants’ responses fell equally into the following two
categories:
1. Program-generated rewards and feelings of accomplishment obtained
by completing program problems/lessons (n = 7). Teachers stated that
their students enjoyed receiving “stars” or “tickets” when they
completed work correctly, and one teacher recounted that students
liked “showing me the screen when they passed a lesson.”
2. Engaging program features including the songs, stories, and avatars
(n = 7).
Teacher suggestions for improving the Imagine MyPath student platform to
increase student learning.
Finally, teachers provided suggestions that they felt would help the Imagine
MyPath student platform to be more effective in increasing student learning. With nine
participants responding (34.6%), two replied, “N/A,” and two more responded either
“Nothing,” or “Not sure.” Of the others, several participants suggested that the program
difficulty level should be adjusted, noting that program content was “difficult, confusing
or difficult to understand,” particularly for students at the kindergarten level, who also
Imagine MyPath Moline-Coal Valley 18
© Johns Hopkins University, 2023
found the explanation videos to be over their heads. One teacher provided the following
example:
Students who don't know their numbers need more lessons on 1-5 before
counting to 100. It seemed some students were over their heads right off
the bat and just kept moving along without passing lessons, so a slower
approach would be better if they keep failing lessons.
Other than simplifying content, teachers also recommended that changes be
made to make the program more engaging for kindergarten students. Some suggested
adding more games, activities, or incentives for students. Individual teacher comments
suggested “better, more enticing and relatable videos pertaining to the subject matter”
or noted that students had trouble distinguishing between the sounds of different
letters when having to choose an answer.
Discussion
The current study was a mixed-methods evaluation designed to provide efficacy
evidence for Imagine Learning’s Imagine MyPath program, as well as data regarding
program implementation and teacher perceptions. Achievement impacts were
determined by comparing treatment students in kindergarten in the Moline-Coal Valley
School District to comparison students identified by NWEA’s Similar Schools Report who
did not use the program.
Results of the main impact analyses showed mixed patterns of results regarding
the efficacy of Imagine MyPath. A significant positive impact of Imagine MyPath on
student mathematics achievement gains was evidenced, with treatment students
outgaining virtual comparison students by slightly more than 2 points on the NWEA MAP
Growth Mathematics assessment. Perceptions of the Imagine MyPath program from
Moline-Coal Valley School District teachers were generally positive. Overall, the majority
of teachers agreed they would like to use the program again in the future and would
recommend it to others. Teachers were highly positive regarding the organization of the
program and its impact on student learning. Most teachers agreed that Imagine MyPath
helped students improve their reading and mathematics skills and that the program
addressed gaps in their knowledge about these subjects. They also largely agreed that
the program placed students at the appropriate level initially, challenged them
appropriately throughout the program, presented them with content appropriate to
their skill level, motivated them to persist through difficult content, and met the needs
of diverse learners.
Teacher perceptions of professional development relating to Imagine MyPath
were also positive, but more mixed. While teachers generally agreed that the
professional development enhanced their understanding of using data and Teacher
Resources to individualize instruction and review students’ Portfolios to inform
© Johns Hopkins University, 2023
found the explanation videos to be over their heads. One teacher provided the following
example:
Students who don't know their numbers need more lessons on 1-5 before
counting to 100. It seemed some students were over their heads right off
the bat and just kept moving along without passing lessons, so a slower
approach would be better if they keep failing lessons.
Other than simplifying content, teachers also recommended that changes be
made to make the program more engaging for kindergarten students. Some suggested
adding more games, activities, or incentives for students. Individual teacher comments
suggested “better, more enticing and relatable videos pertaining to the subject matter”
or noted that students had trouble distinguishing between the sounds of different
letters when having to choose an answer.
Discussion
The current study was a mixed-methods evaluation designed to provide efficacy
evidence for Imagine Learning’s Imagine MyPath program, as well as data regarding
program implementation and teacher perceptions. Achievement impacts were
determined by comparing treatment students in kindergarten in the Moline-Coal Valley
School District to comparison students identified by NWEA’s Similar Schools Report who
did not use the program.
Results of the main impact analyses showed mixed patterns of results regarding
the efficacy of Imagine MyPath. A significant positive impact of Imagine MyPath on
student mathematics achievement gains was evidenced, with treatment students
outgaining virtual comparison students by slightly more than 2 points on the NWEA MAP
Growth Mathematics assessment. Perceptions of the Imagine MyPath program from
Moline-Coal Valley School District teachers were generally positive. Overall, the majority
of teachers agreed they would like to use the program again in the future and would
recommend it to others. Teachers were highly positive regarding the organization of the
program and its impact on student learning. Most teachers agreed that Imagine MyPath
helped students improve their reading and mathematics skills and that the program
addressed gaps in their knowledge about these subjects. They also largely agreed that
the program placed students at the appropriate level initially, challenged them
appropriately throughout the program, presented them with content appropriate to
their skill level, motivated them to persist through difficult content, and met the needs
of diverse learners.
Teacher perceptions of professional development relating to Imagine MyPath
were also positive, but more mixed. While teachers generally agreed that the
professional development enhanced their understanding of using data and Teacher
Resources to individualize instruction and review students’ Portfolios to inform
Imagine MyPath Moline-Coal Valley 19
© Johns Hopkins University, 2023
instruction, fewer teachers felt it enhanced their understanding of grouping and
regrouping students based on their growth and progress. These findings were reflected
in many of the open-ended survey responses that highlighted teacher appreciation for
data use. Some respondents highlighted a need for greater training on Imagine MyPath
in general and how to use its resources, while others sought training in specific
technical components of the program and computer interface. These discrepancies may
be attributed to the mixed role of respondents and how they perceived the usefulness
of training: some were DLSAs who were directly responsible for student activity in
Imagine MyPath, while others were classroom teachers who were not.
It is important to note that teachers’ attitudes toward the Imagine MyPath
program were generally positive, especially as they related to student achievement and
engagement, as well as professional development. However, program implementers
may wish to reconsider and integrate more closely certain Imagine MyPath resources
which teachers reported low levels of use. These include the assignment builder,
teacher toolkits, and reports. In particular, all teachers reported either never using the
assignment builder or using it only on a monthly basis, while most teachers reported
never or monthly use of the teacher toolkit to individualize instruction and over half of
teachers reported never using Imagine MyPath reports to individualize instruction. In
addition, while responses were generally positive, teachers rated the ability of the
Imagine MyPath program to accelerate learning for those behind grade level and
motivate students to persist as lower than in other areas. Consideration may be taken
to attempt to address these issues in future versions of the Imagine MyPath program.
The above interpretations should be considered in view of several limitations of
the present study. Results are derived from data in one unique district; as such, we
caution against wider generalization of the results found here. Another limitation is
related to the nature of the comparison group. While NWEA matches students on the
basis of prior achievement, school type (urban, suburban, or rural), and school-level
FARMS percentage, it is possible that comparison students are still fundamentally
different from treatment students. Thus, while the comparison group is similar to the
treatment group in terms of baseline achievement and school-level characteristics, it is
still possible that important differences may exist between treatment and comparison
students, especially on individual-level variables. Further, given that the analyses were
conducted at the student level using NWEA comparison-student matches unique to the
district, the district constitutes a single context in which core curricula and other
potentially influential district conditions are confounded with Imagine MyPath. For
example, if a particular district were excelling or struggling with its reading and math
curricula or with retaining effective teachers in those subjects, the overall achievement
effects could reflect the latter factors much more than the impacts of a supplementary
program.
© Johns Hopkins University, 2023
instruction, fewer teachers felt it enhanced their understanding of grouping and
regrouping students based on their growth and progress. These findings were reflected
in many of the open-ended survey responses that highlighted teacher appreciation for
data use. Some respondents highlighted a need for greater training on Imagine MyPath
in general and how to use its resources, while others sought training in specific
technical components of the program and computer interface. These discrepancies may
be attributed to the mixed role of respondents and how they perceived the usefulness
of training: some were DLSAs who were directly responsible for student activity in
Imagine MyPath, while others were classroom teachers who were not.
It is important to note that teachers’ attitudes toward the Imagine MyPath
program were generally positive, especially as they related to student achievement and
engagement, as well as professional development. However, program implementers
may wish to reconsider and integrate more closely certain Imagine MyPath resources
which teachers reported low levels of use. These include the assignment builder,
teacher toolkits, and reports. In particular, all teachers reported either never using the
assignment builder or using it only on a monthly basis, while most teachers reported
never or monthly use of the teacher toolkit to individualize instruction and over half of
teachers reported never using Imagine MyPath reports to individualize instruction. In
addition, while responses were generally positive, teachers rated the ability of the
Imagine MyPath program to accelerate learning for those behind grade level and
motivate students to persist as lower than in other areas. Consideration may be taken
to attempt to address these issues in future versions of the Imagine MyPath program.
The above interpretations should be considered in view of several limitations of
the present study. Results are derived from data in one unique district; as such, we
caution against wider generalization of the results found here. Another limitation is
related to the nature of the comparison group. While NWEA matches students on the
basis of prior achievement, school type (urban, suburban, or rural), and school-level
FARMS percentage, it is possible that comparison students are still fundamentally
different from treatment students. Thus, while the comparison group is similar to the
treatment group in terms of baseline achievement and school-level characteristics, it is
still possible that important differences may exist between treatment and comparison
students, especially on individual-level variables. Further, given that the analyses were
conducted at the student level using NWEA comparison-student matches unique to the
district, the district constitutes a single context in which core curricula and other
potentially influential district conditions are confounded with Imagine MyPath. For
example, if a particular district were excelling or struggling with its reading and math
curricula or with retaining effective teachers in those subjects, the overall achievement
effects could reflect the latter factors much more than the impacts of a supplementary
program.
Imagine MyPath Moline-Coal Valley 20
© Johns Hopkins University, 2023
Appendix A: Imagine MyPath Teacher Survey
Imagine MyPath Teacher Survey
Background Information
Please indicate your school.
____________________________________________
What is your primary role?
• Classroom teacher
• Instructional Aide/Paraprofessional
• Other (please specify): __________________________________________________
What grade(s) do you teach? Select all that apply.
• Kindergarten
• First grade
• Second grade
• Third grade
• Fourth grade
• Fifth grade
• Other (Please specify):
__________________________________________________
About what percentage of students in your classroom are...? (Values must add up to 100%.)
• Below-grade level readers: _______
• On-grade level readers: _______
• Above-grade level readers: _______
• Unknown: _______
• Total: ________
Experience with Digital Learning
Did you use Imagine MyPath during the previous school year (2021-2022)?
• Yes
• No
© Johns Hopkins University, 2023
Appendix A: Imagine MyPath Teacher Survey
Imagine MyPath Teacher Survey
Background Information
Please indicate your school.
____________________________________________
What is your primary role?
• Classroom teacher
• Instructional Aide/Paraprofessional
• Other (please specify): __________________________________________________
What grade(s) do you teach? Select all that apply.
• Kindergarten
• First grade
• Second grade
• Third grade
• Fourth grade
• Fifth grade
• Other (Please specify):
__________________________________________________
About what percentage of students in your classroom are...? (Values must add up to 100%.)
• Below-grade level readers: _______
• On-grade level readers: _______
• Above-grade level readers: _______
• Unknown: _______
• Total: ________
Experience with Digital Learning
Did you use Imagine MyPath during the previous school year (2021-2022)?
• Yes
• No
Imagine MyPath Moline-Coal Valley 21
© Johns Hopkins University, 2023
Please list any additional digital learning tools you use to support students' literacy development
during the current school year (2022-23).
Please list any additional digital learning tools you use to support students' math development
during the current school year (2022-23).
How confident are you implementing technology in your classroom?
• Unconfident
• Somewhat unconfident
• Somewhat confident
• Confident
Program Implementation
The following questions ask about your implementation of Imagine MyPath during the 2022–
2023 school year.
How often did you implement Imagine MyPath?
• Never
• Monthly
• Weekly
• Daily
When implementing Imagine MyPath, how often did you do the following?
• Never
• Monthly
• Weekly
• Daily
Worked with individual students who were struggling.
Worked with individual students who were not struggling.
Worked with groups of students.
Used offline resources with students.
Motivated students with extra incentives (outside the program).
Were the additional incentives you put in place effective at encouraging students to pass
lessons?
• Yes
• No
© Johns Hopkins University, 2023
Please list any additional digital learning tools you use to support students' literacy development
during the current school year (2022-23).
Please list any additional digital learning tools you use to support students' math development
during the current school year (2022-23).
How confident are you implementing technology in your classroom?
• Unconfident
• Somewhat unconfident
• Somewhat confident
• Confident
Program Implementation
The following questions ask about your implementation of Imagine MyPath during the 2022–
2023 school year.
How often did you implement Imagine MyPath?
• Never
• Monthly
• Weekly
• Daily
When implementing Imagine MyPath, how often did you do the following?
• Never
• Monthly
• Weekly
• Daily
Worked with individual students who were struggling.
Worked with individual students who were not struggling.
Worked with groups of students.
Used offline resources with students.
Motivated students with extra incentives (outside the program).
Were the additional incentives you put in place effective at encouraging students to pass
lessons?
• Yes
• No
Imagine MyPath Moline-Coal Valley 22
© Johns Hopkins University, 2023
What types of incentives did you use? Briefly describe.
Were these incentives differentially effective for various groups of students (e.g., different
grades, genders)?
What percentage of students wore headphones while using Imagine MyPath?
• None
• Few
• Most
• All
How often did you do the following in Imagine MyPath?
• Never
• Monthly
• Weekly
• Daily
• N/A
Used the class reports to monitor usage, growth, and progress.
Used the student reports to monitor individuals’ usage, growth, and progress.
Used Imagine MyPath reports to individualize instruction.
Used the assignment builder.
Used the teacher toolkit to reteach lessons to students who needed it.
To what degree would you agree or disagree with the following statements about Imagine
MyPath?
• Disagree
• Somewhat disagree
• Somewhat agree
• Agree
• N/A
I found it easy to access reports that displayed students’ usage, growth, and progress.
I found it easy to read reports that displayed students’ usage, growth, and progress.
I found it easy to navigate the Imagine MyPath platform.
The professional development was essential for me to navigate the Imagine MyPath
platform with little frustration.
© Johns Hopkins University, 2023
What types of incentives did you use? Briefly describe.
Were these incentives differentially effective for various groups of students (e.g., different
grades, genders)?
What percentage of students wore headphones while using Imagine MyPath?
• None
• Few
• Most
• All
How often did you do the following in Imagine MyPath?
• Never
• Monthly
• Weekly
• Daily
• N/A
Used the class reports to monitor usage, growth, and progress.
Used the student reports to monitor individuals’ usage, growth, and progress.
Used Imagine MyPath reports to individualize instruction.
Used the assignment builder.
Used the teacher toolkit to reteach lessons to students who needed it.
To what degree would you agree or disagree with the following statements about Imagine
MyPath?
• Disagree
• Somewhat disagree
• Somewhat agree
• Agree
• N/A
I found it easy to access reports that displayed students’ usage, growth, and progress.
I found it easy to read reports that displayed students’ usage, growth, and progress.
I found it easy to navigate the Imagine MyPath platform.
The professional development was essential for me to navigate the Imagine MyPath
platform with little frustration.
Imagine MyPath Moline-Coal Valley 23
© Johns Hopkins University, 2023
To what degree would you agree or disagree with the following statements about Imagine
MyPath?
• Disagree
• Somewhat disagree
• Somewhat agree
• Agree
• N/A
The program initially placed students at a level appropriate for their skills.
The program appropriately adopted instruction for students based on their needs.
Throughout the school year, the program presented students with content appropriate for
their skill level.
The program moved students through material efficiently.
The program met the needs of diverse learners.
The program worked for students with few technical interruptions.
Professional Development
Please rate the extent to which you agree or disagree about whether Imagine MyPath
Professional Development enhanced your understanding of the following.
• Disagree
• Somewhat disagree
• Somewhat agree
• Agree
• N/A
Grouping and regrouping students based on growth and progress.
Using data to individualize instruction.
Using Teacher Resources to individualize and/or reteach concepts.
Reviewing students’ Portfolios to inform instruction.
Teacher Perceptions
Student Engagement
© Johns Hopkins University, 2023
To what degree would you agree or disagree with the following statements about Imagine
MyPath?
• Disagree
• Somewhat disagree
• Somewhat agree
• Agree
• N/A
The program initially placed students at a level appropriate for their skills.
The program appropriately adopted instruction for students based on their needs.
Throughout the school year, the program presented students with content appropriate for
their skill level.
The program moved students through material efficiently.
The program met the needs of diverse learners.
The program worked for students with few technical interruptions.
Professional Development
Please rate the extent to which you agree or disagree about whether Imagine MyPath
Professional Development enhanced your understanding of the following.
• Disagree
• Somewhat disagree
• Somewhat agree
• Agree
• N/A
Grouping and regrouping students based on growth and progress.
Using data to individualize instruction.
Using Teacher Resources to individualize and/or reteach concepts.
Reviewing students’ Portfolios to inform instruction.
Teacher Perceptions
Student Engagement
Imagine MyPath Moline-Coal Valley 24
© Johns Hopkins University, 2023
Please rate the extent to which you agree or disagree with each of the following statements
about student engagement with Imagine MyPath.
• Disagree
• Somewhat disagree
• Somewhat agree
• Agree
• N/A
Students were motivated to persist through difficult content in the program.
Students were appropriately challenged by the program.
Student Growth
Please rate the extent to which you agree or disagree with each of the following statements
about student growth with Imagine MyPath.
• Disagree
• Somewhat disagree
• Somewhat agree
• Agree
• N/A
The program improved students’ math skills.
The program improved students’ reading skills.
The program accelerated students who were behind in math to grade-level content by the
end of the school year.
The program accelerated students who were behind in reading to grade-level content by the
end of the school year.
The program addressed students’ knowledge gaps in math.
The program addressed students’ knowledge gaps in reading.
Teacher Satisfaction
Please rate the extent to which you agree or disagree with each of the following statements
about student engagement with Imagine MyPath.
• Disagree
• Somewhat disagree
• Somewhat agree
• Agree
• N/A
I would like to use the program in the future.
I would recommend the program to other teachers.
© Johns Hopkins University, 2023
Please rate the extent to which you agree or disagree with each of the following statements
about student engagement with Imagine MyPath.
• Disagree
• Somewhat disagree
• Somewhat agree
• Agree
• N/A
Students were motivated to persist through difficult content in the program.
Students were appropriately challenged by the program.
Student Growth
Please rate the extent to which you agree or disagree with each of the following statements
about student growth with Imagine MyPath.
• Disagree
• Somewhat disagree
• Somewhat agree
• Agree
• N/A
The program improved students’ math skills.
The program improved students’ reading skills.
The program accelerated students who were behind in math to grade-level content by the
end of the school year.
The program accelerated students who were behind in reading to grade-level content by the
end of the school year.
The program addressed students’ knowledge gaps in math.
The program addressed students’ knowledge gaps in reading.
Teacher Satisfaction
Please rate the extent to which you agree or disagree with each of the following statements
about student engagement with Imagine MyPath.
• Disagree
• Somewhat disagree
• Somewhat agree
• Agree
• N/A
I would like to use the program in the future.
I would recommend the program to other teachers.
Imagine MyPath Moline-Coal Valley 25
© Johns Hopkins University, 2023
Extended Response
Educator platform
What Imagine MyPath resources helped you be more effective in the classroom?
What could the Imagine MyPath online educator platform do differently to help you be more
effective in the classroom?
Student platform
What did students enjoy most about the Imagine MyPath student platform?
What could the Imagine MyPath student platform do differently to increase student learning?
© Johns Hopkins University, 2023
Extended Response
Educator platform
What Imagine MyPath resources helped you be more effective in the classroom?
What could the Imagine MyPath online educator platform do differently to help you be more
effective in the classroom?
Student platform
What did students enjoy most about the Imagine MyPath student platform?
What could the Imagine MyPath student platform do differently to increase student learning?