Online Credit Recovery Study: Effects on High School Students' Proximal and Distal Outcomes

Principal Investigator(s): View help for Principal Investigator(s) Jordan Rickles, American Institutes for Research; Sarah Peko-Spicer, American Institutes for Research; Kyle Neering, American Institutes for Research

Version: View help for Version V1

Name	File Type	Size	Last Modified
Online Credit Recovery Study - Public Data Release.zip	application/zip	1.5 MB	06/28/2024 10:27:AM

Project Citation:

Rickles, Jordan, Peko-Spicer, Sarah, and Neering, Kyle. Online Credit Recovery Study: Effects on High School Students’ Proximal and Distal Outcomes. Ann Arbor, MI: Inter-university Consortium for Political and Social Research [distributor], 2024-06-28. https://doi.org/10.3886/E202181V1

Project Description

Summary: The American Institutes for Research conducted a multisite randomized study that tested an online learning model for credit recovery at 24 high schools in Los Angeles, California in 2018 and 2019. The study focused on first-year high school students who failed Algebra 1 or English 9 (their ninth-grade English course) and retook the course during the summer before their second year of high school. Within each participating school, we used a lottery to determine whether each student was placed in either the school’s typical teacher-directed class (business-as-usual control condition) or a class that used an online learning model (treatment condition). For the online learning model, an online provider supplied the main course content, and the school provided a subject-appropriate, credentialed in-class teacher who could supplement the digital content with additional instruction.

The study compared outcomes of students assigned to the treatment condition to outcomes of students assigned to the control condition. Analyses focused both on proximal outcomes (ex: student course experiences, content knowledge, and credit recovery rates) and distal outcomes (ex: on-time graduation and cumulative credits earned by the end of the 4th year of high school). We estimated average treatment effects for the intent-to-treat sample using regression models that control for student characteristics and randomization blocks. We conducted separate analyses for students who failed Algebra 1 and students who failed at least one semester of their English 9 course.

This ICPSR data deposit includes our final analytical dataset and three supplemental files. Data come from three sources: (1) extant district data on student information and academic outcomes, (2) end-of-course surveys of students’ and teachers’ experiences, and (3) end-of-course test of students’ content knowledge. Data fields include:

Sample information: term, school (anonymized), teacher (anonymized), course, randomization block, student cohort, treatment status
Demographics: sex, race/ethnicity, National School Lunch Program status, inclusion in the Gifted/Talented program, Special Education status, and English language learner status
Pre-treatment information (treatment group only): 9th grade GPA, 9th grade attendance rate, number of 9th grade courses failed, 8th grade test scores
Online course engagement information: percentage of online course completed, average score on online activities, minutes spent in online platform
Student survey data: responses a survey administered at the end of the course for treatment and control students. Questions cover degree of student engagement with the course, perceptions of teacher support and course difficulty, and clarity of course expectations.
End-of-course test data: answers and scores on an end-of-course assessment administered to treatment and control students to evaluate content knowledge (Algebra 1 or English 9). The test did not count towards the final course grade and included 17-20 multiple choice questions.
Academic outcomes: grade in credit recovery course, credits attempted/earned in each year of high school, GPA in each year of high school, credits/GPA in math and ELA in each year of high school, indicator for on-time high school graduation, 10th grade PSAT scores
Teacher survey and logs: teacher-reported logs on the use of different instructional activities and responses to surveys about course pacing, content, goals, and degree of student support

Funding Sources: United States Department of Education. Institute of Education Sciences (R305A170152)

Scope of Project

Subject Terms: credit recovery; online courses; secondary education; high school students; randomized control trial

Geographic Coverage: Los Angeles, California

Time Period(s): 2018 – 2022 (Summer 2018 through Summer 2022)

Collection Date(s): 2018 – 2022 (Summer 2018 through Summer 2022))

Universe: Students in 24 Los Angeles high schools who entered 9th grade in the 2017-18 school year (Cohort 1) or the 2018-19 school year (Cohort 2), failed their Algebra 1 and/or at least one semester of their English 9 course, and enrolled in a credit recovery course the summer between their 9th and 10th grade year.

Data Type(s): administrative records data; survey data

Methodology

Response Rate: The response rate for the survey was 66% for Algebra 1 (60% for online classes and 73% for teacher-directed classes) and 59% for English 9 (57% for online classes and 61% for teacher-directed classes). The response rate for the test was 66% for Algebra 1 (59% for online classes and 72% for teacher-directed classes) and 59% for English 9 (57% for online classes and 60% for teacher-directed classes). For the PSAT outcome, Grade 10 students in the district are supposed to take the PSAT. Missing data for the PSAT measures could be due to non-response or students in the ITT sample leaving the district prior to the PSAT testing date. For the on-time high school graduation analysis, all students were coded as graduating from the district or not based on the district’s administrative data.

Data Source: Administrative records, surveys, study-administered test

Unit(s) of Observation: Student

Related Publications

Download this project

Published Versions

V1 [2024-06-28]

Export Metadata

Dublin Core

DDI 2.5

Report a Problem

Found a serious problem with the data, such as disclosure risk or copyrighted content? Let us know.

This material is distributed exactly as it arrived from the data depositor. ICPSR has not checked or processed this material. Users should consult the investigator(s) if further information is desired.