Sign Up For Newsletter

Blueprints For Healthy Youth Development logo

Early College High School Model

A high school model designed to increase students' access to a postsecondary credential, particularly for underrepresented students. The goal is to minimize challenges in the transition to postsecondary education for students for whom access has historically been problematic.

Program Outcomes

  • Academic Performance
  • Dropout/High School Graduation
  • Post Secondary Education

Program Type

  • Academic Services
  • School - Environmental Strategies
  • School - Individual Strategies

Program Setting

  • School

Continuum of Intervention

  • Universal Prevention

Age

  • Late Adolescence (15-18) - High School

Gender

  • Both

Race/Ethnicity

  • All

Endorsements

Blueprints: Model Plus

Program Information Contact

Julie Edmunds, Ph.D., Evaluator
Program Director for Secondary School Reform, SERVE Center
University of North Carolina at Greensboro
Email: jedmunds@serve.org
Phone: (336) 315-7415

Clarisse Haxton, Ph.D., Evaluator
Research, Evaluation, and Assessment (REA) Department
Palo Alto Unified School District
Email: chaxton@pausd.org
Phone: (650) 833-4229 ext. 6914


Brief Description of the Program

An Early College High School (EC) is a high school model that offers enrolled students an opportunity to earn an associate's degree or up to 2 years of college credits toward a bachelor's degree during high school at no or low cost to the students. Often referred to as "small schools that blur the line between high school and college" (Edmunds et al., 2017, p. 297), the model is designed to enable students to take college courses while still receiving support from high school staff. Many early college models target students who are traditionally underrepresented in postsecondary education, including minority students, students from low-income families, and students who are in the first generation of their families to go to college.

Outcomes

Study 1
Treatment students were more likely than control to: 

  • Enroll in college within 2 years (Haxton et al., 2016) and within 6 years (Song & Zeiser, 2019) of expected high school graduation.
  • Enroll in 2-year colleges within 2 years (Haxton et al., 2016) and within 6 years (Song & Zeiser, 2019) of expected high school graduation.
  • Earn a credential within 2 years (Haxton et al., 2016) and within 6 years (Song & Zeiser, 2019) of expected high school graduation.
  • Earn an associates or certificate within 6 years of expected high school graduation (Song & Zeiser, 2019).
  • Earn a bachelor's degree within 6 years of expected high school graduation (Song & Zeiser, 2019).

As for risk and protective factors, Haxton et al. (2016) found that, as compared to the control group, treatment students:

  • Were more likely to earn college credits in high school.
  • Report stronger college-going high school cultures and support from instructors while in high school.

Study 2
Edmunds et al. (2017) found that, as compared to the control group, treatment students were more likely to have:

  • Enrolled in postsecondary education (2 years after high school graduation in a typical time frame).
  • Attained a postsecondary degree (2 years after high school graduation in a typical time frame).

In terms of risk & protective factors, compared to the control group, students in the treatment group:

  • Took more core college preparatory courses and succeeded in them at the end of 9th grade (Edmunds et al., 2012).
  • Had better school attendance and fewer suspensions at the end of 9th grade (Edmunds et al., 2012).
  • Earned more college credits by the end of 12th grade (Edmunds et al., 2017).

Brief Evaluation Methodology

Study 1 (Haxton et al., 2016; Song & Zeiser, 2019) and Study 2 (Edmunds et al., 2012, 2017) each conducted a multisite randomized controlled trial (using lottery assignments) to evaluate the effects of the Early College High School model. Study 1 recruited from a five state sample (i.e., North Carolina, Ohio, South Carolina, Texas, and Utah) of 17 lotteries across 10 schools and 3 cohorts of students graduating from high school in 2005-06, 2006-07, and 2007-08, which resulted in a sample size of 2,458 students. Study 2 included students who applied to one of 12 early college schools in North Carolina, and included 18 cohorts of students who enrolled in ninth grade in the 2005-06, 2006-07, 2007-08 and 2008-09 school years (n =1,689). Haxton et al. (2016) followed participants from the year they entered Grade 9 (i.e., Year 1) to 2 years after expected high school graduation (i.e., Year 4). Song & Zeiser (2019) followed these same participants from the year they entered Grade 9 (i.e., Year 1) to 6 years after expected high school graduation (i.e., Year 10). Similar to Haxton et al. (2016), Study 2 (Edmunds et al., 2012, 2017) followed participants up to 6 years after students completed 9th grade (or 2 years after high school graduation in a typical time frame). Primary outcomes for both studies included high school graduation rates, as well as college enrollment and completion rates. Outcomes in Study 2 also included course taking patterns and success, attendance, suspension, and college aspirations at the end of 9th grade.

Study 1

Haxton, C., Song, M., Zeiser, K., Berger, A., Turk-Bicakci, L., Garet, M. S., . . . Hoshen, G. (2016). Longitudinal findings from the Early College High School Initiative Impact Study. Educational Evaluation and Policy Analysis, 38(2), 410-430.


Study 2

Edmunds, J. A., Unlu, F., Glennie, E., Bernstein, L., Fesler, L., Furey, J., & Arshavsky, N. (2017). Smoothing the transition to postsecondary education: The impact of the early college model. Journal of Research on Educational Effectiveness, 10(2), 297-325.


Protective Factors

School: Opportunities for prosocial involvement in education


* Risk/Protective Factor was significantly impacted by the program

See also: Early College High School Model Logic Model (PDF)

Race/Ethnicity/Gender Details

The treatment sample in Haxton et al. (2016) was majority (53%) minority and nearly half of the students (47%) were from low-income families. Findings showed that minority students in the treatment group were more likely to earn a postsecondary degree (of any type) than minority students in the control group. In addition, low-income students who attended an Early College High School were more likely to earn "any" postsecondary degree than low-income students in the control group. For Edmunds et al. (2017), the overall sample was around 40% minority, roughly 40% were first-generation college going, and half were low-income. Findings showed that under-represented minority students in the treatment earned more college credits while in high school, and were more likely to enroll in college and receive a postsecondary credential compared to minority students in the control group. These findings were consistent across other sub-groups, including low-income and first-generation students.

Program Benefits (per individual): $66,716
Program Costs (per individual): $4,034
Net Present Value (Benefits minus Costs, per individual): $62,682
Measured Risk (odds of a positive Net Present Value): 92%

Source: Washington State Institute for Public Policy
All benefit-cost ratios are the most recent estimates published by The Washington State Institute for Public Policy for Blueprint programs implemented in Washington State. These ratios are based on a) meta-analysis estimates of effect size and b) monetized benefits and calculated costs for programs as delivered in the State of Washington. Caution is recommended in applying these estimates of the benefit-cost ratio to any other state or local area. They are provided as an illustration of the benefit-cost ratio found in one specific state. When feasible, local costs and monetized benefits should be used to calculate expected local benefit-cost ratios. The formula for this calculation can be found on the WSIPP website.


No information is available


No information is available

Program Developer/Owner

Program Outcomes

  • Academic Performance
  • Dropout/High School Graduation
  • Post Secondary Education

Program Specifics

Program Type

  • Academic Services
  • School - Environmental Strategies
  • School - Individual Strategies

Program Setting

  • School

Continuum of Intervention

  • Universal Prevention

Program Goals

A high school model designed to increase students' access to a postsecondary credential, particularly for underrepresented students. The goal is to minimize challenges in the transition to postsecondary education for students for whom access has historically been problematic.

Population Demographics

High school students, including those traditionally underrepresented in postsecondary education

Target Population

Age

  • Late Adolescence (15-18) - High School

Gender

  • Both

Race/Ethnicity

  • All

Race/Ethnicity/Gender Details

The treatment sample in Haxton et al. (2016) was majority (53%) minority and nearly half of the students (47%) were from low-income families. Findings showed that minority students in the treatment group were more likely to earn a postsecondary degree (of any type) than minority students in the control group. In addition, low-income students who attended an Early College High School were more likely to earn "any" postsecondary degree than low-income students in the control group. For Edmunds et al. (2017), the overall sample was around 40% minority, roughly 40% were first-generation college going, and half were low-income. Findings showed that under-represented minority students in the treatment earned more college credits while in high school, and were more likely to enroll in college and receive a postsecondary credential compared to minority students in the control group. These findings were consistent across other sub-groups, including low-income and first-generation students.

Risk/Protective Factor Domain

  • School

Risk/Protective Factors

Risk Factors

Protective Factors

School: Opportunities for prosocial involvement in education


*Risk/Protective Factor was significantly impacted by the program

See also: Early College High School Model Logic Model (PDF)

Brief Description of the Program

An Early College High School (EC) is a high school model that offers enrolled students an opportunity to earn an associate's degree or up to 2 years of college credits toward a bachelor's degree during high school at no or low cost to the students. Often referred to as "small schools that blur the line between high school and college" (Edmunds et al., 2017, p. 297), the model is designed to enable students to take college courses while still receiving support from high school staff. Many early college models target students who are traditionally underrepresented in postsecondary education, including minority students, students from low-income families, and students who are in the first generation of their families to go to college.

Description of the Program

An Early College High School (EC) is a high school model that offers enrolled students an opportunity to earn an associate's degree or up to 2 years of transferable college credits toward a bachelor's degree during high school at no or low cost to the students. In early colleges, all students take a curriculum that includes the high school courses necessary for entrance into a four-year university (thus ensuring an academically rigorous course of study) and teachers are expected to receive support in implementing instructional strategies designed to prepare students for the level of thinking they will need to do in college. Often referred to as "small schools that blur the line between high school and college" (Edmunds et al., 2017, p. 297), the model is designed to enable students to take college courses while still receiving support from high school staff. Many early college models target students who are traditionally underrepresented in postsecondary education, including minority students, students from low-income families, and students who are in the first generation of their families to go to college. Thus, while some early colleges are structured as 4-year high schools, most allow students five years to complete the curriculum having recognized that students who are members of the target populations may not always be able to complete all of the necessary credits in only four years.

The early college is a comprehensive school reform model that focuses explicitly and purposefully on preparing all of its students for college. Core "design" principles include: 1) partnering with colleges and universities for enrolled high school students to take college courses; 2) providing opportunities to take college-level courses to all students, not only those who are academically advanced - with some models specifically focusing on dropouts or students at-risk of dropping out of high school; 3) giving students a wide variety of academic and social supports-from personalized relationships to academic tutoring, advising, and help with study skills, time management, self-advocacy, other college "life skills," and college preparation. In addition, early colleges provide students with supports in the formal transition to college, such as assistance in completing college applications and financial aid forms. Some early colleges also have other design principles for adults in the school (for example, professional development focused on a common vision and a collaborative, learning environment for staff).

According to this model, all students are required to take college courses. For most, this starts in the ninth grade when they might take physical education or college success skills, often in classes composed only of early college high school students. In 10th grade, most early college students begin to take core academic courses along with regular college students. By 11th and 12th grade, students take the majority of their courses on the college campus along with regular college students. Many early colleges are actually located on a 2-year or 4-year college campus. For Haxton et al. (2016) and Edmunds et al. (2017), all sites included schools of choice in which resident pupils applied to enroll in schools for which they were not zoned.

Theoretical Rationale

Among the characteristics cited in the literature associated with college access and success are academic preparation, early access to courses that carry college credit, a college-going culture, and assistance with logistical preparation. The model is expected to provide college exposure, rigorous academics, and student supports, which in turn is expected to promote improved high school outcomes, including high school achievement and graduation. Students' high school outcomes, including completing sufficient postsecondary credits while in high school, may lead them to engage in further college education or lead directly to college degree attainment. The framework also acknowledges that student background characteristics may affect student outcomes both during and after high school, and may also moderate the program effects on student outcomes.

Brief Evaluation Methodology

Study 1 (Haxton et al., 2016; Song & Zeiser, 2019) and Study 2 (Edmunds et al., 2012, 2017) each conducted a multisite randomized controlled trial (using lottery assignments) to evaluate the effects of the Early College High School model. Study 1 recruited from a five state sample (i.e., North Carolina, Ohio, South Carolina, Texas, and Utah) of 17 lotteries across 10 schools and 3 cohorts of students graduating from high school in 2005-06, 2006-07, and 2007-08, which resulted in a sample size of 2,458 students. Study 2 included students who applied to one of 12 early college schools in North Carolina, and included 18 cohorts of students who enrolled in ninth grade in the 2005-06, 2006-07, 2007-08 and 2008-09 school years (n =1,689). Haxton et al. (2016) followed participants from the year they entered Grade 9 (i.e., Year 1) to 2 years after expected high school graduation (i.e., Year 4). Song & Zeiser (2019) followed these same participants from the year they entered Grade 9 (i.e., Year 1) to 6 years after expected high school graduation (i.e., Year 10). Similar to Haxton et al. (2016), Study 2 (Edmunds et al., 2012, 2017) followed participants up to 6 years after students completed 9th grade (or 2 years after high school graduation in a typical time frame). Primary outcomes for both studies included high school graduation rates, as well as college enrollment and completion rates. Outcomes in Study 2 also included course taking patterns and success, attendance, suspension, and college aspirations at the end of 9th grade.

Outcomes (Brief, over all studies)

Study 1 (Haxton et al., 2016) found that, as compared to the control group, treatment students were more likely to be enrolled in college and attain a college degree while in high school. In addition, compared to the control group, treatment students were more likely to attain a postsecondary degree after high school. In a follow-up to Haxton et al. (2016), Song & Zeiser (2019) found treatment students were more likely than control to (1) enroll in college and (2) enroll in 2-year colleges each year between the fourth year of high school and 6 years after expected high school graduation. In addition, Song & Zeiser (2019) reported treatment students were more likely than control students to (1) earn a credential, (2) earn an associates or certificate, and (3) earn a bachelor's degree each year between the fourth year of high school and 6 years after expected high school graduation.

At the end of 9th grade, Study 2 (Edmunds et al., 2012) found that compared to the control group, students in the treatment group had significantly higher school attendance and lower suspension rates at the end of 9th grade. Study 2 found that, as compared to the control group, treatment students earned more college credits while in high school (Edmunds et al., 2012), and 2 years after high school graduation in a typical time frame, treatment students were more likely to be enrolled in postsecondary education and attain a postsecondary degree (Edmunds et al., 2017).

As for risk and protective factors, Study 1 (Haxton et al., 2016) found that, as compared to the control group, treatment students were more likely to earn college credits in high school, and report stronger college-going cultures and support from instructors in their high schools. Meanwhile, Study 2 (Edmunds et al., 2012) reported that a significantly higher proportion of students in the treatment group were taking core college preparatory courses and succeeding in them and had better school attendance and fewer suspensions at the end of 9th grade, compared to students in the control group. In addition, compared to control, treatment students earned more college credits by the end of 12th grade ((Edmunds et al., 2012).

Outcomes

Study 1
Treatment students were more likely than control to: 

  • Enroll in college within 2 years (Haxton et al., 2016) and within 6 years (Song & Zeiser, 2019) of expected high school graduation.
  • Enroll in 2-year colleges within 2 years (Haxton et al., 2016) and within 6 years (Song & Zeiser, 2019) of expected high school graduation.
  • Earn a credential within 2 years (Haxton et al., 2016) and within 6 years (Song & Zeiser, 2019) of expected high school graduation.
  • Earn an associates or certificate within 6 years of expected high school graduation (Song & Zeiser, 2019).
  • Earn a bachelor's degree within 6 years of expected high school graduation (Song & Zeiser, 2019).

As for risk and protective factors, Haxton et al. (2016) found that, as compared to the control group, treatment students:

  • Were more likely to earn college credits in high school.
  • Report stronger college-going high school cultures and support from instructors while in high school.

Study 2
Edmunds et al. (2017) found that, as compared to the control group, treatment students were more likely to have:

  • Enrolled in postsecondary education (2 years after high school graduation in a typical time frame).
  • Attained a postsecondary degree (2 years after high school graduation in a typical time frame).

In terms of risk & protective factors, compared to the control group, students in the treatment group:

  • Took more core college preparatory courses and succeeded in them at the end of 9th grade (Edmunds et al., 2012).
  • Had better school attendance and fewer suspensions at the end of 9th grade (Edmunds et al., 2012).
  • Earned more college credits by the end of 12th grade (Edmunds et al., 2017).

Mediating Effects

Song & Zeiser (2019) conducted a mediation analysis to examine the mechanisms of treatment impacts. Results show that students' high school experiences partially explained treatment impact on overall college enrollment, but they did not explain the treatment impact on enrollment in 2-year colleges. Meanwhile, accumulation of college credits during high school explained most of the treatment impact on bachelor's degree completion (even though the college credits accrued during high school were earned at a 2-year college for the majority of students).

Effect Size

For the original article comprising Study 1 (Haxton et al., 2016), effect sizes for behavioral outcomes ranged from small (OR = 1.13) to large (OR = 35.37). Effect sizes for risk and protective factors ranged from small (OR = 1.31) to large (OR = 18.61). Song & Zeiser (2019) computed effect sizes for binary outcomes by dividing the logged odds ratio of each outcome (reported in Appendix F, p. 65 and p. 68) by 1.65 (i.e., the Cox index). Effect sizes ranged from small (.157) to large (.725). In Study 2 (Edmunds et al., 2017), effect sizes were not reported.

Potential Limitations

Study 1: Haxton et al. (2016), Song & Zeiser (2019)

  1. Baseline difference on 1 of 6 variables in favor of treatment, though controlled for in analysis (Haxton et al., 2016). However, Exhibit 3 (p. 9) of Song & Zeiser, 2019) shows no significant differences between groups at baseline. It is unclear why the authors report baseline equivalence differently across publications.
  2. High attrition on 1 (high school graduation) of 3 outcomes but differential attrition not tested (Haxton et al., 2016).
  3. Missing baseline covariates were handled using multiple imputation and doing the analysis without baseline covariates and therefore without the imputation gave slightly weaker results for 1 (i.e., impact on bachelor's degree) of 5 significant outcomes (Song & Zeiser, 2019).

Study 2: Edmunds et al. (2012, 2017)

  • Baseline differences (in favor of treatment) on 2 of 13 variables, though controlled for in analysis.
  • High attrition on 1 (college credits received) of 3 primary outcomes but differential attrition not tested.

Notes

Berger et al. (2013, 2014) are full reports written by the American Institutes for Research (AIR) and Haxton et al. (2016) is a peer-reviewed publication. All three articles are the same report, just published in different formats. This write-up refers to Haxton et al. (2016), which is the same as Berger et al. (2013, 2014).

Endorsements

Blueprints: Model Plus

Program Information Contact

Julie Edmunds, Ph.D., Evaluator
Program Director for Secondary School Reform, SERVE Center
University of North Carolina at Greensboro
Email: jedmunds@serve.org
Phone: (336) 315-7415

Clarisse Haxton, Ph.D., Evaluator
Research, Evaluation, and Assessment (REA) Department
Palo Alto Unified School District
Email: chaxton@pausd.org
Phone: (650) 833-4229 ext. 6914

References

Study 1

Berger, A., Turk-Bicakci, L, Garet, M, Song, M., Knudson, J., Haxton, C., . . . Cassidy, L. (2013). Early College, Early Success: Early College High School Initiative Impact Study. Washington, DC: American Institutes for Research.

Berger, A., Turk-Bicakci, L., Garet, M., Knudson, J., & Hoshen, G. (2014). Early College, Continued Success: Early College High School Initiative Impact Study. Washington, DC: American Institutes for Research.

Certified Haxton, C., Song, M., Zeiser, K., Berger, A., Turk-Bicakci, L., Garet, M. S., . . . Hoshen, G. (2016). Longitudinal findings from the Early College High School Initiative Impact Study. Educational Evaluation and Policy Analysis, 38(2), 410-430.

Song, M., & Zeiser, K. (2019). Early college, continued success: Longer-term impact of early college high schools. Washington, DC: American Institutes for Research.

Study 2

Edmunds, J. A., Bernstein, L., Unlu, F., Glennie, E., Willse, J., Smith, A. & Arshavsky, N. (2012). Expanding the start of the college pipeline: Ninth-grade findings from an experimental study of the impact of the Early College High School Model. Journal of Research on Educational Effectiveness, 5(2), 136-159.

Certified Edmunds, J. A., Unlu, F., Glennie, E., Bernstein, L., Fesler, L., Furey, J., & Arshavsky, N. (2017). Smoothing the transition to postsecondary education: The impact of the early college model. Journal of Research on Educational Effectiveness, 10(2), 297-325.

Study 1

Evaluation Methodology

Design:

Recruitment: To be eligible for this retrospective study, an EC (Early College high school) had to meet the following criteria for at least 1 of 3 school years (2005-2006 through 2007-2008): (a) enrolled students in Grades 9 to 12; (b) had high school graduates; (c) were oversubscribed and used lotteries in their admission processes for incoming ninth graders; (d) retained the lottery records; and (e) implemented the EC model as a whole-school program. The potential study sample was restricted to all Early College High Schools that were open by fall 2007 to ensure that students in the study would have had the opportunity to complete at least 2 years of college by the end of data collection (e.g., 2012-2013). Of the 154 Early College High Schools open nationwide by fall 2007, about two-thirds were not eligible for the study because they were undersubscribed. Of the remaining models, 10 met the criteria for inclusion in this study.

Assignment: This study capitalized on retrospective admission lotteries that occurred between 2005-06 and 2007-08. The sample included 17 lotteries across 10 sites and 3 cohorts, with 1,044 students randomly assigned to treatment and 1,414 students randomly assigned to control (total n = 2,458). Treatment students were lottery applicants offered enrollment either through the initial lottery or from a randomized waitlist prior to the first day of school, and control students were lottery applicants who were not offered enrollment. Two schools in the first two cohorts conducted the lottery themselves.

Attrition: Only 2,207 students had high school graduation data (an attrition rate of 10%), and the reasons were not provided (See Table 3; Haxton et al., 2016). The authors reported no attrition (n = 2,458 at both the 2- and 6-year follow-up) for the postsecondary outcomes (i.e., college entrance and completion). In terms of risk & protective factors, Haxton et al. (2016) did not survey students from Cohort 1 because the authors were not confident in students' ability to recall their high school experiences multiple years after leaving high school. The survey response rate was 94% for treatment students and 88% for control students. Non-response-adjusted survey weights were applied to all analyses of survey data.

Sample: The 10 Early College high school were located in five states (North Carolina, Ohio, South Carolina, Texas, and Utah); five schools were located in urban areas, two in midsized cities, and three in small towns. Nine schools opened as new schools, and one was an existing school that became an Early College high school. Eight of the treatment schools had partnerships with 2-year colleges, and the other two had partnerships with 4-year colleges. All 10 schools were small schools (i.e., fewer than 150 students per grade), with an average enrollment of 290 students. Across the 10 schools, 49% of the students were non-White (ranging from 12% to 100%), and 44% were eligible for free or reduced-price lunch (ranging from 9% to 99%). The authors reported student characteristics by condition. Among the treatment group, 51.8% were female; 52.5% were minority; 21.6% were first-generation college going; and 47.3% were low-income students. Among the control group, 52.9% were female; 53.7% were minority; 19.9% were first-generation college going; and 45.2% were low-income.

Measures: The original study (Haxton et al., 2016) gathered administrative data from Early College High Schools, districts, and state departments of education on high school outcomes. The specific data sources for these variables differed by site, and in some sites, they were able to obtain data for the same measure from multiple sources. For high school graduation, the percentage of students earning a high school diploma or general equivalency diploma (GED) was measured.

Data for postsecondary enrollment and degree attainment came from the National Student Clearinghouse (NSC), which collects data from higher education institutions across the country.

  • For "college enrollment" and "college degree attainment", whether students enrolled in college or earned any degree by the end of study data collection was examined, including either during or after high school.
  • "College degree" meant any postsecondary credential, including certificates, associate's degrees, or bachelor's degrees.

For the original article (Haxton et al., 2016), college enrollment and degree attainment outcomes were measured in three ways:

  1. Outcomes at any point in the study period;
  2. Cumulative outcomes by Year 4, Year 5, and Year 6; and
  3. Outcomes after high school (after Year 4).

Year 4 reflected the period when students would traditionally be in their final year of high school. For students on a traditional trajectory, Year 5 was the year immediately following high school graduation. Year 6 (i.e., 2 years after high school graduation in a typical time frame) was the last year for which the authors had data for all students in the study. The study analyzed college outcomes at different points in time to parse out the timing of the effects, and thus examine whether the treatment impact was concentrated in the high school years or whether the impact persisted after high school.

Both articles (Haxton et al., 2016; Song & Zeiser, 2019) divided postsecondary outcomes into two domains with three primary outcomes in each domain:

  1. Outcomes in the college enrollment domain.
    1. Enrolled in college within 2 years (Haxton et al., 2016) and within 6 years (Song & Zeiser, 2019) after expected high school graduation.
    2. Enrolled in a 2-year college within 2 years (Haxton et al., 2016) and within 6 years (Song & Zeiser, 2019) after expected high school graduation.
    3. Enrolled in a 4-year college within 2 years (Haxton et al., 2016) and within 6 years (Song & Zeiser, 2019) after expected high school graduation. Note - Song & Zeiser (2019) also examined enrollment in a selective 4-year college using information from Barron's Profiles of American Colleges.
  2. Outcomes in the degree attainment domain.
    1. Completed any postsecondary degree within 2 years (Haxton et al., 2016) and within 6 years (Song & Zeiser, 2019) after expected high school graduation.
    2. Completed an associate's degree or certificate within 2 years (Haxton et al., 2016) and within 6 years (Song & Zeiser, 2019) after expected high school graduation.
    3. Completed a bachelor's degree within 2 years (Haxton et al., 2016) and within 6 years (Song & Zeiser, 2019) after expected high school graduation.

Haxton et al. (2016) also collected data on students' high school experiences through a student survey. The survey was administered in 2012-after their expected high school graduation-to 1,416 randomly selected students in the two oldest cohorts. Students' experiences during high school were measured through college exposure and student supports. For college exposure, students' college course-taking and credit accumulation as well as their Advanced Placement (AP) course-taking and exam passage in high school were measured. For student supports, the college-going culture in the school (a 1 to 4 scale based on three survey items, α reliability = .80), instructor support (a 1 to 4 scale based on six items, α reliability = .88) and whether students had access to general information about college were measured.

Analysis:

To estimate the overall program effects across lotteries, authors constructed a two-level model that took into account the clustering of students within lotteries. The treatment indicator was group-mean centered at the student level to make sure the comparisons of treatment students and control students were made within, rather than across, lotteries and thus produced unbiased estimates. The authors modeled the intercept as a random effect to take into account the clustering of student outcomes within lotteries and modeled the treatment effect as fixed at the lottery level because the number of lotteries in the study was too small to generate stable estimates of the variation in treatment effects across lotteries. Across all analyses, all models included the following student background characteristics as covariates: gender, race and ethnicity, first-generation college-going status, low-income status, and academic achievement prior to high school.

To guard against the risk for inflated Type I error due to multiple comparisons, Song & Zeiser (2019) designated a single outcome in each outcome domain (i.e., ever enrolled in college and ever completed a postsecondary degree or certificate) as the "confirmatory" outcome, and their main conclusion about the treatment effect in each domain is based on the confirmatory outcome. To check the robustness of findings to potentially increased Type I error due to multiple comparisons, the authors also used the Benjamini-Hochberg method to correct for multiple comparisons within each outcome domain. Because there are three primary outcome measures within each outcome domain, Song & Zeiser (2019) corrected for three comparisons within each domain. There is, however, no mention of confirmatory outcomes or of corrections for multiple comparisons in Haxton et al. (2016).

In addition to main effects, Haxton et al. (2016) also conducted moderation analyses on student subgroups (e.g., gender, minority status, first generation college-going status, low-income status, or level of prior mathematics and ELA achievement) for the three key outcomes: 1) high school graduation, 2) college enrollment, and 3) college degree attainment. Moderation analyses in Song & Zeiser (2019) examined the impact on college enrollment and completion of treatment interacted with gender, race/ethnicity, income status and level of prior mathematics and ELA achievement.

Intent-to-Treat: All participants were analyzed according to the condition in which they were assigned and all available data were utilized in the analysis which is in line with intent-to-treat protocol. The primary impact analyses used multiple imputation for missing covariates, and included all background and outcome variables available from both the extant data and the student survey data. Researchers generated 10 imputed data sets, conducted all analyses using each imputed data set separately, and then combined estimates across the 10 data sets, taking into account the uncertainty in imputed values both within and across the imputed data sets. Administrative records assessing postsecondary outcomes came from the National Student Clearinghouse (NSC), which covers more than 98% of all student enrollments in public and private colleges and universities. The authors assumed that students for whom the NSC could not find matching records did not attend college or attain a postsecondary degree; however students with missing outcome data were excluded from the analysis.

Outcomes

Implementation Fidelity: Not reported.

Baseline Equivalence: Baseline equivalence was tested on a variety of background characteristics including gender, minority status (i.e., non-White), eligibility for free or reduced-price lunch, first-generation status, and English language arts (ELA) and mathematics test scores prior to high school, and there was a significant difference in English language arts scores in favor of treatment. This difference was not reported as significant (reported as p = .068) in Song and Zeiser (2019). It may be the case that this was caused by recalculating imputed data (which they did for missing covariate data), but it is ultimately unclear what caused this inconsistency.

Differential Attrition: There was high attrition (10%) for high school graduation rates (Haxton et al., 2016), but tests for differential attrition were not conducted. However, the authors reported no attrition on the postsecondary outcome measures measured in Haxton et al. (2016) and Song & Zeiser (2019).

Posttest: Haxton et al. (2016) reported no significant difference between treatment and control in high school graduation rates.

As for risk and protective factors, Haxton et al. (2016) found that, as compared to the control group, treatment students were more likely to:

  • earn "any" college credits (OR = 8.04) and 1 year of college credit (OR = 18.61) in high school,
  • report stronger college-going cultures (ES = .32) and support from instructors (ES = .32) in their high schools.
  • have access to general college information in school than the control group (OR = 1.31), though this effect was marginally significant (p = .06). I

In contrast, students in the treatment group, compared to the control group, were less likely to take at least one Advanced Placement (AP) course (OR = 0.23) and pass at least one AP exam (OR = 0.19).

Long-Term: Haxton et al. (2016) reported positive long-term impacts for the overall sample on three of six outcomes. In terms of college access, within two years of expected high school graduation, treatment students were more likely to:

  1. Enroll in college
    1. OR = 1.63
    2. Marginal effects: 80.9% of treatment students had at least one record of college enrollment, whereas roughly 9 percentage points higher than the 72.2% college enrollment rate for control students.
  2. Enroll in a 2-year college
    1. OR = 2.33
    2. Marginal effects: 60.8% of treatment students had enrolled in a 2-year college compared to 40% of control students.
  3. Attain a postsecondary degree
    1. OR = 6.71
    2. Marginal effects: 24.9% of treatment students earned postsecondary degrees compared with 4.7% of control students.

Haxton et al. (2016) also found impacts in favor of treatment were detected for college enrollment by Year 4 (OR = 5.41), Year 5 (OR = 1.72), and Year 6 (OR = 1.73).

When examining enrollment by school type (2 vs. 4-year colleges), compared to the control group, students in the treatment group were more likely to enroll in a:

  • 2-year college by Year 4 (OR = 6.82), Year 5 (OR = 2.85), and Year 6 (OR = 2.46)
  • 4-year college by Year 4 (OR = 2.92) and Year 6 (OR = 1.32).

In terms of college attainment, compared to control, students in the treatment group were more likely to have earned:  

  • "any degree" by Year 4 (OR = 35.37), Year 5 (OR = 21.38) and Year 6 (OR = 14.66).
  • an Associate's degree (OR = 12.14) or a Bachelor's degree by (OR = 3.80) by Year 4

Song & Zeiser (2019) reported significant, positive effects on five of six outcomes measured six years after expected high school graduation. Note: Song & Zeiser (2019) computed effect sizes for binary outcomes by dividing the logged odds ratio of each outcome (reported in Appendix F, p. 65 and p. 68) by 1.65 (i.e., the Cox index). Each year between the fourth year of high school and 6 years after expected high school graduation, findings show treatment students were more likely than control to:

  1. Enroll in college
    1. ES = .281
    2. Marginal effects: 84.2% of treatment students enrolled in college, roughly 7 percentage points higher than the college enrollment rate of 77% for control students
  2. Enroll in 2-year colleges
    1. ES = .474
    2. Marginal effects: 65.8% for treatment students compared to 46.8% for control students
  3. Earn a credential
    1. ES = .305
    2. Marginal effects: 45.4% of treatment students completed postsecondary degrees, compared with 33.5% of control students
  4. Earn an associates or certificate
    1. ES = .725
    2. Marginal effects: 29.3% of treatment students compared with 11.1% of control students)
  5. Earn a bachelor's degree
    1. ES = .157
    2. Marginal effects: 30.1% of treatment students compared with 24.9% of control students) each year between the fourth year of high school and 6 years after expected high school graduation

Note: the authors re-estimated treatment impacts after removing the one treatment school where student demographic and prior achievement data had to be imputed for all treatment and control students because their state did not allow researchers to link student background data to outcome data. Results from this set of analyses mirror the ITT results presented in the main text with one exception: the treatment impact on bachelor's degree completion within 6 years after expected high school graduation (i.e., by Year 10) was slightly smaller and only marginally significant (p = .096) after removing one site where background data were imputed for all students.

By the end of the sixth year after expected high school graduation, the two groups of students did not significantly differ in the rate of enrolling in

  • 4-year colleges (57.6% of EC students compared with 56.7% of control students)
  • Selective 4-year colleges (by the end of Year 10, approximately 18% of both treatment and control students had enrolled in selective 4-year colleges).

Regarding the impact over time,

  • For enrollment in any type of postsecondary institutions, treatment had a significant impact on enrollment rates by the end of each year between Year 4 (i.e., the fourth year of high school) and Year 10 (i.e., 6 years after expected high school graduation), although the size of the impact tended to decrease over time (Song & Zeiser, 2019).
  • This finding was consistent with enrollment in 2-year colleges.
  • Similarly, while the difference in degree completion rate between the treatment and control students remained statistically significant over time, the size of the difference decreased from a maximum of 22.5 percentage points (by the end of Year 6 and Year 7) to 11.9 percentage points (by the end of Year 10).
  • However, regarding the completion of an associate's degree or certificate, the results show that the differences between treatment and control students were largely stable in both size and statistical significance over time.
  • Where bachelor's degree completion is concerned, the difference between the two study groups first widened and then narrowed (but remained statistically significant) over the period examined.

In terms of subgroup analyses,

  • The program effect on college degree attainment (in favor of the treatment group) within two years of expected high school graduation was significantly stronger for minority students and low-income students, and students who entered high school with higher math and ELA scores (Haxton et al., 2016).
  • Song & Zeiser (2019) found no subgroup effects in terms of family background characteristics within 6 years after expected high school graduation.
  • Treatment impacts on enrollment in 2-year colleges and completion of an associate's degree or certificate within 6 years after expected high school graduation (i.e., by Year 10) were stronger for students with higher levels of Grade 8 achievement (Song & Zeiser, 2019).

Study 2

Evaluation Methodology

Design:

Recruitment: The schools recruited for this study were located in rural and urban settings in all regions of North Carolina. The authors reported that while demographically similar to traditional schools, the early colleges included in this study were smaller than the traditional schools in their counties, had much lower enrollments of students with disabilities, and enrolled students with higher initial levels of achievement. To participate in the study, schools had to have more applicants than available slots and had to agree to use a lottery to randomly assign students. Schools entered the study on a rolling basis, and as long as they continued to use the lottery, they could continue to contribute cohorts of students to the study.

Assignment: Edmunds et al. (2017) reported a total of 1,689 students who applied to 12 different early colleges across North Carolina and enrolled in ninth grade in the 2005-2006, 2006-2007, 2007-2008, and 2008-2009 school years were enrolled in the lottery (though Edmunds et al., 2012, reported 1,607 students were in the lottery, so the exact sample size is unclear). These 12 schools enrolled a total of 18 cohorts of students, with 5 schools enrolling multiple cohorts. Beginning with the 2007-2008 cohort, the research team began conducting the lotteries (though 2 schools in the first cohort conducted their own lottery). Lottery assignments operated as follows. Each applicant who met the school's eligibility criteria was assigned a random number and the list of students was ordered from lowest to highest, with the lowest numbers being selected into the early college until all available slots were filled. As a result, 953 students were assigned to treatment and 736 were assigned to the control condition (see Figure 2).

Attrition: Using 1,607 as the randomized sample, Edmunds et al. (2012) reported attrition rates of 18% (1315/1607) for algebra, and less than 1% for college prep math courses (1607/1607), college prep English courses (1607/1607), and absences/suspensions/plans to attend a 4-year college (1604/1607). Meanwhile, Edmunds et al. (2017) reported, "Because we use extant administrative data, we are able to include almost all students from the original lottery samples in our analyses" (p. 304). Overall attrition rates, however, varied according to the outcome measures. For enrollment in postsecondary education and attainment of a postsecondary credential, overall attrition rates were 2% (1651/1689). For high school graduation, it was 6% (1594/1689), and for college credits attained in high school, it was 15% (1437/1689).

Sample: The overall sample was 41% male, and the racial/ethnic composition of the students was 60.2% white, 26.7% black, and 8.3% Hispanic. Forty-one percent of the students were first-generation college going and 50.7% were eligible for free or reduced lunch. Participating schools (n=12) served about 142 students (on average); and had a teacher turnover rate of 17% and a novice teacher rate of 31.2%.

Measures:

Edmunds (2012) used administrative data to measure college-prep math courses taken in 9th grade, defined as taking/passing Algebra I, Geometry, or Algebra II. Administrative records were also used to measure 9th grade school attendance and the percentage of students who had been suspended at least once in 9th grade. Students' plans to attend a 4-year college were assessed using data from a survey that accompanied each state-mandated end-of-course exam during the period of the study.

Edmunds et al. (2017) used administrative data collected by 3 primary sources: 1) the North Carolina Department of Public Instruction, 2) the National Student Clearinghouse, and 3) the North Carolina Community College System.

  • College credits earned while in high school: The number of college credits transferable to a 4-year college earned though the end of 12th grade was measured. Vocational courses and remedial or developmental courses were excluded as these credits were not transferable to a 4-year college, and "passing" was defined as earning a C or above. Data came from the North Carolina Community College System, and thus excluded 36 students who were enrolled in 4-year colleges while in high school. As for AP exam, data were not available before 2009-2010. However, the authors report sensitivity analyses that excluded 248 students who would have been in 11th and 12th grades (the years in which most students take AP classes) prior to 2009-2010 showed very little difference.
  • Graduation from high school: The authors reported 5-year graduation rates (for both treatment and control) because the majority of early colleges are 5 year programs.
  • Enrollment in postsecondary education: The authors reported whether a student was ever enrolled in any type of postsecondary education (part-time or full-time). This enrollment could have occurred at any point over the time period from 9th grade through the fall semester of the 6th year after the student started high school.
  • Postsecondary credentials: Any type of postsecondary credential, including associate degree, technical credential, or bachelor's degree, was assessed. The sample only went through the sixth year after students' entrance into high school (or two years after the student's expected graduation from high school), allowing two years for students in two-year institutions to complete their degrees.

Analysis: Edmunds et al. (2012) conducted an ITT analysis. Primary impact estimates were obtained from a multivariate linear regression model, with a fixed Treatment X Block (school-level) interaction term. Authors calculated an overall ITT impact estimate by averaging these block-specific effects, weighting them proportionally to the total number of students (treatment and control) in each block. In addition, several sensitivity and specification tests were run (none of which yielded results substantively different from those yielded by the primary analytic strategy). Authors adjusted for multiple comparisons for all core outcomes assessing main effects using the Benjamini-Hochberg multiple comparisons correction.

Edmunds et al. (2017) used multivariate linear regression models that included lottery indicators (or lottery fixed effects), a treatment indicator capturing the initial group to which a student was randomly assigned, and baseline student characteristics including demographic characteristics such as gender, race/ethnicity, age, free/reduced price lunch status, whether a student was retained prior to eighth grade, and eighth-grade academic performance. The statistical inference took into account clustering of students within schools by calculating cluster-robust standard errors. The models were estimated using weights that were based on students' probability of being selected into the early college. This was done because some schools conducted a stratified lottery, which led to different probabilities of selection into the treatment condition. Similar models were run on sub-group analyses, which examine results by race/ethnicity, income, and first-generation status.

In addition to the ITT analysis, Edmunds et al. (2012) presented instrumental variable estimates to represent the average effect of attending an early college (local average treatment effect [LATE]).

Intent-to-Treat: All participants were analyzed according to the condition in which they were assigned and all available data were utilized in the analysis, which is in line with intent-to-treat protocol. The authors also stated they conducted an ITT analysis.

Outcomes

Implementation Fidelity: Not reported.

Baseline Equivalence: Out of 13 baseline variables, including 9 socio-demographic (Black, Hispanic, White; gender; first-generation college, free/reduced price lunch eligibility; disabled/impaired, gifted, retained) and 4 pretest scores (8th grade reading and math scores; percent passed reading and math in 8th grade), 2 significant differences were detected between treatment and control at baseline - being retained in elementary or middle school (effect size = 0.37) and passing the eighth-grade math exam (effect size = 0.16), both in favor of the treatment group.

Differential Attrition: Not conducted. Attrition was >5% for 1 (college credits received) of the 3 outcomes; otherwise attrition was low.

Posttest: In the ITT analysis, Edmunds et al. (2012) found that compared to the control group, students in the treatment group had significantly higher school attendance and lower suspension rates at the end of 9th grade. Though the LATE estimates were similar and presented in Table 3, no significance levels were reported. Edmunds et al. (2017) found that treatment students earned significantly more college credits than control students while in high school. That is, by the end of 12th grade, treatment students had earned an average of 21.6 transferable college credits compared to an average of 2.8 credits earned by the control group. Edmunds et al. (2017) reported no impact on five-year high school graduation rates

Long-Term: At the posttest (2 years after high school graduation in a typical time frame), treatment students were more likely to be enrolled in postsecondary education than control students. Specifically, by the beginning of the 6th year after entering 9th grade, 89.9% of the treatment group had enrolled in postsecondary education at least once (including enrollment while in high school), compared to 74.3% of the control group. In addition, more treatment students attained some sort of postsecondary credential compared to the control group; 30.1% of treatment group students, compared to 4.2% of the control group, had attained a postsecondary credential.

Contact

Blueprints for Healthy Youth Development
University of Colorado Boulder
Institute of Behavioral Science
UCB 483, Boulder, CO 80309

Email: blueprints@colorado.edu

Sign up for Newsletter

If you are interested in staying connected with the work conducted by Blueprints, please share your email to receive quarterly updates.

Blueprints for Healthy Youth Development is
currently funded by Arnold Ventures (formerly the Laura and John Arnold Foundation) and historically has received funding from the Annie E. Casey Foundation and the Office of Juvenile Justice and Delinquency Prevention.