Print Page
Blueprints logo

Career Academies

School-based programs designed to provide students with the credentials and skills needed to make successful transitions to post-secondary education and careers.

Fact Sheet

Program Outcomes

  • Academic Performance
  • Dropout/High School Graduation
  • Employment
  • Post Secondary Education
  • Truancy - School Attendance

Program Type

  • Academic Services
  • Employment - Vocational
  • School - Environmental Strategies
  • Truancy Prevention

Program Setting

  • School

Continuum of Intervention

  • Selective Prevention
  • Universal Prevention

Age

  • Late Adolescence (15-18) - High School

Gender

  • Both

Race/Ethnicity

  • All

Endorsements

Blueprints: Promising
Crime Solutions: Effective
OJJDP Model Programs: Effective
Social Programs that Work:Top Tier
What Works Clearinghouse: Meets Standards Without Reservations - Positive Effect

Program Information Contact

Erin Fender
College and Career Academy Support Network
University of California
Berkeley Graduate School of Education
Berkeley, CA
Email: efender@berkeley.edu
(707) 591-6375
http://casn.berkeley.edu/index.php

Program Developer/Owner

College and Career Academy Support Network


Brief Description of the Program

Career Academies are school-based programs that seek to reduce dropout rates and improve school performance and career readiness among high-school youth. A Career Academy (CA) is organized as a school-within-a-school, where students work in "small learning communities." Each small learning community involves a small number of students working with the same group of teachers for three or four years of high school with the aim being to create a more personalized and supportive learning environment for students. CAs offer students a combination of academic and career-technical curricula and use a career theme to integrate the two. In an effort to build connections between school and work and to provide students with a range of career development and work-based learning opportunities, CAs establish partnerships with local employers. To encourage post-graduate education they also build linkages to local colleges through curricular articulation, dual enrollment programs, and field trips to 2- and 4-year institutions.

A Career Academy is a personalized small learning community within a high school, comprised of a subset of students and teachers for a two-, three-, or four-year span. The goals of the Career Academy are to enhance students' engagement and performance in high school and provide them with the credentials and skills needed to make successful transitions to post-secondary education and, eventually, a career. Students enter an academy in 9th or 10th grade through a voluntary process; they must apply and be accepted, with parental knowledge and support. As academies function within the larger high school, they require a great deal of administrator and counselor support. Academies vary in size, usually including one to three sections of students at each grade level, or 100-300 students in all. Academy classes are generally blocked back-to-back in the daily schedule, and students attend them as a group. Students are able to complete academy requirements within the regular school day, with the exception of work internships and possible college classes.

An academy involves teachers from different subject areas working together as a team. This team manages the program, with one member usually serving as the coordinator. Teams usually participate in professional development, particularly in implementing the key features of the learning model and gaining exposure to the career field. Team members have shared planning time and usually a daily common planning period. The joining of a group of students for several periods each day with teachers whom they have come to know well provides a family-like atmosphere and serves to nurture close student-teacher ties. Academy students may also participate in required and elective classes outside the academy, as well as other activities such as clubs and sports.

Students in a Career Academy have a mixture of career classes (usually one or two) and academic classes (usually three or four) at a time. The academic and some of the career-technical classes meet entrance requirements for four-year colleges and universities. They are linked to academic and industry standards and encourage high achievement. They show students how their subjects relate to each other and the career field.

The career classes develop knowledge in a given field. They are designed to expose students to the full range of careers in that field. Special projects require students to bring together academic skills across their subjects and apply these to community and work settings outside the school. Usually the junior and/or senior year includes work experience, a paid or unpaid work internship or community service assignment. During the senior year students are provided with college and career counseling, forming a post-graduate plan which may include college, a mixture of work and college, or full-time work. The majority of graduates do attend college at some level, most commonly at two-year institutions.

The academy career theme is selected locally, based on an industry that is healthy and can provide a cadre of partners interested in supporting the program. Employers from a group of companies in the selected field work as partners in the academy, serving on a steering committee (along with teachers, administrators, and often parents and students) that governs the program's development and operation. This committee helps to plan the various activities in which employee volunteers participate: as speakers at the school, informing students of the industry and career options; as field trip and job shadowing hosts at their companies; as individual mentors, career-related "big brothers and sisters;" as work internship supervisors during the summer following the junior year or part-time during the senior year; and as community service coordinators. The employer partners may also hire graduates. Postsecondary educational institutions are usually included as well, providing course articulation and concurrent enrollment options.

Outcomes

At the 12th grade survey:

  • Career Academies (CAs) increased both the level of interpersonal support students experienced during high school and their participation in career awareness and work-based learning activities.
  • Among students with a high-risk of dropping out, CAs reduced dropout rates, improved attendance, increased academic course-taking, and increased the likelihood of earning enough credits to graduate on time.
  • Among students with a low-risk of dropping-out, CAs increased the likelihood of graduating on time. CAs also increased career-related course-taking for these students without reducing their likelihood of completing a basic core academic curriculum.
  • In sites where CAs produced particularly dramatic enhancements in the interpersonal support that students received from teachers and peers, the CAs reduced dropout rates and improved school engagement for both high-risk and medium-risk subgroups. CAs that did not enhance these supports actually increased dropout rates and reduced school engagement for some students.
  • CAs did not improve standardized math and reading achievement scores.

8 Years After Students' Scheduled Graduation:

  • CAs students averaged 11% more in earnings than their non-CA group counterparts; when disaggregated by sex, significant effects were found for males only.
  • CAs students worked 1.7 hours more per week.
  • CAs students had a significantly higher percentage of living independently with children and a spouse/partner than non-Career Academy students, in the eight years following graduation.
  • No effects on high school completion rates, postsecondary enrollment, and postsecondary attainment.

Compared to California students statewide, students attending California Partnership Academies showed higher

  • graduation rates
  • completion rates of course requirements for college entrance

Brief Evaluation Methodology

A large scale, multi-site, random assignment research design was utilized to evaluate the effectiveness of Career Academies (CAs). The researchers studied nine CAs with a sample size of 1,764 students. Students in the sample were identified in the 8th or 9th grade and were followed through their senior year of high school and up until eight years post-graduation. Data utilized in the study consisted of survey information provided by both Academy and non-Academy students in the study sample, performance indicators obtained from school records and transcripts, and standardized test scores from a test the researchers administered to the sample of students. Qualitative data was collected during field visits to each participating site.

Another study compared students in 467 California Partnership Academies, which used the Career Academy model, to students statewide. The design involved no assignment, matching, or comparisons of students over time.

Blueprints Certified Studies

Study 1

Kemple, J. J. (2004). Career Academies: Impacts on labor market outcomes and educational attainment. Manpower Demonstration Research Corporation (MDRC), www.mdrc.org.


Kemple, J. J., & Snipes, J. C. (2000). Career Academies: Impacts on students' engagement and performance in high school. Manpower Demonstration Research Corporation (MDRC), www.mdrc.org.


Kemple, J. J., & Willner, C. J. (2008a). Career Academies: Long-term impacts on labor market outcomes, educational attainment, and transitions to adulthood. Manpower Demonstration Research Corporation (MDRC), www.mdrc.org.


Risk and Protective Factors

Risk Factors

School: Low school commitment and attachment*

Protective Factors

Individual: Academic self-efficacy*

Family: Parental involvement in education

School: Opportunities for prosocial involvement in education*, Rewards for prosocial involvement in school*


* Risk/Protective Factor was significantly impacted by the program

See also: Career Academies Logic Model (PDF)

Race/Ethnicity/Gender Details

Gender Specific Findings
  • Male
Race/Ethnicity/Gender Details
Career Academies were developed to be utilized with multiple racial and ethnic groups, and have demonstrated efficacy within schools that serve large minority populations. They also appear to be more effective with male participants, both in terms of labor market and social outcomes.

Training and Technical Assistance

Training and technical assistance is offered in how to establish an individual Career Academy within a given high school or more broadly restructure one or more high schools around this approach. There is a wide range of services provided, and the cost depends on a number of factors, such as the size of the academy or high school or district, number of staff members to be involved in such training, where they are in the knowledge/implementation process (e.g., brand new, expanding), how much of what kinds of help is needed, travel costs, etc. The range of services provided is available on the website at: casn.berkeley.edu./services.php.

As a first step, it is recommended that interested persons start with a telephone conversation so that they can be directed to materials, all of which are free and available for downloading.

Benefits and Costs

Source: Washington State Institute for Public Policy
All benefit-cost ratios are the most recent estimates published by The Washington State Institute for Public Policy for Blueprint programs implemented in Washington State. These ratios are based on a) meta-analysis estimates of effect size and b) monetized benefits and calculated costs for programs as delivered in the State of Washington. Caution is recommended in applying these estimates of the benefit-cost ratio to any other state or local area. They are provided as an illustration of the benefit-cost ratio found in one specific state. When feasible, local costs and monetized benefits should be used to calculate expected local benefit-cost ratios. The formula for this calculation can be found on the WSIPP website.

Program Costs

Start-Up Costs

Initial Training and Technical Assistance

There are a number of national organizations that offer training and technical assistance to high schools interested in implementing a Career Academy; however, there is no required training or technical assistance in order to start an Academy. There are many guides and resources available on-line to help school staff interested in starting an Academy as well as many national organizations offering technical assistance. Some of these include: The College & Career Academy Support Network (CCASN), The National Academy Foundation (NAF), and the National Career Academy Coalition (NCAC). For information on other organizations offering technical assistance, see: http://casn.berkeley.edu/resources.php?r=242.

Curriculum and Materials

There is no single curriculum for Career Academies. The model requires academic courses connected to career and industry themes as well as career and technical education courses. There is a wide array of curricula available on-line. The College & Career Academy Support Network (CCASN) has an on-line database of curricula searchable by career/industry as well as academic subject area, http://casn.berkeley.edu/curriculum.php.

Licensing

None.

Other Start-Up Costs

Start-up of a Career Academy requires significant commitment and time from key administrators and teaching staff. The model requires: 1) a small learning community with a team of teachers working together to provide a supportive environment; 2) an integrated curriculum between academics and career-technical instruction; and 3) employer partnerships and involvement that allow students to have career experiences. Establishing a new Career Academy requires 8 months to 1 year of planning by an advisory board that includes industry partners from the career field, higher education representatives, teachers and administrators, community members, and parents.

Intervention Implementation Costs

Ongoing Curriculum and Materials

No information is available

Staffing

Ratios: The Career Academy model is implemented as a way of structuring learning in high schools so the teacher to student ratio is determined by the school district. However, a core element of the model is that students are organized into smaller learning communities within high schools, and participation by students is voluntary. A single Career Academy cluster within a high school could have 30-60 students per grade who take Academy classes together and are taught by a team of teachers. (Variations of the model in terms of the size of the learning community are being piloted in the interest of reaching a greater scale of students; however in the evaluations that have shown positive outcomes, the sizes of the learning clusters in the Academy model were in the 30-60 student range.)

Qualifications: The teaching and administrator qualifications required by the system would also apply to the Career Academy.

Time to Deliver Intervention: The core curriculum is delivered as part of the regular school day for high school students. The model does require designation of a lead teacher and a lead administrator who need to dedicate time to coordinating the Academy. The lead teacher should be provided at least a teaching period for planning and/or be reimbursed for time spent outside of the regular school day. Teachers' schedules should be structured so they have ongoing team planning and curricula time built in.

Other Implementation Costs

Costs that will vary by locality include administrative support; the staff time dedicated to outreach to employers, organizing and supporting students in internships and other career experiences; and space and equipment.

Implementation Support and Fidelity Monitoring Costs

Ongoing Training and Technical Assistance

Ongoing training and technical assistance can be obtained from many national organizations working to support quality implementation of Career Academies and costs are negotiated individually with consultants.

Fidelity Monitoring and Evaluation

There is no single required fidelity-monitoring tool. There are several sets of standards used in different settings. Perhaps the most widely applicable are those in the "Career Academy National Standards of Practice", available on both the CCASN (http://casn.berkeley.edu) and National Career Academy Coalition (NCAC) websites (www.ncacinc.com). NCAC has developed a set of rubrics for the standards of practice and will evaluate Academies for a fee. The National Academy Foundation (www.naf.org) also has such a set of standards for its 500 Academy National Network of Career Academies.

Ongoing License Fees

None.

Other Implementation Support and Fidelity Monitoring Costs

No information is available

Other Cost Considerations

No information is available

Year One Cost Example

In this example, one high school is implementing the Career Academy model with two cohorts including a total of 200 students. Costs are based on a study of costs from the California Multiple Pathways Program that includes Career Academy models (Ace Parsi, David Plank, and David Stern: Costs of California Multiple Pathway Programs. Policy Analysis for California Education (PACE), University of California, Berkeley, 2010). These are estimates; actual costs will vary depending on the local model and costs.

Start-up ($297 per student) $59,400.00
Teacher and administrator planning time/coordination ($500 per student) $100,000.00
Total One Year Cost $159,400.00

With 200 students participating, estimated year one costs per student would be $797 in addition to regular per pupil spending in the high school. Once the program was beyond start-up costs, the per-student cost would be closer to $500 in addition to per pupil spending.

Funding Strategies

Funding Overview

The Career Academy is a school improvement approach that brings together a strong focus on both career and college readiness. Thus, it is generally implemented in place of other curricula and school improvement approaches and can be supported with the full range of federal, state, and local funds that support core K-12 education, as well as workforce development, and vocational and technical education funds.

Funding Strategies

Improving the Use of Existing Public Funds

High schools that implement a Career Academy will likely choose to shift funds spent on other curriculum and teacher professional development to this evidence-based model, as well as allocating teacher and administrator time to coordinate and implement the model.

Allocating State or Local General Funds

State education funds allocated to local school systems as well as locally-appropriated public school funding can support Career Academies. Some states, for example California and Florida, have created state grant programs dedicated to supporting the Career Academy model.

Maximizing Federal Funds

Formula Funds:

  • Title I is the funding stream most typically used to help cover the cost of the lead teacher, professional development, curricula, internships and field trips.
  • The Perkins Career and Technical Education formula grant can also be used to support Career Academies.
  • Workforce Investment Act formula funds administered by regional Workforce Development Boards within states can also be used to support Career Academies.

Discretionary Grants: Federal discretionary grants from the U.S. Department of Education and the U.S. Department of Labor can be used to fund the initial training, ongoing staffing and coordination, technical assistance, and classroom materials. Relevant discretionary grants include the Career and Technical Education discretionary grants from the U.S. Department of Education, and the Department of Labor Youth Career Connect discretionary grants.

Foundation Grants and Public-Private Partnerships

Because of the focus on a selected career or field, partnerships with employers are critical to the success of the model. Industry partners provide important in-kind resources including staff time for mentoring and guiding field trips of students; equipment and materials; staff time for organizing and supervising internships or job shadowing; and staff time for participation in the advisory group. Foundations, especially those with a stated interest in improving educational achievement and career outcomes for disadvantaged youth can also provide funding for training, coaching, technical assistance, staffing, and materials.

Debt Financing

No information is available

Generating New Revenue

While support for the model is most typically taken on as a school system responsibility, fundraising can provide additional flexible dollars, especially when the school has many competing needs and priorities. Parent Teacher Associations, business and local civic associations can potentially serve as sponsors of fundraising campaigns, and can also provide important volunteer support.

Data Sources

No information is available

Evaluation Abstract

Program Developer/Owner

College and Career Academy Support NetworkUniversity of California Berkeley1608 Tolman HallBerkeley, CA 94720-1670510-643-5748 casn.berkeley.edu

Program Outcomes

  • Academic Performance
  • Dropout/High School Graduation
  • Employment
  • Post Secondary Education
  • Truancy - School Attendance

Program Specifics

Program Type

  • Academic Services
  • Employment - Vocational
  • School - Environmental Strategies
  • Truancy Prevention

Program Setting

  • School

Continuum of Intervention

  • Selective Prevention
  • Universal Prevention

Program Goals

School-based programs designed to provide students with the credentials and skills needed to make successful transitions to post-secondary education and careers.

Population Demographics

High-school students.

Target Population

Age

  • Late Adolescence (15-18) - High School

Gender

  • Both

Gender Specific Findings

  • Male

Race/Ethnicity

  • All

Race/Ethnicity/Gender Details

Career Academies were developed to be utilized with multiple racial and ethnic groups, and have demonstrated efficacy within schools that serve large minority populations. They also appear to be more effective with male participants, both in terms of labor market and social outcomes.

Risk/Protective Factor Domain

  • Individual
  • School
  • Family

Risk/Protective Factors

Risk Factors

School: Low school commitment and attachment*

Protective Factors

Individual: Academic self-efficacy*

Family: Parental involvement in education

School: Opportunities for prosocial involvement in education*, Rewards for prosocial involvement in school*


*Risk/Protective Factor was significantly impacted by the program

Brief Description of the Program

Career Academies are school-based programs that seek to reduce dropout rates and improve school performance and career readiness among high-school youth. A Career Academy (CA) is organized as a school-within-a-school, where students work in "small learning communities." Each small learning community involves a small number of students working with the same group of teachers for three or four years of high school with the aim being to create a more personalized and supportive learning environment for students. CAs offer students a combination of academic and career-technical curricula and use a career theme to integrate the two. In an effort to build connections between school and work and to provide students with a range of career development and work-based learning opportunities, CAs establish partnerships with local employers. To encourage post-graduate education they also build linkages to local colleges through curricular articulation, dual enrollment programs, and field trips to 2- and 4-year institutions.

Description of the Program

A Career Academy is a personalized small learning community within a high school, comprised of a subset of students and teachers for a two-, three-, or four-year span. The goals of the Career Academy are to enhance students' engagement and performance in high school and provide them with the credentials and skills needed to make successful transitions to post-secondary education and, eventually, a career. Students enter an academy in 9th or 10th grade through a voluntary process; they must apply and be accepted, with parental knowledge and support. As academies function within the larger high school, they require a great deal of administrator and counselor support. Academies vary in size, usually including one to three sections of students at each grade level, or 100-300 students in all. Academy classes are generally blocked back-to-back in the daily schedule, and students attend them as a group. Students are able to complete academy requirements within the regular school day, with the exception of work internships and possible college classes.

An academy involves teachers from different subject areas working together as a team. This team manages the program, with one member usually serving as the coordinator. Teams usually participate in professional development, particularly in implementing the key features of the learning model and gaining exposure to the career field. Team members have shared planning time and usually a daily common planning period. The joining of a group of students for several periods each day with teachers whom they have come to know well provides a family-like atmosphere and serves to nurture close student-teacher ties. Academy students may also participate in required and elective classes outside the academy, as well as other activities such as clubs and sports.

Students in a Career Academy have a mixture of career classes (usually one or two) and academic classes (usually three or four) at a time. The academic and some of the career-technical classes meet entrance requirements for four-year colleges and universities. They are linked to academic and industry standards and encourage high achievement. They show students how their subjects relate to each other and the career field.

The career classes develop knowledge in a given field. They are designed to expose students to the full range of careers in that field. Special projects require students to bring together academic skills across their subjects and apply these to community and work settings outside the school. Usually the junior and/or senior year includes work experience, a paid or unpaid work internship or community service assignment. During the senior year students are provided with college and career counseling, forming a post-graduate plan which may include college, a mixture of work and college, or full-time work. The majority of graduates do attend college at some level, most commonly at two-year institutions.

The academy career theme is selected locally, based on an industry that is healthy and can provide a cadre of partners interested in supporting the program. Employers from a group of companies in the selected field work as partners in the academy, serving on a steering committee (along with teachers, administrators, and often parents and students) that governs the program's development and operation. This committee helps to plan the various activities in which employee volunteers participate: as speakers at the school, informing students of the industry and career options; as field trip and job shadowing hosts at their companies; as individual mentors, career-related "big brothers and sisters;" as work internship supervisors during the summer following the junior year or part-time during the senior year; and as community service coordinators. The employer partners may also hire graduates. Postsecondary educational institutions are usually included as well, providing course articulation and concurrent enrollment options.

Theoretical Rationale

Provides interpersonal support in the school environment, basically creating a school within a school.

Theoretical Orientation

  • Self Efficacy
  • Attachment - Bonding

Brief Evaluation Methodology

A large scale, multi-site, random assignment research design was utilized to evaluate the effectiveness of Career Academies (CAs). The researchers studied nine CAs with a sample size of 1,764 students. Students in the sample were identified in the 8th or 9th grade and were followed through their senior year of high school and up until eight years post-graduation. Data utilized in the study consisted of survey information provided by both Academy and non-Academy students in the study sample, performance indicators obtained from school records and transcripts, and standardized test scores from a test the researchers administered to the sample of students. Qualitative data was collected during field visits to each participating site.

Another study compared students in 467 California Partnership Academies, which used the Career Academy model, to students statewide. The design involved no assignment, matching, or comparisons of students over time.

Outcomes (Brief, over all studies)

Career Academies reduced dropout rates and improved school engagement among students least likely to do well in a regular school environment (12th grade survey). However, over the longer term (4 and 8 years post-graduation), the program had no impact on high school completion rates, postsecondary enrollment, and postsecondary attainment. While the academies produced more modest effects for other students, they created a more supportive school environment for all students, and provided them with more opportunities to explore careers and engage in work-based learning opportunities.

Over the long term, students who attended Career Academies averaged 11% more in earnings over the eight years after graduation than those students in the non-Academy group. The Career Academies also produced an increase in the percentage of young people living independently with children and a spouse/partner.

The California Partnership Academies showed higher graduation rates and success in completing course requirements for college entrance than California students statewide.

Outcomes

At the 12th grade survey:

  • Career Academies (CAs) increased both the level of interpersonal support students experienced during high school and their participation in career awareness and work-based learning activities.
  • Among students with a high-risk of dropping out, CAs reduced dropout rates, improved attendance, increased academic course-taking, and increased the likelihood of earning enough credits to graduate on time.
  • Among students with a low-risk of dropping-out, CAs increased the likelihood of graduating on time. CAs also increased career-related course-taking for these students without reducing their likelihood of completing a basic core academic curriculum.
  • In sites where CAs produced particularly dramatic enhancements in the interpersonal support that students received from teachers and peers, the CAs reduced dropout rates and improved school engagement for both high-risk and medium-risk subgroups. CAs that did not enhance these supports actually increased dropout rates and reduced school engagement for some students.
  • CAs did not improve standardized math and reading achievement scores.

8 Years After Students' Scheduled Graduation:

  • CAs students averaged 11% more in earnings than their non-CA group counterparts; when disaggregated by sex, significant effects were found for males only.
  • CAs students worked 1.7 hours more per week.
  • CAs students had a significantly higher percentage of living independently with children and a spouse/partner than non-Career Academy students, in the eight years following graduation.
  • No effects on high school completion rates, postsecondary enrollment, and postsecondary attainment.

Compared to California students statewide, students attending California Partnership Academies showed higher

  • graduation rates
  • completion rates of course requirements for college entrance

Mediating Effects

Not examined.

Effect Size

The tables presented figures on percentage change, defined as the mean difference between conditions as a percentage of the control group mean. They did not present Cohen's d or similar effect sizes.

Generalizability

The strongest positive effects were found among students at high risk of school failure and there are some good outcomes for low-risk students also. There is not much impact for students in the medium risk group. Long-term outcomes relating to higher earnings were more apparent for male participants. Also, male participants experienced improved outcomes in relation to marriage and being custodial parents.

Within the follow-up sample (at four years), male participants and high-risk participants were underrepresented, with female participants and low-risk participants being overrepresented.

The California state study generalizes to most of the state but is limited methodologically.

Potential Limitations

  • Dropped one school that decided to no longer use the program, without attempting to follow the subjects in the school.
  • Many outcomes relating to the labor market, educational attainment, and family life could not be measured or controlled at baseline, although the models used an extensive set of other baseline controls.
  • Some schools appear to have implemented the program poorly.
  • The program produced a possible iatrogenic follow-up effect for postsecondary enrollment among male and high-risk subjects, although the lower enrollment did not lead to worse labor market outcomes.
  • The study of California Partnership Academies did not use randomization or matching, relied on a self-selected sample, and lacked longitudinal comparisons.

Endorsements

Blueprints: Promising
Crime Solutions: Effective
OJJDP Model Programs: Effective
Social Programs that Work:Top Tier
What Works Clearinghouse: Meets Standards Without Reservations - Positive Effect

Program Information Contact

Erin Fender
College and Career Academy Support Network
University of California
Berkeley Graduate School of Education
Berkeley, CA
Email: efender@berkeley.edu
(707) 591-6375
http://casn.berkeley.edu/index.php

References

Study 1

Certified Kemple, J. J. (2004). Career Academies: Impacts on labor market outcomes and educational attainment. Manpower Demonstration Research Corporation (MDRC), www.mdrc.org.

Certified Kemple, J. J., & Snipes, J. C. (2000). Career Academies: Impacts on students' engagement and performance in high school. Manpower Demonstration Research Corporation (MDRC), www.mdrc.org.

Certified Kemple, J. J., & Willner, C. J. (2008a). Career Academies: Long-term impacts on labor market outcomes, educational attainment, and transitions to adulthood. Manpower Demonstration Research Corporation (MDRC), www.mdrc.org.

Kemple, J. J., & Willner, C. J. (2008b). Technical resources for Career Academies: Long-term impacts on labor market outcomes, educational attainment, and transitions to adulthood. Manpower Demonstration Research Corporation (MDRC), www.mdrc.org.

Study 2

Bradby, D., Dayton, C., Hanna, T., & Malloy, A. (2007). A profile of the California Partnership Academies, 2004-2005. Berkeley, CA: ConnectEd, and Career Academy Support Network, Graduate School of Education, University of California.

Dayton, C., Hester, C. H., & Stern, D. (2011). Profile of the California Partnership Academies 2009-2010. Career Academy Support Network, University of California Berkeley, and California Department of Education.

Study 1

Evaluation Methodology

Design: A large scale, multi-site, random assignment research design was utilized to evaluate the effectiveness of Career Academies (CAs) in achieving their goals. Students in the sample (n=1764) were identified at the end of 8th or 9th grade and were followed through their senior year of high school, and up until eight years following their expected graduation. The treatment group (n=959) was comprised of students who applied to one of the 9 academies and who were randomly selected to participate. Students who applied to CAs but were randomly turned down served as the comparison group (n=805). Students were identified as either posing a high-risk (25% of sample), medium-risk (50%), or low-risk (25%) for dropping out of school, based on selected background characteristics and prior school experiences.

Data on outcomes were collected at three time points: posttest (at the time of expected graduation and the program end), 4 years after posttest, and 8 years after posttest. The posttest data came from a student survey, school records, and achievement tests while the follow-up data came from surveys. Long-term data from the two follow-up surveys cover the first four years and second four years post-graduation.

In order for schools to be eligible for the study, they had to have implemented and sustained the core features of the Career Academy approach - small learning communities, career-themed curricula, and partnerships with employers - for at least two years prior to commencement of the study.

The sample of 1,764 subjects from nine schools came from an initial sample of 1,953 students from 10 schools (Kemple & Snipes, 2000, p. 20, footnote 1). However, one of the original 10 schools dropped the program after 2 years and "was not able to provide follow-up information needed for the analyses in this report." The study did not appear to follow the 126 students in the school. In addition, 59 of the initial group of students were dropped "because they should not have been included in the study sample" and four students were found to be deceased.

Attrition. At posttest, response rates were 82% for school records data and 85% for the 12th-grade survey. The study also administered achievement tests to the 691 students in the study sample who were scheduled to be in 12th grade at the end of the 1997-98 school year (Kemple & Snipes, 2000, p. 18, footnote 24). The 490 students who completed the achievement tests represent 71% of those requested to take the tests but only 28% of the full sample.

At the 4-year follow-up, survey data was collected from 1,458 youth who completed the survey. This represents 83% of the 1,764 young people in the full study sample: 83% of the Academy group and 82% of the non-Academy group.

At the 8-year follow-up, survey data were collected from 1,428 youth. This represents 81% of the 1,764 young people in the full study sample: 82% of the Academy group and 80% of the non-Academy group.

Sample: Forty-four percent (44%) of the subjects were male. Fifty-five percent (55%) were Hispanic, 30% Black, 6% White, and 7% Asian/Native American. The remainder (n=34) were unaccounted for in the demographic breakdown. Twenty-six percent (26%) of the subjects were categorized as "high-risk", with 50% as "medium-risk" and 24% as "low-risk".

Each of the nine schools was located in or near large urban school districts with above average dropout rates, unemployment rates, and low-income families.

Measures: Posttest outcome measurements included high school enrollment rates, credits earned and course-taking patterns, math and reading achievement test scores, use of non-school hours and involvement in negative risk-taking behaviors, and steps taken towards further education and work and plans for the future. The researchers obtained data for evaluation from three sources: 1) school records and transcripts, which included information about attendance, academic credits, and course-taking patterns (for the 12% dropping out, up to the time of dropping out); 2) student surveys that asked a wide range of questions about school experiences, employment and work-related experiences, extracurricular activities, preparation for college and post-secondary jobs, and plans for the future; and 3) standardized math computation and reading comprehension tests, which were administered to 490 students from the sample.

The follow-up studies included a diverse set of measures relating to labor market outcomes (monthly earnings, hours worked), educational attainment (high school completion, postsecondary enrollment, postsecondary attainment), family formation (living independently with spouse and children), and social adjustment (health insurance, welfare, registered to vote, drug use).

To divide the sample in three groups based on the risk for dropping out (high, medium, and low), the study used six measures obtained at baseline: daily school attendance, GPA, credits toward graduation, over age for grade, sibling who dropped out of high school, and two or more transfers across schools. The six measures were used to predict the probability of dropping out, and the probability was used to define the three risk groups.

Additionally, to study the cross-site variation in the impacts of CAs, each CA was identified to be a low-, medium-, or high-contrast site. A high-contrast CA is a site where the school environment is most different from a non-CA site. Likewise, a low-contrast CA is a site where the school environment is not very different from a non-CA site. The primary construct used to distinguish among sites in the study was the difference in the level of interpersonal supports that CA and non-CA students received. A CA site was ranked according to the difference between the percentage of CA and non-CA students who reported a high level of support.

Analysis: The posttest used multiple regression analysis with controls for background characteristics. Results were presented separately for each of the three risk groups and for males and females. In the long-term analysis, estimates were regression adjusted to control for background characteristics, clustering of students within schools, and cluster within random assignment years. The studies reported .10 significance levels, but indicated the level of significance so that it is possible to identify condition differences at the .05 probability level or lower.

Many outcomes relating to the labor market, educational attainment, and family life could not be measured or controlled at baseline. However, the models used an extensive set of other baseline controls.

Concerning intent to treat, the study dropped 126 students who were part of the original sample of 1,953 but attended a school that disbanded the program. It is not clear if these students could have been followed and included in the study. Otherwise, the study included all students with data in both conditions, even those assigned to the intervention who never enrolled in or left the program (42%) and those assigned to the control group who enrolled in the intervention (6%). A statistic based on the impact per enrollee was also presented to show the effect on those who actually enrolled in the intervention.

Outcomes

Implementation fidelity: One of the eligibility criteria for study schools was that they had successfully maintained the core components of the Career Academy approach for at least two years before the study commenced. This suggests the programs were well developed. However, the difference in the results between high-contrast schools and low-contrast schools suggests that some schools deviated substantially from the program ideal.

Baseline equivalence: Table 2.1 (Kemple & Snipes, 2000, p. 25) shows no statistically significant differences between the background characteristics and prior school experiences of students in the academy and non-academy groups. The comparison included 26 measures of demographic and family characteristics, school performance characteristics, and characteristics associated with dropping out of school. There were also no significant differences on these background characteristics within the high-risk, medium-risk, and low-risk subgroups (Kemple & Snipes, 2000, Appendix B). A regression on the probability of being assigned to the intervention found only one significant variable for the full sample: intervention subjects tended to be older at baseline than control subjects.

Differential attrition: At posttest, response rates were similar across the intervention and control groups: 81.5% and 83.5% for the school records data, 86.2% and 84.8% for the 12th-grade survey, and 71.8% and 69.9% for those requested to take the achievement tests.

To examine differential attrition for background characteristics, regressions tested for the probability of being assigned to the intervention group among the analysis samples (i.e., the 81% with school records data, the 85% completing the survey data, and the 28% for the achievement test data). Only one of the background variables was associated significantly with being in the intervention group: the intervention group was older than the control group for the survey data analysis sample.

For the 4-year follow-up, the study found some differences between respondents and non-respondents (Kemple, 2004, Unit 1, p. 2): "Young men and high-risk students were somewhat underrepresented in the respondent sample, while young women and low-risk students were slightly overrepresented." However, the loss of subjects was similar across conditions. Comparison of baseline characteristics for the respondents in the analysis sample showed no significant differences between those in the intervention and control groups (Kemple, 2004, Exhibits 1.3 and 1.4).

For the 8-year follow-up, the analysis sample did not differ systematically on any of the background characteristics (Kemple & Willner, 2008a, p. 7). Details are found in the accompanying technical report (Kemple & Willner, 2008b). Exhibit 1.2 shows significantly lower response rates among males, older youth, students living in single parent households, mobile youth, youth missing classes, low attendance, poor academic performance, and risk of dropping out. However, the loss of subjects was similar across conditions. Exhibit 1.3 compares conditions for the analysis sample, finding a significant difference only in hours spent watching TV.

Posttest: The posttest examined condition differences for 50 outcomes within three groups - high-risk, medium-risk, and low-risk students - or 150 total tests. To summarize, the intervention group showed significantly better outcomes on 15 of the 50 tests for the high-risk students, 3 of 50 tests for the medium-risk students, and 5 of 50 tests for the low-risk students.

The treatment group experienced a statistically significant increase in their reported level of interpersonal support from faculty as well as an increase in their participation in career awareness and work-based learning activities. Among treatment group students with a high risk of dropping out, there was a significant reduction in dropout rates, improvement in attendance, increase in academic course-taking, and increase in the likelihood of earning enough credits to graduate on time. Among treatment group students with a low risk of dropping out, there was an increase in the likelihood of graduating on time. Treatment group students across risk levels experienced an increase in vocational course-taking without reducing their likelihood of completing a basic core academic curriculum. When findings were averaged across groups, there were only slight reductions in dropout rates and modest increases in other measures of school engagement, but the only significant impact was for more course credits.

High-contrast CAs, those intervention sites where the school environment is very different from non-CA schools, produced a statistically significant lower dropout rate among medium-risk students, as well as an increase in completion of a core academic curriculum. Conversely, low-contrast CAs produced higher dropout rates, reduced attendance, and lower rates of academic course-taking for medium-risk students.

Long-term (48 months post-graduation): In terms of the full sample, CA students earned significantly more ($107, p<0.05) per month than their control group counterparts at the 48-month follow-up. CA students also averaged 1.9 more working hours per week (p<0.05) and $0.47 more per hour than non-academy students (p<0.05). The program had no effect on high school completion rates, postsecondary enrollment, and postsecondary attainment, but the authors note that the program focus on careers did not worsen academic success. A summary of the results for the full sample in Exhibits 3.1, 3.6, and 3.9 of Kemple (2004) shows that 5 of 43 tests reached statistical significance at the .05 level.

When analyzed by subgroup, it was young men, rather than young women who experienced the greatest change due to the program. The male subgroup average monthly earnings were $212 more than their control group counterparts. This was reflected in significantly greater months of employment over the four years (2.8), greater hours worked per week (4.2), and higher hourly wages ($0.74). In terms of the low-, medium-, and high-risk subgroups, the medium risk group was the only academy group to have significantly higher monthly ($141) and hourly ($0.53) earnings than their non-academy equivalents.

One possible iatrogenic effect identified was that male and high-risk students within the Career Academy group were less likely than their non-Academy counterparts to have enrolled in post-secondary education. However, the lower enrollment did not adversely affect labor market outcomes.

Note: The report at 8-years post-graduation indicates slightly varied results in terms of the outcomes for years 1-4 compared to the original report produced at year 4. See Kemple & Willner (2008a, p. 13) for results over the full period.

Long-term (96 months post-graduation): For the full sample over years 5-8 post-graduation, intervention subjects earned $216 more (p<0.01) per month than their control group counterparts. They also worked on average 1.7 hours more per week (p<0.05). With regard to social outcomes, the academy group had higher levels of being a custodial parent (p=0.006), lower levels of being a noncustodial parent (p=0.026) and higher levels of living independently with children and a partner (p=0.01). The program had no effect on high school completion rates, postsecondary enrollment, or postsecondary attainment, but the authors note that the program focus on careers did not worsen academic success. A summary of the results for the full sample in Exhibits 3.1B, 3.4, and 3.6 in Kemple & Willner (2008b) shows that 7 of 40 tests reached statistical significance at the .05 level.

When analyzed by subgroup, young men again experienced the greatest labor-related outcomes from the program. Young men from the academy group earned on average $361 (p<0.05) more per month than their control group counterparts. They also were employed for 2.8 months more over the 4-year period (p<0.01) and worked 4.1 hours more per week (p<0.01). With regards to social outcomes, male academy group participants were more likely to be married and living with their partner (p=0.020), more likely to be a custodial parent (p=0.003), less likely to be a noncustodial parent (p=0.023), and less likely to be living independently with no children (p=0.015). Female academy group participants were less likely to be living with parents/guardians with or without children (p=0.005).

In terms of the high-, medium-, and low-risk subgroups, the analyses of monthly earnings showed no significant differences across the three subgroups in the effect of the program. More detailed analyses for the subgroups are presented in the technical report (Kemple & Willner 2008b). There were also no differences in the null effect of the program on educational outcomes across the three subgroups.

Study 2

Evaluation Methodology

California Partnership Academies use the Career Academies model, usually in grades 10-12. Bradby et al. (2007) present data for 2004-2005, but Dayton et al. (2011) incorporate the earlier study in examining data for 2004-2005 and 2009-2010.

Design:

This study presents a profile of schools and students in California Partnerships Academies (CPAs), with some comparison to statistics for the state of California as a whole. It did not assign or match schools or students to treatment and control groups. Given that students self-selected into the program schools and likely differed substantially from other students in motivation, skills, and background characteristics, the study cannot establish the causal impact of the program.

The study obtained data in 2004-2005 and 2009-2010 from the California Department of Education on schools and students from 467 CPAs that operated in 278 of California's 1,264 comprehensive high schools. The schools are located in 36 of California's 58 counties, with the largest concentration in the six most populous counties. The 467 CPAs enrolled 48,436 students in grades 10-12 - about 3% of all California students in those grades.

Each CPA submits an annual report to the California Department of Education. The study used data from these annual reports on students in grades 10-12. The cross-sectional comparisons do not follow students or schools over time or compare outcomes for higher grades to lower grades.

Sample: By law, at least 50% of the students in each incoming class of CPA sophomores must meet three of six at risk-requirements (e.g., poor attendance record, low motivation for school, low test scores). Most (54%) of the schools fell in the bottom 40% of schools statewide. The students were 53% female and 47% male and were primarily Hispanic (59%), but also white (16%), Asian (10%), and black (9%).

Measures: CPAs must submit annual reports on how many students meet specified targets for attendance, credits, and graduation. Under the performance-based program, CPAs receive funding only for students who meet the targets. The measures depend on the accuracy of reports from the schools, which deliver the program and have financial incentives to show positive results.

Analysis: The report listed percentages for CPAs and state schools without tests of significance, controls for baseline outcomes, or adjustments for clustering within schools. Along with overall percentages, the report presented figures for CPA and state schools within gender and race/ethnicity groups.

Outcomes

Implementation Fidelity: Fidelity showed in high percentage of CPA seniors having postsecondary plans, working with a mentor, and participating in work-based learning. Older academies did better on test scores than new academies, likely because they had more time to implement the program well.

Baseline Equivalence: The distribution by race and ethnicity in the CPAs differed from that in the state overall. Hispanics made up 59% of CPAs and 47% of schools statewide; whites made up 16% of CPAs, and 30% of schools statewide. CPAs had slightly more females than schools statewide, 53% versus 49%. The study presented no other tests for preexisting student differences.

Differential Attrition: Since subjects were not followed over time, the study could not test for differential attrition.

Posttest: Comparison of outcomes with data for both CPAs and state schools showed the following results.

Pass Rates for Math and English Language Arts. Test results for the 10th-grade California High School Exit Exam demonstrated higher pass rates for CPAs than schools statewide in both math (80% versus 74%) and English (84% versus 76%) in 2004-2005 but little difference in 2009-2010 (83% versus 81% in math and 82% versus 81% in English).

Graduation Rates. The outcome measures the percentage receiving a diploma among those who remained in high school until their senior year (but doesn't account for potentially different dropout rates before the senior year). On this measure, CPAs did better than statewide schools in 2004-2005 (96% versus 87%) and 2009-2010 (95% versus 85%).

Meeting California College Course Requirements. Higher percentages of CPA students than students statewide completed course requirements for entrance into the University of California and California State colleges in 2004-05 (50% versus 35%) and 2009-2010 (57% versus 36%).

Modifiers. The differences between CPAs and all state schools appear similar for males and females, but males perhaps benefitted more from the program. The differences among race/ethnic groups also appear similar, but students of color may have benefitted more from the program. There is some evidence that Asians did worse in CPA schools. Also, CPA schools with a focus on manufacturing and product development had higher exam pass rates (100%) than CPA schools with a focus on marketing sales and service (67%) and building trades and construction (57%).