A career pathways program designed for community college students from low-income backgrounds to graduate with an associate degree or industry-recognized certificate in a high-demand occupation and find employment that pays a living wage.
Blueprints: Promising
What Works Clearinghouse: Meets Standards Without Reservations - Positive Effect
Felida Villarreal, President & CEO
Valley Initiative for Development and Advancement
fvillarreal@vidacareers.org
www.vidacareers.org
Valley Initiative for Development and Advancement
Valley Initiative for Development and Advancement (VIDA) supports postsecondary students with the goal that they will graduate with an associate degree or industry-recognized certificate in a high-demand occupation and achieve living-wage employment. The program's central elements are three-fold: (1) a requirement for full-time college enrollment, (2) mandatory and intensive advising, and (3) substantial financial assistance. The program provides financial assistance with tuition, textbooks, tools, uniforms, transportation, and childcare enabling participants to attend college full-time. Weekly group counseling and individual meetings are held with participants to address academic and personal challenges that otherwise might derail training.
Valley Initiative for Development and Advancement (VIDA) supports postsecondary students with the goal that they will graduate with an associate degree or industry-recognized certificate in a high-demand occupation and achieve living-wage employment. To support participants' successful completion of training, VIDA provides comprehensive and intensive counseling services and substantial financial assistance at a cost of approximately $15,000 per participant over an average of two and a half years.
The primary program features are:
Together these features and other less central program components provide a substantial set of services for which a participant must meet a demanding set of requirements. The program's aim is to move students quickly through standard college coursework in order to obtain a certificate or a degree and secure employment.
Rolston et al. (2017) described one additional program feature, the College Prep Academy. This 16-week, full-time basic skills ("bridge") program was designed to prepare individuals who were not college ready but had tenth grade skill levels or better to pass college entrance exams. Roughly nine percent of the intervention group participated in the academy; however, this program feature was discontinued due to difficulties of recruitment.
In Study 1, Rolston et al. (2017, 2021) and Rolston and Walton (2022) found compared to the control group, treatment group members:
For study 1, between November 2011 and September 2014, Rolston et al. (2017, 2021) and Rolston and Walton (2022) randomly assigned 958 individuals to either the treatment or the control group. Data were collected via self-report surveys and administrative records measuring postsecondary education and individual and family well-being outcomes at 24 months (Rolston et al., 2017), 3 years (Rolston et al., 2021), 3-5 years (Rolston et al., 2021), and 6 years (Rolston & Walton, 2022) after random assignment.
Study 1
Rolston, H., Copson, E., & Gardiner, K. (2017). Valley Initiative for Development and Advancement: Implementation and Early Impact Report, OPRE Report #2017-83. Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.
Rolston, H., Copson, E., Buron, L., & Dastrup, S. (2021). Valley Initiative for Development and Advancement (VIDA): Three-Year Impact Report. OPRE Report #2021-96. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.
Rolston, H., & Walton, D. (2022). Valley Initiative for Development and Advancement (VIDA): Six-Year Impact Report. OPRE Report 2022-58. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.
School: Instructional Practice, Rewards for prosocial involvement in school
*
Risk/Protective Factor was significantly impacted by the program
See also: Valley Initiative for Development and Advancement (VIDA) Logic Model (PDF)
Most (98 percent) of the sample in Study 1 (Rolston et al., 2017, 2021; Rolston & Walton, 2022) was Hispanic.
Source: Washington State Institute for Public Policy
All benefit-cost ratios are the most recent estimates published by The Washington State Institute for Public Policy for Blueprint programs implemented in Washington State. These ratios are based on a) meta-analysis estimates of effect size and b) monetized benefits and calculated costs for programs as delivered in the State of Washington. Caution is recommended in applying these estimates of the benefit-cost ratio to any other state or local area. They are provided as an illustration of the benefit-cost ratio found in one specific state. When feasible, local costs and monetized benefits should be used to calculate expected local benefit-cost ratios. The formula for this calculation can be found on the WSIPP website.
No information is available
No information is available
No information is available
No information is available
No information is available
VIDA's Management Team includes:
VIDA's Career Counselor-Case Managers are responsible for monitoring the academic progress of each participant, providing individualized support for their personal needs, holding weekly counseling sessions (either in groups or one-on-one), and ensuring that participants comply with program requirements. Typically, these Counselors maintain a caseload of approximately 50 participants.
A Retention Specialist-Placement Coordinator attempts to reengage participants who have temporarily suspended enrollment. Additionally, this person collects data from VIDA graduates (e.g., employment status, wages, benefits).
Case Aides assist with data entry for program enrollment and for the Counselors.
At the time of their evaluation, Rolston et al. (2021, Exhibit 7-1) reported the following program expenses for a total cost per treatment group member of $11,070:
Program Services:
Financial Assistance for Participants:
No information is available
No information is available
No information is available
No information is available
No information is available
No information is available
Local city and county governments, economic development corporations, regional workforce development boards, and institutions of higher education may identify growing sectors of their regional economies where there are needs for skilled labor. These entities provide funding to cover the costs of training participants in select occupations with expectations that dollars are spent on residents within these localities.
Foundations may be a good source of funding for recruitment to increase program participation.
Valley Initiative for Development and Advancement417 S. Ohio Ave.Mercedes, TX 78570(800) 478-1770 vidacareers.org
A career pathways program designed for community college students from low-income backgrounds to graduate with an associate degree or industry-recognized certificate in a high-demand occupation and find employment that pays a living wage.
Low-income adults who are 18 years of age or older (on public assistance, with incomes below 200 percent of the poverty level, or who are either unemployed or underemployed with family responsibilities) with high school credentials, at or close to college ready.
Most (98 percent) of the sample in Study 1 (Rolston et al., 2017, 2021; Rolston & Walton, 2022) was Hispanic.
Lack of access to economic opportunity
School: Instructional Practice, Rewards for prosocial involvement in school
*Risk/Protective Factor was significantly impacted by the program
Valley Initiative for Development and Advancement (VIDA) supports postsecondary students with the goal that they will graduate with an associate degree or industry-recognized certificate in a high-demand occupation and achieve living-wage employment. The program's central elements are three-fold: (1) a requirement for full-time college enrollment, (2) mandatory and intensive advising, and (3) substantial financial assistance. The program provides financial assistance with tuition, textbooks, tools, uniforms, transportation, and childcare enabling participants to attend college full-time. Weekly group counseling and individual meetings are held with participants to address academic and personal challenges that otherwise might derail training.
Valley Initiative for Development and Advancement (VIDA) supports postsecondary students with the goal that they will graduate with an associate degree or industry-recognized certificate in a high-demand occupation and achieve living-wage employment. To support participants' successful completion of training, VIDA provides comprehensive and intensive counseling services and substantial financial assistance at a cost of approximately $15,000 per participant over an average of two and a half years.
The primary program features are:
Together these features and other less central program components provide a substantial set of services for which a participant must meet a demanding set of requirements. The program's aim is to move students quickly through standard college coursework in order to obtain a certificate or a degree and secure employment.
Rolston et al. (2017) described one additional program feature, the College Prep Academy. This 16-week, full-time basic skills ("bridge") program was designed to prepare individuals who were not college ready but had tenth grade skill levels or better to pass college entrance exams. Roughly nine percent of the intervention group participated in the academy; however, this program feature was discontinued due to difficulties of recruitment.
The program is grounded in the career pathways framework, which guides the development and operation of interventions aiming to improve the occupational skills of low-income adults, primarily older nontraditional students, by increasing their entry into, persistence in, and completion of postsecondary training. Central to accomplishing these outcomes, the framework describes strategies for overcoming barriers to education and training that these students can face. Key features of programs within this framework include: a series of well-defined training steps, promising instructional approaches targeted to adult learners, services to address academic and non-academic barriers to program enrollment and completion, and connections to employment (Fein, 2012).
In addition (and related), the program is premised on the belief that part-time enrollment plays an important role in student dropout and failure to complete credentials. It is hypothesized that the combination of extensive financial assistance and frequent, required meetings with a counselor will help participants engage in full-time college programs, persist, and earn a credential.
Note: Fein, David J. 2012. "Career Pathways as a Framework for Program Design and Evaluation." OPRE Report #2012-30. Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services. https://www.acf.hhs.gov/opre/resource/career-pathways-as-a-framework-for-program-design-and-evaluation-a-working.
For study 1, between November 2011 and September 2014, Rolston et al. (2017, 2021) and Rolston and Walton (2022) randomly assigned 958 individuals to either the treatment or the control group. Data were collected via self-report surveys and administrative records measuring postsecondary education and individual and family well-being outcomes at 24 months (Rolston et al., 2017), 3 years (Rolston et al., 2021), 3-5 years (Rolston et al., 2021), and 6 years (Rolston & Walton, 2022) after random assignment.
Study 1 found that, compared to the control group, treatment group members earned significantly more: 1) college credits and credentials over an initial 24-month follow-up period (Rolston et al., 2017) and 2) college credentials requiring a year or more of training, a confirmatory outcome, three years after random assignment (Rolston et al., 2021). In addition, three years post randomization, compared to control students, treatment students earned significantly more associate degrees (or higher) and college credits (Rolston et al., 2021). At six years post-randomization, Rolston and Walton (2022) found similar effects on college credentials received and associate degrees.
In Study 1, Rolston et al. (2017, 2021) and Rolston and Walton (2022) found compared to the control group, treatment group members:
Not examined.
Not reported.
Study 1 (Rolston et al., 2017, 2021; Rolston & Walton, 2022) sampled older low-income students, only a small percentage of whom had basic skills deficits.
Study 1 (Rolston et al., 2017, 2021; Rolston & Walton, 2022):
Study 1 (Rolston et al., 2017, 2021) registered their outcomes on the Open Science Framework site: https://osf.io/bpm8u/.
The Valley Initiative for Development and Advancement (VIDA) study is one of nine similar studies conducted under Pathways for Advancing Careers and Education (PACE) Evaluation. Funded by the Administration for Children and Families (ACF) within the U.S. Department of Health and Human Services, the PACE evaluation is a 10-year study of nine programs that include key features of a "career pathways framework." PACE researchers published an analysis plan in October 2015 specifying key hypotheses and outcome measures (https://www.acf.hhs.gov/sites/default/files/documents/opre/pace_analysis_plan_final_10_19_15_b508.pdf).
Blueprints: Promising
What Works Clearinghouse: Meets Standards Without Reservations - Positive Effect
Felida Villarreal, President & CEO
Valley Initiative for Development and Advancement
fvillarreal@vidacareers.org
www.vidacareers.org
Certified
Rolston, H., Copson, E., & Gardiner, K. (2017). Valley Initiative for Development and Advancement: Implementation and Early Impact Report, OPRE Report #2017-83. Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.
Certified
Rolston, H., Copson, E., Buron, L., & Dastrup, S. (2021). Valley Initiative for Development and Advancement (VIDA): Three-Year Impact Report. OPRE Report #2021-96. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.
Certified
Rolston, H., & Walton, D. (2022). Valley Initiative for Development and Advancement (VIDA): Six-Year Impact Report. OPRE Report 2022-58. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.
Evaluation Methodology
Design:
Recruitment: Participants in the study had to be unemployed, underemployed, meet federal poverty income levels, or be on public assistance. Additionally, they had to be residents of the Rio Grande Valley, 18 years or older, and eligible to work in the United States. The program required at least a high school diploma or GED and enrolled participants with prior college experience. Individuals had to be committed and able to enroll full time in college in an occupational program. To meet its recruitment goals, during the period of the study, the intervention was extensively marketed through media, partners and word-of-mouth referrals.
Assignment: Between November 2011 and September 2014, the 958 program applicants were randomly assigned to either the treatment (n=478) or the control group (n=480). The control group received normal services and were embargoed for two years from treatment. Control group members had access to other programs and services available in the community, including training at the five colleges with which the program partners, but after two years, control students could apply again for the treatment and enroll if they met eligibility criteria.
Assessments/Attrition: Assessments came from a 20-month survey, a 3-year survey, and a 6-year survey, and from administrative records at 1 year, 3 years, 4 years, 5 years (for the subset of early cohorts who started five years previous), and 6 years. The 20-month survey had an 85% completion rate, the 3-year survey had a 76% retention rate, and the records data had a 100% completion rate (excepting the exclusion of late cohorts at the 5-year follow-up). The 6-year survey had a 77% response rate, and the records data again had a 100% completion rate.
Sample: Approximately half the sample had annual household incomes of less than $15,000, and more than 85 percent had incomes less than $30,000. About two-thirds received benefits from the Supplemental Nutrition Assistance Program (SNAP, formerly known as Food Stamps) or Special Supplemental Nutrition Program for Women, Infants, and Children (WIC). About two-thirds also reported experiencing financial hardship in the past year. More than 60 percent were age 25 and older, and more than one-fifth were age 35 or older. Virtually all had at least a high school degree or equivalent, and more than half had a year or more of college. Most study participants (about 70 percent) were female. The vast majority (95.8%) were Hispanic, and most (64.9%) were not working at the time of random assignment.
Measures:
The study identified confirmatory, secondary, and exploratory outcomes, which came from surveys and administrative records, both independent sources. The measures have good face validity, but the records data has some gaps. Educational records come from four local colleges and universities and capture 96% of enrollments; earnings records come from state unemployment systems and miss employment outside the formal sector.
The three confirmatory outcomes included measures of:
The 14 secondary outcomes included measures of college degrees, credits, certification, and enrollment; employment, earnings, and job skills; career knowledge and supports; and family financial well-being.
The 39 exploratory outcomes included measures of confirmatory and secondary outcomes over a four- and five-year period plus measures at the 3-year survey of job quality, family financial status, parent engagement, child outcomes, childbearing, and living arrangements.
The 6-year follow-up (Rolston & Walton, 2022) listed two confirmatory outcomes:
A count from the tables listing results indicates tests for approximately 16 secondary outcomes relating to college degrees, enrollment, quarterly earnings, and full-time work. The tables also list about 77 exploratory outcomes. The exploratory outcomes at six years of follow-up included measures of employment and earnings in a variety of other years and quarters, additional measures of college enrollment and credential receipt, measures of financial well-being, and several measures related to parenting and child well-being.
The data for the 6-year follow-up came from survey results and from two sources of administrative records (both matched to the student sample in June 2021). The first source, the National Student Clearinghouse (NSC), had data on student enrollment, degrees earned, and other credential completion from most U.S. institutions of higher education. The second source, the National Directory of New Hires (NDNH), aggregates wage records reported on a quarterly basis to states by employers.
Analysis: In conducting impact regression analyses, the research team controlled for baseline differences by using baseline values as covariates. For confirmatory outcomes, authors applied a one-tailed test of significance because each has a hypothesized direction (an increase in the average level of each outcome).
Intent-to-Treat: The authors reported they followed an intent-to-treat protocol. The 20-month survey data (Rolston et al., 2017) were used to help impute values for missing data on job and education spells from the three-year follow-up survey (Rolston et al., 2021). In addition, Rolston et al. (2021) used National Student Clearinghouse data to adjust for non-response bias in the survey (see Appendix C, Rolston et al., 2021).
Outcomes
Implementation Fidelity: No quantitative information about fidelity was provided. However, implementation data showed most treatment group members received a substantial dose of individual and group counseling. Almost all participants (99 percent) received assistance with tuition and expenses related to attending and completing school, and the program provided an average of almost $7,000 in direct financial assistance per participant within 24 months of random assignment. Almost all treatment group members (98 percent) participated in an education or training activity, which included both basic skills education in the College Prep Academy and occupational training. Ninety-seven percent of all treatment group members participated in occupational training, including those who advanced from the College Prep Academy into college-level programs of study. Ninety-one percent of treatment group members earned at least one college credit, and 66 percent completed 30 or more college credits (by some standards the equivalent of a year of full-time college).
Baseline Equivalence: Significant differences were observed in 2 of 11 baseline characteristics measured: treatment group members were more likely than control group members to have college experience and to be working 20 hours or more (see Exhibit 2-2, p. 17 of Rolston et al., 2017 or Exhibit 1-1, p. 6 of Rolston et al., 2021). There were no baseline differences in age, gender, race/ethnicity, income, receipt of public assistance, and expected work hours in the next few months.
Differential Attrition: There was no attrition for primary and secondary confirmatory outcomes (administrative records were used). Differential attrition tests for survey data (measuring exploratory outcomes) were not reported. Rolston & Walton (2022) noted only that retention rates were 81% for the treatment group and 73% for the control group.
Posttest: The average length of participation in VIDA is 18 months and the Rolston et al. (2017) evaluation was conducted around 24 months post randomization. However, because some treatment participants first entered the College Prep Academy, others were newly enrolling in college, and still others were already enrolled in an occupational program, the program expected that only a subset of treatment and control group members would have had time to complete their programs during the 24-month follow-up period. Indeed, two years after random assignment, 42 percent of the treatment group was still attending training. Among those still attending, about half were still working towards a credential while the other half had obtained a credential, suggesting they continued in their educational pathway to attain further certificates or degrees.
Thus, Rolston et al. (2017) reported on college credits earned as the relevant confirmatory posttest outcome and attainment of a college credential as a secondary outcome. As reported in Exhibit ES-1 (p. viii), treatment significantly increased the total college credits earned within 24 months of random assignment by roughly six percentage points: 33.1 credits for the treatment group compared to 27.5 credits for the control group (p < .001 reported for a one-tailed test). Additionally, 53 percent of treatment group members earned a college credential compared to 45 percent of control group members (an 8 percentage point difference; p = .003, one-tailed test).
Long-Term: To summarize the long-term results, Rolston et al. (2021) found that the intervention, relative to the control group, significantly improved 1 of 2 confirmatory outcomes (college credentials but not earnings) and 4 of 13 secondary outcomes (associate degree, any college credit, enrollment, and number of college credits). The 39 exploratory outcomes were consistently and significantly improved by the intervention in the education domain but not improved in the earnings domain and rarely improved in the life outcomes domain.
Specifically, three years after random assignment, Rolston et al. (2021) found that 61 percent of the treatment group earned a college credential requiring a year or more of training compared to 52 percent of the control group (a statistically significant difference of 9 percentage points; Exhibit ES-1; p = .001 reported for a one-tailed test). In addition, compared to control, three years after random assignment, treatment significantly increased receipt of an associate degree or higher by 7 percentage points (p = .007 reported for a one-tailed test) and "any college credential" by 13 percentage points (p = .001 reported for a one-tailed test). These results are reported in Exhibit 3-1 (p. 23). However, there were no long-term impacts on earnings, which was a confirmatory outcome (both treatment and control groups earned somewhat over $6,000 in quarters 12-13 after random assignment; see Exhibit 4-1, p. 30).
At three years after random assignment, Rolston et al. (2021) found no statistically significant impact on average hourly wages or employment, including working in jobs requiring at least mid-level skills. The evaluation also conducted exploratory analyses of indicators of job quality, such as working at least 32 hours per week, working in a job that offers health insurance, and working in a job with a supportive environment. No impacts on these job quality indicators were detected.
At three years post randomization, there were no significant long-term effects on financial outcomes, including reducing student debt or other forms of debt, indicators of financial distress, or receipt of means-tested public benefits. And there was no effect on having health insurance (Rolston et al., 2021).
Sub-group analyses found that three years after randomization, significantly fewer women in the treatment group compared to the control group had a child since study enrollment or were pregnant at the time of the survey, and women in the treatment group were less likely to be living with a spouse or partner compared to women in the control group.
Some analyses reported in Rolston et al. (2021) extend outcomes to 5 years of education records for an early cohort of participants representing about 80% of the total sample. Five years post randomization, findings show treatment students - compared to control - received significantly more 1) college credentials requiring 1 one or more years to earn, 2) associate degrees (or higher), and 3) "any college credential" (see Exhibit 3-1, p. 23).
Six-Year Follow-up (Rolston & Walton, 2022)
The authors first noted that for about 20% of the sample, the follow-up period overlapped with the COVID-19 pandemic. However, because the program had no detectable impact on quarterly earnings in 2019 or 2020, the research team did not further investigate the pandemic's effects on program impacts.
Overall, the program showed strong educational benefits but little impact on employment, earnings, or several measures of well-being. The authors noted that the inability of educational advances to translate into economic improvement was unexpected and puzzling.
In the education domain, the confirmatory outcome of receiving a college credential proved significantly higher in the intervention group than the control group. Additionally, in response to a Blueprints request, the lead author provided results that indicated this effect was consistent across all age groups - ages 18-24, ages 25-34, and ages 35 and older. Further, six of the seven secondary and exploratory outcomes, which related to enrollment and completion of early credentials, showed significant intervention effects.
In the earnings domain, the confirmatory outcome of average earnings in quarters 23 and 24 did not differ significantly across conditions. Only one of the 50 secondary and exploratory outcomes (access to career network) was statistically significant.
In the other domains, only three of the 36 outcomes were statistically significant. Compared to the control group, the intervention group experienced reduced receipt of means-tested public benefits, and women in the intervention group had a reduced likelihood of childbearing and were less likely to be living with a child.