Print Page
Blueprints logo

Bottom Line College Advising Model

A college advising program designed to support low-income and first-generation students during the college application process and throughout college enrollment.

Fact Sheet

Program Outcomes

  • Post Secondary Education

Program Type

  • Academic Services
  • School - Individual Strategies

Program Setting

  • School
  • Community

Continuum of Intervention

  • Selective Prevention

Age

  • Late Adolescence (15-18) - High School

Gender

  • Both

Race/Ethnicity

  • All

Endorsements

Blueprints: Promising
Crime Solutions: Promising
OJJDP Model Programs: Promising
Social Programs that Work:Top Tier
What Works Clearinghouse: Meets Standards Without Reservations - Potentially Positive Effect

Program Information Contact

Steve Colón, Chief Executive Officer
Erin Harris, Director of Data and Evaluation
eharris@bottomline.org

Bottom Line
50 Milk St., 16th Floor
Boston, MA 02109
857-415-4810
www.bottomline.org

Program Developer/Owner

Andrew Barr, Evaluator
Department of Economics, Texas AM University


Brief Description of the Program

Bottom Line provides individualized college advising to high school students starting at the end of their junior year. Eligibility criteria for the program includes: 1) a high school GPA of at least 2.3; 2) family income below 200 percent of the poverty line; and 3) being in the first generation in their family to go to college in the United States. Students are initially admitted into the "Access" program, which provides students with college and financial aid application support starting the summer before their senior year in high school through the summer after high school graduation. During this time, advisors help students identify well-matched colleges and then complete and submit college applications. The Bottom Line "Access" program also helps students evaluate the affordability of different postsecondary options, and then advisors actually help students apply for financial aid. Barr & Castleman (2017) reported that Bottom Line operates in several cities in 3 states (Massachusetts, New York, and Illinois). As reported in Barr & Castleman (2017), a unique component of the program is that counselors encourage students to attend a set of target colleges and universities within New York and Massachusetts that the program has identified as providing students with an optimal combination of quality and affordability. Students who enroll at one of the target institutions (approximately 50 percent of the students in Barr & Castleman, 2017) are eligible for the Bottom Line "Success" program, in which individualized and campus-based support is provided for up to six years following high school. Two important contrasts between the Bottom Line model and other advising programs are: 1) Bottom Line employs professional advisors (all have a minimum of a college degree) who work full-time; and 2) the intensity of the model - the average student spends 10-15 hours with their advisor before transitioning to college. Advisors have an average caseload of 50-60 students and meet with each student for an hour every three or four weeks during senior year, at Bottom Line's office in each community.

Bottom Line provides individualized college advising to high school students starting at the end of their junior year. Eligibility criteria for the program includes: 1) a high school GPA of at least 2.3; 2) family income below 200 percent of the poverty line; and 3) being in the first generation in their family to go to college in the United States. Students are initially admitted into the "Access" program, which provides students with college and financial aid application support starting the summer before their senior year in high school through the summer after high school graduation. During this time, advisors help students identify and apply to colleges that are a strong match in terms of affordability, competitiveness and fit. The Bottom Line "Access" program advisors also help students apply for financial aid.

Barr & Castleman (2017) reported that Bottom Line operates in three cities in 3 states (Boston, Massachusetts, New York, NY, and Chicago, Illinois). As reported, a unique component of the program is that counselors encourage students to attend a set of target colleges and universities typically in and around the cities where they work that the program has identified as providing students with an optimal combination of quality and affordability. Students who enroll at one of the target institutions (approximately 50 percent of the students in Barr & Castleman, 2017) are eligible for the Bottom Line "Success" program, in which individualized and campus-based support is provided for up to six years following high school.

Through the "Success" program, advisors provide individualized and campus-based support to college students throughout their college enrollment for up to 6 years, with the goal that students will graduate with a Bachelor's degree in 6 years or less. Bottom Line provides ongoing advising during the summer after high school to help students navigate and complete required pre-matriculation tasks, such as attending orientation, completing placement tests, and setting up a tuition payment plan. Campus-based advisors at each target institution continue to meet regularly with students once they have matriculated into college. First-year students meet with advisors an average of 3 to 4 times per semester, and older students meet with an advisor and average of twice a semester. Advisors in the Bottom Line "Success" program provide a combination of academic support (e.g., course selection and making use of advising and tutoring services) and social support (e.g., helping students adjust to a new environment, getting involved with activities and student groups), and they also advise students on how to balance academic, work, social, and family commitments while in college.

Two important contrasts between the Bottom Line model and other advising programs are: 1) Bottom Line employs professional advisors (all have a minimum of a college degree) who work full-time; and 2) the intensity of the model - the average student spends 10-15 hours with their advisor before transitioning to college. Advisors have an average caseload of 50-60 students and meet with each student for an hour every three or four weeks during senior year, at Bottom Line's office in each community.

Outcomes

Barr and Castleman (2017, 2021) found that, as compared to control students, treatment students showed higher rates in:

  • College enrollment
  • College enrollment in 4-year universities
  • College persistence
  • College degree from a 4-year university within 5-6 years of expected high school graduation
  • College degree from a high-quality 4-year university within 5-6 years of expected high school graduation

Brief Evaluation Methodology

Study 1

Barr and Castleman (2017, 2021) conducted a randomized controlled trial to evaluate the effect of Bottom Line on students' college enrollment (overall) and enrollment into 4-year and 2-year colleges. The sample included 2,422 students from 2 cohorts (high school graduating classes of 2015 and 2016) across 2 regions (Boston & Worcester, MA and New York City, NY). Outcome measures were assessed using college enrollment data from the National Student Clearinghouse.

Blueprints Certified Studies

Study 1

Barr, A., & Castleman, B. (2017). The Bottom Line on college counseling. Working Paper.


Risk and Protective Factors

Race/Ethnicity/Gender Details

Race/Ethnicity/Gender Details

In Study 1, Barr & Castleman (2017) showed that college enrollment rates were larger for Hispanic students in the treatment group as compared to Hispanic students in control group. However, Barr & Castleman (2021) reported few differences in treatment impact on student degree attainment across gender, race, or ethnicity.

Training and Technical Assistance

Bottom Line, founded in 1997, began programming in Massachusetts (serving Boston and Worcester) and has since replicated its model in Brooklyn, New York and Chicago, Illinois. While Bottom Line does not sell materials or a training program, it readily engages with organizations and/or community leaders interested in similar and accurate replications. Bottom Line operates with a centralized organizational model which ensures the integrity and consistency of the program. In addition to a National Program Team that leads program design, training, and evaluation, centralized support services are leveraged to oversee the organization's finance, operations, and human resources functions. Regions are led by an Executive Director, who is supported by program and development leadership.

In terms of training, Bottom Line Advisors go through an intensive initial training period upon hire. Following the on-boarding, training is provided on each successive portion of the Bottom Line curriculum and materials. An annual new employee orientation is held each summer which provides both behavioral and technical trainings, including program-specific trainings for all new staff. The National Human Resource department also leads trainings for all staff on core competencies; performance management; Diversity, Equity, and Inclusion; and other topics as needed. These trainings occur in person when possible, or virtually as needed. In addition, regions coordinate their own trainings on topics of interest to their specific staff. Ongoing training and professional development are key factors contributing to Bottom Line's college success rates.

Benefits and Costs

Source: Washington State Institute for Public Policy
All benefit-cost ratios are the most recent estimates published by The Washington State Institute for Public Policy for Blueprint programs implemented in Washington State. These ratios are based on a) meta-analysis estimates of effect size and b) monetized benefits and calculated costs for programs as delivered in the State of Washington. Caution is recommended in applying these estimates of the benefit-cost ratio to any other state or local area. They are provided as an illustration of the benefit-cost ratio found in one specific state. When feasible, local costs and monetized benefits should be used to calculate expected local benefit-cost ratios. The formula for this calculation can be found on the WSIPP website.

Program Costs

Start-Up Costs

Initial Training and Technical Assistance

For organizations and/or communities interested in replication of the Bottom Line model, see Program Information Contact listed on the Fact Sheet.

Curriculum and Materials

No information is available

Licensing

No information is available

Other Start-Up Costs

No information is available

Intervention Implementation Costs

Ongoing Curriculum and Materials

No information is available

Staffing

No information is available

Other Implementation Costs

No information is available

Implementation Support and Fidelity Monitoring Costs

Ongoing Training and Technical Assistance

No information is available

Fidelity Monitoring and Evaluation

No information is available

Ongoing License Fees

No information is available

Other Implementation Support and Fidelity Monitoring Costs

No information is available

Other Cost Considerations

No information is available

Year One Cost Example

Funding Strategies


No information is available

Evaluation Abstract

Program Developer/Owner

Andrew Barr, EvaluatorDepartment of Economics, Texas AM Universityabarr@tamu.edu

Program Outcomes

  • Post Secondary Education

Program Specifics

Program Type

  • Academic Services
  • School - Individual Strategies

Program Setting

  • School
  • Community

Continuum of Intervention

  • Selective Prevention

Program Goals

A college advising program designed to support low-income and first-generation students during the college application process and throughout college enrollment.

Population Demographics

Low-income and first-generation high school students

Target Population

Age

  • Late Adolescence (15-18) - High School

Gender

  • Both

Race/Ethnicity

  • All

Race/Ethnicity/Gender Details

In Study 1, Barr & Castleman (2017) showed that college enrollment rates were larger for Hispanic students in the treatment group as compared to Hispanic students in control group. However, Barr & Castleman (2021) reported few differences in treatment impact on student degree attainment across gender, race, or ethnicity.

Risk/Protective Factors

Risk Factors

Protective Factors


*Risk/Protective Factor was significantly impacted by the program

Brief Description of the Program

Bottom Line provides individualized college advising to high school students starting at the end of their junior year. Eligibility criteria for the program includes: 1) a high school GPA of at least 2.3; 2) family income below 200 percent of the poverty line; and 3) being in the first generation in their family to go to college in the United States. Students are initially admitted into the "Access" program, which provides students with college and financial aid application support starting the summer before their senior year in high school through the summer after high school graduation. During this time, advisors help students identify well-matched colleges and then complete and submit college applications. The Bottom Line "Access" program also helps students evaluate the affordability of different postsecondary options, and then advisors actually help students apply for financial aid. Barr & Castleman (2017) reported that Bottom Line operates in several cities in 3 states (Massachusetts, New York, and Illinois). As reported in Barr & Castleman (2017), a unique component of the program is that counselors encourage students to attend a set of target colleges and universities within New York and Massachusetts that the program has identified as providing students with an optimal combination of quality and affordability. Students who enroll at one of the target institutions (approximately 50 percent of the students in Barr & Castleman, 2017) are eligible for the Bottom Line "Success" program, in which individualized and campus-based support is provided for up to six years following high school. Two important contrasts between the Bottom Line model and other advising programs are: 1) Bottom Line employs professional advisors (all have a minimum of a college degree) who work full-time; and 2) the intensity of the model - the average student spends 10-15 hours with their advisor before transitioning to college. Advisors have an average caseload of 50-60 students and meet with each student for an hour every three or four weeks during senior year, at Bottom Line's office in each community.

Description of the Program

Bottom Line provides individualized college advising to high school students starting at the end of their junior year. Eligibility criteria for the program includes: 1) a high school GPA of at least 2.3; 2) family income below 200 percent of the poverty line; and 3) being in the first generation in their family to go to college in the United States. Students are initially admitted into the "Access" program, which provides students with college and financial aid application support starting the summer before their senior year in high school through the summer after high school graduation. During this time, advisors help students identify and apply to colleges that are a strong match in terms of affordability, competitiveness and fit. The Bottom Line "Access" program advisors also help students apply for financial aid.

Barr & Castleman (2017) reported that Bottom Line operates in three cities in 3 states (Boston, Massachusetts, New York, NY, and Chicago, Illinois). As reported, a unique component of the program is that counselors encourage students to attend a set of target colleges and universities typically in and around the cities where they work that the program has identified as providing students with an optimal combination of quality and affordability. Students who enroll at one of the target institutions (approximately 50 percent of the students in Barr & Castleman, 2017) are eligible for the Bottom Line "Success" program, in which individualized and campus-based support is provided for up to six years following high school.

Through the "Success" program, advisors provide individualized and campus-based support to college students throughout their college enrollment for up to 6 years, with the goal that students will graduate with a Bachelor's degree in 6 years or less. Bottom Line provides ongoing advising during the summer after high school to help students navigate and complete required pre-matriculation tasks, such as attending orientation, completing placement tests, and setting up a tuition payment plan. Campus-based advisors at each target institution continue to meet regularly with students once they have matriculated into college. First-year students meet with advisors an average of 3 to 4 times per semester, and older students meet with an advisor and average of twice a semester. Advisors in the Bottom Line "Success" program provide a combination of academic support (e.g., course selection and making use of advising and tutoring services) and social support (e.g., helping students adjust to a new environment, getting involved with activities and student groups), and they also advise students on how to balance academic, work, social, and family commitments while in college.

Two important contrasts between the Bottom Line model and other advising programs are: 1) Bottom Line employs professional advisors (all have a minimum of a college degree) who work full-time; and 2) the intensity of the model - the average student spends 10-15 hours with their advisor before transitioning to college. Advisors have an average caseload of 50-60 students and meet with each student for an hour every three or four weeks during senior year, at Bottom Line's office in each community.

Theoretical Rationale

Through personalized one-on-one guidance and support provided by advisors, students can achieve their goals of college entrance and graduation.

Brief Evaluation Methodology

Study 1

Barr and Castleman (2017, 2021) conducted a randomized controlled trial to evaluate the effect of Bottom Line on students' college enrollment (overall) and enrollment into 4-year and 2-year colleges. The sample included 2,422 students from 2 cohorts (high school graduating classes of 2015 and 2016) across 2 regions (Boston & Worcester, MA and New York City, NY). Outcome measures were assessed using college enrollment data from the National Student Clearinghouse.

Outcomes (Brief, over all studies)

Study 1

Barr and Castleman (2017) found that treatment students were more likely to be enrolled in college than control students, and they were more likely to enroll in 4-year universities and less likely to enroll in 2-year colleges (compared to control students). Treatment students were also significantly more likely to persist in college (i.e., be enrolled in a second year, have more total enrolled semesters, and be continuously enrolled). Barr & Castleman (2021) extended the finding by showing that treatment students were more likely than control students to attain degrees within 5-6 years from a 4-year college and from a high-quality 4-year college, and they were less likely to attain an associate degree from a 2-year college.

Outcomes

Barr and Castleman (2017, 2021) found that, as compared to control students, treatment students showed higher rates in:

  • College enrollment
  • College enrollment in 4-year universities
  • College persistence
  • College degree from a 4-year university within 5-6 years of expected high school graduation
  • College degree from a high-quality 4-year university within 5-6 years of expected high school graduation

Mediating Effects

In Study 1, Barr & Castleman (2021) reported from a partial mediation analysis that most of the treatment effect on degree attainment at 4-year colleges resulted from advisors shaping student applications and college choices toward higher-quality institutions.

Effect Size

Effect sizes were not reported.

Generalizability

In Study 1, Barr & Castleman (2017, 2021) included low-income and first-generation high school students in Boston and Worcester, MA and New York City, NY.

Potential Limitations

Study 1 (Barr and Castleman, 2017, 2021)

  • Some baseline differences between treatment and control (2 out of 19 variables)
  • Pretest control for college outcomes not possible but used other controls such as high school achievement

Endorsements

Blueprints: Promising
Crime Solutions: Promising
OJJDP Model Programs: Promising
Social Programs that Work:Top Tier
What Works Clearinghouse: Meets Standards Without Reservations - Potentially Positive Effect

Program Information Contact

Steve Colón, Chief Executive Officer
Erin Harris, Director of Data and Evaluation
eharris@bottomline.org

Bottom Line
50 Milk St., 16th Floor
Boston, MA 02109
857-415-4810
www.bottomline.org

References

Study 1

Certified Barr, A., & Castleman, B. (2017). The Bottom Line on college counseling. Working Paper.

Barr, A. C., & Castleman, B. L. (2021). The Bottom Line on College Advising: Large increases in degree attainment (EdWorkingPaper 21-481). Retrieved from Annenberg Institute at Brown University: https://doi.org/10.26300/xdsa-5e22.

Study 1

Evaluation Methodology

Design:

Recruitment: Eligibility for the program and study included having a GPA of at least 2.5 and family income below 200% of the poverty line. To recruit participants, staff actively promoted their program through high schools and non-profit partners in three community sites: Boston, New York City, and Worcester, Massachusetts. A total of 2,422 students joined the study in two separate cohorts (the high school graduating classes in 2014 and 2015). The number of high schools from which students attended was not reported.

Assignment: Students were randomized via a lottery system to treatment (n = 1,687) or control (n = 735) across two regions (Boston-Worcester and New York City) and two cohorts (graduating classes of 2014 and 2015). The uneven condition numbers stemmed from "minimum commitments to its funders and community partners of the number of students it had to serve." Intervention students received an offer to participate in the Bottom Line Access program, while control participants did not receive any services from Bottom Line but were able to access college planning guidance and support from other sources. Some treatment students got Bottom Line "Access" and "Success" and some got just Bottom Line "Access," but all were randomized to one treatment group.

Attrition: The study followed participants from high school to two years after high school in Barr & Castleman (2017) and to five years (for the second cohort) or six years (for the first cohort) after high school in Barr & Castleman (2021). Administrative records on college enrollment were collected on 2,422 students, so there was no attrition. However, the most up-to-date measures in both reports (Barr & Castleman, 2017, 2021) were limited to the first cohort (n = 1429), which started earlier; the second cohort (n = 993) started later but otherwise had no missing data. For the two surveys examined in Barr & Castleman (2021), the completion rate was about 56%.

Sample: Sample characteristics were provided by condition (see Table 1 in Barr & Castleman, 2017). Among the treatment students, 70.1% were female, and the racial/ethnic composition of the students was: 32.4% Black, 31.7% Hispanic, 23.7% Asian, and 9.5% other races. Among the control students, 69.7% were female, and in terms of race/ethnicity, students were 30.2% Black, 32.5% Hispanic, 24.6% Asian, and 9.4% other races. Both treatment and control groups had a first-generation student rate of 81.1%. It appeared that students were low income, as their parents' average gross income was $22,520 a year.

In terms of counselors, most counselors were female (75%), with roughly a quarter black and a quarter Hispanic. The median counselor age was 26 years.

Measures:

Barr & Castleman (2017): Outcome data came from the National Student Clearinghouse, which provided students' term-level college enrollment data into 2-year and 4-year colleges. College entrance (as in the period of time between high school graduation and entrance into college), however, was not defined, although one cohort (2015) had up to 2 years to potentially enroll in college after high school graduation, while the other (2016) had up to 1 year of potential opportunity to enroll in college. College persistence was measured as enrollment in a second year, total enrolled semesters, and continuous enrollment. College quality was measured as enrollment in the student's target college and enrollment in a college with a high graduation rate, high SAT/ACT entrance scores, and a low default rate.

Outcome data were collected from the National Student Clearinghouse (NSC), "with coverage across 96% of college enrollments in the country" (p. 9). Given that no missing outcome data were reported, it appeared that students without a record in the NSC database were assigned a 0, indicating they did not enroll in college. This, however, was not explicitly stated.

Barr & Castleman (2021): Along with measures on enrollment and degree attainment from the National Student Clearinghouse, this report examined the quality of the colleges in which the students enrolled. Participants in both cohorts were followed for five years after high school, and participants in the first cohort were followed for six years. College quality was measured by being a designated Bottom Line target college and by graduation rates, loan default rates, post-graduation earnings, and post-graduation intergenerational "social mobility" that was above the median. In addition, two surveys conducted with students during the spring of their senior year in high school and the fall after high school graduation measured several risk and protective factors related to progress toward college and financial milestones (e.g., completing college applications and financial aid forms).

Analysis: Barr and Castleman (2017, 2021) pooled across cohorts to conduct the analysis. Main effects were tested using linear and logistic regression models, controlling for site-by-cohort fixed effects. Other covariates included demographics, family resources and background, academic aptitude (standardized GPA, state standardized test scores), and measures of college guidance resources (whether the student was working with another advising organization). The models estimated robust standard errors.

Intent-to-Treat: All participants were analyzed according to their assigned condition, which is in line with the intent-to-treat protocol.

Outcomes

Implementation Fidelity:

Barr & Castleman (2017) reported data on dosage. Bottom Line counselor interaction data indicated that 97% of the treatment students had at least one interaction with a counselor. In addition, over a 15-month period, counselors interacted with students an average of 13 times, with the majority of these interactions occurring as in-person meetings in the counselor's office.

Barr & Castleman (2021) further reported that 29 out of 30 advisors had a positive effect on the 4-year enrollment of advisees and that advisor gender or race has no relationship with student success.

Baseline Equivalence:

Tests in Barr & Castleman (2017, 2021) showed significant differences (p < .05) between the treatment and control groups in 2 (out of 19) variables: treatment students were: 1) less likely to be a U.S. citizen, and 2) more likely to have a father who was employed. While the main effects analysis controlled for socio-demographic variables, it was not clear whether these specific variables were included in the model.

Differential Attrition:

The main analysis of college outcomes had no attrition. The attrition rates of the two surveys examined in Barr & Castleman (2021) were roughly equal among treatment and control group students. Tests for baseline equivalence in the analysis sample showed the same two significant differences in 19 tests as for the tests using the full randomized sample (Table A5).

Posttest:

For at least some of the intervention participants, those participating in the Bottom Line Success program for college students, the program was ongoing during the assessment period. Only the measures of degree attainment served as full posttest outcomes.

Barr & Castleman (2017): Results after two years, when the program was ongoing, showed that treatment students were significantly more likely to be enrolled in any college than control students, and the effect was larger when looking at 4-year college enrollment. At the posttest, treatment students were 7% points more likely to enroll in any college than the control students. A larger effect was found on 4-year college enrollment, with a 10%-point increase among treatment students as compared to the control group. In contrast, the program significantly reduced enrollment in 2-year colleges. Additional tests showed that the program significantly increased enrollment in the student's target college and enrollment in colleges with high graduation rates, high SAT/ACT entrance scores, and a low default rate.

Supplemental analyses indicated that treatment effects in college enrollment were larger for Hispanics and individuals with lower high school GPAs.

College persistence was reported for only one cohort (students in the 2014 high school graduating cohort). The results, again during the period when the program was ongoing, indicated that compared to control students, treatment students experienced significantly higher second-year enrollment in any college, second-year enrollment in a 4-year college, total enrolled semesters, and continuous enrollment.

Barr & Castleman (2021): Tests for program main effects replicated the results for enrollment in Barr & Castleman (2017), but the study also presented results on degree attainment over a 5-6-year period. The intervention students had significantly higher degree attainment rates at a 4-year college and significantly higher degree attainment at a college classified as high quality (i.e., having higher graduation rates, loan non-default rates, post-graduation earnings, and post-graduation intergenerational "social mobility" above the median). The program effects on degree attainment at 2-year colleges were negative, as the program diverted students from 2-year to 4-year colleges. Moderation tests found similarly beneficial effects across sites, cohorts, advisors, and student characteristics.

Tests using the survey data to examine the completion of college and financial aid milestones at the early stages of the program found only one significant effect (p < .05): The intervention students completed a larger number of applications.

Additional tests using recently developed causal forest methods found that increases in bachelor's degree attainment were due primarily to shifting students to enroll at higher-quality institutions. Mediation analysis suggested a modest role of ongoing college advising for degree attainment.

Long-Term: Because program students who attended college continued to receive counseling for up to six years after high school, the study did not examine long-term effects.