
Positive Family Support

A family-based, 3-tiered intervention that aims to reduce problem behavior and risk for substance abuse and depression and improve family management practices and communication skills, as well as adolescents' self-regulation skills and prosocial behaviors.
Program Outcomes
- Alcohol
- Depression
- Sexual Risk Behaviors
- Tobacco
Program Type
- Family Therapy
- Parent Training
- School - Individual Strategies
Program Setting
- School
Continuum of Intervention
- Indicated Prevention
- Selective Prevention
- Universal Prevention
Age
- Early Adolescence (12-14) - Middle School
Gender
- Both
Race/Ethnicity
- All
Endorsements
Blueprints: Promising
Crime Solutions: Effective
OJJDP Model Programs: Effective
Program Information Contact
For Training Inquiries:
Marianne Fillhouer, M.A., Ed.S.
REACH Institute
Arizona State University
P.O. Box 876005
Tempe, AZ 85287-6005
Phone: (480) 727-7420
Fax: (480) 965-5430
Email: marianne.fillhouer@asu.edu
Website: reachinstitute.asu.edu/programs/positivefamilysupport
For all Other Inquiries:
Elizabeth Stormshak, Ph.D.
University of Oregon
Eugene, OR 97403
https://fcu.uoregon.edu/
Program Developer/Owner
Drs. Thomas Dishion (deceased) and Elizabeth Stormshak
University of Oregon
Brief Description of the Program
The Positive Family Support program (PFS) is an evidence-based, school intervention system with universal, selected and individualized levels of support. Goals of the program are to increase family to school engagement through developing positive working relationships between caregivers and schools, resulting in improved outcomes for children. PFS (formerly Adolescent Transitions Program and EcoFit) is a three-tiered model with universal, selected, and individualized levels of support. Positive Family Support is designed to enhance schools' current Multi-Tiered Systems of Support (MTSS), such as Positive Behavioral Interventions and Support (PBIS) and Response to Intervention (RtI) by adding a family component at each level of support. At the universal level, a 6-week prevention program called SHAPe (Success, Health, and Peace) is implemented in participating children's homeroom classes. Supports are also available for all school families. These supports include a Family Resource Center (FRC), Universal Screeners for caregivers and for teachers, and Positive Family Outreach. The FRC provides a structured place for school staff and families to collaborate. Norms of protective, prosocial parenting behaviors are set and information on prosocial family management are available to families as are Parenting worksheets to help them practice family management techniques. The selected level includes early, brief interventions in which schools collaborate with caregivers to effect positive behavior change in students where concerns around their school success (i.e., attendance and completion of work) are beginning to emerge. Home incentive-driven monitoring tools are used to engage caregivers in the behavior change process. The third, individualized level involves the School-Based Family Check-Up model. Here, families participate in an interview and assessment session that is followed by a motivational feedback session in which caregivers collaborate with trained school staff to select available intervention programs the family can receive. This level addresses individualized problems through a brief treatment program, which may include academic and social behavior monitoring, parent resources and/or groups, and outside resources. The School-based FCU is also beneficial for students at the Selected Level.
PFS has recently been expanded to target students in grades K-8. However, the effects of expanding the program to additionally include children in kindergarten and elementary grades has not been evaluated. Blueprints has only certified Positive Family Support as effective for youth in middle school.
The Positive Family Support program (PFS) is an evidence-based, school intervention system with universal, selected and individualized components. Goals of the program are to increase family to school engagement through developing positive working relationships between caregivers and schools, resulting in improved outcomes for children. It can be adapted based on the different family needs and the ecology of each school environment, including differing intervention targets and doses that can be most effective to a particular family. This often results in a more effective use of resources and greater treatment compliance. There are three levels to the overall program:
Universal Level of Support - Components at this level include a classroom prevention program, the Family Resource Center (FRC), Universal Screeners for caregivers and teachers, and Positive Family Outreach. A 6-week prevention program called SHAPe (Success, Health, and Peace) is implemented in participating children's homeroom classes. The FRC provides a structured place for school staff and parents to communicate and collaborate, and at least one school staff member is trained in techniques of operating the center. Norms of protective, prosocial parenting behaviors are set and information on prosocial family management are available to families as are Parenting worksheets to help them practice family management techniques. Universal screeners are sent to all families and to teachers. The parent screener allows caregivers to indicate their perspective on the strengths and challenges regarding their student's social, behavioral, and academic well-being. If caregivers indicate concern and wish to be contacted, a meeting is set up in the FRC where caregivers can utilize the resources available or consult with the Family Support Specialist. At this level, all families receive at least one positive contact from the school. Additionally, schools are given strategies and resources for creating additional positive opportunities for all families (e.g. Family Topic Events and Family Event Nights).
Selected Level of Support - At this level, school staff identify and focus on those students who are showing signs of difficulty, academically or behaviorally, and whose needs are not currently being met with universal level supports. The aim is to promote general caregiver involvement/collaboration in making positive changes in their student. Teachers are trained to make early contact with a student's caregiver when there are concerns about their student's attendance, behavior, or completion of work (ABCs). There are resources for teachers to use to support caregiver involvement in student interventions, which are simple and brief. Another component at this level is the development of student support teams to identify and track students in need of additional support. Finally, a behavior intervention system is used to target specific behaviors with effective interventions. This system engages caregivers to work with the school to help students meet their behavioral goals. It also enables schools to document student progress, track behavior change, and involve families in the behavior change process by using home-based incentives.
Individualized Level of Support - The third level of this program reaches students who are struggling and have not responded to less intensive interventions at the selected level, and/or students whose behavioral concerns are beyond the resources available at the lower levels. An adaptation of the behavioral health model Family Check-Up is utilized at this level. The School-based FCU includes a Getting to Know You interview, a Family Questionnaire, and a Motivational Feedback session. The FCU seeks to boost parent motivation and engage families in interventions. During the interview session, trained school staff explore caregiver concerns and stage of change and encourage parents to engage in a family assessment. The assessment consists of a questionnaire completed by caregivers about student and parenting behaviors. The feedback session allows staff to review the results of the assessment and collaborate with caregivers to select available intervention programs the family can receive. Finally, a menu of options consisting of parent support materials (worksheets, brochures, and videos), parent topic events/groups, student intervention supports, and community referrals is explored based on the assessment results. The School-based FCU (interview, questionnaire and feedback session) is designed to be completed in 1.5 hours. The School-based FCU is also beneficial for students at the Selected Level.
Outcomes
Significant program outcomes among intervention youth compared to the control group include:
- Reductions in substance use (cigarettes and alcohol) by ninth grade (Connell, Dishion, & Deater-Deckard, 2006; Dishion, Nelson & Kavanagh, 2003).
- Reductions in depression by ninth grade among high-risk youth, with significant differences in the number of youths scoring in the clinical range, both for youth and mother reports (Connell & Dishion, 2008).
- Improvements in overall youth self-regulation, with small to medium effect sizes in associations between seventh grade self-regulation and youth depressive symptoms from sixth to eighth grade (Stormshak, Fosco, & Dishion, 2010).
- Reduced growth in school absenteeism and reduced decline in GPA from grades 6 to 9 among high-risk youth (Stormshak, Connell, & Dishion, 2009).
- Reduced antisocial behavior by age 18 and parent conflict through adolescence (Connell et al., 2007; Van Ryzin & Dishion, 2012).
- Reduced arrests for adolescent onset offenders (Connell et al., 2011).
- Increases in effortful control (self-regulation), which in turn has significant associations with less growth in antisocial behavior, deviant friendships, and substance use over time (Fosco et al., 2013).
- Indirect program effects on high risk sexual behavior by age 22 (Caruthers et al., 2014), mediated by family relationship quality.
Significant Program Effects on Risk and Protective Factors:
- Increased parent monitoring (Connell, Dishion, & Deater-Deckard, 2006; Dishion, Nelson & Kavanagh, 2003).
Compared to the control group, Connell et al. (2017) found no significant intervention effects at ages 28-30 on past-year or lifetime major depressive disorder diagnoses.
Brief Evaluation Methodology
The initial program development study randomized 119 youth to two components of the indicated level of the ATP, a peer group and parent group intervention. This study revealed that the family component was the most effective for reducing risk for delinquency and drug use. The peer component led to iatrogenic effects on delinquency and drug use over a three year period. Following the initial component study, the ATP was rendered as a three-tiered family-centered program. A large-scale study conducted with random assignment and multiple outcome measures and one replication study of the Adolescent Transitions Program have been evaluated. The main study, Project Alliance 1, conducted in three public middle schools in the Pacific Northwest, evaluated two cohorts of adolescents (672 students in cohort 1 and 998 adolescents in the combined cohort 1 and 2 studies) who were randomly assigned at the individual level to either ATP (intervention condition) or school as usual (comparison condition). Youth were assessed each year beginning in sixth grade, with the long-term follow-up surveying youth at age 22, for a total of 7 waves of data collection. Several outcome measures were assessed, including self-reported alcohol, tobacco, and marijuana use, antisocial behavior, depression, and academic achievement and school attendance. Youth self-reported on deviant peer involvement, parental monitoring, and family conflict as variables predicting initial involvement in problem behavior and engagement in treatment. Both intent-to-treat and complier average causal effect (CACE) analyses were conducted on outcome measures. A second replication study, Project Alliance 2, took place in three urban middle schools with 377 6th-9th grade adolescents and their families and used intent-to-treat analysis.
Connell et al. (2017) randomized 6th grade students in three schools to intervention and control groups and studied the long-term outcomes of the combined cohort sample (n = 998) in early adulthood (ages 28-30). The outcomes included past-year and lifetime depression diagnoses.
Study 1
Connell, A. M., & Dishion, T. J. (2008). Reducing depression among at-risk early adolescents: Three-year effects of a family-centered intervention embedded within schools. Journal of Family Psychology, 22, 574-585.
Connell, A. M., Dishion, T. J., & Deater-Deckard, K. (2006). Variable- and person-centered approaches to the analysis of early adolescent substance use: Linking peer, family, and intervention effects with developmental trajectories. Merrill-Palmer Quarterly, 52, 421-448.
Risk Factors
Individual: Early initiation of antisocial behavior, Favorable attitudes towards antisocial behavior, Favorable attitudes towards drug use, Rebelliousness, Stress, Substance use
Peer: Interaction with antisocial peers, Peer substance use
Family: Family conflict/violence, Parental attitudes favorable to antisocial behavior, Parental attitudes favorable to drug use, Poor family management*
School: Low school commitment and attachment
Protective Factors
Individual: Coping Skills, Problem solving skills
Peer: Interaction with prosocial peers
Family: Attachment to parents, Opportunities for prosocial involvement with parents, Parental involvement in education, Parent social support, Rewards for prosocial involvement with parents
School: Opportunities for prosocial involvement in education, Rewards for prosocial involvement in school
*
Risk/Protective Factor was significantly impacted by the program
Gender Specific Findings
- Male
- Female
Race/Ethnicity Specific Findings
- White
- Hispanic or Latino
- African American
Race/Ethnicity/Gender Details
The sample was roughly half female, 40% African American and 60% European American. In Cohort 1 studies, there were no gender or ethnicity differences.Training includes a readiness assessment and meeting (virtual or in-person) with school administrators and key school personnel. Follow-up PFS trainings range from 2-3 days total and involve various school staff. The Tier 1 (Universal) training session includes as many school staff as possible, including teachers, administrators, and support staff. Subsequent trainings (Tier 2: Selected and Tier 3: Individualized) include school participants that are going to be implementing more targeted interventions, including a modified version of Family Check-Up for schools. The training format includes didactic presentation of the model with digital taped and live role-modeled examples of the specific clinical components, strategies and techniques, as well as small-group activities and planning time for school teams. Role-plays are used to give experiential practice and assist participants in gaining clinical feedback and support to begin to gain mastery of the skills involved. The training content is the same if the training is directly for a particular school but may be broken up differently depending on the resources such as current fiscal and logistic contingencies in any particular school.
Program Benefits (per individual):
$9,064
Program Costs (per individual):
$46
Net Present Value (Benefits minus Costs, per individual):
$9,018
Measured Risk (odds of a positive Net Present Value):
70%
Source: Washington State Institute for Public Policy
All benefit-cost ratios are the most recent estimates published by The Washington State Institute for Public Policy for Blueprint programs implemented in Washington State. These ratios are based on a) meta-analysis estimates of effect size and b) monetized benefits and calculated costs for programs as delivered in the State of Washington. Caution is recommended in applying these estimates of the benefit-cost ratio to any other state or local area. They are provided as an illustration of the benefit-cost ratio found in one specific state. When feasible, local costs and monetized benefits should be used to calculate expected local benefit-cost ratios. The formula for this calculation can be found on the WSIPP website.
Start-Up Costs
Initial Training and Technical Assistance
For the school-based Positive Family Support Program (formerly Adolescent Transitions Program, Ecofit) with follow-up modules, the first year training costs are approximately $15,000 exclusive of trainer travel costs. This training cost can be shared among multiple school sites as the cost is based on number of participants and not the number of school sites.
Travel expenses for trainers (airfare, hotel, and per diem) are approximately $2,500 for on-site training. However, these expenses are billed at actual cost.
Curriculum and Materials
The School-Based Positive Family Support three-tiered program manual and materials, including videotape examples for parents, are included as part of the initial training costs.
Website access (in development) and electronic materials are also included as is electronic access to surveys and screeners that are part of PFS.
Optional: Everyday Parenting: $18.95-Research Press
Licensing
None.
Other Start-Up Costs
For Positive Family Support program, setup for a Family Resource Room/Space within a school can be a cost consideration; it is generally $2,000-$4,000.
Intervention Implementation Costs
Ongoing Curriculum and Materials
Once training and website access (in development) are purchased, materials are free to be used as necessary within a trained program and/or site for the first year following training. Typical printing costs are incurred depending on numbers. Beyond year one, costs for continued access to online materials and assessments is $600 per year.
Staffing
Ideal staffing requirements for the PFS Family Support Specialist generally involve a .50 - 1.0 staff allocation. Many schools also train additional staff to use the program. However, many schools absorb this allocation within other funded positions.
Other Implementation Costs
If a state has business or licensing fees that are assessed, these costs are fully borne by the agency that is requesting program implementation services.
Implementation Support and Fidelity Monitoring Costs
Ongoing Training and Technical Assistance
First-year consultation services for PFS are included in the first-year implementation costs. These services include virtual consultation (via webinar or telephone conference) up to 10 hours per year.
Additional consultation and technical assistance can be purchased for an additional fee.
Fidelity Monitoring and Evaluation
At the current time, schools complete self-evaluations to assess adherence to the PFS program. These self-assessments are included in the manual and materials. Further development of fidelity monitoring and evaluation is in process.
Ongoing License Fees
No information is available
Other Implementation Support and Fidelity Monitoring Costs
No information is available
Other Cost Considerations
No information is available
Year One Cost Example
The following Year 1 costs are based on training which can include multiple school sites (training costs based on number of participants). The implementation services listed below include 2-3 days of training, manual and materials (paper and electronic), website subscription (in development), and consultation. An additional expense, not included in this example, is the cost of furnishing a family resource room whenever such space is not already available within a school.
Readiness Process and Implementation Services (based on number of training participants) | $15,000.00 |
Trainer Travel | $2,500.00 |
Total One Year Cost | $17,500.00 |
The total cost of implementing the Positive Family Support Program in Year One is $17,500 and this cost may be shared among school sites.
Funding Overview
No information is available
Allocating State or Local General Funds
States may have funds for school improvement and/or as part of state or district level ongoing MTTS or SW-PBS implementation and training. Title I funds could also be used for PFS as it has been shown to increase both family engagement and academic achievement. In addition, some states or counties may have mental health or prevention funds that can be used to support PFS implementations.
Maximizing Federal Funds
Entitlements: Mental health agencies may be able to use Medicaid monies for the funding of these programs, depending on what a particular state allows to be billed under their Medicaid program.
Formula Funds potential funding sources:
- Juvenile Accountability Block Grants (JABG) under Office of Juvenile Justice and Delinquency Prevention, Department of Justice
- Mental Health Services Block Grants (MHSBG) within the US Department of Health and Human Services, SAMHSA
Foundation Grants and Public-Private Partnerships
When available, foundations can provide funding for start-up training and support for certification.
Generating New Revenue
Fee for service can be used to support ongoing PFS.
Program Developer/Owner
Drs. Thomas Dishion (deceased) and Elizabeth StormshakUniversity of Oregon
Program Outcomes
- Alcohol
- Depression
- Sexual Risk Behaviors
- Tobacco
Program Specifics
Program Type
- Family Therapy
- Parent Training
- School - Individual Strategies
Program Setting
- School
Continuum of Intervention
- Indicated Prevention
- Selective Prevention
- Universal Prevention
Program Goals
A family-based, 3-tiered intervention that aims to reduce problem behavior and risk for substance abuse and depression and improve family management practices and communication skills, as well as adolescents' self-regulation skills and prosocial behaviors.
Population Demographics
The Positive Family Support program (formerly Adolescent Transitions Program and EcoFit) is a three-tiered program that is designed to address three levels of student and family need. Level One is the universal program targeting all middle-school aged youths and their caregivers. Level Two is the selected program targeting at-risk youths and their families. Level Three addresses indicated youth by utilizing assessments and motivational interviewing to engage students and families into empirically supported treatment programs. Implementation research tracked samples of multiethnic youth from grades 6 through 9.
Target Population
Age
- Early Adolescence (12-14) - Middle School
Gender
- Both
Gender Specific Findings
- Male
- Female
Race/Ethnicity
- All
Race/Ethnicity Specific Findings
- White
- Hispanic or Latino
- African American
Race/Ethnicity/Gender Details
The sample was roughly half female, 40% African American and 60% European American. In Cohort 1 studies, there were no gender or ethnicity differences.
Risk/Protective Factor Domain
- Individual
- School
- Peer
- Family
Risk/Protective Factors
Risk Factors
Individual: Early initiation of antisocial behavior, Favorable attitudes towards antisocial behavior, Favorable attitudes towards drug use, Rebelliousness, Stress, Substance use
Peer: Interaction with antisocial peers, Peer substance use
Family: Family conflict/violence, Parental attitudes favorable to antisocial behavior, Parental attitudes favorable to drug use, Poor family management*
School: Low school commitment and attachment
Protective Factors
Individual: Coping Skills, Problem solving skills
Peer: Interaction with prosocial peers
Family: Attachment to parents, Opportunities for prosocial involvement with parents, Parental involvement in education, Parent social support, Rewards for prosocial involvement with parents
School: Opportunities for prosocial involvement in education, Rewards for prosocial involvement in school
*Risk/Protective Factor was significantly impacted by the program
See also: Positive Family Support Logic Model (PDF)
Brief Description of the Program
The Positive Family Support program (PFS) is an evidence-based, school intervention system with universal, selected and individualized levels of support. Goals of the program are to increase family to school engagement through developing positive working relationships between caregivers and schools, resulting in improved outcomes for children. PFS (formerly Adolescent Transitions Program and EcoFit) is a three-tiered model with universal, selected, and individualized levels of support. Positive Family Support is designed to enhance schools' current Multi-Tiered Systems of Support (MTSS), such as Positive Behavioral Interventions and Support (PBIS) and Response to Intervention (RtI) by adding a family component at each level of support. At the universal level, a 6-week prevention program called SHAPe (Success, Health, and Peace) is implemented in participating children's homeroom classes. Supports are also available for all school families. These supports include a Family Resource Center (FRC), Universal Screeners for caregivers and for teachers, and Positive Family Outreach. The FRC provides a structured place for school staff and families to collaborate. Norms of protective, prosocial parenting behaviors are set and information on prosocial family management are available to families as are Parenting worksheets to help them practice family management techniques. The selected level includes early, brief interventions in which schools collaborate with caregivers to effect positive behavior change in students where concerns around their school success (i.e., attendance and completion of work) are beginning to emerge. Home incentive-driven monitoring tools are used to engage caregivers in the behavior change process. The third, individualized level involves the School-Based Family Check-Up model. Here, families participate in an interview and assessment session that is followed by a motivational feedback session in which caregivers collaborate with trained school staff to select available intervention programs the family can receive. This level addresses individualized problems through a brief treatment program, which may include academic and social behavior monitoring, parent resources and/or groups, and outside resources. The School-based FCU is also beneficial for students at the Selected Level.
PFS has recently been expanded to target students in grades K-8. However, the effects of expanding the program to additionally include children in kindergarten and elementary grades has not been evaluated. Blueprints has only certified Positive Family Support as effective for youth in middle school.
Description of the Program
The Positive Family Support program (PFS) is an evidence-based, school intervention system with universal, selected and individualized components. Goals of the program are to increase family to school engagement through developing positive working relationships between caregivers and schools, resulting in improved outcomes for children. It can be adapted based on the different family needs and the ecology of each school environment, including differing intervention targets and doses that can be most effective to a particular family. This often results in a more effective use of resources and greater treatment compliance. There are three levels to the overall program:
Universal Level of Support - Components at this level include a classroom prevention program, the Family Resource Center (FRC), Universal Screeners for caregivers and teachers, and Positive Family Outreach. A 6-week prevention program called SHAPe (Success, Health, and Peace) is implemented in participating children's homeroom classes. The FRC provides a structured place for school staff and parents to communicate and collaborate, and at least one school staff member is trained in techniques of operating the center. Norms of protective, prosocial parenting behaviors are set and information on prosocial family management are available to families as are Parenting worksheets to help them practice family management techniques. Universal screeners are sent to all families and to teachers. The parent screener allows caregivers to indicate their perspective on the strengths and challenges regarding their student's social, behavioral, and academic well-being. If caregivers indicate concern and wish to be contacted, a meeting is set up in the FRC where caregivers can utilize the resources available or consult with the Family Support Specialist. At this level, all families receive at least one positive contact from the school. Additionally, schools are given strategies and resources for creating additional positive opportunities for all families (e.g. Family Topic Events and Family Event Nights).
Selected Level of Support - At this level, school staff identify and focus on those students who are showing signs of difficulty, academically or behaviorally, and whose needs are not currently being met with universal level supports. The aim is to promote general caregiver involvement/collaboration in making positive changes in their student. Teachers are trained to make early contact with a student's caregiver when there are concerns about their student's attendance, behavior, or completion of work (ABCs). There are resources for teachers to use to support caregiver involvement in student interventions, which are simple and brief. Another component at this level is the development of student support teams to identify and track students in need of additional support. Finally, a behavior intervention system is used to target specific behaviors with effective interventions. This system engages caregivers to work with the school to help students meet their behavioral goals. It also enables schools to document student progress, track behavior change, and involve families in the behavior change process by using home-based incentives.
Individualized Level of Support - The third level of this program reaches students who are struggling and have not responded to less intensive interventions at the selected level, and/or students whose behavioral concerns are beyond the resources available at the lower levels. An adaptation of the behavioral health model Family Check-Up is utilized at this level. The School-based FCU includes a Getting to Know You interview, a Family Questionnaire, and a Motivational Feedback session. The FCU seeks to boost parent motivation and engage families in interventions. During the interview session, trained school staff explore caregiver concerns and stage of change and encourage parents to engage in a family assessment. The assessment consists of a questionnaire completed by caregivers about student and parenting behaviors. The feedback session allows staff to review the results of the assessment and collaborate with caregivers to select available intervention programs the family can receive. Finally, a menu of options consisting of parent support materials (worksheets, brochures, and videos), parent topic events/groups, student intervention supports, and community referrals is explored based on the assessment results. The School-based FCU (interview, questionnaire and feedback session) is designed to be completed in 1.5 hours. The School-based FCU is also beneficial for students at the Selected Level.
Theoretical Rationale
The program is based on the social interactional model, which states that global traits (anti- or pro-social personality, for example) are attributed to people based on their patterns of social exchange with parents and peers, as the orderliness of these interpersonal exchanges are related to behavioral shifts. Based also on coercion theory, which posits that a child's interpersonal style is learned primarily within the family, the program aims to correct coercive family exchanges that lead to antisocial traits and maladjustment in other social contexts. The program involves the development of treatment technology for parents and use of naturalistic, home observation of children.
Theoretical Orientation
- Social Learning
Brief Evaluation Methodology
The initial program development study randomized 119 youth to two components of the indicated level of the ATP, a peer group and parent group intervention. This study revealed that the family component was the most effective for reducing risk for delinquency and drug use. The peer component led to iatrogenic effects on delinquency and drug use over a three year period. Following the initial component study, the ATP was rendered as a three-tiered family-centered program. A large-scale study conducted with random assignment and multiple outcome measures and one replication study of the Adolescent Transitions Program have been evaluated. The main study, Project Alliance 1, conducted in three public middle schools in the Pacific Northwest, evaluated two cohorts of adolescents (672 students in cohort 1 and 998 adolescents in the combined cohort 1 and 2 studies) who were randomly assigned at the individual level to either ATP (intervention condition) or school as usual (comparison condition). Youth were assessed each year beginning in sixth grade, with the long-term follow-up surveying youth at age 22, for a total of 7 waves of data collection. Several outcome measures were assessed, including self-reported alcohol, tobacco, and marijuana use, antisocial behavior, depression, and academic achievement and school attendance. Youth self-reported on deviant peer involvement, parental monitoring, and family conflict as variables predicting initial involvement in problem behavior and engagement in treatment. Both intent-to-treat and complier average causal effect (CACE) analyses were conducted on outcome measures. A second replication study, Project Alliance 2, took place in three urban middle schools with 377 6th-9th grade adolescents and their families and used intent-to-treat analysis.
Connell et al. (2017) randomized 6th grade students in three schools to intervention and control groups and studied the long-term outcomes of the combined cohort sample (n = 998) in early adulthood (ages 28-30). The outcomes included past-year and lifetime depression diagnoses.
Outcomes (Brief, over all studies)
There was a significant program effect on reductions in grade 9 substance use. There was no significant difference between at-risk youth and typically developing youth, indicating that the program produced significant effects regardless of risk-level. Effects also did not vary by gender or ethnicity. For the identified high-risk sample that participated in Level Two programming (selected Family Check-Up intervention), intervention families decreased in parental monitoring from seventh to eighth grade, but increased monitoring between eighth and ninth grade. Control families decreased parental monitoring slightly from seventh to eighth grade and continued to decrease monitoring more sharply from eighth to ninth grade. After controlling for prior substance use, the intervention condition was associated with a lower probability of substance use. High-risk youth in the control group were more likely to report substance use than intervention youth. There were also significant program effects for treatment youth who engaged in the intervention on arrest rates, likelihood of being diagnosed with lifetime alcohol, tobacco, or marijuana use, long-term tobacco use, depression, internalizing symptoms, and school absences and GPA.
Characteristics of youth and families most likely to engage in treatment: Treatment engagement was significantly related to the likelihood of biological fathers being absent from the home, youth reports of elevated family conflict and deviant peer affiliation, and teacher reports of elevated risk behaviors at school. This means that families who had the highest level of vulnerability towards negative outcomes and experienced the most difficulty were more likely to engage in treatment. Female gender also predicted higher level of engagement.
For the identified high-risk sample that participated in Level Two programming (selected Family Check-Up intervention), significant program outcomes found among treatment youth who engaged in the treatment vs. treatment youth who did not engage in treatment include:
- Inhibited growth in substance use trajectories by age 17.
- Lower arrest rates by late adolescence.
- Lower likelihood of being diagnosed with lifetime alcohol, tobacco, or marijuana use by late adolescence.
- Less growth in tobacco use by age 22.
- Impact of program on substance use greater for girls than boys.
- Less growth in self-reported depressive symptoms.
- Less growth in mother-reported internalizing symptoms.
- Reductions in growth of school absences and declines in GPA from 6th through 11th grades.
Replication: Replication results indicate significant program effects on overall youth self-regulation, as well as an association between seventh grade self-regulation and decreases in youth depressive symptoms from sixth to eighth grade, and eighth grade self-regulation and school engagement by ninth grade. In addition, participation in the Family Check-Up results in associations between increases in self-regulation and the prevention of problem behavior (antisocial behavior and involvement with deviant peers) and substance use (cigarettes, alcohol, and marijuana).
Long-term follow-up on depression diagnoses: Connell et al. (2017) found no significant effects of the intervention on either past-year major depressive disorder diagnoses or lifetime diagnoses, measured at age 28-30.
Outcomes
Significant program outcomes among intervention youth compared to the control group include:
- Reductions in substance use (cigarettes and alcohol) by ninth grade (Connell, Dishion, & Deater-Deckard, 2006; Dishion, Nelson & Kavanagh, 2003).
- Reductions in depression by ninth grade among high-risk youth, with significant differences in the number of youths scoring in the clinical range, both for youth and mother reports (Connell & Dishion, 2008).
- Improvements in overall youth self-regulation, with small to medium effect sizes in associations between seventh grade self-regulation and youth depressive symptoms from sixth to eighth grade (Stormshak, Fosco, & Dishion, 2010).
- Reduced growth in school absenteeism and reduced decline in GPA from grades 6 to 9 among high-risk youth (Stormshak, Connell, & Dishion, 2009).
- Reduced antisocial behavior by age 18 and parent conflict through adolescence (Connell et al., 2007; Van Ryzin & Dishion, 2012).
- Reduced arrests for adolescent onset offenders (Connell et al., 2011).
- Increases in effortful control (self-regulation), which in turn has significant associations with less growth in antisocial behavior, deviant friendships, and substance use over time (Fosco et al., 2013).
- Indirect program effects on high risk sexual behavior by age 22 (Caruthers et al., 2014), mediated by family relationship quality.
Significant Program Effects on Risk and Protective Factors:
- Increased parent monitoring (Connell, Dishion, & Deater-Deckard, 2006; Dishion, Nelson & Kavanagh, 2003).
Compared to the control group, Connell et al. (2017) found no significant intervention effects at ages 28-30 on past-year or lifetime major depressive disorder diagnoses.
Mediating Effects
Results of a mediated model show that increases in parental monitoring significantly account for reductions in substance use in ninth grade, controlling for previous levels of substance use and random assignment. There was also a significant effect of family relationship quality on HRSB mediated by differences in parental monitoring and early sexual activity (Caruthers et al., 2014) at age 22. The effects of the mediators were varied based on gender and race.
Effect Size
One of the key studies (Connell & Dishion, 2008) presented Cohen's d effect sizes. The program had medium effects on growth of depressive symptoms from 7th to 9th grade (d = .56) and on depressive symptoms in 9th grade (d = .42). In separate analyses of a compliance subsample rather than the intent to treat sample, effect sizes became much stronger, reaching 1.35 for growth of depressive symptoms and 1.07 for youth internalizing.
Generalizability
Generalizability is solid because of the use of random assignment, the stability of effects across gender, ethnicity and risk-level, and the large and ethnically diverse sample. There is no information available regarding the effectiveness of the program in rural areas.
Potential Limitations
The initial study (Dishion et al., 2002) had missing data for 216 children that reduced the sample size from 672 to 460, indicating only a 68% retention rate. Checks on a model with fewer variables and 13% missing data gave the same results, but it appears from footnote 3 that subjects with missing data had higher substance use. Dishion, Nelson, & Kavanagh (2003) examined the results of the high-risk sample (that received Family Check-Up). Of 150 families with high-risk children, 71 agreed to participate, and tests for program effects included 47 of the 71 subjects (or 34% missing data). It is unknown if estimates are biased because of the missing data or differential attrition. Also in this study, there appears to be non-equivalence across conditions in the high-risk sample, given the small numbers. The intervention went to those who were best able to benefit from it (e.g., females and those less likely to be on food stamps). Since there is a greater willingness to monitor girls than boys, this opens the possibility that the observed effects are related to differential prevalence of girls vs. boys in the high-risk sample (see Dishion, Nelson, & Kavanagh, 2003). Last, tests in Dishion, Nelson, and Kavanagh were one-tailed, eliminating the possibility of finding iatrogenic effects.
There are other questions from the two studies of the revised program. First, there is a discrepancy in the two papers, with the 2002 paper stating that 85% completed all assessments from grade 6-9, but the 2003 paper stating 71% completed all assessments. Second, in the Dishion et al. (2002, 2003) papers, 71 high-risk youth ultimately participated in the Family Checkup Intervention component; however, in one study 137 youth were identified as high-risk using the TRISK and in the other study 150 were identified.
In Connell and Dishion (2008), depressive symptoms were assessed only during the course of the yearly assessments for high-risk youth, and not around the time of the parent completion of FCU, which limited the potential for higher intervention effects that followed immediately after intervention sessions.
Limitations noted in the academic outcomes study (Stormshak et al., 2009) include the inability to test the mediating impact of the program on parenting strategies such as homework monitoring, implementing skills to alleviate family conflict, and positive parenting. Also, some of the indices used (i.e., paternal presence and family conflict) were broad and researchers were unable to identify more sensitive family configuration and more exact measures of types of family conflict. Finally, a change in the school district's record keeping precluded the collection of 11th grade GPA for the second cohort of the study. However, since there were no significant differences found in the cohort data at earlier waves of data collection, there is a reasonable assumption that the intervention effects would not have changed significantly with the additional data.
Limitations noted in the long-term tobacco use study (Connell, 2009) include ignoring the nesting of families within schools, which may have underestimated standard errors. However, the problem appears minor, as examination of nesting effects by schools yielded extremely small intraclass correlations for either engagement rates or tobacco use outcomes. Also, the intervention had no influence for the intent-to-treat sample, while the researchers indicated that CACE results are susceptible to bias in the face of violations of the exclusion restriction. Finally, engagement with treatment was considered as a binary variable instead of a continuous variable, which limited the ability to examine any finer relationship between treatment engagement and youth outcomes.
Connell et al (2017):
- Unclear if interviewers providing the diagnosis were blind to condition
- No controls for baseline depression
- No tests for baseline equivalence reported
- No tests for differential attrition
- No long-term intervention effects on outcome (depression)
Notes
As an upstream preventive intervention, this program targets and reduces problem behaviors that are associated with increased risk of developing substance use disorder or opioid use disorder later in life.
The primary studies qualifying for Blueprints were Connell, Dishion, and Deater-Deckard, 2006; and Connell and Dishion, 2008.
Endorsements
Blueprints: Promising
Crime Solutions: Effective
OJJDP Model Programs: Effective
Program Information Contact
For Training Inquiries:
Marianne Fillhouer, M.A., Ed.S.
REACH Institute
Arizona State University
P.O. Box 876005
Tempe, AZ 85287-6005
Phone: (480) 727-7420
Fax: (480) 965-5430
Email: marianne.fillhouer@asu.edu
Website: reachinstitute.asu.edu/programs/positivefamilysupport
For all Other Inquiries:
Elizabeth Stormshak, Ph.D.
University of Oregon
Eugene, OR 97403
https://fcu.uoregon.edu/
References
Study 1
Connell, A. M., & Dishion, T. J. (2017). Long-term effects of the family check-up in public secondary school on diagnosed major depressive disorder in adulthood. Journal of Youth and Adolescence, 46(3), 570-581.
Caruthers, A. S., Van Ryzin, M. J., & Dishion, T. J. (2014). Preventing high-risk sexual behavior in early adulthood with family interventions in adolescence: Outcomes and developmental processes. Prevention Science, 15(Supplement 1), S59-S69.
Connell, A. (2009). Employing complier average causal effect analytic methods to examine effects of randomized encouragement trials. The American Journal of Drug and Alcohol Abuse, 35, 253-259.
Certified Connell, A. M., & Dishion, T. J. (2008). Reducing depression among at-risk early adolescents: Three-year effects of a family-centered intervention embedded within schools. Journal of Family Psychology, 22, 574-585.
Certified Connell, A. M., Dishion, T. J., & Deater-Deckard, K. (2006). Variable- and person-centered approaches to the analysis of early adolescent substance use: Linking peer, family, and intervention effects with developmental trajectories. Merrill-Palmer Quarterly, 52, 421-448.
Connell, A. M., Dishion, T. J., Yasui, M., & Kavanagh, K. (2007). An adaptive approach to family intervention: Linking engagement in family-centered intervention to reductions in adolescent problem behavior. Journal of Consulting and Clinical Psychology, 75, 568-579.
Connell, A., M., Dishion, T. J., & Klostermann, S. (2012). Family Check Up effects on adolescent arrest trajectories: Variation by developmental subtype. Journal of Research on Adolescence, 22(2), 367-380.
Dishion, T. J., Kavanagh, K., Schneiger, A., Nelson, S., & Kaufman, N. K. (2002). Preventing early adolescent substance use: A family-centered strategy for the public middle school. Prevention Science, 3(3), 191-201.
Dishion, T. J., Nelson, S. E., & Kavanagh, K. (2003). The Family Check-up with high-risk young adolescents: Preventing early-onset substance use by parent monitoring. Behavior Therapy, 34, 553-571.
Stormshak, E. A., Connell, A., & Dishion, T. J. (2009). An adaptive approach to family-centered intervention in schools: Linking intervention engagement to academic outcomes in middle and high school. Prevention Science, 10, 221-235.
Study 2
Stormshak, E. A., Fosco, G. M., & Dishion, T. J. (2010). Implementing intervention with families in schools to increase youth school engagement: The Family Check-up model. School Mental Health, 2, 82-92.
Study 3
Fosco, G. M., Frank, J. L., Stormshak, E. A., & Dishion, T. J. (2013). Opening the "Black Box": Family Check-Up intervention effects on self-regulation that prevents growth in problem behavior and substance use. Journal of School Psychology, 51, 455-468.
Study 4
Smolkowski, R. R., Gau, J. M., Dishion, T. J., Stormshak, E. A., Moore, K. J., Falkenstein, C. A., … Garbacz, S.A. (2017). Effectiveness evaluation of the Positive Family Support intervention: A three-tiered public health delivery model for middle schools. Journal of School Psychology, 62, 103-125.
Study 1
Evaluation Methodology
Cohort 1
Design (Dishion et al., 2002; Dishion, Nelson, and Kavanagh, 2003):
Recruitment: Using the revised model of the Adolescent Transitions Program (ATP), 672 students and their families were recruited from a population of sixth grade students from three middle schools in a metropolitan area of the Pacific Northwest. School principals distributed letters to parents to introduce the study.
Assignment: Most (95%) of the population gave consent and were randomized, by student, to an experimental or control group. In 7th grade, all experimental students were assigned to the same homerooms so the SHAPe program could be implemented. Parents of intervention youths were also targeted for Level Two (Family Check-Up) programming.
Attrition: Eighty-five percent participated in all assessment waves, which occurred yearly from grades 6 through 9. However, the sample was reduced by 216 children due to missing data. This N of 460 represents a 68% retention rate. Students were assessed through mail and in the classroom and were paid $20 for completing each of the first 3 waves and $30 for completing the fourth assessment. Teacher-ratings were collected as was information from the Parent Consultant on parent and/or family involvement.
Risk Group Formation: Researchers utilized a multiple-gating procedure to identify participants for the second and third program levels. Teacher-ratings of risk, using a 16-item measure, were scored to define an at-risk (AR) level and a high-risk level for each individual. These items were put into a hierarchical cluster analysis using squared Euclidean distance as the index of similarity. Two groups emerged: at-risk (AR) and typically developing (TD). One hundred thirty-seven were identified post hoc as AR; 71 (22.1%) were in the treatment group and 66 (20.4%) were in the control group.
Sample:
The schools were located in an ethnically diverse metropolitan area in the Northwest. For the overall sample of 672 students, 52% of both the experimental and control groups were male and 41% of both groups were European American. Of the experimental group, 34% were African American and 6.9% were Hispanic, compared to 30.5% and 7.6% of the control group, respectively. There were no differences between groups in terms of percentage of AR students.
As reported in a 2003 publication that evaluates the Family Check-Up level with a randomized design, treatment and control youth were different on some demographic variables: 31% of FCU youth were male versus 47% of controls; 26% of FCU youth were European American versus 39% of controls; 57% of FCU youth were African American, 6% were Hispanic, and 11% were mixed, compared to 44%, 0%, and 17% of controls, respectively; 28% of FCU youths were from married families compared to 39% of controls. The average number of children was 2.7 for FCU families (2.6 for controls). Fourteen percent of FCU families received Food Stamps compared to 25% of controls. The authors report these differences were not statistically reliable but no statistical information was presented.
A high-risk subset of this population was used to examine parental monitoring as a mediator of substance use outcomes (Dishion et al., 2003). The 150 youth were identified by teachers as high risk using the TRISK, and 75 (50%) agreed to participate in videotaped home assessments in grades 7, 8, and 9. Ultimately data were available for 75 families (35 in the Intervention group and 36 in the Control group).
Measures:
Teacher-ratings were measured with a 16-item Teacher Perception of Risk (TRISK) scale. Self-reported alcohol, tobacco, and marijuana use by students were also assessed with measures of frequency of use in the last month. Measures of antisocial behavior were included in the combined cohort 1 and 2 assessment and these measures included items of self-reported problem behavior - frequency over the past month on six items (lying to parents, skipping school, staying out all night without permission, stealing, panhandling, and carrying a weapon). Intervention engagement was measured by Parent Consultant-collected data on minutes spent in contact with the Parent Consultant, the purpose of the contact (measuring intensity), and method of contact (in-person or by phone). Videotaped family assessment allowed for a measure of parental monitoring.
For the full sample assessment studies (n=998), youths also self-reported on deviant peer involvement, parental monitoring, and family conflict. Arrest records were accessed with parental permission to identify status violations, misdemeanors, and felonies. At age 19, youths also completed the Composite International Diagnostic Interview to report on lifetime substance abuse diagnoses. On the measure of depressive symptoms (Connell and Dishion, 2008) for the identified high-risk students (n=106), additional measures were collected. These included the self-reported Child Depression Inventory (CDI)and the maternal Child Behavior Checklist (CBCL) for internalizing and externalizing behavior. These assessments were collected in the fall of the child's seventh, eighth, and ninth grades. Academic achievement and attendance were measured using student GPA (grades 6 - 11) and attendance records.
Analysis:
Substance use outcomes were generated using logistic regression. Data are presented separately for AR & TD youth, as well as cumulatively, with ninth grade substance use as a dependent variable. Mediating effects of parent monitoring were determined through correlation and logistic regression.
Cohorts 1 and 2
(Connell et al., 2006; Connell et al., 2007; Connell and Dishion, 2008; Stormshak et al., 2009; Connell, 2009; Connell et al., 2017)
This study combined the original cohort above with an additional cohort of adolescents and their families, for a combined total sample population of 998. The study design and measures are described above. Various outcome measures and analytical strategies were used to determine both immediate and long-term program effects.
Sample:
This sample included 998 adolescents and their families, who were randomly assigned to either the treatment (n=500) or control (n=498) conditions. The majority of the adolescents were male (52.7%). The sample was ethnically diverse, with 42.3% Caucasian, 29.1% African-American, 6.8% Latino, 5.2% Asian, and 16.4% other ethnicity youth. In 58.6% of families, biological fathers were present. Approximately 80% of the youths were retained in the assessment throughout each wave of the study. In the intervention condition, 115 families (23%) elected to receive the FCU and 88 of these progressed to the third level to receive further intervention services. For cohort 1, 46% of FCUs were completed following the seventh grade family assessment, 53% were completed following the eighth grade family assessment, and 1% was completed following the ninth grade family assessment. In cohort 2, most (93%) FCUs were completed following the seventh grade family assessment and the remainder following the eighth grade family assessment. These families had an average of 8.9 direct contact hours with intervention staff. There was also a long-term diagnostic interview assessment (Connell et al., 2017), conducted when the individuals were aged 28-30 years. A total of 754 (76%) youth were retained for the long-term assessment.
A high-risk sample of adolescents (n=106) was identified using the multiple gating approach to risk assessment (TRISK), with problem behaviors including aggression, moodiness, oppositionality, peer relationship problems, and school problems. Students who scored 3 or higher or whom teachers suspected of substance use were designated high risk. These students completed more comprehensive assessments, including the measurement of depression symptoms. There were 52 youths assigned to the control condition and 54 in the intervention condition. Most parents (88.9%) of the high-risk youth participated in the universal level of the intervention, and of these, 60% also completed FCU and the linked intervention services through the family resource room. These families received an average of 7.89 hours of services over the course of 3 years, from seventh through ninth grade.
A subset of the combined cohort population was used to examine substance-use patterns in grades 6-9 (Connell et al., 2006). This subset included 698 youth and their families who self-identified as either Caucasian (59.6%) or African-American (40.4%). These two subgroups were chosen because the numbers in each were sufficient to conduct analyses for ethnicity effects.
Measures:
For the long-term follow-up analysis (Connell et al., 2017), depression was diagnosed using the Composite International Diagnostic Interview (CIDI), based on criteria from the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV). Individuals were assessed for past year and lifetime major depressive disorder diagnoses. It was unclear if the diagnostician was aware of condition.
Analysis:
A technique known as Complier Average Causal Effect (CACE) analysis was used on several outcome measures to limit the amount of potential bias introduced in the results from an intent-to-treat analysis among the families that voluntarily participated in the selected and indicated levels of the program. This type of analysis identifies a subset of the randomized control group who resemble those who do actively participate in the voluntary intervention, which should provide a more accurate picture of how youth receiving the FCU would have developed without intervention. A mixture-modeling framework is applied to identify the optimal comparison group from the control condition to compare with intervention engagers in the intervention condition. The Connell et al., 2007 analysis focused on separate substance use types and antisocial behavior through grade 12. CACE analyses were conducted as mixture models and used full information maximum likelihood estimation to account for missing data. The assumption was that for noncompliers in either the intervention or control condition (i.e., never-takers or always-takers), the distribution of potential outcomes was independent of the treatment assignment. Connell (2009) also used CACE analysis to examine the long-term impact of ATP on trajectories of adolescent tobacco use from early adolescence through early adulthood, using 7 assessment waves when youth were ages 11, 12, 13, 14, 16-17, 18-19, and 22 (at wave 7, only cohort 1 data were available). Stormshak and colleagues (2009) used this technique to examine program effects on academic achievement and school attendance.
The primary analysis for youth depression outcomes (Connell and Dishion, 2008) were intent-to-treat analyses using two latent growth models (LGM). Analyses also used full information maximum likelihood estimation (FIML) to accommodate missing data. Youth-reported or mother-reported conduct problems at each wave were used as time-varying covariates in the model. CACE analyses were also conducted in order to examine the possibility that the intervention was largely driven by familial participation in the selected and indicated levels of intervention.
The study of trajectories of early adolescent substance use (Connell et al., 2006) used three levels of analyses. First, a zero-inflated Poisson (ZIP) growth model using M-plus 3.01 was used to examine predictors of change in substance use over early adolescence. Next, latent growth mixture modeling (LGMM) using M-plus 3.01 was used to examine heterogeneity in developmental trajectories of early adolescent self-reported substance use from a person-centered perspective. Finally, a logistic regression framework was used to examine the predictors of trajectory class membership, with results comparing the extent to which each predictor was related to the likelihood of being in a given trajectory relative to a normative trajectory.
In the long-term follow-up analyses, logistic regression models were used to examine the intervention's effect on depression diagnoses at age 28-30, but without controls for baseline depression. Follow-up analyses for significant interactions were carried out by subsetting the data by the median split on the moderator variable, and then examining intervention effects in the subsamples.
Intent-to-treat: Although the primary results presented are based only on individuals with long-term data, intent-to-treat analyses were also carried out in a subsequent analysis, with missing data accounted for using multiple imputation. The authors reported that the results from both analyses were virtually identical.
Outcomes
Cohort 1 studies
Dishion et al., 2002:
Intervention Engagement: Two groups emerged from TRISK analysis, the AR (at-risk) group and TD (typically developing). Typically developing students were those not at-risk. The program staff made more contacts with AR families (on average, 13) than TD families (on average, 7). The total personal contact time was brief - only 5.6 hours for AR families and 3.1 hours for TD families. Only 3 control families sought FRC services during the program period and these families, according to a decision the research group made prior to the study's initiation, were afforded services they wanted.
Intervention Outcomes: There is very little instance of substance use, especially for TD children, until grade 9. There is a significant program effect on reductions in grade 9 substance use, controlling for use in prior grades, favoring the ATP group. There was not a significant difference between AR youth and TD youth, indicating that the program produced significant effects regardless of risk-level. Effects did not vary by gender or ethnicity.
Dishion, Nelson, and Kavanagh, 2003: Using a directional test, the intervention, relative to the control condition, reduced grade 9 substance use among higher risk students.
Mediation Model: Intervention families decreased parental monitoring from seventh to eighth grade, but increased monitoring between eighth and ninth grade. Control families decreased parental monitoring slightly from seventh to eighth grade and continued to decrease monitoring more sharply from eighth to ninth grade. After controlling for prior substance use, the intervention condition was associated with a significant lower probability of substance use. High-risk youth in the control group were more likely to report substance use than intervention youth. Results of the mediated model show an increase in parental monitoring significantly accounted for a reduction in substance use in ninth grade, controlling for previous levels of substance use and random assignment.
Combined cohorts 1 and 2 studies
The Connell et al., 2007 study discriminated between families within the intervention group who engaged in the FCU from those who did not (engagers vs. non-engagers). They found that treatment engagement was significantly related to the likelihood of biological fathers being absent from the home, youth reports of elevated family conflict and deviant peer affiliation, and teacher reports of elevated risk behaviors at school. That is, the families that had the highest level of vulnerability and experienced the most difficulty were more likely to engage in treatment. This finding is significant in itself, as it demonstrates that the intervention was able to successfully engage and motivate the at-risk and high-risk portion of the larger sample to participate in the targeted intervention phases. Female gender predicted a higher level of engagement for tobacco and marijuana use models, and deviant peer affiliation also predicted engagement in the tobacco use model. Those who engaged in treatment showed significantly inhibited growth in substance use and antisocial behavior from age 12 to 17, compared to the control group. The intervention also had significant effects on arrest rates, and likelihood of being diagnosed with alcohol, tobacco, or marijuana use disorders by late adolescence. Long-term tobacco outcomes (Connell, 2009) indicated that treatment engagement predicted significantly less growth in use by age 22. Effect sizes at age 22 were between 1.18 and 1.27, depending on the model, which used either a moderate or stringent definition of "engagement" (completion of FCU only (25.5% of families) or both FCU and follow-up services (19.5% of families), respectively). In this study, family engagement was predicted by female child and elevated peer deviance.
The depression analysis conducted with 106 high risk youth from Cohorts 1 and 2 (Connell and Dishion, 2008) found that treatment effects did not emerge until the youth were in ninth grade, when there were significant reductions in CDI scores for treatment youth compared to control group youth at that time. By ninth grade, there were significant differences in the number of youths in the clinical range between treatment and control group (effect size = .56). On mother-reported youth symptoms, the intervention produced a small to medium sized difference in depression symptoms at ninth grade (.42). CACE model results indicate that youths' self-reports of growth in depressive symptoms were significantly lower for "compliers" (63.2% of the sample) than youth in the control group, with a treatment effect size of 1.35. Also, mother reports of youth compliers (65.3%) reported significantly less growth in youth internalizing symptoms than mothers of youth in the control group, with an effect size of 1.07.
On the measure of academic outcomes (Stormshak et al., 2009), intervention engagement reduced growth in school absences and declines in GPA from 6th through 11th grade. Engager vs. non-engager intervention status was as follows: On the measure of academic performance, sixth grade GPA in the non-engager class was positively related to male gender, the presence of a biological father in the home, and to parental monitoring efforts. There was a negative association with African American or Hispanic ethnicity, elevated family conflict, and teacher perceptions of engagement in risky behavior. For engagers, sixth grade GPA was negatively related to teachers' perceptions of school risk behavior. On the measure of school absences in sixth grade, non-engager intervention youth were more likely to come from homes without a biological father present, low parental monitoring, and teacher report of school risk behavior. For engagers, sixth grade absences were positively related to teachers' reports of risk behavior. Hispanic youth who participated in the intervention showed greater declines in GPA and greater acceleration in the number of school absences over time compared to non-Hispanic youth who participated in the intervention.
The examination of the trajectories of early adolescent substance use (Connell et al., 2006) found that about half (53.1%) of the students reported no substance use over the course of the study. The likelihood of initiating substance use in sixth grade was related to deviant peer affiliation and less-effective parental monitoring. Among those youth who indicated having used a substance at least once between sixth and ninth grade, those in the intervention condition had higher initial (sixth grade) substance use and showed significantly less growth in substance use over time, compared to youth in the control condition. There were also gender differences, with program impacts in slowing growth in substance use as significantly greater for girls compared to boys. Five trajectories also emerged, representing growth parameters across classes and the predictors of trajectory class membership. The majority of youth were classified as no-use (51.82%) or low/rare-use (25.79%). The remaining classes of youth who reported substance use included a decelerating class (7.02%), youth who exhibited significant levels of substance use in sixth grade and significant decreases in the level of use over time; an early accelerating class (6.30%), youth who exhibited significant levels of substance use in sixth grade, and also reported significant growth in substance use over time; and a late-accelerating class (9.03%), youth who exhibited a burst of substance use during the transition from middle to high school (eighth to ninth grade). Of these youth who exhibited higher levels of initial substance use, those assigned to the intervention reported more dramatic declines and were disproportionately more likely to engage in the most intensive form of treatment relative to any other trajectory class. The youth were more likely to be Caucasian and to have reported high levels of sixth grade deviant peer affiliation.
In Caruthers et al. (2014), growth curve modeling was used to determine direct and indirect program effects over time on high risk sexual behavior (HRSB) by age 22. Key variables examined were family relationship quality and parental monitoring as mediators, as well as early sexual activity. High risk sexual behavior was the average of five dichotomous items (sexual partners with multiple partners, familiarity with partner, condom use, sexually transmitted disease, and pregnancy). Differential effects of race and gender were also examined as moderators. There was no direct program effect on HRSB, but there was a significant indirect program effect on HRSB with family relationship quality as an intervening variable. There was also a significant effect of family relationship quality on HRSB mediated by differences in parental monitoring and early sexual activity. The effects of the mediators were varied based on gender and race.
Long-term follow-up of combined cohorts (Connell et al., 2017):
Implementation Fidelity:
The families that received the Family Check Up intervention were in contact with intervention staff for an average of 8.9 hours over the course of the study, but no other information was given on attendance or implementation fidelity.
Baseline Equivalence:
There was no information on baseline equivalence.
Differential Attrition:
The authors note that "preliminary analyses suggested that attrition was consistent with Missing at Random assumptions," even though overall attrition was fairly high (24%) by the long-term follow-up.
Long-Term:
Connell et al. (2017) found no effect of intervention on past-year depression diagnoses or on lifetime depression diagnoses. They tested for multiple moderation effects, finding some program benefits for subjects with low baseline family conflict and with low baseline GPA.
Study 2
Evaluation Methodology:
Design: This study, called Project Alliance 2, focused primarily on home-to-school linkages and the family-school partnership and outcomes related to academic success and school engagement. The evaluation consisted of an auto-regressive design over four waves of data and targeted youth from 6th - 9th grade. The design of the intervention followed the three level model of the program. In the first level, families had access to a parent consultant through the Family Resource Center. The consultant provided services to families within the school context, including attending behavioral support meetings, teacher meetings, and any other important school meeting related to child behavior. At the second level, families participated in the Family Check-Up (FCU) protocol, which included an initial home visit and interview, an assessment and video-taped parent-child interaction session, and a feedback session. At this point, families decided either to decline further services, or participated in level three, in which consultation was provided for further support and training services.
Sample: A sample of 377 adolescents and their families was drawn from three public middle schools in an urban area. Researchers used an unbalanced approach to randomization in order to enhance the power of intervention effects specifically for the families that elected to utilize services tailored for their needs. More of the sample (n=277, 73%) was assigned to the treatment condition, while the remaining 100 families (27%) were randomly assigned to the school as usual condition. The majority of adolescents were male (51%) and participants were ethnically diverse (36% White, 18% Latino/Hispanic, 16% African American, 8% Asian, 3% American Indian, and 19% biracial/mixed ethnicity). Average household income was between $30,000 and $40,000.
Across the 4 years of the study, approximately 80% of youth remained in the study. Families who moved were tracked and followed, via mailed surveys and phone interviews. Of the 277 students in the intervention condition, 46% received parent consultation services and 38% participated in the full FCU intervention (24% of these received additional services following feedback). The median amount of time families participated in FCU was 168 minutes, with most contacts occurring when the child was between 7th and 8th grade (80%). More families with a male child (34%) participated in the FCU level, compared to families with a female child (30%). There were no differences by gender or for outcome variables of self-regulation, depression, or school engagement.
Measures: Students were surveyed at school each spring by completing a questionnaire. Self-regulation was measured using items from the Early Adolescent Temperament Questionnaire. Depression was measured using items from the Child Depression Inventory. School engagement was measured using two items that reflect enjoyment of learning and effort to achieve.
Analysis: An intent-to-treat analysis was conducted to evaluate generalized effects of the FCU in an autoregressive model. Three analyses were conducted. The first tested the possibility that treatment group assignment was associated with increases in self-regulation in seventh grade, accounting for sixth grade self-regulation. Next, the extent to which seventh grade self-regulation predicted changes in depressive symptoms in eighth grade was tested. Finally, eighth grade self-regulation and depressive symptoms as predictors of youths' academic engagement in ninth grade was examined.
Outcomes: There was a significant effect of the intervention on overall youth self-regulation between sixth and seventh grade. Seventh grade self-regulation was also associated with decreases in youths' depressive symptoms from sixth to eighth grade, with a small to medium effect size (-.25). Eighth grade self-regulation was associated, with a medium effect size increase (.37), with school engagement by ninth grade. There were no significant gender differences.
Limitations: Limitations include using adolescent self-reports as the primary measure for outcomes; this may have created some bias based on social desirability of reporting.
Study 3
Evaluation Methodology
Design: Participants were 593 adolescents and their families from three urban, public middle schools in the Pacific Northwest. Families were randomly assigned to either the intervention (n=386; 65%) or control (n=207; 35%) condition. Youth and families assigned to the control condition experienced "business as usual" school and did not have access to the intervention services available to the intervention condition families.
To increase the level of engagement of families in the intervention condition, all intervention families were invited to participate in Family Check-Up (FCU) services, whether or not their child was identified as at risk. Intervention families who engaged in the FCU services participated in a three session assessment and feedback process. Parent consultants used motivational interviewing techniques to determine the family dynamic and assess needs. During the feedback session, the families were presented with a list of intervention options and parenting strategies, as well as other school- or community-based resources to address the specific needs of the family. Families were free to accept or decline the services. The parent consultants used for this study were trained researchers (not school personnel) and, wherever possible, parent consultant ethnicity was matched with family ethnicity. Consultants received intensive training and supervision over the three year course of the study.
Utilization of FCU services was as follows: 51% of intervention families received consultation from a parent consultant and 42% received the full FCU intervention. Of those receiving FCU intervention, 78% received additional follow-up assistance after the feedback, including parent skills training, and address of education-related concerns as needed (homework, attendance, grades, etc.). Intervention families received an average of 94.2 minutes of intervention time.
Sample Characteristics: Students were in 6th grade when the intervention began. There were slightly more male than female participants (51.4% vs. 48.6%), and the racial/ethnic makeup of the sample was diverse: 36.1% White, 18.0% Latino/Hispanic, 15.0% African American, 7.1% Asian, 2.4% American Indian/Native American, 1.9% Pacific Islander, and 19.3% reporting mixed ethnicity/biracial.
Measures: Measures were collected from students in the spring of each academic year from 6th through 8th grades and the survey tool used was developed at the Oregon Research Institute. Primary outcome measures assessed from this tool included measures of antisocial behavior, deviant peers, and substance use (cigarette, alcohol, marijuana). Additionally, the Effortful Control scale was used to assess self-regulation. Reliability across all measures and waves ranged from .72 - .86.
Analysis: Structural equation modeling was used, using full information maximum likelihood estimation to reduce bias resulting from missing data. An alpha of .05 was used. Standard measures of fit were reported for antisocial behavior and deviant peer affiliation, including the comparative fit index, standardized root mean square residual, and root mean square error of approximation. Because random assignment occurred within schools, analyses were conducted at the individual level. Analysis was intent to treat.
Outcomes
Fidelity Monitoring: Supervision of the parent consultants by the program developers was ongoing throughout the study, and included monitoring of videotaped FCUs, feedback to consultants, planning for FCUs, conducting role plays, and providing guidance for use of the family management curriculum.
Posttest: Results indicate significant program impact on increases in self-regulation for intervention students from 6th to 7th grades across all growth models, compared to students in the control condition. There were then small to moderate associations with growth in all primary outcomes. Self-regulation was associated with less growth in antisocial behavior (ES=.12) and deviant friendships (ES=.18). There was a stronger association between higher levels of self-regulation and less growth in substance use, with effect sizes of .30 (cigarettes), .22 (alcohol) and .39 (marijuana) from 6th to 8th grades. There was a gender effect on deviant friendship growth for girls (ES=.12).
Limitations: Data collection measures were single informant, relying on responses only from students. Also, effect size for the intervention was small, partially due to the chosen analytical approach (to determine the average benefit at the school level), which may have resulted in an underrepresentation of the effect size, as less than half of the intervention group did not engage in the FCU. Also, the parenting interventions did not directly target effortful control (self-regulation), also potentially limiting the effect size. Randomization occurred at the student level to avoid nesting effects, so schools were limited in their implementation of the universal-level component of the program (as students could be in either condition), potentially limiting the effects of that program component. The parent consultants were trained researchers.
Study 4
Evaluation Methodology
Design:
Smolkowski et al. (2017) conducted a cluster randomized control trial with 41 middle schools in which schools were blocked at the district level. The treatment schools received the three-level Positive Family Support intervention, while the control group delayed receiving the program for three years. The authors followed two cohorts of students. Cohort 1 entered sixth grade immediately after randomization, 1 year before implementation of the treatment in intervention schools. Cohort 2 entered sixth grade the following year, 1 year after randomization and during the first implementation year in treatment schools. Thus, within treatment schools, all students in Cohort 1 finished sixth grade before implementation and all students in Cohort 2 began sixth grade after implementation. Each cohort was followed through eighth grade. Treatment students in both cohorts experienced 2 years of exposure, but insufficient time prohibited schools in Cohort 2 to complete the final wave of data collection. Smolkowski et al. (2017) therefore reported results for students in Cohort 1. Though not clearly stated, it appeared that student-, parent- and teacher-reported data were collected among Cohort 1 students at baseline (i.e., the beginning of seventh grade), with 2 follow-ups (one at the end of seventh grade and another at the end of eighth grade). Because the intervention was two years, however, the posttest was at the end of eight grade.
Recruitment:
Schools with a minimum of 50 students in the sixth grade that had implemented school wide Positive Behavioral Interventions and Support systems were eligible for the study. Approximately 200 schools in the Pacific Northwest were identified that met eligibility criteria. The parents of 14,331 students received a request for dissent or consent, with the parents of 1,419 students having actively dissented to participation, thus leaving 12,912 participating students across both cohorts and all data collection time points (in which there were four rounds of data collection in total) from 41 middle schools in 27 school districts.
Assignment:
Project staff recruited and randomized 14 schools before the 2009-2010 academic year, 13 schools before the 2010-2011 academic year, and 14 schools before the 2011-2012 academic year. A total of 41 schools were consequently assigned to condition, in which single schools within districts were matched with other single schools by enrollment, though in 1 year (2010-11) one school was unmatched and assigned at random to condition by itself, resulting in 21 treatment schools and 20 control schools. For Smolkowski et al. (2017), Cohort 1 included 5,133 students from 41 schools with pre-intervention data (i.e., those randomly assigned and assessed at T1 - which for students in Cohort 1 was the beginning of seventh grade).
Attrition:
There were 4,078 students in Cohort 1 who completed an assessment at the end of eighth grade (which was considered the posttest), for an overall attrition rate of 8%. There was no school-level attrition.
Sample:
Of the 12,912 overall students who participated in the study, about half (51%) were female and a majority (65%) were white. The average age for students at baseline was 11.9 years (SD = 1.4). A sizable proportion (45%) of students reported their families had "just enough money to get by." Demographics for the involved schools were as follows. The proportion of students who received free or reduced-price lunch was 58%. The proportion of English language learners was 3%. The proportion of minority students was 28%.
Measures:
Nineteen measures were conducted at the student, parent and teacher levels to assess program effects on a myriad of behavioral outcomes and risk and protective factors. These measures are listed below.
Student-level measures were obtained during the annual student survey, and included:
- Parental Monitoring - a risk and protective measure (sample α = .91)
- Family Conflict - a risk and protective measure (sample α = .78)
- Conduct problems (sample α = .63)
- Emotional problems (sample α = .69)
- Substance use for tobacco, alcohol, marijuana or other drugs (sample α = .87)
- Positive Peers (sample α = .77)
- School participation - a risk and protective factor (sample α = .56)
- Positive school structures - a risk and protective factor (sample α = .86)
Parent-level measures were obtained from annual parent surveys, and included:
- School Success - a parental report of the child's academic performance (sample α = .90)
- Student risk - a risk and protective factor measuring parent concern for child's internalizing behavior, externalizing behavior, attention, sociability, and adherence to rules (sample α = .82)
- Parental monitoring behavior - a risk and protective factor (sample α = .76)
- Parent involvement - a risk and protective factor measuring parents' interactions with child (sample α = .66)
- School structures to monitor school work, attendance and behavior - a risk and protective factor (sample α = .77)
Teacher-level measures were obtained from annual teacher survey, and included:
- School readiness - a four-item scale to identify school-level statements specific to Positive Family Support (sample α = .78)
- Student Risk - teacher concern for child's internalizing behavior, externalizing behavior, attention, sociability, and adherence to rules; a risk and protective factor (sample α = .89)
Administrative student-level records were obtained from the Oregon Department of Education for three primary outcomes (end-of-year state standardized math and reading test scores, and number of days absent from school) and one risk & protective factor (negative and positive school contact, as measured by calls home to parents concerning children's behavior in school).
Analysis:
Multi-level models were conducted to assess main effects with students nested within schools across time. Additional analyses were conducted to assess whether risk-level, gender, minority status and cohort moderated outcomes.
Intent-to-Treat:
All students with data were assessed in their assigned treatments, therefore intent-to-treat protocol was followed. The authors utilized restricted maximum likelihood to deal with missing data.
Outcomes
Implementation Fidelity:
Treatment schools had significantly greater scores of implementation fidelity for the three tiers of the program and a significantly greater number of resources available to parents.
Baseline Equivalence:
There were no school-level significant differences between conditions at baseline. In addition, students from both conditions were comparable on all 19 outcomes and most demographic characteristics at baseline, with the exception of the student report of how much money their family had.
Differential Attrition:
Attrition was just over 5% (i.e., 8%) from pretest to posttest. While the authors did not report differential attrition (i.e., the difference between completers and attritors on baseline and demographic variables), they reported results of attrition by condition tests and found no differences by condition on baseline measures.
Posttest:
At posttest, no main effects were detected. Moderation analysis found that treatment students at-risk for behavioral problems significantly outperformed control students at-risk on measures of parent-reported negative school contacts.
Long-Term:
Not conducted.
Limitations:
- Tests of differential attrition are incomplete
- No main effect