Sign Up For Newsletter

KEEP SAFE

KEEP SAFE

A six-session group-based intervention, facilitated by paraprofessionals, for youth in foster care as they transition to middle school. The program aims to prevent internalizing and externalizing problems that may lead to more serious longer term outcomes such as delinquency, substance use, and high-risk sexual behavior. Foster parents also attend a six-session program.

Fact Sheet

Program Outcomes

  • Illicit Drug Use
  • Positive Social/Prosocial Behavior
  • Sexual Risk Behaviors
  • Tobacco

Program Type

  • Foster Care and Family Prevention
  • Parent Training
  • Skills Training

Program Setting

  • Social Services

Continuum of Intervention

  • Selective Prevention

Age

  • Early Adolescence (12-14) - Middle School

Gender

  • Female

Race/Ethnicity

  • All

Endorsements

Blueprints: Promising

Program Information Contact

Patricia Chamberlain
Oregon Social Learning Center
10 Shelton McMurphey Boulevard
Eugene, OR 97401-4928

Program Developer/Owner

Patricia Chamberlain and Leslie Leve
Oregon Social Learning Center


Brief Description of the Program

The KEEP SAFE (Middle School) program begins during the summer prior to middle school entry and consists of two parallel components (both led by paraprofessionals): a six-session group-based intervention for the foster-care youth and a six-session, group-based intervention for the foster parents. The groups meet twice weekly for 3 weeks during the summer. Facilitators teach youth about setting goals, establishing positive relationships with peers and adults, building confidence, and developing decision-making. Problem-solving skills and opportunities are also provided so they can practice positive behaviors. A ceremony is conducted at the end of the summer in which the youth proclaim their goals and commitments to each other and their foster parents. Sessions for foster parents focus on maintaining stability in the home, preparing the youth for middle school, and developing behavioral reinforcement techniques and realistic expectations. Parents are given homework assignments designed to encourage them to practice their new skills at home. Continued sessions (i.e., ongoing training and support) are provided to foster parents (group-based) and youth (one-on-one sessions) once a week for two hours during the first year of middle school.

Outcomes

Intervention girls, relative to control girls, demonstrated significant effects for:

  • Internalizing and externalizing problem behaviors at six months.
  • Prosocial behavior combined across the 6-month and 12-month assessments.
  • Placement stability at the 12-month posttest.
  • Substance use at 36 months beyond baseline (2 years post-intervention).
  • Health-risking sexual behavior at 36 months beyond baseline (2 years post-intervention).
  • Small to moderate mediating effects of prosocial, internalizing, and externalizing behaviors on long-term outcomes of substance use.
  • Small to moderate mediating effect of tobacco and marijuana use on long-term health-risking sexual behavior.

Brief Evaluation Methodology

This randomized controlled trial of the program was conducted with girls in foster care from two counties (one urban, one rural) in Oregon. Researchers recruited foster girls who were finishing elementary school between 2004 and 2007. They were randomized into one of two groups – the KEEP SAFE (Middle School) intervention (n=48) or the control group (n=52), which received standard services from local child welfare organizations.

Data were collected at five points in time: baseline, six months, 12 months (program completion), and one and two years post-intervention. A five- to seven-year post-intervention assessment is ongoing.

The effects of the program on internalizing, externalizing, and prosocial behavior were examined at six months, as well as pathways through which the program may have affected delinquency, health-risking sexual behavior, foster-care placement stability, and substance use over the course of two years after program completion.

Study 1

Kim, H., & Leve, L. (2011). Substance use and delinquency among middle school girls in foster care: A three-year follow-up of a randomized controlled trial. Journal of Consulting and Clinical Psychology, 79(6), 740-750.


Risk Factors

Individual: Stress

Peer: Interaction with antisocial peers

Family: Family conflict/violence, Family history of problem behavior, Neglectful parenting, Parent stress, Poor family management

Protective Factors

Individual: Problem solving skills, Prosocial involvement, Rewards for prosocial involvement, Skills for social interaction

Peer: Interaction with prosocial peers

Family: Opportunities for prosocial involvement with parents, Rewards for prosocial involvement with parents

KEEP SAFE is intended for middle school youth (ages 11-14) in foster care. The program has been evaluated with girls in foster care who have just left elementary school and are beginning middle school in the coming months. An ongoing study is evaluating the program for boys.

Race/Ethnicity/Gender Details

Although a program for foster-care boys and girls, the KEEF SAFE program was evaluated with foster care girls. In the girls-only trial, the race distribution of the sample was 63% White, 10% Latino, 9% African American, 4% Native American, and 14% multiracial. The analysis did not examine results by race and ethnicity.

The training model for KEEP SAFE facilitators closely parallels the services that the facilitator will provide to the foster parents. The initial 5-day training is experiential, and the trainees conduct a KEEP SAFE group though all of the sessions with colleagues. In this way the content of the KEEP SAFE curriculum is reviewed along with discussions about group process, logistics, and supervision.

Following the initial training, each KEEP SAFE group meeting is recorded and uploaded to a secure website. Weekly consultations with the facilitator are used to highlight issues with program fidelity, to support the delivery of the program content with accuracy, and to assist with questions specific to the current group of foster parents. We have found that by the conclusion of a third group, most facilitators feel comfortable with the material and the delivery of the material to a group of foster parents. It is following the third group that facilitators may opt to apply for certification. If the fidelity benchmarks are achieved, the facilitator may continue to hold KEEP SAFE without weekly consultations. A certified facilitator will have quarterly checks of program fidelity, with support plans developed only if difficulties are noted.

Certified facilitators are eligible to received further training in the delivery of the 5-day initial training course and consultation as a trained trainer. Such trainers also receive support in their first training, and in the coding of recorded sessions for intervention fidelity. Such trainers can enhance the sustainability of the practice by training others within their organization.

Program Costs

Start-Up Costs

Initial Training and Technical Assistance

$40,000 for a five-day, on-site training for a facilitator and co-facilitator; execution of a contract along with site readiness activities; weekly recording and upload of KEEP SAFE group sessions via a HIPAA-compliant system; weekly consultations with a KEEP SAFE consultant; and fidelity review for certification after the facilitator has competed three KEEP SAFE groups. Up to four additional teams (facilitator and co-facilitator) can be added to the training for an additional $24,000 per team. These costs do not include travel and related expenses (airfare, hotel, per diem) for the trainers.

Curriculum and Materials

The costs of the curriculum are included in the initial training costs.

Materials Available in Other Language: KEEP is available in Danish and Spanish. Contact developers for costs as they are not billed in any consistent manner.

Licensing

There are no additional licensing costs.

Other Start-Up Costs

$1,950 for initial planning consultation, plus travel costs. Many teams find a one-day stakeholders meeting to be helpful in establishing a referral base and galvanizing community interest, support and understanding. Holding a stakeholders meeting is not a requirement to receive training or support.

Running KEEP SAFE groups requires sufficient space for a group of up to 12 participants. The space should be readily accessible to the foster parents. Most providers find that they are able to use agency and community resources.

Intervention Implementation Costs

Ongoing Curriculum and Materials

Facilitators are required to record and upload each KEEP SAFE session to a secure website. This requires a Windows-based computer and internet access. It is also helpful to have an external microphone to make the recordings clear. The use of computers dedicated to recording and uploading of session recordings helps to facilitate this process.

Staffing

Qualifications: Facilitators and co-facilitators are child welfare professionals.

Ratios: 1 facilitator and 1 co-facilitator lead a group of 12 participants.

Time to Deliver Intervention: 90 minute weekly group sessions for 20 weeks.

It is helpful, but not required to have an on-site supervisor who is involved in the training and support processes. While there will be weekly consultation regarding the KEEP SAFE intervention and fidelity, this consultation does not replace local knowledge about foster parent certification, policies, and legal requirements. Trained supervisors are also able to provide coverage and support for facilitators.

Other Implementation Costs

Once facilitators have run three KEEP SAFE groups, they are eligible to be certified as KEEP SAFE facilitators. This certification permits the facilitator to continue to run KEEP SAFE groups without the need or cost of weekly consultation. Certified facilitators are also eligible (with additional selection criteria) for a subsequent three-day training which would permit them to train others within their organization. This on-site training costs $12,900 (with travel costs additional) and includes the three-day training, shadowing of the trainee’s first training, and support in achieving reliability of the fidelity tracking system.

KEEP SAFE has a strong commitment to sustaining the participation of foster parents who start the groups. In the randomized controlled trials, participating foster parents were given training credits, incentives for participation (as reimbursement for travel expenses) and make-up sessions in their homes when they were unable to meet with the group. These practices resulted in a high percentage of foster parents receiving all of the KEEP SAFE material, and reinforced the status of the foster parents as partners in the care of the young people in their homes.

Implementation Support and Fidelity Monitoring Costs

Ongoing Training and Technical Assistance

There are no additional or ongoing trainings required. If a facilitator finds that he/she is having difficulty maintaining intervention fidelity, additional training can be outlined.

Fidelity Monitoring and Evaluation

$3,000 per year. Once a facilitator is certified, the individual must run at least one KEEP SAFE group each year, and then continue to upload sessions to the website. Access to the site costs $3,000 per year and includes a quarterly review of available sessions for fidelity.

Ongoing License Fees

None.

Other Implementation Support and Fidelity Monitoring Costs

No information is available

Other Cost Considerations

Staff turnover is an important cost consideration. Certifications are held by the individual, rather than the organization. Making plans for sustainability and potential staff turnover are an important part of the site readiness and initial planning.

Year One Cost Example

This example assumes that a community-based organization would offer the KEEP SAFE program to 3 groups of 12 parents, each with 1 facilitator and 1 co-facilitator, for 20 sessions, with three cohorts of parent groups per year.

Fringe at 30% $36,000
Facilitator and co-facilitator's salary $120,000
Fidelity monitoring and evaluation $3,000
Initial planning consultation $1,950
Estimated trainer travel costs and expenses for 1 trainer $1,500
Facilitator and co-facilitator initial training costs $40,000
Total Year One Cost $202,450

With 108 foster parents participating, the initial cost of the program is approximately $1,874/parent.

Funding Strategies

Funding Overview

As a program designed for foster or kinship caregivers with young people aged 12-17 years, KEEP SAFE is designed to increase the parenting skills of foster and kinship caregivers, decrease the number of placement disruptions, improve child outcomes and increase the number of positive placement changes. KEEP SAFE can potentially be supported by child welfare funding streams, Medicaid and fees for service.

Funding Strategies

Improving the Use of Existing Public Funds

The child welfare system is a potential source of funding for KEEP SAFE. Funds from system-wide savings accrued from stabilizing placements, reducing foster parent attrition, and maintaining youth with caregivers upon their return from foster care can be reinvested in the KEEP SAFE program.

Allocating State or Local General Funds

State and local child welfare funding sources are key sources of support for the KEEP SAFE program. States that are building KEEP SAFE into their ongoing training of foster parents have built support for KEEP SAFE into their child welfare budgets.

Maximizing Federal Funds

Entitlement Funds:

  • Some localities have successfully accessed Medicaid to cover the cost of the program for qualifying participants; individual sessions of the group are documented for payment by Medicaid.
  • Title IV-E training dollars are federal matching funds to support training of child welfare professionals, as wells as foster parents, and prospective adoptive parents and guardians. States can potentially claim federal IV-E matching funds for ongoing KEEP SAFE training. Title IV-E reimburses 75% of allowable training costs.

Formula Funds: Title IV-B is the primary federal block grant supporting child welfare preservation and prevention services. Title IV-B can be used fairly flexibly to support a range of child welfare services and could potentially be used to pay for initial training and/or ongoing costs of KEEP SAFE training.

Discretionary Grants: Federal discretionary grants from the Department of Health and Human Services, Administration for Children and Families (DHHS, ACF) can be used to cover the costs of initial training, consultations, certification and training of trainers.

Foundation Grants and Public-Private Partnerships

Foundations, especially those with a stated interest in the well-being of vulnerable children in foster care, can provide funding for initial training and staffing, as well as meeting space.

Debt Financing

No information is available

Generating New Revenue

No information is available

Data Sources

No information is available

Evaluation Abstract

Program Developer/Owner

Patricia Chamberlain and Leslie Leve Oregon Social Learning Center 10 Shelton McMurphey Boulevard Eugene, OR 97401-4928 US (541) 485-2711 pattic@oslc.org lesliel@oslc.org

Program Outcomes

  • Illicit Drug Use
  • Positive Social/Prosocial Behavior
  • Sexual Risk Behaviors
  • Tobacco

Program Specifics

Program Type

  • Foster Care and Family Prevention
  • Parent Training
  • Skills Training

Program Setting

  • Social Services

Continuum of Intervention

  • Selective Prevention

Program Goals

A six-session group-based intervention, facilitated by paraprofessionals, for youth in foster care as they transition to middle school. The program aims to prevent internalizing and externalizing problems that may lead to more serious longer term outcomes such as delinquency, substance use, and high-risk sexual behavior. Foster parents also attend a six-session program.

Population Demographics

KEEP SAFE is intended for middle school youth (ages 11-14) in foster care. The program has been evaluated with girls in foster care who have just left elementary school and are beginning middle school in the coming months. An ongoing study is evaluating the program for boys.

Target Population

Age

  • Early Adolescence (12-14) - Middle School

Gender

  • Female

Race/Ethnicity

  • All

Race/Ethnicity/Gender Details

Although a program for foster-care boys and girls, the KEEF SAFE program was evaluated with foster care girls. In the girls-only trial, the race distribution of the sample was 63% White, 10% Latino, 9% African American, 4% Native American, and 14% multiracial. The analysis did not examine results by race and ethnicity.

Other Risk and Protective Factors

Risk factors for developing internalizing and externalizing problems are especially acute for children with foster care histories because they are more likely to have been subject to childhood maltreatment. In addition, foster care girls are at increased risk compared to foster care boys because they have significantly higher rates of childhood abuse than boys and high rates of co-occurring physical abuse. Girls in foster care typically come from families with higher levels of stress and criminality than boys in foster care.

Compared to non-foster girls, foster girls are more likely to experience an unstable rearing environment, which is associated with early pubertal onset, and early pubescence is associated with internalizing and externalizing behavior problems.

Risk/Protective Factor Domain

  • Individual
  • Peer
  • Family

Risk/Protective Factors

Risk Factors

Individual: Stress

Peer: Interaction with antisocial peers

Family: Family conflict/violence, Family history of problem behavior, Neglectful parenting, Parent stress, Poor family management

Protective Factors

Individual: Problem solving skills, Prosocial involvement, Rewards for prosocial involvement, Skills for social interaction

Peer: Interaction with prosocial peers

Family: Opportunities for prosocial involvement with parents, Rewards for prosocial involvement with parents

Brief Description of the Program

The KEEP SAFE (Middle School) program begins during the summer prior to middle school entry and consists of two parallel components (both led by paraprofessionals): a six-session group-based intervention for the foster-care youth and a six-session, group-based intervention for the foster parents. The groups meet twice weekly for 3 weeks during the summer. Facilitators teach youth about setting goals, establishing positive relationships with peers and adults, building confidence, and developing decision-making. Problem-solving skills and opportunities are also provided so they can practice positive behaviors. A ceremony is conducted at the end of the summer in which the youth proclaim their goals and commitments to each other and their foster parents. Sessions for foster parents focus on maintaining stability in the home, preparing the youth for middle school, and developing behavioral reinforcement techniques and realistic expectations. Parents are given homework assignments designed to encourage them to practice their new skills at home. Continued sessions (i.e., ongoing training and support) are provided to foster parents (group-based) and youth (one-on-one sessions) once a week for two hours during the first year of middle school.

Description of the Program

The KEEP SAFE (Middle School) program begins during the summer prior to middle school entry and consists of two parallel components (both led by paraprofessionals): a six-session group-based intervention for the foster-care youth and a six-session, group-based intervention for the foster parents. The groups meet twice weekly for 3 weeks, with approximately 7 people per group.

The parent sessions are led by a facilitator and a co-facilitator, each of whom are experienced foster parents with bachelor's degrees. Sessions are also often led by child welfare case workers and case work supervisors in “real world” implementation formats. The youth groups are led by a facilitator and three assistants. These individuals are supervised by a masters or doctoral-level clinician. Using paraprofessionals as the primary facilitators reduces the costs of the program dramatically.

Continued sessions (i.e. ongoing training and support) are provided to foster parents and youth (one-on-one sessions) once a week for two hours during the first year of middle school.

The content of the program is as follows:

Focus for youth: Facilitators teach youth about setting goals, establishing positive relationships with peers and adults, building confidence, and developing decision-making skills. Problem-solving skills and opportunities are also provided so they can practice positive behaviors. A ceremony is conducted at the end of the summer in which the youth proclaim their goals and commitments to each other and their foster parents.

Focus for foster parents: These sessions are primarily concerned with enabling foster parents to maintain stability in the home, prepare the youth for middle school, and develop behavioral reinforcement techniques and realistic expectations. Parents are given homework assignments designed to encourage them to practice their new skills at home.

Theoretical Rationale

The developers based the program on "developmental theories and intervention work with at-risk youth" and uses a behavioral model of change, emphasizing reinforcement and contingency management.

Theoretical Orientation

  • Behavioral
  • Skill Oriented

Brief Evaluation Methodology

This randomized controlled trial of the program was conducted with girls in foster care from two counties (one urban, one rural) in Oregon. Researchers recruited foster girls who were finishing elementary school between 2004 and 2007. They were randomized into one of two groups – the KEEP SAFE (Middle School) intervention (n=48) or the control group (n=52), which received standard services from local child welfare organizations.

Data were collected at five points in time: baseline, six months, 12 months (program completion), and one and two years post-intervention. A five- to seven-year post-intervention assessment is ongoing.

The effects of the program on internalizing, externalizing, and prosocial behavior were examined at six months, as well as pathways through which the program may have affected delinquency, health-risking sexual behavior, foster-care placement stability, and substance use over the course of two years after program completion.

Outcomes (Brief, over all studies)

At the six-month mid-program assessment, the KEEP SAFE (Middle School) program for foster girls was significantly related to improvements in internalizing and externalizing problem behaviors relative to the foster girls in the control group.

At 12 months after baseline, the program significantly reduced placement changes and increased prosocial behavior.

At 36 months after baseline, or two years after program completion, the intervention foster girls reported moderately lower levels of substance use (Cohen's d = -.47) than girls in the control group. The intervention also affected substance use via prosocial behavior and internalizing and externalizing behaviors. The program did not have a direct effect on delinquency, but indirectly affected it via prosocial behavior and internalizing and externalizing behaviors.

Also at 36 months after baseline, or two years after program completion, the intervention foster girls reported significantly lower tobacco use, marijuana use, and health-risking sexual behavior. The reduction in rates of health-risking sexual behavior was mediated by the intervention-driven reduction in substance use (tobacco and marijuana use).

Outcomes

Intervention girls, relative to control girls, demonstrated significant effects for:

  • Internalizing and externalizing problem behaviors at six months.
  • Prosocial behavior combined across the 6-month and 12-month assessments.
  • Placement stability at the 12-month posttest.
  • Substance use at 36 months beyond baseline (2 years post-intervention).
  • Health-risking sexual behavior at 36 months beyond baseline (2 years post-intervention).
  • Small to moderate mediating effects of prosocial, internalizing, and externalizing behaviors on long-term outcomes of substance use.
  • Small to moderate mediating effect of tobacco and marijuana use on long-term health-risking sexual behavior.

Mediating Effects

The researchers found that there were small-to-moderate mediating effects of prosocial, internalizing, and externalizing behaviors on the long-term outcomes of substance use (beta = -.04) and delinquency (beta = -.07), and small-to-moderate mediating effects of tobacco and marijuana use on health-risking sexual behavior (beta = -.12).

Effect Size

Effect sizes were generally moderate at the two-year follow-up, with Cohen’s d ranging from .45 to .50 for significant outcomes and with correlations of the intervention and outcomes ranging from -.22 to -.28.

Generalizability

These studies are generalizable to urban and rural foster girls entering middle school in the northwestern U.S.

Limitations

Limitations of studies (Smith et al., 2011; Kim & Leve, 2011; Kim et al., 2013) included:

  • Models for long-term outcomes of substance use, delinquency, and sexual behavior lacked controls for baseline outcomes.
  • Results were typically reported at six months and 36 months post-baseline. There were no reports of posttest and one year posttest results for substance use, delinquency, and sexual behavior.
  • Although the measure for externalizing and internalizing at 6 months is a composite measure of caregivers and parents, the measurement at 2 years post intervention is by caregiver report only. Since the caregivers are direct recipients of the program, just as the girls, the reporting using this measure could be biased.

Endorsements

Blueprints: Promising

Peer Implementation Sites

Britany Binkowski
Assistant to the Commissioner
Office of Child Welfare Reform
UBS Tower, 10th Floor
315 Deaderick St., Nashville, TN 37243
o. 615-741-1405
c. 615-708-9084
Britany.Binkowski@tn.gov

Sylvia Rowlands
Senior Vice-President
Evidence-Based Programs
590 Avenue of the Americas
New York, New York 10011
(212) 660-1342
sylvia.rowlands@NYFoundling.org

Program Information Contact

Patricia Chamberlain
Oregon Social Learning Center
10 Shelton McMurphey Boulevard
Eugene, OR 97401-4928

References

Study 1

Certified Kim, H., & Leve, L. (2011). Substance use and delinquency among middle school girls in foster care: A three-year follow-up of a randomized controlled trial. Journal of Consulting and Clinical Psychology, 79(6), 740-750.

Kim, H. K., Pears, K. C., Leve, L. D., Chamberlain, P. C., & Smith, D. K. (2013). Intervention effects on health-risking sexual behavior among foster care girls: The role of placement disruption and substance use. Journal of Child and Adolescent Substance Abuse, 22(5), 370-387.

Kim, H., & Leve, L. (2011). Substance use and delinquency among middle school girls in foster care: A three-year follow-up of a randomized controlled trial. Journal of Consulting and Clinical Psychology, 79(6), 740-750.

Smith, D., Leve, L., & Chamberlain, P. (2011). Preventing internalizing and externalizing problems in girls in foster care as they enter middle school: Immediate impact of an intervention. Prevention Science, 12(3), 269-277.

Study 1

Evaluation Methodology

Design: This randomized controlled trial of the KEEP SAFE (Middle School) program was conducted with girls in foster care from two counties (one urban, one rural) in Oregon. Girls were eligible if they were finishing elementary school between 2004 and 2007 and were referred to foster care through the local child welfare system. Researchers recruited girls and their foster parents from the pool of eligible participants (N=145). Eligibility criteria required that girls not yet be reunited with their original family and that the foster parents provide consent. Recruitment occurred on a rolling basis and stopped when enrollment reached 100. The participants were randomly assigned to the intervention (n=48) and the control condition (n=52) which consisted of the usual services provided by the local child welfare organizations.

After randomization the girls and their foster parents were assessed at baseline (T1) and at 6 months post baseline (T2). The girls were assessed again at 12-, 24-, and 36-months post baseline, which represent the periods shortly after the full program ended (T3), one year after the program ended (T4), and two years after program ended (T5), respectively. The baseline assessment consisted of a standardized interview and questionnaires for each girl and foster parent, an interview with the girl's caseworker, and the collection of child welfare records. The interviews lasted approximately two hours and were conducted in person by assessors who were blind to group assignments. The data collection instruments were designed to measure child and family characteristics, child behaviors, and parenting practices.

The summer component of the program was implemented in the summer before commencement of middle school. The mean time between baseline (T1) and the 6-month follow-up (T2) was 147 days (s.d.= 45.6). When changes in placement/caregiver occurred between baseline and T2 (20% of sample), the girl was followed to her new placement and the girl's new caregiver was recruited to participate in the study. The authors did not report when the transitions to new caregivers occurred.

Booster sessions took place during the first year of middle school. On average, parents attended ten weekly meetings between the completion of the summer session and the T2 assessment. The authors did not report on the specific frequency of the girls' attendance in follow-up sessions, but "participation rates mirrored those of their caregivers" (Kim and Leve, 2011).

The baseline (T1) sample size for the intervention group was 48 and subsequent sample sizes were 48, 47, 44, and 45, respectively, for T2, T3, T4, and T5. Of the three subjects that were lost, one could not be located and two refused to participate. The baseline (T1) sample size for the control group was 52 and subsequent sample sizes were 50, 50, 48, 45, respectively, for T2, T3, T4, and T5. Of the seven subjects that were lost, three could not be located and four refused to participate. Thus, attrition was about 10% percent from baseline to the 2-year post-intervention follow-up.

Sample Characteristics: The average age of the girls at baseline was 11.5 years. The ethnic distribution was 63% European American, 10% Latino, 9% African American, 4% Native American, and 14% multiracial. At baseline, the girls had been in foster care an average of 2.9 years and had experienced an average of 1.4 out-of-home placements. In this sample, 56% of the girls had at least one documented incident of physical abuse, 67% had at least one documented incident of sexual abuse, and 82% had at least one documented incident of physical or sexual abuse.

Measures: Measures were divided into categories of short-term outcomes, long-term outcomes, and predictors.

For short-term outcomes, the Parent Daily Report Checklist (PDR) was used to measure internalizing problems, externalizing problems, and prosocial behavior. This 34-item measure of child behavior problems was administered by telephone individually to foster parents and girls on three consecutive or closely spaced days. In each call, subjects were asked about the preceding 24-hours to improve reliability of the measure. The PDR has been shown to be both valid and reliable.

  • Internalizing problems: An internalizing problems composite was computed based on five of the PDR items. The scores were averaged across the three telephone calls at T2 (alpha=.72 for parents and .74 for girls). The parent and girl scores were significantly correlated and were thus combined into a composite internalizing problems score at 6 months. At two years, the report was by caregivers only.
  • Externalizing problems: An externalizing problems composite was computed based on 18 of the PDR items. The scores were averaged across the three telephone calls at T2 (alpha=.85 for parents and .81 for girls). The parent and girl scores were significantly correlated and were thus combined into a composite externalizing problems score at six months. At two years the report was by caregivers only.
  • Prosocial behavior: A prosocial behavior composite was computed based on 11 of the PDR items. The scores were averaged across the three telephone calls at T2 (alpha=.74 for parents and .75 for girls). The parent and girl scores were significantly correlated and were thus combined into a composite prosocial behavior score.

Placement changes in foster care were measured from welfare system records as the sum from the start of the study through posttest. The number of placement changes ranged from 0 to 7 during this period, and the mean was .56.

Long-term outcomes included substance use, delinquency, and health-risking sexual behavior.

  • Substance use: Three indicators were used to assess substance use at T5: How many times in the past year have you (a) smoked cigarettes or chewed tobacco; (b) drunk alcohol; and (c) used marijuana. The response scale ranged from 1 (never) through 9 (daily). Tobacco and marijuana use were combined to create a two-item composite score. The authors did not report on reliability or validity for this measure.
  • Delinquency: Delinquency was assessed with two indicators. First, girls' own delinquent behavior was measured with the 36 items from the general delinquency scale, which is a part of the Self-Report Delinquency Scale (SRD). Girls were asked how many times in the past year they had committed various acts (e.g., damaging or destroying property, stealing). The mean of the frequencies across the items was used to represent the level of delinquency. Internal reliability of this measure with this sample was alpha=.85. Second, the girls' association with delinquent peers was measured with 30 items from a modified version of the general delinquency scale from the SRD. Girls were asked how many of their friends were involved in delinquent acts (26 items scored 0="none" to 4="all") and how often their friends used alcohol, tobacco, and marijuana (4 items scored 0="none" to 4="a lot"). Because of different response scales for the first 26 items and the last 4 items, the two item sets were each standardized before combining to compute a mean value. The internal reliability of this scale was alpha=.96.
  • Health-risking sexual behavior: Eight items from the girls’ in-person interviews were used to assess health-risking sexual behavior at T5. The girls reported on items such as touching a boy’s body above or below the waist, having sexual intercourse, having sex with someone who they just met, or having sex with someone using drugs in the past 12 months. Positive answers to these items were totaled to represent the cumulative number of health-risking sexual behaviors. Internal reliability of the measure was .67.

Predictor measures included:

  • Maltreatment History: The girls' cumulative maltreatment history at T1 was coded from child welfare case files using a modified version of the Maltreatment Classification System. Trained coders examined these case files to identify incidents of physical and sexual abuse. Two-thirds of the files were double-coded to compute interrater agreement. Agreement on the identification of the number of physical and sexual abuse incidents was high (85.7% and 86.2%, respectively). The maltreatment history variable consisted of the sum of the number of incidents prior to the study.
  • Pubertal Development: Pubertal development was measured at T1 using girl and foster parent reports on the Pederson Scales of Development. This measure has been shown to be reliable and valid. The Pederson Scales of Development uses a 4-point Likert scale to rate items such as body hair changes, skin changes, breast growth, and changes in height. Menstruation was measured as a dichotomous variable. A pubertal development score was computed using parent data and girl data. These scores were highly related and therefore averaged to create a mean pubertal development score.

Analysis: For the 6-month assessment, Smith et al. (2011) ran separate stepwise hierarchical linear regression models, one for each T2 outcome (internalizing, externalizing, prosocial behaviors). The first step consisted of the general predictor variables (baseline age, maltreatment history, pubertal development, and behavior) and the second step included the intervention dummy. The study reported beta coefficients, but did not state that the coefficients were standardized (authors state in personal correspondence that coefficients were standardized).

The two-year posttest follow-ups (Kim & Leve, 2011; Kim et al., in press) conducted structural equation modeling. To use the full intent-to-treat sample (n=100), full information maximum likelihood (FIML) estimation was used. This method has been shown to provide unbiased estimates when data are missing at random. Little's MCAR test indicated that the data were, in fact, missing completely at random. The composite tobacco and marijuana use variable was log transformed and the number of placement changes was square root transformed to more closely resemble normal distributions. Baseline outcomes of substance use, delinquency, and health-risking sexual behavior were not measured at baseline or used as predictors.

Outcomes

Fidelity: The interventionists were supervised weekly, and would regularly be shown videotapes of sessions and provided with feedback regarding adherence to the clinical model. The authors did not report how well the interventionists adhered to these guidelines. Caregivers attended 5.62 of the 6 summer sessions, and 20 of the 40 follow-up sessions during the year offered, on average. Participation rates of girls in the summer sessions mirrored those of their caregivers. Of the 40 follow-up sessions offered to youth, average attendance was 56.4%.

Baseline equivalence: Kim and Leve (2011) found no significant differences on demographic characteristics, age at first placement, number of placements, foster care type, history of delinquency, history of special services, internalizing behavior, or externalizing behavior. Only the baseline measure of severity of neglect differed significantly, but it had no influence in any of the models.

Differential attrition: The authors did not report on the characteristics of attriters vs. non-attriters, yet Little's MCAR test indicated that data were missing completely at random.

Posttest: Smith et al. (2011) examined internalizing problems, externalizing problems, and prosocial behavior at about six months after baseline (conclusion of the summer component, but still during booster sessions occurring in the first middle school year). The program was significantly related to a decrease in internalizing (beta = -.28, p<.01) and externalizing (beta = -.21, p<.01) problems, but not significantly related to prosocial behavior.

Kim and Leve (2011) examined placement changes, internalizing behavior, externalizing behavior, and prosocial behavior at 6-month and posttest combined. Girls in the intervention had significantly fewer placement changes (Cohen's d = .50) and significantly greater prosocial behavior (Cohen's d = .46). There were no significant differences on externalizing and internalizing symptoms (Cohen's d = .02).

Long-term: In Kim and Leve (2011), the intervention group reported moderately lower levels of substance use (composite of tobacco, alcohol, and marijuana) than girls in the control group (Cohen's d = .47) at the two-year posttest follow-up. When the substances were examined separately, the program did not have a significant effect on alcohol use but did significantly affect tobacco use (Cohen's d = .45) and marijuana use (Cohen's d = .57). Delinquency (a composite of the girl's own delinquency and her association with delinquent peers) was marginally significantly lower for intervention girls.

In Kim et al. (in press), girls from the intervention showed significantly lower levels health-risking sexual behavior (Cohen’s d = .48) than the control group. Post-hoc analysis indicated that only 4.4% of the girls in the program, as opposed to 17.8% of the girls in the control condition, reported having sexual intercourse in the past year (chi-square = 4.05, df = 1, p = .04).

Mediation: First, the program had significant indirect effects (beta=-.04) on substance use through increased positive prosocial behavior at T2 and T3 and reduced internalizing and externalizing behaviors at T3 and T4. Second, the program did not have a direct effect on delinquency but did have an indirect effect (b=-.07) on delinquency through increased prosocial behavior at T2 and T3. Third, the program had a significant indirect effect (beta=-.12) on health-risking sexual behavior through reduced tobacco and marijuana use.

Contact

Blueprints for Healthy Youth Development
University of Colorado Boulder
Institute of Behavioral Science
UCB 483, Boulder, CO 80309

Email: blueprints@colorado.edu

Sign up for Newsletter

If you are interested in staying connected with the work conducted by Blueprints, please share your email to receive quarterly updates.

Blueprints for Healthy Youth Development is
currently funded by Arnold Ventures (formerly the Laura and John Arnold Foundation) and historically has received funding from the Annie E. Casey Foundation and the Office of Juvenile Justice and Delinquency Prevention.