Treatment Foster Care Oregon

Study 1

Chamberlain et al. (1997) reported that boys were in treatment for from 1 day up to 9 months.

Summary

Chamberlain (1997), Chamberlain et al. (1996), Eddy et al. (2004), and additional articles used a randomized controlled trial to examine 79 boys who had been mandated for out-of-home care and were assigned to the intervention or control group. The boys were followed for two years and then surveyed as adults to investigate delinquency, substance use, and arrests.

Chamberlain (1997), Chamberlain et al. (1996), Eddy et al. (2004), and additional articles found that, relative to the control group, the intervention group had significantly:

• lower negative peer influence and frequency of problem behaviors at three months
• less time incarcerated, fewer arrests, and fewer criminal activities at 12 months
• lower use of tobacco, marijuana, and other drugs at 18 months
• lower violent offending at 24 months.

Evaluation Methodology

Design:

Recruitment: The study recruited 85 boys aged 12-18 who were mandated into out-of-home care by the juvenile court between 1991 and 1995. All of the boys had previously been placed out of their homes at least once and had spent an average of 76 days in detention, but they were not deemed in need of substance abuse treatment or incarceration. They lived in a medium-sized metropolitan region and surrounding rural areas in the Pacific Northwest.

Assignment: The study randomly assigned the 85 participants to a group-care, peer-mediated control group (n = 40) or to the Multidimensional Treatment Foster Care intervention group (n = 45). Consent followed randomization, and parents of three boys in each condition declined to participate after assignment, leaving 79 total, 42 in the control group, and 37 in the intervention group. Five boys assessed at baseline failed to be placed in either the intervention (n = 1) or control program (n = 4) and were sent home to live. However, these five were analyzed in their originally assigned condition.

Assessments/Attrition: Assessments occurred at baseline and at 3, 6, 12, 18, and 24 months following baseline. One additional assessment followed 9-13 years after baseline. The 24-month and 9-13-year assessments served as long-term follow-ups.

All assessments lost 7% of the sample immediately after randomization. Otherwise, reported attrition varied across reports. Chamberlain and Reid (1998) reported degrees of freedom consistent with no other missing data, but Chamberlain et al. (1996, 1997) appeared to have additional missing data. Eddy and Chamberlain (2000) had complete data on only 67% of the 79 participants. Eddy et al. (2004) reported that, across 79 youths at four time points, 8% of the data were missing, and that at 24 months after baseline, 15% of the data were missing.

Sample: The mean age at entry into the study was 14.9 years. By race, 85% were Caucasian, 6% African American, 3% American Indian, and 6% Hispanic.

Measures:

The outcomes included a variety of self-reported and official-record measures. Researchers who were blind to condition obtained 90% inter-rated reliability in coding criminal referrals.

• Chamberlain et al. (1996) obtained measures three months after placement from interviews of caretakers and boys using the Parent Daily Report Checklist to assess problem behaviors over the past 24 hours. The authors reported high inter-interviewer reliability.
• Chamberlain et al. (1997) and Chamberlain and Reid (1998) examined measures from official records, either arrests or criminal referrals, and from self-reports of delinquency and offending.
• Eddy et al. (2004) focused specifically on official records of violent offenses and self-reported violent behavior.
• Smith et al. (2010) focused on self-reported substance use (tobacco, alcohol, marijuana, and other drugs.
• Rhoades et al. (2016) examined adult arrests at 9-13 months after baseline.

Analysis:

Chamberlain et al. (1996) used bivariate ANOVA without controls for baseline outcomes, while Chamberlain et al. (1997) and Chamberlain and Reid (1998) used multiple regression with controls for the baseline outcome or several baseline risk measures (age at first arrest, age at baseline, and number of prior offenses). Eddy and Chamberlain (1998) and Smith et al. (2010) used structural equation models.

Eddy et al. (2004) used generalized estimating equations for repeated measures, controlling for baseline outcome, age at first criminal referral, and age at placement in treatment. The tests focused on a binary measure of condition rather than on the condition-by-time interaction typically used in repeated-measures models.

Missing Data Method: Chamberlain et al. (1996, 1997) used complete-case analysis, but the analytic sample size was not always made explicit. Chamberlain and Reid (1998) appeared to have no missing data. Eddy and Chamberlain (2000) used complete-case analysis. Eddy et al. (2004) and Smith et al. (2010) included all participants, despite having attrition and missing data, by using generalized estimating equations or full information maximum likelihood estimation.

Intent to Treat: None of the analyses included the six participants who dropped out immediately after randomization and had no baseline data. Chamberlain et al. (1996, 1997) provided no details on the extent or causes of missing data, whereas Chamberlain and Reid (1998) appeared to have no missing data, and Eddy et al. (2004) included all participants.

Outcomes

Implementation Fidelity:

Chamberlain and Reid (1998) reported that intervention youths completed their program significantly more often and remained in their placement significantly longer than control participants.

Baseline Equivalence:

Table 1 in Chamberlain et al. (1996) lists condition means on age at first arrest, age at intake, number of arrests, and 17 risk factors. Of the 20 baseline measures, two were significantly different: perpetrated a sex offense (10% in the control group, 38% in the treatment group), and being two years below grade level (66% in the control group, 22% in the intervention group. Two significant differences would be expected in 20 tests, and the direction of the differences does not favor one condition over the other, but the percentage differences are quite large. Chamberlain and Reid (1997) reported on 16 baseline risk factors that did not differ significantly. Smith et al. (2010) add that "no significant treatment condition differences on baseline levels of substance use were found."

Differential Attrition:

Chamberlain et al. (1996, 1997) did not mention attrition, although there appears to be a loss of data at follow-up. Attrition in Eddy and Chamberlain (2000) was related to baseline criminal referrals in the days prior to detention. However, the sample with complete data did not differ significantly for baseline age, criminal referrals, self-reported crimes, felony referrals, or the age of first criminal referral.

Eddy et al. (2004) provided more detail on the 8-15% missing data. They noted that group assignment, self-reported delinquency, and official criminal referrals were not significantly associated with the presence of missing data. The missing data rates were 10% in the control group and 6% in the intervention group, which meets both the WWC cautious and optimistic standards.

Smith et al (2010) stated that "The results of using Little's test of missingness for all data in the SEM models indicated that these data met the assumptions for missing completely at random."

Posttest:

Three Months. Chamberlain et al. (1996) examined interim outcomes at three months post-baseline and found that the intervention increased caretaker use of discipline and reduced contact with and influence of peers.

12-Month Posttest. Chamberlain et al. (1997) and Chamberlain and Reid (1998) examined outcomes at posttest (one year after baseline). The intervention group had significantly fewer official arrests and criminal referrals than the control group. Also, intervention youths were incarcerated significantly less than control youths. Significant results similarly emerged for measures of self-reported general delinquency, index offenses, and felony assaults.

12 and 18 Months. Smith et al. (2010) found that the boys in the experimental condition had significantly lower levels of self-reported other drug use at 12 months (but not tobacco, marijuana, or alcohol use), and lower levels of tobacco, marijuana, and other drug use at 18 months (but not alcohol use). Models with and without a control for baseline substance use were not substantively different, so the authors reported the results without the control.

Long-Term:

24 Months. Eddy et al. (2004) examined a combined period from baseline through 24 months after baseline (one year after the posttest). Table 1 shows a significant effect (p < .05, two-tailed) on self-reported violence but only a marginal effect on official violent referrals (p < .05, one-tailed test).

24-Month Mediating Effects. In Eddy and Chamberlain (2000), family management skills and deviant peer association at three months after baseline significantly mediated the effect of treatment condition on youth antisocial behavior (i.e., a scale combining self-reported delinquency and official criminal referrals) at one year after exit from treatment. Leve and Chamberlain (2005) examined a sample that combined 72 boys from this study with 81 girls from another study. Their results demonstrated that the association with deviant peers during treatment mediated the effect of the treatment condition on deviant peer associations at 12 months post-treatment.

Nine to 13 Years. Rhoades et al. (2016) examined the sample boys about 9-13 years after recruitment, when most had reached adulthood; it excluded those not yet age 20. The analysis focused on the determinants of adult arrests, with condition included only as a control variable. Nonetheless, the Cox proportional hazard models of the time to first offense after age 18 (Table 3, columns 1-4) found no significant effects of the intervention on any arrest (p = .88) or felony arrest (p = .52).

Study 2

Summary

Chamberlain (1990) used a quasi-experimental design to examine 16 youths in foster care who received the program and 16 matched comparison youths who received other treatments. The study followed the youths for two years to measure time spent incarcerated.

Chamberlain (1990) found that, relative to the control group, the intervention group had significantly:

• more successful completion of treatment at posttest
• fewer days institutionalized at one year
• fewer days incarcerated at two years.

Evaluation Methodology

Design: Participating youth (n=16) had been committed to a training school in Oregon and then diverted to the Specialized Foster Care (SFC) program at the Oregon Social Learning Center. The 16 comparison group members were randomly selected from a pool of 435 youths based on their commitment and diversion to traditional community treatment programs, such as group homes, intensive parole supervision, or residential treatment centers, and matched to the SFC participants on sex, and age and date of commitment (within a three-month window. Four pairs were matched within a six-month window). The treatment youth were placed with foster parents who were chosen on the basis of their positive parenting skills and family environment.

The application process was three-fold. First, applications were filled out and references checked. Then a home visit occurred in order to explain the program and observe the home and family environment. Lastly, eligible families attended an eight-hour training session held by an experienced foster parent and the program director. Parents who were chosen were trained and supervised specifically according to the Social Learning Family Therapy approach. Participating families received one SFC youth, and followed the Treatment Foster Care model, including the behavior management level system for youth, daily monitoring through progress reports with a case manager, weekly individual youth therapy sessions, and family therapy with biological parents (when available). Among the comparison group subjects, eight were placed in group homes, four in a secure residential treatment center, two in their parents' homes with intensive parole supervision, and two in a program conducting an application of the SFC model in another community.

Sample: Participants in this study were six girls and ten boys. Comparison youth were chosen according to the criteria outlined above. Average age of youth in both groups was 14.6. Group differences were then assessed based on a multitude of risk indicators of child maladjustment, including family risk factors, child risk factors, child dangerousness, and child school adjustment. The only significant difference found between the treatment and comparison group was on the greater proportion of treatment youth who had been adopted. Overall, subjects in the treatment group were comparable to but somewhat more at-risk than their counterparts in the control group.

Measures: Oregon Children's Service Division (CSD) records, which track the number of days in out-of-home placements were examined at three time periods: pretreatment - the number of days that the youth was incarcerated in either of the two Oregon state training schools during the one-year period prior to placement in the diversion program; days in treatment - the number of days that the youth participated in a CSD-funded diversion program; follow-up - the number of days that the youth was incarcerated during the two years post-treatment.

Analysis: A comparison of means was used to determine the average amount of treatment received by both groups. The number of days incarcerated was counted independently by two research assistants for each of the three time periods. Intercoder agreement was calculated by dividing the number of agreements by the number of agreements plus disagreements, yielding a reliability coefficient of 96%.

Outcomes
Post-test: There were no statistically significant differences between the treatment and comparison groups prior to treatment. There were also no statistically significant differences found in the average amount of days spent in the treatment phase (142 for treatment group compared to 146 among comparison group). A higher proportion of youth placed in the SFC program successfully completed their treatment (75%) than comparison youth (31%). Interestingly, two of the five youth in the comparison group who were successful in completing treatment were those participating in the other community application of SFC. Reasons for failure to complete treatment among both groups included revocation through incarceration (three treatment youth compared to 4 comparison youth), or runaway (one treatment youth compared to seven comparison youth).

Long-term: During the year following treatment, 38% (6) of the treatment youth were reinstitutionalized, compared to 88% (14) of the control group youth. In year two of the follow-up, seven of the treatment youth and ten of the comparison youth were incarcerated. Comparison youth also spent a higher average of days incarcerated (66.8 days compared to 44.3 days among treatment youth). Overall, during the two years following treatment, eight of the SFC youth and 15 of the comparison group youth were reincarcerated at least once. This difference was statistically significant. There was also a significant correlation between the number of days in treatment and the number of days of subsequent incarceration. The more days spent in treatment, the fewer days later spent incarcerated. No such relationship was found for the comparison group.

Study 3

Summary

Chamberlain and Reid (1991) used a randomized controlled trial to examine 20 youths in foster care who were assigned to intervention or alternative treatment groups. Assessment occurred at three and seven months after baseline and measured emotional disturbance, problem behavior, and institutionalization.

Chamberlain and Reid (1991) found that, relative to the control group, the intervention group had significantly:

• faster rates of placement in the community after three months in the program
• lower daily reporting of problem behavior after three months in the program.

Evaluation Methodology

Design: Participating youth (n=20) were referred from the state mental hospital and then randomly assigned to either the Specialized Foster Care (SFC) group or to existing alternative treatment groups in their communities. This evaluation looked specifically at how the treatment foster care program would benefit a population of severely disturbed youth.

The SFC model used for treatment participants included recruitment and screening of foster parents, preservice training, daily management of the child in the home and community, ongoing supervision and support for foster parents, family treatment, individual child treatment, and case management and community liaison services. Of the 10 control participants, 7 were placed in community settings during the evaluation. The other three youth remained in the state hospital. Of the 7 community placements, 3 were sent to residential centers, including a juvenile corrections training school, a group home, and a secure residential treatment center. Four went to family or relative's homes. The treatment received by control subjects included milieu therapy for those in the residential centers and the hospital. Types of therapy ranged from highly structured to more general. All but one control youth received some individual therapy. Amount of therapy ranged, with the least amount received by youth placed in a home setting. Group therapy also occurred for youth placed in the residential settings and for 2 of the 4 subjects placed in the home settings.

Sample: Participants were referred to the study by a multidisciplinary team of staff from the Oregon State Hospital. Team members had worked with each case. Referrals were ready for community placement. After referral, subjects were randomly assigned to the treatment or control condition. A total of eight males and 12 females participated in the evaluation. Average age of participants was 13.9 years for treatment group and 15.1 for control group. Treatment participants had an average of 5.1 out-of-home placements prior to the current hospitalization, whereas control participants had an average of 5.0. Analyses revealed no significant differences between the two groups with regard to family make up, risk variables, or special clinical concerns (suicide attempts, drug/alcohol dependency, multiple runaways, chronic truancy, sexual abuse). Between both groups, four treatment and two control subjects had IQ scores that were at least one standard deviation below the normal range. On average, treatment participants had slightly lower IQ scores than those in the control group, although this difference was not statistically significant. During the year preceding referral to the study, treatment youth had been hospitalized an average of 245.1 days, compared to the 236 days for control youth.

Measures: All youth were assessed at baseline on their severity of emotional disturbance, level of social competency, self-reported symptoms, and the occurrence of problem behaviors. At 3 and 7 months later, all measures except the severity of emotional disturbance were assessed again. To measure severity of emotional disturbance, youth took the Child Global Assessment Scale (CGAS) to measure level of functioning. Measures taken at baseline, and then again at 3 and 7 months included: Parent Daily Report Checklist (PDR) to measure occurrence of problem behaviors on a daily basis (phone monitoring by case manager); Behavior Symptom Inventory (BSI) to measure level of symptoms and distress (self-reported); Social Interaction Tasks used to measure child's level of social skills and problem solving. The Adolescent Problem Inventory (API) was used for youth 12 years and older, and the Taxonomy of Problematic Social Situations (TPOS) was used for children under 12. To gauge success or failure of each case, institutionalization rates were tabulated during three time periods: 1) the year prior to referral, 2) time from referral to initial placement out of the hospital, and 3) time from initial placement through the subsequent 365 days.

Analysis: A comparison of means was used to determine differences in institutionalization rates. Analysis of variance (ANOVA) was used on the measure of occurrence of problem behaviors. On the measure of level of social competency, analysis on the TPOS was calculated by dividing the number of agreements by the number of agreements plus disagreements at baseline.

Outcomes
Post-test: Once referred, experimental subjects were placed outside the hospital after an average of 81 days, as compared to the 182 days to placement for control youth (p = .05). Once placed, treatment youth spent an average of 288 days living in their communities. Three of these participants were rehospitalized during the first 6 months, and one was briefly hospitalized (for 10 days) and then returned to the foster home. Control subjects placed outside of the hospital (n=7) spent 261 days in the community. Two were rehospitalized during the first 6 months and 1 was briefly hospitalized (for 3 days) and returned to community placement. The difference in days spent in the community was not statistically different, although it should be noted that fewer control subjects received community placement. For those participants between both groups who were only placed in family homes (all treatment, 4 of 10 control), the difference in number of days in the community (288 for treatment, 251 for control) was not statistically significant. On the pretreatment measure of severity of emotional disturbance, analysis revealed that both groups fell into the second lowest category - major impairment in functioning in several areas. On the measure of occurrence of problem behaviors, complete PDR data were available for 7 youth in each group at baseline, 3 months, and 7 months post baseline. At baseline, mean daily rates of problem behaviors were high (over 20 reported problems per day). At 3 months, treatment youth rates dropped over 50%, while control youth rates showed no decrease. At 7 months, mean daily rates among the control group decreased, but not to the level of the treatment group. There was no overall significant group by time interaction at 7 months, although the group by time interaction did reach significance when comparing baseline to 3 month data (p<.05). However, it must be noted that at the 3 month period, only 3 of the control subjects had been placed in a community setting, whereas 7 of the treatment youths had been placed. Due to the small sample size and potential differences in the quality of ratings between hospital caretakers and parents, caution must be exercised in evaluating these results. However, data does suggest that youth did seem to show behavioral improvements when moved to less restrictive specialized foster care settings. On the measure of self-reports of symptoms, at baseline, treatment youth reported twice as many problems as control youth. At 7 months, treatment youth were still reporting more distress than control youth, but these differences were no longer statistically significant. On the measure of social competency, no improvement was seen for either group. In fact, declines for both groups were evident from pre- to post-tests (differences were not statistically significant). On the measure of success (i.e., those living in the community at follow-up; n=14) or failure (i.e., those rehospitalized at follow-up; n=6) of cases at follow-up, only 2 of 6 cases where the subject had a below average full scale IQ score was successful, compared to 12 of 14 cases where IQ score fell within the normal range.

No long-term follow-up evaluation was conducted.

Study 4

Summary

Chamberlain et al. (1992) used a randomized controlled trial to examine 72 children placed in foster care and assigned to an enhanced training group, an increased payment group, or a foster care as-usual group. Assessments at baseline and three months later measured retention and stability of foster care.

Chamberlain et al. (1992) found that, compared to the control group, the intervention group had significantly:

• fewer failures in their foster care placement
• problem behaviors.

Evaluation Methodology

Design: This study was designed to evaluate the effects of an increased stipend and enhanced support and training for foster parents. Participating children and their foster parents were placed into one of three conditions: enhanced support and training (ES&T) plus an increased payment of $70/month (n=31); increased payment of $70/month only (IPO; n=14); or foster care as usual (n=27).

Sample: The sample included 72 children from three Oregon counties placed in foster care between 1988-1990 and their foster parents. Children were from four to seven years old and were expected to have been in foster care for at least three months. The children were predominantly Caucasian, and a majority were female. The leading stated reason for placement into foster care was parental neglect, followed by physical and sexual abuse. Foster parents were largely two-parent households (85%), with both parents in their early 40s. Average level of education among foster parents was some college, but not completed degrees. Average annual income was $20,000-$24,900. Families had an average of three biological children in the household. Foster parents in this evaluation had cared for an average of 21 foster children, indicating a high level of experience at providing care. There were no significant differences between the three conditions with regard to demographics or level of experience.

Measures: Child behavior was measured using the Parent Daily Report, which measured problem behaviors on a daily basis. Data was collected at baseline and then at the three month mark. Foster parent measures included a dropout/retention rate, collected from Children's Services Division certifiers. A Staff Impressions Measure was used by study staff to rate impressions of foster parents' skills at discipline, their impression of the foster mothers' levels of personal strength, and the foster parents' levels of social skill. Surveys were also used by foster parents and caseworkers to determine the effectiveness of weekly training and support groups. These were administered only to those in the ES&T group. Children were also monitored on stability of foster care, by ongoing checks to determine whether foster children had left the home and if so, under what circumstances (returned home, ran away, or was moved to another foster home, residential or group care, juvenile detention, or psychiatric hospital.

Analysis: A repeated measures analysis of change from baseline to the three month period was conducted on the parent daily report data. On the measure of staff impressions of foster mothers' ability to discipline appropriately, social skills, and level of personal strength, mean scores from each scale were compared.

Outcomes
Post-test: On the measure of foster parent retention, after two years, 16.6% of the foster families participating in the study discontinued providing foster care (compared to 40% statewide). Among the individual groups, the ES&T (9.6%) and IPO (14.3%) groups, which received additional services or a larger stipend, had lower dropout rates than the foster care as usual group (25.9%). On the measure of child outcomes, 18 of the 72 participating children were returned home during the two year period, due primarily to improvements observed by the caseworker in the family of origin's situation. Among those remaining, children stayed with their original foster home, were moved to another home, ran away, or were placed in a more restrictive setting. Among these remaining children, those in the ES&T group had significantly more successful days in care than children in either of the other two conditions. When combining the two treatment groups and comparing them to the control condition, the treatment group children had significantly fewer failures in their foster care placements (29% compared to 54%). On the measure of child behavior problems, at baseline, the ES&T group had a higher-than-normal frequency of daily problem behaviors than the other two conditions. However, by the three month follow up, foster parents in the ES&T group reported the greatest decline in problem behavior rates relative to the other two conditions. By three months, all three conditions were reporting problem behaviors in the normal range (3.85 - 4.56). The foster care as usual group actually slightly worsened over the three month period, as they had initially reported below normal levels of problem behaviors. On the measure of foster parent and caseworker satisfaction, foster parents overall reported satisfaction with the weekly group meetings, that the groups helped them deal effectively with their foster child's problems, and that they would definitely recommend the groups to other foster parents. Caseworkers also felt that the parents who participated in the weekly meetings benefited from the meetings, and that their ability to manage children's behavior problems improved. On the measure of staff impressions of the foster mothers' ability to discipline appropriately, social skills, and level of personal strength, significant differences were found on the discipline scale only. Foster mothers in the IPO group were rated as having significantly better discipline practices than those in the other two conditions.

Study 5

This study examined a version of the program for use with preschoolers that is considered an extension rather than a revision of the program for older foster children.

Summary

Fisher and Kim (2007) evaluated the preschool version of the program using a randomized controlled trial to examine 137 three- to five-year-old foster children (intervention n = 64 and regular foster care control n = 73). Assessments over the next 12 months measured child attachment behavior as rated by foster parents.

Fisher and Kim (2007) found that the intervention group, relative to the control group, did not differ significantly on any of the three outcomes at 12 months but showed greater improvement over time in two of the outcomes, parent-rated secure behavior and avoidance behavior.

Evaluation Methodology

Design:

Recruitment: Foster children of preschool age (3 to 5 years) who were entering into foster care placement through the Lane County Branch of the Oregon Department of Human Services were targeted for program participation. The children could be new to foster care, reentering care, or moving between foster placements, but their placement needed to be expected to last for at least three months. The study identified 137 eligible children over a 3.5-year recruitment period.

Assignment: With assignment occurring prior to consent, the 137 children were randomized to either the intervention condition (n = 64) or a regular foster care control condition (n = 73). Caseworkers and foster parents were then contacted for consent to participate by research staff members who were unaware of study condition. Consent was obtained for 57 (89%) in the intervention group and 60 (82%) in the control group. The authors stated that the refusal rates did not differ significantly across conditions.

Assessments/Attrition: Data collection occurred at each of five three-month intervals, including baseline, three months, six months, nine months, and 12 months. About 69% had data for all time points. As children typically received services for 9-12 months, including the period of transition to a permanent placement, the 12-month assessment represented a posttest.

Sample Characteristics: The sample was 49-58% male, 4.34-4.54 years of age on average, 89% European American, 5% Latino, 5% Native American, and 1% African American. The children had spent an average of 171 days in foster care prior to the baseline assessment.

Measures:

Foster parents rated children on three attachment-related measures using the Parent Attachment Diary: secure behavior (proximity seeking, contact maintenance), avoidant behavior (ignoring, moving away from caregiver), and resistant behavior (displaying angry behaviors towards caregivers). The authors reported no information on validity or reliability, only that the three items were correlated from -.39 to -.73.

Analysis:

Latent growth curve modeling was used to analyze data from T1 to T5 (baseline to 12-months). The models were estimated using the full information maximum likelihood estimator that included all participants, even those with only partial data. To capture non-linearity in over-time changes, a linear spline was used for individuals. The robust standard errors adjusted for clustering due to including siblings in the sample.

Intent to Treat: The FIML estimation used data from all 117 randomized participants.

Outcomes

Implementation Fidelity:

The study monitored treatment fidelity through progress notes and checklists completed by the clinical staff but did not report any quantitative information.

Baseline Equivalence:

The conditions did not differ significantly at baseline on child's age, gender, ethnicity, prior time spent in foster care, type of current foster placement, number of permanent placements that occurred during the study period, or the three outcome measures.

Differential Attrition:

The retention rates were significantly higher for the intervention group at three months, six months, and 12 months. However, there were no systematic differences between those who remained in the study and those who did not in terms of attachment-related behaviors and other internalizing and externalizing problem behaviors assessed at baseline.

Posttest:

None of the three outcomes differed significantly at 12 months (as shown by the coefficients for the intervention effects on the intercept). However, two of the three outcomes, secure behavior and avoidance behavior, differed significantly in the change over time (as shown by the coefficients for the intervention effects on the slopes). Although they were similar to the control children at the end of the study, the intervention children showed significantly more positive change for these two outcomes.

The interaction tests between intervention status and age at first foster placement revealed mixed moderation effects. Intervention children who were older when first placed made the greatest increases in secure behavior. However, those placed when older were also more likely to show resistant behaviors.

Long-Term:

Not examined.

Study 6

Summary

Leve et al. (2005), Leve and Chamberlain (2007), Chamberlain et al. (2007), and Buchanan (2008) used a randomized controlled trial to examine 81 girls mandated to out-of-home care for problems with chronic delinquency. The girls were randomly assigned to intervention or control groups and assessed through 24 months post-baseline to measure delinquency and educational engagement.

The results showed that, compared to the control group, the intervention group had significantly:

• fewer days in locked settings at 12 and 24 months post-baseline
• fewer criminal referrals at 12 months and 24 months post-baseline
• lower caregiver-reported delinquency at 12 months post-baseline
• more time spent on homework at 12 months post-baseline
• and lower self-reported delinquency at 24-months.

Evaluation Methodology

Design:

Recruitment: The study recruited 103 girls referred by juvenile court judges in Oregon between 1997 and 2002. The girls, who had all been mandated to out-of-home care for problems with chronic delinquency, were screened for eligibility on four criteria: (1) ages 13-17 years, (2) not currently pregnant, (3) at least one criminal referral in the prior 12 months, and (4) placement in out-of-home care within 12 months following the referral. A total of 81 (78.6%) met eligibility criteria and consented.

Assignment: The 81 participants were randomized to either the intervention group (n = 37) or a treatment-as-usual control group (n = 44) that received community-based care in group homes.

Assessments/Attrition: Girls and their current caregivers completed a baseline assessment and  12- and 24-month post-baseline assessments. Juvenile court records were obtained at 12 and 24 months. The length of the intervention averaged 174 days or about six months, but a large standard deviation of 144 days indicated that treatment for some continued for at least a year. The 12-month assessment accordingly serves as a posttest and the 24-month assessment as a long-term follow-up.

• Leve et al. (2005) reported no attrition for delinquency measures at 12 months.
• Leve et al. (2007) reported completion rates of 88% for school attendance and homework measures and 98% for lockup measures at 12 months.
• Chamberlain et al. (2007) reported a completion rate of 83% at 24 months.

Sample:

Girls were of an average age of 15.3 years. 74% were Caucasian, 2% were African American, 9% were Hispanic, 12% were Native American, 1% were Asian, and 2% were biracial or of another ethnicity. Over half (63%) were residing in a single-parent family and 32% lived in families with an income of less than $10,000. Prior to study entry, the average lifetime criminal referrals per girl was 11.9 and 70% had committed at least one felony. Nearly all girls had experienced prior maltreatment: 88% had documented physical abuse and 69% had documented sexual abuse.

Measures:

Measures were drawn from youth self-reports, caregiver reports about the youth, and official records. Since the caregiver components of the program taught parenting skills, and caregivers thereby helped deliver the program, these ratings may not be independent. Otherwise, staff members collecting the data were blind to group assignment.

In Leve et al. (2005) and Chamberlain et al. (2007), the outcome measures included delinquency, referrals, and days in locked settings. Referrals came from police and court records, days in locked settings came from youth reports, and delinquency measures came from both caregivers and youths. The delinquency scales used well-validated instruments and had good alpha reliabilities for the study sample. However, Leve et al. (2005) commented "that a number of girls underreported their delinquent activity, a potential reporting bias".

In Leve et al. (2007), the outcomes included measures of homework completion and school attendance, both combining highly correlated self- and caregiver reports.

Analysis:

The 12-month analyses in Leve et al. (2005, 2007) used ANCOVA with baseline outcomes as covariates. The 24-month analyses in Chamberlain et al. (2007) used Structural Equation Models, either latent outcome models or latent growth models, which both allowed for FIML estimation and a control for baseline outcomes.

Missing Data Method: Leve et al. (2005) reported that the 12-month analysis had no missing data. Leve et al. (2007) reported missing data and appeared to use complete cases for the main analysis and FIML for the mediation tests. Chamberlain et al. (2007) included all cases with FIML estimation.

Intent to Treat: The analyses included all randomized participants or all randomized participants with complete data, regardless of treatment duration.

Outcomes

Implementation Fidelity:

No information was presented, but Leve et al. (2007) demonstrated that, consistent with the program's activities, the intervention led girls to work more on homework during the program.

Baseline Equivalence:

The reports consistently affirmed baseline equivalence. Leve et al. (2005) stated that "There were no group differences on the rates or types of pre-BL offenses or on other demographic characteristics." They are more specific about the four outcome measures at baseline: "As indicated in Table 1, there were no significant mean-level differences on the BL delinquency measures by group." Leve et al. (2007) stated that "None of the baseline variables differed significantly by group." Chamberlain et al. (2007) stated that "There were no group differences on the rates or types of prebaseline offenses or on other demographic characteristics."

Differential Attrition:

At 12 months, Leve et al. (2005) made no mention of attrition or missing data, while Leve et al. (2007) reported attrition and missing data and noted that the data were missing completely at random according to the Little test. Neither report listed the figures needed to compute attrition by condition.

At 24 months, Chamberlain et al. (2007) also noted that the data were missing completely at random according to the Little test. They further stated that "there was no differential rate of attrition for those participants lost to follow up by group." The statement refers to attrition rates by condition, 13.5% for the intervention group and 20.5% for the control group. Although not statistically significant, the difference in attrition rates does not meet the WWC cautious standard.

Posttest:

6 Months Post-Baseline. Buchanan (2008) reported interim findings obtained while many participants remained in the program. There was no effect of the intervention on school engagement.

12 Months Post-Baseline: Leve et al. (2005) found two significant effects in four tests. The intervention group relative to the control group had significantly fewer days in locked settings (eta squared = .05) and lower caregiver-reported delinquency (eta squared = .07). Leve et al. (2007) found significant effects for both outcomes tested. The intervention group relative to the control group reported significantly more time on homework (eta squared = .09) and significantly more days of school attendance (eta squared = .07).

Mediation: Leve et al. (2007) found that the intervention indirectly reduced days in locked settings at the 12-month posttest via the mediator of homework completion during the intervention (i.e., at 3 to 6 months post-baseline).

Long-Term:

24-Months Post-Baseline: Chamberlain et al. (2007) found that the intervention significantly lowered a latent delinquency construct based on criminal referrals, days in locked settings, and self-reported delinquency (d = .65, eta-squared = .08). Additional analyses using a linear spline indicated a significantly greater rate of decrease in the latent construct from baseline to the 24-month follow-up for the intervention group than the control group.

Study 7

The intervention examined in this study added a component that targeted substance use and risky sexual behaviors.

Summary

Kerr et al. (2009) and nine other articles used a randomized controlled trial to examine 166 girls committed to out-of-home care because of chronic delinquency. The study randomly assigned the girls to intervention or group care conditions and measured multiple outcomes over  periods ranging from two years to ten years after baseline

The results showed that, compared to the control group, the intervention showed significantly:

• fewer pregnancies at two years
• fewer psychotic symptoms at two years
• lower illicit drug use at nine years
• fewer depressive symptoms at two years and nine years

Evaluation Methodology

Design:

Recruitment: Participants were 166 girls enrolled in one of two consecutive randomized controlled trials beginning in 1997. The girls, all Oregon residents, had been mandated to community-based, out-of-home care because of problems with chronic delinquency. Girls were 13-17 years of age at baseline and were only recruited if they had at least one criminal referral in the prior 12 months, were not currently pregnant, and were placed in out-of-home care within 12 months following referral. The sample combined two separate trials or cohorts, with the second adding some new components to the program.

Assignment: Girls were randomly assigned within each trial or cohort to the intervention group (n = 81) or a community-based group-care control group (n = 85). Assignment followed completion of the baseline assessment. However, in the first trial or cohort, the initial measure of depressive symptoms was obtained for both conditions at three months post-baseline rather than at baseline. Similarly, the initial self-reported measure of psychotic symptoms was obtained at three months post-baseline, and the initial clinical diagnostic measure of psychotic symptoms was obtained one year after baseline.

Assessments/Attrition: The initial set of assessments occurred at baseline and at 6, 12, 18, and 24 months. Table 2 in Kerr et al. (2009) shows completion rates of 95% at 12 months and 96% at 24 months for the study of pregnancy. Harold et al. (2013) reported completion rates of 99% at 12 months and 92% at 24 months for the study of depression. Van Ryzin and Leve (2012) and Poulton et al. (2014) also reported completion rates of 92% at 24 months.

Beginning at seven years after baseline, when the participants averaged 22 years of age, the study completed six telephone surveys, one approximately every six months, and one in-person survey. Although most analyses did not use all the surveys, the sample was followed for up to nearly 10 years after baseline. According to the detailed CONSORT diagram in Leve, Khurana et al. (2015), completion rates for individual surveys ranged from 80% to 91%. However, most reports noted the percentages of participants who completed at least one survey, which ranged from 89% to 94%.

Sample: By race, 74% of the girls were Caucasian, 2% African American, 7% Hispanic, 4% Native American, 1% Asian, and 13% reported mixed ethnic heritage. At baseline, 61% of the girls lived with single-parent families, and 32% of the girls lived with families earning less than $10,000.

Measures:

Measures came from participant self-reports, official records, and ratings by researchers unaware of condition (although assessors were indirectly made aware of condition at six months by the location of the interview). The numerous reports examined a wide variety of outcomes over the many assessments, including delinquency and crime, depression, substance use, and risky sex. The scales used had good reliability. Many measures came from single items that appeared straightforward but otherwise lacked validity information.

Analysis:

• Kerr et al. (2009) and Leve et al. (2013) used logistic regression models that controlled for baseline outcomes when available or controlled for other covariates when not available (e.g., pregnancy, miscarriages, and child welfare involvement). Kerr et al. (2009) was the only analysis to adjust the standard errors for non-independence of girls within foster-care or group-care sites.
• Harold et al. (2013) and Kerr et al. (2014) used linear growth multilevel models (time as level 1 and person as level 2) in which a time variable indirectly controlled for the baseline outcome.
• Van Ryzin and Leve (2012), Leve, van Ryzin et al. (2015), Poulton et al. (2014), and Rhoades et al. (2014), and Leve, Khurana et al. (2015) used structural equation models. The models controlled for baseline outcomes when available or controlled for other covariates when not available.
• Rhoades et al. (2016) used Cox proportional hazard models to compare the occurrence and timing of an adult arrest.

Missing Data Method: Kerr et al. (2009) and Leve et al. (2013) used complete-case analysis, while all the other analyses used FIML and/or multiple imputation to include participants with missing data.

Intent to Treat: The analyses used all participants or all participants with complete data, regardless of program participation.

Outcomes

Implementation Fidelity:

Not presented.

Baseline Equivalence:

Kerr et al. (2009) reported that "the conditions did not differ on any baseline measure." The wording implies no significant differences but does not specify the variables used in the tests.

Harold et al. (2013) reported that "there were no group differences on demographic characteristics, delinquency (self-report, days in locked settings, or official records), or childhood maltreatment at baseline. However, the depression symptom outcome measure differed significantly across conditions and was therefore included as a covariate.

Poulton et al. (2014) offered a few more details in stating that "Examination of baseline characteristics (criminal referrals; alcohol, marijuana, and other illicit drug use; and demographic information, including ethnicity, age, maltreatment history, single-parent family, income, parent criminality) indicated no significant differences between groups (all p > .10)."

Differential Attrition:

Kerr et al. (2009) had attrition of 5% or less, and nearly all the studies with higher attrition reported that the Little test indicated the data were missing completely at random. The supplement to Kerr et al. (2014) presents a detailed CONSORT diagram showing condition attrition rates that were similar enough to meet the WWC cautious and optimistic standards.

Rhoades et al. (2014) presented one additional test by comparing participants who did not complete any of the young adult follow-up assessments (7% of the total sample) to those who completed at least one follow-up assessment. The two groups did not differ on baseline drug use, days in treatment, intervention assignment, cohort, or ethnicity (all p > .10). The only significant difference was that those who did not participate were more likely to have been in trial or cohort 1 versus trial or cohort 2 (p < .05).

Posttest and Long-Term:

Two-Year Pregnancy (Kerr et al., 2009). From baseline through 24 months, the intervention group had significantly fewer self-reported pregnancies than the control group (OR = .41). A moderation test found that the intervention reduced the effect of baseline criminal referrals on pregnancy.

Two-Year Depression (Harold et al., 2013). Over the five waves of data from baseline to 24 months, the intervention condition showed a significantly greater linear decrease in depressive symptoms than the control condition. The difference was significantly greater in the second trial than in the first, and may have been affected by the higher baseline depressive symptoms in the intervention group. The same results held for the binary measure of clinical depression (OR = .57). Moderation tests found that the intervention benefited girls with higher levels of initial depression more than other girls.

Two-Year Psychotic Symptoms (Poulton et al., 2014). The results showed that, relative to the control group, the intervention group had a significantly greater decline in self-reported psychotic symptoms and significantly fewer diagnostic psychotic symptoms at the last follow-up. However, in a letter to the journal editor, Sinclair et al. (2017) argued that the positive intervention impact may have been due to regression to the mean. They pointed out that the initial assessment of psychotic symptoms often came after the program began and that the intervention group scored significantly higher on initial self-reported symptoms than the control group. Leve et al. (2017) responded that the initial assessment appropriately occurred at baseline for one of the cohorts and that intervention effects were still positive when examining only that cohort.

Two-Year Mediation (van Ryzin and Leve, 2012). The mediation models examined the outcomes of general delinquency and a latent construct of the number of days in a locked setting and the number of criminal referrals. The intervention significantly predicted delinquent peer associations at 12 months, which in turn significantly predicted both the latent construct and self-reported delinquency. Indirect effects on both outcomes were statistically significant but small (standardized coefficient = -.04 and -.06).

Seven-Year Outcomes (Leve et al., 2013). The intervention did not significantly affect any of the four seven-year, young-adult outcomes (marijuana use, illicit drug use, miscarrying a new pregnancy, and child welfare involvement). Also, pregnancy at two years did not significantly moderate the intervention effects.

Nine-Year Suicide and Depression (Kerr et al. 2014). The results showed that, relative to the control group, the intervention group had a marginally greater decline in suicide ideation and a significantly greater decline in depressive symptoms. The intervention did not affect suicide attempts.

Nine-Year Risky Sex (Leve, Van Ryzin et al., 2015). The correlations in Table 2 provided the clearest test of the intervention. It was not significantly correlated with either safe sexual practices or contraction of a sexually transmitted infection. The mediation model also showed no significant effects of the intervention.

Nine-Year Illicit Drug Use (Rhoades et al., 2014). Latent growth curve analysis indicated that a model allowing the drug use intercepts and slopes to vary across conditions improved significantly on a model constraining the two groups to be equal. The authors did not report directly on the significance of condition differences in the slopes but noted that intervention girls reported significant decreases in drug use over a two-year period in young adulthood (7-9 years after the study began), while those assigned to treatment as usual did not report significant decreases in drug use during this time. Further tests found that the relationship between the drug use of participants and their partners was significantly weaker in the intervention group than in the control group.

Ten-Year Child Maltreatment (Leve, Khurana et al., 2015). The results showed that the intervention group did significantly better than the control group on one of the three perpetration outcomes: self-reported maltreatment chronicity but not official maltreatment records or self-reported maltreatment contacts.

Adult Arrests (Rhoades et al., 2016). For the subsample of girls who had reached age 20 during the young adult follow-up period, the Cox proportional hazard models of the time to first offense after age 18 (Table 3, columns 5-8) found no significant effects of the intervention on any arrest (p = .80) or felony arrest (p = .67).

Study 8

This study evaluated the preschool version of the program, called Early Intervention Foster Care. The authors considered the program to be an extension of Multidimensional Treatment Foster Care to younger ages.

Summary

Fisher et al. (2005) evaluated the preschool version of the program using a randomized controlled trial to examine 90 three- to six-year-old foster children (intervention n = 47 and regular foster care control n = 43). Focusing on only the 54 assigned a permanent placement, the study examined the outcome of failed placements over 24 months.

Fisher et al. (2005) found that, compared to the control children, the intervention children had significantly:

• fewer failed placements (but only for the bivariate results, not the multivariate results)

Evaluation Methodology

Design:

Recruitment: Eligible participants included all 3- to 6-year-old foster children in Lane County, Oregon, who needed a new foster placement (n = 90). The sample selected those expected to remain in care for more than three months, whether new to the foster care system, reentering foster care, or moving between placements.

Assignment: The study randomly assigned participants to the intervention (n = 47) or the regular foster care control condition (n = 43). The control group received services-as-usual in which children were placed in state foster homes that followed standard policies and procedures.

However, consent came after random assignment and was provided by the child's caseworker and foster parent. Although recruitment rates were not significantly different for the two groups, the gap between consent rates of 89% for the intervention group and 80% for the control group could compromise the randomization. Also, the analysis included only the subsample of 54 children who received a permanent placement after randomization. The subsample included 62% of the intervention children (n = 29) and 58% of the control children (n = 25). The condition differences were small, but any such difference would compromise the randomization.

Assessments/Attrition: The baseline assessment came approximately three to five weeks after entrance to a new foster-care placement. Subsequent assessments came at three-month intervals over 24 months. In addition, salivary cortisol was collected from the children at four-week intervals. As noted, the study examined only 54 of the 90 randomized children. Of these 54 children, eight (15%) did not complete the full assessment. Although not stated in the article, the final analysis sample would appear to be 46.

Sample: Based on Table 1, the sample was 60-66% male, 79-92% white, and 4-18% Latino. The average age was between 4.22 and 4.50 years. The most common type of maltreatment was neglect.

Measures:

Measures came from case records provided by child welfare services. The measures included:

• Type of permanent placement, including reunification with the biological parent, relative adoption, and nonrelative adoption.
• Failure of a permanent placement, defined as the child having returned to foster care.
• Number of foster care placements, obtained from contacts at four-week intervals.

Analysis:

The analysis used Cox regression models of permanent placement failure rates, controlling for prior foster placements, foster placements after baseline, prior time in foster care, time in foster care after baseline, and gender. Although clustering within families may stem from siblings in the sample, the authors noted (p. 66) that removing the sibling families from the Cox regression models "did not affect the results significantly."

Intent-to-Treat: The restriction of the sample to children receiving a permanent placement may have been related to program effectiveness and therefore may violate the intent-to-treat criterion.

Outcomes

Implementation Fidelity:

Not examined.

Baseline Equivalence:

Table 1 lacks significance tests and effect sizes but shows large condition differences at baseline. The intervention group had more Hispanic children (18% versus 4%) and children who experienced physical abuse (24% versus 4%). Otherwise, the authors said that there were no significant group differences in prior placements, prior time in foster care, internalizing, externalizing, or the type of mistreatment.

Differential Attrition:

Not presented.

Posttest:

The 24-month follow-up period exceeded the typical six- to nine-month program period by at least one year, but the analyses did not distinguish posttest outcomes from long-term outcomes.

Long-Term:

A bivariate comparison found that the intervention group had a significantly lower number of failed permanent placements (p. 66), while bivariate percentage differences in Table 1 indicated that "no significant differences were found by type of permanent placement." The multivariate failure model in Table 2 included an insignificant condition effect along with a significant interaction with the number of prior placements. The significant interaction effect means that those with more placements benefitted more from the program than those with fewer placements.

Study 9

Summary

Westermark et al. (2011) used a randomized controlled trial to examine 35 Swedish youths at risk of out-of-home placement. The youths were assigned to intervention and treatment-as-usual control groups. Assessments through 24 months after baseline included measures of problem behavior.

Westermark et al. (2011) found that, relative to the control group, the intervention group showed significantly:

• lower externalizing and internalizing.

Evaluation Methodology

Design: This evaluation of Multidimensional Treatment Foster Care (MTFC) focused on 35 Swedish antisocial youths and presents outcomes at 24-months post-baseline. A total of 35 Swedish youth (20 treatment, 15 control) participated in the evaluation. Data were collected at baseline, 6-months, 12-months and 24-months post-baseline, but results are only presented from 24-months. Multiple sources of information were used, including self-reports and mother reports.

The treatment condition consisted of MTFC, and the control condition was 'treatment as usual' but included some intervention from the local child welfare authority. A total of 38 participants were referred by Swedish social agencies, but 3 declined to participate. Participants were referred for intervention due to serious behavioral problems. Criteria for inclusion include (a) diagnosis of a conduct disorder according to the DSM-IV and (b) were at risk of immediate out-of-home placement. Individuals were excluded from the study if they met one of the following criteria: (1) ongoing treatment by another provider (2) substance abuse without another antisocial behavior (3) sexual offending (4) acute psychosis (5) imminent risk of suicide (6) placement of the individual in a foster home that would cause a serious threat to the safety of the foster family.

Attrition: Overall treatment attrition rate was 11%. A total of 2 participants were lost from the treatment group and 2 from the control group. However, following an intent-to-treat model, these youth were included in the final analysis.

Sample: The sample included 35 Swedish youths (17 girls and 18 boys) with a mean age of 15.4 years. Almost half the sample had a history of previous interventions.

Measures: A number of measures were used including the Youth Self-Report (YSR), Child Behavior Checklist (CBCL) and the Symptom Checklist-90 (SCL-90).

• The YSR is a self-report measure completed by participants. The YSR contains 119 items that explores youths' behavior over the previous six months. Alpha coefficients for the YSR range from .71 - .95.
• The CBCL was completed by participants' mothers and included 113 items that describe the behavior of their child over the last six months. The CBCL includes both internalizing and externalizing subscales and has alpha coefficients between .78 - .97.
• The SCL-90 is a self-report rating scale for measuring psychiatric symptoms. Subscales of this measure were also used to measure mother's depression and anxiety. Alpha coefficients for the SCL-90 were between .81 - .91.

Analysis: The study followed a 2 X 2 condition (treatment versus control X baseline versus post-test). ANOVA was used to examine the effects of MTFC on youth's behavior problems. A general linear model (GLM) was used to analyze the variation within groups at two different periods (baseline and post-baseline) and the statistical interaction effect between groups.

Researchers measured the clinical significance of MTFC in two ways:

1. Reduction in standard deviation: This is a change in an individual's symptoms compared to the normative level of the symptom. If the change was (at least) as large as one standard deviation from pre- to post-intervention in a positive direction, researchers considered this clinically significant.
2. Minimum 30% reduction: A symptom reduction was also estimated if at least a 30% improvement (in a positive direction) was noted between baseline and post-test.

Outcomes

Implementation fidelity: There were no measures of implementation fidelity.

Baseline equivalence and Differential Attrition: There were no significant differences between treatment and control groups at baseline. Participants who dropped out of either the treatment or control group (n = 4) were entered in the intent-to-treat analysis. The missing value was imputed by last observation carried forward.

Post-test (24-Months)

Externalizing: There was a significant difference between treatment and control conditions (favoring treatment) in the YSR externalizing subscale. This significant difference was noted in an ANOVA and in both clinical markers (reduction in standard deviation and minimum 30% reduction). There was also a significant difference between treatment and control conditions in the CBCL externalizing subscale, as measured by a minimum 30% reduction.

Internalizing: There was a significant difference between treatment and control conditions in the CBCL internalizing subscale, as measured by a reduction in standard deviation as well as a minimum 30% reduction. There was no significant reduction in the internalizing subscale of the YSR.

Psychiatric symptoms: There were no significant differences between treatment and control conditions in clinical reduction of psychiatric symptoms, as measured by the SCL-90. However, the ANOVA showed differences favoring MTFC for Depression and for the Global Severity Index.

Study 10

This study examined the preschool version of the program. Laurent et al. (2014) focused primarily on cortisol levels but gave tangential attention to symptoms of emotional and behavior problems. Lynch et al. (2014) examined permanent placements and focused on the economic benefits of the program. Fisher et al. (2009) examined only a subset of the sample that had previously experienced foster-care placement instability.

Summary

Fisher et al. (2009), Lynch et al. (2014), and Laurent et al. (2014) used a randomized controlled trial to examine the preschool program. Although 137 children ages 3-6 were randomly assigned, non-consent and other exclusions after randomization left 57 in the intervention group and 60 in a usual-care control group. The study examined permanent placements two years after baseline and emotional and behavioral problems six years after baseline.

Fisher et al. (2009), Lynch et al. (2014), and Laurent et al. (2014) found no significant intervention effects on the child behavioral outcomes for the full sample.

Evaluation Methodology

Design:

Recruitment: The study recruited 137 preschool children (ages 3-5) who were entering new foster-care placements. All came from a public child welfare agency in a moderate-sized Pacific Northwest city. Laurent et al. (2014) also included a group of 60 same-aged, low-income community children who had not been involved in the child welfare system to serve as an additional comparison group.

Assignment: According to Lynch et al. (2014), 137 children were randomized but consent from both caseworkers and parents came after randomization. Of the 137 children, 20 were excluded after randomization because of non-consent, scheduling problems, or placement having already occurred. After exclusions, the intervention condition had 57 children (89% of the 64 randomized) and a usual-care control condition had 60 children (82% of the 73 randomized). The community sample in Laurent et al. (2014) was not part of the randomization.

Assessments/Attrition: Fisher et al. (2009) and Lynch et al. (2014) followed the children for 24 months and appeared to have complete data on the primary outcome. Laurent et al. (2014) assessed the children 29 times over six or more years, initially at one-month intervals and later at six-month intervals. A final assessment included diagnostic data, but only 96 of the sample of 177 (54%) provided data at this assessment.

Sample:

Although ages 3-6 at baseline, the children were followed long enough to reach ages 9-13. Reflecting the make-up of the community, the sample was 89% European American, 1% African American, 5% Latino, and 5% Native American.

Measures:

Fisher et al. (2009) and Lynch et al. (2014) examined one primary outcome, successful permanent placement, that was obtained from agency records. Permanent placement included re-uniting with a biological parent, relative adoption, and nonrelative adoption.

Laurent et al. (2014) used data collected by a research team that was blind to the child's assignment. At each assessment, saliva samples were collected for cortisol levels - a measure related to stress and adversity - in the morning and evening over two consecutive days. At the final assessment only, the Diagnostic Interview Schedule for Children was used to measure total symptom counts of (1) anxiety disorders, (2) eating disorders, (3) mood disorders, (4) disruptive behavior disorders, and (5) alcohol and substance use disorders.

Analysis:

Fisher et al. (2009) and Lynch et al. (2014) compared condition means without controls. For the cortisol outcomes, Laurent et al. (2014) used hierarchical linear models with time (i.e., the 29 assessments) nested within persons. The models included a term for lagged cortisol level at the previous assessment. For the behavioral outcomes, Laurent et al. (2014) used one-way ANOVA without baseline controls.

Fisher noted that the sample included 10 sibling pairs. None of the analyses adjusted for sibling clustering, but Fisher et al. (2009) stated that selecting only the younger of the pairs and excluding the other produced similar findings as when all siblings were included.

Intent-to-Treat: Lynch et al. (2014) used all participants after the initial exclusions. Laurent et al. (2014) dropped those with missing data at the final assessment but otherwise used FIML to include participants with some missing cortisol assessments. Fisher et al. (2009) examined only a subsample of participants.

Outcomes

Implementation Fidelity:

Not presented.

Baseline Equivalence:

Neither Laurent et al. (2014) nor Lynch et al. (2014) presented figures on baseline condition means, but Laurent et al. (2014) stated that "children in the two foster care groups did not differ on placement type (i.e., first-time foster placement vs. change in foster home vs. re-entry into foster care following failed permanent placement) or number of days in foster care" and that "there were no differences between the three groups on child age, gender, or ethnicity."

Differential Attrition:

In Laurent et al. (2014), the only analysis with attrition, a comparison across conditions showed significantly fewer control children than intervention children who completed the final assessment. There were no demographic differences between completers and dropouts, but completers had significantly higher mean cortisol values.

Posttest:

Not examined.

Long-Term:

Lynch et al. (2014) found that the intervention group did not differ significantly from the control group on permanent placements. For the subsample of children who exhibited prior placement instability, the intervention group had significantly more permanent placements. Overall, total costs were significantly lower for the intervention group than the usual care group in both the full sample and the placement instability sample. For the subsample of children with prior placement instability, Fisher et al. (2009) also found that the intervention children had a significantly higher rate of permanent placements.

For the five behavioral outcomes in Laurent et al. (2014), there were no significant differences between the intervention and control children. The only difference involved lower disruptive behavior disorders for the community comparison sample.

For the cortisol outcome in Laurent et al. (2014), children in the control group had significantly lower and less stable cortisol levels than the intervention group. The authors interpreted the findings to indicate that "the intervention moved foster care children toward the patterns of HPA function found in community children."

Study 11

The English version of Multidimensional Treatment Foster Care that was evaluated in this study is called Intensive Fostering (IF). The English program closely followed the original program and included distance-supervision by program developers in Oregon.

Summary

Biehal et al. (2010, 2011) used a quasi-experimental design with non-random assignment to compare 23 intervention youth who were placed in foster care with 24 comparison youth who were sentenced to custody or community-based supervision. Reconviction rates and entrance into custody were examined at the end of the foster-care intervention and one year afterward.

Biehal et al. (2010, 2011) found that, compared to the control group, the intervention group had significantly:

• fewer reconvictions and entrances to custody at posttest but not at long-term follow-up.

Evaluation Methodology

Design:

Recruitment: The sample consisted of 47 serious and persistent youth offenders in England who faced an imminent custodial sentence and were at high risk of reoffending. The eligibility requirements included having severity scores of three or more (on a four-point rating scale) on the family and personal relationships and lifestyle subscales of a screening assessment tool used by the criminal justice system.

Assignment: In a quasi-experimental design, 23 participants were non-randomly assigned to the intervention group that received a nine-month foster-care placement and 24 to the comparison group that was either sentenced to custody or, in four cases, sentenced to a community-based program at the request of the funding agency. Because of the limited number of placements available, only 23 young people could be assigned to the intervention group. The comparison youths met the same eligibility criteria as the intervention youths and therefore were roughly matched on the ratings from the assessment tool.

Assessments/Attrition: Baseline assessments came at the date of the sentence to foster placement or custody/supervision. For the intervention group, outcomes were measured at two points: 1) one year after the date of entry to the intervention placement (Stage 1), while the youths were typically under supervision, and 2) one year after the date of exit from intervention placement, when the youths were no longer under supervision. For the comparison group, outcomes were measured at a single point. A further complication was that the single assessment for the comparison group came at two different time points: 1) one year after the date of their release for those in custody, as they only had the opportunity to reoffend once they returned to the community, and 2) one year after the sentence began for those under supervision, as they remained at liberty throughout. Thus, the intervention group at one year and two years after baseline was compared to the comparison group at two years after baseline for most and one year after baseline for some. There was no attrition for the primary outcome measures, although one intervention participant died and was excluded from the study.

Interviews with young people and parents came at baseline and one year after entry to placement or release from custody. Of the 47 participants, the youth interviews included 79% of the sample and the parent interviews included only 19% of the sample.

Sample:

The sample, 83% male and age 15 on average, consisted of highly vulnerable youth. For example, the average age of first conviction was 13 years, more than half had experienced maltreatment, and one-third had difficulties with basic literacy or numeracy.

Measures:

The primary outcome measures were reconviction and the entrance into custody. These measures came from official records. The reconviction measures included days to first reoffence, total number of offenses, number of offenses per day at liberty, and a gravity score of the seriousness of the offense.

Secondary outcomes were also obtained from semi-structured interviews and included living situation, participation in education, training or employment, and peer relationships.

Analysis:

The analyses typically used bivariate chi-square and Mann-Whitney tests to compare conditions but sometimes used multivariate logistic regression. The logistic regression controlled for pre-baseline offending.

Intent-to-Treat: The primary analysis used all participants, and the secondary analysis used all available data.

Outcomes

Implementation Fidelity:

The authors referred to problems with staff recruitment and turnover that may have limited program delivery at times but likely no more than in other implementations. Otherwise, foster-care placement lasted from one week to nearly 17 months. Two-thirds of the young people remained in their placement for nine months or more. Two-thirds formally completed the program, a figure similar to that found in other studies.

Baseline Equivalence:

In Table 1, one of 10 tests for condition differences reached statistical significance. The comparison youth were more likely to have committed an index offense of violence against the person - 21% in the intervention group versus 50% in the comparison group. Other substantial differences, such as 52% maltreatment in the intervention group versus 63% maltreatment in the comparison group, did not reach statistical significance but nonetheless suggest non-equivalence.

Differential Attrition:

No attrition for the primary outcomes, and no tests for attrition for the secondary outcomes.

Posttest:

The Stage 1 results referred to outcomes at the end of foster placement for the intervention youth and one year after release from custody for most of the comparison group. The multivariate results found that the intervention group had significantly fewer reconvictions and entrances into custody than the comparison group.

The results for the secondary outcomes were largely descriptive, but one test indicated that the intervention group reported living with parents significantly more often than the comparison group.

Long-Term:

The Stage 2 results referred to outcomes at one year after the end of foster placement for the intervention group and, as at Stage 1, one year after release from custody for most of the comparison group. None of the outcomes differed significantly after intervention youth left their foster placement.

Study 12

Summary

Biehal et al. (2012), Dixon et al. (2014), Green et al. (2014), and Sinclair et al. (2016) used both a randomized controlled trial (n = 34) and a propensity-score matched QED (n = 185). The posttest came one year after baseline and included measures of overall functioning, educational outcomes, and offending.

Biehal et al. (2012), Dixon et al. (2014), Green et al. (2014), and Sinclair et al. (2006) found no significant intervention effects for the full sample.

Evaluation Methodology

Design:

Recruitment: Children eligible for the study were 10-17 years old, had complex or severe emotional or behavioral difficulties, and were in a care placement that was unstable, at risk of breakdown, not meeting the child's needs, or leading to custody or secure care. The final sample of 219 youths came from 23 local authorities in England over the period from June 2005 to December 2008.

Assignment: The study used two forms of assignment. First, 34 young people were randomly assigned, with 20 in the intervention group of nine months placement plus a short period of aftercare and 14 in the usual-care control group. Second, 185 young people were non-randomly assigned, with 92 in the intervention group and 93 in the usual-care control group. The assignment was made by a panel of social workers who sought to balance youth needs with placement availability. Propensity score matching was used to equalize differences in the non-randomized sample. Dixon et al. (2014) described the difficulties in obtaining a sample that local agencies would allow to be randomized, and concerns about the potential problems led to the pre-registration of the non-randomized component.

Assessments/Attrition: Assessments came at three time points: baseline, three months into the placements, and posttest (one-year post-baseline). According to the CONSORT diagram in Green et al. (2014), the analysis included 85% of the randomized sample and 81% of the non-randomized sample.

Sample:

The sample of young people ages 11-17 was 54% male. At least two-thirds of the sample had clinically significant mental health difficulties, with externalizing problems being particularly common. About 36% had been charged or convicted in the six months prior to baseline.

Measures:

The two primary outcomes measured overall adaptive functioning and were obtained from the Children's Global Assessment Scale and the Health of the Nation Outcome Scales for Children and Adolescents. Researchers who were unaware of condition obtained information for the measures from the youths, caregivers, school and police records, and standard mental health measures of behavior and social functioning (Child Behavior Checklist, Strengths and Difficulties Questionnaire). Two researchers then coded the information to assign scores. Interrater reliability was high.

Secondary outcomes included ratings for education-related domains of scholastic performance and attendance and criminal offending incidents reported by caregivers and social workers.

Analysis:

The analysis of the posttest outcomes for both samples used linear regression with controls for the baseline outcomes. For the non-randomized sample, baseline covariates of gender, age, prior placement, and the two overall functioning scores generated propensity scores. The propensity score matching first selected a trimmed data set that eliminated those with extreme propensity scores above .95 or below .05. The regression then adjusted for inverse probability weights to equalize the propensity scores across groups. The matched data included 153 participants.

Intent-to-Treat: Despite substantial crossover, the analysis of the randomized sample examined participants in their originally assigned group, subject to data availability. However, the reasons for attrition were not clearly specified for all dropouts in the CONSORT diagram. Analysis of the non-randomized sample included all matched participants, regardless of program participation.

Outcomes

Implementation Fidelity:

At the program end-point, 45% of intervention participants were still in their placement. The authors noted that the randomized sample had a relatively high proportion of crossovers between the two arms of the trial. Eight of 20 young people assigned to the intervention were placed in usual care placements.

Baseline Equivalence:

The randomized sample was said to have very good baseline matching between the groups. The non-randomized sample had several statistically significant differences between the groups. The use of propensity scores reduced condition differences but not completely: There was still evidence that baseline age, one functioning score, and residential care prior to the study were imbalanced. The effects of remaining differences on outcomes were taken into account in the multivariate analysis.

Differential Attrition:

Not examined, though attrition of only 9%.

Posttest:

Neither the randomized sample nor the propensity-score matched sample showed significant condition differences at posttest. The non-significant results held for the two primary measures of functioning and for the three secondary measures of scholastic performance, school attendance, and offending. Moderation tests found some evidence of a significant intervention benefit in the non-randomized sample for youth with high baseline antisocial behavior.

Sinclair et al. (2016) examined moderation by baseline antisocial behavior in more depth. Using a propensity-score matched sample of 171 (88 in the intervention group and 83 in the control group), the analysis divided the youths into an antisocial group and a less antisocial group. The outcome, Children's Global Assessment of overall functioning, was coded so that high scores indicated better functioning. The conditions did not differ significantly for the full matched sample. However, for the high antisocial group, overall functioning was significantly higher in the intervention group than the control group. For the less antisocial group, overall functioning was either significantly higher in the control group than the intervention group or statistically equivalent across the groups. For the high antisocial group, mediation tests showed a significant indirect effect of the program on overall functioning via reduced antisocial behavior.

Long-Term:

Not examined.

Study 13

Summary

Hansson and Olsson (2012) and Bergström and Höjman (2016) used a randomized controlled trial to examine youths with a diagnosed conduct disorder and at risk for out-of-home placement. The youths were assigned to intervention (n = 19) and control (n = 27) conditions and followed for 36 months after baseline to assess problem behavior, crime, and substance abuse.

Hansson and Olsson (2012) found no significant intervention effects for the full sample. Bergström and Höjman (2016) found that, compared to the control group, the intervention group committed significantly:

• fewer crimes of personal violence (in year 1 and years 1-3).

Evaluation Methodology

Hansson & Olsson (2012) examined the 24-month post-baseline outcomes, whereas Bergström & Höjman (2016) examined the three-year post-baseline outcomes.

Design:

Recruitment: The participants in this Swedish evaluation first went through two months of assessment at a single residential home. Eligible youths had to be 12-17 years old, meet the diagnostic criteria for conduct disorder, and be at risk for immediate out-of-home placement. A total of 46 youths and their families joined the study.

Assignment: The 46 youths were randomly allocated to either the intervention group (n = 19) or the treatment-as-usual control group (n = 27). The uneven numbers came from the initial assignment of one to the intervention for two to the control and from later assignment using a one-to-one ratio. The control group received services from social agencies that included residential care, foster care, and home-based interventions.

Assessments/Attrition: Assessments came at one year after baseline (posttest), two years after baseline (one-year follow-up), and three years after baseline (two-year follow-up). Of the 46 participants, four (9%) were lost to follow-up at year two, but Bergström & Höjman (2016) had no attrition through year three because they used data from administrative records rather than child and parent reports.

Sample:

Girls made up about 40% of the Swedish sample, and 35% of the families had a least one parent who was an immigrant to Sweden.

Measures:

The measures in Hansson & Olsson (2012) came from youth self-reports and from mother reports, which may be biased given that the program involves a parenting component. Most of the reported alpha values came from other studies.

• Youth self-report of problem behavior using the Youth Self Report (alpha = .95).
• Mother report of problem behavior using the Child Behavior Checklist (alpha = .97).
• Youth sense of coherence using the Sense of Coherence Questionnaire (alpha = .84).
• Mother's psychiatric symptom load (alpha = .97).
• Mother's sense of coherence.

Measures of clinical change for youth- and mother-reported problem behavior supplemented the continuous measures.

The measures in Bergström & Höjman (2016) came from casework and institutional care records. The team coded the measures using a specially developed manual but may have been aware of condition. Four measures of the treatment process included: 1) the number of out-of-home placements, 2) whether the juvenile was placed in a locked ward, 3) whether the juvenile was without a place to live, and 4) whether the juvenile experienced a breakdown exit from the placement.

Criminality was measured by confirmed reports from police or by convictions for any crime and for personal violence crimes.

Substance abuse was measured with drug tests, drug treatment, or drug convictions.

Analysis: The main analyses in Hansson & Olson (2012) used general linear models with repeated measures and a time-by-group interaction coefficient to test for differences across conditions in changes in the outcomes. A subsample analysis of clinical change used chi-square tests that also controlled for baseline outcomes. Neither analysis included covariates. The study also compared family breakdown in the intervention group to other studies but not to the control group. Bergström & Höjman (2016) also used general linear models with repeated measures for one outcome but mostly used chi-square and t-tests without baseline outcome controls.

Intent-to-Treat: The main analysis in Hansson & Olson (2012) included all participants by replacing missing follow-up data with the data from a previous assessment. However, the analysis of clinical change excluded those with normal values on the baseline outcomes. Bergström & Höjman (2016) used all participants.

Outcomes

Implementation Fidelity:

Not examined.

Baseline Equivalence:

Table 1 in Hansson & Olson (2012) shows one significant difference in five tests, with the intervention group having substantially more youths with an immigrant background (53%) than the control group (22%). Table 3 shows no significant baseline differences for the five outcomes. Bergström & Höjman (2016) added tests for multiple baseline measures and found no significant differences in 12 tests (Table 1) and 24 tests (Table 2). However, the small sample size may have made it difficult to detect statistical significance in the baseline tests.

Differential Attrition:

Not examined, although attrition in Hansson & Olson (2012) was only 9%, and missing data were imputed using the last observation carried forward. Bergström & Höjman (2016) had no attrition due to the use of administrative records.

Posttest and Long-term:

Hansson & Olson (2012): The results in Table 3 showed that the time-by-group interaction terms were non-significant for all outcomes, indicating no benefit of the intervention. The subsample results in Table 4 for those with clinical problems at baseline showed that the reduction in clinically relevant change was significantly greater for the intervention group than the control group for two measures of child behavior but only for the period from baseline to treatment end and not for the period from baseline to the long-term follow-up.

Bergström & Höjman (2016): The results reported in Tables 3-7 and the text showed four significant condition differences in 26 tests. For risk and protective factors, the intervention group relative to the control group had fewer placements in a locked setting in year 1 (but not in year 2, year 3, or years 1-3). For behavioral outcomes, the intervention group relative to the control group committed fewer crimes in year 3 (but not in year 1, year 2, or years 1-3), and fewer personal violence crimes in year 1 and years 1-3 (but not in year 2 or year 3).

Study 14

Summary

Jonkman et al. (2017) used a randomized controlled trial with 42 participants, a quasi-experimental design with 89 non-randomly assigned participants, and a non-experimental comparison with 30 regular foster children. The study followed the participants for nine months and used measures of problem behavior as the primary outcomes.

Jonkman et al. (2017) found no effects for the randomized sample and very few effects for the non-randomized sample.

Evaluation Methodology

This study evaluated the preschool version of the program. It was pre-registered at the Medical Ethical Committee (Academic Medical Center Amsterdam, The Netherlands; April 2009; METC 09/046).

Design:

Recruitment: The study examined children ages 3-7 in the Netherlands who were in permanent foster-care placement and had severe behavioral problems. Participants were recruited from child protective services and foster parents, and the recruitment took place from June 2009 to January 2013. The study ended up with three groups of participants that differed on type of assignment: 42 were randomly assigned, 89 were non-randomly assigned, and 30 were selected from regular foster care.

Assignment: Children were initially assigned randomly to intervention or control conditions, but the need to fill empty intervention slots led to subsequent non-random assignment. Figure 1 lists 42 children as randomly assigned, but consent came after randomization. After removing those declining to participate, the randomized sample had 34 participants (81%), 23 in the intervention group and 11 in the control group. The QED sample included the randomized participants plus the non-randomized participants. Of 89 assigned participants, 78 (88%) consented, with 55 in the intervention group and 23 in the control group. The two control groups received Therapeutic Foster Care, which included "two-weekly home visits of social workers" who could also arrange for additional services as needed. A third comparison group of 30 children in regular foster care was selected separately and received minimal services.

Assessments/Attrition: Assessments came at baseline, three months (interim), six months (interim), and nine months (posttest). Based on figures in Table 2, posttest data was provided by 88% of the foster parents and 69% of the teachers.

Sample:

About 64% of the sample was male, with 28% having experienced physical abuse and 74% having experienced neglect. The study provided no information on the racial, ethnic, or socioeconomic composition of the sample.

Measures:

The primary outcomes included six measures of problem behavior: internalizing, externalizing, and total problems from reports of foster parents and internalizing, externalizing, and total problems from reports of teachers. The measures used standardized instruments such as the Child Behavioral Checklist and reported acceptable alpha reliabilities.

The secondary outcomes included measures of child disturbed attachment reported by foster parents, child trauma symptoms reported by foster parents, self-reported caregiver stress, child cortisol, and caregiver cortisol. Reported alpha reliabilities were acceptable.

Analysis:

The analyses used ANOVA mixed models for repeated measures. Time in current foster family was treated as a covariate to adjust for baseline differences.

Intent-to-Treat: The QED analyses were "per protocol" with data excluded because of non-consent and attrition. Data from the randomized sample was analyzed according to intent-to-treat principles, with the last observed values moved forward to replace missing values.

Outcomes

Implementation Fidelity:

Not presented.

Baseline Equivalence:

Tables 1 and 2 present tests for the QED sample. Of seven background measures, one indicated a significant condition difference. Time spent in the current family was 2.38 months for the intervention group versus 16.48 months for the control group. Of six tests for baseline measures of problem behavior, none reached statistical significance.

Differential Attrition:

Not examined.

Posttest:

For the QED sample at posttest, the per protocol analysis of the six primary outcomes in Table 2 showed no significant differences between conditions. The per protocol analysis of secondary outcomes in Table 4 showed one significant effect in five tests. Foster parents reported fewer trauma symptoms for the intervention children than the control children.

For the RCT sample that used all participants in the intent-to-treat analysis, changes in the outcomes did not differ significantly across conditions.

Comparisons to the regular foster care sample (called Study II by the authors) found that the intervention group improved significantly more on foster-parent reports of internalizing.

Long-Term:

Not examined.