Sign Up For Newsletter

Blueprints For Healthy Youth Development logo

Multisystemic Therapy® (MST®)

A juvenile crime prevention program designed to improve the real-world functioning of youth by changing their natural settings - home, school, and neighborhood - in ways that promote prosocial behavior while decreasing antisocial behavior.

Program Outcomes

  • Close Relationships with Parents
  • Conduct Problems
  • Delinquency and Criminal Behavior
  • Externalizing
  • Internalizing
  • Mental Health - Other
  • Positive Social/Prosocial Behavior
  • Prosocial with Peers
  • Violence

Program Type

  • Family Therapy
  • Juvenile Justice - Other

Program Setting

  • Transitional Between Contexts
  • Social Services
  • Home
  • Correctional Facility
  • School
  • Mental Health/Treatment Center

Continuum of Intervention

  • Indicated Prevention

Age

  • Late Adolescence (15-18) - High School
  • Early Adolescence (12-14) - Middle School

Gender

  • Both

Race/Ethnicity

  • All

Endorsements

Blueprints: Model Plus
Crime Solutions: Effective
OJJDP Model Programs: Effective
SAMHSA : 2.9-3.2

Program Information Contact

Tom Pietkiewicz
Director of Business Development
MST Services, Inc.
3490 Piedmont Road NE, Suite 1250
Atlanta, GA 30305
Office: (843) 352-4306
Cell: (404) 395-6038
Email: tom.pietkiewicz@mstservices.com

Program Developer/Owner

Scott W. Henggeler, Ph.D.
Medical University of South Carolina


Brief Description of the Program

Multisystemic Therapy® (MST®) is an intensive family- and community-based treatment that addresses the multiple causes of serious antisocial behavior in juvenile offenders. The MST program seeks to improve the real-world functioning of youth by changing their natural settings - home, school, and neighborhood - in ways that promote prosocial behavior while decreasing antisocial behavior. Therapists work with youth and their families to address the known causes of delinquency on an individualized, yet comprehensive basis. By using the strengths in each system (family, peers, school, and neighborhood) to facilitate change, MST addresses the multiple factors known to be related to delinquency across the key systems within which youth are embedded. The extent of treatment varies by family according to clinical need. Therapists generally spend more time with families in the initial weeks (daily if needed) and gradually taper their time (to as infrequently as once a week) over the 3- to 5-month course of treatment.

Outcomes

Primary Evidence Base for Certification

Study 3

Borduin et al. (1995) found that at the posttest, relative to the control group, the MST intervention group showed significantly

  • Decreased youth behavior problems
  • Improved family relations
  • Lower recidivism rates (re-arrested at least once) through 4-year follow-up.

Study 7

At the posttest, Ogden et al. (2004) found that the MST intervention group, relative to the usual-services control group, showed significantly:

  • Decreased youth internalizing symptoms
  • Increased social competence
  • Decreased out-of-home placements.

Study 9

Weiss et al. (2013) found that at the posttest, relative to the control group, the MST intervention group showed significantly lower:

  • Externalizing problems
  • Absenteeism in school.

Study 11

Asscher et al. (2013, 2014) and Deković et al. (2012) found that, relative to the control group, the MST intervention group showed significant reductions at both the posttest and six-month follow up in:

  • Externalizing
  • Oppositional defiant disorder
  • Conduct disorder
  • Property offenses
  • Relationship quality.

Study 13

Butler et al. (2011) found that, relative to the control group, the MST intervention group showed significant decreases in

  • Aggression and delinquency at posttest
  • Psychopathic traits at posttest
  • Nonviolent offenses at the 18-month follow-up assessment.

Brief Evaluation Methodology

Primary Evidence Base for Certification

Of the 23 studies Blueprints has reviewed, 5 (studies 3, 7, 9, 11 and 13) meet Blueprints evidentiary standards (specificity, evaluation quality, impact, dissemination readiness). In addition, Study 3 was done by the developer; Studies 7, 9, 11 and 13 were conducted by independent researchers.

Study 3

Borduin et al. (1995), Sawyer et al. (2011), Schaeffer et al. (2005), and Wagner et al. (2014) used a randomized controlled trial that assigned 176 juvenile offenders ages 12-17 to an MST intervention group or an individual therapy control group. Assessments were conducted at posttest through a 21.9-year follow-up and measured problem behavior, arrests, and family relations.

Study 7

Ogden and Halliday-Boykins (2004), Ogden and Hagen (2006, 2009), and Ogden et al. (2007) used a randomized controlled trial that assigned 100 Norwegian youths with serious antisocial behavior to an MST intervention group or a control group receiving child welfare services. Assessments at posttest and two-year follow-up measured problem behavior, delinquency, and out-of-home placement.

Study 9

Weiss et al. (2013, 2015) used a randomized controlled trial that assigned 164 youths enrolled in behavior intervention classrooms to an MST intervention group or a no additional treatment control group. Assessments from posttest to a one-year follow-up measured conduct problems, externalizing, and criminal offending.

Study 11

Asscher et al. (2013, 2014, 2018), Deković et al. (2012), and Manders et al. (2013 used a randomized controlled trial that assigned 256 Dutch adolescents with antisocial behavior to an MST intervention group or a treatment-as-usual control group. Assessments from posttest to a three-year follow-up measured a variety of outcomes relating to antisocial behavior.

Study 13

Butler et al. (2011) used a randomized controlled trial that assigned 108 juvenile offenders in London, England, to an MST intervention group or a treatment-as-usual control group. Assessments at baseline and from posttest to an 18-month follow-up measured reoffending and problem behavior.

Study 3

Borduin, C. M., Mann, B. J., Cone, L. T., Henggeler, S. W., Fucci, B. R., Blaske, D. M. & Williams, R. A. (1995). Multisystemic treatment of serious juvenile offenders: Long-term prevention of criminality and violence. Journal of Consulting and Clinical Psychology, 63, 569-578.


Study 7

Ogden, T., & Halliday-Boykins, C. A. (2004). Multisystemic treatment of antisocial adolescents in Norway: Replication of clinical outcomes outside of the US. Child and Adolescent Mental Health, 9(2), 77-83.


Study 9

Weiss, B., Han, S., Harris, V., Catron, T., Ngo, V. K., Caron, A., . . . Guth, C. (2013). An independent randomized clinical trial of Multisystemic Therapy with non-court-referred adolescents with serious conduct problems. Journal of Consulting and Clinical Psychology, 81(6), 1027-1039.


Study 11

Asscher, J. J., Deković, M., Manders, W. A., van der Laan, P. H., Prins, P. J. M., & the Dutch MST Cost-Effectiveness Study Group 4. (2013). A randomized controlled trial of the effectiveness of Multisystemic Therapy in the Netherlands: Post-treatment changes and moderator effects. Journal of Experimental Criminology, 9, 169-187.


Asscher, J. J., Deković, M., Manders, W. A., van der Laan, P. H., Prins, P. J. M., van Arum, S., & the Dutch MST Cost-Effectiveness Study Group 4. (2014). Sustainabilityof the effects of multisystem therapy for juvenile delinquents in The Netherlands: Effects on delinquency and recidivism. Journal Experimental Criminology, 10, 227-243.


Deković, M., Asscher, J. J., Manders, W. A., Prins, P. J. M., & van der Laan, P. (2012). Within-intervention change: Mediators of intervention effects during Multisystemic Therapy. Journal of Consulting and Clinical Psychology, 80(4), 574-587.


Study 13

Butler, S., Baruch, G., Hickey, N., & Fonagy, P. (2011). A randomized controlled trial of Multisystemic Therapy and a statutory therapeutic intervention for young offenders. Journal of the American Academy of Child and Adolescent Psychiatry, 50(12), 1220-1235.


Risk Factors

Individual: Early initiation of antisocial behavior, Early initiation of drug use, Favorable attitudes towards antisocial behavior, Rebelliousness, Substance use

Peer: Interaction with antisocial peers, Peer substance use

Family: Family conflict/violence*, Neglectful parenting, Parental attitudes favorable to antisocial behavior, Parental attitudes favorable to drug use, Parent history of mental health difficulties*, Parent stress, Poor family management*, Violent discipline

School: Low school commitment and attachment*, Poor academic performance

Neighborhood/Community: Community disorganization, Laws and norms favorable to drug use/crime, Low neighborhood attachment

Protective Factors

Individual: Clear standards for behavior, Problem solving skills, Prosocial involvement*, Rewards for prosocial involvement, Skills for social interaction*

Peer: Interaction with prosocial peers*

Family: Attachment to parents, Nonviolent Discipline*, Opportunities for prosocial involvement with parents, Parental involvement in education, Parent social support, Rewards for prosocial involvement with parents

School: Opportunities for prosocial involvement in education, Rewards for prosocial involvement in school

Neighborhood/Community: Opportunities for prosocial involvement, Rewards for prosocial involvement


* Risk/Protective Factor was significantly impacted by the program

See also: Multisystemic Therapy® (MST®) Logic Model (PDF)

Gender Specific Findings
  • Male
Race/Ethnicity/Gender Details

Subgroup differences by race, ethnicity, or gender (coded in binary terms as male/female):

  • There were no subgroup findings of differences in program impact by race or ethnicity in any of the Blueprints-certified studies for MST.
  • In terms of gender (coded in binary terms as male/female), Asscher et al. (2013; Study 11) found that for adolescent cognitions, the treatment had larger (and more positive) effects for boys than for girls.

Sample demographics including race, ethnicity, and gender for Blueprints-certified studies:

  • The sample in Study 3 (Borduin et al., 1995) was 30% Black or African American and 70% White (ethnicity was not reported).
  • For Study 7 (Ogden & Halliday-Boykins, 2004), the sample demographics were not reported.
  • In Study 9 (Weiss et al., 2013), the sample was 60% Black or African American and 40% White (ethnicity was not reported).
  • For Study 11 (Deković et al., 2012), 55% of youths had a Dutch background. Of those belonging to ethnic minoritized populations, 34% had a Moroccan and 32% had a Surinamese background (race was not reported).
  • The sample for Study 13 (Butler et al., 2011) was 34% White British/European, 32% Black African, 5% Asian, and 29% of the sample's race was not specified (ethnicity was not reported).
  • The samples for all certified studies were reported in binary categories of male and female, and most of the samples were comprised of males.
  • Specifically, 67% of the sample in Study 3 (Borduin et al., 1995) was male, 63% in Study 7 (Ogden & Halliday-Boykins, 2004) were male, 83% in Study 9 (Weiss et al., 2013) were male, 73% in Study 11 (Deković et al., 2012) were male, and 82% of youth in Study 13 (Butler et al., 2011) were male.  

MST Group LLC (doing business as MST Services) offers comprehensive assistance with the full development of MST programs by providing program start-up assistance, initial and on-going clinical training and program quality assurance support services.

MST Services program development and support consists of a comprehensive package of services designed to do "what it takes" to ensure that the MST program will be successful and sustainable. These services cover four areas: 1) program start-up including initial staff training, 2) ongoing clinical support activities, 3) ongoing organization support activities, and 4) quality assurance support.

The program start-up services include technical assistance and materials designed to produce a program description, projected budget, and implementation timeline. Key critical elements include clear articulation of the target population definition and prioritization process, referral and discharge criteria and processes, recommendations regarding clinical record-keeping practices, and initial program evaluation planning. The MST Program Developer will visit the community to provide an overview presentation and meet with community stakeholders to assure the buy-in needed for program success after start-up. Next, staff recruitment assistance includes sample job descriptions, help with advertising, interviewing and selecting staff most qualified to implement MST successfully. Finally, all selected initial staff will complete the 5-day MST Orientation Training.

The ongoing MST clinical support is provided to replicate the characteristics of training, clinical supervision, consultation, and monitoring provided in the successful clinical research trials of MST. This program implementation protocol has been refined through extensive experience with communities and providers in numerous sites in the U.S. and internationally. After start-up, training continues through weekly telephone MST consultation for each team of MST clinicians aimed at monitoring treatment fidelity and adherence to the MST treatment model, and through quarterly on-site booster trainings (1 ½ days each). Fully trained MST Experts will teach the on-site MST supervisor to implement a manualized MST supervisory protocol and collaborate with the supervisor to promote the ongoing clinical development of all team members. The MST Expert will also assist at the organizational level.

Ongoing organizational assistance aims to overcome barriers to achieving successful clinical outcomes through services that may include business planning, promotion of the MST program within the broader service community and developing program-level interventions designed to increase referrals, reduce staff attrition, or restructure program funding mechanisms to increase sustainability.

Quality assurance support activities focus on monitoring and enhancing program outcomes through increasing therapist and supervisor adherence to the MST treatment model. The research on MST has consistently indicated that adherence to the model is critical to achieving reduced rates of recidivism and incarceration. The MST Therapist Adherence Measure (TAM) and the MST Supervisor Adherence Measure (SAM) were validated in the research on MST with antisocial and delinquent youth and are now being implemented by all licensed MST programs. Additionally, new measures of supervisor practices, organizational, and broader systems-level influences on client outcomes are under development and are available to interested MST sites.

Successful programs require an economic environment that promotes the excellence of the services as well as the financial health of the provider organization. MST Services offers assistance to funding organizations to assure that funding structures are sufficient and the funder's program requirements are compatible with MST program standards. Examples of this type of assistance include providing materials and technical assistance to help with developing practice standards, writing a Request for Proposals (RFP), and reviewing provider responses if requested. At the funding organization's discretion, MST Services will provide technical assistance to organizations responding to funding RFPs to assure that selected proposals contain the necessary elements and address or remove barriers to implementation.

MST Services assists interested programs in conducting a feasibility study at no cost to determine if MST is the best choice given the community needs and provider organization interests. Program development costs cover all activities that prepare the MST team to accept clients and initiate program operations. The cost of ongoing program support services is based on an all-inclusive annual per-team fee within provider organizations. Those organizations wishing to take on MST Services' supporting role within their organization may be considered for Network Partner status. Consideration is based on the organization's MST program size and growth plan, its staff demonstrating high treatment fidelity and adherence to the MST model, its administration committing to execute the required quality assurance responsibilities, and their community stakeholders' commitment to financially supporting this added element.

Training Certification Process

Administratively, training certification relationships are structured as a license agreement for MST between the Medical University of South Carolina (MUSC) and the provider/implementing organization/agency. MUSC holds the intellectual property rights to MST, and MST Services is the MUSC-affiliated organization that grants license agreements and provides program development and training services for MST worldwide. Certification, in the form of MST Licensure, is not available on an individual basis but is rather granted to an organization that is fully committed to supporting the adherent implementation through all levels of implementation, from staff selection, agency practices and policies, support of the model at the agency Executive level, and by championing the model as necessary with funding and referral sources across time as system-level issues put pressure on the agency and clinicians to modify practices in ways that may not be consistent with the MST model.

Program Benefits (per individual): $25,554
Program Costs (per individual): $8,471
Net Present Value (Benefits minus Costs, per individual): $17,083
Measured Risk (odds of a positive Net Present Value): 99%

Source: Washington State Institute for Public Policy
All benefit-cost ratios are the most recent estimates published by The Washington State Institute for Public Policy for Blueprint programs implemented in Washington State. These ratios are based on a) meta-analysis estimates of effect size and b) monetized benefits and calculated costs for programs as delivered in the State of Washington. Caution is recommended in applying these estimates of the benefit-cost ratio to any other state or local area. They are provided as an illustration of the benefit-cost ratio found in one specific state. When feasible, local costs and monetized benefits should be used to calculate expected local benefit-cost ratios. The formula for this calculation can be found on the WSIPP website.

Start-Up Costs

Initial Training and Technical Assistance

Program Development and Start-up: $14,500 plus travel

  • Includes one on-site 5-day Orientation training for up to three teams. Programs with more than three teams, or teams starting on a staggered schedule will need to purchase additional on-site 5-day Orientation trainings (see Optional Services below).
  • $2,000 per additional team plus travel

MST clinical "teams" are defined to consist of a Ph.D. or master's level supervisor and two to four master's level counselors operating together to provide MST services in a specific area or region.

Increasing the number of teams trained at one time can produce economies of scale.

Curriculum and Materials

Included in costs above.

Materials Available in Other Language: All materials needed for therapist training and implementation are available in Spanish.

Licensing

Master License: $4,750 per year per agency
Team License: $2,950 per team per agency

Other Start-Up Costs

Mileage considerations.  Since MST is an in-home therapy that also works in the youth's ecology, MST teams accrue considerable mileage annually which should be factored into overall program cost.  On average, we find an MST Supervisor travels 6,000 miles per year in delivering the therapy while Therapists travel 12,000 miles per year.  Travel will vary based on population concentration of service area (more travel to serve rural populations).

Intervention Implementation Costs

Ongoing Curriculum and Materials

Included.

Staffing

  • Ratios: An MST Team of a supervisor and 4 therapists each serves an average of 3-5 families at a time.
  • Qualifications: Supervisors must be mental health professionals with a minimum of a master's degree. Therapists should also be master's level.  MST Services licenses individual teams who successfully complete the Multisystemic Therapy Program Development, Licensing and Training processes and who remain in fidelity with MST's quality assurance standards.  MST Services does not license nor require outside (state) licensor for the individual clinicians who comprise the MST team.
  • Time to Deliver Intervention: Each family is served for an average of four months with at least one weekly visit to the home.

Other Implementation Costs

Costs that will vary by locality include administrative support, space, travel, supplies, and communications. Implementation costs vary significantly across the country. MST Services provides an MST Program Budget Template to assist communities in estimating costs.

Implementation Support and Fidelity Monitoring Costs

Ongoing Training and Technical Assistance

The following fee structure applies to each team or group of teams that comes together for on-going Booster training.  In the case where teams begin operating at different times, the fee structure applies to each separate group of teams that join together for on-going Booster training (exclusive of Program Development and Start-up or 5-day Orientation training fees).

Single team programs: $37,200 per year
Plus required quality assurance services: $6,000 per year for Therapist Adherence Measure (TAM) data collection*

Two or three teams training jointly: $28,500 per team per year**

*For single team programs, the TAM data collection services provided by the MST Institute is a required part of program implementation at a cost of $6,000 per year.
**In some unusual instances, with approval from MST Services, four teams may train jointly. In this case, annual program support and training fees will be $28,500 per team per year.

Additional orientation training required for "replacement staff" (staff hired due to attrition of previously trained MST program personnel) or for teams starting up after "initial" program start-up can be provided for the following additional amounts:

  • 5-day Orientation for $12,500 per training delivered on site, plus travel costs, or
  • $950 per participant for those who attend 5-day Orientation trainings in Atlanta, Georgia (workshop fee only, exclusive of travel, lodging and meal costs).

Optional Services (provided by MST Services)

  • On-site 5-day Orientation (excludes travel expenses): $12,500 per training
  • Supervisor Workshops (excludes travel expenses): $450 per person

Fidelity Monitoring and Evaluation

Optional third-party services

  • TAM data collection services provided by MST Institute: $6,000 per team per year
  • Program evaluation services by MST Institute: TBD

Ongoing License Fees

Master License: $4,750 per year per agency
Team License: $2,950 per team per agency

Other Implementation Support and Fidelity Monitoring Costs

No information is available

Other Cost Considerations

MST Services, Inc. recommends that, when estimating costs, one should consider the implementation of the "highest quality program" in order to assure accountability for the best possible client outcomes. Economies of scale can be achieved when multiple teams can be supported at one time. It is often more difficult to locate start-up monies as opposed to ongoing implementation funding.

Year One Cost Example

In this example, an organization is setting up one MST team, with a supervisor and four therapists to serve approximately 48 families over the course of a year.

Note: The Year One costs listed below include expenses that will vary by locale (e.g., salaries and benefits, mileage, cell phone use, testing kits, and flexible funds).

Start-up and Initial Training $14,500.00
Initial Agency License Fees $4,750.00
Developer Consultation/Support Fee $37,200.00
Team License/Certification $2,950.00
Data System/Collection $6,000.00
Travel for MST staff to site assessment and trainings $12,000.00
Salary for Supervisor $78,000.00
Salary for 4 Counselors @ $57,200 $228,800.00
Fringe Benefits $76,700.00
Cell Phone $3,000.00
Mileage (54,000 miles @ $.575 per mile) $31,050.00
Flexible Funds $4,800.00
SA Testing kits $4,800.00
Indirect Costs (estimated at 20% of Direct Costs) $85,430.00
Total One Year Cost $589,980.00

With one team of four therapists and a supervisor serving 48 families, the Year One cost per family is $12,291. 

Note: The estimated cost per family served will usually be higher (by 11% to 34%) for smaller MST teams (teams with a Supervisor and 2-3 Therapists).

For assistance in developing a detailed budget, reach out to the Program Information Contact listed on the Fact Sheet.

Funding Overview

The strong track record of MST in helping states and localities achieve savings on costly out-of-home placements has led to the leverage of significant state and local funds for MST, most typically in the budgets of juvenile justice and child welfare agencies. Medicaid is also an important source of support for MST, and many states have included MST in their Medicaid State Plans as a mental health therapy. Foundation support and public-private partnerships can play an important role in helping states and localities get MST programs up and running so that they can begin to divert youth from costly placements and reinvest the savings on those placements in the continued operation and expansion of the program.

Funding Strategies

Improving the Use of Existing Public Funds

Redirection: Many states and communities have redirected state and local funding from detention and residential placements to MST. The reasons for redirecting funds include: an interest in keeping youth with their families and in their communities; poor results with current strategies and the strong track record of results for MST; and the potential for cost savings by implementing MST and keeping young people out of costly out-of-home placements.

Reinvestment: Reinvestment is a strategy that can help to bring MST to scale by seeking a commitment from public agencies that they will reinvest the savings generated by implementing MST and reducing the use of out-of-home placements into sustaining and replicating the program. It may be helpful to pair this strategy with a public-private partnership in which a private funder helps to facilitate and support the development of commitments from public agencies to reinvest savings. See the Opportunity Compact example below.

Allocating State or Local General Funds

Many states have chosen to fund MST with general funds as part of a commitment to evidence-based practices and in an effort to achieve better outcomes for youth. Some counties and cities have made additional contributions to funding the program.

Maximizing Federal Funds

Entitlements: Medicaid is an option for funding MST as a family therapy. Also, there is now a billing code assigned directly to MST (HCPCS code H2033). Some states also bill MST as a psychosocial rehabilitative service. Any Medicaid approach requires the state to provide state matching funds, with the state share percentage set by the federal government. This option is limited to the Medicaid eligible portion of the population to be served.

Formula Funds:

  • Juvenile Accountability Block Grant (JABG) Funds are focused on reducing juvenile offending through efforts that promote accountability, including providing effective early intervention through mental health screening and treatment.
  • Office of Juvenile Justice and Delinquency Prevention (OJJDP) Formula Funds support a variety of improvements to delinquency prevention programs and juvenile justice programs in states. Evidence-based programs are an explicit priority for these funds, which are typically administered on a competitive basis from the state administering agency to community-based programs.
  • The Mental Health Services Block Grant (MHSBG) can fund a variety of mental health promotion and intervention activities and is a potential source of support for functional family therapy programs.
  • Title IV-B, Parts 1 & 2 provides fairly flexible funding to state child welfare agencies for child welfare services, including prevention and family preservation activities.

Discretionary Federal Grants: Such grants have mostly been used for start-up expenses. Federal agencies including SAMSHA and OJJDP administer relevant discretionary grant programs.

Foundation Grants and Public-Private Partnerships

A number of states have used foundation grants to provide start-up funding for MST programs. Foundations are not as good a source of ongoing implementation funding. An Opportunity Compact is an example of a relevant public/private partnership, where private funding initiates an intervention such as MST, with the potential of saving money from the avoidance of a costly alternative. Savings are then used to sustain the intervention.

Debt Financing

Program-related investment, social impact bonds or government bonds can all be used for start-up and initial implementation funding for programs such as MST, which target cost avoidance for youth who would otherwise need an expensive alternative such as out-of-home placement. Savings from avoided costs would repay the investment as well as sustain the intervention.

Generating New Revenue

While presenting a challenge in securing needed public support, new revenue should be considered in the form of taxpayer referenda, new taxes and fees or dedicated revenue streams such as tax form check-offs.

Program Developer/Owner

Scott W. Henggeler, Ph.D.Medical University of South CarolinaDepartment of Psychiatry and Behavioral SciencesFamily Services Research CenterCharleston, SC 29425-0742(843) 876-1800(843) 876-1808henggesw@musc.edu

Program Outcomes

  • Close Relationships with Parents
  • Conduct Problems
  • Delinquency and Criminal Behavior
  • Externalizing
  • Internalizing
  • Mental Health - Other
  • Positive Social/Prosocial Behavior
  • Prosocial with Peers
  • Violence

Program Specifics

Program Type

  • Family Therapy
  • Juvenile Justice - Other

Program Setting

  • Transitional Between Contexts
  • Social Services
  • Home
  • Correctional Facility
  • School
  • Mental Health/Treatment Center

Continuum of Intervention

  • Indicated Prevention

Program Goals

A juvenile crime prevention program designed to improve the real-world functioning of youth by changing their natural settings - home, school, and neighborhood - in ways that promote prosocial behavior while decreasing antisocial behavior.

Population Demographics

MST® targets chronic, serious, violent, and substance abusing juvenile offenders ages 11-17.

Target Population

Age

  • Late Adolescence (15-18) - High School
  • Early Adolescence (12-14) - Middle School

Gender

  • Both

Gender Specific Findings

  • Male

Race/Ethnicity

  • All

Race/Ethnicity/Gender Details

Subgroup differences by race, ethnicity, or gender (coded in binary terms as male/female):

  • There were no subgroup findings of differences in program impact by race or ethnicity in any of the Blueprints-certified studies for MST.
  • In terms of gender (coded in binary terms as male/female), Asscher et al. (2013; Study 11) found that for adolescent cognitions, the treatment had larger (and more positive) effects for boys than for girls.

Sample demographics including race, ethnicity, and gender for Blueprints-certified studies:

  • The sample in Study 3 (Borduin et al., 1995) was 30% Black or African American and 70% White (ethnicity was not reported).
  • For Study 7 (Ogden & Halliday-Boykins, 2004), the sample demographics were not reported.
  • In Study 9 (Weiss et al., 2013), the sample was 60% Black or African American and 40% White (ethnicity was not reported).
  • For Study 11 (Deković et al., 2012), 55% of youths had a Dutch background. Of those belonging to ethnic minoritized populations, 34% had a Moroccan and 32% had a Surinamese background (race was not reported).
  • The sample for Study 13 (Butler et al., 2011) was 34% White British/European, 32% Black African, 5% Asian, and 29% of the sample's race was not specified (ethnicity was not reported).
  • The samples for all certified studies were reported in binary categories of male and female, and most of the samples were comprised of males.
  • Specifically, 67% of the sample in Study 3 (Borduin et al., 1995) was male, 63% in Study 7 (Ogden & Halliday-Boykins, 2004) were male, 83% in Study 9 (Weiss et al., 2013) were male, 73% in Study 11 (Deković et al., 2012) were male, and 82% of youth in Study 13 (Butler et al., 2011) were male.  

Other Risk and Protective Factors

Individual: Attributional bias
Family: Low warmth, parental problems, and low support
Community: Transiency, criminal subculture

Risk/Protective Factor Domain

  • Individual
  • School
  • Peer
  • Family
  • Neighborhood/Community

Risk/Protective Factors

Risk Factors

Individual: Early initiation of antisocial behavior, Early initiation of drug use, Favorable attitudes towards antisocial behavior, Rebelliousness, Substance use

Peer: Interaction with antisocial peers, Peer substance use

Family: Family conflict/violence*, Neglectful parenting, Parental attitudes favorable to antisocial behavior, Parental attitudes favorable to drug use, Parent history of mental health difficulties*, Parent stress, Poor family management*, Violent discipline

School: Low school commitment and attachment*, Poor academic performance

Neighborhood/Community: Community disorganization, Laws and norms favorable to drug use/crime, Low neighborhood attachment

Protective Factors

Individual: Clear standards for behavior, Problem solving skills, Prosocial involvement*, Rewards for prosocial involvement, Skills for social interaction*

Peer: Interaction with prosocial peers*

Family: Attachment to parents, Nonviolent Discipline*, Opportunities for prosocial involvement with parents, Parental involvement in education, Parent social support, Rewards for prosocial involvement with parents

School: Opportunities for prosocial involvement in education, Rewards for prosocial involvement in school

Neighborhood/Community: Opportunities for prosocial involvement, Rewards for prosocial involvement


*Risk/Protective Factor was significantly impacted by the program

See also: Multisystemic Therapy® (MST®) Logic Model (PDF)

Brief Description of the Program

Multisystemic Therapy® (MST®) is an intensive family- and community-based treatment that addresses the multiple causes of serious antisocial behavior in juvenile offenders. The MST program seeks to improve the real-world functioning of youth by changing their natural settings - home, school, and neighborhood - in ways that promote prosocial behavior while decreasing antisocial behavior. Therapists work with youth and their families to address the known causes of delinquency on an individualized, yet comprehensive basis. By using the strengths in each system (family, peers, school, and neighborhood) to facilitate change, MST addresses the multiple factors known to be related to delinquency across the key systems within which youth are embedded. The extent of treatment varies by family according to clinical need. Therapists generally spend more time with families in the initial weeks (daily if needed) and gradually taper their time (to as infrequently as once a week) over the 3- to 5-month course of treatment.

Description of the Program

Multisystemic Therapy® (MST®) is an intensive family- and community-based treatment that addresses the multiple causes of serious antisocial behavior across key settings, or systems within which youth are embedded (family, peers, school, and neighborhood). Because MST emphasizes promoting behavior change in the youth's natural environment, the program aims to empower parents with the skills and resources needed to independently address the inevitable difficulties that arise in raising teenagers, and to empower youth to cope with the family, peer, school, and neighborhood problems they encounter.

Within a context of support and skill building, the therapist places developmentally appropriate demands on adolescents and their families to reduce problem behavior. Initial therapy sessions identify the strengths and weaknesses of the adolescent, the family, and their transactions with extrafamilial systems (e.g., peers, friends, school, parental workplace). Problems identified by both family members and the therapist are explicitly targeted for change by using the strengths in each system to facilitate such change. Treatment approaches are derived from well-validated strategies such as strategic family therapy, structural family therapy, behavioral parent training, and cognitive behavioral therapy.

While MST focuses on addressing the known causes of delinquency on an individualized comprehensive basis, several types of interventions are typically identified for serious juvenile offenders and their families. At the family level, MST interventions aim to remove barriers to effective parenting (e.g., parental substance abuse, parental psychopathology, low social support, high stress, and marital conflict), to enhance parenting competencies, and to promote affection and communication among family members. Interventions might include introducing systematic monitoring, reward, and discipline systems; prompting parents to communicate effectively with each other about adolescent problems; problem solving day-to-day conflicts; and developing social support networks. At the peer level, interventions frequently are designed to decrease affiliation with delinquent and drug-using peers and to increase affiliation with prosocial peers. Interventions in the school domain may focus on establishing positive lines of communication between parents and teachers, parental monitoring of the adolescent's school performance, and restructuring after-school hours to support academic efforts. Individual level interventions generally involve using cognitive behavior therapy to modify the individual's social perspective-taking skills, belief system, or motivational system, and encouraging the adolescent to deal assertively with negative peer pressure.

A Master's-Level therapist, with a caseload of 4 to 6 families, provides most mental health services and coordinates access to other important services (e.g., medical, educational, and recreational). While the therapist is available to the family 24 hours a day, 7 days a week, the direct contact hours per family varies according to clinical need. Generally, the therapist spends more time with the family in the initial weeks of the program (daily if needed) and gradually tapers off (as infrequently as once a week) during a 3- to 5-month course of treatment. Treatment fidelity is maintained by weekly group supervision meetings involving 3 to 4 therapists and a Doctoral-Level or advanced Master's-Level clinical supervisor. The group reviews the goals and progress of each case to ensure the multisystemic focus of the therapists' intervention strategies, identify barriers to success, and facilitate the attainment of treatment goals. In addition, an MST expert consultant reviews each case with the team weekly to promote treatment fidelity and favorable clinical outcomes.

The design and implementation of MST interventions are based on the following nine core principles of MST. An extensive description of these principles, with examples that illustrate the translation of these principles into specific intervention strategies are provided in comprehensive clinical volumes (Henggeler et al., 1998; 2009).

  1. The primary purpose of assessment is to understand the "fit" between the identified problems and their broader systemic context.
  2. Therapeutic contacts emphasize the positive and use systemic strengths as levers for change.
  3. Interventions are designed to promote responsible behavior and decrease irresponsible behavior among family members.
  4. Interventions are present-focused and action-oriented, targeting specific and well-defined problems.
  5. Interventions target sequences of behavior within and between multiple systems that maintain the identified problems.
  6. Interventions are developmentally appropriate and fit the developmental needs of the youth.
  7. Interventions are designed to require daily or weekly effort by family members.
  8. Intervention effectiveness is evaluated continuously from multiple perspectives with providers assuming accountability for overcoming barriers to successful outcomes.
  9. Interventions are designed to promote treatment generalization and long-term maintenance of therapeutic change by empowering caregivers to address family members' needs across multiple systemic contexts.

Theoretical Rationale

MST is based upon the social-ecological model of behavior. According to this perspective, behavior is determined through the reciprocal interplay of the child and his or her social ecology, including the family, peers, neighborhood, and other community settings. Research demonstrates that there are factors within the social settings youth are embedded that put youth at greater risk for criminal and antisocial behavior. Likewise, there are also factors within the social environment that encourage involvement in prosocial behavior and protect youth from involvement in antisocial and criminal behavior. Problem behavior may be a function of difficulty within any of these social settings and/or difficulties that characterize the interfaces between these settings (i.e., family-school relations or family-neighborhood relations). Based on this theoretical rationale, MST interventions are tailored to address the specific risk and protective factors that are salient to the social environments of the individual and family receiving the treatment.

Theoretical Orientation

  • Person - Environment

Brief Evaluation Methodology

Primary Evidence Base for Certification

Of the 23 studies Blueprints has reviewed, 5 (studies 3, 7, 9, 11 and 13) meet Blueprints evidentiary standards (specificity, evaluation quality, impact, dissemination readiness). In addition, Study 3 was done by the developer; Studies 7, 9, 11 and 13 were conducted by independent researchers.

Study 3

Borduin et al. (1995), Sawyer et al. (2011), Schaeffer et al. (2005), and Wagner et al. (2014) used a randomized controlled trial that assigned 176 juvenile offenders ages 12-17 to an MST intervention group or an individual therapy control group. Assessments were conducted at posttest through a 21.9-year follow-up and measured problem behavior, arrests, and family relations.

Study 7

Ogden and Halliday-Boykins (2004), Ogden and Hagen (2006, 2009), and Ogden et al. (2007) used a randomized controlled trial that assigned 100 Norwegian youths with serious antisocial behavior to an MST intervention group or a control group receiving child welfare services. Assessments at posttest and two-year follow-up measured problem behavior, delinquency, and out-of-home placement.

Study 9

Weiss et al. (2013, 2015) used a randomized controlled trial that assigned 164 youths enrolled in behavior intervention classrooms to an MST intervention group or a no additional treatment control group. Assessments from posttest to a one-year follow-up measured conduct problems, externalizing, and criminal offending.

Study 11

Asscher et al. (2013, 2014, 2018), Deković et al. (2012), and Manders et al. (2013 used a randomized controlled trial that assigned 256 Dutch adolescents with antisocial behavior to an MST intervention group or a treatment-as-usual control group. Assessments from posttest to a three-year follow-up measured a variety of outcomes relating to antisocial behavior.

Study 13

Butler et al. (2011) used a randomized controlled trial that assigned 108 juvenile offenders in London, England, to an MST intervention group or a treatment-as-usual control group. Assessments at baseline and from posttest to an 18-month follow-up measured reoffending and problem behavior.

Outcomes (Brief, over all studies)

Primary Evidence Base for Certification

Study 3

Borduin et al. (1995) found that at the posttest, relative to the control group, the MST intervention group showed significantly decreased problem behaviors and improved family relations. Additionally, intervention group participants had lower re-arrest rates through the four-year follow-up.

Study 7

Ogden & Halliday-Boykins (2004) found that at the posttest, the MST intervention group, relative to the usual-services control group, showed significantly decreased youth internalizing symptoms, increased social competence, and decreased out-of-home placements.

Study 9

Weiss et al. (2013) found that at the posttest, relative to the control group, the MST intervention group showed significantly lower externalizing problems and absenteeism in school.

Study 11

Deković et al. (2012) and Asscher et al. (2013, 2014) found that, relative to the control group, the MST intervention group showed significant reductions at both the posttest and six-month follow up in externalizing, oppositional defiant disorder, conduct disorder, relationship quality, and property offenses.

Study 13

Butler et al. (2011) found that, relative to the control group, the MST intervention group showed significant decreases in aggression, delinquency, and psychopathic traits at posttest and nonviolent offenses at the 18-month follow-up assessment.

Outcomes

Primary Evidence Base for Certification

Study 3

Borduin et al. (1995) found that at the posttest, relative to the control group, the MST intervention group showed significantly

  • Decreased youth behavior problems
  • Improved family relations
  • Lower recidivism rates (re-arrested at least once) through 4-year follow-up.

Study 7

At the posttest, Ogden et al. (2004) found that the MST intervention group, relative to the usual-services control group, showed significantly:

  • Decreased youth internalizing symptoms
  • Increased social competence
  • Decreased out-of-home placements.

Study 9

Weiss et al. (2013) found that at the posttest, relative to the control group, the MST intervention group showed significantly lower:

  • Externalizing problems
  • Absenteeism in school.

Study 11

Asscher et al. (2013, 2014) and Deković et al. (2012) found that, relative to the control group, the MST intervention group showed significant reductions at both the posttest and six-month follow up in:

  • Externalizing
  • Oppositional defiant disorder
  • Conduct disorder
  • Property offenses
  • Relationship quality.

Study 13

Butler et al. (2011) found that, relative to the control group, the MST intervention group showed significant decreases in

  • Aggression and delinquency at posttest
  • Psychopathic traits at posttest
  • Nonviolent offenses at the 18-month follow-up assessment.

Generalizability

Five studies meet Blueprints standards for high quality in methods with strong evidence of program impact (i.e., "certified" by Blueprints): Study 3 (Borduin et al., 1995), Study 7 (Ogden & Halliday-Boykins, 2004), Study 9 (Weiss et al., 2013), Study 11 (Deković et al., 2012) and Study 13 (Butler et al., 2011).  The samples for all these studies included youths with serious behavior problems.

  • The setting for Study 3 (Borduin et al., 1995) was the Missouri Department of Youth Services (in the United States), in which MST was compared to an individual therapy control group.
  • Study 7 (Ogden & Halliday-Boykins, 2004) took place in Europe within the Norwegian municipal child welfare services, in which MST was compared to a control group receiving usual child welfare services.
  • Study 9 (Weiss et al., 2013) took place in the southeastern U.S. in self-contained behavior intervention classrooms in a restrictive setting, in which MST plus classroom services was compared to classroom services only.
  • Study 11 (Deković et al., 2012) was conducted by the Bureau of Youth Care or Child Protection Council in Europe within 3 Dutch cities in which MST was compared to a treatment-as-usual control group.
  • Study 13 (Butler et al., 2011) was conducted within two youth offending services in North London (Europe) where MST was compared to an alternative treatment (Youth Offending Teams).

Potential Limitations

Additional Studies (not certified by Blueprints)

Study 1 (Henggeler et al., 1992, 1993):

  • Evidence of differential attrition
  • No intent-to-treat analysis

Henggeler, S. W., Melton, G. B., & Smith, L. A. (1992). Family preservation using Multisystemic Therapy: An effective alternative to incarcerating serious juvenile offenders. Journal of Consulting and Clinical Psychology, 6, 953-961.

Henggeler, S. W., Melton, G. B., Smith, L. A., Schoenwald, S. K., & Hanley, J. H. (1993). Family preservation using multisystemic treatment: Long-term followup to a clinical trial with serious juvenile offenders. Journal of Child and Family Studies, 2, 283-293.

Study 2 (Henggeler et al., 1991):

  • No information on attrition
  • Incomplete tests for differential attrition

Henggeler, S. W., Borduin, C. M., Melton, G. B., Mann, B. J., Smith, L. A., Hall, J. A., . . . Fucci, B. R. (1991). Effects of multisystemic therapy on drug use and abuse in serious juvenile offenders: A progress report from two outcome studies. Family Dynamics of Addiction Quarterly, 1(3), 40-51.

Study 4 (Henggler et al., 1997, 2000):

  • Very few effects on behavioral outcomes

Henggeler, S. W., Melton, G. B., Brondino, M. J., Scherer, D. G., & Hanley, J. H. (1997). Multisystemic Therapy with violent and chronic juvenile offenders and their families: The role of treatment fidelity in successful dissemination. Journal of Consulting and Clinical Psychology, 65, 821-833.

Huey, S. J., Henggeler, S. W., Brondino, M. J., & Pickrel, S. G. (2000). Mechanisms of change in Multisystemic Therapy: Reducing delinquent behavior through therapist adherence and improved family and peer functioning. Journal of Consulting and Clinical Psychology, 68(3), 451-467.

Study 5 (Henggeler et al., 1986):

  • Quasi-experimental design (QED) with non-random assignment and limited matching
  • Design confound
  • No intent-to-treat analysis

Henggeler, S. W., Rodick, J. D., Borduin, C. M., Hanson, C. L., Watson, S. M., & Urey, J. R. (1986). Multisystemic treatment of juvenile offenders: Effects on adolescent behavior and family interaction. Developmental Psychology, 22, 132-141.

Study 6 (Brown et al., 1999; Schoenwald et al., 1996; Henggeler et al., 1999, 2002):

  • Differences between conditions at baseline
  • Evidence of differential attrition
  • Few effects were maintained at six-month follow-up

Brown, T. L., Henggeler, S. W., Schoenwald, S. K., Brondino, M. J., & Pickrel, S. G. (1999). Multisystemic treatment of substance abusing and dependent juvenile delinquents: Effects on school attendance at posttreatment and 6-month follow-up. Children's Services: Social Policy, Research, and Practice, 2(2), 81-93.

Henggeler, S. W., Clingempeel, W. G., Brondino, M. J., & Pickrel, S. G. (2002). Four-year follow-up of Multisystemic Therapy with substance-abusing and substance-dependent juvenile offenders. Journal of the American Academy of Child and Adolescent Psychiatry, 41(7), 868-874.

Henggeler, S. W., Pickrel, S. G., & Brondino, M. J. (1999). Multisystemic treatment of substance-abusing and dependent delinquents: Outcomes, treatment fidelity, and transportability. Mental Health Services Research, 1(3), 171-184.

Schoenwald, S. K., Ward, D. M., Henggeler, S. W., Pickrel, S. G., & Patel, H. (1996). Multisystemic Therapy treatment of substance abusing or dependent adolescent offenders: Costs of reducing incarceration, inpatient, and residential placement. Journal of Child and Family Studies, 5(4), 431-444.

Study 8 (Leschied & Cunningham, 2002):

  • Differences between conditions at baseline
  • Evidence of differential attrition

Leschied, A., & Cunningham, A. (2002). Seeking effective interventions for serious young offenders: Interim results of a four-year randomized study of Multisystemic Therapy in Ontario, Canada. London, Canada: Centre for Children and Families in the Justice System.

Study 10 (Fain et al., 2014):

  • Used a quasi-experimental design (QED) with no matching
  • Ethnicity differed between groups at baseline
  • No tests for missing data on several of the outcomes
  • One iatrogenic effect for African Americans and a possible iatrogenic effect for the full sample (without controls)

Fain, T., Greathouse, S. M., Turner, S. F., & Weinberg, H. D. (2014). Effectiveness of Multisystemic Therapy for minority youth: Outcomes over 8 years in Los Angeles County. Journal of Juvenile Justice, 3(2), 24-37.

Study 12 (Timmons-Mitchell et al., 2006):

  • Some condition differences in the timing of assessments
  • Possible violation of the intent-to-treat criterion
  • Not possible to use baseline outcome controls in all analyses
  • Differences between conditions at baseline
  • Evidence of differential attrition

Timmons-Mitchell, J., Bender, M., Kishna, M. A., & Mitchell, C. (2006). An independent effectiveness trial of Multisystemic Therapy with juvenile justice youth. Journal of Clinical Child and Adolescent Psychology, 35(2), 227-236.

Study 14 (Sundell et al., 2008; Lofhölm et al., 2009):

  • Mostly used baseline outcome controls but sometimes not possible
  • No baseline-by-condition tests for differential attrition
  • No significant effects on behavioral outcomes
  • Only one significant effect for risk and protective factors

Sundell, K., Hansson, K., Lofhölm, C. A., Olsson, T., Gustle, L.-H., Kadesjö, C. (2008). The transportability of Multisystemic Therapy to Sweden: Short-term results from a randomized trial of conduct-disordered youths. Journal of Family Psychology, 22(3), 550-560.

Lofhölm, C. A., Olsson, T., Sundell, K., & Hansson, K. (2009). Multisystemic Therapy with conduct- disordered young people: Stability of treatment outcomes two years after intake. Evidence & Policy, 5(4), 373-397.

Study 15 (Fonagy et al., 2018, 2020a, 2020b):

  • No significance tests for baseline equivalence, but the conditions appear balanced
  • Some differential attrition and no tests for baseline-by-condition attrition
  • A few iatrogenic effects in the many tests
  • No posttest effects were maintained long-term

Fonagy, P., Butler, S., Cottrell, D., Scott, S., Pilling, S., Eisler, I., . . . Goodyer I. M. (2018). Multisystemic therapy versus management as usual in the treatment of adolescent antisocial behaviour (START): A pragmatic, randomised controlled, superiority trial. The Lancet Psychiatry, 5(2), 119-133.

Fonagy, P., Butler, S., Cottrell, D., Scott, S., Pilling, S., Eiser I., . . ., Goodyer, I. M. (2020a). Multisystemic therapy versus management as usual in the treatment of adolescent antisocial behaviour (START): 5-year follow-up of a pragmatic, randomised controlled, superiority trial. Lancet Psychiatry, 7, 420-430. doi: https://doi.org/10.1016/ S2215-0366(20)30131-0

Fonagy, P., Butler, S., Cottrell, D., Scott, S., Pilling, S., Eisler, I., . . . Goodyer, I. M. (2020b). Multisystemic therapy compared with management as usual for adolescents at risk of offending: The START II RCT. Health Services and Delivery Research, 8(23), 1-114.

Study 16 (Stambaugh et al., 2007):

  • Quasi-experimental design (QED) with non-random assignment and no matching
  • Differences between conditions at baseline
  • Tests for differential attrition are incomplete

Stambaugh, L. F., Mustillo, S. A., Burns, B. J., Stephens, R. L., Baxter, B., Edwards, D., & DeKraai, M. (2007). Outcomes from wraparound and Multisystemic Therapy in a center for mental health services system-of-care demonstration site. Journal of Emotional and Behavioral Disorders, 15(3), 143-155. doi: http://dx.doi.org/10.1177/10634266070150030201

Study 17 (Glisson et al., 2010):

  • Baseline control was not possible for out-of-home placement
  • Tests for baseline equivalence are incomplete

Glisson, C., Schoenwald, S. K., Hemmelgarn, A., Green, P., Dukes, D., Armstrong, K. S., & Chapman, J. E. (2010). Randomized trial of MST and ARC in a two-level evidence-based treatment implementation strategy. Journal of Consulting and Clinical Psychology, 78(4), 537-550.

Study 18 (Mayfield, 2011):

  • Quasi-experimental design (QED) with non-random assignment and limited matching
  • No reliability or validity information
  • No effects on behavioral outcomes
  • Differences between conditions at baseline

Mayfield, J. (2011). Multisystemic therapy outcomes in an evidence-based practice pilot. Olympia: Washington State Institute for Public Policy.

Study 19 (Baglivio et al., 2014):

  • Quasi-experimental design (QED) with good but not ideal propensity score matching
  • Possible validity problems with one outcome measure
  • Not possible to control for baseline recidivism measures
  • No significant MST effects
  • Possible iatrogenic effect relative to another high-quality intervention

Baglivio, M. T., Jackowski, K., Greenwald, M. A., & Wolff, K. T. (2014). Comparison of Multisystemic Therapy and Functional Family Therapy effectiveness: A multiyear statewide propensity score matching analysis of juvenile offenders. Criminal Justice and Behavior, 41(9), 1033-1056.

Study 20 (Eeren et al., 2018):

  • Quasi-experimental design (QED) with limited propensity score matching
  • No intent-to-treat analysis
  • Evidence of differential attrition
  • Very few effects on behavioral outcomes

Eeren, H. V., Goossens, L. M. A., Scholte, R. H. J., Busschbach, J. V., & van der Rijken, R. E. A. (2018). Multisystemic Therapy and Functional Family Therapy compared on their effectiveness using the propensity score method. Journal of Abnormal Child Psychology, 46, 1037-1050. doi: https://doi.org/10.1007/s10802-017-0392-4

Study 21 (Blankestein et al., 2019):

  • Quasi-experimental design (QED) with limited propensity score matching
  • Limited information on reliability for secondary measures
  • No tests for differential attrition
  • No effects on behavioral outcomes

Blankestein, A., van der Rijken, R., Eeren, H. V., Lange, A., Scholte, R., Moonen, X., . . ., & Didden, R. (2019). Evaluating the effects of Multisystemic Therapy for adolescents with intellectual disabilities and antisocial or delinquent behaviour and their parents. Journal of Applied Research on Intellectual Disabilities, 32, 575-590. doi: https://doi.org/10.1111/jar.12551

Study 22 (Vidal et al., 2017):

  • QED with limited propensity score matching

Vidal, S., Steeger, C. M., Caron, C., Lasher, L., & Connell, C. M. (2017). Placement and delinquency outcomes among system-involved youth referred to Multisystemic Therapy: A propensity score matching analysis. Administrative Policy in Mental Health, 44, 853-866. https://doi.org/10.1007/s10488-017-0797-y

Study 23 (Wagner et al., 2019):

  • Differences between conditions at baseline
  • Some evidence of differential attrition and incomplete tests
  • No effects on behavioral outcomes
  • Very small or specialized sample

Wagner, D. V., Borduin, C. M., Mazurek, M. O., Kanne, S. M., & Dopp, A. R. (2019). Multisystemic Therapy for disruptive behavior problems in youths with autism spectrum disorder: Results from a small randomized clinical trial. Evidence-Based Practice in Child and Adolescent Mental Health, 4(1), 42-54. doi: 10.1080/23794925.2018.1560237

Notes

Study 15 (Fonagy et al., 2015, 2020a, 2020b) is registered, number ISRCTN7713221.

Endorsements

Blueprints: Model Plus
Crime Solutions: Effective
OJJDP Model Programs: Effective
SAMHSA : 2.9-3.2

Peer Implementation Sites

Domestic
Mike Williams
MST Program Manager, Network Partner Director
Advanced Behavioral Health
mwilliams@abhct.com
Phone number: (860) 704-6436

International
Cathy James
MST Programme Lead MSTUK
National Implementation Service
cathy.james@kcl.ac.uk
0207 848 5843

Program Information Contact

Tom Pietkiewicz
Director of Business Development
MST Services, Inc.
3490 Piedmont Road NE, Suite 1250
Atlanta, GA 30305
Office: (843) 352-4306
Cell: (404) 395-6038
Email: tom.pietkiewicz@mstservices.com

References

Study 1

Henggeler, S. W., Melton, G. B., & Smith, L. A. (1992). Family preservation using Multisystemic Therapy: An effective alternative to incarcerating serious juvenile offenders. Journal of Consulting and Clinical Psychology, 6, 953-961.

Henggeler, S. W., Melton, G. B., Smith, L. A., Schoenwald, S. K., & Hanley, J. H. (1993). Family preservation using multisystemic treatment: Long-term followup to a clinical trial with serious juvenile offenders. Journal of Child and Family Studies, 2, 283-293.

Study 2

Henggeler, S. W., Borduin, C. M., Melton, G. B., Mann, B. J., Smith, L. A., Hall, J. A., . . . Fucci, B. R. (1991). Effects of multisystemic therapy on drug use and abuse in serious juvenile offenders: A progress report from two outcome studies. Family Dynamics of Addiction Quarterly, 1(3), 40-51.

Study 3

Certified Borduin, C. M., Mann, B. J., Cone, L. T., Henggeler, S. W., Fucci, B. R., Blaske, D. M. & Williams, R. A. (1995). Multisystemic treatment of serious juvenile offenders: Long-term prevention of criminality and violence. Journal of Consulting and Clinical Psychology, 63, 569-578.

Sawyer, A. M, & Borduin, C. M. (2011). Effects of Multisystemic Therapy through midlife: A 21.9-year follow-up to a randomized clinical trial with serious and violent juvenile offenders. Journal of Consulting and Clinical Psychology, 79(5), 643-652.

Schaeffer, C. M., & Borduin, C. M. (2005). Long-term follow-up to a randomized clinical trial of Multisystemic Therapy with serious and violent juvenile offenders. Journal of Consulting and Clinical Psychology, 73(3), 445-453.

Wagner, D. V., Borduin, C. M., Sawyer, A. M., & Dopp, A. R. (2014). Long-term prevention of criminality in siblings of serious and violent juvenile offenders: A 25-year follow-up to a randomized clinical trial of Multisystemic Therapy. Journal of Consulting and Clinical Psychology, 82(3), 492-499.

Study 4

Henggeler, S. W., Melton, G. B., Brondino, M. J., Scherer, D. G., & Hanley, J. H. (1997). Multisystemic Therapy with violent and chronic juvenile offenders and their families: The role of treatment fidelity in successful dissemination. Journal of Consulting and Clinical Psychology, 65, 821-833.

Huey, S. J., Henggeler, S. W., Brondino, M. J., & Pickrel, S. G. (2000). Mechanisms of change in Multisystemic Therapy: Reducing delinquent behavior through therapist adherence and improved family and peer functioning. Journal of Consulting and Clinical Psychology, 68(3), 451-467.

Study 5

Henggeler, S. W., Rodick, J. D., Borduin, C. M., Hanson, C. L., Watson, S. M., & Urey, J. R. (1986). Multisystemic treatment of juvenile offenders: Effects on adolescent behavior and family interaction. Developmental Psychology, 22, 132-141.

Study 6

Brown, T. L., Henggeler, S. W., Schoenwald, S. K., Brondino, M. J., & Pickrel, S. G. (1999). Multisystemic treatment of substance abusing and dependent juvenile delinquents: Effects on school attendance at posttreatment and 6-month follow-up. Children's Services: Social Policy, Research, and Practice, 2(2), 81-93.

Henggeler, S. W., Clingempeel, W. G., Brondino, M. J., & Pickrel, S. G. (2002). Four-year follow-up of Multisystemic Therapy with substance-abusing and substance-dependent juvenile offenders. Journal of the American Academy of Child and Adolescent Psychiatry, 41(7), 868-874.

Henggeler, S. W., Pickrel, S. G., & Brondino, M. J. (1999). Multisystemic treatment of substance-abusing and dependent delinquents: Outcomes, treatment fidelity, and transportability. Mental Health Services Research, 1(3), 171-184.

Schoenwald, S. K., Ward, D. M., Henggeler, S. W., Pickrel, S. G., & Patel, H. (1996). Multisystemic Therapy treatment of substance abusing or dependent adolescent offenders: Costs of reducing incarceration, inpatient, and residential placement. Journal of Child and Family Studies, 5(4), 431-444.

Study 7

Ogden, T., & Hagen, K.A. (2006). Multisystemic Therapy of serious behaviour problems in youth: Sustainability of therapy effectiveness two years after intake. Journal of Child and Adolescent Mental Health, 11, 142-149.

Ogden, T., & Hagen, K.A. (2009). What works for whom? Gender differences in intake characteristics and treatment outcomes following Multisystemic Therapy. Journal of Adolescence, 32, 1425-1435.

Certified

Ogden, T., & Halliday-Boykins, C. A. (2004). Multisystemic treatment of antisocial adolescents in Norway: Replication of clinical outcomes outside of the US. Child and Adolescent Mental Health, 9(2), 77-83.

Ogden, T., Hagen, K. A., & Andersen, O. (2007). Sustainability of the effectiveness of a programme of multisystemic treatment (MST) across participant groups in the second year of operation. Journal of Children's Services, 2, 4-14.

Study 8

Leschied, A., & Cunningham, A. (2002). Seeking effective interventions for serious young offenders: Interim results of a four-year randomized study of Multisystemic Therapy in Ontario, Canada. London, Canada: Centre for Children and Families in the Justice System.

Study 9

Certified Weiss, B., Han, S., Harris, V., Catron, T., Ngo, V. K., Caron, A., . . . Guth, C. (2013). An independent randomized clinical trial of Multisystemic Therapy with non-court-referred adolescents with serious conduct problems. Journal of Consulting and Clinical Psychology, 81(6), 1027-1039.

Weiss, B., Han, S. S., Tran, N. T., Gallop, R., & Ngo, V. K. (2015). Test of "facilitation" vs. "proximal process" moderator models for the effects of Multisystemic Therapy on adolescents with severe conduct problem. Journal of Abnormal Child Psychology43, 971-983. https://doi.org/10.1007/s10802-014-9952-z

Study 10

Fain, T., Greathouse, S. M., Turner, S. F., & Weinberg, H. D. (2014). Effectiveness of Multisystemic Therapy for minority youth: Outcomes over 8 years in Los Angeles County. Journal of Juvenile Justice, 3(2), 24-37.

Study 11

Certified

Asscher, J. J., Deković, M., Manders, W. A., van der Laan, P. H., Prins, P. J. M., & the Dutch MST Cost-Effectiveness Study Group 4. (2013). A randomized controlled trial of the effectiveness of Multisystemic Therapy in the Netherlands: Post-treatment changes and moderator effects. Journal of Experimental Criminology, 9, 169-187.

Certified

Asscher, J. J., Deković, M., Manders, W. A., van der Laan, P. H., Prins, P. J. M., van Arum, S., & the Dutch MST Cost-Effectiveness Study Group 4. (2014). Sustainabilityof the effects of multisystem therapy for juvenile delinquents in The Netherlands: Effects on delinquency and recidivism. Journal Experimental Criminology, 10, 227-243.

Certified

Deković, M., Asscher, J. J., Manders, W. A., Prins, P. J. M., & van der Laan, P. (2012). Within-intervention change: Mediators of intervention effects during Multisystemic Therapy. Journal of Consulting and Clinical Psychology, 80(4), 574-587.

Manders, W. A., Deković, M., Asscher, J. J., van der Laan, P. H., & Prins, P. J. M. (2013). Psychopathy as predictor and moderator of Multisystemic Therapy outcomes among adolescents treated for antisocial behavior. Journal of Abnormal Child Psychology, 41, 1121-1132.

Asscher, J. J., Dekovic, M., Van den Akker, A. L., Prins, P. J. M., & Van der Laan, P. H. (2018). Do extremely violent juveniles respond differently to treatment? International Journal of Offender Therapy and Comparative Criminology, 62(4), 958-977. doi: 10.1177/0306624X16670951

Study 12

Timmons-Mitchell, J., Bender, M., Kishna, M. A., & Mitchell, C. (2006). An independent effectiveness trial of Multisystemic Therapy with juvenile justice youth. Journal of Clinical Child and Adolescent Psychology, 35(2), 227-236.

Study 13

Certified Butler, S., Baruch, G., Hickey, N., & Fonagy, P. (2011). A randomized controlled trial of Multisystemic Therapy and a statutory therapeutic intervention for young offenders. Journal of the American Academy of Child and Adolescent Psychiatry, 50(12), 1220-1235.

Study 14

Sundell, K., Hansson, K., Lofhölm, C. A., Olsson, T., Gustle, L.-H., Kadesjö, C. (2008). The transportability of Multisystemic Therapy to Sweden: Short-term results from a randomized trial of conduct-disordered youths. Journal of Family Psychology, 22(3), 550-560.

Lofhölm, C. A., Olsson, T., Sundell, K., & Hansson, K. (2009). Multisystemic Therapy with conduct- disordered young people: Stability of treatment outcomes two years after intake. Evidence & Policy, 5(4), 373-397.

Study 15

Fonagy, P., Butler, S., Cottrell, D., Scott, S., Pilling, S., Eisler, I., . . . Goodyer I. M. (2018). Multisystemic therapy versus management as usual in the treatment of adolescent antisocial behaviour (START): A pragmatic, randomised controlled, superiority trial. The Lancet Psychiatry, 5(2), 119-133.

Fonagy, P., Butler, S., Cottrell, D., Scott, S., Pilling, S., Eiser I., . . ., Goodyer, I. M. (2020a). Multisystemic therapy versus management as usual in the treatment of adolescent antisocial behaviour (START): 5-year follow-up of a pragmatic, randomised controlled, superiority trial. Lancet Psychiatry, 7, 420-430. doi: https://doi.org/10.1016/ S2215-0366(20)30131-0

Fonagy, P., Butler, S., Cottrell, D., Scott, S., Pilling, S., Eisler, I., . . . Goodyer, I. M. (2020b). Multisystemic therapy compared with management as usual for adolescents at risk of offending: The START II RCT. Health Services and Delivery Research, 8(23), 1-114.

Study 16

Stambaugh, L. F., Mustillo, S. A., Burns, B. J., Stephens, R. L., Baxter, B., Edwards, D., & DeKraai, M. (2007). Outcomes from wraparound and Multisystemic Therapy in a center for mental health services system-of-care demonstration site. Journal of Emotional and Behavioral Disorders, 15(3), 143-155. doi: http://dx.doi.org/10.1177/10634266070150030201

Study 17

Glisson, C., Schoenwald, S. K., Hemmelgarn, A., Green, P., Dukes, D., Armstrong, K. S., & Chapman, J. E. (2010). Randomized trial of MST and ARC in a two-level evidence-based treatment implementation strategy. Journal of Consulting and Clinical Psychology, 78(4), 537-550.

Study 18

Mayfield, J. (2011). Multisystemic therapy outcomes in an evidence-based practice pilot. Olympia: Washington State Institute for Public Policy.

Study 19

Baglivio, M. T., Jackowski, K., Greenwald, M. A., & Wolff, K. T. (2014). Comparison of Multisystemic Therapy and Functional Family Therapy effectiveness: A multiyear statewide propensity score matching analysis of juvenile offenders. Criminal Justice and Behavior, 41(9), 1033-1056.

Study 20

Eeren, H. V., Goossens, L. M. A., Scholte, R. H. J., Busschbach, J. V., & van der Rijken, R. E. A. (2018). Multisystemic Therapy and Functional Family Therapy compared on their effectiveness using the propensity score method. Journal of Abnormal Child Psychology, 46, 1037-1050. doi: https://doi.org/10.1007/s10802-017-0392-4

Study 21

Blankestein, A., van der Rijken, R., Eeren, H. V., Lange, A., Scholte, R., Moonen, X., . . ., & Didden, R. (2019). Evaluating the effects of Multisystemic Therapy for adolescents with intellectual disabilities and antisocial or delinquent behaviour and their parents. Journal of Applied Research on Intellectual Disabilities, 32, 575-590. doi: https://doi.org/10.1111/jar.12551

Study 22

Vidal, S., Steeger, C. M., Caron, C., Lasher, L., & Connell, C. M. (2017). Placement and delinquency outcomes among system-involved youth referred to Multisystemic Therapy: A propensity score matching analysis. Administrative Policy in Mental Health, 44, 853-866. https://doi.org/10.1007/s10488-017-0797-y

Study 23

Wagner, D. V., Borduin, C. M., Mazurek, M. O., Kanne, S. M., & Dopp, A. R. (2019). Multisystemic Therapy for disruptive behavior problems in youths with autism spectrum disorder: Results from a small randomized clinical trial. Evidence-Based Practice in Child and Adolescent Mental Health, 4(1), 42-54. doi: 10.1080/23794925.2018.1560237

Study 1

Summary

The study used a randomized controlled trial that assigned 84 youths at risk of out-of-home placement for serious criminal activity to an MST intervention group or a control group receiving usual youth services. Assessments at posttest and 2.4-year follow-up measured aggression, criminal activity, and arrests.

The study reported that, relative to the control group, the MST intervention group showed significantly:

  • Lower self-reported delinquency at posttest
  • Less aggression with peers at posttest
  • Greater family cohesion at posttest
  • Fewer arrests and incarcerations at 59 weeks
  • Longer time to rearrest through 2.4 years.

Evaluation Methodology

Design: A pretest-posttest control group design, with random assignment to conditions and follow-up for arrest and incarceration measures was used to compare the effectiveness of family preservation using MST versus the usual services delivered by the Department of Youth Services (DYS). Eligible youths were referred by the DYS in yoked pairs, with one youth randomly selected to receive MST (n=43) and the other to receive the usual services (n=41). The study excluded six youths assigned to MST because they did not participate in the program, likely violating the intent-to-treat criterion. Assessments were administered to both groups at approximately the same time.

Therapists with caseloads of 4 delivered the MST program to the treatment group. On average, the treatment duration was 13.4 weeks encompassing 33 hours of direct contact, with 24-hour-a-day case coverage. Depending on the stage of treatment, sessions were held as often as every day or as infrequently as once a week usually in the family's home and ranged in duration from 15 to 90 minutes.

Youths in the usual service condition met monthly with a probation officer who monitored their compliance with court ordered stipulations (e.g., curfew, school attendance, participation with other agencies). Although youths and families were often referred for mental health services, few substantive services were actually delivered.

Pretreatment and posttreatment assessments were completed by 77% of the families in the MST condition (n=33) and 56% of families in the control condition. Criminal histories and demographics of the participants completing both assessments are essentially the same as the larger sample. Tests of attrition showed families who completed both assessments, compared to those who did not, were more likely to be African-American, to have participated in the MST condition, and to have reported higher maternal symptomatology, more structured family relations, and greater social competence in the youth. Attrition analyses on premature treatment terminators were not appropriate for this study because it was not possible to terminate usual services.

Sample: Out of 96 youths who were originally referred by DYS staff because they were at-risk for out-of-home placement due to involvement in serious criminal activity, 84 participated in the study (n=43 in MST and n=41 usual-services). Over half of the youths in the sample had been convicted of a serious violent offense and the sample averaged 3.5 previous arrests. The mean age of the sample was 15.2 years, 77% were male, 56% African American, 42% were Caucasian, and 2% were Hispanic American. The MST and usual services groups did not differ significantly on demographic variables or criminal history.

Measures: A multimethod, multifocus measurement battery was used to assess variables related to the ultimate and instrumental goals. Ultimate outcomes included decreases in criminal activity and incarceration. Instrumental goals included improved family relations, peer relations, and social competence and decreased symptomatology in youths and parents.

Archival records from the time of referral to the MST or control condition were evaluated for postreferral arrests and postreferral incarceration. In addition, the full-scale score of the Self-Report Delinquency scale (SRD) assessed youths' reports of criminal behavior during the previous 4 months. The Family Adaptability and Cohesion Evaluation scales (FACES) were used to assess parental and youth perceptions of family cohesion and adaptability. The Missouri Peer Relations Inventory (MPRI) evaluated parental and youth perceptions of the adolescent's friendships on the dimensions of emotional bonding, aggression, and social maturity. Adolescent symptomatology was assessed through maternal reports on the Revised Behavior Problem Checklist (RBPC). Parental symptomatology was assessed with the self-report Symptom Checklist-90-Revised (SCL-90-R) and adolescent social competence was assessed with the Social Competence scale of the Child Behavior Checklist (SCS-CBC).

Analysis: One-way analyses of variance (ANOVA) were used to evaluate between-groups differences for arrests and incarceration following referral to the project (n=84), and one way analyses of covariance were used to evaluate between groups differences on the SRD and the psychosocial measures at the posttreatment assessment, with the corresponding pretreatment score serving as the covariate (n=56).

Outcomes

Posttest: Shortly after treatment ended, self-reported offenses were lower for MST youth compared to youth receiving usual services (Mean=2.9 vs. 8.6). At 59 weeks postreferral, MST youths had more than half as many arrests (Mean=0.87 vs. 1.52) as youths receiving usual services (58% of MST youths experienced no rearrests vs. 38% of usual services group). MST youths also spent an average of 73 fewer days incarcerated in DYS facilities (Mean=5.8 vs. 16.2) than usual services youth (20% of MST youths vs. 68% of usual services youth experienced incarceration). Aggression with peers decreased significantly for MST youth, while remaining the same for youth receiving usual services. Families receiving MST also reported more cohesion, whereas family cohesion decreased in the usual services condition.

Long-term: A 2.4-year follow-up (Henggeler, Melton, Smith, Schoenwald, and Hanley, 1993) showed that MST doubled the percentage of youth not rearrested, in comparison with the usual services group.

Study 2

Summary

The study used a randomized controlled trial that assigned 47 juvenile offenders to an MST intervention group or a control group receiving usual youth services. Assessments at posttest measured substance abuse and delinquency.

The study found that, relative to the control group, the MST intervention group reported significantly:

  • Less soft drug use at posttest.

Evaluation Methodology

Design: A pretest-posttest control group design, with random assignment to conditions and follow-up for arrest and incarceration measures was used to compare the effectiveness of family preservation using MST versus the usual services (US) delivered by the Department of Youth Services (DYS). At the time of this study, assessment data had been obtained for 47 youths, 28 in the MST condition and 19 in the US condition. In the MST condition, 89% of the referred families participated in treatment.

Therapists with caseloads of 4 delivered the MST program to the treatment group. On average, the treatment duration was 13.4 weeks encompassing 33 hours of direct contact, with 24-hour-a-day case coverage. Depending on the stage of treatment, sessions were held as often as every day or as infrequently as once a week, usually in the family's home, and ranged in duration from 15 to 90 minutes.

Youths in the usual service condition met monthly with a probation officer who monitored their compliance with court ordered stipulations (i.e., curfew, school attendance, participation with other agencies). Although youths and families were often referred for mental health services, few substantive services were actually delivered.

Sample: The mean age of the sample was 15.1 years, 72% were male, 74% African American, and 26% were Caucasian. The MST and usual services groups did not differ significantly on demographic variables or criminal history.

Measures: Self-reported substance abuse was measured with the soft drug use and hard drug use subscales of the self-report delinquency scale in the National Youth Survey.

Analysis: One-way analyses of variance (ANOVA) were used to evaluate between-groups differences for self-reported drug use.

Outcomes

Post-test: Self-reported soft drug use was significantly lower at posttreatment for the youths in the MST condition than for the youths in the Usual Services condition. Differences between samples for hard drug use could not be tested due to a very low base rate (i.e., only three youths reported such use).

Study 3

Summary

The study used a randomized controlled trial that assigned 176 juvenile offenders ages 12-17 to an MST intervention group or an individual therapy control group. Assessments at posttest through a 25-year follow-up measured problem behavior, arrests, and family relations.

The study found that, relative to the control group, the MST intervention group showed significantly:

  • Decreased parent-reported youth behavior problems
  • Improved family relations
  • Fewer arrests (through 21.9 years from treatment end)
  • Fewer arrests of siblings.

Evaluation Methodology

Design: A pretest-posttest control group design, with random assignment to conditions and multiple follow-ups for arrests, was used to compare the effectiveness of MST with that of individual therapy (IT). Participants were 200 twelve- to seventeen- year old juvenile offenders and their families who were referred to the project by juvenile court personnel and agreed to participate in a pretreatment assessment session; five other families were referred but did not agree to participate. Twenty-four (12%) of the families refused service. The remaining 176 families were randomly assigned to receive either MST (n=92) or IT (n=84). Of these, 140 (79.5%) completed treatment, and 36 (21.5%) dropped out. Dropout rates were not significantly different for the treatment or control group. Statistical tests showed no between-group differences in the criminal histories or demographic characteristics for the IT dropouts or the MST treatment refusers. Pre and posttreatment assessments were available for 126 families. The average treatment for MST involved 23.9 hours and 28.6 hours for the IT completers. These means were significantly different.

Sample: Juvenile offenders and their families with (a) at least two arrests (b) currently living with at least one parent figure, and (c) showed no evidence of psychosis were included in the study. The juvenile offenders were involved in extensive criminal activity as evidenced by their average 4.2 previous arrests and the fact that 63% had been incarcerated. Their average age was 14.8 years; 67% were male; 70% White, and 30% African American; 65% were from families characterized by low socioeconomic class; and 53% lived with two parental figures.

Measures: Individual adjustment included measures of psychiatric symptomatology in mothers, fathers, and adolescents and a measure of adolescent behavior problems assessed through mothers' reports. Family relations were measured by parental and adolescent perceptions of family functioning and video-recorded observed family interactions. Maternal and teacher perceptions of peer relations were evaluated with the 13-item Missouri Peer Relations Inventory (MPRI). Juvenile court, local police, and state police records, collected an average of 3.95 years, were used to obtain data on post-probation arrests. Substance abuse was measured as an arrest for a substance-related offense.

Analysis: Repeated measures multivariate analyses of variance (MANOVAs) and ANOVAs were used to evaluate whether significant changes pre- to post assessment were experienced by the 70 MST youths and families or the 56 IT youths and families who completed pretreatment and posttreatment assessments. Survival analysis was employed to determine the proportion of participants not arrested in each group by the length of time from release from probation. Additional analyses examined the number of arrests and the seriousness of those arrests among recidivists in the MST and IT groups. Hierarchical multiple regression analysis was used to evaluate the effect of treatment on violent offending and to evaluate the effects of potential moderators (age, race, social class, gender, pretreatment arrests) of MST effectiveness.

Outcomes

Post-test: At posttest (within one week of treatment completion), MST resulted in decreased symptomatology in parents (self-reports), compared to an increase in the IT families (mothers) and no change (fathers); and decreased behavior problems in MST youth (parent reports), whereas mothers of IT youth reported an increase in behavior problems. MST families experienced favorable effects on perceived family relations (increased cohesion and adaptability) whereas cohesion and adaptability decreased in the IT condition. Observed family interactions among the MST families improved (increased supportiveness and decreased conflict-hostility across family dyads), compared to IT families in which dyadic relations deteriorated (mother-adolescent supportiveness), conflict-hostility increased (father-adolescent), or no change indicated (on measures of supportiveness and conflict-hostility).

4 Year Follow-up: By the end of 4 years, 26.1% of youth in the MST group had been arrested at least once, compared to 71.4% of the IT group. Specifically, MST completers had lower recidivism rates (n=77, 22.1%) than MST dropouts (n=15, 46.6%), IT completers (n=63, 71.4%), IT dropouts (n=21, 71.4%), and treatment refusers (n=24, 87.5%). Additionally, recidivists in the MST group had been arrested less often and for less serious crimes than IT youth. MST youth were less likely to be arrested for violent crimes (e.g., rape, attempted rape, sexual assault, aggravated assault, assault/battery) following treatment than IT youth. MST youth also had a significantly lower rate of substance-related arrests than IT youth (4% vs. 16%) (Henggeler et al., 1991). MST was shown to be equally effective with youths of different gender and ethnic backgrounds.

13.7 Years Post Treatment (Schaeffer and Borduin, 2005): This long-term follow-up sample consists of 87 MST and 78 IT (usual treatment) individuals. Participants were on average 28.8 years old. Both juvenile and adult substantiated (i.e., charges that were dismissed at trial were excluded) criminal records were obtained. Survival analysis to time of first arrest for various types of offenses was conducted. MST participants were at lower risk of rearrest (i.e., more likely to survive) during follow-up than were IT participants. By the end of 13.7 years, 81% of the IT group had been arrested at least once, compared with 50% of the MST group. With regard to specific offenses, MST participants were at lower risk of arrest for violent offenses, nonviolent offenses, and drug offenses during follow-up than IT participants. Additionally, MST participants had 54% fewer arrests, were sentenced to 61% fewer days of confinement in adult detention facilities and 37% fewer days of probation as adults than were comparison counterparts.

21.9-Year Follow-Up (Sawyer and Borduin, 2011): This study began with the same 176 subjects examined in earlier studies and, like the other follow-ups, it gathered data on arrests from Missouri official records. After 21.9 years from the end of the MST program, subjects had reached an average age of 37.3 years. A search of juvenile and adult criminal arrest records in the state identified arrests that occurred after the program and that led to conviction. Those without recorded arrests and with a Missouri driver's license were counted as having lived in the state without an arrest and conviction. Note that the records search was done for subjects who completed the program as well as for subjects who had dropped out.

The records search determined that 84.1% of the sample (n = 148) had lived in the state since the prior follow-up. The remaining 15.9% of the sample for whom residency could not be verified were considered lost to follow-up. Attrition rates did not differ significantly across the treatment and control groups, and there were no differences in the pretreatment criminal histories or demographic characteristics of participants included in the follow-up versus participants considered lost to follow-up. The authors concluded that the loss of subjects leaving the state and unavailable for follow-up did not bias the findings.

Outcome measures included arrests with convictions for felonies, violent felonies, nonviolent felonies, and misdemeanors (excluding traffic). The outcome measures also included civil suits involving family instability or financial problems. Since the study sampled subjects after an arrest, the new arrests indicated recidivism.

The results revealed clear long-term benefits of MST. The treatment group had significantly better outcomes for four of the six measures: felonies, violent felonies, nonviolent felonies, and civil suits involving family instability. For example, 54.8% of control participants versus 34.8% of MST participants had been rearrested at least once for a felony offense by the end of the 21.9-year follow-up period. The odds ratio of 2.27 for control subjects relative to intervention participants indicates a medium-sized effect. The largest group difference occurred for violent felonies: The odds ratio of 4.08 for arrest for a violent felony translates into a large effect.

Survival analyses, which model the time to arrest or censoring rather than just the occurrence of an arrest, confirmed the descriptive results. The hazard ratio for all felonies of .616 suggests a weak to medium effect of the program. Further, zero-inflated Poisson regression models examined program effects on the number of rearrests and length of sentences. The results proved less strong for these measures: The program significantly reduced only the number of arrests for misdemeanors. Tests for moderation by background factors (age, socioeconomic status, pretreatment arrests) in the zero-inflated Poisson regression showed that the results proved similar across subjects from divergent backgrounds.

Columbia, Missouri, 25 Year Sibling Follow-Up (Wagner et al., 2014):

To examine the general effects of MST beyond the youth participants, the 25-year follow-up measured arrest rates and incarceration times for the closest sibling of those targeted in the original MST study.

Evaluation Methodology

Design: Of the 176 randomized juveniles included in the original study, 129 had siblings in the home during the period of intervention. These 129 closest-in-age siblings, rather than the original sample, served as subjects in the 25-year follow-up.

Assignment: The original subjects, but not the sibling subjects, were randomized to the treatment and control groups. Among the 129 siblings participating, 67 participants (51.9%) belonged to the treatment group and 62 participants (48.1%) to the control group.

Attrition: The study obtained publicly available Missouri court records of the closest sibling to the original participants in the study. Of the 129 available participants, 19 (14.73%) appeared to have left the state, lacked data on court records, and were designated as lost to follow-up.

Sample: Of the selected participants, 60% were the younger sibling of the MST participant, while 40% were older siblings. Half of the siblings included in the study are male. The majority of participants are white (86.4%).

Measures: The study used publicly available adult (age 17+) criminal records. Researchers coded crime classification (felony or misdemeanor) and date of arrest. In addition, the study used the number of days sentenced to incarceration or probation. The inability to obtain data for states other than Missouri might limit the measures, as might the inability to obtain data on juvenile offenses.

Analysis: Based on the publicly available criminal records, the study analyzed the relative odds of arrest of control and treatment group participants. In addition, the study reported the cumulative survival function for arrests. Finally, the number of arrests and years sentenced were analyzed using a Zero Inflated Poisson Regression, which evaluates the impact of treatment condition on the number of posttreatment arrests and years sentenced to incarceration or probation.

Intent-to-Treat: The analysis excluded the 19 individuals who were lost to follow-up because of residence outside the state but otherwise used all available data.

Outcomes:

Baseline Equivalence: A comparison of the treatment and control siblings for the analytic sample of 110 subjects showed no significant differences on demographic variables, but the study lacked baseline measures for other variables.

Differential Attrition: No baseline differences on condition or demographic variables were reported between those located and those lost to follow-up, but the study lacked baseline measures for other variables.

Long-Term:

  • Siblings in the control group were significantly more likely to have been arrested at least once as compared to siblings in the treatment group (OR = 3.36).
  • A log-rank test of the survival functions for the treatment and control group showed that the treatment group was at a significantly lower risk of arrest (p=.004) and misdemeanor arrest (p=.02) during the follow-up period than the control group.
  • The Zero Inflated Poisson Regression analysis included the odds estimate for any arrests and rate estimates for number of arrests. The treatment significantly reduced odds estimate of arrest for any crime, any felony, and any misdemeanor but did not influence the number of arrests.
  • The Zero Inflated Poisson Regression also examined adult sentencing, finding that the treatment significantly reduced the odds of incarceration and probation and the years of the sentence for incarceration and probation.

Study 4

Summary

The study used a randomized controlled trial that assigned 155 juvenile offenders to an MST intervention group or a usual-services control group. Assessments at posttest and 1.7-year follow-up measured re-offending and family relations.

The study found that, relative to the control group, the MST intervention group showed significantly:

  • Better emotional-behavioral functioning
  • Fewer days incarcerated.

Evaluation Methodology

Design: This study followed a 2 x 2 x 2 Condition (MST vs. usual services [US] x Time (pretest vs. posttest) x Site (Site 1 [S1] vs Site 2 [S2]) mixed factorial design, with random assignment. Eighty-two families were randomly assigned to MST treatment conditions and 73 to usual services. Families from each group were paired to further control for historical and related threats to validity. Nine of the families could not be paired. Between pretest and posttest, 9.7% (n=15) of the families dropped out of the study. Mothers from the dropout group were slightly better educated than mothers from the completers. Two sites, covering a three county area, were chosen to achieve representation of racial group and urban and rural settings. S1 encompassed urban and rural areas and had a majority Caucasian population, while S2 was mostly rural and majority African American.

The average treatment for the MST groups involved 122.6 days for S1 and 116.6 for S2. Youths in the US condition were placed on probation for a minimum of 6 months and typically seen by his or her probation officer at least once per month. School attendance was monitored and referrals were made to other social service agencies for help in particular problem areas (e.g., alcohol and drug abuse programming, vocational counseling or training).

Sample: To be included in the study, adolescents had to (a) be between 11 and 17 years (b) have committed a serious criminal offense or have at least three prior criminal offenses other than status offenses, and (c) be at imminent risk of being placed outside the home because of serious criminal involvement. The juvenile offenders averaged 3.07 previous arrests and 59% had been incarcerated at least once. At the time of referral their average age was 15.22 years; 81.9% were male; 19.4% were White, and 80.6% were African American; approximately 50% were from two parent households and the median family income was between $5,000 and $10,000 per year.

Measures: Individual emotional adjustment and adolescent behavior problems, criminal activity, family relations, parental monitoring, peer relations, and MST treatment adherence were all measured in this study. Primary caregiver and adolescent psychological distress were assessed by the Global Severity Index. Adolescent behavior was measured through caregiver reports. A self-report delinquency scale was utilized to assess criminal activity during the previous 3 months. Arrest and incarceration histories were collected approximately 1.7 years from the end of the project and included offense, arrest, adjudication, and incarceration histories. Assessments of family functioning, parental monitoring, and peer relations were provided by reports from the primary caregiver and the adolescent. The MST Adherence Measure was completed by the parents, adolescents, and therapists after randomly selected therapy sessions.

Analysis: A series of 2 x 2 x 2 Time (pretreatment vs. posttreatment) x Treatment Condition (MST vs. US) x Site (S1 vs. S2) ANOVAs were conducted on the psychosocial measures collected during the pretreatment and posttreatment assessments. In addition, 2 x 2 (Treatment Condition x Site) ANOVAs were used to evaluate longer term outcomes for rearrest and incarceration through the 1.7 year follow-up. Hierarchical multiple regression analyses were conducted to test whether outcomes in the MST condition were associated with treatment adherence.

Outcomes

Youths in the MST condition reported substantially reduced psychiatric symptomatology, whereas their US counterparts reported slightly increased symptomatology. There were no significant treatment effects for youth reports on the self-reported delinquency assessment. The annualized rate of rearrest and the average seriousness of rearrests did not differ significantly between groups. On the basis of archival incarceration records, the annualized rate of days incarcerated was 47% lower for youths in the MST condition (33.2 days per year per youth) than their US counterparts (70.4 days per year per youth). No treatment effects were observed on the family relations measures or for the peer relations measure.

Results in this study show that MST program fidelity is significantly associated with many of the outcome measures. High adherence based on parent, adolescent, and therapist reports predicted favorable outcomes, and low adherence predicted poor outcomes on the following measures: adolescent symptomatology, rates of parental and emotional distress, adolescent self-reports of index offenses, rates of rearrest, and rates of incarceration.

Mediation: (Huey, et al., 2000): Across two independent samples, the study above and the Charleston County, SC, Drug Study (Study 6 in this writeup), results supported a family-centered mediation model. The model results demonstrated that MST improved family relations (i.e., quality of family functioning, family cohesion, and parent monitoring), and improved family relations in turn predicted decreased delinquent peer affiliation and subsequent delinquent behavior.

Study 5

Summary

The study used a quasi-experimental design with matching that examined 155 youths who were non-randomly assigned to three groups: an MST intervention group for juvenile offenders, an alternate family intervention for juvenile offenders, or a usual-services control group of non-offenders. Assessments at posttest measured conduct problems, anxiety, and aggression.

The study found that, relative to the comparison group, the MST intervention group showed significant:

  • Decreases in behavior problems at posttest
  • Improvements in family relations at posttest.

Evaluation Methodology

Design: A 3 X 2 mixed factorial design was used with three groups of adolescents and their families (family-ecological treatment, alternative treatment families, and normal controls). Offenders were not randomly assigned to treatment groups, but the groups were matched on demographic variables and pretreatment arrest histories. One hundred sixteen families of juvenile offenders were referred to the family-ecological treatment between June 1978 and June 1982. Alternative treatment families were the delinquent control group and consisted of 40 juvenile offenders and their families, who were referred to other mental health agencies for services. To control for developmental maturation and to provide a frame of reference, 50 nonpathological adolescents and their families who matched the demographic characteristics of the treatment families were recruited from local high schools for participation in the study. Pre- and post-treatment (3 weeks after treatment terminated) data was gathered for 57 treatment families, 23 alternate treatment families, and for 44 control families. The style and quantity of the family-ecological treatment was based on the family's needs and varied widely (from 2 to 47 hours, M=20 hours). Alternative treatment hours averaged 24 hours of intervention over an approximately 3-month period.

Sample: Specific characteristics of the sample, beyond the information listed above, were not provided in this study.

Measures: Child psychopathology was measured through youths self-reports, family member's self-reports of their perceptions of family relations, and observational measures of family relations based on the audio-recording of members' discussions.

Analysis: To determine an appraisal of any differential changes experienced by the three groups during the treatment period, a test of significance was used to show whether there was a significant multivariate effect for the set of dependent measures. If a significant effect was found, a 3 x 2 mixed factorial ANOVA was performed on each measure in the set. If the ANOVA revealed a significant interaction effect, the Scheffe test was used to evaluate the pre-post changes for each group on that measure.

Outcomes

Results showed that adolescents in the family-ecological condition had a decrease in behavior problems on each of the subscales: conduct problem, anxiety-withdrawal, immaturity, and socialized aggression. Alternative treatment and normal control adolescents showed no change. No significant multivariate interaction effects emerged in self-reported family relations. However, observational ratings indicated significant positive change among the family-ecological group (e.g., the mother-adolescent and marital dyads were warmer and more affectionate, and the adolescent was more actively involved in family discussions after treatment). Such changes were not observed for the alternate treatment group, and some family relations had deteriorated in this group (the marital relationship and the father-adolescent relationship in these families showed decreased warmth and affection following treatment).

Study 6

Summary

The study used a randomized controlled trial that assigned 118 youths with diagnosed substance abuse to an MST intervention group or a usual-services control group receiving outpatient substance use treatment. Assessments from posttest to a four-year follow-up measured drug use, criminal activity, and out-of-home placement.

The study found that, relative to the control group, the MST intervention group showed significantly:

  • Lower substance use at posttest but not six-month follow-up
  • Better school attendance at six months
  • Fewer aggressive crimes at four years.

Evaluation Methodology

Design: The study followed a 2 (treatment type: MST vs. usual services) x 3 (time: pretreatment [T1], posttreatment [T2], and 6-month follow-up [T3]) mixed factorial design, with random assignment of participating families to the treatment condition (N=58) and usual services control condition (N=60). Out of 140 screened adolescents, 84% (N=118) agreed to participate in the study. Fifty-seven out of the 58 (98%) of the families assigned to the MST condition completed a full course of treatment, while 47 (78%) of the families assigned to the usual services condition completed treatment.

The amount of MST therapeutic services each family received was based on clinical need. Families in the MST condition received services for an average of 130 days, with an average of 40 hours direct contact. Youths in the comparison condition were referred by their probation officer to receive outpatient substance abuse services from the local office of the state substance abuse commission. Youths in this condition received few substance abuse or mental health services during the first 5 months following recruitment into the project. In fact, 78% of these families received no treatment.

An examination of the baseline comparability between youths and families in the treatment and control groups revealed youths in the MST condition reported higher rates of drug use prior to treatment.

Sample: Participants were 118 twelve to seventeen year-old adolescents recruited from the Department of Juvenile Justice in Charleston County, South Carolina. To be included in the study, youth met diagnostic criteria for substance abuse or dependence, had formal or informal probationary status, and lived with at least one parent figure. The average age of the sample was 15.7 years at the time of referral with 79% male, 50% African American, 47% Caucasian, 1% Asian, 1% Hispanic American, and 1% Native American. Based on socioeconomic measures, the sample is relatively disadvantaged.

Measures: A multimethod (self-report, parent report, biological, and archival) strategy was used to examine the following three outcomes: drug use, criminal activity, and out-of-home placement. Adolescent drug use was assessed through adolescent self-reports and urine toxicology screenings. Adolescent criminal activity was measured through youth self-reports on the Self-Report Delinquency Scale (SRD) and computerized arrest records. Days in out-of-home placements were documented through the monthly service utilization survey. MST program adherence was assessed using the 26-item MST Adherence Measure, which was administered to the primary caregivers and youths in the MST condition and completed by the project's three therapists following randomly selected therapy sessions. Three measures were used to obtain school attendance data: a school data form (SDF), the Child Behavior Checklist, and the monthly service utilization survey completed by the MST therapist (Brown, Henggeler, Schoenwald, Brondino, and Pickrel, 1999).

Analysis: To determine the effects of self-report drug use and delinquency, comparisons were conducted contrasting the behavior between the MST and usual services groups over the three time intervals. Analyses were conducted within the context of 2 (treatment condition) x 3 (time: T1, T2, and T3) mixed model analyses of variance (ANOVA), with one between- and one within-subjects factor. Due to the between group differences between reported drug use at baseline, analyses of covariance were also conducted on the drug use measures. One-way ANOVAs were conducted to measure arrests, out-of-home placements, and the rates of positive drug testing during treatment. Regression analyses were conducted to test whether adherence scores were associated with key outcomes. A 2 (condition: MST vs. US) x 3 (Time: pretreatment, posttreatment, 6-month follow-up) repeated measures ANOVA was conducted to examine changes in school participation across the treatment conditions over time.

Outcomes

Post-test: There were no significant treatment effects on the drug use measures (using ANCOVA), self-reported criminal activity, or arrest records. Although more youth in the MST group were incarcerated (19), compared with 16 youth in the US group, the MST youths were incarcerated for substantially shorter durations (569 days vs. 1051 days). MST fidelity outcomes showed modest results: decreased drug use at T2 but not at T3; a 50% decrease in days in out-of-home placement; and no effects for self-reported offending or the urine screens.

Comparisons of MST adherence across three studies, showed the current study on adolescent drug abuse were the worst (e.g., lowest adherence and highest non-productive sessions). Overall, adherence scores were marginally associated, in the expected direction, with criminal activity and out-of-home placement. The findings are contradictory in the observed associations between treatment adherence and adolescent drug use. Some results demonstrate higher fidelity is related to improved youth outcomes (e.g., adolescent reports of high treatment adherence were associated with reduced probability of using drugs other than alcohol and marijuana at T3). On the other hand, several findings showed the opposite effects (e.g., caregiver ratings of high adherence were associated with increased adolescent alcohol and marijuana use at T2).

MST effects were moderated in two instances. For females, MST was effective in decreasing alcohol and marijuana use from T1 to T2 in comparison to the US group, but between T2 and T3 female alcohol and marijuana use significantly increased for females in the MST condition, whereas counterparts in the US condition improved. A similar finding was revealed for age. For younger adolescents (age 15 and below), MST was effective in decreasing alcohol and marijuana use from T1 to T2 in comparison to the US group who increased, but between T2 and T3 younger adolescents in the MST group significantly increased their alcohol and marijuana use, whereas counterparts in the US condition decreased their use.

Participants in the MST condition showed a sustained increase in the percentage of adolescents in school through the 6-month follow-up. In contrast, the percentage of adolescents in school in the US condition decreased at posttreatment but then increased at the 6-month follow-up (Brown, Henggeler, Schoenwald, Brondino, and Pickrel, 1999).

Notes: Previous studies support the potential viability of MST for substance abusing adolescents. The modest results in this study may be due to the difficulty in transporting MST from the program developers to the supervisors and therapists implementing the program. Although the findings were not entirely consistent, high fidelity in this study tended to be associated with improved clinical outcomes.

Cost or Cost-Benefit: (Schoenwald, Ward, Henggeler, Pickrel, and Patel, 1996)

A study was completed to determine the monetary costs of MST during the 11-month period from referral into the program to the six-month follow-up. The total cost of services in the US condition during the 11-month period was $198,729. The total cost of seeing youth in the MST condition was $298,724 (this includes MST program costs + costs of mental health and substance abuse services). Costs per youth were $5,063 for youth in the MST condition and $3,369 for youth in the US condition. However, based on the clinical outcomes in this study, a 46% reduction in incarceration days and a 64% reduction in hospitalization/residential treatment, the costs of implementing the MST program are nearly offset by the savings in incarceration costs. Since the average incarceration cost is $100/day, the decrease in days incarcerated resulted in a $48,200 savings. In addition, the cost of MST compares favorably to the estimated lifetime costs of untreated delinquency and drug abuse which range from $309,000 to $1 million.

Four-Year Follow-up of Drug Outcomes - Charleston County, South Carolina (Henggeler, Clingempeel, Brondino, and Pickrel, 2002)

Evaluation Methodology

Design: This analysis examines the 4-year outcomes from the randomized clinical trial of MST with 118 juvenile offenders meeting formal diagnostic criteria for substance abuse or dependence. Eighty (43 MST and 37 usual services) of the 118 adolescents who participated in the original study completed the follow-up assessment. Between-group differences were determined at time 5 (T5) by comparing mean scores or percentages of the participants in the MST and usual services conditions on each of the dependent measures. Attrition analyses showed that study dropouts and completers did not differ significantly on the measures assessed. Comparisons between the MST and usual services participants at baseline showed two significant differences. MST participants were older and reported more frequent use of marijuana.

Sample: The average age of the participants at follow-up was 19.6 years; 76% male, 60% African American, and 40% were Caucasian. Approximately 48% had not obtained a high-school education or GED and 12% had completed some college or technical school beyond high school.

Measures: Adult criminal activity was measured through youth self-reports on the Self-Report Delinquency Scale (SRD) and computerized arrest records. Drug use was assessed through self-report and urine and head hair samples. Psychiatric symptoms were measured by the Externalizing and Internalizing scales of the Young Adult Self-Report (YAS).

Analysis: Multivariate analyses of covariance (MANCOVAs) were conducted on measures of (1) aggressive criminal behavior, (2) property crimes, (3) self-reported illicit drug use, (4) psychiatric symptoms to determine the differences between the MST and usual services groups at T5. Chi-square analyses were conducted on the biological drug use indicators (i.e., no positive test versus any positive test).

Outcomes

MST participants showed a 75% reduction in convictions for aggressive crimes since the age of 17 years and reported committing significantly fewer aggressive crimes during the past 12 months, compared with the usual services group. There were no significant differences in self-reported drug use between the two groups. However, the biological measures showed young adults in the MST condition had significantly higher rates of marijuana abstinence (55% versus 28%) than did their usual services counterparts. There were no significant differences in psychiatric symptoms between the two groups.

Moderator analyses showed that the impact of treatment did not vary as a function of demographic characteristics, comorbid psychopathology, or initial (T1) levels of drug use and criminal behavior.

Study 7

Summary

The study used a randomized controlled trial that assigned 100 Norwegian youths with serious antisocial behavior to an MST intervention group or a control group receiving child welfare services. Assessments at posttest and two-year follow-up measured problem behavior, delinquency, and out-of-home placement.

The study found that the MST intervention group, relative to the usual-services control group, showed significantly:

  • Decreased youth internalizing symptoms at posttest
  • Increased social competence at posttest
  • Decreased out-of-home placements at posttest
  • Lower delinquency at the two-year follow-up
  • Fewer behavior problems, internalizing, and externalizing at the two-year follow-up.

Evaluation Methodology

Design: This study followed a 2 (treatment type: MST vs. Child Welfare Services (CS)) x 2 (time: pretreatment, post-treatment) x 4 (county municipality) mixed factorial design, with random assignment to treatment conditions. One hundred adolescents and their families from four counties in Norway, referred to treatment for antisocial behavior, participated in this study. A weighted randomization procedure was utilized to assign families to the treatment or control condition, with each family having a 6/10 chance of receiving MST and a 4/10 chance of receiving the usual CS services. This resulted in 62 families being assigned to MST and 38 families to CS. Assessments were conducted at study entry and after termination of MST treatment (approximately 6 months after intake). Comparison of the two groups at baseline showed CS caregivers were more likely than MST caregivers to be divorced, and MST caregivers were more likely to be married to someone other than the child's biological parent. These differences had no moderating effect on the outcomes.

Four families dropped out of the MST program early in the treatment, but were replaced with new families. In addition, 4 families withdrew from the study prior to the post-treatment assessment, resulting in a 96% retention rate. Analysis of attrition was not possible because the government-approved informed consent allowed dropouts to have their data expunged from research records.

MST for youths with serious antisocial behavior was implemented as detailed in the MST treatment manual with no major modifications to the original intervention model. Treatment options for the CS group included foster care and institutional placement. If out-of-home placement was not warranted, less intensive services such as home-based treatment or social work was provided. In this study, 14 youths received long-term institutional placement, 5 were placed in a crisis institution for assessment, 6 were supervised by a social worker in their homes, 7 were given other home-based treatment, and 6 refused the services offered.

Sample: To be included in the study, adolescents had to (a) be between 12 and 17 years, (b) exhibit serious problem behavior, and (c) have parents that were sufficiently involved and motivated to start MST. The sample consisted of 63 boys and 37 girls, who averaged 14.95 years of age. Ninety-five percent of the caregivers had a Norwegian background. Thirty-nine percent of the sample had been previously placed out of the home, 54% had a history of running away from home, 30% had been suspended from school, and 90% had a history of school truancy.

Measures: Because Norway's criminal justice system does not make arrests and convictions for youth under 15 years, and offenders under the age of 18 are usually sent to Child Welfare Services rather than prosecuted, archival arrest data is not available for this study. Instead, behaviors for the main outcomes were assessed using multiple informants (parents, teachers, and youths). Youth symptomatology and social competence was measured by caregiver, adolescent, and teacher ratings on the Child Behavior Checklist (CBCL). The Self-Report Delinquency Scale (SRD) was utilized to measure delinquent behavior. Social competence with peers was assessed from adolescent, caregiver, and teacher reports on the Social Competence with Peers Questionnaire (SCPQ). The Social Skills Ratings System (SSRS) was utilized to measure a broad array of social skills. The Family Adaptability and Cohesion Evaluation Scales-III (FACES-III) was completed by both caregivers and adolescents. This instrument measured two key family constructs: cohesion and adaptability. This study also provided assessed out-of-home placement and family satisfaction with the services they received.

Analysis: ANOVAs and chi-square analyses were conducted to evaluate between and within group differences on the measures collected during the pretreatment and post-treatment assessments.

Outcomes

Based on combined caregiver, youth, and teacher reports, youths in the MST condition demonstrated a significant decrease in internalizing behavior and a marginally significant decrease in externalizing behavior (p=.07) at post-treatment, compared with youths in the CS condition. Likewise, MST youths demonstrated a significant increase in social competence, compared to CS youths. There were no significant between group differences on family functioning measures. However, family cohesion increased significantly over time within the treatment condition. MST youths were maintained in the home significantly more often than CS youths. Of those at home during the pretreatment assessment (n=84), 90.6% of MST youths were also at home during the post-treatment assessment, compared with 58.1% of CS youth.

Follow-up Two Years After Intake (Ogden and Hagen, 2006)

The follow-up sample consisted of 75 participants. One of the four sites was dropped from the analysis because of poor fidelity, thus this is not an intent to treat analysis. The specific goals of the long-term follow-up were to investigate whether MST was successful in preventing out of home placement and examine reductions in behavior problems, such as delinquency. MST participants were less likely to have been placed out of the home than were their RS (received regular child welfare services) counterparts. This was especially true for boys and older MST participants. At the two-year follow-up, MST youths self-reported less delinquency than RS youths, parents reported fewer behavioral problems on the CBCL total problem scale and less internalizing, and teachers reported fewer behavior problems, less internalizing and less externalizing problems.

Non-significant differences were reported by youth on the total problem scale of the YSR and the YSR Externalizing and Internalizing scales, as well as the parent reports on the CBCL Externalizing.

Quasi-Experimental Comparisons with New Intervention Group (Ogden et al., 2007)

This study used 50 participants from two of the four sites in the original year 1 sample. It also examined 55 new participants at the two sites who were all assigned to the MST condition during year 2 of the program implementation. The design compared the non-randomized MST participants in year 2 to the randomized MST and control participants in year 1. The study lacked randomization in year 2 and made comparisons across two different years. The results demonstrated sustained effectiveness of the program over time. Compared to the control participants in year 1, the intervention participants in year 2 reported significantly better outcomes for preventing out-of-home placement, youth-reported delinquency and internalizing, and teacher-reported internalizing and externalizing. Further, comparisons of intervention participants showed similar improvements in year 2 as in year 1.

Notes: The community services available to the control group, to which the MST group was compared, are much more comprehensive and treatment-oriented than the services typically available to juvenile offenders in previous MST research. The more modest treatment effects obtained in this study, compared to other evaluations, may be attributable to the nature of the usual services condition.

Gender differences in MST outcomes (Ogden and Hagen, 2009)

This analysis uses students from the previous program evaluation to look at gender differences in treatment outcomes.

Sample: This study used data from 2 previous studies in Norway. The first was an RCT in which 100 adolescents were randomly assigned to either MST (n=62) or regular services (n=38). An additional group of 55 non-randomized adolescents received MST and participated in another study during the second year of program operation and were added for the purpose of studying the sustainability of program effectiveness. The current study included the 117 youth who received MST treatment in the program's first and second year of operation. Of these, 41 were girls and 76 were boys (65%). Adolescents ranged in age from 12-17 years. While somewhat fewer of the girls' caregivers were married or cohabiting (54%) compared to boys' caregivers (62%), the difference was not statistically significant. There were no significant differences on girls' and boys' caregivers' mean age (41.26 years) and family income. At intake to treatment, 91.3% of the adolescents had been living at home with their caregivers for the last 6 months. No gender differences appeared regarding living situation at intake.

Measures: Adolescents' behavior problems were assessed with multi-informant ratings from caregivers, teachers and adolescents using the CBCL, the Teacher's Report Form (TRF) and the Youth Self-Report (YSR). Youth alcohol and drug use was measured by the Personal Experience Inventory, which was completed by the youth themselves, as was the self-report delinquency scale. A subscale from the Family Adaptability and Cohesion Evaluation Scale was used to assess family cohesion. Caregivers' satisfaction with the MST treatment was measured using the Family Satisfaction Survey. Finally, the youth's place of living was reported by both parents and intake teams at intake and again at treatment termination.

Analysis: To test for gender differences at intake, chi-square tests and one-way ANOVAS were run. Four MANOVA models were run to test for gender differences in behavioral improvement following treatment.

Outcomes
At intake, girls and boys looked far more similar than different, although some differences were apparent at posttest. Girls were rated by their parents as presenting fewer externalizing problems than boys at post-treatment, after controlling for their wave 1 scores. This suggested that girls improved more than did boys on this measure. There were no gender differences on parent reports of internalizing or attention problems. Results revealed that girls rated themselves as having significantly more internalizing problems than did boys at the post-assessment, after controlling for their pretreatment scores. Boys, however, scored significantly higher on the self-report delinquency scale than did girls. This suggests that girls improved more regarding self-report delinquency, whereas boys reported greater reductions in self-assessed internalizing problems. No gender differences were found in terms of changes following treatment on the externalizing and attention problem scales of the YSR, on self-reported drug use or out of home placement. According to teachers, boys and girls showed similar levels of behavioral change following treatment.

Overall, although girls present a problem profile different than boys, and risk factors are somewhat different, MST is effective regardless of gender.

Study 8

Summary

The study used a randomized controlled trial that assigned 411 Canadian youths at high risk for criminal offending to an MST intervention group or a usual-services control group. Assessments from posttest to a three-year follow-up measured family relations and criminal convictions.

The study found that, relative to the control group, the MST intervention group was significantly:

  • Less likely to be sentenced to a term of secure custody
  • Improved on parent reports of family adaptability
  • Improved on youth externalizing behavior and internalizing symptoms.

Evaluation Methodology

Design: This study employed an experimental design whereby each of the 411 referred youth from four Canadian communities were randomly assigned to either the MST group or the usual services group, which functioned as a control group. Two categories of criteria were used to determine eligibility for MST. The first assessed the appropriateness of a family preservation intervention for the youth and the second verified that the presenting issues of the youth were among those for which MST has been empirically validated.

Pretesting was administered once the family gave consent and before random assignment was made. Data was collected from youth, parents, and teachers. However for the teacher information there was a low response rate of 57% at intake and 35% at discharge. While the response rate was the same for the two groups, there were also some significant differences between the aggregate scores of the MST and the control groups. Among the 57% of cases for which the teacher information was available, the members of the control group had higher levels of externalizing behavior problems at pretest. The instruments from the intake battery were readministered when an MST case was closed. If the MST case closed prematurely, post-testing was not re-administered. The members of the control group were contacted five months after intake and asked to complete the testing again.

In total, post-testing for the youth is available in only 62% of the cases and was more likely to be available for the MST group. There was no post-testing available for either a parent or a youth in 49% of control cases and for 28% of the MST group. The members of the MST group had a significantly higher response rate at discharge (71%), compared with the members of the usual services group (52%). Nineteen percent of the MST group dropped out prior to concluding the treatment, however, dropouts did not differ significantly from those who stayed on any of the intake testing. The authors' report no response bias could be identified. Analyses at post-test comparing MST dropouts and MST completers show that MST drop outs performed poorly compared with both the MST completers and the usual services group. The researchers were unable to identify the usual services drop outs.

All youths were tracked for three years to gauge offending and levels of correctional service utilization at 6, 12, 24, and 36 months after the case was closed. The case was closed for the MST group at the last session with the family. For the control group, the case was considered closed six months after the family signed the consent to participate in the study. At this report 380 youths were tracked at least six month post-discharge, 323 were reached one-year post discharge, 192 were two years post-discharge, and 82 reached three years.

Sample: To be included in the study, youth had had to be at high risk for committing a criminal offense and meet the eligibility criteria. One of the sites included youth under the age of 12 (27 youth). The average age was calculated for each of the four sites and ranged from 13.9 to 15.3, with an overall average of 14.6 years. Twenty-six percent of the sample was female. Thirteen percent self-identified as aboriginal. Family SES varied in the sample, about one-third were welfare dependent with poor educational achievement and low SES. Also, about 30% of the sample was middle class families with good educations and high incomes.

In all, 64% of the youth were referred to the MST project by probation officers. Although there had to be evidence of past criminal behavior to qualify for MST, about one third had no record of prior criminal convictions at referral. Thirty percent had been sentenced to at least one sentenced custody stay prior to referral. On average, youth with prior convictions had served 47 days in sentenced custody.

A clinical profile was drawn from psychometric testing completed by the youth, caregivers, and teachers. According to parent ratings, 84% of youth were over the clinical cutoff for conduct problems and half were over the cutoff for depression. Youth self-reports were only slightly lower, with 61% placing themselves over the cutoff for conduct and 48% for depression. One third of parents placed themselves over the cutoff for caregiver depression and poor family functioning. Teachers rated the youths as low on academic competence and social skills, placing almost all of them at or below the tenth percentile.

Measures: The following instruments were used as baseline information and to assess the impact of MST on the youth: (1) Standard Client Information System (SCIS); (2) Beliefs and Attitudes Scale; (3) Family Adaptability and Cohesion Scale - II (FACES) - this is a 30-item scale that measures family adaptability (negotiation style, roles, assertiveness, leadership, discipline, child control, rules) and family cohesion (emotional bonding, coalitions, space, family boundaries, shared time/friends, decision-making, and shared activities); (4) Social Skills Rating System; (5) Parental Supervision Index.

Follow-up data on recidivism and correctional service utilization are collected using a "CR" check of the Canadian Police Information Centre (CPIC). Records are checked at six months and after one, two, and three years post-discharge.

Analysis: The analyses included a comparison of group means and a survival curve analyses at four follow-up time periods for rates of conviction.

Outcomes

Post-test: There were no significant differences between groups in rates of convictions when the means are compared. Likewise, the survival curve did not demonstrate significant group differences regarding convictions. There was no difference in the rate at which the two groups were sentenced to custody during the follow-up or in the length of time before being convicted after being in treatment. Members of the MST group were significantly more likely to be sentenced to a term of open custody and significantly less likely to be sentenced to a term of secure custody, compared to the usual services group. This pattern was observed in three sites. There were no significant differences between groups for the length of time youths spent in custody or for the total number of offenses for which youth were prosecuted.

MST youth improved significantly on some problems as measured by psychological testing. Compared to the control group, the MST youth improved significantly on parent reports of family adaptability, caregiver depression, and youths' externalizing behavior. The MST group also improved significantly on youth report of internalizing symptoms when compared to the control group. While there were no significant differences between groups regarding parental supervision, these data need to be interpreted with caution because there were significant group differences at intake and because pre/post data are only available for only half the sample.

Notes: The Canadian MST program was implemented with a lower budget and less supervision by MST Inc. than previous MST studies. This may have undermined the fidelity of program implementation. Another possible factor that could account for the lack of results in the Canadian study, compared with studies in the U.S., is that the usual services that the control group received were of higher quality than the services youth receive in the U.S.

The results reported here are from an interim report and does not represent the final report.

Study 9

Summary

The study used a randomized controlled trial that assigned 164 youths enrolled in behavior intervention classrooms to an MST intervention group or a no additional treatment control group. Assessments from posttest to a one-year follow-up measured conduct problems, externalizing, and criminal offending.

The study found that, relative to the control group, the MST intervention group showed significantly lower:

  • Parent- and adolescent-reported externalizing problems
  • Absenteeism in school.

Evaluation Methodology

Design:

Recruitment: The authors contacted public junior and high schools to recruit adolescents between the ages of 11 and 18 who were enrolled in self-contained behavior intervention classrooms that served as restrictive settings designed for adolescents with conduct problems. Of 213 families with children in classrooms, 164 (77%) provided consent to participate in the study. The sample scored highly on baseline measures of externalizing and internalizing.

Assignment: After baseline measures were collected, 84 adolescents were randomly assigned to the treatment group and 80 were randomly assigned to a treatment-as-usual control group that did not receive the treatment.

Assessments/Attrition: Outcomes were assessed at baseline, three months (mid-program), six months (posttest), and 18 months (one-year follow-up). By the one-year follow-up, 7% of the families withdrew from the study or could not be located.

Sample:

Participants were enrolled in the 7th through 11th grades at baseline, with a mean age of 14.6 years. The vast majority of adolescents (83%) were male, 60% were black, and 40% were white. Parents or guardians averaged 40.8 years of age, 71% had at least completed high school, and the median reported family income was $17,500.

Measures:

Weiss et al. (2013): All measures were obtained from separate interviews for parents and adolescents at a location of their choosing, while teachers were given the assessment materials to complete on their own time. Parents both participated in the program to help their child and provided many outcome measures. However, an additional analysis found that adjusting for a measure of positive impression among parents did not change the results.

The primary outcomes were adolescent conduct problems as assessed by parent, adolescent, and teacher reports on the Child Behavior Checklist (CBCL) and criminal charges obtained from court records. The CBCL is used to obtain a broad measure of child behavioral and emotional problems, and its 118 items produce separate scales for externalizing and internalizing problems. The authors report that these scales have average 1-week test-retest reliability of .89. Criminal records were obtained with parental and child assent from juvenile court records from 1 year prior to baseline through 2.5 years post-baseline. Charges were coded as status offenses, misdemeanors, or felonies, with the analysis focusing on incidents wherein the adolescent was charged with a felony.

Secondary outcomes included several areas of conduct problems such as delinquency and drug use that were measured using the Self Report Delinquency Scale (validity and reliability were not reported). School functioning data was also included, and consisted of the student's average grade across all core academic subjects, school attendance, and the number of days suspended during the study period.

The authors also detailed the measurement of nine factors related to the study's risk and protective factors. To assess family relationships, parents and youth completed the Family Adaptability and Cohesion Evaluation Scales III; to assess authoritarian, authoritative, and permissive parenting behavior, the adolescent's primary caregiver completed the Parental Authority Questionnaire; and to assess parent mental health problems, parents completed the measures of externalizing and internalizing from the Personality Assessment Inventory.

Weiss et al. (2015): Three externalizing scales from the parent-reported Child Behavior Checklist, the Youth Self-Report form, and the Teacher Report Form served as the outcomes (alpha values were .89, .85, and .93, respectively).

Analysis:

The analysis used hierarchical linear models for repeated measures with time nested within persons. The models specified time as a random effect with linear and quadratic terms. These models inherently adjusted for baseline outcomes. However, the models did not adjust for clustering within the unspecified number of schools.

Cox proportional hazards models were used to analyze the court data, and estimated whether time to first felony arrest differed by treatment group, adjusting for whether the adolescent had been arrested in the previous year.

Intent-to-Treat: The study adhered to the principles of intent-to-treat, using all available data for all respondents, regardless of program completion.

Outcomes:

Implementation Fidelity: Quantitative measures indicated that therapists' overall adherence to Multisystemic Therapy principles was moderately high to high, and consistent with the other studies. With the average treatment lasting 5.19 months, 88% of cases were considered to have been closed successfully, and 8% were considered to have been closed partially successfully.

Baseline Equivalence: Of 15 demographic and baseline primary outcome measures, one differed significantly between treatment groups. The treatment group had a higher proportion of parents who had graduated from high school. The authors state that this variable was not related to treatment outcomes, so it was not included in any of the outcome analyses. Also, the study did not appear to test for differences in the other 14 secondary and risk/protective outcomes.

Differential Attrition: Over the course of the study, 3 families in the treatment group (3.6%) and 8 families (10%) in the control group withdrew from the study or could not be located by the 1-year follow-up assessment The authors did not assess differential attrition.

Posttest and Long-Term: In Weiss et al. (2013), two of four primary outcome measures were improved relative to the control group, with intervention-group parents and adolescents reporting greater decreases in externalizing problems; however, teacher reports and arrest data did not show significant treatment effects. Only one of five secondary outcomes differed significantly between groups, with a quadratic treatment effect for the number of days absent in favor of the treatment group. Among nine risk and protective factors, two showed significant improvement in the treatment group: decreased permissive parenting behavior and decreased parent internalizing mental health problems.

Weiss et al. (2015) briefly summarized the main effect results, with significantly greater declines for the treatment than the control group for parent-reported externalizing and youth-reported externalizing but not for teacher-reported externalizing. The major part of the analysis examined moderation of intervention effects by parenting measures. The results showed that families sometimes gained from MST when parents reported maladaptive processes (e.g., low baseline levels of adaptive child discipline skills) but more often gained from MST when parents reported strengths that facilitated the treatment (e.g., high levels of adaptive functioning).

Study 10

Summary

The study used a quasi-experimental design that examined 1,137 juvenile offenders from Los Angeles County who were assigned to MST treatment. Those participating in the treatment served as the intervention group and those not participating served as the control group. Assessments after six months measured arrests and incarcerations.

The study found no overall beneficial effects of MST relative to the comparison group but showed improvements in:

  • Rates of rearrest, incarceration, and completion of community service among Hispanic youth only.

Evaluation Methodology

Design:

The program was evaluated using a QED without randomization of 1,137 juvenile offenders that qualified for MST participation based on the Los Angeles County Juvenile Justice and Crime Prevention Act's (JJCPA) eligibility criteria. The intervention group consisted of 757 individuals who were accepted into and participated in Los Angeles' MST program between January 2003 and December 2010, and the comparison group contained 380 youth who were eligible but did not participate in MST between January 2001 and December 2010, most often because of lack of Medicaid coverage. There is little information on the kind of treatment that the comparison group received.

Data from all participants were tracked for 6 months after qualifying for (in the comparison group) or entering the program (in the intervention group) using automated databases maintained by the Los Angeles County Probation Department, but not all measures applied to all participants. The full sample (100%) had arrest and incarceration data, with fewer having data on completing probation (N= 724, 96% of treatment group; N= 353, 93% of comparison group), probation violations (N= 722, 95% of treatment group; N= 353, 93% of comparison group), completing restitution (N= 477, 63% of treatment group; N= 255, 67% of comparison group), and completing community service (N= 363, 48% of treatment group; N= 153, 40.3% of comparison group). Risk and protective factors targeted by the intervention were only collected from the treatment group beginning in 2004, resulting in 508 participants (67% of treatment group) with data in this area.

Sample:

The mean age of program participants was 15.3 at baseline, and almost all youth (96%) had at least one prior arrest. The majority of participating youth were Male (77%) and Hispanic/Latino (about 74%), with a smaller number identifying as African American (about 20%) and very few as White or some other ethnicity (about 7%). Most participants were arrested for violent (30%) or other crimes (43%) that led to their qualifying for MST, with fewer arrested for property (21%) or drug crimes (5%).

Measures:

Juvenile justice outcomes, all measured as zero for no and one for yes, included 1) arrests, 2) incarcerations, 3) successful completion of probation, 4) successful completion of restitution, 5) successful completion of community service, and 6) probation violations. All were measured continuously for 6 months after qualifying for MST and were obtained from an automated database maintained by the Los Angeles County Probation Department.

Five risk and protective factors targeted by the program were measured among the treatment group at baseline and discharge from the program, beginning in 2004. The factors included parenting skills, family relations, network of social supports, success in educational or vocational settings, and involvement with prosocial peers. Performance in each area was rated as satisfactory or unsatisfactory according to criteria specified by MST caseworkers; for example, improvement in parenting skill required parents to demonstrate at least two of the following: 1) increased limit setting, 2) established and enforced consequences, 3) increased monitoring. MST caseworkers who delivered the therapy provided the scores.

Analysis:

Logistic regression was used to evaluate all juvenile justice outcomes with race/ethnicity, treatment group, age, gender, and type of offense at recent arrest included as covariates. Race-stratified analyses were also performed, comparing the effects of MST treatment for African American and Hispanic participants. McNemar's test (paired chi-square) was used to evaluate the significance of pre- to post- changes in risk and protective factors and did not include adjustments for sociodemographic or criminogenic factors.

The study adhered to the principles of intent-to-treat, analyzing all participants with relevant data in their original condition.

Outcomes:

Implementation Fidelity: No measure of implementation fidelity was reported.

Baseline Equivalence: The treatment and comparison groups were similar at baseline, significantly differing only on ethnicity (MST participants were more likely to be Hispanic). While baseline equivalence was not examined for the study outcomes, similar measures such as having a prior arrest and offense type at last arrest did not differ by group. However, the groups likely differed on unmeasured socioeconomic characteristics that partly determined assignment.

Differential Attrition: While the study did not examine differential attrition, all subjects were retained for the analysis of arrests and incarcerations. The smaller sample sizes for other outcomes are likely a result of variation in the sentences given to participants at trial.

Posttest: After controlling for race, age, gender, and type of offense, the treatment group showed significant improvement in 3 of 6 outcomes (arrests, incarcerations, and completion of community service) relative to the comparison group. Without the controls, the intervention group had significantly more probation violations than the comparison group.

Race-specific analysis of these outcomes revealed that the program's positive effects were observed for Hispanic youth only. While Hispanic MST youth had significantly lower rates of arrest, lower rates of incarceration, and greater odds of completing probation than Hispanic comparison youth, African American MST youth had higher rates of arrest than the comparison group and did not differ significantly on any other outcomes.

Of the 6 functional areas targeted by the program, all (parenting skills, family relations, network of social supports, educational/vocational success, involvement with prosocial peers) were significantly improved at posttest within the treatment group (though with no comparison to the non-MST group).

Study 11

Summary

The study used a randomized controlled trial that assigned 256 Dutch adolescents with antisocial behavior to an MST intervention group or a treatment-as-usual control group. Assessments from posttest to a three-year follow-up measured a variety of outcomes relating to antisocial behavior.

The study found that, relative to the control group, the MST intervention group showed significant reductions in:

  • Externalizing
  • Oppositional defiant disorder
  • Conduct disorder
  • Property offenses

Evaluation Methodology

Design:

Recruitment: Adolescents with various types of serious antisocial behavior were referred to the study from multiple community sources across the Netherlands and during the years 2006 to 2010. Of 318 assessed for eligibility, 256 met inclusion criteria and consented to participate.

Assignment: The study randomly assigned the 256 participants to MST (N = 147) or a treatment-as-usual control group (N = 109). The randomization was adjusted using a 1:2 ratio in favor of MST for a six-month period due to a low number of referrals.

Assessments/Attrition: Assessments took place at baseline before the start of the program, about 6 months later at posttest, and at 1.08 years after pretest (roughly 6 months postintervention). Official recidivism data were collected at 3.06 years (mean length, called a 2-year follow-up in the paper) after pretest. They also occurred monthly during the intervention period. About 13% of the participants were lost at posttest and 23% were lost at follow-up.

Sample Characteristics: Most of the subjects had a Dutch background (55%). Many belonged to ethnic minorities (34% Moroccan and 32% Surinamese). Half lived in single-parent homes, and 50% of the mothers and 36% of the fathers were unemployed. More than half the families lived below minimum income levels. Judicial records showed that 71% had been arrested at least once.

Measures: Baseline and posttest assessments took place in the homes of the subjects, and "the majority" of research assistants doing the assessments were blind to the assigned condition. Measures came from parent reports, adolescent reports, and observations and showed acceptable reliability. A brief list summarizes the numerous measures:

  • Cognitions based on parent-reported sense of competence and adolescent-reported self-esteem, personal failure, and hostility
  • Parenting based on parent-, adolescent-, and observer-reported positive discipline, inept discipline, and relationship quality
  • Deviant peers and prosocial peers reported by adolescents
  • Externalizing behaviors from the parent-reported Child Behavior Checklist
  • Oppositional defiant disorder and conduct disorder from the parent-reported Defiant Behavior Disorders rating scales
  • Externalizing behaviors from the adolescent-reported Youth Self Report
  • Violent offending and property offending from the adolescent-reported Self-Report Delinquency scale
  • Official recidivism data from the Judicial Registration System, Dutch Ministry of Justice

Assessments completed within the intervention period were done by phone with an abbreviated set of items for each scale.

Analysis: Using an intent-to-treat strategy, the study employed three different statistical approaches. First, the analysis examined posttest outcomes for the groups with ANOVA and controls for baseline outcomes. Second, the analysis examined within-intervention trajectories using latent growth models. Third, structural equation models tested for mediation effects.

Missing Data Method: An email to Blueprints from the lead author noted that the expectation maximum likelihood algorithm in LISREL replaced missing data for sociodemographics and relevant baseline outcomes.

Intent to Treat: Given evidence of data missing completely at random, the analysis used all cases with the expectation maximum likelihood algorithm in LISREL.

Outcomes

Implementation Fidelity: A Therapist Adherence Measure that the investigators collected monthly from the parents assessed adherence to the nine principles of MST. The mean of 4.36 on a scale from 1 to 5 showed acceptable adherence.

Baseline Equivalence: No significant baseline differences were found on any of the demographic or outcome variables.

Differential Attrition: Participants lost to postintervention assessment did not differ significantly on any of the assessed variables from those that remained in the study, based on the the additive test for completers vs. attritors (Asscher et al., 2013; 2014; Dekovic et al., 2012). The authors stated that Multiple Imputation was carried out by the expectation maximum likelihood algorithm. Additionally, Dekovic et al. (2012) used the Little's test to provide evidence that data were missing completely at random.

Posttest: Deković et al. (2012) presented results on 1) intervention effects at posttest, 2) within-intervention change, and 3) mediated effects. First, the program significantly improved four of the five posttest measures compared to the control group: parental sense of competence, positive discipline, relationship quality, and externalizing problems. Only inept discipline did not improve. Second, as shown by differences across groups in the mean slopes, the program significantly increased the within-intervention rates of growth in the same four outcomes as above. Effect sizes similarly increased over time. By the end of the sixth month, effect sizes for the four significant outcomes ranged from about .2 to .5. Third, mediation models showed that participation in MST was significantly related to a greater increase in maternal sense of competence, which in turn predicted an increase in positive discipline. These changes then predicted a decrease in adolescent externalizing. However, a similar model using relationship quality rather than positive discipline did not significantly mediate between the program and externalizing behavior.

Asscher et al. (2013) found significant improvement in the treatment group relative to the control group for 5 of 6 primary measures: parent-reported externalizing, opposition defiant disorder, and conduct problems, and adolescent-reported externalizing and property offending. Effect sizes ranged from .25-.36. They also found significant program improvement for 9 of 15 secondary measures. The significant outcomes included measures of parent-reported sense of competency, youth-reported hostility, parent-, adolescent-, and observed-reported positive parenting, parent- and observer-reported quality of relationship, observer-reported inept discipline, and adolescent-reported prosocial peers. However, one significant iatrogenic effect emerged for adolescent-reported sense of personal failure. Effect sizes for the secondary measures ranged from .26-.47.

Asscher et al. (2013) also tested for moderation effects by ethnicity and age, with little evidence of differences across groups. Tests for moderation by gender found that for adolescent cognitions, the treatment had larger (and more positive) effects for boys than for girls.

Manders et al. (2013) tested for additional moderation effects. They found that MST was more effective than the control in decreasing externalizing problems for the "lower callous/unemotional" and "lower narcissism" group, but not for the "high callous/unemotional" and "high narcissism" group.

Long-term (Asscher et al., 2014): There were no differences between the conditions in frequency or number of arrests, time to re-arrest, or type of arrest, at the six-month and 2-year follow-ups. At the end of the final follow-up period, 63% of the MST group and 53% of the TAU group had been rearrested at least once, but these differences were not significant.

Moderation (Asscher et al., 2018): The study examined moderation tests for differences in program effects for extremely violent and not extremely violent youth. The results showed no significant moderation at posttest for externalizing or relationship quality. However, additional analyses of monthly data from pretest to posttest demonstrated different patterns of change. The extremely violent youth worsened initially but then showed large improvements in both externalizing and relationship quality.

Study 12

Summary

The study used a randomized controlled trial that assigned 93 juvenile offenders to an MST intervention group or a treatment-as-usual control group. Assessments through 18-month follow-up measured reoffending and youth functioning in eight domains.

The study found that, relative to the control group, the MST intervention group showed significantly:

  • Lower rates of reoffending and rearrest
  • Greater improvements in four areas of functioning (home, school, community, and moods and emotions)

Evaluation Methodology

Design
:

Recruitment: Participants were 93 youth who appeared before a family county court in a Midwestern State between October 1998 and April 2001. Youth were recruited for participation if they met the following inclusion criteria: (a) a felony conviction, (b) a suspended commitment to the Department of Youth Services incarcerating facility, and (c) parents' consent to participate. All youth were either on probation at the time of the study or had been on probation previously.

Parents or legal guardians of youth meeting the study inclusion criteria were asked to consent to random assignment to either MST or TAU. The court agreed to randomize (into the MST or TAU conditions) families in which caregivers and youths agreed to participate in the study. Both parents and youth provided informed consent, which was obtained either by court personnel or by the MST supervisor. Overall, 89% of eligible participants who met inclusion criteria agreed to participate in the study (n=105). If parents or guardians did not agree but were eligible for MST, it was at the discretion of the court whether to assign them to MST without study participation.

Assignment: Randomization was accomplished by having the court administrator flip a coin. 48 youths were randomly assigned to the MST condition and 45 youth were assigned to the TAU condition.

Assessments/Attrition: 89% of participants completed the study (i.e., 11% of study participants dropped out resulting in a final analysis sample of 93 participants). Recidivism was tracked through an 18-month post-treatment follow-up.

Sample: The mean age of all youths was 14.1 years at the time of enrollment in the study. Twenty-two percent of the participants were female, and seventy-eight percent were male. The racial composition of the sample was as follows: 15.5% African American, 77.5% European-American, 4.2% American Hispanic and 2.8% biracial.

Measures:

Official charge data: The county family court keeps detailed information regarding juvenile arrests. The recidivism analyses in this study were based on those charges for which the youth was formally arraigned following discharge from treatment (for the MST group) or at 6 months post-recruitment (for the TAU group). Charge data were examined through 24-month post-recruitment for both groups. Despite the level of detail in the court record concerning rearrests, few details were available on the specific type of new charge; however, each charge was designated as either a misdemeanor or a felony.

CAFAS: Ratings for the Child and Adolescent Functional Assessment Scale (CAFAS) were made corresponding with the beginning of treatment, discharge, and 6 months following discharge for MST youth. For TAU youth, ratings approximated timing of administration for MST youth (i.e., baseline, 6-month post-recruitment, and 12-month post-recruitment). The CAFAS measures youth functioning in eight areas: school and work, home, community, behavior, substance use, and thinking. For each subscale, the child receives a score of 0, 10, 20, or 30. A score of zero indicates no or minimal impairment, 10 indicates mild impairment, 20 indicates moderate impairment, and 30 indicates severe impairment. A total score based on the summation of the subscales reflects overall youth functioning. The range of possible scores on the CAFAS is 0 to 240. The higher the CAPAS score, the greater the functional impairment.

Brief subscale descriptions provided in the current edition of the CAFAS manual are as follows: (a) School/Work: ability to function satisfactorily in a group education environment, (b) Home: extent to which youth observes reasonable rules and performs age appropriate tasks, (c) Community: respect for the rights of others and their property and conformity to laws, (d) Behavior Toward Others: appropriateness of youth's daily behavior, (e) Moods/Emotions: modulation of the youth's emotional life, (f), Self-Harmful Behavior: extent to which the youth can cope without resorting to harmful behavior or verbalizations, (g) Substance Use: youth's substance use and the extent to which it is disruptive, and (h) Thinking: ability of youth to use rational thought processes.

To provide a standardized method of CAFAS assessment, research assistants used court records to rate MST and TAU youth at baseline, at immediate posttreatment (for MST youth) or 6-month post-recruitment (for TAU), and at 6-month post-treatment (for MST youth) or 12-month post-recruitment (for TAU youth).

Analysis: A 2 (treatment condition: MST vs. TAU) x 3 (time: pretreatment, immediate post-treatment, and 6-month post-treatment follow-up) mixed factorial design was used to evaluate functioning. Relative odds ratios were calculated for the likelihood of rearrest. Binary logistic regression was conducted to compute the relative risk of rearrest in the TAU versus the MST group. The survival analysis was conducted using a Fleming-Harrington test that weighted offenses later in time more heavily than offenses earlier in time. The rationale for using this weighting scheme with post-treatment arrest data derives primarily from knowing that rates of reoffending typically increase over time as active supervision of youths' activities decreases.

General linear modeling repeated measures were used to examine changes in six CAFAS subscale scores over time. The CAFAS total score is not presented as there is limited evidence that this score presents information separate from that already covered in the subscales. To minimize the effect of multiple tests made on the data, alpha was set at .008. Using the adjusted alpha, time by interaction effects were tested.

Missing Data Methodology: The study used complete case analysis without imputation or FIML estimation.

Intent-to-Treat:  The study referred to including participants who "completed the treatment and completed the study." The wording implies that only program completers were kept for the analysis.

Outcomes

Implementation Fidelity:

Baseline Equivalence: There were no statistically significant between-group differences with respect to race or sex. There were also no statistically significant between-group differences on court-related variables such as age at first offense, number of pre-treatment offenses, number of pre-treatment misdemeanors, and number of pre-treatment felonies. However, these results appeared to use the analysis sample rather than the randomized sample. Further, tests using the analysis sample in Table 2 show several significant differences between the conditions at baseline (i.e., time 1).

Differential Attrition:

The study did not refer to tests for attrition or show condition differences in attrition rates, but the tests for baseline equivalence of conditions using the analysis offer evidence of differential attrition (see Table 2 for time 1).

Posttest and Long-term:

At the 18-month post-treatment follow-up, the recidivism rate for the MST group (66.7%) was significantly lower than the overall recidivism rate for the TAU group (86.7%). Youth in the MST group were also arrested and arraigned for significantly fewer offenses. There were no significant between-group differences in the percentage of felonies versus misdemeanors. Youths in the TAU group were 3.2 times more likely than youths in the MST group to be rearrested. For youth with at least one rearrest, the average time to first arrest was 135 days for youths in the MST group and 117 days for youths in the TAU group, a nonsignificant difference.

With regards to youth functioning, both groups evidenced improvement in functioning over time, with the MST CAFAS scores significantly better on four of six subscales: home, school, community, and moods and emotions. The difference between the groups was nonsignificant for substance use and behavior towards others.

Study 13

The main goal of this study was to evaluate whether Multisystemic Therapy (MST) is more effective in reducing youth offending than an equally comprehensive management protocol called Youth Offending Teams (YOTs). This study can be seen as true independent evaluation since it was conducted without the program developers close involvement. Whereas most other MST evaluations were conducted in the U.S., this study was fielded in the United Kingdom.

Summary

The study used a randomized controlled trial that assigned 108 juvenile offenders in London, England, to an MST intervention group or a treatment-as-usual control group. Assessments from posttest to an 18-month follow-up measured reoffending and problem behavior.

The study found that, relative to the control group, the MST intervention group showed significant decreases in:

  • Aggression, delinquency, and psychopathic traits at posttest
  • Nonviolent offenses at the 18-month follow-up assessment.

Evaluation Methodology

Design:

Recruitment /Sample size/Attrition:
For the recruitment process, this study used referrals from two local youth offending services in North London. Young persons were included in the study if they were between 13 and 17 years of age; living in the home of and being brought up by a parent or principal caretaker; and being on a court referral order for treatment, supervision, or following imprisonment. Youth were excluded if they were a sex offender; presented only with substance misuse; were diagnosed with a psychotic illness; or posed a risk to research personnel. A total of 478 young persons were referred to the study team, of whom 370 (77%) were excluded because they could not be contacted, refused to consent to assessment, or did not meet the inclusion criteria. The remaining 108 participants were randomized into Multisystemic Therapy (MST) treatment group (n = 56) or Youth Offending Teams (YOT) control group (n = 52). For primary outcome measures, one participant was excluded from the MST group "due to lack of evidence of offence at intake" (p. 1222) (attrition of < 1%). For secondary outcome measures, 3 participants in the MST group and 1 participant in the YOT group did not complete the questionnaires (attrition of 4%).

Study type/Randomization/Intervention:
The study employed a randomized control trial design. The MST team comprised three therapists and a supervisor. The therapists had low caseloads, usually visited the families at least 3 times per week, and were available by telephone to support them 24 hours per day and 7 days per week. The lengths of the interventions ranged from 11 to 30 weeks (mean = 20.4 weeks). Students in the YOT control group received the usual services. As in MST, YOT interventions are extensive and multi-component and included helping the young person to re-engage in education; help with substance misuse problems and anger management; training in social problem-solving skills, and programs for crime awareness. The YOT treatments were delivered by professional social workers, therapists, or probation officers. During the intervention period, participants in the YOT group received approximately 21 professional appointments. There are two key differences between MST and YOT. 1) In contrast to YOT, MST is delivered in a family context by a single person. 2) There is no overarching model or set of principles that governs the selection of treatments in YOT, which could be compared to those in MST. Thus, YOT interventions are offered on an "as needed" basis by specialist agencies to which the young person is referred.

Assessment:
The primary outcome measure (offending behavior) was measured at 6-monthly intervals: for the 6 months before randomization, for the 6 months covering the intervention period, and then every 6 months until 12-months after posttest (to which the authors refer as an 18-month follow-up assessment). All secondary outcome measures were obtained at baseline and at posttest, after the MST treatment was completed (6 months after randomization).

Sample Characteristics: The majority of participants were male (82%) and were, on average, 15 years of age. The sample was racially diverse with 34% white, 32% black, 5% Asian, and 24% classified as Mixed/Other. Participants had an average of more than two offenses at intake with more than half the convictions constituting violent offenses. Only a small minority of individuals were living with two parents; more than two-thirds lived with their mothers but not their fathers, and less than 10% with their fathers but not their mothers. Only one-third were in mainstream education. Of the parents, 31% had left school with no academic qualifications; 40% had no vocational qualifications; and 54% were without income. In sum, almost all subjects lived in socioeconomically disadvantaged families.

Measures:
Validity of measurements:
All measures were borrowed from prior questionnaires for which validity and reliability had been demonstrated. Alpha values of the employed scales are not reported.

Primary outcome measures:

  • The number of records of offending behavior (count data) was obtained from the National Young Offender Information System (YOIS) database. In addition, a dichotomous variable measured the occurrence of 6-month periods free of any offending behavior.

Secondary outcome measures:
Self- and parent-rated symptoms of antisocial behavior, delinquency-linked cognitions, personality functioning, and parenting variables were measured using the following items/scales:

  • Self-Report of Youth Behavior (SRYB): Based on youth's self-report, this scale measured antisocial behavior such as vandalism, theft, burglary, and fraud.
  • Youth Self-Report (YSR) questionnaire: The delinquency and aggression subscales of the YSR were employed.
  • Child Behavior Checklist (CBCL): Was completed by parents or guardians.
  • Antisocial Beliefs and Attitudes Scale (ABAS): Based on youth's self-reports, this scale measured beliefs and attitudes toward standards of acceptable behavior in social and family contexts.
  • Positive Parenting and Disciplinary Practices (PP) measure: This questionnaire was completed by the primary caregiver.
  • Subjective Family Image Test (SFIT): Completed by the family, this questionnaire assessed the quality of the emotional bond between adolescent and parent.
  • Antisocial Process Screening Device (APSD): This parent-completed questionnaire measured youth psychopathic traits.
  • Youth's Involvement with Delinquent Peers (IDP) scale: This parent-completed 16 items scale was adapted from the Youth in Transition Study.

In addition, a demographic data form was used to gather information regarding participants' ethnicity and socioeconomic background.

Analysis: The authors employed multilevel models to account for the hierarchical data structure (measurement points are nested within persons). Both intercepts and slopes of time were allowed to vary randomly across individuals. Depending on the investigated outcome measure, linear, logistic, or Poisson models were used. All models implicitly control for baseline characteristics.

Intention-to-treat: The study followed the intent-to-treat principle.

Outcomes

Implementation fidelity: Therapists that delivered the intervention received thorough training in MST programs and procedures as part of the study. In addition, therapists received weekly supervision from the MST supervisor, as well as weekly 1-hour consultation (via telephone) with an MST outside expert, on-site booster training sessions four times per year, and twice-yearly implementation reviews. The research team closely followed the MST Organizational Manual. In addition, the MST Therapist Adherence Measure (TAM) was used to assess adherence to the nine MST treatment principles.

Other than to note that YOT youth attended a significantly greater number of appointments than the MST youth, the study does not provide measures of fidelity.

Baseline Equivalence: The intervention and control group did not differ significantly at baseline on any measured variable.

Differential attrition: No test for differential attrition was performed. The authors point out that "missing values were not a significant problem in the analysis of the data set for primary outcomes (< 5%) and data for all participants were used for secondary outcomes, although three individuals in the MST and one in the YOT group provided no self-report information" (p. 1225).

Post-test/Long-term: Due to the reporting of time x group interaction, it is not possible to disentangle post-test and long-term effects, except for instances in which the authors provide these details in the text.

Primary outcomes:
Significant group x time interactions suggest a greater decrease in recorded offenses for youths in the MST group compared to youths in the control group. The number of offenses between the two study groups did not differ at post-test, and 6-month follow-up but became significant (p<.001) at the 12-month follow-up assessment. In addition, a significant group x time interaction was found for participants with 6-month periods free of offenses (p<.001). This effect was more pronounced in the MST group than in the YOT group. A sub-category analysis showed that the group x time interaction for both outcome measures was significant for nonviolent offenses (6-month period free of offenses, p<.005; No. of recorded offenses, p<.02) but not for violent offenses. For example, at the 12 months follow-up assessment only 8% in the MST group compared with 34% in the control group had one record or more of a nonviolent offense during the past 6 months (p<.001).

Secondary outcomes:
Out of 21 tests for secondary outcome measures, 5 (24%) were significant. Significant program effects were observed for measures of aggression and delinquency (p<.05). For all significant findings, the group x time interaction suggested that the problem behavior declined stronger for the MST compared to the YOT group. In addition, a number of significant group differences were found for personality, relational, and cognitive measures. The measure for parent reported psychopathic traits (ASPD) declined substantially more over time in the intervention compared to the control group (p<.02). In addition, positive parenting increased in the MST group but decreased in the control group, resulting in a significant (p<.05) group difference.

Effect size: The study reported that the effect size of the change over time for the MST group was medium for both aggression (ES = 0.42) and delinquency (ES = 0.64), while effect sizes were small for the YOT group on aggression (ES = 0.09) and delinquency (ES = 0.25). However, the study does not report effect sizes for differences between groups across time.

Mediating effects: A mediator analysis demonstrated that positive parenting did not account for program related changes in offending behavior among youths. Similarly, adherence to MST standards (based on parent's independent reports about their therapy) did not mediate the group differences on the primary outcome measure of offense frequency.

Study 14

Summary

The study used a randomized controlled trial that assigned 156 Swedish youths with a diagnosed conduct disorder to an MST intervention group or a treatment-as-usual control group. Assessments at posttest and two-year follow-up measured delinquency, alcohol and drug use, and school attendance.

The study found no effects of MST on parenting, mental health, delinquency, social competence, alcohol use, or drug use.

Evaluation Methodology

Using a clinical sample of youths diagnosed with conduct disorder, Sundell et al. (2008) examined the seven-month posttest results, while Lofhölm et al. (2009) examined the 17-month follow-up results.

Design:

Recruitment: The sample consisted of youths ages 12-17 years who met the criteria for a clinical diagnosis of conduct disorder. Exclusion criteria were (a) on-going treatment by another provider; (b) substance abuse without other antisocial behavior; (c) sexual offending; (d) autism, acute psychosis, or imminent risk of suicide; and (e) the presence of the youth in the home constituting a serious risk to the youth or the family. All youths were referred to the study from March 2004 through February 2005 by child welfare services in 27 local authorities (municipalities) from Sweden's three largest cities and one town. A total of 168 families were asked to participate in the study, and 156 (93%) accepted.

Assignment: Randomization of youths to the intervention group (n = 79) or control group (n = 77) immediately followed initial data collection, when research staff opened a sealed and numbered envelope that contained the result of computer-generated randomization. The randomization used the six MST sites as a blocking variable. The control group youths received treatment as usual, which consisted of social services such as individual counseling, family therapy, mentorship, out-of-home care, and residential care.

Assessments/Attrition: The posttest came about seven months after randomization, and a follow-up came 24 months after randomization or about 17 months after posttest. At the posttest, 14 of the 156 families lacked data (9%). Seven had withdrawn from the evaluation, and seven refused to complete the questionnaire. For specific posttest measures, missing data varied between 9-12% for youths and between 8-12% for guardians, with the exception of the guardian's mental health, where missing data reached 19%. At the follow-up, 16 young people and 15 parents did not complete the questionnaire. For specific measures, missing data varied between 10-11% for young people and between 10-16% for guardians.

Sample:

The sample consisted of 61% boys and 39% girls with a mean age of 15.0 years. At study intake, 67% of the youths had been arrested at least once and 32% had been placed outside of the home at some point during the six months before the study intake. Almost half (47%) of the families spoke a language other than Swedish in the home. A majority of the youths (67%) lived in a single-parent home. Of the mothers, 18% had a college education and 51% were unemployed. Of the families, 61% lived entirely or in part on social welfare grants.

Measures:

Youth Psychiatric Symptoms. Six measures from both caregiver and child ratings of total, internalizing, and externalizing symptoms came from the Child Behavior Checklist and Youth Self-Report. Internal consistency was high (.88-.96).

Sense of Coherence. One youth-reported measure from the Sense of Coherence scale reflected the youth rating of life as comprehensible, manageable, and meaningful. Internal consistency was acceptable (.78).

Self-Report Delinquency. Youths answered 40 questions designed to measure antisocial behavior pertaining to violence, general delinquency, and status offenses. Internal consistency was high (.92). In addition, social workers and parents reported on the number of police reports involving the youth.

Alcohol and drug consumption. Youths reported on the frequency and amount of their consumption of five types of alcoholic beverages (e.g., beer and wine) during the six months before measurement. Youths also reported on their use of six types of drugs (e.g., cannabis and cocaine) during the six months before measurement. The Alcohol Use Disorder Identification Test assessed risky alcohol consumption, dependency, and alcohol-related harm. Internal consistency was high (.88). The Drug Use Disorder Identification Test measured current drug-related problems, excluding alcohol. Internal consistency was high (.93).

Anti-Social Peers. One subscale from the Pittsburgh Youth Study measured relationships with antisocial peers. This measure had moderate internal consistency (.71).

Social Competence with Peers. Youth social competence was measured through caregiver and adolescent ratings of social interactions with peers. Internal consistency was .77 for youths and .84 for guardians.

Social Skills. A scale was adapted from the Social Skills Rating System. Internal consistency was acceptable (.72).

School attendance. Information on school attendance came from the school authorities and was based on the 95% of students who continued on to high school after completing compulsory education at Grade 9.

Out-of-Home Care and Social Services. The two measures came from case-file reviews. Social workers also reported on the type and extent of services received by families before referral.

Parenting Skills. Caregivers as well as youths described parenting skills, with a focus on parental knowledge, parental monitoring, parental soliciting, youth disclosure, and family decision making. Internal consistency was high for youth reports (.88) and parent reports (.89).

Mother's Mental Health. Mothers completed the Symptom Checklist-90 with nine symptom constructs and one global index. Internal consistency was high (.98).

Analysis:

The analyses used repeated measures ANOVA for outcomes with both a pretest and posttest and used one-way ANOVA for outcomes with no baseline measure. The key coefficients came from the treatment-by-time interaction terms.

Intent-to-Treat: The analyses used all cases in their original assignment. Missing data at the posttest were imputed by carrying the baseline measure forward, while missing data at follow-up were imputed with multiple imputation.

Outcomes

Implementation Fidelity:

About 73% of the MST clients completed the program on the basis of the mutual agreement of the primary caregiver(s) and the MST team. This was similar to the average treatment completion rate (74%) for MST programs worldwide. Fidelity scores for 160 interviews with 60 families (alpha = .86) had a mean of 4.00 (range of 1.73 to 5.00), approximately one standard deviation lower than that reported in a U.S. study.

Baseline Equivalence:

Intervention parents exhibited significantly more mental health symptoms than control parents, and control parents reported more children with early-onset behavior problems than intervention parents. No other significant differences existed between the treatment and control groups on demographic or psychosocial variables.

Differential Attrition:

At posttest, the 14 youths who did not complete the questionnaire did not differ significantly from the 142 youths who did on either demographic or psychosocial characteristics. At follow-up, the 16 young people and 15 parents who did not complete the questionnaire did not differ significantly on any of the baseline measures from the 140 young people and 141 parents who did.

Posttest:

Tests reported by Sundell et al. (2008) for 20 outcomes revealed only one significant result: The intervention youth received more days of service than the control youth. Numerous tests for moderation found few subgroup differences in program effects.

Long-Term:

Tests reported by Lofhölm et al. (2009) for 20 outcomes revealed no significant results. Numerous tests for moderation found few subgroup differences in program effects.

Study 15

Summary

The study used a randomized controlled trial that assigned 684 English youths with antisocial behavior to an MST intervention group or a treatment-as-usual control group. Assessments through five years measured out-of-home placement and criminal offending.

The study found several posttest effects of the MST intervention, but the effects disappeared after one year and the program showed no long-term effects.

Evaluation Methodology

Design:

Recruitment: The sample came from nine sites across England. Young people ages 11-17 with moderate-to-severe antisocial behavior were recruited from local service organizations. Those eligible had at least three severity criteria indicating past difficulties across several settings and one of five general inclusion criteria for antisocial behavior: persistent (weekly) and enduring (≥6 months) violent and aggressive interpersonal behavior; at least one conviction plus three additional warnings, reprimands, or convictions; conduct disorder diagnosed according to DSM-IV criteria and not responding to treatment; permanent school exclusion for antisocial behavior; and significant risk of harm to others or self. Between February 4, 2010, and September 1, 2012, 1,076 families of young people were referred as eligible, and 684 families consented to baseline assessment.

Assignment: Eligible families were randomly assigned (1:1) by an independent researcher to the management-as-usual control group (n = 342) or to three to five months of Multisystemic Therapy followed by management as usual (n = 342). The randomization stratified for treatment site, sex, age at study enrolment (<15 years or ≥15 years), and age at onset of antisocial behavior. Management as usual involved the best available local services for young people, as identified by a multi-agency referral panel.

Assessments/Attrition: Outcomes were assessed at baseline and six, 12, 18, 24, 36, 48, and 60 months after randomization. With the program lasting three to five months, the assessments included a posttest and six-month follow-up plus long-term follow-ups ranging from one year to 4.5 years after treatment end. For the primary outcome, which was measured at two time points, 99% of the participants had data at 18 months and 89% at 60 months. For secondary outcomes, which were measured at six time points, 85% of the 684 randomized families remained in the study at six months, 76% at 12 months, 72% at 18 months, 70% at 24 months, 63% at 36 months, and 51% at 48 months.

Sample:

More than 80% of the sample youths met DSM-IV criteria for a conduct disorder (including oppositional defiant disorder). Their mean age was 13.8 years, with 63.5% being male. About 75-78% of the families had low income, and most were white (76-80%).

Measures:

Investigators and research assistants were masked to treatment allocation and were located separately to avoid contamination. The primary outcome in Fonagy et al. (2018) was the proportion of participants in out-of-home placement at 18 months. Secondary outcomes included time to first criminal offense and the number of offenses (total, non-violent, and violent), based on official police records. In addition, Fonagy et al. (2018) examined about 39 other secondary outcomes (see Tables 4-7 for a list). Many relating to conduct problems, delinquency, ADHD, and strengths and difficulties came from questionnaires completed by young people, parents, and teachers. Parents rated their children as well as their own parenting skills and mental health. Because parents and youths participated jointly in the program, parent ratings of the youths can be considered independent. Measures of school attendance and exclusions came from educational records. Psychiatric disorders were assessed by clinicians at baseline and 12 months using the Development and Well-Being Assessment instrument. Appendix Table A.6 shows good reliabilities for nearly all scales.

The primary outcome in Fonagy et al. (2020a, 2020b) was the proportion of young people with any criminal conviction at 18 months and 60 months. The secondary outcomes included the total number of recorded offenses with convictions, time to conviction, and whether the conviction resulted in a custodial sentence, community sentence, or a caution. The numerous self-reported secondary outcomes completed by the young person and their parent or caregiver measured antisocial behavior and attitudes, parenting, family functioning, and the use of healthcare, social care, education, and criminal justice services (see Table 2 in Fonagy et al., 2020b, pp. 10-11). With a few exceptions, the measures had acceptable reliability coefficients (see Appendix 1, Table 20 in Fonagy et al., 2020b, p. 59).

Analysis:

The analysis of out-of-home placement at 18 months in Fonagy et al. (2018) used a mixed-effects logistic regression model, and the analysis of the time-to-event outcomes for first criminal offense used Cox regression. Random effects adjusted for clustering by therapist. The logistic regression model included covariates for site, number of past convictions, sex, and age at onset of criminal behavior. The analysis of secondary outcomes used linear mixed models, with adjustment for baseline values, and Poisson mixed models for count variables.

The analyses in Fonagy et al. (2020a) used mixed-effects logistic regression models for dichotomous outcomes, Cox proportional hazards models for time-to-event outcomes, Poisson mixed models for count outcomes, and linear mixed-effects models for continuous outcomes. All models included the following covariates: prior criminal convictions, sex, age at onset of criminal behavior, site as a random effect, and the baseline outcome. The models used Full Information Maximum Likelihood estimation under the assumption of data missing at random. In addition, the analysis used multiple imputation for all baseline and outcome variables. For the numerous secondary outcomes, the Benjamini-Hochberg multiple testing procedure adjusted for multiple comparisons.

Intent-to-Treat: The analyses included all randomized participants with available data. A sensitivity analysis used multiple imputation but found only minor differences for the results (see the Appendix in Fonagy et al., 2018).

Outcomes

Implementation Fidelity:

The study did not present quantitative analyses, but the authors stated that eight of nine sites performed well above the standards expected by the developers. Detailed cost analyses showed no savings from MST.

Baseline Equivalence:

As shown in Table 1 (Fonagy et al., 2018), the two conditions were similar at baseline, but the study lacked significance tests or d values. The authors noted that slightly more young people with attention deficit hyperactivity disorder were in the intervention group (33%) than in the control group (27%).

Differential Attrition:

At 18 months, the control group had fewer families available for assessment (68%) than the intervention group (75%). Three direct observational points were available for 86% of the intervention families and 81% of the control families. Appendix ii (Fonagy et al., 2018, p. 18) compared the baseline scores of participants who completed the 12-month assessment with those who did not. Although there were no significant differences between these groups at p < .01, there were four significant differences at p < .05 in 42 tests.

Fonagy et al. (2020a) noted that "missing data due to loss to follow-up was not associated with any baseline characteristics, corroborating the representativeness of the sample."

Posttest:

At six months (posttest) or 12 months (six-month follow-up), the program did not significantly affect the number of total crimes, violent crimes, or non-violent crimes.

At six months, Tables 4-6 in Fonagy et al. (2018) show significant program effects for 17 of 33 tests, while at 12 months, the tables show significant program effects for eight of 33 tests. The intervention group did better than the control group on total strength and difficulties (youth reported), emotional problems (youth reported), mood and feelings (youth reported), family satisfaction (parent reported), total strengths and difficulties (parent-reported), emotional problems (parent reported), hyperactivity (parent reported), and parent mental health (parent-reported). In addition, school attendance and exclusion did not differ significantly across conditions, and clinician-rated mental health at 12 months showed no significant effects in six tests (Table 7).

Long-Term:

At 18 months (one-year follow-up), Fonagy et al. (2018) reported no significant program effect on the primary outcome of out-of-home placement or on the secondary outcome of time to first offense (Table 2). At 18 months, there was a significant iatrogenic effect on the mean number of crimes (Table 3). Moderation tests showed three significant subgroup differences in program effects in 19 tests, but all three indicated harmful effects of the program for high-risk youth.

Also at 18 months in Fonagy et al. (2018), Tables 4-6 list significant program effects for two of 33 tests: youth callous and unemotional traits and parent mental health (labeled as General Health Questionnaire in Table 6). By the long-term assessment, none of the earlier significant youth effects persisted, and among the parent risk and protective measures, only self-rated mental health remained significant.

Fonagy et al. (2020a) found no significant differences between conditions at the 60-month follow-up for the primary outcome of a criminal conviction. For the secondary outcomes related to offending, the MST group did significantly worse than the control group at 24 and 48 months, but the difference fell to non-significance by 60 months. For the secondary outcomes at 24, 36, and 48 months, the results showed no condition differences in youth- or parent-reported outcomes after adjusting for multiple tests (see Appendix 3, Tables 22-35 and Appendix 4, Tables 36-45 in Fonagy et al., 2020b).

Study 16

Summary

The study used a quasi-experimental design that examined 320 youths with serious emotional disorders. The participants were non-randomly assigned to an MST-only group, a wraparound-only group, or a wraparound plus MST group that received both interventions. Measures of behavioral problems and global functioning were obtained over an 18-month period.

The study found that, relative to the wraparound-only condition, the MST-only condition showed significantly greater reductions in:

  • Parent-rated youth behavioral problems.

Evaluation Methodology

Design:

Recruitment: The sample consisted of 320 children and adolescents ranging in age from 4 to 17.5 years. Participants came from a care center covering a rural and frontier area of Nebraska and targeting youth with serious emotional disorders. Referrals came from schools, welfare agencies, justice organizations, and other child-serving agencies. To qualify for wraparound services, the youths must have demonstrated a diagnosable mental health disorder, functional impairment, and high risk for restrictive placement, dropping out of school, or involvement in the juvenile justice system. Addition criteria for MST services included physical aggression, delinquency, and substance use.

Assignment: In a quasi-experimental design that used no matching, the study examined three groups of youth: a wraparound-only group (n = 213), an MST-only group (n = 54), and a wraparound plus MST group (n = 53) that received both interventions, sometimes at the same time and sometimes in sequence. Assignment appeared to be based on the extent of problems of the youth, as wraparound and MST had some different requirements for treatment and wraparound plus MST was provided to youth who did not respond to their original treatment. The authors noted (p. 159) that "Propensity score matching was also considered to account for baseline differences; however, the sample size was insufficient to allow for reliable estimates."

Assessments/Attrition: Assessments occurred at baseline and three consecutive six-month follow-ups over 18 months. Attrition rates were 11% at six months, 28% at 12 months, and 37% at 18 months. The final analysis sample, which allowed for use of participants without data for all four time points, included 298 participants (93%). Average treatment lengths ranged from 5.5 to 15 months, which means the assessments did not include a long-term follow-up.

Sample:

The sample averaged 12 years of age, and 73% were male. The racial distribution of the sample was 90% white, 4% American Indian, and 6% other. About 11% of the participants were of Hispanic ethnicity. Most families (57%) reported a gross household income of $25,000 or less.

Measures:

The study examined two outcome measures, one reported by parents (total behavioral problems from the Child Behavior Checklist that combined internalizing and externalizing subscales) and one reported by caregivers or clinicians (global functioning across eight life domains from the Child and Adolescent Functional Assessment Scale). Parents participated in but did not deliver the intervention and therefore provided independent assessments of their children. Both instruments are commonly used and well validated, though the authors did not report reliability and validity information for the study sample.

Analysis:

The analysis used repeated measures mixed models with maximum likelihood estimation and an unstructured covariance structure, random coefficients, and random slopes. The models controlled for baseline outcomes, gender, age, minority status, family income, and number of living placements reported at each six-month follow-up. Condition-by-wave interaction terms tested for program effects.

Intent-to-Treat: Models were estimated with imputed and non-imputed data, but because the results did not differ, the study reported results from the non-imputed data.

Outcomes

Implementation Fidelity:

MST-only youth spent 5.5 months in therapy and reported higher participation in family preservation and family therapy than the wraparound-only youths.

Baseline Equivalence:

Table 1 showed eight significant baseline condition differences in 11 tests. The MST-only group was older, included more males, and had fewer problems than the other groups. Youth who received both wraparound and MST had more severe problems than those of the other two groups, likely because they did not respond to the first assigned intervention.

Differential Attrition:

The authors reported that attrition at 18 months did not differ by treatment group, gender, age, minority status, family income, or the baseline outcomes.

Posttest:

The mixed models found, as indicated by the group-by-wave interactions, significantly greater improvements for the MST-only group than the wraparound-only group for one of two outcomes - total behavioral problems from the parent-rated Child Behavior Checklist but not global functioning from the Child and Adolescent Functional Assessment Scale.

Long-Term:

Not examined.

Study 17

Summary

The study used a randomized controlled trial that assigned 674 youths with serious emotional disorders in rural Tennessee to an MST condition or a usual-services condition. Measures of behavioral problems and out-of-home placements were obtained over an 18-month period and used to examine MST effectiveness in counties with and without a countywide service-facilitation program.

The study found that, relative to the usual-services condition, the MST condition showed significantly fewer:

  • Out-of-home placements
  • Behavioral problems but only in counties with a countywide service-facilitation program

Evaluation Methodology

The study focused on testing the benefits of an organizational intervention called ARC (Availability, Responsiveness, Continuity) that aims to improve the delivery of community-based mental health services. In so doing, the study examined MST alone as well as MST in combination with ARC.

Design:

Recruitment: The sample came from 14 of the poorest, least populated counties in the rural Appalachian region of eastern Tennessee. Between October 2003 and September 2007, youths referred to juvenile court in each of the 14 counties were recruited for the study if they met a list of 11 eligibility criteria, including ages 9-17, referral for a status offense or delinquent behavior, at risk of out-of-home placement, and a diagnosis of serious functioning problems or psychiatric symptoms. A total of 674 youth and their caregivers met the eligibility criteria and agreed to participate.

Assignment: First, randomization at the county level assigned six counties to the ARC intervention and six to no ARC intervention. Second, delinquent youth were randomly assigned within each county to either the MST program or to the usual services program. The second assignment used predetermined, concealed randomization of sequence numbers based on the order of recruitment. Youth assigned to usual services received a variety of inpatient and outpatient mental health services. The double randomization created four conditions that allowed for 1) the comparison of youth receiving MST (n = 349) with youth receiving usual services (n = 325) and 2) the county-level ARC intervention to moderate MST effects.

Assessments/Attrition: The four assessments occurred at baseline, six months, 12 months, and 18 months. Completion rates were 77% at six months, 69% at 12 months, and 52% at 18 months. According to the CONSORT diagram, however, the analysis included 84-88% of the participants who had data for at least two time points. With the MST intervention lasting an average of about 3.5 months, the 18-month assessment defined a long-term follow-up.

Sample: The youth sample averaged 14.9 years of age, was 69% male, and reflected the racial characteristics of the rural Appalachian region of eastern Tennessee (91% white). Over half (53%) of the youths had two or more mental health diagnoses.

Measures: The outcomes included two measures. Total youth behavior problems, a combination of internalizing and externalizing, was measured by the caregiver-rated Child Behavior Checklist (alphas ranged between .94 and .95 across waves). Youth out-of-home placements in state custody were measured by interviews with the youth's caregiver. As caregivers participated in but did not deliver the program, the measures appeared to be independent.

Analysis: The analysis used multilevel random-effects models with youth nested within counties, linear functions for the continuous outcome, and a logit link function for the dichotomous outcome. The equations for the models on pages 545-546 included county as a level-2 factor with random effects that adjusted for within-county clustering. Although not stated explicitly, the models imply the use of FIML estimation.

Missing Data Methodology: The analysis excluded only those participants with data for less than two time points and used mixed models with FIML to include others with incomplete data.

Intent-to-Treat: The analysis did not include all participants but included many with incomplete data.

Outcomes

Implementation Fidelity: Detailed analyses compared MST fidelity with and without the ARC. The findings indicated that the inter-organizational components of ARC facilitated greater therapist sensitivity to, and improvement in relations between, the family, community, and service system.

Baseline Equivalence: The authors reported only that there were no significant differences between ARC and non-ARC county means on total population, per capita income, and percentage of children in poverty. No figures or tests were presented for baseline differences in youth characteristics.

Differential Attrition: The authors reported that "a pattern-mixture model for the analysis of missing data found no differences in the effects of ARC and MST between youth with and without data at each wave" and that "no differences were found in the effects of ARC and MST between youth with and without data at each wave." Pattern mixture models typically use available data to identify missing data patterns and then include indicators of the patterns as covariates. Also, the use of imputation may moderate potential attrition bias.

Calculations based on the CONSORT diagram indicated attrition of 13-18% in the MST condition versus attrition of 10-14% in the control condition. Based on the overall attrition rate and the difference in attrition rates between conditions, the study met both the WWC cautious and optimistic standards at all assessments.

Posttest and Long-term: Over the 18-month follow-up period, out-of-home placements were significantly lower for the youths who received MST than for youths who received usual services (OR = .472). ARC did not significantly moderate the effect of MST.

Over the first six months and the next 12 months (18 months total), the measure of behavioral problems did not differ significantly between the MST group and the usual services group. However, moderation tests found that MST was significantly more effective than usual services in ARC countries in the first but not the second period.

Study 18

Summary

The study used a quasi-experimental design that examined 215 youths with serious behavioral problems in Washington State who had been referred to receive MST services. These youths were then matched to comparison youths who had an encounter with the public mental health system. Measures were obtained for criminal convictions and mental health services over one year.

The study found that, relative to a matched comparison group, the MST group showed significantly higher:

  • Utilization of mental health services.

Evaluation Methodology

Design:

Recruitment: The study began with adolescents in Thurston and Mason counties of Washington State who met the following criteria: ages 12 to 17 years, exhibited behavioral challenges and significantly interrupted functioning across multiple domains, and faced high risk of being placed out of home. Referrals came mostly from the juvenile justice or mental health systems (71%). Of 219 referrals made from April 2007 through June 2010, 215 enrolled in the study and 183 remained after removing duplicates and those without matches in the administrative database.

Assignment: In this quasi-experimental design, the 183 youths referred to MST served as the intervention group. Comparison youths were ages 9-17 and had an encounter in the public mental health system over the same study period used for the intervention youths. The study identified matched pairs of youths who were similar in age, gender, ethnicity, primary diagnosis, level of functioning, prior mental health service utilization, and criminal history. It was unclear if the comparison youths came from the same counties. For crime outcomes, 101 participants were successfully matched, and for mental health service utilization outcomes, 126 participants were successfully matched.

Assessments/Attrition: The study assessed outcomes over a one-year follow-up period. As it included the treatment time, the one-year assessment period did not include a long-term follow-up. The retrospective study used a sample of youths with existing data, and there was no attrition.

Sample:

The intervention group included 60% males, averaged 14 years of age (with a range from under 10 to 17), and was 83% white, 9% Hispanic, 5% Native American, 2% Asian and Pacific Islander, and 1% African American. About 68% had a misdemeanor or felony conviction in the previous year, and the majority were diagnosed with behavioral and disruptive disorders.

Measures:

Outcome measures came from administrative data gathered by multiple state agencies. The eight measures included criminal convictions (total, misdemeanor, felony, violent) and mental health services utilization (inpatient admission, outpatient treatment, support services, and crises services). The study provided no details on the reliability or validity of the measures.

Analysis:

The analysis used logistic regression with controls for age, gender, race, primary mental health diagnoses, level of functioning, and prior measures for the outcome of interest.

Intent-to-Treat: The analysis used all available data.

Outcomes

Implementation Fidelity:

The study stated that "MST . . . has met adherence standards throughout the project," but also that the program served some youths who did not fit the eligibility criteria.

Baseline Equivalence:

Before matching, MST youths were more likely than comparison youths to be involved in the criminal justice system, use intensive inpatient and other mental health services, be diagnosed with behavioral or attention deficit disorders, and have experienced multiple out-of-home placements. After matching (Exhibit A6.1, p. 7), the conditions still differed on five of 15 baseline measures.

Differential Attrition:

No attrition.

Posttest:

Tests showed no significant condition differences over the one-year follow-up period on any of the four criminal conviction measures (although the percentages favored the MST youth and the effect size for any conviction was -.10). Additional tests for use of mental health services found one significant intervention effect: the MST group had higher utilization rates of mental health support services. However, the author noted that increased use of support services could be characterized as a good outcome (MST promotes access to services) or a negative outcome (MST is associated with increased dependence on public services).

Long-Term:

Not examined.

Study 19

Summary

The study had no control group, instead offering a comparison of MST with Functional Family Therapy. It used a quasi-experimental design that examined 1,256 youths under the care of the Florida Department of Juvenile Justice who were referred to MST or Functional Family Therapy for treatment. The study matched participants in the two interventions using propensity scores (628 in each) and measured criminal offending during the intervention and over the next 12 months.

The study found that, after statistical correction for multiple tests, the MST group did not differ significantly from the matched Functional Family Therapy group on offending during the treatment period or during the 12 months after the treatment.

Baglivio et al. (2014) found no significant effects favoring the MST group compared to the matched Functional Family Therapy group.

Evaluation Methodology

Design:

Recruitment: The sample included all 2,312 juvenile offenders under the care of the Florida Department of Juvenile Justice who were referred to MST or FFT between July 1, 2009, and June 30, 2011. Excluding 109 youth who turned out to not qualify for the programs left 2,203 for the study. The statewide sample came from 10 MST sites and 11 FFT sites within 50 of Florida's 67 counties (both rural and urban for each service).

Assignment: The quasi-experimental design began with 629 youths referred to MST and 1,574 youths referred to FFT. Hundreds of probation officers made the referrals over the two-year period. With MST and FFT rarely available at the same time and the same sites, officers seldom had a choice between the two programs. Officers at all sites used the same form and the same inclusionary/exclusionary criteria for both programs. However, officers in sites with MST may have used different selection criteria for the program than officers in sites with FFT.

The study used propensity score matching to adjust for observed differences across the conditions with nearest-neighbor matching, no replacement, and a caliper of .05 (all analyses were replicated using a more restrictive caliper of .01 and nearest-neighbor matching without replacement). Predictors included gender, race, region of the state, age at admission to service, overall risk to reoffend, criminal history, a global social risk measure, most serious prior offense, age at first offense, antisocial peer association, alcohol use, drug use, and parental authority. Separate matching was done for the full sample, for males, females, whites, non-whites, low-risk youth, and moderate-high to high-risk youth.

The final matched sample included 628 youths in each condition (1,256 total).

Assessments/Attrition: The study examined data for the period of service and for the 12 months after the end of service. Service lasted about four months for MST and about three months for FFT. The retrospective data had no attrition.

Sample:

The matched sample was 69% male and 48% non-white.

Measures:

The two outcome measures came from state administrative records. First, a dichotomous measure of offending during service included any new criminal offense referral (arrest) or violation of probation while receiving or enrolled in MST or FFT. The longer time of service for the MST group could affect the offending measure.

Second, a dichotomous measure of recidivism captured whether the youth was adjudicated or convicted for an offense that occurred within 12 months of termination of MST or FFT (termination of service could be either successful or unsuccessful completion). The measure included data from adult records for youths who turned 18 years of age during the follow-up period. The recidivism measure, unlike the offending measure, used adjudicated/convicted offenses rather than arrests.

Analysis:

The analysis used t-tests with a Bonferroni correction for two outcomes (.05/2 = .025) in comparing the two matched conditions.

Intent-to-Treat: The analysis used all youth referred to the services, even if they did not successfully complete the program.

Outcomes

Implementation Fidelity:

The study noted only that the completion rate for MST of 71.9% was significantly lower than for FFT (76.5%).

Baseline Equivalence:

Many significant condition differences existed before matching, with MST having lower scores on percent male, risk, criminal history, and seriousness of priors. However, after matching, there were no significant differences in tests for 13 matching variables. Effect sizes for the differences were all below .25.

Differential Attrition:

No attrition.

Posttest:

With the usual significance tests, the FFT group had significantly lower offending during services than the MST group, but the effects fell to non-significance with a Bonferroni correction and with matching using a .01 caliper. Some evidence suggests that the significant effects favoring FFT were strongest for young women and low-risk youth.

Long-Term:

Recidivism for the 12 months after the end of treatment did not differ significantly across the two conditions.

Study 20

Summary

The study had no control group, instead offering a comparison of MST with Functional Family Therapy. It used a quasi-experimental design that examined 697 youths from a Dutch institute for behavioral problems who were referred to MST or Functional Family Therapy for treatment. The study matched participants in the two interventions using propensity scores and measured posttest externalizing as the primary outcome.

The study found that the MST group did significantly better than the matched Functional Family Therapy group on:

  • Engagement in school or work at posttest.

Evaluation Methodology

Design:

Recruitment: The sample consisted of 1,714 adolescents and their families who started either Functional Family Therapy (FFT) or MST at an institute for personality disorders and behavioral problems in the Netherland between October 2009 and June 2014. About 45% of the youths had a court order for treatment and were considered high risk. Of the 1,714 youths undergoing treatment, 697 completed the treatment and comprised the final sample.

Assignment: Adolescents with more serious problems were assigned to MST (n = 1074) and those with less serious problems to FFT (n = 640). After dropping those not completing the treatment, 422 youths (39%) remained in MST and 275 youths (43%) remained in FFT. To adjust for differences, the study used weights based on propensity scores. The covariates used in the propensity model (Table 1) included measures of sociodemographic characteristics, family relationships, and the baseline outcomes. Use of covariate missing values as predictors allowed for inclusion of all participants in the propensity score model.

The MST weight was 1, while the FFT weight was an odds ratio (i.e., the propensity score divided by one minus the propensity score). The propensity scores were estimated for the full sample and also for the subsamples of youths with and without court orders.

Assessments/Attrition: The posttest assessment occurred at the end of treatment. With data only on participants who finished treatment, 697 (40.7%) completed the primary outcome measures.

Sample:

Of the adolescents who completed MST, about 65% were male and about 81% were born in the Netherlands. They had an average age of 15.7 years.

Measures:

Externalizing as reported by both parents (Child Behavior Checklist) and youths (Youth Self Report) served as the primary outcome, with the measures having alphas of .94 and .88, respectively. The three secondary outcomes included 1) living at home (i.e., no out-of-home placement); 2) engagement in school or work for at least 20 hours per week at the end of treatment; and 3) new police contact due to inappropriate or illegal behavior during the treatment period. The therapists reported the three secondary outcomes as part of a standard scoring procedure that supervisors and consultants monitored.

Analysis:

The analysis used linear regression (with the propensity score weights) for continuous outcomes and logistic regression for binary outcomes. Tests of significance used 5,000 bootstrap samples to obtain non-parametric 95% confidence intervals.

Intent-to-Treat: Using only participants who completed treatment, the study examined a treatment-of-treated sample rather than an ITT sample.

Outcomes

Implementation Fidelity:

The mean adherence score within MST was .53, which was in accordance with the norm of ≥0.50 provided by MST Institute.

Baseline Equivalence:

Before matching, tests showed numerous significant differences and large standardized differences (d > .25) between the conditions (Table 1). After matching, the standardized differences all fell to below .25. Also after matching, the variance of key measures was similar across conditions. However, the authors noted that eight participants from the MST group and 12 participants from the FFT group were removed from the baseline tests because "there was no overlap" in the propensity scores.

Differential Attrition:

Those completing the treatment and having outcome data differed significantly from those who did not with regard to their country of birth, living situation, and whether or not they had a court order before treatment, as well as the level of education and employment status of their primary caregiver, and whether or not this primary caregiver had a partner.

Posttest:

One of five outcomes showed significant condition differences at posttest: A significantly higher proportion of adolescents who had completed MST versus FFT were engaged in school or work after treatment (d = .19). Tests for subgroup effects found that among adolescents without a court order, MST did significantly more than FFT to reduce both parent- and child-reported externalizing problems.

Long-Term:

Not examined.

Study 21

Summary

The study had no control group, instead offering a comparison of MST with an adapted version of MST for adolescents with intellectual disabilities (MST-ID). It used a quasi-experimental design that examined 63 Dutch adolescents with intellectual disabilities who were referred to MST or an adaptation of MST for adolescents with intellectual disabilities. The study matched participants in the two interventions using propensity scores and measured living arrangements, school/work, and police contacts at posttest and six-month follow-up as the primary outcomes.

The study found no significant effects favoring the MST-only group.

Evaluation Methodology

Design:

Recruitment: The sample, obtained in the Netherlands between March 2014 and October 2015, included families of adolescents with antisocial or delinquent behavior and with intellectual disabilities (i.e., an IQ score of 50 to 85). In addition, eligible participants were at least age 12 and were receiving no other treatments. Of the 214 families who met the inclusion criteria, 128 families (60%) consented to participate.

Assignment: Dutch agencies referred families to standard MST (n = 73) or MST‐ID (n = 55), a program that adapted MST for adolescents with intellectual disabilities. The alternate treatment was provided by two teams from one organization specializing in care for people with intellectual disabilities, while the standard MST was offered by 24 teams from seven Dutch organizations.

The study used propensity score weights based on 27 predictors to adjust for condition differences. All observed baseline variables, as well as missing indicators for all baseline variables with missing data, were included as predictors in the propensity score model. The missing data indicators allowed all families to be included in the propensity score estimation. The resulting propensity scores were used as weights equalizing condition differences. However, limiting the matched participants sample to those with overlapping propensity scores reduced the sample size from 128 to 63. The differences for 40 families in the MST group and 25 in the alternative group were too great to include in the analysis (i.e., scores did not fall in the range of scores observed in the other treatment group).

Assessments/Attrition: Assessments occurred at baseline, posttest, and six-month follow-up. Six months after treatment, 87 families (68%) provided data. As noted above, additional losses came from non-matching propensity scores, which reduced the sample to 63 families (49%).

Sample:

Across the two conditions, the percent female in the sample ranged from 36-44%. The average ages were 14.9 and 15.2. About 95% of the sample was born in the Netherlands.

Measures:

The outcome measures came from the reports of therapists and parents at baseline and posttest and from parents at follow-up. The three primary outcomes included if the adolescent was living at home, attending school or working for at least 20 hours a week, and free of police involvement since the start of treatment. At posttest, therapist reports were monitored by the team supervisor and the MST consultant.

The study also examined eight secondary outcomes at posttest relating to parenting, family relations, and youth externalizing. At the follow-up, only the rule-breaking component of externalizing was examined. The study reported no information on reliability or validity for the secondary measures.

Analysis:

The analysis used logistic regression for dichotomous outcomes and linear regression for continuous outcomes. Bootstrapping was used to calculate 90% confidence intervals for all outcome measures, with 5,000 bootstrap samples drawn from the weighted sample.

Intent-to-Treat: The study did not drop participants for not completing the treatment, though many subjects were lost in the matching.

Outcomes

Implementation Fidelity:

On a scale ranging from 1 (not at all) to 5 (very much), the average therapist adherence score was 4.38 for standard MST.

Baseline Equivalence:

Using the full sample of 128, there were five significant condition differences before matching. After matching, significance tests were not reported, but all baseline variables as well as the missing indicators used in the propensity score estimation had standardized mean differences lower than 0.25. This result came from excluding the 65 families with non‐overlapping propensity scores and using the analysis sample of 68.

Differential Attrition:

The study did not test for differences between those with and without posttest and follow-up data.

Posttest:

All significant effects favored the alternative treatment. Based on a 90% confidence interval, none of the three primary outcomes differed significantly at posttest, while one, living at home, favored the alternate treatment at six-month follow-up. Several secondary outcomes were significantly better for the alternate treatment group.

Long-Term:

Not examined.

Study 22

Summary

The study used a quasi-experimental design that examined 740 youths referred to MST by the state of Rhode Island. The intervention group completed the treatment, while the comparison group did not complete the treatment for reasons unrelated to case progress. After using propensity scores to match the two conditions, the study examined delinquency outcomes over the next six years.

The study found that the intervention group relative to the comparison group showed significantly lower risks for

  • Out-of-home placements
  • Adjudications
  • Juvenile training school.

Evaluation Methodology

Design:

Recruitment: The sample came from five MST providers that were active in Rhode Island from 2008-2011. A total of 772 youths ages 11-17 were referred to MST by either Child Welfare Services or Juvenile Correctional Services in Rhode Island, primarily for reasons of behavioral problems, disobedience, truancy, and delinquency. The youth who received an out-of-home placement prior to initial program contact were excluded, leaving a final sample of 740.

Assignment: The treatment condition included all youth referred to MST who either completed treatment or had the opportunity to do so. The latter group included youth who were unable to complete treatment due to either lack of engagement or a placement occurring while involved in treatment. Of the total sample of 740, 78% (n = 577) met criteria for inclusion in the treatment condition, with 90% (n = 522) completing treatment, 8% (n = 46) discharged due to placement, and 2% (n = 9) discharged due to lack of engagement. The median length of stay in the program for this treatment group was 4.4 months. The comparison group included the remaining 163 youth (22%) who were referred to MST but did not have the opportunity to complete treatment as a result of reasons unrelated to case progress. Of these, 94% (n = 154) were removed from treatment due to administrative or funding/referral reasons and 6% (n = 9) moved out of the program's service area. The median time for the comparison group from initial program contact to program discharge was 1.45 months.

The study used propensity score matching to balance pre-treatment characteristics of youth across treatment conditions. After comparing multiple methods of matching, the authors settled on the Kernel-based matching approach with a bandwidth of 0.08 and a common support restriction. The covariates included age, gender, race, number of prior episodes of out-of-home care, number of placements, history of maltreatment, history of adjudication, history of juvenile training school placement, and case assignment prior to or during MST treatment. The matched sample lost 20 participants from outside the common support region.

Assessments/Attrition: The study outcomes were examined over a six-year period after the initial program contact. The retrospective use of pre-existing data meant there was no attrition.

Sample:

The majority (57%) of youth were male, and the average age was 14.9 years at the time of referral. For self-reported race and ethnicity, 49% were White, 14% Black, 29% Hispanic, and 8% other or multiracial. In terms of case history, 36% of youth had a history of maltreatment, and more than half (57%) had a history of out-of-home placement. More than one-quarter of youth had previously been adjudicated (26%) and placed in a juvenile training school (27%).

Measures:

The data came from administrative records of the MST providers and the Rhode Island Department of Children, Youth, and Families. The three outcomes measured the time-to-event for youth of out-of-home placement (i.e., removal from parental custody due to reasons such as child behavior or disability, parental inability to cope, abuse or neglect), adjudication (i.e., court determination that the youth committed a crime or delinquent offense), and placement in a juvenile training school.

Analysis:

The analysis used Cox proportional hazards regression models to examine the risk of removal and delinquency outcomes following initial contact with the program. Youth who did not experience the outcome were censored at the end of the study observation period. For youth who experienced a subsequent MST referral or treatment prior to the end of the study observation period (n = 48), the outcomes were censored at the beginning of the second MST referral for treatment. In addition, youth who turned 18 prior to event occurrence or the end of the study period were censored on their 18th birthday. The covariates included prior history for the outcomes.

Intent-to-Treat: The study used all referred participants who began the program, though 4% were excluded because they experienced the outcome before the program began.

Outcomes

Implementation Fidelity:

No quantitative information.

Baseline Equivalence:

Prior to matching, significant differences were found between the treatment and comparison groups on race/ethnicity, the number of removals from parental custody, and the number of out-of-home placement settings in the first removal episode. Relative to the comparison group, the treatment group had fewer racial and ethnic minority youth, experienced fewer out-of-home care episodes, and had fewer placement transitions in their first episode of care. After matching, no significant differences were found between the treatment and comparison groups on these covariates.

Differential Attrition:

No attrition.

Posttest and Long-Term:

The Cox survival models showed significantly lower risks for all three outcomes - out-of-home placement, adjudication, and juvenile training school - in the intervention group than the comparison group. The hazard ratios ranged from .57 to .61.

Study 23

Summary

The intervention examined in this study adapted MST principles and techniques for youth with an autism spectrum disorder. The study used a randomized controlled trial that assigned 15 youths with an autism spectrum disorder and severe disruptive behavior to an MST condition or a usual-services condition. Measures of conduct problems, internalizing, and aggression were obtained at six months and 12 months after recruitment.

The study found that the MST intervention group showed significantly better family adaptability at six- and 12-months post-recruitment than the usual-services control group.

Wagner et al. (2019) found that, relative to the usual-services control group, the MST intervention group showed significantly better:

  • Family adaptability at six- and 12-months post-recruitment.

Evaluation Methodology

Design:

Recruitment: The sample came from families and youths registered at an interdisciplinary academic medical center specializing in the diagnosis and treatment of youths with an autism spectrum disorder. Eligible youths had a confirmed autism spectrum disorder, had recently evidenced severe disruptive behavior, and were 10 to 17 years of age. Of 42 families screened for eligibility, 17 were actually eligible and consented to participate and two withdrew during the assessment phase (due to identified factors that affected their eligibility).

Assignment: The 15 participating families were randomized (with a 1:1 ratio using a computerized random number generator) to the MST (n = 8) or usual community services (n = 7) conditions. The control group received a variety of services, but all focused on the individual youth rather than on the systems in which the youth was embedded.

Assessments/Attrition: Assessments occurred at baseline, six months, and 12 months after recruitment. The completion rates were 87% (n = 13) at six months and 73% (n = 11) at 12 months. The 12-month assessment referred to the period after recruitment rather than after program completion and therefore did not define a long-term follow-up.

Sample:

The sample youths had a mean age of 13.8 years. They were 87.5% boys, 75% white, 6.25% African American, 12.5% Asian American, and 6.25% Hispanic/Latino. About 62.5% lived with two caregivers (e.g., biological parents, stepparents).

Measures:

The outcomes, all from caregiver reports, included measures of youth conduct problems, internalizing, family relations, and peer relations. Reliabilities for the measures were high but came from reports of other studies rather than from the sample.

Analysis:

The analysis used analyses of covariance models with a time-by-group term to test for condition differences in changes from baseline to posttest and follow-up. Baseline outcomes were included as part of the time variable and sometimes as covariates as well. Supplemental analyses replaced missing data by carrying the last observation forward.

Intent-to-Treat: The four families that did not complete all three assessments were excluded from primary analyses, but supplemental analyses estimated program effects for the full sample by imputing missing data.

Outcomes

Implementation Fidelity:

Not examined.

Baseline Equivalence:

Tests showed no differences in demographic characteristics. However, caregivers in the MST condition reported higher levels of caregiver stress, lower levels of family adaptability, and more youth conduct problems than did caregivers in the control condition.

Differential Attrition:

Retention was greater in the MST group than the control group at both follow-ups: 100% versus 71.4% at six months and 87.5% versus 57.1% at 12 months. However, the authors reported that "no between-condition differences on demographic or psychosocial measures were found."

Posttest:

One significant effect emerged in 13 tests: The MST group scored better than the control group on family adaptability. Several marginally significant effects also favored the MST group: youth emotional bonding with peers, intensity of youth aggression, and family cohesion. As reported by the authors, the results remained unchanged when imputing missing data and using the full sample.

Long-Term:

Not examined.

Contact

Blueprints for Healthy Youth Development
University of Colorado Boulder
Institute of Behavioral Science
UCB 483, Boulder, CO 80309

Email: blueprints@colorado.edu

Sign up for Newsletter

If you are interested in staying connected with the work conducted by Blueprints, please share your email to receive quarterly updates.

Blueprints for Healthy Youth Development is
currently funded by Arnold Ventures (formerly the Laura and John Arnold Foundation) and historically has received funding from the Annie E. Casey Foundation and the Office of Juvenile Justice and Delinquency Prevention.