Positive Family Support

Study 1

Evaluation Methodology

Cohort 1
Design (Dishion et al., 2002; Dishion, Nelson, and Kavanagh, 2003):

Recruitment: Using the revised model of the Adolescent Transitions Program (ATP), 672 students and their families were recruited from a population of sixth grade students from three middle schools in a metropolitan area of the Pacific Northwest. School principals distributed letters to parents to introduce the study.

Assignment: Most (95%) of the population gave consent and were randomized, by student, to an experimental or control group. In 7th grade, all experimental students were assigned to the same homerooms so the SHAPe program could be implemented. Parents of intervention youths were also targeted for Level Two (Family Check-Up) programming.

Attrition: Eighty-five percent participated in all assessment waves, which occurred yearly from grades 6 through 9. However, the sample was reduced by 216 children due to missing data. This N of 460 represents a 68% retention rate. Students were assessed through mail and in the classroom and were paid $20 for completing each of the first 3 waves and $30 for completing the fourth assessment. Teacher-ratings were collected as was information from the Parent Consultant on parent and/or family involvement.

Risk Group Formation: Researchers utilized a multiple-gating procedure to identify participants for the second and third program levels. Teacher-ratings of risk, using a 16-item measure, were scored to define an at-risk (AR) level and a high-risk level for each individual. These items were put into a hierarchical cluster analysis using squared Euclidean distance as the index of similarity. Two groups emerged: at-risk (AR) and typically developing (TD). One hundred thirty-seven were identified post hoc as AR; 71 (22.1%) were in the treatment group and 66 (20.4%) were in the control group.

Sample:

The schools were located in an ethnically diverse metropolitan area in the Northwest. For the overall sample of 672 students, 52% of both the experimental and control groups were male and 41% of both groups were European American. Of the experimental group, 34% were African American and 6.9% were Hispanic, compared to 30.5% and 7.6% of the control group, respectively. There were no differences between groups in terms of percentage of AR students.

As reported in a 2003 publication that evaluates the Family Check-Up level with a randomized design, treatment and control youth were different on some demographic variables: 31% of FCU youth were male versus 47% of controls; 26% of FCU youth were European American versus 39% of controls; 57% of FCU youth were African American, 6% were Hispanic, and 11% were mixed, compared to 44%, 0%, and 17% of controls, respectively; 28% of FCU youths were from married families compared to 39% of controls. The average number of children was 2.7 for FCU families (2.6 for controls). Fourteen percent of FCU families received Food Stamps compared to 25% of controls. The authors report these differences were not statistically reliable but no statistical information was presented.

A high-risk subset of this population was used to examine parental monitoring as a mediator of substance use outcomes (Dishion et al., 2003). The 150 youth were identified by teachers as high risk using the TRISK, and 75 (50%) agreed to participate in videotaped home assessments in grades 7, 8, and 9. Ultimately data were available for 75 families (35 in the Intervention group and 36 in the Control group).

Measures:

Teacher-ratings were measured with a 16-item Teacher Perception of Risk (TRISK) scale. Self-reported alcohol, tobacco, and marijuana use by students were also assessed with measures of frequency of use in the last month. Measures of antisocial behavior were included in the combined cohort 1 and 2 assessment and these measures included items of self-reported problem behavior - frequency over the past month on six items (lying to parents, skipping school, staying out all night without permission, stealing, panhandling, and carrying a weapon). Intervention engagement was measured by Parent Consultant-collected data on minutes spent in contact with the Parent Consultant, the purpose of the contact (measuring intensity), and method of contact (in-person or by phone). Videotaped family assessment allowed for a measure of parental monitoring.

For the full sample assessment studies (n=998), youths also self-reported on deviant peer involvement, parental monitoring, and family conflict. Arrest records were accessed with parental permission to identify status violations, misdemeanors, and felonies. At age 19, youths also completed the Composite International Diagnostic Interview to report on lifetime substance abuse diagnoses. On the measure of depressive symptoms (Connell and Dishion, 2008) for the identified high-risk students (n=106), additional measures were collected. These included the self-reported Child Depression Inventory (CDI)and the maternal Child Behavior Checklist (CBCL) for internalizing and externalizing behavior. These assessments were collected in the fall of the child's seventh, eighth, and ninth grades. Academic achievement and attendance were measured using student GPA (grades 6 - 11) and attendance records.

Analysis:

Substance use outcomes were generated using logistic regression. Data are presented separately for AR & TD youth, as well as cumulatively, with ninth grade substance use as a dependent variable. Mediating effects of parent monitoring were determined through correlation and logistic regression.

Cohorts 1 and 2
(Connell et al., 2006; Connell et al., 2007; Connell and Dishion, 2008; Stormshak et al., 2009; Connell, 2009; Connell et al., 2017)
This study combined the original cohort above with an additional cohort of adolescents and their families, for a combined total sample population of 998. The study design and measures are described above. Various outcome measures and analytical strategies were used to determine both immediate and long-term program effects.

Sample:

This sample included 998 adolescents and their families, who were randomly assigned to either the treatment (n=500) or control (n=498) conditions. The majority of the adolescents were male (52.7%). The sample was ethnically diverse, with 42.3% Caucasian, 29.1% African-American, 6.8% Latino, 5.2% Asian, and 16.4% other ethnicity youth. In 58.6% of families, biological fathers were present. Approximately 80% of the youths were retained in the assessment throughout each wave of the study. In the intervention condition, 115 families (23%) elected to receive the FCU and 88 of these progressed to the third level to receive further intervention services. For cohort 1, 46% of FCUs were completed following the seventh grade family assessment, 53% were completed following the eighth grade family assessment, and 1% was completed following the ninth grade family assessment. In cohort 2, most (93%) FCUs were completed following the seventh grade family assessment and the remainder following the eighth grade family assessment. These families had an average of 8.9 direct contact hours with intervention staff. There was also a long-term diagnostic interview assessment (Connell et al., 2017), conducted when the individuals were aged 28-30 years. A total of 754 (76%) youth were retained for the long-term assessment.

A high-risk sample of adolescents (n=106) was identified using the multiple gating approach to risk assessment (TRISK), with problem behaviors including aggression, moodiness, oppositionality, peer relationship problems, and school problems. Students who scored 3 or higher or whom teachers suspected of substance use were designated high risk. These students completed more comprehensive assessments, including the measurement of depression symptoms. There were 52 youths assigned to the control condition and 54 in the intervention condition. Most parents (88.9%) of the high-risk youth participated in the universal level of the intervention, and of these, 60% also completed FCU and the linked intervention services through the family resource room. These families received an average of 7.89 hours of services over the course of 3 years, from seventh through ninth grade.

A subset of the combined cohort population was used to examine substance-use patterns in grades 6-9 (Connell et al., 2006). This subset included 698 youth and their families who self-identified as either Caucasian (59.6%) or African-American (40.4%). These two subgroups were chosen because the numbers in each were sufficient to conduct analyses for ethnicity effects.

Measures:

For the long-term follow-up analysis (Connell et al., 2017), depression was diagnosed using the Composite International Diagnostic Interview (CIDI), based on criteria from the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV). Individuals were assessed for past year and lifetime major depressive disorder diagnoses. It was unclear if the diagnostician was aware of condition.

Analysis:

A technique known as Complier Average Causal Effect (CACE) analysis was used on several outcome measures to limit the amount of potential bias introduced in the results from an intent-to-treat analysis among the families that voluntarily participated in the selected and indicated levels of the program. This type of analysis identifies a subset of the randomized control group who resemble those who do actively participate in the voluntary intervention, which should provide a more accurate picture of how youth receiving the FCU would have developed without intervention. A mixture-modeling framework is applied to identify the optimal comparison group from the control condition to compare with intervention engagers in the intervention condition. The Connell et al., 2007 analysis focused on separate substance use types and antisocial behavior through grade 12. CACE analyses were conducted as mixture models and used full information maximum likelihood estimation to account for missing data. The assumption was that for noncompliers in either the intervention or control condition (i.e., never-takers or always-takers), the distribution of potential outcomes was independent of the treatment assignment. Connell (2009) also used CACE analysis to examine the long-term impact of ATP on trajectories of adolescent tobacco use from early adolescence through early adulthood, using 7 assessment waves when youth were ages 11, 12, 13, 14, 16-17, 18-19, and 22 (at wave 7, only cohort 1 data were available). Stormshak and colleagues (2009) used this technique to examine program effects on academic achievement and school attendance.

The primary analysis for youth depression outcomes (Connell and Dishion, 2008) were intent-to-treat analyses using two latent growth models (LGM). Analyses also used full information maximum likelihood estimation (FIML) to accommodate missing data. Youth-reported or mother-reported conduct problems at each wave were used as time-varying covariates in the model. CACE analyses were also conducted in order to examine the possibility that the intervention was largely driven by familial participation in the selected and indicated levels of intervention.

The study of trajectories of early adolescent substance use (Connell et al., 2006) used three levels of analyses. First, a zero-inflated Poisson (ZIP) growth model using M-plus 3.01 was used to examine predictors of change in substance use over early adolescence. Next, latent growth mixture modeling (LGMM) using M-plus 3.01 was used to examine heterogeneity in developmental trajectories of early adolescent self-reported substance use from a person-centered perspective. Finally, a logistic regression framework was used to examine the predictors of trajectory class membership, with results comparing the extent to which each predictor was related to the likelihood of being in a given trajectory relative to a normative trajectory.

In the long-term follow-up analyses, logistic regression models were used to examine the intervention's effect on depression diagnoses at age 28-30, but without controls for baseline depression. Follow-up analyses for significant interactions were carried out by subsetting the data by the median split on the moderator variable, and then examining intervention effects in the subsamples.

Intent-to-treat: Although the primary results presented are based only on individuals with long-term data, intent-to-treat analyses were also carried out in a subsequent analysis, with missing data accounted for using multiple imputation. The authors reported that the results from both analyses were virtually identical.

Outcomes

Cohort 1 studies
Dishion et al., 2002:
Intervention Engagement: Two groups emerged from TRISK analysis, the AR (at-risk) group and TD (typically developing). Typically developing students were those not at-risk. The program staff made more contacts with AR families (on average, 13) than TD families (on average, 7). The total personal contact time was brief - only 5.6 hours for AR families and 3.1 hours for TD families. Only 3 control families sought FRC services during the program period and these families, according to a decision the research group made prior to the study's initiation, were afforded services they wanted.

Intervention Outcomes: There is very little instance of substance use, especially for TD children, until grade 9. There is a significant program effect on reductions in grade 9 substance use, controlling for use in prior grades, favoring the ATP group. There was not a significant difference between AR youth and TD youth, indicating that the program produced significant effects regardless of risk-level. Effects did not vary by gender or ethnicity.

Dishion, Nelson, and Kavanagh, 2003: Using a directional test, the intervention, relative to the control condition, reduced grade 9 substance use among higher risk students.

Mediation Model: Intervention families decreased parental monitoring from seventh to eighth grade, but increased monitoring between eighth and ninth grade. Control families decreased parental monitoring slightly from seventh to eighth grade and continued to decrease monitoring more sharply from eighth to ninth grade. After controlling for prior substance use, the intervention condition was associated with a significant lower probability of substance use. High-risk youth in the control group were more likely to report substance use than intervention youth. Results of the mediated model show an increase in parental monitoring significantly accounted for a reduction in substance use in ninth grade, controlling for previous levels of substance use and random assignment.

Combined cohorts 1 and 2 studies
The Connell et al., 2007 study discriminated between families within the intervention group who engaged in the FCU from those who did not (engagers vs. non-engagers). They found that treatment engagement was significantly related to the likelihood of biological fathers being absent from the home, youth reports of elevated family conflict and deviant peer affiliation, and teacher reports of elevated risk behaviors at school. That is, the families that had the highest level of vulnerability and experienced the most difficulty were more likely to engage in treatment. This finding is significant in itself, as it demonstrates that the intervention was able to successfully engage and motivate the at-risk and high-risk portion of the larger sample to participate in the targeted intervention phases. Female gender predicted a higher level of engagement for tobacco and marijuana use models, and deviant peer affiliation also predicted engagement in the tobacco use model. Those who engaged in treatment showed significantly inhibited growth in substance use and antisocial behavior from age 12 to 17, compared to the control group. The intervention also had significant effects on arrest rates, and likelihood of being diagnosed with alcohol, tobacco, or marijuana use disorders by late adolescence. Long-term tobacco outcomes (Connell, 2009) indicated that treatment engagement predicted significantly less growth in use by age 22. Effect sizes at age 22 were between 1.18 and 1.27, depending on the model, which used either a moderate or stringent definition of "engagement" (completion of FCU only (25.5% of families) or both FCU and follow-up services (19.5% of families), respectively). In this study, family engagement was predicted by female child and elevated peer deviance.

The depression analysis conducted with 106 high risk youth from Cohorts 1 and 2 (Connell and Dishion, 2008) found that treatment effects did not emerge until the youth were in ninth grade, when there were significant reductions in CDI scores for treatment youth compared to control group youth at that time. By ninth grade, there were significant differences in the number of youths in the clinical range between treatment and control group (effect size = .56). On mother-reported youth symptoms, the intervention produced a small to medium sized difference in depression symptoms at ninth grade (.42). CACE model results indicate that youths' self-reports of growth in depressive symptoms were significantly lower for "compliers" (63.2% of the sample) than youth in the control group, with a treatment effect size of 1.35. Also, mother reports of youth compliers (65.3%) reported significantly less growth in youth internalizing symptoms than mothers of youth in the control group, with an effect size of 1.07.

On the measure of academic outcomes (Stormshak et al., 2009), intervention engagement reduced growth in school absences and declines in GPA from 6th through 11th grade. Engager vs. non-engager intervention status was as follows: On the measure of academic performance, sixth grade GPA in the non-engager class was positively related to male gender, the presence of a biological father in the home, and to parental monitoring efforts. There was a negative association with African American or Hispanic ethnicity, elevated family conflict, and teacher perceptions of engagement in risky behavior. For engagers, sixth grade GPA was negatively related to teachers' perceptions of school risk behavior. On the measure of school absences in sixth grade, non-engager intervention youth were more likely to come from homes without a biological father present, low parental monitoring, and teacher report of school risk behavior. For engagers, sixth grade absences were positively related to teachers' reports of risk behavior. Hispanic youth who participated in the intervention showed greater declines in GPA and greater acceleration in the number of school absences over time compared to non-Hispanic youth who participated in the intervention.

The examination of the trajectories of early adolescent substance use (Connell et al., 2006) found that about half (53.1%) of the students reported no substance use over the course of the study. The likelihood of initiating substance use in sixth grade was related to deviant peer affiliation and less-effective parental monitoring. Among those youth who indicated having used a substance at least once between sixth and ninth grade, those in the intervention condition had higher initial (sixth grade) substance use and showed significantly less growth in substance use over time, compared to youth in the control condition. There were also gender differences, with program impacts in slowing growth in substance use as significantly greater for girls compared to boys. Five trajectories also emerged, representing growth parameters across classes and the predictors of trajectory class membership. The majority of youth were classified as no-use (51.82%) or low/rare-use (25.79%). The remaining classes of youth who reported substance use included a decelerating class (7.02%), youth who exhibited significant levels of substance use in sixth grade and significant decreases in the level of use over time; an early accelerating class (6.30%), youth who exhibited significant levels of substance use in sixth grade, and also reported significant growth in substance use over time; and a late-accelerating class (9.03%), youth who exhibited a burst of substance use during the transition from middle to high school (eighth to ninth grade). Of these youth who exhibited higher levels of initial substance use, those assigned to the intervention reported more dramatic declines and were disproportionately more likely to engage in the most intensive form of treatment relative to any other trajectory class. The youth were more likely to be Caucasian and to have reported high levels of sixth grade deviant peer affiliation.

In Caruthers et al. (2014), growth curve modeling was used to determine direct and indirect program effects over time on high risk sexual behavior (HRSB) by age 22. Key variables examined were family relationship quality and parental monitoring as mediators, as well as early sexual activity. High risk sexual behavior was the average of five dichotomous items (sexual partners with multiple partners, familiarity with partner, condom use, sexually transmitted disease, and pregnancy). Differential effects of race and gender were also examined as moderators. There was no direct program effect on HRSB, but there was a significant indirect program effect on HRSB with family relationship quality as an intervening variable. There was also a significant effect of family relationship quality on HRSB mediated by differences in parental monitoring and early sexual activity. The effects of the mediators were varied based on gender and race.

Long-term follow-up of combined cohorts (Connell et al., 2017):

Implementation Fidelity:

The families that received the Family Check Up intervention were in contact with intervention staff for an average of 8.9 hours over the course of the study, but no other information was given on attendance or implementation fidelity.

Baseline Equivalence:

There was no information on baseline equivalence.

Differential Attrition:

The authors note that "preliminary analyses suggested that attrition was consistent with Missing at Random assumptions," even though overall attrition was fairly high (24%) by the long-term follow-up.

Long-Term:

Connell et al. (2017) found no effect of intervention on past-year depression diagnoses or on lifetime depression diagnoses. They tested for multiple moderation effects, finding some program benefits for subjects with low baseline family conflict and with low baseline GPA.

Study 2

Evaluation Methodology:

Design: This study, called Project Alliance 2, focused primarily on home-to-school linkages and the family-school partnership and outcomes related to academic success and school engagement. The evaluation consisted of an auto-regressive design over four waves of data and targeted youth from 6th - 9th grade. The design of the intervention followed the three level model of the program. In the first level, families had access to a parent consultant through the Family Resource Center. The consultant provided services to families within the school context, including attending behavioral support meetings, teacher meetings, and any other important school meeting related to child behavior. At the second level, families participated in the Family Check-Up (FCU) protocol, which included an initial home visit and interview, an assessment and video-taped parent-child interaction session, and a feedback session. At this point, families decided either to decline further services, or participated in level three, in which consultation was provided for further support and training services.

Sample: A sample of 377 adolescents and their families was drawn from three public middle schools in an urban area. Researchers used an unbalanced approach to randomization in order to enhance the power of intervention effects specifically for the families that elected to utilize services tailored for their needs. More of the sample (n=277, 73%) was assigned to the treatment condition, while the remaining 100 families (27%) were randomly assigned to the school as usual condition. The majority of adolescents were male (51%) and participants were ethnically diverse (36% White, 18% Latino/Hispanic, 16% African American, 8% Asian, 3% American Indian, and 19% biracial/mixed ethnicity). Average household income was between $30,000 and $40,000.

Across the 4 years of the study, approximately 80% of youth remained in the study. Families who moved were tracked and followed, via mailed surveys and phone interviews. Of the 277 students in the intervention condition, 46% received parent consultation services and 38% participated in the full FCU intervention (24% of these received additional services following feedback). The median amount of time families participated in FCU was 168 minutes, with most contacts occurring when the child was between 7th and 8th grade (80%). More families with a male child (34%) participated in the FCU level, compared to families with a female child (30%). There were no differences by gender or for outcome variables of self-regulation, depression, or school engagement.

Measures: Students were surveyed at school each spring by completing a questionnaire. Self-regulation was measured using items from the Early Adolescent Temperament Questionnaire. Depression was measured using items from the Child Depression Inventory. School engagement was measured using two items that reflect enjoyment of learning and effort to achieve.

Analysis: An intent-to-treat analysis was conducted to evaluate generalized effects of the FCU in an autoregressive model. Three analyses were conducted. The first tested the possibility that treatment group assignment was associated with increases in self-regulation in seventh grade, accounting for sixth grade self-regulation. Next, the extent to which seventh grade self-regulation predicted changes in depressive symptoms in eighth grade was tested. Finally, eighth grade self-regulation and depressive symptoms as predictors of youths' academic engagement in ninth grade was examined.

Outcomes: There was a significant effect of the intervention on overall youth self-regulation between sixth and seventh grade. Seventh grade self-regulation was also associated with decreases in youths' depressive symptoms from sixth to eighth grade, with a small to medium effect size (-.25). Eighth grade self-regulation was associated, with a medium effect size increase (.37), with school engagement by ninth grade. There were no significant gender differences.

Limitations: Limitations include using adolescent self-reports as the primary measure for outcomes; this may have created some bias based on social desirability of reporting.

Study 3

Evaluation Methodology

Design: Participants were 593 adolescents and their families from three urban, public middle schools in the Pacific Northwest. Families were randomly assigned to either the intervention (n=386; 65%) or control (n=207; 35%) condition. Youth and families assigned to the control condition experienced "business as usual" school and did not have access to the intervention services available to the intervention condition families.

To increase the level of engagement of families in the intervention condition, all intervention families were invited to participate in Family Check-Up (FCU) services, whether or not their child was identified as at risk. Intervention families who engaged in the FCU services participated in a three session assessment and feedback process. Parent consultants used motivational interviewing techniques to determine the family dynamic and assess needs. During the feedback session, the families were presented with a list of intervention options and parenting strategies, as well as other school- or community-based resources to address the specific needs of the family. Families were free to accept or decline the services. The parent consultants used for this study were trained researchers (not school personnel) and, wherever possible, parent consultant ethnicity was matched with family ethnicity. Consultants received intensive training and supervision over the three year course of the study.

Utilization of FCU services was as follows: 51% of intervention families received consultation from a parent consultant and 42% received the full FCU intervention. Of those receiving FCU intervention, 78% received additional follow-up assistance after the feedback, including parent skills training, and address of education-related concerns as needed (homework, attendance, grades, etc.). Intervention families received an average of 94.2 minutes of intervention time.

Sample Characteristics: Students were in 6th grade when the intervention began. There were slightly more male than female participants (51.4% vs. 48.6%), and the racial/ethnic makeup of the sample was diverse: 36.1% White, 18.0% Latino/Hispanic, 15.0% African American, 7.1% Asian, 2.4% American Indian/Native American, 1.9% Pacific Islander, and 19.3% reporting mixed ethnicity/biracial.

Measures: Measures were collected from students in the spring of each academic year from 6th through 8th grades and the survey tool used was developed at the Oregon Research Institute. Primary outcome measures assessed from this tool included measures of antisocial behavior, deviant peers, and substance use (cigarette, alcohol, marijuana). Additionally, the Effortful Control scale was used to assess self-regulation. Reliability across all measures and waves ranged from .72 - .86.

Analysis: Structural equation modeling was used, using full information maximum likelihood estimation to reduce bias resulting from missing data. An alpha of .05 was used. Standard measures of fit were reported for antisocial behavior and deviant peer affiliation, including the comparative fit index, standardized root mean square residual, and root mean square error of approximation. Because random assignment occurred within schools, analyses were conducted at the individual level. Analysis was intent to treat.

Outcomes

Fidelity Monitoring: Supervision of the parent consultants by the program developers was ongoing throughout the study, and included monitoring of videotaped FCUs, feedback to consultants, planning for FCUs, conducting role plays, and providing guidance for use of the family management curriculum.

Posttest: Results indicate significant program impact on increases in self-regulation for intervention students from 6th to 7th grades across all growth models, compared to students in the control condition. There were then small to moderate associations with growth in all primary outcomes. Self-regulation was associated with less growth in antisocial behavior (ES=.12) and deviant friendships (ES=.18). There was a stronger association between higher levels of self-regulation and less growth in substance use, with effect sizes of .30 (cigarettes), .22 (alcohol) and .39 (marijuana) from 6th to 8th grades. There was a gender effect on deviant friendship growth for girls (ES=.12).

Limitations: Data collection measures were single informant, relying on responses only from students. Also, effect size for the intervention was small, partially due to the chosen analytical approach (to determine the average benefit at the school level), which may have resulted in an underrepresentation of the effect size, as less than half of the intervention group did not engage in the FCU. Also, the parenting interventions did not directly target effortful control (self-regulation), also potentially limiting the effect size. Randomization occurred at the student level to avoid nesting effects, so schools were limited in their implementation of the universal-level component of the program (as students could be in either condition), potentially limiting the effects of that program component. The parent consultants were trained researchers.

Study 4

Evaluation Methodology

Design:

Smolkowski et al. (2017) conducted a cluster randomized control trial with 41 middle schools in which schools were blocked at the district level. The treatment schools received the three-level Positive Family Support intervention, while the control group delayed receiving the program for three years. The authors followed two cohorts of students. Cohort 1 entered sixth grade immediately after randomization, 1 year before implementation of the treatment in intervention schools. Cohort 2 entered sixth grade the following year, 1 year after randomization and during the first implementation year in treatment schools. Thus, within treatment schools, all students in Cohort 1 finished sixth grade before implementation and all students in Cohort 2 began sixth grade after implementation. Each cohort was followed through eighth grade. Treatment students in both cohorts experienced 2 years of exposure, but insufficient time prohibited schools in Cohort 2 to complete the final wave of data collection. Smolkowski et al. (2017) therefore reported results for students in Cohort 1. Though not clearly stated, it appeared that student-, parent- and teacher-reported data were collected among Cohort 1 students at baseline (i.e., the beginning of seventh grade), with 2 follow-ups (one at the end of seventh grade and another at the end of eighth grade). Because the intervention was two years, however, the posttest was at the end of eight grade.

Recruitment:

Schools with a minimum of 50 students in the sixth grade that had implemented school wide Positive Behavioral Interventions and Support systems were eligible for the study. Approximately 200 schools in the Pacific Northwest were identified that met eligibility criteria. The parents of 14,331 students received a request for dissent or consent, with the parents of 1,419 students having actively dissented to participation, thus leaving 12,912 participating students across both cohorts and all data collection time points (in which there were four rounds of data collection in total) from 41 middle schools in 27 school districts.

Assignment:

Project staff recruited and randomized 14 schools before the 2009-2010 academic year, 13 schools before the 2010-2011 academic year, and 14 schools before the 2011-2012 academic year. A total of 41 schools were consequently assigned to condition, in which single schools within districts were matched with other single schools by enrollment, though in 1 year (2010-11) one school was unmatched and assigned at random to condition by itself, resulting in 21 treatment schools and 20 control schools. For Smolkowski et al. (2017), Cohort 1 included 5,133 students from 41 schools with pre-intervention data (i.e., those randomly assigned and assessed at T1 - which for students in Cohort 1 was the beginning of seventh grade).

Attrition:

There were 4,078 students in Cohort 1 who completed an assessment at the end of eighth grade (which was considered the posttest), for an overall attrition rate of 8%. There was no school-level attrition.

Sample:

Of the 12,912 overall students who participated in the study, about half (51%) were female and a majority (65%) were white. The average age for students at baseline was 11.9 years (SD = 1.4). A sizable proportion (45%) of students reported their families had "just enough money to get by." Demographics for the involved schools were as follows. The proportion of students who received free or reduced-price lunch was 58%. The proportion of English language learners was 3%. The proportion of minority students was 28%.

Measures:

Nineteen measures were conducted at the student, parent and teacher levels to assess program effects on a myriad of behavioral outcomes and risk and protective factors. These measures are listed below.

Student-level measures were obtained during the annual student survey, and included:

1. Parental Monitoring - a risk and protective measure (sample α = .91)
2. Family Conflict - a risk and protective measure (sample α = .78)
3. Conduct problems (sample α = .63)
4. Emotional problems (sample α = .69)
5. Substance use for tobacco, alcohol, marijuana or other drugs (sample α = .87)
6. Positive Peers (sample α = .77)
7. School participation - a risk and protective factor (sample α = .56)
8. Positive school structures - a risk and protective factor (sample α = .86)

Parent-level measures were obtained from annual parent surveys, and included:

1. School Success - a parental report of the child's academic performance (sample α = .90)
2. Student risk - a risk and protective factor measuring parent concern for child's internalizing behavior, externalizing behavior, attention, sociability, and adherence to rules (sample α = .82)
3. Parental monitoring behavior - a risk and protective factor (sample α = .76)
4. Parent involvement - a risk and protective factor measuring parents' interactions with child (sample α = .66)
5. School structures to monitor school work, attendance and behavior - a risk and protective factor (sample α = .77)

Teacher-level measures were obtained from annual teacher survey, and included:

1. School readiness - a four-item scale to identify school-level statements specific to Positive Family Support (sample α = .78)
2. Student Risk - teacher concern for child's internalizing behavior, externalizing behavior, attention, sociability, and adherence to rules; a risk and protective factor (sample α = .89)

Administrative student-level records were obtained from the Oregon Department of Education for three primary outcomes (end-of-year state standardized math and reading test scores, and number of days absent from school) and one risk & protective factor (negative and positive school contact, as measured by calls home to parents concerning children's behavior in school).

Analysis:

Multi-level models were conducted to assess main effects with students nested within schools across time. Additional analyses were conducted to assess whether risk-level, gender, minority status and cohort moderated outcomes.

Intent-to-Treat:

All students with data were assessed in their assigned treatments, therefore intent-to-treat protocol was followed. The authors utilized restricted maximum likelihood to deal with missing data.

Outcomes

Implementation Fidelity:

Treatment schools had significantly greater scores of implementation fidelity for the three tiers of the program and a significantly greater number of resources available to parents.

Baseline Equivalence:

There were no school-level significant differences between conditions at baseline. In addition, students from both conditions were comparable on all 19 outcomes and most demographic characteristics at baseline, with the exception of the student report of how much money their family had.

Differential Attrition:

Attrition was just over 5% (i.e., 8%) from pretest to posttest. While the authors did not report differential attrition (i.e., the difference between completers and attritors on baseline and demographic variables), they reported results of attrition by condition tests and found no differences by condition on baseline measures.

Posttest:

At posttest, no main effects were detected. Moderation analysis found that treatment students at-risk for behavioral problems significantly outperformed control students at-risk on measures of parent-reported negative school contacts.

Long-Term:

Not conducted.

Limitations:

• Tests of differential attrition are incomplete
• No main effect