Coping Power Universal

Study 1

In this pilot study, both Coping Power Program (CPP)-certified trained psychologists and teachers delivered the program. In later studies, teachers delivered the program with CPP staff support.

Evaluation Methodology

Design:

Recruitment: Two primary schools (grades 1-5) from the Tuscany area of Italy agreed to participate in the study. Within the schools, nine teachers concerned about the lack of social skills and disruptive behavior of many children in their classrooms agreed to participate in the study. The nine classes of the teachers included 184 first- and second-grade students, and all parents consented to the participation and assessment of their children.

Assignment: The nine classrooms were randomly assigned to the intervention and control conditions. The intervention group comprised three first-grade classes and one second-grade class from the Capannori area and one second-grade class from the Lucca area. The control sample comprised two first-grade classes from the Lucca area and two second-grade classes from the Capannori area. The intervention classrooms added the weekly Coping Power Program sessions, while the control classrooms continued their academic curriculum as usual.

Assessments/Attrition: Assessments occurred at baseline, posttest immediately following program completion, and at 12-month follow-up. There was no attrition from baseline to posttest, but there was an 8% (N=22) loss from posttest to the follow-up assessment on behavioral measures and a 23% loss (N=42) at follow-up on academic records.

Sample: The average age of the first- and second-grade students was 7.5. There was a near even split between boys and girls, while 80-84% of the children were Italian. Based on the percentile distribution (e.g., 9.2% has scores over the 90th percentile), the sample appeared typical in terms of overall stress or total problems.

Measures:

All measures came from the teachers who helped deliver the program.

Five Behavioral Measures came from the Strengths and Difficulties Questionnaire (with the Italian translation and norms). The SDQ is a 25-item questionnaire used to assess conduct problems (e.g., bullying), hyperactivity (e.g., squirming), emotional symptoms (e.g., worrying), peer problems (e.g., disliked by other children), and prosocial behavior (e.g., helping). The SDQ-Overall Stress is the total score of the scale indicating general problematic behaviors. The mean Cronbach's alpha was .82.

Academic Performance was assessed using course grades for language arts and mathematics, as obtained from school records.

Analysis: To adjust for the nesting of individuals within classes and the randomization of classes, the analysis used linear mixed model estimation. Tests of the intervention came from coefficients for group-by-time interactions and implicitly controlled for baseline outcomes.

In the follow-up, a mixed factorial ANOVA (three time points x two conditions) was used to test the intervention effect, and a cross-lagged meditational model was used to test the intervention's indirect mechanism of behavioral improvements on academic performance.

Intent-to-Treat: The analysis likely used an intent-to-treat sample, although it did not state that explicitly.

Outcomes

Implementation Fidelity: Using videos of all the intervention sessions of four classes, a certified supervisor completed a checklist about variations from the manual during the three phases included in a session. The mean score of the intervention adherence standard checklist, based on a Likert scale from 1 to 4, was 2.90 though the follow-up lists the mean score as 3.60.

Baseline Equivalence: The study stated that there were no significant baseline differences between conditions on demographic factors or teacher-reported SDQ scales. The intervention group had three first-grade classes compared to two in the control group, but the mean age of the two groups did not differ significantly.

Differential Attrition: There was no attrition at posttest, but 8% of students dropped out between posttest and follow-up. Analyses revealed that dropouts had significantly lower baseline levels of prosocial behaviors. Course grades at 12-month follow-up could only be obtained from 77% of the baseline sample, with no assessment of whether this differed by condition or other sample characteristics.

Posttest: A significant group-by-time interaction emerged for 3 of 6 tests. The intervention group did significantly better than the control group on prosocial behaviors, hyperactivity/inattention problems, and overall stress (or total problems).

Long-Term: At 12-month follow-up, compared to the control group, the intervention group showed a significant improvement in overall stress, hyperactivity/inattention, prosocial behavior, and school grades. Mediation analyses revealed that the intervention's significant reduction of hyperactivity/attention problems accounted for the significant increases in academic performance.

Study 2

Evaluation Methodology

Design:

Recruitment: Pubic elementary schools in Lucca, Pisa, and Spoleto Italy were recruited to participate in the study. Six schools agreed to participate, and all third- and fourth-grade classes (n = 40 classes, 901 students) in these schools were selected.

Assignment: The 40 classes were randomly assigned to either treatment or control (20 classes and 488 students in treatment, 20 classes and 413 students in control). The treatment group included 10 third-grade classes and 10 fourth-grade classes, and though not explicitly stated it appeared the control group had the same breakdown.

Assessments/Attrition: Teachers completed assessments at baseline and posttest immediately following program completion. It was reported on p.757 that: "The students with complete data at both times were 841; attrition involved 6.72% of the students."

Sample: Sample characteristics were not reported.

Measures: Teachers delivered the program and also completed the outcome measures. The assessment included the Strengths and Difficulties Questionnaire (SDQ; Goodman, 1997; Tobia et al., 2011), which is a 25-item questionnaire designed to assess the occurrence of particular behaviors that have been associated with conduct problems, hyperactivity, emotional symptoms, peer problems, and prosocial behavior in children aged 4-16. In this study, SDQ-Overall Stress was measured, which is the total score of the four scales indicating general problematic behaviors: 1) conduct problems, 2) hyperactivity, 3) emotional symptoms, and 4) peer problems. In addition to assessing the total scale score, except for peer problems, each scale was also measured separately. Sample alphas were .83 for conduct problems, .86 for hyperactivity, and .84 for emotional symptoms.

Analysis: A 3-level model (which measured occasion within individuals, within classes, and within schools) was conducted with each student outcome using a random-intercept (fit to account for within classes and within schools). There were no controls for demographic variables.

Intent-to-Treat: It appeared that all participants were analyzed according to the condition in which they were assigned and all available data were utilized which is in line with intent-to-treat protocol.

Outcomes

Implementation Fidelity: Interventionists completed a measure of fidelity after completing each intervention session, rating whether they had covered each session objective "completely," "partially," or "not at all." The program leaders indicated that they "completely" (89% child group) or "partially" (11% child group) completed group intervention objectives, indicating a high rate of self-reported intervention fidelity. After each intervention session, teachers also completed a questionnaire that investigated the theoretical and practical preparation of the teachers on the daily session activities and principles. A certified Coping Power supervisor verified the percentage of correct answers: 87% of the answers were corrected in the current study.

Baseline Equivalence: Baseline equivalence was tested only on outcome measures, and it was unclear whether significance tests were conducted (see Table 2).

Differential Attrition: It was reported on p.757 that: "No significant differences were found between those who completed both data collections and those who completed only the baseline evaluation." Attrition-by-condition tests, however, were not conducted.

Posttest: At the posttest, compared to control group students, students in the treatment group were significantly less likely to experience conduct problems and less likely to exhibit hyperactive behaviors and emotional problems.

Long-Term: Not conducted.

Study 3

Evaluation Methodology

Design:

Recruitment: Six schools in a disadvantaged area of an Italian city were selected for the study. The researchers contacted the principals of these schools and all agreed to participate. Inclusion criteria were: (1) the child had to be in fourth or fifth grade; and (2) the child's parents agreed to be involved. No parents asked to exclude their children from the study. The sample comprised 1,030 students attending 70 primary school classrooms in the six schools.

Assignment: Cluster randomization by classroom (35 classrooms assigned to treatment and 35 classrooms assigned to control) was completed by an independent researcher after all participants had completed the baseline assessment. Of the 1,030 youth in the six schools that agreed to participate, 511 attended classrooms assigned to the treatment condition and 519 attended classrooms assigned to the control condition that continued business as usual. The program lasted from November 2016 to April 2017.

Assessments/Attrition: Assessments occurred at baseline, in the fall of the school year, and at posttest, near the end of the school year and about one month after program completion. Complete data at both time points were obtained for 994 students, indicating minimal attrition (< 5%).

Sample: Students were in fourth or fifth grade and had an average age of 9.24 years. Approximately 11% of the sample identified as African American, with the remainder identifying as Caucasian. Thirty-five children had an intellectual disability and eight children had a sensory disability. Teachers involved in the project had a mean number of 10 years of experience with children.

Measures: Teachers (who delivered the program) and parents (who were blind to condition) reported on youth total problem behaviors, conduct problems, hyperactivity, emotional symptoms, peer problems, and prosocial behaviors using the Strengths and Difficulties Questionnaire (SDQ). Alpha reliabilities in the current sample ranged from α = 0.80 (parent-reported emotional symptoms) to α = 0.88 (teacher-reported prosocial behaviors).

Analysis: The researchers tested for intervention effects using four-level linear mixed-effects random-intercept models to account for within-subjects correlations, within-classroom correlations, and within-school correlations. Analyses were conducted using full maximum likelihood estimation, with measurement occasion (Level 1) nested within individuals (Level 2) nested within classrooms (Level 3) nested within schools (Level 4). The time measure included the baseline outcomes, and tests for program effects examined the time-by-experimental group interactions.

Intent-to-Treat: All available data were used in FIML analyses.

Outcomes

Implementation Fidelity: Teachers self-reported that they delivered 88% of the elements of the intervention sessions. They also showed a mean fidelity score of 62 out of 72 based on questions about their understanding of key program principles. Finally, the school psychologist observed recordings of 20% of the sessions, which showed that 84% of the reported levels of adherence were good or excellent.

Baseline Equivalence: The authors only stated that no significant differences were detected in baseline demographics or outcomes.

Differential Attrition: Minimal attrition (< 5%) occurred over the course of the study. In addition, the researchers stated that "no significant differences were found between children who completed evaluations at both time points and those who completed only the baseline evaluation" (p. 15-16).

Posttest: One month after the intervention, relative to control participants, children in the intervention condition showed significantly greater improvements in five of six non-independent teacher-rated outcomes (youth total problem behaviors, conduct problems, hyperactivity, peer problems, and prosocial behavior) and three of six independent parent-rated outcomes (youth total problem behaviors, conduct problems, prosocial behavior).

Long-Term: Not conducted.