Cognitive Behavioral Intervention for Trauma in Schools (CBITS)

Study 1

Kataoka et al. (2011) extended the outcomes used in Stein et al. (2003) to include measures of academic performance.

Evaluation Methodology

Design: The first study was a randomized control trial examining the effectiveness of a cognitive behavioral therapy group intervention. The intervention was a 10-session Cognitive Behavioral Intervention for Trauma in Schools (CBITS). It was designed for use in an inner-city school mental health clinic with a multicultural population. The intervention incorporated cognitive-behavioral therapy skills in a group format. The 10 sessions were implemented on a continuous basis from late autumn through spring of the 2001-2002 academic year, in two middle schools in Eastern Los Angeles.

After parents agreed to have their children participate and children agreed to be screened, the mental health clinicians administered a self-report questionnaire regarding exposure to violence and symptoms of PTSD to 769 sixth-grade students. The students were screened for eligibility to the program; they were eligible to participate if (1) they had substantial exposure to violence; (2) they had symptoms of PTSD in the clinical range; (3) they had symptoms of PTSD related to exposure to violence; and, (4) they did not appear too disruptive to participate in group therapy intervention. One hundred and fifty-nine students met the criteria, of which 126 students chose to participate and complete baseline assessments. These students were randomly assigned into early intervention (n=61) or delayed intervention groups (n=65). Initial baseline scores were taken to account for individual differences. Program assessments were done for baseline (the initial screening), at 3 months, and at 6 months. The 3-month assessment served as a posttest for the early intervention group, who had just completed the program, while the delayed intervention group had yet to begin the intervention. The 6-month assessment served as a 3-month follow-up for the early intervention group and as a posttest for the delayed intervention group, who had just completed the program.

Kataoka et al. (2011) examined 123 students (98%) with grades and standardized state test scores available for the spring semester of the year of program receipt. However, the authors noted that, because the waitlisted control group received the program before the end of the spring semester, "it is difficult to interpret the effect that CBITS has on grades immediately post-treatment."

Sample: The sample contained one hundred and twenty six middle school students from two different East Los Angeles schools. This area of Los Angeles is primarily Latino and socioeconomically disadvantaged. Both males and females participated in the study, however, no exact calculations were presented as to how many males and females participated. Sample attrition was minimal; of the initial one hundred and twenty six student participants, one hundred and seventeen (93%) completed the 3-month follow-up assessments (54 IV, 63 control), and one hundred and thirteen (90%) completed the 6-month follow-up assessments. Overall, 90% of the participants (53 IV, 60 control) completed all assessments.

Measures: The CBITS intervention had been previously piloted for feasibility and acceptability. The school clinicians also followed a treatment manual to ensure that the application of the intervention was standardized across clinicians. The integrity of the intervention was tested through random assessment both to the extent of session completion and also the overall quality of therapy the intervention provided. On the seven items assessing quality, all sessions were rated moderate to high. Students' symptoms of PTSD were assessed using a seventeen-item Child PTSD Symptom Scale (CPSS) self-report measure (range, 0-51 points). This measure has been shown to have good convergent validity, discriminant validity, and high reliability. Students' depression levels were assessed using a twenty six-item Child Depression Inventory (CDI). This inventory assesses children's cognitive, affective, and behavioral symptoms of depression. The CDI was chosen for its good test-retest reliability and validity. Child psychosocial dysfunction was assessed using the thirty five-item Pediatric Symptom Checklist (PSC), in which the student's parent rated the frequency of emotional and behavioral problems (scale: 0=never and 2=often). Classroom behavior was assessed by the student's teacher using the six-item Teacher-Child rating scale (1= not a problem, 5= a very serious problem).

Kataoka et al. (2011) examined four outcome measures for the spring semester that related to academic performance: math grades, language arts grades, math passing grades, and language arts passing grades. Note that grades were not measured exactly after treatment, and many in the control group would have received the intervention by the time of final grades.

Analysis: Baseline measures were compared for the early intervention group and the delayed intervention group to assess individual differences. All clinical and demographic characteristics were compared in the baseline estimates. To assess the effectiveness of the intervention, a linear regression analysis was used to estimate the mean difference in outcome scores between the two groups at three and six months, adjusted for scores at baseline. Effect sizes were calculated to assess the magnitude of intervention effects. These were calculated as the odds ratio of the estimated treatment effect to the pooled standard deviation at baseline.

Kataoka et al. (2011) used hierarchical linear models to account for clustering within each of the 5-8-person program treatment groups (with program non-participants treated as a separate group). The models treated the two schools as fixed effects and controlled for the standardized test score from the prior school year, as well as the baseline PTSD symptom score, total violence score, sex, school site, parent employment status, and time of screening.

Outcomes

Intervention Integrity: Completion of required elements of the intervention program ranged from 67% to 100%, with a mean rate of 96%. Quality of sessions was assessed as moderate to high across all sessions.

Baseline Characteristics: In baseline characteristics, there were no significant differences between the students in the early intervention and delayed intervention groups. The 3-month assessment was completed by 117 students and 113 students completed the 6-month assessment. At baseline, compared with students who had completed all assessments, non-completers had significantly higher CPSS scores (mean difference, 5.4; 95% CI, 1.5-9.4), CDI scores (mean difference, 8.1; 95% CI, 3.0-13.2), acting out classroom behaviors (mean difference, 7.7; 95% CI, 3.4-11.9), and classroom learning problems (mean difference, 5.2; 95% CI, 0.4-10.0).

Posttest: At the 3-month assessment, students in the early intervention group had significantly lower self-reported symptoms of PTSD than students in the delayed intervention group. The mean difference between groups was -7.0 (95% CI, -10.8 to -3.2), with an effect size of 1.08. The results of the logistic regression analysis (adjusted for baseline scale scores) indicated that at the 3-month assessment, 86% of students in the early intervention group had significantly lower self-reported PTSD symptoms than students who did not undergo CBITS intervention. Students in the early intervention group had 67% lower self-reported symptoms of depression than students in the delayed CBITS intervention, with an adjusted mean difference of -3.4 (9.4 vs. 12.7) and an effect size of 0.45. At 6 months, when the delayed group received intervention, significant differences were no longer present between the delayed group and experimental group, in self-report PTSD scores or depression.

Parents of students in the early intervention group at 3-months reported significantly less psychosocial dysfunction (78%) than parents in the delayed intervention group. The adjusted mean difference was -6.4 (95% CI,-10.4 to 2.3), with an effect size of 0.77. Again, no significant differences existed after the parents in the delayed group received intervention; this finding was true for all measures. At the 6 month follow-up, after the delayed group had received CBITS intervention, the results indicated that students in the early intervention group had no significant differences on any tests from the delayed group. Teachers reported no differences between groups on classroom behaviors.

Kataoka et al. (2011) found one marginal and two significant intervention effects in four tests for academic outcomes: language arts passing grade (p = .033), math grade (p = .048), and math passing grade (p = .053).

Long-term: This was the first study to date to use a randomized controlled trial to evaluate the effectiveness of an intervention for children with substantial levels of symptoms of PTSD. As this was a preliminary evaluation, the long-term effects will need to be revealed in time and with replication. However, the students that received this brief intervention had significantly fewer self-reported symptoms of PTSD and parent-reported psychosocial dysfunction than the delayed intervention group. While it is positive that the students in the delayed intervention had significant positive changes after receiving intervention and those in early intervention maintained improvement at the six month assessment, only short-term effectiveness of the intervention is known.

Study 2

Evaluation Methodology

Design: From an original potential pool of 609 fourth to eighth graders at three schools, after consents, 195 students were screened and 118 who met inclusion criteria were randomized into one of two programs, CBITS or Trauma-Focused Cognitive-Behavioral Therapy (TF-CBT; Cohen, Mannarino, & Deblinger, 2006). Students were assessed at baseline, at 5 months, and at 10 months. The CBITS groups were implemented from March to May 2007 and TF-CBT was implemented from February to September 2007. Jaycox, et al. (2010) reports only on the 10 month assessment results.

TF-CBT, received by the comparison group, is a 12-session individual or conjoint intervention that includes child and parents and is typically delivered in clinics. For this evaluation, the 10 CBIT group sessions were provided along with 1-3 individual sessions designed specifically for use in schools. The earlier study by Stein et al., 2003, makes no mention of the individual sessions.

Sample: Three schools were selected to represent schools participating in a larger project, Project Fleur-de-Lis. An introductory letter was sent home with consent forms to the parents of 609 fourth to eighth graders at the three schools. 438 consent forms were returned (72%), and 202 (33%) granted permission to participate. Six students declined to participate and one moved, resulting in 195 (32%) participants in the screening. The consent rate was 41%, 27% and 46%, for the three schools. School 1 was predominately African American (74%), with 75% participating in the free/reduced lunch program (FRLP). School 2 was predominately Caucasian (90%) with 11% participating in the FRLP, and School 3 was predominately African American (97%) with 80% participating in the FRLP.

Of the 195 students who participated, 118 (61%) screened positive for PTSD symptoms. Students were randomized within strata to receive each intervention, resulting in 58 students being offered CBITS and 60 students offered TF-CBT (16, 57, and 45 participants in the three schools).

Participating students included slightly more girls (55.9%) than boys. Forty-eight percent of participants were non-Hispanic White, 46% were African American, 5% were Hispanic, and 2% were from other racial/ethnic groups. More girls (63%) than boys were determined to be at risk based on PTSD symptoms scores, with an average age of 11.5. More than half (52%) were African American, 42% were non-Hispanic White, 4% were Hispanic, and 2% were from other racial/ethnic groups.

Of children randomized to CBITS, 57 of 58 began treatment and 53 (91%) completed treatment. For children randomized to TF-CBT, 22 of the 60 (37%) attended the initial assessment, and of this number, seven (32%) did not meet PTSD criteria on the K-SADS and were not provided treatment. Another child had a pervasive developmental disorder that precluded inclusion in the study, and thus 14 students (23%) began TF-CBT and 9 completed treatment at the 10 month follow-up.

Measures: At baseline, students completed the self-report measure, Disaster Experiences Questionnaire (Scheeringa, 2005), and the UCLA PTSD Reaction Index for DSM-IV (Pynoos, et al., 1998).

At baseline and at follow-up, students completed the Child PTSD Symptom Scale (Foa, et al., 2001), the Children's Depression Inventory (Kovacs, 1981), and the Social Support Scale for Children (Harter, 1985; subscales for friends and family). Teachers reported on student behavior problems using the Strengths and Difficulties Questionnaire (Goodman, 1997; Goodman, et al., 1998).

As part of the normal TF-CBT protocol, children assigned to TF-CBT were scheduled for a clinic intake consisting of the PTSD section of a diagnostic interview (the Schedule for Affective Disorders and Schizophrenia for School Aged Children - Present and Lifetime Version; K-SADS-PL-PTSD; Kaufman, et al., 1996). Children whose responses did not indicate PTSD were not included in the TF-CBT treatment.

Analysis: Missing items were imputed 5 times separately for each treatment group and time point using ProcMI in SAS. Descriptive analyses were conducted at baseline, and compared groups receiving CBITS and TF-CBT. The plan to examine predictors of improvement within each group was not possible due to limited numbers of children who participated in TF-CBT. Thus, predictors of improvement are examined for the CBITS group only, via regressions controlling for baseline PTSD levels, adjusted for clustering within CBITS intervention groups. In the TF-CBT group, researchers examined predictors of uptake of TF-CBT via logistic regression predicting intake attendance.

Outcomes: Students in the study report a median of one type of hurricane exposure (range 0-7; most common experiences included having seen something very upsetting or being separated from parent or caregiver). Lifetime trauma was common with students reporting a median of 4 traumatic events (range 0-10, most commonly learning about the death or injury of a loved one, witnessing violence, car accidents, being victims of violence, medical procedures) at baseline. At the 10-month assessment, students reported on traumatic events since the last assessment, and students reported a median of 3 additional recent traumatic exposures.

Measures of PTSD, depression, and teacher reported behavior problems demonstrated that the two groups of children were comparable to each other.

At 10 month follow-up, PTSD scores improved for youth in both interventions, compared with baseline scores. Mean PTSD scores for the TF-CBT group had moved to the normal range whereas mean scores in the CBITS group were in the low clinical range. Thirty-seven of the 57 children (65%) in the CBITS group remained in the at-risk range on the Child PTSD Symptom Scale at 10 month follow-up, whereas 6 of the 14 children (43%) in TF-CBT remained at risk. A Fisher's Exact test revealed these two rates of clinical change were comparable across the two groups.

Changes in depressive symptoms also improved for both groups, but the improvement was only statistically significant for the CBITS group, with mean depression scores moved to the normal range for both groups.

Predictors of treatment outcome for the CBITS group were examined, predicting PTSD symptoms at 10 months while controlling for baseline PTSD. Baseline PTSD was a strong predictor of PTSD at 10 months. Support from family predicted lower PTSD scores, whereas higher baseline depressive symptoms and additional exposures to traumatic events as reported at follow-up predicted higher PTSD scores. Gender, school, teacher reported behavior problems at baseline, hurricane exposures, and social support from friends were unrelated to PTSD at follow-up.

Study 3

The study evaluated the program with college students rather than middle school students. In order to suit college students, the sessions were reduced from ten to six, once a week for up to one hour. Also, the individual-based classes, two hours of education for parents, and the one-hour of education with a teacher were excluded from the program.

Evaluation Methodology

Design:

Recruitment: The sample consisted of 100 undergraduate students who were recruited from a mandatory psychology class at Methodist University in Fayetteville, North Carolina. Participation was voluntary but extra credit was offered. A total of 73 students filled out the screening questionnaires, resulting in a response rate of 73%. A total of 28 students (38%) met eligibility criteria and participated in the study. The eligibility criteria included having a traumatic event within five years and a high score on a trauma screening instrument.

Assignment: The study randomly assigned the 28 participants to an intervention group receiving CBITS and a control group not receiving therapy. The intervention group was split into three treatment groups.

Assessments/Attrition: A posttest came at the end of the six-week program. Of the 28 students, 21 had some missing data, and the results show that only 12-23 (43-83%) were used in the analyses.

Sample:

The sample included nine men (32%) and 19 women (68%).

Measures:

The three outcome measures came from independent sources. GPA measured assigned grades as obtained from school records. Student self-reported problematic behavior in school (e.g., completing assignments on time, attendance) came from surveys. Teacher reports on student problematic behavior came from surveys, but since teachers did not deliver the program or appear to be aware of condition assignment, the measure was likely independent. The student and teacher reports on problematic behavior in school came from the Weiss Functional Impairment Rating Scale and had good reliabilities (alphas = .74 and .78).

Analysis:

The analysis used independent sample t-tests to compare conditions on the outcome measures. Controls for the baseline outcomes were not used.

Missing Data Strategy: The analysis used only those participations with complete posttest data for each outcome.

Intent-to-Treat: The analysis used all participants with data for each of the outcomes.

Outcomes

Implementation Fidelity:

Not examined.

Baseline Equivalence:

No tests were conducted, but the study described several differences between condition. There were more freshmen in the control group than in the experimental group, the control group had more positive baseline teacher evaluations (0.77) than the experimental group (2.42), and the control group rated themselves higher on problematic school behavior (14.3) than the experimental group (8.3) at baseline.

Differential Attrition:

Not examined.

Posttest:

For GPA, the intervention groups had a significantly higher value at posttest than the control group (r = .56), but had there been a control for the baseline outcome, the effect would have been eliminated. Figure 1 shows that the intervention group increased less than the control group from baseline to posttest. For self-reported problematic behavior at school, the intervention group had a significantly better score at posttest than the control group (r = .39), and Figure 2 shows that the intervention group experienced a greater decline than the control group. For teacher reports, the conditions did not differ significantly at posttest.

Long-Term:

Not examined.

Study 4

Evaluation Methodology

Design:

Recruitment: The sample came from students attending two primary schools located in an area of Kenya outside of Nairobi with informal settlements (i.e., poor housing, lack of faculties, and insecure tenancy). It included adolescents ages 10-14 (grades five to seven of the Kenyan primary system of education) who experienced mild, moderate, or severe PTSD symptoms. Among the students meeting the eligibility criteria, every other one was selected to join the study. Of 698 students ages 10-14, 285 met the trauma screening criterion and 212 were selected to participate.

Assignment: The author referred to using a quasi-experiment design but randomly assigned one of the two schools to the intervention condition and one to the control condition. Each school had 106 participating students. The sample of one school in each condition creates a confound that prevents the study from isolating the impact of the program from the unique characteristics of the two schools.

Assessments/Attrition: The posttest assessment occurred 10 weeks after the program start and a follow-up occurred 12 weeks after the posttest. Of the 212 participants, 194 completed the follow-up (92%). The authors stated (p. 89) that participants "who abandoned more than two sessions would also be excluded from the study." It was not clear if the attrition figures counted these exclusions.

Sample:

Girls made up 54% of the sample, and their mean age was 12.2 years.

Measures:

The measures came from self-reports of PTSD symptoms (alpha = .67) administered by trained research assistants.

Analysis:

The analysis used t-tests for independent samples without controls for baseline outcomes or adjustment for clustering within schools (the unit of assignment).

Missing Data Strategy. The analysis used complete cases analysis, dropping those without posttest data and, it appears, those not attending at least eight of 10 sessions.

Intent-to-Treat: The analysis did not use an intent-to-treat sample, as students who attended fewer than eight of 10 sessions were excluded.

Outcomes

Implementation Fidelity:

Not examined.

Baseline Equivalence:

In Table 4.13 (p. 124), a t-test for equality of means found that baseline PTSD symptoms did not differ significantly across conditions.

Differential Attrition:

No tests were conducted, but the attrition rates were similar across conditions (90% in the intervention group and 93% in the control group).

Posttest:

In Table 4.13 (p. 124), t-tests for equality of means found that the intervention group had significantly lower PTSD symptoms at posttest (d = .79) and follow-up (d = .82).

Long-Term:

Not examined.

Study 5

School social workers (rather than therapists) implemented the program.

Evaluation Methodology

Design:

Recruitment: The sample came from 12 middle schools located within one large urban school district in northern California that served a diverse population. Eligible participants within the schools were in grade six, had experienced one or more traumatic events, and had elevated traumatic stress. Of the 4,076 students who returned screening consent forms from the fall of 2011 to the fall of 2015, 550 (13.5%) endorsed at least one event on a trauma exposure checklist and showed elevated levels on a trauma symptom checklist. Of these eligible students, 296 (54%) consented to participate in the study.

Assignment: The study randomly assigned the 296 students to the intervention group (n = 152) or to a services-as-usual control group (n = 144). Assignment of individuals was done within each school. After randomization, two intervention students and one control student declined to participate, and they were removed from the study sample. Most students in the control group received a range of non-program services (e.g., individual short-term goal-oriented supports, restorative justice groups, small group counseling, social skills groups, anger management groups).

Assessments/Attrition: Assessments occurred at baseline, posttest, and one-year follow-up. According to the CONSORT diagram (Figure 1), completion rates were 99% of the randomized sample at baseline, 98% at posttest, and 95% at follow-up. However, the authors noted that across the 16 outcome measures, the proportion of students with missing data ranged from 2-10% at baseline, 4-14% at posttest, and 7-25% at one-year follow-up.

Sample:

The sample included equal portions of males and females. Approximately 39% of the sample identified as Latinx, 24% as Asian, 12% as White, 7% as Black, and 18% declined to answer.

Measures:

The 16 outcome measures came from five instruments.

• The student self-reported Trauma Symptom Checklist-Child Version consisted of 44 items and had internal consistencies ranging from 0.76 to 0.90 across the three waves of data collection.
• The Achenbach System of Empirically Based Assessment-Teacher's Report Form consisted of three subscales that measured internalizing, externalizing, and total problems and, as reported by the scale developers, had alpha values at or above .90.
• The Achenbach System of Empirically Based Assessment-Youth Self‐Report consisted of three subscales that measured internalizing, externalizing, and total problems and, as reported by the scale developers, had alpha values at or above .90.
• The Woodcock-Johnson III Normative Update Brief Battery was administered by research assistants, consisted of four subscales assessing reading and math achievement, and had internal consistency coefficients of 0.95 to 0.97.
• One measure of Academic Engaged Time came from classroom observations by research assistants unaware of condition.

Analysis:

The analysis used hierarchical linear models to account for students being nested in schools. The models controlled for baseline outcome measures, race/ethnicity, and gender.

Missing Data Method. The hierarchical linear models used full information maximum likelihood estimation to account for missing data.

Intent-to-Treat: The analysis used all available data with FIML.

Outcomes

Implementation Fidelity:

On a scale of 0 to 3, the average rating of session adherence was 2.85 and the average rating of session quality was 2.89. However, the authors noted that there were challenges to holding the parent education meetings.

Baseline Equivalence:

Using all participants with baseline data (99% of the randomized sample) and hierarchical linear models, tests for equivalence on 22 baseline measures (see Table 1) found two significant differences. Students in the intervention group self-reported significantly more symptoms of anger and depression than the control group.

Differential Attrition:

A Little test found that data were missing completely at random (MCAR). In addition, the authors reported that, across 16 outcomes at posttest, the treatment group attrition rate ranged from 4-15%, the control group attrition rate ranged from 4-12.5%, and the differential attrition rate ranged from 0-2.5%. Across 16 outcomes at follow-up, the treatment group attrition rate ranged from 5-23%, the control group attrition rate ranged from 9-26%, and the differential attrition rate ranged from 3-4%. The authors concluded that "According to the WWC standards (2017), the overall and differential attrition rate is low for this study" (p. 688).

Posttest:

In tests for the 16 outcomes at posttest (Tables 2 and 3), there was one significant effect. The intervention group reported significantly reduced PTSD symptoms (g = -.21). Moderation tests suggested that the program was significantly more effective for several outcomes (e.g., dissociation, anger) among students with externalizing behavior problems in the clinical range at baseline than for others.

Long-Term:

In tests for the 16 outcomes at one-year follow-up (Tables 2 and 3), there were no significant intervention effects. Moderation tests suggested that the program was significantly more effective for a few outcomes (e.g., externalizing, mathematics achievement) among students with internalizing behavior problems in the clinical range at baseline than for others.