Project Towards No Drug Abuse

Study 1

Summary

Simon et al. (2002), Sun et al. (2006), and Sussman et al. (1998) used a cluster randomized controlled trial that assigned 21 alternative, continuation high schools with 1,578 student participants to an intervention group, an intervention group plus school wide activities, or a control group. Student measures of substance use were obtained over a four-to-five-year period.

Simon et al. (2002), Sun et al. (2006), and Sussman et al. (1998) found that students from program schools, relative to students from control schools, exhibited significant reductions in

• hard drug use prevalence rates at one-year, two-to-three-year, and four-to-five-year follow-ups
• alcohol use prevalence at one-year follow-up among those using alcohol at baseline.

Evaluation Methodology

Design: Twenty-nine school districts containing at least one continuation high school from five Southern California counties were recruited for participation in the study in a procedure approximating random selection. Twenty-one alternative high school settings were selected based on school size (greater than 50 students but less than 500) and were blocked by characteristics such as substance abuse prevalence based on student and staff interviews, ethnic composition of school and achievement scores. Schools were then randomly assigned by block to one of three conditions: 1) control group; 2) TND curriculum; and 3) TND curriculum plus school wide activities such as meetings, job training and drug-free party events. Pretest assessments were administered just prior to school-wide implementation of the TND program. Approximately one month later (after completion of the TND program) the posttest assessment was administered. Follow-up surveys were administered immediately post-program, as well as 13.5-, 24-, 36-, 48-, and 60-months post program. The majority of follow-ups were conducted via telephone interview (Year 1: 77%; Year 2 82.5%). Follow-up surveys for years 3 to 5 were administered by telephone interviews only.

Researchers initially had access to 75% of the population in the study, or 2,863 students. Only those students who participated in the classes on campus where the TND curriculum was administered were eligible for participation in the study. The pretest assessment was completed by 2,060 students. Of the 2,060, a total of 1,578 students received parental consent to participate in the evaluation and were therefore eligible for follow-up data collection. Of this sample, 1,074 students, or 68% of the eligible sample with parental consent (38% of the original population), submitted one-year follow-up data (approximately 13.5 months after the pretest). In order to increase the sample size for analysis, survey data at years 2 and 3 were combined as middle-term follow-up data, and years 4 and 5 were combined as long-term follow-up data. From the 1,578 subjects with pretest measures, 1,047 (66%) and 725 (46%) subjects had middle and long-term follow-up data, respectively. A total of 530 subjects (34%) had complete data at all 4 time points: pretest, short-term, middle-term, and long-term. The reports for three studies (Sussman, Dent, Stacy, & Craig, 1998; Simon, Sussman, Dahlberg, & Dent, 2002; Sun, Skara, Sun, Dent, & Sussman, 2006) utilize different sample sizes contingent on samples assessed..

Of the sample of 1,074 students who submitted both pre- and one-year follow-up data (Sussman et al., 1998), subjects varied from 14 to 19 years of age, with a mean age of 16.7 years. The pretest sample was 62% male, 37% white, 46% Latino, 4% Asian American, 8% African American, 3% Native American, and 2% other ethnicity. The ethnicity of the sample varied slightly over time (see Sun et al., 2002). An attrition analysis using 31 items or indices showed no statistically significant differences between the consented pretest group and the full pretest sample. An analysis across group conditions at pretest showed no significant differences between groups.

The sub-sample assessed by Simon et al. (2002) for violence-related behavior included the responses of 850 students who provided complete data at both testing periods (pretest and on measures of sex, perpetration, victimization and weapon carrying). This sample consisted of 55% males, 45% females, 49% Latino, 34% white, 9% African American, 4% Asian American, 3% Native American and 1% other ethnicity. Subjects ranged in age from 14 to 19 with a mean age of 16.8 years. In comparing the subsample with the total sample, no significant sex differences were observed. Participants who completed the follow-up assessment were more likely to be white and slightly less likely to be non-Hispanic black. Participants in the subsample were also significantly less likely to report perpetration and tended to be less likely to carry a weapon, compared to those not providing follow-up data.

Measures: Data were collected via self-report surveys, via face-to-face interviews, and telephone surveys.The primary measure was current drug use (Sussman, Dent, Stacy, & Craig, 1998; Sun, Skara, Sun, Dent, & Sussman, 2006). Respondents were asked "How many times in the last month have you used" each of several different drug categories: cigarettes, alcohol, marijuana, cocaine (crack), hallucinogens (LSD, acid, mushrooms), stimulants (ice, speed, amphetamines), inhalants (rush, nitrous), and other drugs (depressants, PCP, steroids, heroin, etc.). The last five drug categories (cocaine through other drugs) were summed to form a hard drug index. In addition, psychosocial measures were included on the questionnaires in order to assess friends' drug use, prevalence estimates of use, peer approval of use, family conflict, perceived stress, sensation-seeking, morality of drug use, violence victimization, baseline program outcome expectancies, assertiveness, health as a value, membership in a gang or tagging crew, depression, and drug use myths.

The Simon et al. (2002) sub-sample provided complete data on measures of gender, perpetration, victimization, and weapon carrying at baseline and one-year follow-up (79% of follow-up sample, 30% of the full baseline sample). Perpetration of violence was measured by asking respondents how many times in the previous twelve months they had "slapped, punched, kicked, or beat-up someone," and "used a weapon to threaten someone." Respondents were also asked to indicate how often they engaged in activities over the past 12 months to protect themselves using a pre-selected list of answers.

Analysis: ANCOVAs were used to test for differences between groups at the one-year follow-up (Sussman et al., 1998). Prediction of the level of current use (in the past 30 days) of cigarette smoking, alcohol use, marijuana use, or hard drug use from pre-test use level, condition, schools nested within condition, method of one-year follow-up collection, and the interaction between pretest level of use and condition was completed with a PROC MIXED procedure. Mean comparisons were evaluated using one-tailed tests of significance. Propensity score analyses were used to detect program effects in middle-term and follow-up data (Sun et al., 2006).

Three multivariate logistic regression analyses were calculated to assess the relationship between exposure to the curriculum and risk of violent victimization, perpetration of violence, and weapon carrying (Simon et al., 2002). Since baseline comparability analysis indicated that race/ethnicity and survey procedure differed across conditions, these variables were controlled in the analyses. As the two intervention conditions did not differ on any of the violence outcomes, the two categories were collapsed for future analysis in comparison with the control group. Three multivariable logistic regression analyses were used to test whether exposure to the curriculum was associated with risk of violent victimization, perpetration of violence, and weapon carrying, controlling for race/ethnicity and survey procedure.

Outcomes

Immediate Posttest: The data collected at the one-month follow-up was primarily used for program implementation evaluation purposes and were not analyzed for the purposes of outcome evaluation.

Long-term: The results at one-year follow-up (Sussman, Dent, Stacy, & Craig, 1998) revealed that students from schools in either program condition exhibited a 25% reduction in hard drug use prevalence rates, relative to students from the control schools. In addition, a 7% reduction in alcohol use prevalence was observed for students in either program condition, relative to controls, but only among those who were using alcohol at baseline (64% of the sample). No reduction effects, relative to controls, were found on the prevalence of cigarette smoking or marijuana use in either program condition. Also, despite the fact that the school-led extracurricular activities component appeared to be successfully carried out, there appeared to be no incremental effect of those activities on problem behaviors above and beyond the presentation of the classroom curriculum.

Middle-term (2 and 3 years) and long-term (4 and 5 years) follow-up data are reported in Sun, Skara, Dent, and Sussman (2006). The analysis of the middle-term follow-up data revealed no significant program effects for past 30-day use of any substance (cigarettes, alcohol, marijuana, and hard drugs). A significant long-term program effect was revealed for hard drug use.

At the long-term follow-up, compared with those in the control group, the group who received the classroom-only intervention had less than half of the past 30-day drug use frequency, and the group who received classroom plus the school-as-community component showed about one-fifth the past 30-day drug use frequency. Both differences were statistically significant at the 4- or 5-year follow-up. The contrast of the classroom-only condition versus the classroom plus the school-as-community component condition was not significant.

Simon, Sussman, Dahlberg, and Dent (2002) assessed violence-related behavior in a subset of the original sample. Long-term effects were measured at the one-year follow-up. Preliminary analyses indicate preventative effects for alcohol and hard drug use for the treatment group, compared to the control group. However, there were no significant differences between the group that received only the TND curriculum and the group who received the TND curriculum plus activities.

Results for the sex-by-program interaction term in the model predicting victimization indicate that the difference between the control condition and the treatment condition was significantly stronger for males than females. Similar patterns were also observed for weapon carrying, though this finding was not statistically significant. Separate logistic regression models were run for each gender. Results indicate that program condition was not significantly associated with risk for any of the violence outcomes for females. However, statistically significant findings were demonstrated for males, indicating higher levels of victimization for those males in the control group, compared to the treatment group. Similar findings were also demonstrated for weapon carrying in the control group, though these results were not significant. When drug variables were added to the equation to test for mediating effects, results indicate that the association between the treatment condition and victimization risk remained statistically significant following adjustment for drug abuse measures.

Study 2

Summary

Dent et al. (2001) and Sussman et al. (2002) used a cluster randomized controlled trial that assigned 26 classrooms with 1,208 students in three regular high schools to the intervention group or a control group. Student measures of substance use were gathered at a one-year follow-up.

Dent et al. (2001) and Sussman et al. (2002) found that, relative to control classrooms, intervention classrooms exhibited significant reductions in

• hard drug use at one-year follow-up
• alcohol use at one-year follow-up
• weapon carrying and victimization among males only.

Evaluation Methodology

Design: Three public high schools were randomly selected from a pool of 78 Los Angeles area regular high schools. Twenty-six classes from grades 9 to 11 were identified and randomly assigned in equal numbers by teacher and school to one of two experimental conditions - control or classroom education program. A total of 1,208 students enrolled at three Los Angeles Area general public senior high schools participated in the study. A pretest survey was conducted in each of the 26 classrooms. The pretest survey was followed by implementation of the TND drug abuse prevention program over the next 3 weeks. The one-year follow-up was conducted with 679 of those students (63%).

Sample: Participating students ranged from 14-17 years of age; 35% were 9th graders, 43% were 10th graders, and 22% were 11th graders at baseline. The sample was 47% male, 34% white, 38% Latino, 26% African American, and 2% other ethnicity. An analysis of attrition showed there were no statistically significant differences on any assessed variable between the sub-sample of subjects measured at pretest and one-year follow-up and all those measured at pretest. A test of condition comparability at pretest found no statistical evidence that would indicate that the condition groups systematically varied on any of the pretest measures.

Measures: The primary measure in this study was current drug use at pretest and again one year later (one-year follow-up). Respondents were asked "How many times in the last month have you used" each of several different drug categories: cigarettes, alcohol, marijuana, cocaine (crack), hallucinogens (LSD, acid, mushrooms), stimulants (ice, speed, amphetamines), inhalants (rush, nitrous), and other drugs (depressants, PCP, steroids, heroin, etc.). In this analysis, the last five drug categories (cocaine through other drugs) were summed to form a hard drug index.

Analysis: An analysis of covariance (ANCOVA) model was utilized to assess whether the program curriculum was effective in the reduction of drug use.

Outcomes

Immediate Posttest: No post-test data was analyzed after implementation of the program. Long-term effects were measured at the one-year follow-up.

One Year Follow-up: There were no significant effects on cigarette and marijuana use frequency. There was an interaction effect on alcohol and drug use frequency, with program condition students with higher levels of pretest alcohol use benefitting more and with pretest non-drug users benefitting more (Dent, Sussman, & Stacy, 2001). As hypothesized, the program condition did reduce the prevalence of problem behaviors at one-year follow-up. Reduction effects were observed on prevalence of hard drug use (25% among baseline non-drug users), alcohol use (12%) among baseline users; and weapon carrying (19%) and victimization (17%) among males (Sussman, Dent, & Stacy, 2002).

Study 3

Summary

Sussman, Dent, Craig et al. (2002), Sussman, Dent, & Stacy (2002), and Sussman et al. (2003) used a cluster randomized controlled trial that assigned 18 continuation high schools with 1,018 students to three conditions: the regular intervention, a self-instructional version of the intervention, or a standard-care control condition. Data on student substance use were obtained at one- and two-year follow-ups.

Sussman, Dent, Craig et al. (2002), Sussman, Dent, & Stacy (2002), and Sussman et al. (2003) found that the health educator-led condition, relative to the control condition, significantly reduced

• problem behavior rates (i.e., hard drug use, alcohol use, weapon carrying, tobacco and marijuana use) at one-year follow-up
• 30-day tobacco and hard drug use at two-year follow-up.

Evaluation Methodology

Design: Three sessions were added to the original TND program in order to create a revised TND curriculum. This 12-session version of the curriculum contained the same motivation-skills-decision-making material as the previous TND trials, with the addition of three new sessions that provided more information about tobacco and marijuana use and violence prevention. This experimental field trial involved 18 continuation high schools. A randomized block design was used to assign these schools to one of three conditions: standard care control condition, a 12-session classroom program, or a 12-session self-instructional version of the program. There were six continuation high schools in each condition. In one program condition, students at the schools received the 12-session version of the TND curriculum delivered by project staff health educators in class. In the second program condition, students completed a self-instruction version of the 12-session TND curriculum. The self-instruction version contained the same material and concepts as the health-educator-led version, but each student completed the lessons on his or her own, while in class. Self-instructional programming is the major means of imparting academic material in continuation high schools. Students at schools in the control condition completed the pretest and follow-up surveys only.

Sample: A total of 1,018 students were surveyed from 18 continuation high schools in southern California. However, only 572 participants had both a pretest and posttest and completed all the items for the posttest analysis, and 575 were successfully resurveyed at the two-year follow-up. Subsequent analysis showed that baseline demographics and knowledge did not differ between those only surveyed at pretest and those who were surveyed at both waves. A total of 320 students participated in the health-educator led program and 252 participated in self-instruction. The sample was ethnically diverse with 27% white students, 50% Latino students, 10% African American students, and 13% "other" ethnic groups. The sample was 60% male.

Measures: The implementation measure consisted of a tally of average attendance in the health educator-delivered condition. In the self-instruction condition, the program was self-paced and participation could reach 100% if the student completed all self-instruction packets. Students were asked to rate each session and to form a general opinion about the program overall. A total of 24 items measured students' knowledge of information gained through the program. Additional measures included substance use (an index based on how many times a particular drug was used in the past 30 days); violence and victimization (a scale consisting of the following three items: how many times has someone injured you with a weapon in the past year, threatened you without injury, injured you without a weapon); and respondents were also provided a list of things that people do to protect themselves and asked how often they engaged in the behaviors. In addition, breath samples were collected in order to measure carbon monoxide content.

At the two-year follow-up (Sussman, Sun, McCuller, & Dent, 2003) the self-report survey form was administered via telephone as only 5% of the targeted sample were still enrolled at the continuation high school.

Analysis: Analyses were completed on the pre-post data knowledge items using a repeated measures general linear model (GLM). ANCOVA was utilized to assess whether the program curriculum was effective in reducing the drug and violence indicators among students in the treatment conditions. Additional analyses were conducted in order to evaluate the effectiveness of the health educator-led and self-taught programs in comparison to the control group.

Outcomes

Immediate Posttest (Sussman, Dent, Craig, Ritt-Olsen, & McCuller, 2002): Students attended an average of two-thirds of the sessions in the health educator-led condition, and students completed an average of 83% of the self-instruction sessions. Those students exposed to the health educator-led version found it more enjoyable, interesting, and less of a waste of time than those exposed to the self-instruction version. There were significant effects on knowledge overall. The average percentage correct changed from 38 to 41 in the control condition, 39 to 48 in the health educator condition, and 34 to 42 in the self-instruction condition.

Long-term: Results from a one-year follow-up (Sussman, Dent, & Stacy, 2002) indicated that only the health educator-led condition provided a reduction in problem behavior rates, relative to each of the other conditions; the self-instruction and control condition observed rates of problem behavior did not differ significantly from each other at one-year follow-up. Reductions in prevalence were found in the health educator-led condition for hard drug use (26% relative reduction), alcohol use (9% relative reduction) among baseline users; and weapon carrying (37% relative reduction among baseline non-weapon carriers) and victimization (6%) among males. In addition, prevalence reduction in cigarette smoking (27% relative reduction) and marijuana use (22% relative reduction) were observed for the health educator-led condition students.

At the two-year follow-up (Sussman, Sun, McCuller, & Dent, 2003; n = 575, 55% of the targeted follow-up sample), the health educator-led condition significantly lowered the probability of 30-day tobacco and hard drug use. Marijuana use among male non-users at pretest was also lower in the health educator-led condition. There was no evidence that the self-instruction program exerted any effects at 2 years post-program.

Study 4

Summary

Sun et al. (2008) used a cluster randomized controlled trial that assigned 18 continuation and regular high schools with 2,734 students to a cognitive-only intervention, a cognitive plus behavior intervention, or a control group. Student data on substance use were obtained at a one-year follow-up.

Sun et al. (2008) found that, relative to the control group, the two intervention groups significantly reduced

• hard drug use.

Evaluation Methodology

Design: Nine school districts from 2 counties in southern California were recruited. From each district a pair of high schools, one regular and one alternative was included, yielding a total sample of 18 schools. Schools were randomly assigned to one of three conditions: control, cognitive perception information only curriculum (Cognitive Only), or combined cognitive perception information + behavioral skills curriculum (Combined). Curriculum lessons were delivered over a four-week period. A total of 3,908 high school students were enrolled in the classrooms selected for study participation; however, access was provided to 2,734 of these students (70%), all of whom completed pretests. Of students with pretests, 2,064 (75.5% with a pretest) also completed the one-year follow-up. This is the final analysis sample.

Of note, the program was administered by both project health educators and classroom teachers, in contrast to prior studies in which only project health educators delivered the program.

Sample: Students varied from 13-19 years of age; 52.1% male; 18.2% white, 62.1% Hispanic, 8.4% Asian, 8.1% African American, and 3.2% other ethnicity. A language other than English was spoken at home by 16.4% of the sample; 61.9% lived with both parents; and approximately 50% of youths' fathers and 56% of youths' mothers completed high school.

Baseline comparability was achieved for age, gender, program provider, attrition rate, and the four drug use outcomes. There were a lower proportion of white subjects and a greater proportion of Latino subjects in schools assigned to the Combined conditions, and higher proportions of African American and Asian students in the Cognitive Only condition. The Control condition contained a lower proportion of alternative students relative to the other conditions. To adjust for possible confounding, subjects' ethnicity and school type were included as covariates in the evaluation.

Measures and Analysis: Prevalence and frequency of drug use was measured using a generalized mixed-linear model. Substance use measures included 30-day use of cigarettes, alcohol, marijuana, and a hard drug use score (across cocaine, hallucinogens, inhalants, stimulants, ecstasy, depressants, PCP, steroids, heroin).

Outcomes

Attrition: Comparison of the analysis sample (n=2,064) to the lost-to-follow-up sample (n=670) on 12 key baseline measures showed five measures to have statistical differences. The retained sample was slightly younger, less likely to smoke cigarettes, less likely to be male, less likely to be African American, and more likely to live with both parents. To adjust for possible attrition bias, a propensity to attrition score was calculated for each subject retained at one year, and adjusted for in the analysis.

One-Year Follow-up: There were no significant findings for prevalence of use of any of the four 30-day substance use outcomes for either of the treatment conditions. The only findings that were statistically significant were for frequency of hard drug use which was reduced in both treatment conditions relative to the control group.

Study 5

Summary

Valente et al. (2007) used a cluster randomized control trial that assigned 73 classrooms with 928 students in 25 continuation high schools in California to the intervention, the intervention plus a social network emphasis, or a control group. Student data on substance use were collected at a one-year follow-up.

Valente et al. (2007) found that the TND-Network group, relative to the control group, significantly reduced

• substance use.

Evaluation Methodology

Design: A total of 25 continuation high school districts in southern California were invited to participate in the study (continuation high schools are alternatives to the normal comprehensive high schools). Ten of the schools refused to participate due to administrative reasons and 7 schools were eliminated due to small populations or restrictions on access. Of the eight remaining districts, one was used as a pilot location and the 75 classrooms from the remaining seven districts (a total of 14 continuation high schools) were assigned randomly to one of three conditions: control (prevention as usual) (n=28), TND (n=22), and TND-Network (n=25).

Of the 1,493 students who were invited to participate, 938 provided consent forms and completed valid baseline surveys. Of these, 344 students were lost to follow-up at one year resulting in an attrition rate of 36.7%. Data from 53 students were unusable at 1-year follow-up due to missingness, resulting in a final analysis sample of 541 participants at 1-year follow-up (retention rate of 57.7%). These 53 students were also removed from the baseline comparisons, resulting in a baseline sample of 885 students.

The 12-session program was delivered over 3-4 weeks and follow-up occurred one year after the program ended. No immediate posttest was conducted.

Sample Characteristics: The participants were high school students with an average age at baseline of 16.3 years. 62% of the students were male, 72% were Hispanic, 6% were African American, and 11% were white. Students had, on average, 4.16 friends with about 1.82 of these from within the same school. At baseline, students used nicotine, alcohol and marijuana approximately 10 times per month. The average mother's education was 2.92, where a score of 3 indicates completed high school. This appears to be a disadvantaged sample; however, the researchers state that these characteristics are "typical of southern California continuation high schools".

Measures: Information on substance use was gathered by measuring monthly use of nicotine, alcohol, marijuana and cocaine on an 11-point scale (1=no use, 2=1-10 times/month, 3=11-20 times/month, etc). A composite score was also created by standardizing the scores on all four substances and calculating the average across all four items.

Social network data were collected by providing each student a roster of the students in the class and students wrote the names of each student they considered a friend. These data were used to measure nominations sent and received. Peer use was measured by averaging the friends' self-report of substance use.

Analysis: Multilevel, lagged regression models that adjust for within-school clustering were used to test for intervention effects using demographic and network variables as controls.

Intention-to-treat: The study used data on all students who provided valid assessments, regardless of dose of intervention received. However, the many dropouts without valid data may have had low participation.

Outcomes

Implementation Fidelity: Sixteen health educators were trained by program staff to provide TND and TND-Network. No further information about implementation fidelity was provided.

Baseline Equivalence: No statistically significant differences among conditions were observed, with the exception of grade, network nominations and peer use. Participants in TND-Network were at a slightly lower grade compared to the other conditions (10.4 versus 10.7); control students made more network nominations than the other conditions (3.42 versus 3.02); peer use was the lowest in TND-Network (2.1) versus TND (2.36) and control group (2.42).

Differential Attrition: Across the three conditions, retention ranged from 56.7% to 63.8%. While the study did report that there were no differences on any variables between those retained in the study and those lost to follow-up, the high level of attrition suggests the need for a more thorough analysis of the problem.

One-Year Follow-up: As previously mentioned, follow-up measures were gathered one year post-intervention. Results revealed that receiving TND was not associated with any changes in substance use relative to the control condition. Receiving TND-Network was associated with decreased marijuana use (p<0.05), decreased cocaine use (p<0.05), and decreased composite use (p<0.01) relative to the control condition. The interaction of peer use and TND-Network condition was associated with increased marijuana use (p<0.05), increased cocaine use (p<0.05), and increased composite use (p<0.01) relative to control condition. No significant findings were revealed using the interaction of peer use and TND condition.

The significant interaction produced iatrogenic effects. For example, peer substance use had no influence on the composite substance use scale for the control and TND groups, but it increased substance use for the TND-Network group. While successful in general, the intervention worsened substance use for those with high substance-use peers.

Researchers also examined the start rate (those who reported no use at baseline and some use at follow-up) and quit rate (use at baseline but none at follow-up). They reported that the quit rate was significantly higher in the intervention conditions (14.3%) than the control condition (12.6%). The adjusted odds ratio (AOR) for quitting in the TND-Network condition was 3.41 (p<0.01). Consistent with the interaction effects, as substance use among peers increased, this AOR (and the benefit of the program) decreased.

Study 6

Summary

Rorhrbach et al. (2010) used a cluster randomized controlled trial that assigned 65 high schools across the country with 3,751 students to one of two intervention conditions with different forms of teacher training or a control group. Student data on substance use were gathered at a one-year follow-up.

Rorhrbach et al. (2010) found that, compared to controls, the program showed significantly

• hard drug use among baseline non-users only.

Evaluation Methodology

Design: Between 2004 and 2007, four cohorts of school districts were recruited to participate in this study. Of the 65 schools that agreed to participate, 59 were regular high schools and 6 were alternative high schools. These schools were from school districts who expressed an interest in Project TND. They spanned a total of 14 school districts across the country. Within each school district, schools were randomly assigned to one of three conditions: TND with teacher support (n=22), TND with regular workshop only (n=21), or standard care control (n=22). Within each school assigned to the program conditions, project staff coordinated with school administration to select at least one teacher to participate in the project and two of the teacher's classes were randomly selected to participate in the program evaluation. In control schools, two classrooms were randomly selected for the program evaluation.

Teachers in the Implementation Support and Regular Training conditions participated in a one-day workshop conducted on-site by certified TND trainers. Teachers in the TND with teacher support intervention also received two on-site sessions of coaching from the TND trainer, web-based support (a discussion forum, teaching tips, and downloadable scientific articles), and additional technical assistance from program specialists via telephone and e-mail, on an as-needed basis.

A total of 4,351 students were enrolled in the classes selected for participation in the study. Of these, 3,751 students (86.2%) consented to participate and 3,346 students took the pretest survey. The one-year follow-up survey was completed by 2,563 students (76.6%). A total of 25 students were dropped from the analysis for inconsistent responses, leaving 2,538 students (1,085 in the TND with teacher support, 772 in the TND regular, and 681 in the control conditions) in the analysis sample.

Data were gathered at baseline and at one-year post intervention during the next school year. Those students who were no longer enrolled in school at one-year follow-up were contacted by telephone and surveys were completed through telephonic interview.

Sample Characteristics: A majority of the participants were enrolled in regular high schools (94.8%). The sample had an average age of 14.8 years, and 46.6% of the sample was male, 41.1% was white, 28.7% was Hispanic, and 15.8% was African American. There was no other information on the sociodemographic characteristics or extent of substance abuse problems of the schools or the students.

Measures: Information on substance use was gathered by measuring monthly use of cigarettes, alcohol, marijuana and "hard drugs" on 8-point scales (0, 1-10 times/month, 11-30 times/month, etc.). A hard drug use index summed responses to six items regarding use of cocaine, hallucinogens, stimulants, inhalants, ecstasy, and other drugs.

Analysis: Generalized mixed linear models were used to analyze program effects on substance use at one-year follow-up. Two-level random coefficients modeling was constructed with school and school district considered as random factors and experimental condition considered as fixed. Baseline levels of substance use, age, gender, ethnicity and propensity for attrition were all adjusted for in the model. The first set of analyses compared intervention students to controls; the second set compared the two intervention conditions; the third set examined program effects on each substance use outcome for users versus non-users for each specific substance.

Researchers calculated a propensity-for-attrition score to adjust for bias introduced by differential attrition. This score comes from a multiple regression of attrition status on selected baseline measures. However, the study gives few details on how the propensity score is estimated or how well the propensity score prediction model works.

Intention-to-treat: The study used data on all students who provided valid assessments, regardless of dose of intervention received.

Outcomes

Implementation Fidelity: No information about implementation fidelity was provided.

Baseline Equivalence: No statistically significant differences among conditions were observed for age, ethnicity, alcohol, marijuana and hard drug use. However, relative to the other conditions the TND-regular had a greater proportion of males (51.0% vs. 43.1%, p=0.05), a higher prevalence of 30-day cigarette use (16.8% vs. 11.6%, p=0.01), and a lower propensity-for-attrition score (0.72 vs. 0.79, p=0.002).

Differential Attrition: Retention ranged across the three conditions between 70.6% and 79.4% and the test for a difference in these rates was marginally significant (p<.10). Differential attrition was not significant across experimental conditions by substance use status. However, when comparing the analyzed sample (n=2,538) to those lost to follow-up (n=706) on nine baseline variables, statistically significant differences were found on all nine variables. Retained students were younger (14.7 vs. 15.1 years old), less likely to be enrolled in an alternative school (4.3% vs. 8.6%), more likely to be living with both parents (66.2% vs. 49.3%), and had parents with a higher level of education. In addition, the retained sample contained more whites (43.9% vs. 31.1%) and fewer Hispanics (27.1% vs. 34.5%), and had a lower prevalence of substance use (12.9% vs. 23.5% for cigarette use, 31.4% vs. 38.6% for alcohol use, 12.6% vs. 24.5% for marijuana use, and 5.5% vs. 9.7% for hard drug use). Controlling for the propensity-for-attrition score aims to adjust for these differences.

One-Year Follow-up: Results revealed that when comparing combined TND conditions to the control condition, there was a marginally significant reduction in marijuana use (p<0.10, OR=.77) and a significant program effect on hard drug use for baseline non-users (p<0.05, OR=.61). Program effects in both instances were weak. Comparisons between the two intervention conditions (teacher support versus regular workshop) showed no significant differences overall or by baseline use status. Of the 24 significance tests, only one reached significance at the .05 level.

Effect size: The study reported weak effect sizes (OR=.77) for reductions in marijuana use when comparing intervention and control conditions and reductions in hard drug use (OR=.61) among non-users.

Study 7

Summary

Barnett et al. (2012), Lisha et al. (2012), and Sussman et al. (2012) used a cluster randomized controlled trial that assigned 24 continuation high schools in California with 1,186 students to the intervention, the intervention plus motivational interviewing, or a control group. Student data on substance use were gathered at a one-year follow-up.

Lisha et al. (2012) and Sussman et al. (2012) replicated the program effects on

• alcohol, overall substance use, and hard drug use at immediate and one-year assessment.

Evaluation Methodology

Design: This randomized clinical trial was conducted with continuation high schools in four counties in southern California. Eligibility criteria (e.g., ethnic diversity, proximity to project headquarters, school size) were established and 61 continuation high schools meeting the inclusion criteria were rank ordered based on drug use risk. Drug use risk was determined by calculating a linear composite score for each school using factors that predicted adolescent drug use. Schools with the highest score for drug use risk were recruited, and when three schools with similar drug use risk scores were recruited, they were randomly assigned to one of three conditions (control, TND only, TND+MI) until there were 8 schools in each condition. Of the initial 61 schools that met inclusion criteria, 22 were unable to participate and 15 were never approached.

Within each of the 24 schools, at least two classrooms were selected to participate in the study. Of the 2,397 students enrolled in the 24 schools and 48 classes, 1,694 (70.7%) consented to participate (Sussman et al. report n=1704) and 1,676 completed baseline measures. Of these students, 1,426 (85.1%) completed an immediate posttest and 1,186 (70.7%) students completed the post-booster assessment.

Project TND was delivered in 12 classroom sessions lasting 45 minutes each and 3-4 weeks overall. An immediate posttest was conducted at the completion of the TND program. The MI booster component was then provided to the youth in the TND+MI schools. The booster component comprised three 20-minute contacts between the youth and an MI interventionist, with the first contact conducted in person 1-3 days after the completion of the immediate posttest and the second and third contacts conducted by phone on 3-4 month intervals.

Sample Characteristics: The participants were continuation high school students with an average age of 16.8 years; 56.6% were male, 64.9% were Hispanic and 11.7% were white. 51.3% of the participants lived with both parents and 55.9% of the youths' mothers had completed high school. At baseline, 70.1% of the youth reported the use of tobacco, alcohol, marijuana and/or hard drugs in the previous 30 days. The selection of continuation high schools with the highest risk of drug abuse problems led to a socioeconomically disadvantaged sample.

Measures:

Measures completed at the initial posttest included:

Program-specific knowledge: using 12 items (e.g., "In terms of damage to your lungs, two joints are equivalent to how many cigarettes?").

Motivation/Cognitive Misperception: using three scales: drug use myths (4 items; alpha=0.56), health as a value (3 items; alpha=0.78), and motivation to improve (3 items; alpha=0.80).

Skills: using three self-report scales: social self-control (10 items; alpha=0.73), assertiveness (4 items; alpha=0.60) and engagement coping (6 items; alpha=0.85).

Decision-making/commitment: using two scales: decision-making confidence (3 items; alpha=0.73) and decision-making avoidance (3 items; alpha=0.73).

Drug use intentions: using four items (e.g., "How likely are you to use cigarettes in the next 12 months?").

Risky sexual behavior intentions and norms: using two scales: sexual behavior intentions (5 items; alpha=.81) and perceived norms (two items; no alpha provided; e.g., "In your school, about how many females (males) your age do you think have sexual intercourse without condoms?")

Measures completed at post-booster assessment one- ear after the delivery of TND included:

Substance use: from asking youth how many times in the past 30 days they had used 12 different substance categories. A hard drug use index (alpha=.82) was calculated by summing youth responses on frequency of hard drug use (e.g., cocaine, hallucinogens, stimulants, inhalants, ecstasy, etc.). An overall substance abuse index was also calculated using all 12 substance categories.

Risky sexual behavior: using four questions related to frequency of intercourse while using substances, condom use, and number of partners in the last 30 days and in the last 12 months.

Analysis:

Lisha et al., 2012: Immediate outcomes were examined using a generalized mixed linear model with the program condition as the fixed effect and school (the unit of randomization) as the random effect. A propensity for attrition score was calculated for each participant in the analytic sample and comparisons were made between program conditions and control, and between the two program conditions; a two-tailed significance test was used.

Sussman et al., 2012: One-year follow-up outcomes were examined using three different types of mixed models: dichotomous outcomes with a logit link function, ordinal count measures using zero-inflated negative binomial distribution modeling, and continuous outcomes using mixed linear regression. The models included the treatment conditions as determinants at the school level and a control for the baseline outcome. However, the results report one-tailed tests of significance.

Intention-to-treat: The study used data on all students who provided valid assessments regardless of dose of intervention received.

Outcomes

Implementation Fidelity: Program fidelity was assessed immediately following program delivery using self-report questions that the program providers (health educators) completed. In 86.1% of the lessons delivered, health educators reported that they did not omit any material specified.

Average student attendance at the program was 77% (Lisha et al., 2012) and students attended an average of 67% of the sessions.

The MI booster component was closely monitored through audiotapes that were then coded to see if they met proficiency standards. Based on the coding of the MI interventionists, 88% exceeded proficiency in MI consistent behaviors. For the MI booster component, 93% of the students were reached for at least one contact.

Baseline Equivalence: The three conditions (control, TND only, and TND+MI) were compared across 16 baseline measures and no statistically significant differences were found on 15 of these measures. The only statistically significant difference was that there were fewer males in the TND only group (52.5%) compared to the control (61%) and TND+MI (59.7%) groups. Gender was included as a covariate in the models.

Differential Attrition: A comparison of youth who completed pretest only versus youth who completed both pretest and immediate posttest was conducted and no significant differences were found on the 16 baseline measures (Lisha et al., 2012).

Comparing youth who completed one-year follow-up (n=1,186) to those who were lost to follow-up (n=494) on 11 baseline measures, 3 significant differences were found: the retained sample was slightly younger (16.7 versus 16.9), more likely to live with both parents (51.3% versus 44.4%) and less likely to smoke marijuana (45.5% versus 53%). The attrition rate was not statistically different across the program conditions and ranged from 30.6% to 36.8% (p=.58). To adjust for attrition bias at one year follow-up, a propensity to attrition score was calculated.

Posttest: Lisha et al., 2012: Because the posttest was conducted before the MI booster, no difference between TND only and TND+MI was expected. Of the seven scales used to measure knowledge, skills and motivation/cognitive misperception, three were significant when comparing the control to program conditions. The program condition significantly predicted a gain in knowledge score (p<.0001) and a decrease in belief of drug use myths (p<.05). Unexpectedly, however, students receiving the program had lower levels on valuing health than did students in the control group (p<.05). No significant differences were found on the three measures of skills, motivation to improve or the two measures of decision-making. All four measures of drug use intentions were significantly different when comparing control to program conditions (p<.0001). On the measures of risky sexual behavior intentions and norms, none of the five measures of sexual behavior intentions were significant, however the two measures of sexual behavior norms were both significant (p<.0007).

Post-booster assessment at one year post-TND: Sussman et al., 2012: Adjusting the reported one-tailed tests to include only those with p values of .025 or lower, 3 of the 12 substance abuse measures showed significant differences between any TND programming and standard care. Significant differences occurred in use of alcohol (p=0.01), the substance use index (p=.014), and the hard drug use index (p=.023). Five other outcomes showed marginally significant differences (p < .10 in a two-tailed test). However, there were no significant differences between TND only and TND+MI groups, indicating that the booster sessions brought little or no benefit.

TND programming did not affect risky sexual behaviors and there were no differences in results between TND only and TND+MI.

Study 8

The study evaluated a nine-session version of the program.

Summary

Mohammad et al. (2010) used a cluster randomized controlled trial that assigned eight Iranian high schools to the intervention group (n = 577 students) or a school-as-usual control group (n = 513 students). At six months after program implementation, a student survey measured self-reported substance use, truancy, and violent behavior.

Mohammad et al. (2010) lacked significance tests and percentages showed only small differences between conditions for substance use, truancy, and violence.

Evaluation Methodology

Design:

Recruitment: The study examined 1,180 students from eight traditional (public) high schools in Zahedan (southeastern Iran). The participants ranged from 15-18 years of age.

Assignment: The study appeared to use random assignment, stating that "sample high schools were selected randomly and assigned to either intervention or control conditions." The sample sizes after attrition were 577 for the intervention group and 513 for the school-as-usual control group.

Assessments/Attrition: Post-baseline assessments came one week and six months after "implementation of the program," but only the intervention students completed the one-week assessment. With 90 students having incomplete questionnaires, the final sample included 1,090 of the 1,180 students (92%).

Sample:

The sample came from high schools in Iran, but the study presented no information on the socioeconomic composition of the sample.

Measures:

Behavioral measures came from the High Risk Behaviors Questionnaire. Students provided self-reports on smoking, illicit drug use, truancy, physical fighting, verbal fighting, and carrying a weapon. Researchers also developed a measure of knowledge with subscales for effective communication, open-mindedness, and self-control.

Analysis:

The analysis consisted of reporting percentages for the two conditions at the six-month posttest, without statistical tests, baseline outcome controls, or adjustments for clustering.

Intent-to-Treat: The analyses appeared to use all available data, excluding only those who failed to complete the surveys.

Outcomes

Implementation Fidelity:

Not examined.

Baseline Equivalence:

Tables 1 and 2 present the means for baseline outcomes of the two conditions but without significance tests and without figures on sociodemographics. The means showed some modest deviations, the largest being for truancy (32% for the intervention group and 23% for the control group).

Differential Attrition:

Not examined.

Posttest:

The figures presented did not include significance tests. Although smoking, illicit drug use, truancy, and fighting appeared to decline more among the intervention youth than the control youth from baseline to the six-month follow-up, the condition differences at the follow-up appeared small. The measures of knowledge, however, showed clear condition differences at the follow-up that favored the intervention group.

Long-Term:

Not examined.