Reducing The Risk

Study 1

Evaluation Methodology

Design: The study took place in 13 high schools in 10 school districts in rural and urban California where 46 classrooms of students that were taking a mandatory health education class were assigned to either treatment or comparison groups. The comparison group received a sexual education curriculum that was already being utilized at the school. The assignment of classrooms was sometimes based on size and otherwise randomly assigned. Survey questionnaires were given to both the treatment and comparison groups before the intervention, immediately after the intervention and then at 6 and 18 months after the intervention. The total number of students in the study started at 1033, however a total of 758 (429 in the treatment group; 329 in the comparison group) completed the final survey at 18-months resulting in an identical attrition rate of 27% across both control and treatment groups. This study only presented results comparing pretest and 18-month results.

The Reducing the Risk curriculum was taught by high school teachers who volunteered to do so and were provided with three days of training focused on delivery of the curriculum, facilitation of role-play sessions and methodological concerns including collection of data and preventing contamination of control classes. In order to address this latter issue, the study randomly assigned half the classes where the treatment and comparison classes were taught by the same teachers and the other half to where the treatment and comparison classes were taught by separate teachers. In this way, half the treatment classes and half the control classes were taught by the same teachers and the other half were taught by separate teachers. The benefit of this was to account for the possibility that the teachers who volunteered to teach the experimental curriculum were better teachers in general, however it did increase the possibility that the teachers who taught both treatment and comparison classes allowed the RTR curriculum to contaminate their regular comparison curriculum. However, no analysis was presented at either the classroom or teacher level to support these assumptions.

Sample Characteristics: The sample was 47% male, 62% white and 20% Latino, 27% in 9th grade, 56% in 10th grade and 17% in 11th and 12th grades with a mean age of 15.3. Based on background characteristics gathered from the students, some students were classified as being at a higher risk for engaging in sexual risk-taking behavior. They had the following characteristics in common: they did not live with both parents; their mother did not complete high school; they were receiving grades that were Ds or lower; they drank one or more times in the preceding month and usually drank five or more drinks on each occasion. These higher risk students, who had at least one of these risk factors, made up 52% of the sample.

Measures: Information was gathered before the intervention, immediately after the intervention and then at 6 and 18 months after the intervention on a variety of different measures including contraceptive knowledge and practice, perception of norms, behavioral intentions, sexual behavior, pregnancy, and communication with parents. Other than for behavioral intentions, no information about the reliability or validity of the measures was provided.

• Contraceptive knowledge - A 20-item knowledge-based test with statements like "A girl cannot get pregnant the first time she has sex" and "A girl can get pregnant any time of the month" and allowing for true, false or I don't know responses.
• Perceptions of norms - One survey question regarding the students' perception of what proportion of students their age have had intercourse with responses ranging from 1 (few) to 5 (all).
• Intentions - An 11-item test that involved watching vignettes of social situations that students encounter and responding with their likely action; the students' intention to use birth control or abstain from sexual intercourse was measured. This test had a reported alpha coefficient of .76.
• Sexual behavior - Three variables from single questions measuring prior sexual intercourse, use of contraception during intercourse and frequency of intercourse in the past month.
• Pregnancy - Survey questions about whether the student was pregnant or had made someone pregnant.
• Communication with parents - Survey questions of youth and parents regarding communication with parents about abstinence, birth control, pregnancy and STDs.

Analysis: Because this study focused on the long-term impact of the program, only data from students who responded to both the pretest and the 18-month follow-up survey were analyzed. An intent-to-treat analysis used logistic regression, chi-squared tests of significance and t-tests at the student-level. Results regarding initiation of intercourse, as well as use of protection during intercourse, were also analyzed at the sub-population level looking at differences by gender, race and risk. This study did not present analysis done comparing pretest and posttest scores. Analysis of change scores incorporated baseline controls. No analysis was done at the classroom level, even though that was the level of randomization.

Outcomes

Fidelity monitoring: Members of the evaluation team observed the treatment classes a minimum of once to assess implementation fidelity, and using Likert scales rated follow through with, and delivery of, the curriculum. Ratings of fidelity were high.

Baseline Equivalence and Attrition: The treatment and comparison groups were statistically equivalent at baseline on a variety of sociodemographic variables. Further, there was no statistical difference in measures of contraceptive knowledge or outcome variables related to behavior and beliefs.

Of the 1033 students who began the program, 758 completed the 18-month follow-up survey resulting in a 27% attrition rate that was identical for the control and treatment groups. No information was provided for the attrition rate during the program or at posttest. The students who did not respond to the 18-month follow-up survey had the following pretest characteristics: older, male, poor grades, did not live with both parents, had already initiated intercourse, and had failed to use contraception in the past. However, no information was provided about whether this differential attrition was the same in the treatment group versus the comparison group.

Posttest: In order to study the long-term effects of the Reducing the Risk curriculum, this study did not report the results of analysis between the pre and posttest scores.

Long-term: Results for knowledge, beliefs and behavior measures were mixed.

• Contraceptive knowledge - the treatment group showed significantly greater gains than the comparison group both at 6-months and 18-months and across all subpopulations on overall scores as well as on 11 of the 20 questions at 18-month follow-up and 9 of the 20 at 6-month follow-up.
• Perceptions and Intentions - there was no significant difference between the comparison and treatment groups at the 18-month level for either changes in perception or changes in intention, though the difference in scores for changes in perception was significant at the 6-month level at p<.01.
• Behavior - difference in the changes in three of seven measures for sexual behavior (intercourse and contraceptive use) were significant between the treatment and comparison groups.
• Pregnancy - no statistically significant difference between the comparison and treatment groups was found for rates of becoming pregnant or making someone pregnant.
• Communication with parents - only one item (talking with parents about abstinence) of the four communication items was significantly different between treatment and comparison groups at the 18-month follow up.

Analysis was also conducted on subpopulations (by gender, ethnicity, risk level), however no significant difference was found except in the case of lower risk students and female students who showed a statistically significant difference in initiating intercourse than the comparison group. Further, among lower risk students and students who had not initiated intercourse before the program, the RTR curriculum reduced the likelihood of unprotected intercourse by 40%.

Study 2

Evaluation Methodology

Design: Using a quasi-experimental design, the study matched five comparison school districts to five treatment school districts in rural and urban Alabama. One classroom in each treatment and comparison school district was randomly selected for study. The comparison group received a sexual education curriculum that was already being utilized in the school district. The study began with 532 students who completed surveys, 512 of which were usable; however, only a total of 212 completed the final survey at 18-months, resulting in a 58% attrition rate. No information was provided whether there was differential attrition across the treatment and control groups.

As with the 1991 study, the Reducing the Risk curriculum was taught by middle and high school teachers who volunteered to do so and were provided with three days of training focused on delivery of the curriculum, facilitation of role-play sessions and accessing health resources.

Sample Characteristics: The sample was 48% male, 85% White and 14% Black, 8% in 9th grade, 80% in 10th and 11th grades and 12% in 12th grade. Unlike the initial study, this study did not differentiate the population based on risk. However, this study did gather information on religious attendance and religious affiliation, with 55% attending services more than once a month and 54% identifying as Baptist.

Measures: Information was gathered before the intervention and at 18 months after the intervention (but no posttest). The assessments included a 28-item survey that used 22 items from the Youth Risk Behavior Survey (1993), 2 items on religious attendance and affiliation, and 4 items about communication with parents.

Analysis: The study used an intent-to-treat analysis and compared data from students in the comparison and treatment groups who responded to the initial and 18-month surveys. Data on initiation of sexual intercourse, use of contraception during intercourse and communication with parents were analyzed using a one-tailed test for difference in proportions. This study did not present analysis done at any subsample level (gender, ethnic or religious) and while analysis was conducted at the individual level, no analysis was presented at the district level, even though that was the level at which the treatment and comparison samples were matched.

Outcomes:

Fidelity Monitoring: Teachers who delivered Reducing the Risk completed a self-report questionnaire on the degree of implementation of the curriculum. Only 29% of teachers delivered all 16 lessons, 29% completed 15 lessons and 42% taught 11-12 of the lessons. No dose-response analysis was conducted to analyze the effect of this differential delivery of the curriculum.

Baseline Equivalence and Attrition: The treatment and comparison groups were statistically equivalent at baseline on characteristics such as grade, race, gender, religious attendance and religious affiliation. There was also no significant difference in outcome variables such as initiation of sexual intercourse or communication with parents. No item-level information was provided about baseline equivalence on any other measures on the enhanced YRB Survey.

Of the 512 students who completed usable surveys at pretest, only 212 responded at 18 months resulting in a 58% attrition rate. Reasons for attrition included graduation, drop-out, absenteeism and missing posttest data. No information was provided on whether there was differential attrition in treatment and comparison groups or if the attrition differed by certain characteristics.

Posttest: No data were gathered immediately after the delivery of the curriculum in this study. Only 18-month follow-up data were gathered.

Long-term: Results for sexual behaviors and communication with parents were mixed. There was a marginally significant difference (p<0.10, z= -1.79) between treatment group students and comparison group students on becoming sexually active after initiating the Reducing the Risk program. However, among students who became sexually active after pretest, the students in the treatment group were significantly more likely to use contraception than students in the control group (p<0.05, z= 2.98). There was no significant difference in the use of contraception between treatment and comparison group students who were already sexually active at the time of the program.

Of the four items used to measure parental communication, differences between treatment and comparison groups were significant on two items - birth control and prevention of STDs/HIV.

Effect Size: No effect size was reported.

Mediating Effects: None reported.

Study 3

Evaluation Methodology

Design: A total of 28 schools from Cleveland, OH and Louisville, KY were invited to participate in this three-year longitudinal study. These schools were selected because they were racially diverse, had at least 200 ninth-grade students, and taught a health education class. Of the schools invited, 17 chose to participate and 11 schools declined because they were concerned about the explicit content of the curriculum. The 17 schools that participated in the study comprised 10 high schools from the Louisville, KY area and seven high schools from the Cleveland, OH area. Using a matched-pairs design, the three curricula (school's standard curricula, Reducing the Risk, modified Reducing the Risk) were randomly assigned to classrooms. However, some schools did not have a standard HIV prevention curriculum and no information was provided about what the students in the comparison group did instead, nor was analysis conducted separately for the comparison classrooms that did not provide any standard curriculum. In addition to the curriculum, which was delivered when the students were in 9th grade, students received a booster session during 10th grade. No information was provided about what this booster session entailed for the two treatment groups.

Students completed surveys at pretest (beginning of 9th grade), 6 months after the delivery of the curriculum (end of 9th grade) and 18 months after delivery of the curriculum (end of 10th grade). The sample size started at 2,647 students of whom 1,944 completed follow-up surveys, though only 1,424 were used in the analysis (48% attrition rate).

As in both previous studies of the Reducing the Risk curriculum, teachers received 2.5 days of training on the delivery of the curriculum. Unlike previous studies, peer leaders were chosen to support the delivery of the modified curriculum based on low scores on sensation-seeking and impulsivity scales, and peer leaders were provided 2 days of training to lead discussions, guide the role-plays, and assist teachers delivering the modified Reducing the Risk curriculum.

Sample Characteristics: The sample was 53.1% female, 51.1% White and 35.5% Black, 86% age 14-15, 60.8% reporting not sexually experienced, and 54.6% of those who were sexually experienced reported using condoms.

Measures: Information was gathered before the intervention, at 6 months, and at 18 months after the intervention on a variety of different measures including socio-demographic variables, personality characteristics, contraceptive knowledge and practice, perceptions and behavioral intentions, sexual behaviors and self-efficacy.

• Demographic information - including variables such as socioeconomic status (measured by asking what the students paid for school lunch), relationship status (are you going out with someone regularly) and educational aspirations (how far do you expect to go in school).
• Contraceptive knowledge - assessed using 10 true/false items (e.g., the best way to use a condom is to leave some space at the tip for the sperm).
• Perceptions - assessed using one survey question regarding the students' perception of what proportion of students their age have had intercourse with responses ranging from 1 (none) to 6 (all).
• Intentions - assessed using 4 items regarding intention to use birth control or abstain from sexual intercourse.
• Sexual behavior - assessed using single-item questions to measure prior sexual intercourse, use of contraception during intercourse and use of alcohol during intercourse.
• Personality characteristics - sensation-seeking and decision making style was assessed using the 16-item adolescent version of the Sensation-Seeking Scale (alpha 0.87) and the 11-item Decision-Making Style Scale (alpha 0.76). Students who scored above the mean on either scale within their racial group by gender were classified as high sensation-seekers and impulsive decision makers respectively.
• Self-efficacy - measures of refusal self-efficacy (6-item scale, alpha 0.89), condom self-efficacy (5-item scale, alpha 0.83) and situational self-efficacy (4-item scale, alpha 0.81) were collected to evaluate students' perceived ability to refuse sexual intercourse, use condoms during intercourse, and respond effectively to different situations of sexual pressure.

Analysis: Hypotheses were tested using logistic regression and mixed model hierarchical linear regression. Analyses were also conducted to determine differences between high and low sensation-seekers as well as impulsive versus rational decision makers controlling for race, gender, educational aspirations and relationship status.

Outcomes:

Fidelity Monitoring: Implementation was monitored using ratings from students about the delivery of the curriculum. Questions included use of role-plays (85% of students in the modified RTR treatment group versus 71% in original RTR treatment group indicated that students did role-plays), who had visited the classroom (75% and 97% of students accurately identified the two guest speakers) and who led the discussions (97% of students in the modified RTR treatment group reported that group leaders led discussions). No external observers monitored the delivery of the curriculum, nor did teachers provide information about their assessment of their delivery of the curriculum.

Baseline Equivalence and Attrition: The comparison and two treatment groups were equivalent at baseline on five of eight characteristics measured (age, city, relationship status, high-school grades, socioeconomic status) but were not equivalent on gender, race and educational aspirations. Treatment groups had higher proportions of male students and the modified Reducing the Risk treatment group had a higher proportion of black students, lower proportion of white students and higher proportion of students who wanted to attain advanced degrees. These differences at baseline were statistically significant.

The sample size started at 2,647, dropped to 73% at baseline due to illegible or incomplete surveys, 68% at Time 2 and 52% at Time 3. The differences in drop-out rates were significant across the three groups. There was also differential attrition across sample characteristics, with students who were male, sexually experienced at baseline, and had lower educational aspirations being more likely to not respond to follow up surveys.

No adjustments were made in the analysis for deviations from baseline equivalence or for differential attrition.

Posttest: No data were gathered immediately after the delivery of the curriculum. Data were gathered at 6 months and 18 months after the delivery of the curriculum.

Long-term: Results presented in the study were mixed. There were no significant bivariate differences between the three curricula (standard, Reducing the Risk and modified Reducing the Risk) in any of the outcome variables (initiation of intercourse, use of condoms, use of alcohol during intercourse) at Time 1, 2 or 3. Further, Reducing the Risk and modified Reducing the Risk did not differ from each other with high-sensation seekers or impulsive students. However, based on multiple logistic regression, the odds of initiating intercourse for students in the comparison group was 2.42 times that of both treatment groups combined.

Mediating Effects: Of the eight mediating variables tested, only one - contraceptive knowledge - increased significantly for both treatment groups over the comparison group between Time 1 and Time 2. Of the seven mediating variables tested between Time 1 and Time 3, no significant differences were found between treatment groups and comparison groups.

Effect size: The odds ratio, at 2.42 for initiating sexual intercourse, the only significant outcome present in the analysis, shows a moderate effect size.

Study 4

Evaluation Methodology

Design:

Recruitment: The sample was recruited from adolescents aged 14-19 years who were classified as being at high risk for teen pregnancy and were involved in out-of-school programs at 23 different youth-focused organizations. The organizations were faith-based agencies, community centers, social service agencies, and resource centers in Louisville, Kentucky schools with the highest rates of poverty and minority students. A total of 1,448 youths were recruited, eligible to participate, and completed at least one day of the two-day intervention or control programs.

Assignment: At each camp, youths were randomly assigned by the research manager to one of three or one of two clusters (A, B or C) using statistical randomization software, depending on the number of youth who presented themselves that morning, with the exception of 5 adolescents who were knowingly assigned to ensure gender balance and to ensure that all members of the same household were in the same cluster. The clusters were then randomly assigned to the intervention group (Reducing the Risk), a control (Power of We) condition, or an intervention group for another program. Of the 39 camps or cohorts, 8 had too few participants to assign to all 3 conditions, so the clusters were randomly assigned to the 2 intervention conditions. The randomization was double-blind, because the evaluators were blinded to each condition.

There were 515 youths in 39 camps assigned to the Reducing the Risk intervention and 422 youths in 31 camps assigned to the control group. The control condition consisted of a program focused on exploring neighborhood assets and ways to bring about positive change in the community, with no mention of sexual health or other overlap with the intervention.

Attrition: Assessments occurred at baseline, 3 months post-baseline, and 6 months post-baseline. Across all three conditions, a total of 1,090 (75%) completed the 3-month follow-up and 991 (68%) completed the 6-month follow-up.

Sample:

Across all participants, the average age was around 16 years, about 63% were female, approximately 7% were non-Hispanic White, 90% were non-Hispanic Black, and 3% were Hispanic. About 15% had had sex without condoms in the past 3 months, 12% had had sex without other forms of birth control in the past 3 months, 40% had ever had sex, and the average number of partners in the past 12 months was 1.

Measures:

Four questions about risky sexual behavior were asked of participants: Sexual intercourse without a condom in the past 3 months and sexual intercourse without using any forms of birth control in the past 3 months were measured with "yes or no" questions; the number of different sexual partners in the past 12 months (past 3 months for the follow-up questionnaires) was an open-ended question; and participants were asked whether they had ever had sexual intercourse. The study provided no information on the validity of the measure but stated that the procedure minimized the potential for social desirability bias.

Analysis:

Hierarchical linear modeling was used to analyze the effects of treatment on the outcome variables, controlling for cluster, cohort, baseline outcomes, and other relevant covariates.

Intent-to-Treat: There was little information given on intent-to-treat. It appears that all participants with data were used in the analyses, but is unclear if efforts were made to follow all participants.

Outcomes

Implementation Fidelity:

The study reported only on attendance, noting that 95% of participants attended both days of the training.

Baseline Equivalence:

Only one demographic variable varied significantly between intervention and control groups, the percentage of non-Hispanic black participants (p = 0.05). As such, this was included as a covariate in all analyses, but it did not substantially affect outcomes.

Differential Attrition:

Participant retention was slightly higher in the control group (80% at 3 months, 73% at 6 months) than in the intervention group (75% at 3 months, 66% at 6 months). In additional material sent by the authors, there were no significant differences in gender or age between those who completed the program and those who dropped out, but there was a significant difference in ethnicity, such that African-Americans plus immigrants from Africa and the Caribbean were more likely to complete the study than other ethnicities. Also in the additional material, a series of t-tests on baseline outcomes using only those participants who completed the entire study found no significant differences across conditions, indicating that attrition did not affect baseline equivalence on the primary outcomes.

Posttest:

There was a significant positive effect of treatment on 2 of 4 outcomes: failure to use birth control (other than condoms) and the number of sexual partners at both the 3- and 6-month follow-ups.

Long-Term: Not examined.

Registry

The clinical trial registry for the study can be found at Clinical.Trials.gov: https://www.clinicaltrials.gov/ct2/show/NCT01411878?term=NCT01411878&rank=1 (registration number NCT01411878 with the title Louisville Teen Pregnancy Prevention Project (CHAMPS)). As the primary outcomes, the registry lists the number of girls that get pregnant, and times a boy impregnates someone over a 2-year time frame. As the secondary outcomes, it lists the number of sexually transmitted infections participants contracted over a 2-year time frame. The registry mentions delaying sexual initiation and use of birth control in the purpose section but not as primary or secondary outcomes.

Study 5

Evaluation Methodology

Design:

Recruitment: The sample was drawn from schools that agreed to participate in one of three sites that received grants through the federal Teen Pregnancy Prevention Program, including six high schools in St. Louis, Missouri, five high schools in Austin, Texas, and six middle, junior high, and high schools in San Diego County, California. School staff identified classes to participate, and all students who obtained parental permission were enrolled. The initial sample consisted of 3314 students across 150 classes.

Assignment: Participating classrooms in each school were assigned to the intervention or a "business as usual" control condition by evaluation staff, with more classes assigned to the treatment condition overall (depending on school and program preferences). The content of the control condition differed across schools, and included regularly-scheduled health, physical education, ROTC, homeroom, and science classes.

Attrition: The follow-up assessment occurred 12 months after baseline and an unspecified period after the end of the 16-session program. Attrition was high at 19%. Of the 3314 students who consented to participate, 98% completed the baseline survey and 2689 (81%) completed the 12-month follow-up survey.

Sample:

The sample included students from public schools in St. Louis, Austin, and San Diego County. Overall, about half of the participants were female, one third were black, almost half were Hispanic, and the rest were about equally divided between White and other races. Participants were 15 years of age on average.

Measures:

All measures were self-developed by the researchers. Alphas for the scales ranged from α = 0.75 to 0.86. No information on validity is reported.

Behavioral outcome measures included the following seven yes/no questions about youth sexual behavior:

1. Ever been sexually active;
2. Been sexually active in the past 90 days;
3. Had sexual intercourse in the past 90 days;
4. Had oral sex in the past 90 days;
5. Had sexual intercourse without birth control in the past 90 days;
6. Had sexual intercourse without a condom in the past 90 days; and
7. Had oral sex without a condom in the past 90 days.

Risk and protective factors were measured through composite measures created from the following scales:

1. 12-item scale for attitudes toward protection;
2. 7-item scale for attitudes toward risky sexual behavior;
3. 4 items about knowledge of pregnancy risk;
4. 12 items about knowledge of STI risk;
5. 3-item scale measuring motivation to delay childbearing;
6. 4 items for intentions towards sexual risk behaviors;
7. 6-item scale for refusal skills; and
8. 7-item scale for condom negotiation skills.

Analysis:

Effects of the program on students' sexual activity in the previous 90 days and sexual intercourse without birth control in the previous 90 days were tested using regression models with a Benjamini-Hochberg correction to adjust for multiple comparisons. Effects on protective factors (attitudes, knowledge, motivation, intentions, skills) were considered exploratory and did not adjust for multiple comparisons. All models controlled for baseline outcomes and randomization blocking variables (site, school, year, semester, and gender). Data were pooled across classrooms, schools, and research sites, with no tests or adjustments for clustering.

Intent-to-Treat: All available data were used in all analyses.

Outcomes

Implementation Fidelity:

Health educators who implemented the program completed fidelity checklists, and sessions were observed by supervisory staff selected by the funder. The researchers report that the program was implemented with fidelity. Attendance varied by replication site; in Austin, only 47% of participating students attended at least 75% of program sessions, compared to 85% of students in San Diego and 73% of students in St. Louis.

Baseline Equivalence:

In tests utilizing the analysis rather than the randomized sample, no significant differences were detected between groups at baseline.

Differential Attrition:

Although attrition was high overall (19%), differential attrition between conditions was low (only 0.52%). Baseline characteristics of dropouts did not significantly differ between conditions. Also, there were no differences in tests for baseline equivalence using the analysis sample.

Posttest:

At 12 months after baseline, no significant differences in sexual behaviors were detected between treatment and control participants. The program had positive impacts on protective factors, including attitudes towards protection, knowledge of pregnancy risk, and knowledge of STI risk, but no overall effects on motivation, intentions, or skills.

Long-Term:

Not examined.