Power Through Choices

Study 1

Evaluation Methodology

Design:

Recruitment: The sample was recruited between 2012 and 2014 from teenagers aged 13-18 years and living in eligible group homes serving youths from the child welfare or juvenile justice systems. A total of 1,036 or 1,037 youths from 44 residential homes in California (N = 19), Maryland (N = 10), and Oklahoma (N = 15) were eligible and consented to participate. Homes were included where 80% of youth or a minimum of six youth consented to the study and "if they had the capacity and commitment to support the study."

Assignment: Group homes were the unit of randomization. Homes were re-randomized up to four times if the population of residents turned over completely between implementations. A total of 44 group homes were randomized a total of 80 times and clustered according to location, recruitment date, number of youth served, and gender of youth served. Equal numbers of homes were assigned to the intervention (N = 40) and control groups (N = 40), and the groups contained approximately equal numbers of youth (intervention N = 517, control N = 519). Group homes in the control condition received "usual care," which was not related to reproductive health but could include some educational programs on topics such as healthy eating, for example.

Assessments/Attrition: Assessments occurred at pre-intervention, post-intervention, 6-month, and 12-month follow-ups. A total of 952 (92%) completed the post-intervention survey, 853 (82%) the 6-month follow-up, and 885 (85%) the 12-month follow-up. No clusters were lost.

Sample:

Of the 1,036 participants, about 78.7% were male and the mean age was 16.1. The participants were 36.6% Hispanic, 23.6% White, 19.5% Black, 13.8% American Indian or Alaska Native, 5.3% Asian or Pacific Islander, and 1.3% unknown. The majority (61.3%) of the group homes served both child welfare and juvenile justice youth, 20.5% were contracted to serve juvenile justice youth, and 18.2% were contracted to serve child welfare youth.

Measures:

Measures of behavioral outcomes came from youth self-reports. Covington et al. (2016) measured history of unprotected sex with binary variables for whether participants had ever had sex without effective protection (also in Oman et al., 2018) and whether they had ever had sex without a condom. For sexual activity, a binary variable was included for whether participants reported having sex within the previous three months.

Pregnancy risk was assessed with binary variables for whether the participant had ever been pregnant or gotten someone pregnant (also in Oman et al., 2018) and whether they had been pregnant or gotten someone pregnant within the prior three months.

To determine STI risk, binary variables measured 1) whether the participant reported multiple sex partners in the prior three months, 2) whether they had been tested for STIs over the preceding 12 months, and 3) whether they had been diagnosed with an STI over the preceding 12 months.

Measures of risk and protective factors were used in both Oman et al. (2016) and Green et al. (2017). The measures came from subject self-reports. Constructs in the areas of knowledge, attitudes, self-efficacy, and behavioral intentions regarding sexual activity were created using exploratory factor analysis on a polychoric matrix (using principal component analysis and varimax rotation). Reliabilities for the scales were generally high.

Knowledge was assessed in the areas of Reproductive Anatomy (summed from 4 items), HIV and other STIs (7 items), and Methods of Protection (10 items); attitudes were measured using two constructs, Support for Methods of Protection (created from 6 items) and Barriers to Methods of Protection (5 items); self-efficacy was examined through two constructs, Ability to Communicate with Partner (from 3 items), Plan for Protected Sex and Avoiding Unprotected Sex (3 items), as well as one additional item measuring the ability to find a place in the community to obtain birth control of some kind; finally, three behavioral intentions toward sexual activity were measured, the intention to have sex in the next year, the intention to use a condom in the next year, and the intention to use other forms of birth control in the next year.

Analysis: In Oman et al. (2016), the posttest data were analyzed with generalized linear mixed models to look at the effects of the intervention on both the continuous and dichotomous outcome variables, with time (pre vs. post-intervention) and condition (intervention vs. control) as the within- and between-cluster factors, respectively, and age, race, and gender included as covariates in all models. Time-by-condition terms tested for differences in change across conditions. Random intercepts adjusted for clustering within the units of assignment.

Green et al. (2017) compared treatment and control groups using random intercept logistic regression to adjust for assignment of the 80 clusters. Covariates included age, race, gender, living in the group home at 12 months, and baseline outcome measures.

Covington et al. (2016) used OLS regression with clustered standard errors to adjust for assignment of the 80 clusters. Results were compared against alternative model specifications including logistic regression for binary outcomes, multilevel models to address the nested data structure, and cluster-level analysis. Covariates included age, race, gender, and baseline outcome measures. P-values were adjusted to account for multiple tests.

Oman et al. (2018) used logistic regression with random intercepts. They controlled for the baseline outcome and whether the subject was still living at the group home at the end of the follow-up period.

Intent-to-Treat: In Oman et al. (2016), there was little information given on intent-to-treat, but authors noted that "all available data were used in the analyses." In Green et al. (2017), the authors stated that, by including all randomized group homes in the analysis, they used an intent-to-treat approach. In a CONSORT diagram, Covington et al. (2016) noted that only those not completing the 12-month survey were excluded. Oman et al. (2018) used all randomized subjects in the analysis.

Outcomes

Implementation Fidelity:

To evaluate program fidelity, data collectors observed one randomly selected intervention session per program implementation and completed checklists and attendance records. The implementation rate was 100%, and attendance was marked as 87% overall.

Baseline Equivalence:

Using the analysis sample, Oman et al. (2016) reported no significant group differences at baseline among the demographic or outcome variables. Also using the analysis sample, Covington et al. (2016) reported no significant differences between the treatment and control conditions on six socio-demographic measures and 10 sexual behaviors.

Using the randomized sample, Green et al. (2017) reported no significant differences between the treatment and control conditions on five socio-demographic measures and two sexual behaviors. Also using the randomized sample, Oman et al. (2018) reported no significant differences between treatment and control conditions on three socio-demographic measures and five sexual behaviors.

Differential Attrition:

In Oman et al. (2016), tests for baseline equivalence using the analysis sample gave no evidence of a problem. The same was true of Covington et al. (2016) (see Tables II.2-II.3, pp. 15-16). Further, Oman et al. (2018) stated that there were no significant differences in baseline characteristics between those who completed all assessments and those who did not, though no significance tests were reported. 

Covington et al. (2016) in Appendix A showed that non-respondents were significantly more likely than respondents to have ever had sex, have first intercourse at a younger age, and have more lifetime partners, but these differences did not translate into condition differences in the tests for baseline equivalence.

Posttest:

There was a significant positive effect of the intervention on all three knowledge areas, one of the two attitude constructs, all three self-efficacy measures, and two of three behavioral intentions (Oman et al., 2016). Six months after the intervention, knowledge, attitudes, self-efficacy and intention to use birth control had increased significantly more among the intervention than the control group (Green et al., 2017).

Oman et al. (2018) tested for two behavioral outcomes at 6 months. The program significantly reduced having had sex without birth control but did not significantly affect having ever been pregnant.

Long-Term:

At 12 months post intervention and as compared to a control group, Covington et al. (2016) found that intervention participants were significantly less likely to have ever been pregnant or gotten someone pregnant. None of the other seven sexual behavior measures differed significantly between conditions. Oman et al. (2018) tested for two behavioral outcomes at 12 months and found the opposite of the 6-month results: The program significantly reduced having ever been pregnant but did not significantly affect having had sex without birth control.

Subgroup analyses by age lacked direct tests of significance for group differences. However, older participants aged 17 and above were significantly less likely to have had sex, to have had sex without protection over the preceding three months, to have been pregnant or gotten someone pregnant over the preceding three months, and to have ever been pregnant or gotten someone pregnant. There were no effects for younger participants. For two of the outcomes, Oman et al. (2018) noted that there were no significant interactions.

Across all participants and also at 12 months, Green et al. (2017) and Covington et al. (2016) reported positive effects on several risk and protective factors: knowledge of reproductive anatomy and fertility, knowledge of HIV and STIs, ability to locate means of protection, support for methods of protection, and self-efficacy regarding sexual health.