Sign Up For Newsletter

Blueprints For Healthy Youth Development logo

Red Light, Purple Light!: A Self-Regulation Intervention

A preschool-based program intended to improve behavioral aspects of self-regulation in children through circle time games that target attention, working memory, and inhibitory control.

Program Outcomes

  • Academic Performance
  • Cognitive Development
  • School Readiness

Program Type

  • Early Childhood Education

Program Setting

  • School

Continuum of Intervention

  • Universal Prevention

Age

  • Early Childhood (3-4) - Preschool

Gender

  • Both

Race/Ethnicity

  • All

Endorsements

Blueprints: Promising

Program Information Contact

Megan McClelland, Ph.D.
Hallie E. Ford Center for Healthy Children & Families
125 Hallie E. Ford Center
263 SW Campus Way
Oregon State University
Corvallis, OR 97331
Phone: 541-737-9225
Email: megan.mcclelland@oregonstate.edu
Wesite: health.oregonstate.edu/labs/kreadiness

Program Developer/Owner

Megan McClelland, Ph.D. & Shauna Tominey, Ph.D.
Oregon State University


Brief Description of the Program

Red Light, Purple Light! consists of circle time games played over an 8-week period and occurring twice per week during approximately 15 to 20-minute sessions that focus on improving three components of executive function: working memory, attentional shifting, and inhibitory control. These games allow children to practice self-regulation skills in group settings.

Outcomes

Study 1 (Tominey & McClelland, 2011) found that children in the intervention group, relative to the control group, showed significantly greater improvements in:

  • Letter-word identification

Study 2 (Schmitt et al., 2015) reported that children in the intervention condition, relative to children in the control condition, showed significantly greater improvements in:

  • Behavioral self-regulation
  • Cognitive and attentional flexibility

Study 3 (Keown et al., 2020) reported that at the posttest, compared to control, children who participated in the intervention demonstrated stronger:

  • Executive functioning skills

Brief Evaluation Methodology

In Study 1, Tominey and McClelland (2011) evaluated the program in a randomized controlled trial (N = 65) in two preschools in Oregon. They measured self-regulation and academic outcomes in a pretest at the fall of the final year of preschool before kindergarten and in a posttest at the spring of that school year.

For Study 2, Schmitt et al. (2015) evaluated the program in a randomized controlled trial (N = 276) of children enrolled in preschool through the Head Start program in the Pacific Northwest. The study randomized 14 classrooms to intervention and control conditions. Outcomes were measured at the fall of the final year of preschool before kindergarten and in a posttest at the spring of that school year. The measures included indicators of self-regulation and academics.

In Study 3, Keown et al. (2020) conducted a cluster randomized control trial with 15 early childhood education centers in Auckland, New Zealand. Children's executive functioning skills were measured pre-, post-, and 4-month post-intervention.

Study 2

Schmitt, S. A., McClelland, M. M., Tominey, S. L., & Acock, A. C. (2015). Strengthening school readiness for Head Start children: Evaluation of a self-regulation intervention. Early Childhood Research Quarterly, 30, 20-31.


Race/Ethnicity Specific Findings
  • Hispanic or Latino
Race/Ethnicity/Gender Details

The program targets all groups but has been tested in predominantly low-income samples. Schmitt et al. (2015) found that Spanish-speaking English language learners showed significantly greater improvements in math scores in the intervention condition than English speakers in the intervention.

The in-person training consists of 2 trainers and a 3-hour workshop for up to 30 early childhood practitioners. Upon completion of the training, participants will receive a certificate for 3 hours of professional development training from Oregon State University. This course qualifies as a Set 2 training with the Oregon Registry (ORO) and can be applied to ORO Core Knowledge categories: Understanding & Guiding Behavior (UGB) and Human Growth and Development (HGD). In addition, this course aligns with OPEC Parenting Educator Core Knowledge & Skills category: Human Growth & Development.

Live training of the intervention is also accessible through online platforms such as WebEx or Skype for a fee.

A self-paced training is also available online through PACE (Professional and Continuing Education), an Oregon State University website (https://pace.oregonstate.edu/catalog/red-light-purple-light-self-regulation-intervention-program). Upon completion of the self-paced course, participants will receive a certificate for 6 hours of professional development training from Oregon State University. This course qualifies as a Set 1 training with the Oregon Registry (ORO) and can be applied to ORO Core Knowledge categories: Understanding & Guiding Behavior (UGB) and Learning Environments and Curriculum (LEC). In addition, this course aligns with OPEC Parenting Educator Core Knowledge & Skills category: Human Growth & Development.

Source: Washington State Institute for Public Policy
All benefit-cost ratios are the most recent estimates published by The Washington State Institute for Public Policy for Blueprint programs implemented in Washington State. These ratios are based on a) meta-analysis estimates of effect size and b) monetized benefits and calculated costs for programs as delivered in the State of Washington. Caution is recommended in applying these estimates of the benefit-cost ratio to any other state or local area. They are provided as an illustration of the benefit-cost ratio found in one specific state. When feasible, local costs and monetized benefits should be used to calculate expected local benefit-cost ratios. The formula for this calculation can be found on the WSIPP website.

Start-Up Costs

Initial Training and Technical Assistance

The cost of the initial in-person half-day training with 2 trainers and up to 30 participants is $1,200. This cost covers trainings held within 60 miles of Oregon State University (OSU). Additional costs will be incurred for trainings with a distance greater than 60 miles from OSU or the necessity of a third trainer.

An online self-paced training is available for $200 (+ $60 registration) per participant. However, the effectiveness of this form of training has not been evaluated, and thus, Blueprints recommends the in-person training.

Curriculum and Materials

Implementation Manual: $25.00.
For orders over 20, the manual cost is reduced to $20.

Optional classroom kits:

  • Red Kit (laminated circles, sticker chart, 24 egg shakers, baton, train whistle and music CD): $40/each
  • Blue Kit (laminated circles, sticker chart, 24 egg shakers, baton, train whistle, drum and music CD): $65/each
  • Purple Kit (laminated circles; sticker chart; 24 egg shakers; baton; train whistle; Stop, Think, Act book by Dr. Megan McClelland and Dr. Shauna Tominey; and music CD): $100/each.

Licensing

No information is available

Other Start-Up Costs

No information is available

Intervention Implementation Costs

Ongoing Curriculum and Materials

No information is available

Staffing

There are no specific requirements to deliver the RLPL intervention; however, it is encouraged that implementers have a general understanding of the early childhood field and preschool classroom practices.

Other Implementation Costs

No information is available

Implementation Support and Fidelity Monitoring Costs

Ongoing Training and Technical Assistance

Technical assistance is available via phone and email at $62.00 per hour.

Fidelity Monitoring and Evaluation

Fidelity monitoring and evaluation tools are available only through prior research agreement established with the RLPL Principal Investigator.

Ongoing License Fees

No information is available

Other Implementation Support and Fidelity Monitoring Costs

No information is available

Other Cost Considerations

No information is available

Year One Cost Example

A preschool program with 10 teachers in 5 classrooms of 20 children could expect to incur the following Year 1 costs to implement the Red Light, Purple Light! intervention:

Initial Training $1,200.00
Trainer Travel at $750 per trainer $1,500.00
Implementation Manual at $25 per teacher $250.00
Purple Kit Materials at $100 per classroom $500.00
Total One Year Cost $3,450.00

For this program serving 100 children, the cost would $34.50 per student in Year 1.

Funding Overview

Financing for preschool program improvements can come from a variety of sources. Public sector preschool funding from all levels of government should be explored. Fees paid by parents for preschool could also be used to enhance the programs offered by preschools. Foundation grant opportunities and fundraising can also be considered.

Allocating State or Local General Funds

State and local education budgets that include preschool could be used to fund training in the Red Light, Purple Light! intervention. In particular, line items for teacher training and curriculum purchase should be considered.

Maximizing Federal Funds

Formula Funds: Preschool programs may be able to receive funding from the federal Department of Health and Human Services through Head Start and through the federal Department of Education through Title I or state preschool special education grants.

Discretionary Grants: Discretionary grants from the federal Department of Education should be evaluated as sources of funding for Red Light, Purple Light! Race to the Top funding is one example.

Foundation Grants and Public-Private Partnerships

Foundations, especially those interested in the impact of preschool on future school performance, may be a source of funds to implement the Red Light, Purple Light! intervention.

Generating New Revenue

Fundraising by a parent teacher organization and support from the business community should be considered for start-up costs for training and curriculum.

Program Developer/Owner

Megan McClelland, Ph.D. & Shauna Tominey, Ph.D.Oregon State UniversityHallie E. Ford Center for Healthy Children & Families125 Hallie E. Ford Center, 2631 SW Campus WayCorvallis, Oregon 97331United Statesmegan.mcclelland@oregonstate.edu health.oregonstate.edu/labs/kreadiness

Program Outcomes

  • Academic Performance
  • Cognitive Development
  • School Readiness

Program Specifics

Program Type

  • Early Childhood Education

Program Setting

  • School

Continuum of Intervention

  • Universal Prevention

Program Goals

A preschool-based program intended to improve behavioral aspects of self-regulation in children through circle time games that target attention, working memory, and inhibitory control.

Population Demographics

The program is designed for preschool children aged 3-5.

Target Population

Age

  • Early Childhood (3-4) - Preschool

Gender

  • Both

Race/Ethnicity

  • All

Race/Ethnicity Specific Findings

  • Hispanic or Latino

Race/Ethnicity/Gender Details

The program targets all groups but has been tested in predominantly low-income samples. Schmitt et al. (2015) found that Spanish-speaking English language learners showed significantly greater improvements in math scores in the intervention condition than English speakers in the intervention.

Other Risk and Protective Factors

Behavioral self-regulations skills including attention, working memory, and inhibitory control

Risk/Protective Factor Domain

  • Individual

Risk/Protective Factors

Risk Factors

Protective Factors


*Risk/Protective Factor was significantly impacted by the program

Brief Description of the Program

Red Light, Purple Light! consists of circle time games played over an 8-week period and occurring twice per week during approximately 15 to 20-minute sessions that focus on improving three components of executive function: working memory, attentional shifting, and inhibitory control. These games allow children to practice self-regulation skills in group settings.

Description of the Program

Red Light, Purple Light! consists of circle time games played during approximately 15 to 20-minute sessions that focus on improving three components of executive function: working memory, attentional shifting, and inhibitory control. These games allow children to practice self-regulation skills in group settings.

The program is administered by teachers to all chidren in their classroom, typically during large group time. The games are played 16 times, typically twice per week over eight weeks. Six different types of circle games are played over the 16 sessions, with activities being repeated and instructions becoming increasingly complex as the program progresses. The games include Red Light, Purple Light; the Freeze Game; the Sleeping Game; Conducting an Orchestra; and Drum Beats. Each session begins and end with a transition song.

Theoretical Rationale

Self-regulation predicts classroom success. Academic skills acquired in early education are cumulative, so targeting self-regulation skills in preschool can be impactful in students' success.

Theoretical Orientation

  • Skill Oriented

Brief Evaluation Methodology

In Study 1, Tominey and McClelland (2011) evaluated the program in a randomized controlled trial (N = 65) in two preschools in Oregon. They measured self-regulation and academic outcomes in a pretest at the fall of the final year of preschool before kindergarten and in a posttest at the spring of that school year.

For Study 2, Schmitt et al. (2015) evaluated the program in a randomized controlled trial (N = 276) of children enrolled in preschool through the Head Start program in the Pacific Northwest. The study randomized 14 classrooms to intervention and control conditions. Outcomes were measured at the fall of the final year of preschool before kindergarten and in a posttest at the spring of that school year. The measures included indicators of self-regulation and academics.

In Study 3, Keown et al. (2020) conducted a cluster randomized control trial with 15 early childhood education centers in Auckland, New Zealand. Children's executive functioning skills were measured pre-, post-, and 4-month post-intervention.

Outcomes (Brief, over all studies)

Study 1 (Tominey & McClelland, 2011) found that children in the intervention group showed significantly greater improvements in one of four outcomes tested (letter-word identification) relative to children in the control condition.

Study 2 (Schmitt et al., 2015) reported that children in the intervention group showed significantly greater improvements in two of six outcomes tested (behavioral self-regulation and cognitive/attentional flexibility) relative to children in the control condition.

Study 3 (Keown et al., 2020) reported that compared to control, children who participated in the intervention demonstrated stronger executive functioning skills at post-intervention. These effects, however, were no longer significant at the 4-month follow-up.

Outcomes

Study 1 (Tominey & McClelland, 2011) found that children in the intervention group, relative to the control group, showed significantly greater improvements in:

  • Letter-word identification

Study 2 (Schmitt et al., 2015) reported that children in the intervention condition, relative to children in the control condition, showed significantly greater improvements in:

  • Behavioral self-regulation
  • Cognitive and attentional flexibility

Study 3 (Keown et al., 2020) reported that at the posttest, compared to control, children who participated in the intervention demonstrated stronger:

  • Executive functioning skills

Mediating Effects

Schmitt et al. (2015) tested for a mediation effect of self-regulation on the relationship between the intervention and changes in academic outcomes and did not find a significant effect.

Effect Size

Tominey and McClelland (2011) reported a standardized coefficent (β) of 0.28 for letter-word identification. Schmitt et al. (2015) reported significant effect sizes of Cohen's d = 0.32 and 0.16 for two different tasks of self-regulation.

Generalizability

Schmitt et al. (2015) can be generalized to preschoolers from low-income families in the Pacific Northwest. While also from the Pacific Northwest, the Tominey and McClelland (2011) study can be generalized to preschoolers from both low-income and middle-upper income families. Keown, Franke and Triggs (2020) conducted their study with early childhood education (ECE) centers across the Auckland region of New Zealand.

Potential Limitations

Study 1 (Tominey & McClelland, 2011) had the following limitations:

  • Tests for baseline equivalence found no differences but used analysis sample
  • Some gaps in tests for differential attrition
  • Small and specialized sample

Study 3 (Keown et al., 2020) had the following limitations:

  • RCT but randomization likely compromised
  • Tests for baseline equivalence are incomplete
  • High attrition with evidence of differential attrition and incomplete tests of differential attrition
  • Very few effects on behavioral outcomes

Notes

Tests for differential attrition were incomplete in the published article for Study 2 (Schmitt et al., 2015) but later provided by the study authors upon request by Blueprints.

Endorsements

Blueprints: Promising

Program Information Contact

Megan McClelland, Ph.D.
Hallie E. Ford Center for Healthy Children & Families
125 Hallie E. Ford Center
263 SW Campus Way
Oregon State University
Corvallis, OR 97331
Phone: 541-737-9225
Email: megan.mcclelland@oregonstate.edu
Wesite: health.oregonstate.edu/labs/kreadiness

References

Study 1

Tominey, S. L. & McClelland, M. M. (2011). Red Light, Purple Light: Findings from a randomized trial using circle time games to improve behavioral self-regulation in preschool. Early Education and Development, 22(3), 489-519.

Study 2

Certified Schmitt, S. A., McClelland, M. M., Tominey, S. L., & Acock, A. C. (2015). Strengthening school readiness for Head Start children: Evaluation of a self-regulation intervention. Early Childhood Research Quarterly, 30, 20-31.

Study 3

Keown, L. J., Franke, N., & Triggs, C. M. (2020). An evaluation of a classroom-based intervention to improve executive functions in 4-year old children in New Zealand. Early Childhood Education Journal, 48(5), 621-631. doi:10.1007/s10643-020-01023-x

Study 1

Evaluation Methodology

Design:

Recruitment: The researchers mailed letters to the parents of all 4-year-olds in nine classrooms at the two participating preschools inviting them to participate in the study. A total of 74 children were selected to participate based on their kindergarten eligibility the following year.

Assignment: Half of the children in each classroom were randomly assigned to receive the intervention program (with the other half serving in the control condition).

Assessments/Attrition: Baseline data were collected in the fall (November and December) of the final year of preschool before kindergarten, and posttest data were collected in the spring (April and May) of that year. Of the initial recruited sample of 74 participants, data were obtained from all children at baseline, and data were obtained from 65 children at posttest, resulting in an overall attrition rate of 12%. Of the nine non-completers, four moved away, one left school for a family vacation, three declined to participate in the posttest, and one withdrew from the study due to newly diagnosed developmental delays.

Sample:

The majority of the children in the study attended preschool in a university child development center and laboratory school. Placement in the center was available either through tuition or at no cost to children enrolled in Head Start. Approximately half of the children in each classroom paid tuition, and half received care at no cost because of enrollment in Head Start. Thus, approximately half the sample was low-income as defined by Head Start status. Mothers of children enrolled in Head Start had an average education level of 12.2 years with a range of 6-16 years. Mothers of children who were not enrolled in the Head Start program had an average education level of 17 years with a range of 12-21 years. The average age of children was 4.6 years, with 39 females (60%) and 26 males (40%).

Measures:

The primary outcomes of interest were behavioral self-regulation and early academic achievement. Behavioral self-regulation was measured with the Head-Toes-Knees-Shoulders task, a validated and well-established measure for children. Twenty test items are scored as 0, 1, or 2, resulting in a total possible range of scores from 0 to 40. The measure had an interrater reliability of kappa = 0.92 for the present study sample, and demonstrated external validity based on correlations with teacher-rated behavioral self-regulation at each time point (fall r=.20, p=.09; spring r=.24, p=.06).

Academic outcomes were measured using three subtests of the Woodcock Johnson Psycho-Educational Battery-III Tests of Achievement or the Batería III Woodcock-Muñoz: the Applied Problems subtest, Letter-Word Identification subtest, and Picture Vocabulary subtest. This is a well-established standardized measure of early childhood academic competence.

Analysis:

The researchers used t-tests to examine baseline equivalence and then used multiple regression analyses to estimate intervention effects. The regression model for behavioral self-regulation predicted self-regulation change scores (spring minus fall) from group while controlling for Head Start status, age, gender, baseline outcome scores, and the interaction between group and baseline outcomes. The regression models for academic achievement (applied problems, word-letter identification, and picture vocabulary subtests) predicted difference scores on each subtest (spring minus fall) from group while controlling for age, gender, and Head Start status. No basis was provided for inclusion of these covariates aside from previous research documenting the importance of these variables for predicting the outcomes of interest. The researchers stated that the intraclass correlation for classrooms on baseline self-regulation scores was 0.06 so multilevel modeling was not used despite the nesting of children within classrooms and schools.

Intent-to-Treat: The researchers appear to have followed intent-to-treat. Only participants missing data from one or both assessment periods were dropped from the analysis.

Outcomes

Implementation Fidelity:

Children attended an average of 11.3 of the total 16 sessions in the program (range = 5-16). Most of the missed sessions were due to illness or vacation.

Baseline Equivalence:

The researchers used the analysis sample rather than the randomized sample to test for baseline equivalence on nine demographics and outcomes and did not find any significant differences, though the intervention group scored higher than the control group on all measures evaluated.

Differential Attrition:

The researchers reported that the children who left the study did not differ significantly from those who remained in the study on age or gender. A higher percentage of children who left the study were enrolled in Head Start (67%) as compared to the overall sample (43%), although this difference was not statistically significant (p > .05). The lack of significant differences between conditions in tests for baseline equivalence using the analysis sample suggested no problem with differential attrition.

Posttest:

Children in the intervention group showed significantly greater improvements in letter-word identification relative to children in the control group, controlling for age, gender, and whether they were enrolled in Head Start. No group effects were found for the other two subtests of the academic achievement test (applied problems and picture vocabulary) or for behavioral self-regulation.

Post-hoc subgroup analyses also revealed that the program significantly improved self-regulation scores for children in the bottom half of scores at baseline. The number of sessions attended also significantly predicted gains in self-regulation for children in the bottom half of scores at baseline.

Long-Term:

Not examined.

Study 2

Evaluation Methodology

Design:

Recruitment: Children and families were recruited through letters in an enrollment packet about the study sent during the summer prior to the school year. Children were enrolled in one of 14 Head Start classrooms across nine sites in a Pacific Northwest city. The recruited sample included 276 children. Signed consent was obtained from 24 teachers and from a parent of all children in the study.

Assignment: Classrooms (N = 14) were randomly assigned to receive the intervention program or no program. A total of 126 children were randomized to the treatment condition and 150 children to the control group, with seven classes in each condition.

Assessments/Attrition: Participants completed assessments in the fall and spring of their final year of preschool. From baseline to posttest, the sample decreased 275 to 241 (12.7% attrition). Missing data on individual tasks at pretest and posttest, either due to attrition or other extraneous circumstances (e.g., child was sick during testing session), ranged from 6% to 17%.

Sample:

The sample included preschoolers enrolled in the Head Start program from nine schools in a city in the Pacific Northwest.

Measures:

The primary outcomes included self-regulation and academic performance. Self-regulation was measured using a teacher-report survey and two child behavioral tasks. Teachers rated self-regulation using the Child Behavior Rating Scale, which had an alpha of 0.96 in the present sample. The child behavioral tasks included the Head-Toes-Knees-Shoulders task, a well-established and validated measure, and the Dimensional Change Card Sort task, a well-established measure of cognitive and attentional flexibility, which had an alpha of 0.84 in the fall and 0.77 in the spring in the present sample.

Academic outcomes were measured with the Applied Problems subtest, Letter-Word Identification subtest, and Picture Vocabulary subtest of the Woodcock Johnson Psycho-Educational Battery-III Tests of Achievement or the Batería III Woodcock-Muñoz.

Analysis:

The researchers used multilevel regression models clustered by classroom to analyze differences in self-regulation between groups, controlling for baseline outcome scores, age, and English language learner status. These covariates were included based on previous research suggesting their relation to self-regulation and academic achievement. Data were assumed to be missing at random because analyses showed that missing data was not related to ethnicity, parent education marital status, maternal age, parent employment, or residential mobility. Full information maximum likelihood was used to estimate models. Additionally, path models were used to test for mediation of self-regulation on the relationship between the intervention and academic scores.

Intent-to-Treat: The researchers stated they used intent-to-treat. Missing data were handled using full information maximum likelihood estimation under a missing at random assumption.

Outcomes

Implementation Fidelity:

Children in the intervention condition attended an average of 13 out of the 16 sessions, and 66% of the participants attended 80% of the sessions or more. Fidelity of the session content was also monitored and addressed in regular discussions.

Baseline Equivalence:

At baseline, the randomized conditions did not differ significantly by age, gender, parent education, parent marital status, maternal age, English language learner status, or outcome measures.

Differential Attrition:

The control group had a 19% attrition rate and the intervention group had a 5% attrition rate though the authors did not report whether this difference was significant. Missing data was not significantly related to ethnicity, parent education, marital status, maternal age, parent employment status, or residential mobility.

Posttest:

Children in the intervention condition demonstrated stronger improvements than children in the control condition on two of six outcomes tested. Specifically, children in the intervention had stronger self-regulation at posttest on two of three self-regulation measures (the two behavioral tasks, but not the teacher-report survey) relative to children in the control condition. Cohen's d = 0.32 and 0.16 for the two different self-regulation tasks. No group differences were observed for changes in the three subtests of academic achievement.

Subgroup analyses also revealed that English language learners showed significantly greater improvements in math scores in the intervention condition than English speakers in the intervention.

Long-Term:

Not examined.

Study 3

Evaluation Methodology

Design:

Recruitment: Twenty ECE centers were invited to take part in the study. These centers provided full-day early-learning and childcare and were in a range of socioeconomic areas (low SES = 7; mid SES = 8) across the Auckland (New Zealand) region. Centers belonged to a large national early care and education provider, who expressed interest in trialing the program. The study was confined to 4-year-old children for two reasons: (1) In New Zealand children typically start school at age 5-years, (2) the possibility of floor effects on outcome measures used in the study in children under 4 years of age. Signed consent was obtained from 61 teachers; however 15 teachers were included in the study.

Assignment: Sixteen early childhood education centers were randomized to either treatment (8 weeks of twice weekly 20- to 30-min sessions of the intervention; n = 8 centers) or a waitlist control group (business-as-usual, i.e., the same daily activities, routines, and curricula that preceded study participation; n = 8 centers). The intervention took place in the children's regular classroom (on the same two days each week) and the curriculum was taught by ECE qualified teachers. The article states that paired randomization was used to balance center size across the intervention and control groups, which involved pairing centers based on their similarity in the number of consented children. However, 16 centers were assigned in pairs and one of the centers (from the control group) did not participate, but the paired cluster from the treatment group was not dropped. This seems to be a violation of the randomization procedure.

Assessments/Attrition: One center dropped out after randomization, for an attrition rate of 6% at the cluster level. At the child-level, the authors did not report how many children were in the center that dropped out. Based on the 15 centers and 15 teachers included in the analysis, the sample included 212 children (107 assigned to treatment and 105 assigned to control). However, as noted above, this number did not include the count of children from the center that dropped out. At the posttest, 187 children of the initial reported sample of 212 children remained in the study (for an overall attrition rate of 12%). At the follow-up (4 months post-intervention), 124 children remained (for an attrition rate of 42%). In addition, the authors reported missing data for individual outcome measures that resulted from either attrition or other circumstances (e.g., absences on the day of assessments or child refusal to be assessed).

Sample: The children ranged in age from 47 to 58 months (M = 52.52 months, SD = 2.93), at the date of the pre-intervention assessments. Fifty two percent of the children were female. Thirty-one different ethnic groups were represented within the sample. Most parents nominated one ethnic group (n = 154, 73%) for their child, with Asian descent the most frequent, followed by New Zealand European and Pacific Island origins. Fifty-seven parents (27%) reported their child had two or more ethnicities. The majority of parents reported one main language spoken at home (n = 163, 77.1%), with English the most common (n = 114, 69.9%), followed by Indian (n = 19, 11.7%), and Chinese (n = 13, 8%). Forty-eight parents (22.9%) nominated two or more languages spoken at home.

Most of the teachers were female, with an average of 5.9 years of ECE teaching experience.

Measures:

Executive Functioning:

  • The HTKS task (Cameron Ponitz et al. 2009) was used to assess children's integration of working memory, attentional flexibility, and inhibitory control. In the present study, 60 HTKS assessments were jointly observed and scored by pairs of raters, with an interrater agreement of Kw = 0.96.
  • The DCCS task from the National Institute of Health (NIH) Toolbox Cognitive Function Battery (Zelazo et al. 2013), which was administered via an app on an iPad, was used to assess cognitive flexibility and attention. No reliability or validity was reported.

Behavioral Self-regulation

  • Teacher ratings of self-regulation were assessed with the 10-item subscale of the Child Behavior Rating Scale (CBRS). In the current sample, internal consistency for this subscale ranged between α = .95 at pre- and post-test and α = .96 at the follow-up. This measure was not independent (as teachers administered the intervention).

Analysis: Hierarchical linear mixed models were used to estimate the effects of the intervention. Separate models were fitted for each outcome and at each time point. Covariates included baseline outcome scores, baseline receptive English vocabulary (BPVS) scores, children's age, and gender.

Intent-to-Treat: Authors stated they conducted an intent-to-treat analysis; however, one center in the control group did not participate and data were not collected from this site and the paired treatment center that remained in the study was not dropped - which could be a violation of intent-to-treat protocol. Missing data were imputed separately for each output measure at each time point. Thus, for example, for HTKS at the second time point, separate imputations were carried out for the intervention and waitlist control groups and the two sets of imputed complete data were recombined before analysis. Imputation respecting the hierarchical structure of the data was carried out using the R package pan.

Outcomes

Implementation Fidelity: On average, children were enrolled at their center for 32.23 h (SD = 11.78) per week and had spent 100.55 weeks (SD = 57.31) at their current center. Five RLPL sessions, spaced over 8 weeks, were observed by the researchers at each intervention center to assess quality and fidelity of program implementation. Across the eight intervention-group centers the average attendance rate was 12 RLPL sessions (SD = 4.12; range 1 to 16) and 72 children (70%) attended 12 or more intervention sessions.

Baseline Equivalence: Since the authors did not report the cluster of children in the center that withdrew after randomization, baseline equivalence for the randomized sample is unknown. Table 2 (p. 627) reports baseline equivalence for the 212 students within the 15 centers (i.e., the analysis sample). No significant differences were detected between conditions in baseline outcome scores or in baseline receptive English vocabulary (BPVS) scores. Socio-demographic characteristics were not assessed for baseline equivalence.

Differential Attrition: Logistic regressions were run to determine whether demographic variables (i.e., child age, child gender, weekly enrolment hours at ECE, parent employment status) were related to missingness. Child age predicted missingness on the DCCS at T2 and DCCS, HTKS and CBRS at T3. Data on these measures were more likely to be missing for older children in the sample who had started school and could not be contacted for data collection. However, baseline measures were not included in the logistic regression analyses. In addition, attrition by condition was not assessed.

Posttest: Compared to control, children who participated in the intervention demonstrated significantly stronger executive functioning skills at post-intervention, as measured by the DCCS task. These effects, however, were no longer significant at the 4-month follow-up. There were no posttest or follow-up effects for executive functioning (as measured by the HTKS) or teacher-rated self-regulation (as measured by the CBRS).

Long-Term: Not reported.

Contact

Blueprints for Healthy Youth Development
University of Colorado Boulder
Institute of Behavioral Science
UCB 483, Boulder, CO 80309

Email: blueprints@colorado.edu

Sign up for Newsletter

If you are interested in staying connected with the work conducted by Blueprints, please share your email to receive quarterly updates.

Blueprints for Healthy Youth Development is
currently funded by Arnold Ventures (formerly the Laura and John Arnold Foundation) and historically has received funding from the Annie E. Casey Foundation and the Office of Juvenile Justice and Delinquency Prevention.