Sign Up For Newsletter

Blueprints For Healthy Youth Development logo

Early Literacy and Learning Model

A literacy-focused curriculum and support system designed for preschool children ages 3, 4, and 5 years old. The program is designed to enhance existing classroom curricula by specifically focusing on improving children's early literacy skills and knowledge.

Program Outcomes

  • Cognitive Development
  • Preschool Communication/ Language Development

Program Type

  • Academic Services
  • School - Individual Strategies
  • Teacher Training

Program Setting

  • Home
  • School

Continuum of Intervention

  • Selective Prevention
  • Universal Prevention

Age

  • Early Childhood (3-4) - Preschool

Gender

  • Both

Race/Ethnicity

  • All

Endorsements

Blueprints: Promising

Program Information Contact

Dr. Madelaine Cosgrove (mcosgrov@unf.edu)
or Howaida Mousa (h.mousa@unf.edu)
Florida Institute of Education at the University of North Florida
Adam W. Herbert University Center
12000 Alumni Drive
Jacksonville, FL 32224
(904) 620-2496

Curriculum information and materials available at: https://www.unf.edu/fie/ellm/plus/

 

Program Developer/Owner

Dr. Madelaine Cosgrove
Florida Institute of Education at the University of North Florida


Brief Description of the Program

The Early Literacy and Learning Model (ELLM) is a literacy-focused curriculum and support system targeting young children from low-income families. The program is designed to enhance existing classroom curricula by specifically focusing on children's early literacy skills and knowledge. The program is designed to be implemented year-round or during the academic year and supplements the daily activities of the classroom. Teacher support and family involvement opportunities also occur regularly throughout the year.

The ELLM program components include the following:

  • curriculum and literacy building blocks;
  • assessment for instructional improvement;
  • professional development for literacy coaches and teachers;
  • family involvement; and
  • collaborative partnerships.

Outcomes

Primary Evidence Base for Certification

Study 2

Cosgrove et al. (2006), found that in comparison to controls, students in the intervention group showed:

  • Greater recognition of letters
  • Better emerging literacy skills

Brief Evaluation Methodology

Primary Evidence Base for Certification

Of the two studies Blueprints has reviewed, one study (Study 2) meets Blueprints evidentiary standards (specificity, evaluation quality, impact, dissemination readiness). This study was done by the developer.

Study 2

Cosgrove et al. (2006) conducted a cluster-randomized trial with 48 classrooms and 466 preschool children, in which classrooms (one per school) were assigned to either the intervention or a waitlist control condition. Measures of alphabet recognition and early literacy skills were collected at baseline and posttest.

Study 2

Cosgrove, M., Fountain, C., Wehry, S., Wood, J., & Kasten, K. (2006, April). Randomized field trial of an early literacy curriculum and instructional support system. Paper presented at the annual meeting of the American Educational Research Association, San Francisco, CA.


Risk Factors

School: Poor academic performance

Protective Factors

Family: Parental involvement in education

School: Instructional Practice


* Risk/Protective Factor was significantly impacted by the program

See also: Early Literacy and Learning Model Logic Model (PDF)

Subgroup Analysis Details

Sample demographics including race, ethnicity, and gender for Blueprints-certified studies:

  • The sample for Study 2 (Cosgrove et al., 2006) was 71% Black, 12% White, 8% Hispanic, and 9% of other racial background. The gender of participants was not reported.

ELLM/Plus Introduction Training (2-day): $375 per participant (with a minimum of ten participants) for training at program site plus meeting costs (food, A/V) and travel costs for trainers. Training provides not only an overview of the curriculum, but also opportunities to engage in hands-on experiences with curriculum implementation through modeling and practice activities, and using supplemental resources to enhance daily instruction. Content includes:

  • Using the ELLM/Plus Monthly Instructional Packets to organize and implement literacy and language instruction each day
  • Setting up classrooms that promote learning and positive interactions
  • Planning and scheduling daily learning activities
  • Helping children build their background knowledge and vocabulary
  • Using informational assessment to scaffold learning
  • Promoting children's language and literacy development through adult-child and child-child interactions
  • Connecting home and school learning
  • Planning the ELLM/Plus day
  • Using the Teaching Table for small group instruction
  • Integrating the curriculum across the content areas
  • Building children's background knowledge through one-on-one conversations and small group discussions

Strategic Coach Institute (5-day): $2,650 per participant plus travel to training and administrator or coach time. Florida Institute of Education recommends that a program administrator or a coach participate in this training that equips them to provide ongoing fidelity monitoring and training to staff.

Customized training is also available. Customized training is designed to address needs identified by administrators and/or participants overseeing implementation. Lengths of training and follow-up activities are decided on a case-by-case basis to meet implementation needs of participants.

Source: Washington State Institute for Public Policy
All benefit-cost ratios are the most recent estimates published by The Washington State Institute for Public Policy for Blueprint programs implemented in Washington State. These ratios are based on a) meta-analysis estimates of effect size and b) monetized benefits and calculated costs for programs as delivered in the State of Washington. Caution is recommended in applying these estimates of the benefit-cost ratio to any other state or local area. They are provided as an illustration of the benefit-cost ratio found in one specific state. When feasible, local costs and monetized benefits should be used to calculate expected local benefit-cost ratios. The formula for this calculation can be found on the WSIPP website.

Start-Up Costs

Initial Training and Technical Assistance

  • ELLM/Plus Introduction Training (2-day): $375 per participant (with a minimum of ten participants) for training at program site plus meeting costs (food/AV) and travel costs for trainers. There is a one-day training option at $209 per participant, but teachers received two days in the research trials; thus the two-day training model is preferred by Blueprints.
  • Strategic Coach Institute (5-day): $2,650 per participant plus travel to training and administrator or coach time. Florida Institute of Education (FIE) recommends that a program administrator or a coach participate in this training that equips them to provide ongoing fidelity monitoring and training to staff.
  • Additional support for coaches and teachers (training and onsite support) is available, billed at $100 per hour plus travel costs.
  • Pre-implementation Technical Assistance (optional): $600 for consultant cost plus administrator time for six hours needed to discuss material purchase, training, special needs, and implementation fidelity.
  • Technical Assistance available to help set up classrooms to create learning rich environment: $300 for consultant fees (based on estimate of 3 hours to provide guidance on classroom set up) plus travel fees.

Curriculum and Materials

  • ELLM/Plus 10-Month Kit (curriculum materials and children's books/per classroom): $1,532
  • ELLM/Plus 12-Month Kit (curriculum materials and children's books/per classroom): $1,684
  • Classroom lending library: $600 per classroom
  • Manipulatives: $200 per classroom

Licensing

None.

Other Start-Up Costs

Each classroom needs sufficient space to set up a minimum of 5 functioning learning centers, a small group teaching table, word walls, and space to display children's work.

Intervention Implementation Costs

Ongoing Curriculum and Materials

Ongoing material costs associated with ELLM/Plus implementation varies depending on replacement needs. Estimate yearly costs for replacement of books for the classroom lending library at $200 per class and replacement of classroom manipulatives at $100 per class. Recommend replacing the curriculum at least every five years as curriculum is updated every 2 to 3 years.

Staffing

Qualifications: No specific requirements - intervention typically delivered by early care and education teachers.

Ratios: No specific requirements - intervention implemented in early care settings governed by ratio requirements in state licensing standards.

Time to Deliver Intervention: The ELLM/Plus model is a comprehensive curriculum that can be integrated into half or full day early learning schedules.

Other Implementation Costs

None.

Implementation Support and Fidelity Monitoring Costs

Ongoing Training and Technical Assistance

Periodic technical assistance through email and telephone is available at no cost. The cost for on-site technical assistance designed to answer implementation questions, address site-based needs, assist in monitoring progress, and provide additional support varies depending on other needs of the site, and is billed at a rate of $100 per hour plus travel costs.

Fidelity Monitoring and Evaluation

Fidelity monitoring tools and training on how to use them are included in the start-up training. Trained coaches or administrators are responsible for fidelity monitoring on an ongoing basis, including walk-throughs and completion of checklists.

Ongoing License Fees

None.

Other Implementation Support and Fidelity Monitoring Costs

No information is available

Other Cost Considerations

The high level of turnover in the early care and education field means that new staff may need to complete start-up training on an ongoing basis. Building local capacity for ongoing training and fidelity monitoring is an efficient and cost effective strategy to sustain ongoing implementation with fidelity. FIE researchers have worked with local sites to train site coordinators and/or a cadre of coaches who then take on the responsibility for implementation support and fidelity monitoring. This strategy is particularly relevant for sites that have local early care and education quality specialists or trainers who work with local programs.

Year One Cost Example

This example assumes that an early care and education program would offer the Early Literacy and Learning Model Plus in 10 classrooms operating 10 months per year with two teachers per classroom. Costs included are the start-up costs for materials and training recommended by the publishers. This model does not include ongoing costs of teachers and facilities.

Teacher Initial Training Costs: $375 x 22 (20 teachers, 2 admin) $8,250.00
Trainer Travel: $1,000 x 2 $2,000.00
Strategic Coach Institute Training $2,650.00
Coach Travel $1,000.00
ELLM/Plus 10-Month Kit (curriculum, materials): $1,532 x 10 classrooms $15,320.00
Lending Library: $600 x 10 classrooms $6,000.00
Manipulatives: $200 x 10 classrooms $2,000.00
Total One Year Cost $37,220.00

With 200 students in 10 classrooms, the additional cost to the program (beyond their existing staffing and operations cost) is approximately $186/child.

Funding Overview

The Early Learning and Literacy Model Plus is generally integrated into the ongoing operation of early learning and education programs, so the implementation costs are supported by the funding sources that support the ongoing operations of early care programs. These include parent fees and government child-care and preschool funding. The start-up costs can potentially be covered through portions of government child-care and preschool funding that are designated to support quality improvement or administration. Public-private partnerships in which foundation partners help to support initial training and curriculum costs are another promising strategy for supporting ELLM/Plus.

Allocating State or Local General Funds

Many states have created state funded preschool programs that fund school and community-based pre-kindergarten programs. States invest varying levels of resources in supporting the quality of these efforts, but there is potential for states to allocate funds or invest some portion of their existing quality or administrative funds in ELLM/Plus training, curriculum, and coaching.

Maximizing Federal Funds

Formula Funds: The core formula funds supporting early care and education can potentially support ELLM/Plus.

  • Child Care and Development Fund (CCDF) can potentially support curriculum purchase, training, and general operating costs in schools. There are two primary mechanisms for CCDF to support ELLM/Plus: 1) Early care and education programs can access vouchers or act as contractors to serve families who qualify for child care assistance on the basis of income. These dollars are used for general operating support in programs and could support the ongoing implementation of ELLM/Plus. 2) CCDF has a 4% quality set-aside that states must use to improve the quality and availability of early care and education. States or localities that want to implement ELLM/Plus could use child care quality dollars to pay for curriculum and materials, training teachers, administrators and coaches, and ongoing fidelity monitoring.
  • Temporary Assistance for Needy Families (TANF) has historically been used by many states to support early care and education programs - either directly or through transfer of funds to the Child Care and Development Fund. Once transferred, TANF funds can be used in the same manner as CCDF to support ELLM/Plus.
  • Title I, Part A is used by many districts to support preschool programs for disadvantaged children. These funds can support the curriculum, training, fidelity monitoring, and ongoing implementation.

Discretionary Grants: Federal discretionary grants supporting early care and education and early literacy programs can potentially support ELLM/Plus. Most relevant federal discretionary grants are administered by The Child Care Bureau or The Head Start Bureau within the Department of Health and Human Services, or the Department of Education.

  • Head Start Funds offer an important source of support for implementation of ELLM/Plus within Head Start programs. In addition, Head Start grantees can potentially partner with school-based preschool programs and child care centers to support training and fidelity monitoring of coaches, a capacity that could be shared among multiple settings in a locality.
  • Race to the Top: Early Learning Challenge offers competitive grants to state education and social service agencies to improve the quality and availability of early learning programs. States could choose to invest in ELLM as part of a quality improvement strategy.

Foundation Grants and Public-Private Partnerships

Foundation grants and public-private partnerships are an important financing strategy for ELLM/Plus. Early Care and Education Programs that find it difficult to identify dollars to support the initial training and curriculum purchases can potentially partner with a local foundation or corporation to support these costs.

Generating New Revenue

A final category of strategies to consider are mechanisms to generate new revenue and to set-aside funding for specific populations or sets of services. Several states and localities generate funding for early care and education services through the imposition of fees on products that can result in social costs. The most popular of these "sin taxes" generally target tobacco and alcohol use. New tax and fee strategies are generally difficult to put in place, and the mechanism for establishing them depends on the state. In some states new revenue mechanisms can be put in place through ballot initiatives, while in others they require state legislation.

Data Sources

Survey and interview with purveyor, Florida Institute of Education.

Program Developer/Owner

Dr. Madelaine CosgroveFlorida Institute of Education at the University of North FloridaAdam W. Herbert University Center12000 Alumni DriveJacksonville, FL 32224USA(904) 620-2496mcosgrov@unf.eduh.mousa@unf.edu

Program Outcomes

  • Cognitive Development
  • Preschool Communication/ Language Development

Program Specifics

Program Type

  • Academic Services
  • School - Individual Strategies
  • Teacher Training

Program Setting

  • Home
  • School

Continuum of Intervention

  • Selective Prevention
  • Universal Prevention

Program Goals

A literacy-focused curriculum and support system designed for preschool children ages 3, 4, and 5 years old. The program is designed to enhance existing classroom curricula by specifically focusing on improving children's early literacy skills and knowledge.

Population Demographics

The program targets preschool aged children (age 3 to 5 years,) living in low-income neighborhoods. The program has been tested with samples that are primarily African American with smaller percentages of White and Hispanic children living in urban and rural areas. There was an equal mix of male and female children who were an average of 4-5 years old.

Target Population

Age

  • Early Childhood (3-4) - Preschool

Gender

  • Both

Race/Ethnicity

  • All

Subgroup Analysis Details

Sample demographics including race, ethnicity, and gender for Blueprints-certified studies:

  • The sample for Study 2 (Cosgrove et al., 2006) was 71% Black, 12% White, 8% Hispanic, and 9% of other racial background. The gender of participants was not reported.

Other Risk and Protective Factors

The program targets children who reside in low-income neighborhoods and are likely to come to school underprepared for literacy success. It provides an enriched learning environment and supports family involvement in learning. While the program is delivered classroom-wide, additional services are targeted to children assessed with low levels of early literacy skills.

Risk/Protective Factor Domain

  • School
  • Family

Risk/Protective Factors

Risk Factors

School: Poor academic performance

Protective Factors

Family: Parental involvement in education

School: Instructional Practice


*Risk/Protective Factor was significantly impacted by the program

See also: Early Literacy and Learning Model Logic Model (PDF)

Brief Description of the Program

The Early Literacy and Learning Model (ELLM) is a literacy-focused curriculum and support system targeting young children from low-income families. The program is designed to enhance existing classroom curricula by specifically focusing on children's early literacy skills and knowledge. The program is designed to be implemented year-round or during the academic year and supplements the daily activities of the classroom. Teacher support and family involvement opportunities also occur regularly throughout the year.

The ELLM program components include the following:

  • curriculum and literacy building blocks;
  • assessment for instructional improvement;
  • professional development for literacy coaches and teachers;
  • family involvement; and
  • collaborative partnerships.

Description of the Program

The Early Literacy and Learning Model (ELLM) is a literacy-focused curriculum and support system targeting preschool children ages 3, 4, and 5 years old, particularly those from low-income neighborhoods. The program is designed to enhance existing classroom curricula by specifically focusing on improving children's early literacy skills and knowledge. The program is designed to be implemented year-round or during the academic year and supplements the daily activities of the classroom. Teacher support and family involvement opportunities also occur regularly throughout the year.

The ELLM program components include the following:

  • curriculum and literacy building blocks;
  • assessment for instructional improvement;
  • professional development for literacy coaches and teachers;
  • family involvement; and
  • collaborative partnerships.

The ELLM curriculum materials include a set of literacy performance standards; monthly literacy packets; targeted instructional strategies; resource guides for teachers; a classroom book-lending library; family and teacher tip sheets; and literacy calendars.

One hour of daily literacy instruction is provided to implement the ELLM literacy building blocks. Trained literacy coaches provide instructional support to preschool teachers who use the curriculum.

The ELLM program also contains a family involvement action plan. Families have access to many resources, including a classroom book-lending library that enables children to take books home daily to share with their parents. Parents receive monthly family tip sheets and calendars with suggestions for literacy activities they can engage in with their children. Parents also have the opportunity to engage in preschool site-based family activities during the school year.

Theoretical Rationale

If preschool children (especially those at-risk) regularly engage in intensive, explicit, and intentional early literacy instruction; embedded in quality programs using standards- and research-based curricula delivered with fidelity by practitioners who are provided ongoing job-embedded support; then, participating children will acquire the knowledge and skills needed to become successful readers.

Theoretical Orientation

  • Skill Oriented

Brief Evaluation Methodology

Primary Evidence Base for Certification

Of the two studies Blueprints has reviewed, one study (Study 2) meets Blueprints evidentiary standards (specificity, evaluation quality, impact, dissemination readiness). This study was done by the developer.

Study 2

Cosgrove et al. (2006) conducted a cluster-randomized trial with 48 classrooms and 466 preschool children, in which classrooms (one per school) were assigned to either the intervention or a waitlist control condition. Measures of alphabet recognition and early literacy skills were collected at baseline and posttest.

Outcomes (Brief, over all studies)

Primary Evidence Base for Certification

Study 2

Cosgrove et al. (2006) found that the intervention students scored significantly higher on all measures - three early literacy scales and one alphabet recognition scale - than the students in the control group.

Outcomes

Primary Evidence Base for Certification

Study 2

Cosgrove et al. (2006), found that in comparison to controls, students in the intervention group showed:

  • Greater recognition of letters
  • Better emerging literacy skills

Effect Size

In Study 2 (Cosgrove et al., 2006), at the end of the intervention year, with the exception of the differences in the TERA-3 Conventions of Print scale adjusted mean posttest scores, resulting effect sizes were 25 percent of a standard deviation or larger.

  • ARLI ES = .25
  • TERA-3 Reading ES = .28
  • TERA-3 Alphabet = .28
  • TERA-3 Conventions of Print = .17
  • TERA-3 Meaning ES = .29

Generalizability

One study meets Blueprints standards for high quality methods with strong evidence of program impact (i.e., "certified" by Blueprints): Study 2 (Cosgrove et al., 2006). The study took place within three lower-income areas in Florida, in which the treatment was compared with a waitlist control condition.

Potential Limitations

Additional Studies (not certified by Blueprints)

Study 1 PCERC (2008)

  • Children in two sites that dropped out of the study were not followed, and no information was provided on efforts to follow-up children who left prematurely.
  • No detail on differential attrition was presented.
  • All but a few effects were insignificant. The analysis in the national study (PCERC, 2008) was not conducted at the level of randomization.

Preschool Curriculum Evaluation Research Consortium (PCERC) (2008). Effects of Preschool Curriculum Programs on School Readiness (NCER 2008-2009) (pp. 99-108, C-17, C-18, D-17, D-18). National Center for Education Research, Institute of Education Sciences, U.S. Department of Education. Washington, D.C.: Government Printing Office.

Notes

The PCERC (2008) was not as detailed about the methodology used as Cosgrove et al. (2006), so some information on the randomization process as described in the PCERC (2008) study actually came from Cosgrove et al. (2006).

Endorsements

Blueprints: Promising

Program Information Contact

Dr. Madelaine Cosgrove (mcosgrov@unf.edu)
or Howaida Mousa (h.mousa@unf.edu)
Florida Institute of Education at the University of North Florida
Adam W. Herbert University Center
12000 Alumni Drive
Jacksonville, FL 32224
(904) 620-2496

Curriculum information and materials available at: https://www.unf.edu/fie/ellm/plus/

 

References

Study 1

Preschool Curriculum Evaluation Research Consortium (PCERC) (2008). Effects of Preschool Curriculum Programs on School Readiness (NCER 2008-2009) (pp. 99-108, C-17, C-18, D-17, D-18). National Center for Education Research, Institute of Education Sciences, U.S. Department of Education. Washington, D.C.: Government Printing Office.

Study 2

Certified Cosgrove, M., Fountain, C., Wehry, S., Wood, J., & Kasten, K. (2006, April). Randomized field trial of an early literacy curriculum and instructional support system. Paper presented at the annual meeting of the American Educational Research Association, San Francisco, CA.

Study 1

Summary

PCERC (2008) examined 28 preschool classrooms and 244 children. Equal numbers of classrooms were assigned to the treatment and waitlist control conditions; however, as part of a pilot study (see Study 2), the treatment classrooms were randomized a year earlier than the control classrooms. Data were collected at baseline, posttest (end of preschool), and follow-up (end of kindergarten), and included measures of literacy and math skills, behavior, and teacher-child interaction.

PCERC (2008) found:

  • A delayed effect on vocabulary that showed up at the end of kindergarten (1 year post-intervention)

Evaluation Methodology

Design

The Early Literacy and Learning Model (ELLM) was evaluated in a randomized controlled trial the included classrooms from 28 preschools located in three geographic areas of Florida. The sampled preschools included full-day Head Start, subsidized, faith based, and public school-based early intervention programs. Randomization was done during an earlier national pilot study (2002-2003). However, two preschools dropped out during the pilot year and were replaced for the national study year (2003-2004). A total of 30 classrooms and teachers were randomized, but two dropped out leaving 28 classrooms and teachers for the analysis. The intervention teachers were randomized in 2002 and participated in the pilot study (Study 2) as well as the current study discussed here, while a new pool of classrooms was recruited in 2003 and teachers were randomized to the control condition from this pool.

A total of 297 children and 294 parents were recruited for participation in the national study. The final sample included 244 children (137 treatment, 107 control) and 243 parents. Data were collected on 243 children and 204 parents at the time of the baseline data collection. In the follow-up year of the national evaluation, the sample of schools went from 28 preschools to 119 schools with kindergarten classrooms. The sample of classrooms went from 28 preschool classrooms to 175 kindergarten classrooms. The kindergarten sample included 237 children and 236 parents from original sample of participants. Follow up data were collected on 218 children and 177 parents.

Appendix B (page B-9) reports on completion rates at pretest, posttest (end of preschool), and follow-up (end of kindergarten). For children, the rates were 100%, 92%, and 89%. It is not clear to what extent the national research team attempted to collect post-test and/or follow up data from children who left the study prematurely.

Regarding the counterfactual condition, a number of curricula were used in the control classrooms to include Creative Curriculum (Dodge, Colker and Heroman 2002), & Beyond Centers and Circletime (Phelps 2002), High Reach Learning Pre-K (High Reach Learning 1997) and High/Scope (Hohmann and Weikart 2002).

Sample

The research team recruited preschool programs from three distinct geographic locations within the state. The research team first identified elementary school neighborhoods in each geographic location (Counties A, B, and C) with low-performing schools. Using the Florida Department of Education's school grading report card system the research team identified grade D and F elementary schools in each of the three counties. It was expected that children from the preschool programs in these low-performing elementary school neighborhoods would transition into these grade D and F elementary schools during the kindergarten year of the study. As such, preschool programs within the low performing elementary school neighborhoods were randomly selected for inclusion in the sampling pool of preschool programs.

The average age of children was 4.6 years at the time of the baseline data collection and half (50%) was male. The overall sample was primarily African American (71%) with smaller percentages of White (14%) and Hispanic (8%) children. Approximately 13% of the children were reported by their parents as having a disability.

The average age of the primary caregiver (usually the mother) was 31 years. Almost half (40%) of the primary caregivers were never married; 37% were married at the time of the fall assessment data collection. More than one-third of the primary caregivers reported having had some college (36%) or had graduated from college (6%); 37% had a high school diploma or GED; and 22% had not finished high school. More than half (54%) of the primary caregivers were employed full-time; 11% were employed part-time; and 33% were unemployed. At baseline, a higher percentage of parents in the treatment group had completed some post-high school education relative to those assigned to the control group (41% vs. 29%, p < .01).

There were 28 teachers who participated in the national preschool year intervention study. All were female. The majority identified themselves as African American (64%) or White (21%). The preschool teachers had on average 11 years of teaching experience, with an average of seven years teaching preschool. Fifty percent of the teachers had a high school diploma or GED and 21% had a bachelor's degree. Many of the teachers reported having a state-awarded preschool certificate (52%); a teaching license or certificate (46%); or a Child Development Associate (CDA) credential (46%). Eighteen percent reported having no teacher certification credentials. At baseline, teachers in the treatment group had more years of experience teaching in a preschool setting relative to those assigned to the control group (9 years vs. 4 years, p < .01).

Measures

Data were collected by the national research team from parents, teachers, and schools and through direct assessments of the children. No reliability or validity information was provided on the 23 measures used; however, most are commonly used measures in educational intervention research.

Mathematics

  • Woodcock Johnson (WJ) Applied Problems
  • Child Math Assessment-Abbreviated (CMA-A) Composite Score
  • Shape Composition

Reading

  • Test of Early Reading Ability (TERA-3)
  • Woodcock Johnson (WJ) Letter Word Identification
  • Woodcock Johnson (WJ) Spelling

Phonological Awareness

  • Preschool Comprehensive Test of Phonological and Print Processing (Pre-CTOPPP; Elision subtest) (preschool only) & Comprehensive Test of Phonological Processing (CTOPP; Elison subset) (kindergarten only)

Language

  • Peabody Picture Vocabulary Test (PPVT)
  • Test of Language (TOLD) Grammatical Understanding subtest

Behavior

  • Social Skills Rating System (SSRS) Social Skills scale
  • SSRS Problem Behaviors scale
  • Preschool Learning Behaviors Scale (PLBS) (preschool only) & Learning Behaviors Scale (LBS) (kindergarten only)

Overall classroom environment (baseline & posttest only)

  • Early Childhood Environment Rating Scale-Revised (ECERS-R)

Teacher-child interactions (baseline & posttest only)

  • Arnett Detachment scale
  • Arnett Harshness scale
  • Arnett Permissiveness scale
  • Arnett Positive Interactions scale

Classroom Instruction (posttest only)

  • Early literacy (Teacher Book Reading Scale (TBRS) Print and Letter Knowledge)
  • Early literacy (TBRS Written Expression scale)
  • Phonological awareness (TBRS Phonological Awareness scale)
  • Language (TBRS Book Reading)
  • Language (TBRS Oral Language scales)
  • Early mathematics (TBRS Math Concepts scale)

Analysis

Analysis was conducted at the individual child-level (for child specific measures) and the classroom-level (for teacher specific measures). Randomization was done at the preschool classroom level. Appendix B (page B-10) describes the use of hierarchical linear models to account for nesting of children within classrooms. Appendix B (page B-8) also states that the analyses used an intent-to-treat logic by including all randomized subjects, even if they were lost to follow-up.

The unadjusted mean scores, adjusted mean difference scores with standard errors, and effect sizes were reported. Effect sizes were calculated using repeat measures (RM) analysis or analysis of covariance (ANCOVA) depending on the data available at each time point.

Adjustments of child-level measures included the following covariates: (a) child's age, (b) gender, (c) race/ethnicity, (d) disability status (as reported by parent), and (e) mother's education. For all analysis of classroom-level measures adjustments included the following covariates: (a) teacher has a BA degree, (b) previous teaching experience, (c) child/adult ratio in classroom, (d) average class size, (e) city size, and (f) geographic site.

RM analysis was used to detect effects on the phonological awareness and social behavioral measures at baseline and post-test. RM analysis was also used to detect effects on the three mathematics outcomes, the three reading outcomes, the overall classroom environment and teacher-child interactions at all three data collection time points.

ANCOVAs were used to assess the long-term effects on phonological awareness and the three social behavioral measures; the covariates, for which, included the baseline score of the measure plus those listed above. ANCOVAs were also conducted for the measures of classroom instruction using the same covariates as listed above; however, classroom instruction data was only collected at post-test, thus baseline data was not included as a covariate.

OUTCOMES

Implementation Fidelity

The research team collected videotaped data to measure the fidelity of ELLM curriculum implementation. Teachers were videotaped twice during the preschool year. The videotapes were segmented and coded to capture the presence or absence of the ELLM critical elements. Scores ranged from 0 to 147. A score of 118 or more was considered high implementation fidelity, 89 to 117 as medium fidelity, and 88 or below as low implementation fidelity.

Even though 11 of the 14 teachers were in their second year of implementing the ELLM, none were rated as high implementers. The intervention teachers were all rated as low to medium implementers. All but one of the control teachers were rated as low implementers (the one exception was rated as a medium implementer). Overall the intervention teachers were given a global 'medium' rating on implementation fidelity.

Baseline Equivalence

At baseline, the groups were equivalent at on all measured outcomes (as listed below), on all child demographics and on most parent and teacher demographics. A higher percentage of parents in the treatment group had completed some post-high school education relative to those assigned to the control group (41% vs. 29%, p < .01). Additionally, teachers in the treatment group had more years of experience teaching in a preschool setting relative to those assigned to the control group (9 years vs. 4 years, p < .01).

Differential Attrition

One control and one treatment preschool dropped out of the study during the preschool year. There is no discussion of how these two preschools differed from the 28 who remained. For individuals, Appendix B (page B-8) states in general that there was no evidence of different rates of attrition across experimental and control groups but does not present specific analyses of differential attrition for this study.

Posttest

Of the 23 measures assessed at post-test (end of pre-k), none showed a statistically significant effect of the program the ELLM children's outcomes relative to the children in the control condition.

Long-term

At the long-term follow-up (1 year after the program ended i.e. end of K), the two child-level measures of language development showed statistically significant effects. They had small to medium effect sizes (PPVT: ES = .34, p < .05; TOLD Grammatical Understanding: ES = .44, p < .05). The other 10 outcome measures were not significantly influenced by the intervention.

Study 2

Summary

Cosgrove et al. (2006) conducted a cluster-randomized trial with 48 classrooms and 466 preschool children, in which classrooms (one per school) were assigned to either the intervention or a waitlist control condition. Measures of alphabet recognition and early literacy skills were collected at baseline and posttest.

Cosgrove et al. (2006), found that in comparison to controls, students in the intervention group showed:

  • Greater recognition of letters
  • Better emerging literacy skills

Evaluation Methodology

Design

As described above, low-performing public elementary schools housing at least one early intervention pre-kindergarten class were identified in each of the three locations and randomly assigned to either ELLM or wait-list control status. Additionally, two Head Start and two subsidized sites were randomly selected in the neighborhood of each elementary school. One class from each site was randomly selected to participate. Treatment group classes implemented the Early Literacy and Learning Model curriculum in conjunction with existing curriculum. Control group classes only implemented existing curriculum. (Prevalent existing curricula were Creative Curriculum, High/Scope, and High Reach.) The intervention lasted for a year with measures applied at the start and end of the year.

Sample

Four hundred sixty-six  four-year-old children attending preschools in low-income neighborhoods served as the study sample for this investigation. These children were drawn from 48 pre-school classes, from three different geographic locations in Florida. These three locations represented differing degrees of urbanicity. The children were 12% White, 71% Black, 8% Hispanic, and 9% of other racial background.

Measures

Data were collected on the children's ability to recognize the 52 upper- and lowercase letters of the alphabet, and their early literacy abilities were measured by the Test of Early Reading Ability-Third Edition, Form A (TERA-3).

The TERA-3 is composed of three scales measuring unique but related early literacy skills: 1) the Alphabet scale measures graphophomenic knowledge; 2) the Conventions of Print scale measures knowledge of conventions of English print; and 3) the Meaning scale measures ability to comprehend meaning of print. Cronbach's Alpha coefficients of internal consistency for 4-year-old children for the Reading Quotient and the Alphabet, Conventions, and Meaning subtests are .97, .94, .88, and .94 respectively. Alpha coefficients for the 5-year-old children were comparable.

The Alphabet Letter Recognition Inventory (ALRI) is a locally developed inventory of children's ability to recognize the upper- and lowercase letters of the alphabet. Trained assessors presented uppercase letter flashcards, arranged in a fixed non-alphabetic order, to each child. The child was asked to name the letter. Following presentation of the uppercase letters, lowercase letter flashcards were presented in a similar fashion. The children's responses were recorded on scannable forms and computer scored.

Analysis

Hierarchical linear modeling (HLM) was used to test the statistical significance of these observed differences because children experienced ELLM together in classes rather than in one-on-one settings. Child-level variables included in the analyses were the TERA-3 pretest standardized scores, the ALRI pretest scores, the child's age in months on September 1, 2002, and gender. All continuous variables were grand-mean centered. Class-level variables included in the analyses were class assignment as ELLM or wait-list control, urbanicity, and educational attainment of the teacher (coded as either having completed a bachelor's degree or not).

Baseline Equivalence and Differential Attrition

At baseline, and again, near the end of the school year, children were assessed on their emerging literacy abilities and on their ability to recognize the 52 upper- and lowercase letters of the alphabet. Treatment children did not differ significantly from control children on these measures at baseline.

Attrition occurred at both the site and child levels. One wait-list control and one ELLM classroom in the largest city withdrew from the study and were not included in the analysis. In the 48 remaining classrooms, child-level attrition occurred, and 21% of the remaining children with pretest scores were unavailable for posttest. Comparison of the TERA-3 mean pretest scores of children with both pretest and posttest scores to those of the children with only pretest scores provided no evidence of systematic attrition.

Posttest

Examination of the TERA-3 and ALRI posttest scores indicated that those of the ELLM children were higher than those of the wait-list control children on all measures.

The child-level variables were statistically significant predictors in all of the analyses except for the child's gender in the Analysis of the Meaning scale. In all cases, older children scored at lower posttest levels than younger children, and, where gender was significant, boys achieved at lower levels than girls. These findings were the same for children in the ELLM and wait-list control classrooms.

At the class level, only assignment to ELLM or wait-list control classes was a statistically significant predictor in all analyses. The educational attainment of the classroom teacher was a statistically significant predictor only in the analysis of the Conventions of Print scale, and for this analysis, children who were taught by bachelor-degreed teachers achieved at higher levels in both the ELLM and wait-list control classrooms.

Analyses indicated that ELLM was more effective than traditional approaches in raising the emergent literacy achievement of children. As measured by all TERA-3 scales, the children experiencing the ELLM literacy curriculum and instruction support system achieved higher adjusted mean posttest levels of emergent literacy skills than children in the wait-list control classrooms. With the exception of the differences in the TERA-3 Conventions of Print scale adjusted mean posttest scores, resulting effect sizes were 25 percent of a standard deviation or larger (ARLI ES=.25, TERA-3 Reading ES=.28, TERA-3 Alphabet=.28, TERA-3 Conventions of Print=.17, and TERA-3 Meaning ES= .29).

Contact

Blueprints for Healthy Youth Development
University of Colorado Boulder
Institute of Behavioral Science
UCB 483, Boulder, CO 80309

Email: blueprints@colorado.edu

Sign up for Newsletter

If you are interested in staying connected with the work conducted by Blueprints, please share your email to receive quarterly updates.

Blueprints for Healthy Youth Development is
currently funded by Arnold Ventures (formerly the Laura and John Arnold Foundation) and historically has received funding from the Annie E. Casey Foundation and the Office of Juvenile Justice and Delinquency Prevention.