Sign Up For Newsletter

Blueprints For Healthy Youth Development logo

Fraction Face-Off!

A 12-week math tutoring program to improve the understanding of fractions for at-risk 4th graders through increased instruction on measurement interpretation of fractions.

Program Outcomes

  • Academic Performance

Program Type

  • Academic Services
  • Mentoring - Tutoring

Program Setting

  • School

Continuum of Intervention

  • Selective Prevention

Age

  • Late Childhood (5-11) - K/Elementary

Gender

  • Both

Race/Ethnicity

  • All

Endorsements

Blueprints: Promising

Program Information Contact

Lynn Davies
Vanderbilt University
PMB #228
110 Magnolia Circle, Suite MRL 418
Nashville, TN 37203-5721
Phone: (615) 343-4782
Email: lynn.a.davies@vanderbilt.edu
www.intensiveintervention.org/chart/instructional-intervention-tools/12928

Program Developer/Owner

Lynn S. Fuchs
Vanderbilt University


Brief Description of the Program

This program seeks to improve the understanding of fractions for at-risk 4th graders through increased instruction on measurement interpretation of fractions. The 12-week program consists of 36 30-minute lessons that take place during the math block, math center, or intervention time. Tutors instruct three students on a sequence of content that focuses primarily on representing, comparing, ordering, and placing fractions on a 0 to 1 number line, but also includes attention to part-whole interpretation and fair shares representation. Four activities comprise each lesson: introduction of concepts and skills, group work, a speed game to build fluency, and individual work. Tutors also work with students to improve task-oriented behavior, teaching students what on-task behavior means, monitoring behavior, and providing students with prizes for on-task behavior. Full-time or part-time graduate-student employees serve as tutors and are trained to implement the manualized program in a 2-day workshop. Biweekly 1-hr meetings update tutors on upcoming topics and provide time to discuss problems.

Variations on the basic program involve special attention to 1) fluency practice using speed tests for flashcards, or 2) conceptual practice requiring students to explain their reasoning about fractions to the group.

Outcomes

Compared to the control group, the intervention group improved on the following posttest outcomes:

  • compare fractions
  • fraction number line
  • fraction items
  • part-whole interpretation
  • measurement interpretation
  • fraction calculations

Brief Evaluation Methodology

The randomized controlled trials determined the effects of a tutoring program for 4th grade children at risk of having problems learning fractions. Sampling 2-8 students per classroom, one study obtained 290 students from 53 4th grade classrooms in 13 schools, and another study obtained 277 students from 49 4th grade classrooms in 14 schools. Control group students received the usual classroom instruction and remediation classes.

Of the original 290 students in the first study, 31 (10.7%) either moved before the end of the study or had missing data. Of the 277 students in the second study, 34 (12.2%) moved before the end of the study. Screening was conducted in August and September and pretests were given in September and October. The interventions took place from late October to late March, and posttests were administered in early April (within 2 weeks of intervention ending). Key outcome measures included scores on fraction tests.

Study 1

Fuchs, L. S., Schumacher, R. F., Long, J., Namkung, J., Hamlett, C. L., Cirino, P. T., . . . Changas, P. (2013). Improving at-risk learners' understanding of fractions. Journal of Educational Psychology, 105(3), 683-700.


Study 2

Fuchs, L. S., Schumacher, R. F., Sterba, S. K., Long, J., Namkung, J., Malone, A., . . . Changas, P. (2014). Does working memory moderate the effects of fraction intervention? An aptitude-treatment interaction. Journal of Educational Psychology, 106(2), 499-514. doi:10.1037/a0034341


Risk Factors

School: Poor academic performance

Protective Factors

School: Instructional Practice


* Risk/Protective Factor was significantly impacted by the program

See also: Fraction Face-Off! Logic Model (PDF)

Tutor training takes place in a 1-day workshop (typically 7 hours) at the district or school site. The cost for the one-day workshop is $1,500 plus travel. For smaller sites, a half-day workshop is available at a cost of $1,000 plus travel. Appropriate workshop length may be discussed with the trainer at the time of scheduling. In the one-day training workshop for tutors, (a) an overview of the tutoring program, goals, and topics is presented, and (b) tutoring procedures are modeled and practiced for each activity in the first set of tutoring topics. Following demonstration by the trainer, tutors practice techniques and activities in pairs and receive feedback. Additional consultation with the trainer is available by email or phone following training. Instructors may be certified teachers or paraprofessionals.

The training agenda includes the following topics and activities:

  • Fraction Face-Off! Program and RTI Overview
  • Walk-Through of Manuals, Materials, Program Sequence
  • Use of Behavior Management and Point System
  • Implementation Fidelity
  • Trainer Modeling of Lesson Content and Activities
  • Tutor Practice of Lesson Content and Activities, with Feedback Provided by Trainer
  • Availability of Technical Support

Source: Washington State Institute for Public Policy
All benefit-cost ratios are the most recent estimates published by The Washington State Institute for Public Policy for Blueprint programs implemented in Washington State. These ratios are based on a) meta-analysis estimates of effect size and b) monetized benefits and calculated costs for programs as delivered in the State of Washington. Caution is recommended in applying these estimates of the benefit-cost ratio to any other state or local area. They are provided as an illustration of the benefit-cost ratio found in one specific state. When feasible, local costs and monetized benefits should be used to calculate expected local benefit-cost ratios. The formula for this calculation can be found on the WSIPP website.

Start-Up Costs

Initial Training and Technical Assistance

The cost of an initial full-day training workshop is $1,500 plus trainer's travel expenses. For small sites, a half-day workshop is available for $1,000 plus trainer's travel expenses.

Curriculum and Materials

$40 per tutor for implementation manual and master copy of supporting materials.
$25 estimated cost per student to photocopy supporting materials.
$15 per tutor for supplemental materials (Fraction Tiles).
$10 per student for prizes to stock "fractions store", where students apply fraction knowledge to purchase prizes.
Note: Prizes are up to the tutor/school. In evaluation studies, popular prizes included sports-themed pens, pencils and erasers, card games (e.g., Old Maid and Hearts), small beach balls, play jewelry, and jump ropes.

Licensing

$10 per year per tutor to photocopy supporting materials.

Other Start-Up Costs

No information is available

Intervention Implementation Costs

Ongoing Curriculum and Materials

$25 estimated cost per student for photocopies of supporting materials.
$10 per student for prizes to stock "fractions store", where students apply fraction knowledge to purchase prizes.

Staffing

School-based tutoring may be provided by student instructional support services personnel. If school personnel are not available, salaries for tutors would be needed.

Qualifications: Program generally is delivered by teachers or paraprofessionals who are supervised by teachers. Schools may fill these positions with student instructional support staff, substitute teachers, bachelor's level paraprofessionals, graduate students (typically education/social sciences), or school volunteers. Classroom teachers may implement the program in small groups outside of mathematics instructional time.

Ratios: The program is delivered in small groups of 2-3 students per tutor. A typical scenario may be a school with 4 first-grade classrooms and 4-5 small tutoring groups that are covered by 2 tutors.

Other Implementation Costs

No information is available

Implementation Support and Fidelity Monitoring Costs

Ongoing Training and Technical Assistance

Initial training, delivered by trainers who are certified by the developer, prepares tutors to implement the program as designed. It is recommended that sites include back-up tutors in the initial training if tutor turnover is anticipated. Training costs would be incurred if new tutors are hired following the initial training workshop for the site.

Phone and email technical support from the trainer is available at $50/hour. On-site training or technical support is subject to training workshop fees ($1,500 per full day plus trainer's travel expenses).

Fidelity Monitoring and Evaluation

Fidelity monitoring may be completed by school or district personnel.

Ongoing License Fees

A $10 licensing fee allows educators to make additional copies of the manual and supporting materials for up to one year. License expires one year after date of purchase but may be renewed annually.

Other Implementation Support and Fidelity Monitoring Costs

No information is available

Other Cost Considerations

No information is available

Year One Cost Example

In this example, Fraction Face-off! is implemented in a school district with four elementary schools each with four fourth grade classrooms. Three students in each classroom are identified as at-risk for mathematics difficulty and placed in the program. Two tutors per school (8 in all) serve 48 students district wide. It is assumed that the tutors are part of the school-based instructional staff.

One-day on-site training $1,500.00
Trainer's travel expenses $1,000.00
Implementation manuals @ $40/tutor $320.00
Supplemental materials @ $15/tutor $120.00
Licensing fee to photocopy supporting materials @ $10/tutor $80.00
Photocopies of supporting materials @ $25/student $1,200.00
Prizes for fractions store @ $10/student $480.00
Total One Year Cost $4,700.00

With 48 students in the Fraction Face-off! program, the Year 1 cost per student is $97.92.

Funding Overview

No information is available

Funding Strategies

Improving the Use of Existing Public Funds

No information is available

Allocating State or Local General Funds

No information is available

Maximizing Federal Funds

Formula Funds:

  • The Individuals with Disabilities Education Act (IDEA) 2004 provides funding to school districts for early intervening services (EIS) that assist students who need increased academic and behavioral support to succeed in a general education environment. As such, IDEA 2004 is a potential source of funding for Fraction Face-off!
  • Title I funds may be used for Fraction Face-off! Schools with at least a 40 percent poverty rate may choose to operate a schoolwide program that allows Title I funds to be combined with other federal, state and local funds to upgrade the school's overall instructional program. All other schools must operate targeted assistance programs for the most at-risk students; these programs must supplement the regular education program normally provided by state and local educational agencies.

Foundation Grants and Public-Private Partnerships

The National Council of Teachers of Mathematics offers funding to "increase the breadth and depth of mathematics content knowledge" of elementary school teachers committed to mathematics teaching and learning. If a school or district wishes to train teachers in Fraction Face-off! tutoring, this may be a potential source of funding.

President Obama's 2015 budget includes investments to improve science, technology, engineering, and mathematics (STEM) education in K-12 schools, with a priority on excellent teachers, rigorous courses, and regional partnerships that enable school districts to partner with local employers, museums, universities, and others to transform STEM teaching and learning by developing coordinated plans to promote student inspiration, achievement, and preparation in STEM subjects.

Debt Financing

No information is available

Generating New Revenue

No information is available

Data Sources

No information is available

Program Developer/Owner

Lynn S. FuchsVanderbilt University228 PeabodyNashvilleNashville37203U.S.A.lynn.fuchs@vanderbilt.edu www.intensiveintervention.org/chart/instructional-intervention-tools/12928

Program Outcomes

  • Academic Performance

Program Specifics

Program Type

  • Academic Services
  • Mentoring - Tutoring

Program Setting

  • School

Continuum of Intervention

  • Selective Prevention

Program Goals

A 12-week math tutoring program to improve the understanding of fractions for at-risk 4th graders through increased instruction on measurement interpretation of fractions.

Population Demographics

Targets 4th graders with low scores on a broad-based calculations assessment.

Target Population

Age

  • Late Childhood (5-11) - K/Elementary

Gender

  • Both

Race/Ethnicity

  • All

Other Risk and Protective Factors

Risk factor: low fraction proficiency

Risk/Protective Factor Domain

  • Individual

Risk/Protective Factors

Risk Factors

School: Poor academic performance

Protective Factors

School: Instructional Practice


*Risk/Protective Factor was significantly impacted by the program

See also: Fraction Face-Off! Logic Model (PDF)

Brief Description of the Program

This program seeks to improve the understanding of fractions for at-risk 4th graders through increased instruction on measurement interpretation of fractions. The 12-week program consists of 36 30-minute lessons that take place during the math block, math center, or intervention time. Tutors instruct three students on a sequence of content that focuses primarily on representing, comparing, ordering, and placing fractions on a 0 to 1 number line, but also includes attention to part-whole interpretation and fair shares representation. Four activities comprise each lesson: introduction of concepts and skills, group work, a speed game to build fluency, and individual work. Tutors also work with students to improve task-oriented behavior, teaching students what on-task behavior means, monitoring behavior, and providing students with prizes for on-task behavior. Full-time or part-time graduate-student employees serve as tutors and are trained to implement the manualized program in a 2-day workshop. Biweekly 1-hr meetings update tutors on upcoming topics and provide time to discuss problems.

Variations on the basic program involve special attention to 1) fluency practice using speed tests for flashcards, or 2) conceptual practice requiring students to explain their reasoning about fractions to the group.

Description of the Program

This program seeks to improve the understanding of fractions for at-risk 4th graders through increased instruction on measurement interpretation of fractions. The 12-week program consists of 36 30-minute lessons that take place during the math block, math center, or intervention time. Tutors instruct three students on a sequence of content that focuses primarily on representing, comparing, ordering, and placing fractions on a 0 to 1 number line, but also includes attention to part-whole interpretation and fair shares representation. Four activities comprise each lesson: introduction of concepts or skills, group work, a speed game to build fluency, and individual work. Tutors also work with students to improve task-oriented behavior, teaching students what on-task behavior means, monitoring behavior, and providing students with prizes for on-task behavior. Full-time or part-time graduate-student employees serve as tutors and are trained to implement the manualized program in a 2-day workshop. Biweekly 1-hr meetings update tutors on upcoming topics and provide time to discuss problems.

Variations on the basic program involve special attention to 1) fluency practice using speed tests for flashcards, or 2) conceptual practice requiring students to explain their reasoning about fractions to the group.

Theoretical Rationale

This program is based on a model of mathematics learning developed by others, and centers on conceptual understanding with the major focus on the measurement interpretation of fractions.

Theoretical Orientation

  • Skill Oriented

Brief Evaluation Methodology

The randomized controlled trials determined the effects of a tutoring program for 4th grade children at risk of having problems learning fractions. Sampling 2-8 students per classroom, one study obtained 290 students from 53 4th grade classrooms in 13 schools, and another study obtained 277 students from 49 4th grade classrooms in 14 schools. Control group students received the usual classroom instruction and remediation classes.

Of the original 290 students in the first study, 31 (10.7%) either moved before the end of the study or had missing data. Of the 277 students in the second study, 34 (12.2%) moved before the end of the study. Screening was conducted in August and September and pretests were given in September and October. The interventions took place from late October to late March, and posttests were administered in early April (within 2 weeks of intervention ending). Key outcome measures included scores on fraction tests.

Outcomes (Brief, over all studies)

One study (Fuchs et al., 2013) reported significant program effects for each of six fraction outcomes at posttest (compare fractions, fraction number line, fraction items, part-whole interpretation, measurement interpretation, and fraction calculations). The other study (Fuchs et al., 2014) reported significant program effects on three outcomes at posttest (fraction number line, NAEP total, and fraction calculation).

Outcomes

Compared to the control group, the intervention group improved on the following posttest outcomes:

  • compare fractions
  • fraction number line
  • fraction items
  • part-whole interpretation
  • measurement interpretation
  • fraction calculations

Mediating Effects

Significant mediation was reported for two of the three theoretically specified pathways in Fuchs et al. (2013). The study concluded that improvement in measurement interpretation partially mediated the intervention effect on fraction items, and improvement in measurement completely mediated the program effect on part-whole interpretation. Part-whole interpretation improvement did not mediate measurement interpretation.

Fuchs et al. (2014) found that improvement in fraction number line scores mediated the effect of both the fluency and conceptual intervention on the outcome of NAEP total. The indirect effect of .12 for the fluency condition (compared to the control condition) was significant, and the indirect effect of .19 for the conceptual condition (compared to the control condition) was significant.

Effect Size

Effect sizes were small to large in Fuchs et al. (2013), ranging from .29 to 2.50, and were medium to large in Fuchs et al. (2014), ranging from .60 to 1.13.

Generalizability

Generalizability is unknown, as the study did not report details on the characteristics of the participating schools or on recruitment strategies.

Potential Limitations

  • With the exception of the NAEP, outcome measures were closely tied to program content.
  • One study (Fuchs et al., 2013) reported that pretest scores across conditions were comparable and provided means that looked similar, but did not indicate results of significance tests, although the other study (Fuchs et al., 2014) provided these results.
  • Fuchs et al. (2013) reported the coefficients for intervention as program effects, but did not detail whether interaction terms including condition status should change interpretation of main effects.
  • Generalizability is unknown, as the studies did not report details on the characteristics of the participating schools or on recruitment strategies.

Endorsements

Blueprints: Promising

Peer Implementation Sites

Jenna Noonan Davis
4th-Grade Teacher
Glenview Elementary
1020 Patricia Dr.
Nashville, TN 37217
Jenna.Noonan@mnps.org

615-360-2906

Program Information Contact

Lynn Davies
Vanderbilt University
PMB #228
110 Magnolia Circle, Suite MRL 418
Nashville, TN 37203-5721
Phone: (615) 343-4782
Email: lynn.a.davies@vanderbilt.edu
www.intensiveintervention.org/chart/instructional-intervention-tools/12928

References

Study 1

Certified Fuchs, L. S., Schumacher, R. F., Long, J., Namkung, J., Hamlett, C. L., Cirino, P. T., . . . Changas, P. (2013). Improving at-risk learners' understanding of fractions. Journal of Educational Psychology, 105(3), 683-700.

Study 2

Certified Fuchs, L. S., Schumacher, R. F., Sterba, S. K., Long, J., Namkung, J., Malone, A., . . . Changas, P. (2014). Does working memory moderate the effects of fraction intervention? An aptitude-treatment interaction. Journal of Educational Psychology, 106(2), 499-514. doi:10.1037/a0034341

Study 1

Evaluation Methodology

Design: This randomized controlled trial determined effects of a tutoring program for 4th grade children at risk of having problems learning fractions, defined as scoring below the 35th percentile on a broad-based calculations screening assessment. The study reported sampling 2-8 students per classroom, stratified by whether or not the student scored below the 15th percentile on the screening assessment. Eighteen students scoring below the 9th percentile on both subtests of the Wechsler Abbreviated Scales of Intelligence were excluded, resulting in a sample of 290 students. Presumably the study sampled all classrooms in participating schools, but only specified that the 290 students came from 53 fourth grade classrooms in 13 schools. Control group students received the usual classroom instruction and remediation classes, and did not receive any small-group tutoring. Intervention lessons were provided during instructional periods such that math instructional time was similar for control and treatment students.

Of the original 290 students, 31 (10.7%) either moved before the end of the study or had missing data. Screening was conducted in August and September, and pretests were given in September and October. The intervention took place from late October to late March, and posttests were administered in early April (within 2 weeks of intervention ending).

Sample Characteristics: The sample consisted of 4th grade children who scored below the 35th percentile on a broad-based calculations assessment. The sample was mostly African American (51-54%), followed by white (24-26%), Hispanic (19%), and other race/ethnicity (3-4%). A large proportion of students (81-83%) received subsidized lunch, and between 9 to 12% of students were English language learners. Half (50-54%) the sample was male. A small number (5%) of children received special education.

Measures: The study used the following outcome measures, collected at pretest and posttest:

  • Comparing fractions, from the 2010 Fraction Battery, developed by others. This measure assesses magnitude understanding with 15 items (alpha=.84).
  • Fraction number line, developed by others. This measure assesses measurement interpretation of fractions by requiring students to place fractions on a number line. Low scores indicate stronger performance. Test-retest reliability was .79.
  • NAEP total fraction items, taken from fraction items from the 1990-2009 NAEP, a standardized test developed by others. This 18-item outcome had an alpha of .72.
  • Part-whole interpretation, taken from eight items in the 1990-2009 NAEP, a standardized test developed by others (alpha=.60).
  • Measurement interpretation, taken from eight items in the 1990-2009 NAEP, a standardized test developed by others (alpha=.62).
  • Fraction calculations, a combination of Fraction Addition and Fraction Subtraction problems in the 2010 Fraction Battery, developed by others. This 20-item measure had an alpha of .90.

The study collected the following potential moderators at pretest, except attentive behavior, which was collected 6 weeks into intervention:

  • Listening Recall, taken from the Working Memory Test Battery for Children, developed by others. Test-retest reliability ranged from .84 to .93.
  • Counting Recall, taken from the Working Memory Test Battery for Children, developed by others. Test-retest reliability ranged from .84 to .93.
  • Attentive behavior, taken from the inattentive subscale of the Strength and Weaknesses of ADHD-Symptoms and Normal-Behavior, developed by others (alpha=.96). This measure was reported by teachers.
  • Processing speed, taken from the Woodcock-Johnson III Cross Out measures, developed by others (alpha=.91).
  • Listening comprehension, taken from the Woodcock Diagnostic Reading Battery-Listening Comprehension, developed by others (alpha=.80).
  • Whole number calculation skills, from the WRAT-4-Arithmetic, developed by others (alpha=.85).

Analysis: Ordinary least squares path analytical framework estimated intervention effects. Models controlled for baseline outcome levels, and baseline whole-number calculation skill score. For four outcomes (comparing fractions, fraction number line, part-whole interpretation, and fraction calculations) not meeting the homogeneity of regression assumption, an interaction between intervention condition and baseline outcomes was also included as a control. The study specified the reasoning behind testing homogeneity (p.690): "because we relied on a residualized change approach to analyze intervention effects (i.e., covarying pretest scores to reduce within-group error variance), we assessed the homogeneity of regression assumption".

The theoretically specified moderation analyses used the controls described above, but also included a variable for the moderator (listening recall, counting recall, attentive behavior, processing speed, or listening comprehension) and for the interaction between the moderator and condition status. When more than one moderator was significant for a given fraction outcome, the study re-examined each moderator while controlling for performance on the other significant moderator variable. It appears that the study checked models combining all moderators and outcomes, with only significant results reported.

The study reported the coefficients for intervention as main effects, but did not detail whether interaction terms including condition status should change interpretation of main effects.

The study also examined three theoretically specified mediation pathways; it determined whether improvements on fraction number line, measurement interpretation, or part-whole interpretation mediated NAEP total, part-whole interpretation, or measurement interpretation outcomes, respectively. This analysis used two steps: (1) effects of intervention on the mediator, controlling for the covariate(s), (2) effects of intervention on the outcome, controlling for the same covariates(s) and the mediator. All models controlled for the baseline outcome and the mediation model looking at NAEP total as the outcome also used fraction calculations as a covariate. For the indirect (mediation) effect, the study used bootstrapping estimation with 5,000 draws to estimate standard errors and 95% confidence intervals.

The study appears to follow intent-to-treat, as all students who did not move and who had complete data were included in the analyses. It does not appear, however, that the study attempted to follow movers.

Randomization and analysis were at the individual level. Students were clustered in classrooms, and the study determined that accounting for this clustering did not alter the results. Intra-class correlations (0 to .063) indicated that little to no proportion of variance was due to classrooms. The study reported that supplemental analyses using multilevel models demonstrated the same results as the single-level models, and multilevel results are available from the first author.

Outcomes

Implementation Fidelity: Each session was audiotaped, and research assistants listened to a random 20% of these tapes. The mean percentage of points addressed during those sessions was 97.69, suggesting high fidelity.

Baseline Equivalence: The intervention and control groups did not significantly differ on the broad-based calculations assessment, or any of the demographic measures (sex, English language learners, subsidized lunch status, special education, and race). The study reported that pretest outcome scores across conditions were comparable and provided means that looked similar, but it did not indicate results of a significance test.

Differential Attrition: The study stated that the 31 students who moved before the end of the study or had missing data did not significantly differ on pretest measures from the other 259 students.

Posttest: The study reported significant program effects for each of the six fraction outcomes at posttest (compare fractions, fraction number line, NAEP total, part-whole interpretation, measurement interpretation, and fraction calculations). Effect sizes ranged from .29 to 2.50.

Moderation: Models looking at main effects also provided information on moderating effects. First, interaction terms for whole-number calculation skill scores were nonsignificant for all six outcomes. Second, interactions with baseline levels were significant for the four outcomes that included the terms. The study did not interpret these results, but the interaction coefficients in the table indicate that intervention effects were weaker for those with higher pretest scores on compare fractions, part-whole measurement, and fraction calculations, but stronger for those with higher pretest scores for fraction number line.

Third, additional moderation analyses examined five moderators for six outcomes (30 models), and six significant effects emerged. Program effects were stronger for the compare fractions outcome among those with higher listening recall or attentive behavior and for the NAEP total outcome among those with higher attentive behavior. Conversely, the effect of the intervention was weaker on fraction number line for participants with higher counting recall and on fraction calculations for those with higher listening comprehension or processing speed. Table 5 (p. 693) reported listening recall as a moderator for compare fractions and count recall for number line, but this appears to be a mistake, as the text reports, in multiple places, that count recall is a moderator for compare fractions and listen recall for number line.

Mediation: Significant mediation was reported for two of the three pathways examined. The study concluded that improvement in measurement interpretation partially mediated the intervention effect on NAEP total, and improvement in measurement completely mediated the program effect on part-whole interpretation. Part-whole interpretation improvement did not mediate measurement interpretation.

Study 2

Evaluation Methodology

Design: This randomized controlled trial determined effects of a tutoring program for 4th grade children at risk of having problems learning fractions, defined as scoring below the 35th percentile on a broad-based calculations screening assessment. The study reported sampling 2-8 students per classroom, stratified by whether or not the student scored below the 15th percentile on the screening assessment. Eighteen students scoring below the 9th percentile on both subtests of the Wechsler Abbreviated Scales of Intelligence were excluded, resulting in a sample of 277 students from 49 classrooms in 14 schools. Control group students received the usual classroom instruction and remediation classes, and did not receive any small-group tutoring. Intervention lessons were provided during instructional periods such that math instructional time was similar for control and treatment students.

Students were randomly assigned at the individual level to three conditions: 1) treatment with fluency activities, 2) treatment with conceptual activities, and 3) control. Another 281 low-risk classmates (> 34th percentile) were randomly sampled to serve as a comparison group but were not used to test program efficacy.

Of the original 277 students, 34 (12.3%) moved before the end of the study. The analytic sample thus had data on 243 students (84 fluency practice; 79 conceptual practice; and 80 control). Screening was conducted in August and September and pretests were given in September and October. The intervention took place from late October to late March, and posttests were administered in early April (within 2 weeks of intervention ending).

Sample Characteristics: The sample consisted of 4th grade children who scored below the 35th percentile on a broad-based calculations assessment. The sample was mostly African American (58-61%), followed by Hispanic (22-24%) and white (14-17%). A large proportion of students (86-95%) received subsidized lunch, and 5-14% of students were English language learners. There were more girls (59-63%) than boys. A small portion of children (8-12%) received special education.

Measures:

Screening Measures

  • WRAT-4 Arithmetic. Used to screen students below the 35th percentile, this measure has students complete calculations of increasing difficulty and had an alpha of .85 for the sample.
  • Wechsler Abbreviated Scale of Intelligence. This measure assessed vocabulary and matrix reasoning and had an alpha exceeding .92.

Moderator Measures

  • Working Memory. The measure comes from listening recall tasks, with more correct responses indicating greater working memory. The test-retest reliabilities ranged from .84 to .93.
  • Attentive Behavior. The measure comes from the inattentive subscale of the Strength and Weaknesses of ADHD-Symptoms and Normal-Behavior, developed by others (alpha=.96). This measure was reported by teachers.
  • Processing Speed. Students are given tasks to complete in 3 minutes; the measure has a reliability of .91.
  • Language. This measure of vocabulary has a split-half reliability of .86-.87 and correlates highly (r = .72) with the Wechsler Intelligence Test.

Outcome Fraction Measures

  • Fraction Number Line. This measure assesses measurement interpretation of fractions by requiring students to place fractions on a number line. Low scores indicate stronger performance. Test-retest reliability was .80.
  • NAEP Total Fraction Items. The fraction items come from the 1990-2009 NAEP, a standardized test developed by others. This 18-item outcome had an alpha of .81.
  • Fraction Calculations. The measure combines Fraction Addition and Fraction Subtraction problems in the 2010 Fraction Battery, developed by others. This 20-item measure had an alpha of .93.

The study collected the moderators at pretest, except attentive behavior, which was collected 6 weeks into intervention. Testers were blind to condition when administering and scoring the tests.

Analysis . The study estimated cross-classified partially nested multilevel models for each of the three outcomes. The models accounted for nesting at the classroom level for all three conditions and at the small-group tutoring level for the two intervention conditions. That is, students (Level 1 units) were partially nested and cross classified in small groups (Level 2a unit, occurring only in the treatment arms) and classrooms (Level 2b unit). Classroom ICCs ranged from .04 to .07, while small-group ICCs for treatment subjects ranged from 0 to .22. Restricted maximum likelihood estimates with adjustments of standard errors for clustering came from PROC MIXED in SAS. However, the paper also states that tests for moderation used an ordinary least squares framework.

The models controlled for baseline outcomes.

The study appears to follow intent-to-treat, as all students who did not move were included in the analyses. It does not appear, however, that the study attempted to follow movers.

Outcomes

Implementation Fidelity: Each session was audiotaped, and research assistants listened to a randomly selected 20% of these tapes. The mean percentage of points addressed during those sessions was 97.3% in the fluency condition and 97.0% in the conceptual condition.

Baseline Equivalence: The three conditions did not differ significantly (p > .05) on the Wechsler Intelligence Test or on sociodemographic characteristics, including gender, English learners, subsidized lunch, special education, and race.

The study noted that the three groups were comparable at baseline on the three outcome measures. Differences in scores for fraction number line, NAEP, and fraction calculations were not significant (p >.05). Effect sizes for the three measures for pretest comparisons were under .17 with one exception - the fluency condition had higher scores than the conceptual condition on fraction calculations (d = .31).

Differential Attrition: The study stated that the 34 students who moved before the end of the study did not differ statistically from the remaining 243 students on pretest measures or on pretest measure as a function of condition.

Posttest: For all three outcome variables - fraction number line, NAEP total, and fraction calculations - the two intervention groups scored significantly better than the control group, even after a Bonferroni adjustment (p < .001). Effect sizes calculated to take account of the partial nested design ranged from .60 to 1.12 for the fluency condition and from .63 to 1.13 for the conceptual condition. The fluency and conceptual conditions did not differ significantly from one another.

Mediation: The results showed that improvement in fraction number line scores mediated the effects of both the fluency and conceptual interventions on the outcome of NAEP total. The indirect effect of .12 for the fluency condition (compared to the control condition) was significant, and the indirect effect of the conceptual condition (compared to the control condition) of .19 was also significant.

Moderation: The model used fraction number line score as the outcome but appears to use only subjects in the two intervention conditions (not the control condition) (N = 163). The models focused on the interaction between working memory (WM) and the fluency intervention relative to the conceptual intervention. The interaction results showed that "for students with very low WM, effects favored the conceptual condition; however, for students with more adequate WM, effects favored the fluency condition." Interactions involving attentive behavior, processing speed, and language were not statistically significant. The results were in the opposite direction of those predicted by the hypothesis - fluency tasks would work better for those with poorer working memory.

Contact

Blueprints for Healthy Youth Development
University of Colorado Boulder
Institute of Behavioral Science
UCB 483, Boulder, CO 80309

Email: blueprints@colorado.edu

Sign up for Newsletter

If you are interested in staying connected with the work conducted by Blueprints, please share your email to receive quarterly updates.

Blueprints for Healthy Youth Development is
currently funded by Arnold Ventures (formerly the Laura and John Arnold Foundation) and historically has received funding from the Annie E. Casey Foundation and the Office of Juvenile Justice and Delinquency Prevention.