Print Page
Blueprints logo

Olweus Bullying Prevention Program

A multi-level bullying prevention program designed to reduce and prevent school bullying in elementary, middle, and high schools.

Fact Sheet

Program Outcomes

  • Bullying
  • Delinquency and Criminal Behavior
  • Prosocial with Peers
  • Truancy - School Attendance
  • Violent Victimization

Program Type

  • Bullying Prevention
  • School - Environmental Strategies

Program Setting

  • School

Continuum of Intervention

  • Selective Prevention
  • Universal Prevention

Age

  • Late Adolescence (15-18) - High School
  • Early Adolescence (12-14) - Middle School
  • Late Childhood (5-11) - K/Elementary

Gender

  • Both

Race/Ethnicity

  • All

Endorsements

Blueprints: Promising

Program Information Contact

Jan Urbanski, Ed.D.
Director of Safe and Humane Schools
Institute on Family & Neighborhood Life
Clemson University
2037 Barre Hall
Clemson, SC 29634
(864) 656-1836
Email: jurbans@clemson.edu
Website: www.clemson.edu/olweus

Program Developer/Owner

Dan Olweus, Ph.D.
University of Bergen


Brief Description of the Program

The Olweus Bullying Prevention Program is a multi-level, multi-component program designed to reduce and prevent school bullying in elementary, middle, and high schools. Secondary goals include increased awareness and knowledge about bullying, involvement of teachers and parents in bullying prevention, development of clear rules against bullying, and providing support and protection to victims. The program includes school-, classroom-, and individual-level components. The school-level components consist of an assessment of the nature and prevalence of bullying in the school, the formation of a committee to coordinate the prevention program, and development of a system ensuring adult supervision of students outside of the classroom. Classroom components include defining and enforcing rules against bullying, discussions and activities to reinforce anti-bullying values and norms and active parental involvement in the program. Individual components intervene with students with a history of bullying and/or victimization.

The Olweus Bullying Prevention Program targets the problem of bullying at three levels: the school, the classroom, and the individual. Designed for elementary, middle, and high schools, the program addresses the problem of bullying with multiple strategies at each level. At the school level, students are given an anonymous questionnaire (25-45 minutes long) to assess the nature and prevalence of bullying at the school. The survey is administered in spring of the school year prior to program implementation. Secondly, the school administration convenes a conference day, during which program consultants and school staff discuss findings from the student questionnaire, familiarize themselves with the program and its effects (through discussions with program consultants, handbooks and videos), form a Bullying Prevention Coordinating Committee, and plan for program implementation. The coordinating committee includes representatives from all constituencies involved with the school (i.e., administration, teachers, counselors, health professionals, parents, and students). The school component also involves increased adult supervision of school areas that are frequently the setting for bullying (i.e., the playground, cafeteria, and restrooms).

The classroom-level component involves establishing clear and consistently enforced rules against bullying, along with regular class discussions and activities designed to reinforce rules and anti-bullying values and norms. Discussions and activities also present the harm caused by bullying and strategies for preventing it. The program encourages parental involvement through meetings and discussion of the problem and efforts to address it.

Individual-level components include interventions with students who bully, students who are bullied, and their parents. Interventions are designed to ensure the cessation of the bullying behavior and to provide support to victims.

Outcomes

  • Reductions in self-reported bullying are mixed across multiple evaluations, but generally positive.
  • Reductions in self-reported victimization are mixed across multiple evaluations.
  • Decreases in other forms of delinquency and anti-social behavior, such as theft, vandalism and truancy found in the original Norway study and South Carolina replication.
  • Improvements in positive social relationships and school climate found in Norway study.
  • In Pennsylvania, improvements in all 14 bullying outcomes, including a 13% decrease in the likelihood of being bullied and a 29% decrease in the likelihood of bullying others.

Study 13

Two years after baseline, Limber et al. (2018) found significant reductions in:

  • Frequency of students being bullied
  • Frequency of students bullying others

Three years after baseline, for a subgroup of schools who completed year three assessments, Limber et al. (2018) found significant reductions in:

  • Frequency of students being bullied
  • Frequency of students bullying others

Risk and Protective Factors

  • Increased empathy for bullied peers
  • Decreased willingness to join in bullying a disliked peer

Brief Evaluation Methodology

All studies have used quasi-experimental designs. The original Norway study and several subsequent replications used comparisons of adjacent age-cohorts. All but the Toronto, Ireland, and most recent Oslo studies had comparison groups, but none used random assignment to treatment and comparison groups. The primary sources of data for evaluations presented were self-report data using the Bully/Victim Questionnaire (or a modified version of it) and teacher surveys.

Study 13

Limber et al. (2018) evaluated the program in a quasi-experimental extended age cohort design in which posttest results for one cohort were compared to pretest results for the previous cohort for students in the same grade and same school. Bullying and being bullied by others, as well as two risk and protective factors for bullying, were measured at baseline and for up to three consecutive years following the beginning of the intervention, though most schools only completed assessments at baseline and two years after baseline.

Blueprints Certified Studies

Study 1

Olweus, D., & Alsaker, F. D. (1991). Assessing change in a cohort-longitudinal study with hierarchical data. In D. Magnusson, L.R. Bergman, G. Rudinger, & B. Torestad (Eds.), Problems and methods in longitudinal research: Stability and change (pp. 107-132). New York, NY: Cambridge University Press.


Study 9

Schroeder, B. A., Messina, A., Schroeder, D., Good, K., Barto, S., Saylor, J., & Masiello, M. (2011). The implementation of a statewide bullying prevention program: Preliminary findings from the field and the importance of coalitions. Health Promotion Practice. 21 March 2011, epub.


Study 10

Limber, S. P., Olweus, D., Massiello, M., Molnar-Main, S., & Moore, D. (2012). Evaluation of the Olweus Bullying Prevention Program in a large scale study in Pennsylvania. Unpublished report.


Risk and Protective Factors

Risk Factors

Individual: Bullies others*, Early initiation of antisocial behavior, Favorable attitudes towards antisocial behavior, Victim of bullying*

Peer: Interaction with antisocial peers

Protective Factors

Individual: Clear standards for behavior*, Problem solving skills, Prosocial behavior, Prosocial involvement, Refusal skills, Rewards for prosocial involvement, Skills for social interaction

Peer: Interaction with prosocial peers

Family: Parental involvement in education

School: Opportunities for prosocial involvement in education, Rewards for prosocial involvement in school


* Risk/Protective Factor was significantly impacted by the program

See also: Olweus Bullying Prevention Program Logic Model (PDF)

Race/Ethnicity/Gender Details

Gender Specific Findings
  • Male
  • Female
Race/Ethnicity Specific Findings
  • White
Race/Ethnicity/Gender Details
The program targets youth of all races and ethnicities but was designed and originally implemented in northern Europe. The program has been evaluated a number of times; some evaluations give few specific details about the ethnicity of participants and the majority do not examine effects by race. Bauer et al. (2007), however, found program effects on relational and physical victimization for white youths only, relative to youths of other ethnicities, in an evaluation of program implementation in Seattle.

Study 13

Limber et al. (2018) found a significant program by sex interaction for students in grades three through five, such that boys reported greater reductions in the frequency with which they were bullied, relative to girls. In addition, White students in grades three through eight reported greater reductions in the frequency with which they were bullied, relative to Black and Hispanic students. White students in grades three through five also reported greater reductions in their own bullying of others, relative to Black and Hispanic Students.

Training and Technical Assistance

The Olweus Bullying Prevention Program (OBPP) training and technical assistance purveyor is Clemson University, Institute on Family and Neighborhood Life. The OBPP strives to develop local capacity to implement the program systemically and long term to prevent bullying behavior and to intervene effectively when bullying occurs. A "Readiness Checklist" is available at www.violencepreventionworks.org and an archived webinar program overview is available free of charge at www.hazelden.org/webinars. Email and phone consultation is available at no cost for schools interested in implementation. Phone and email contact information can be found at www.clemson.edu/olweus/contact.html.

There are two options for training and technical assistance:

1) Schools may hire an OBPP Certified Olweus Trainer/Consultant to train the school building's leadership team (Bullying Prevention Coordination Committee [BPCC]) and provide technical assistance for implementation.

2) Schools may sponsor a professional within their community to become a Certified OBPP Trainer/Consultant in order to build implementation sustainability.

In either training option, technical assistance for a minimum of 12 months post-training is provided to assist/coach schools in implementation of the program as intended.

Costs for OBPP Trainer/Consultant traveling to school:

Training costs are negotiated with a Certified Olweus Trainer directly. A list of OBPP Trainers is found at www.clemson.edu/olweus or Jane Riese, Director of Training at jriese@clemson.edu may be contacted for assistance in finding a trainer to fit a school's needs. A single OBPP Certified Trainer/Consultant will set his/her own fees when contracting for training and consultation, but may charge no more than:

  • $3,000 for a 2-day training involving 1 or 2 schools' Bullying Prevention Coordinating Committees (BPCC).
  • An additional $250 per half-day of travel time for the trainer to travel to and from the training site.
  • Travel costs for the trainer, including airfare or mileage, lodging, meals, and local transportation, if needed.
  • Maximum of $125/hour for 12 to 24 months for telephone consultation for each school site. The fee is negotiated with the selected OBPP Trainer and includes the cost of phone calls. Consultation is required for schools to coach leadership teams to implement with fidelity.
  • If two trainers are present (required when 3 schools' BPCCs are being trained at one time), fees may increase accordingly, up to a maximum of $4,500 for the 2-day training. (No more than 3 schools may be trained at one time without prior permission from program leaders.) For more information, visit www.clemson.edu/olweus/training_staff.html#scope.
  • If there is a need for further training in following years, the trainer/consultant fees will be determined per agreed upon contract for the work required.

Training Certification Process

Costs for the Trainer Certification Course (TCC) to develop training and consultation expertise locally within district for sustainability:

The cost of participation in the TCC is $3,925, which includes:

  • 40 hours of training
    • Part I (prepares participants to train BPCCs (leadership teams)).
    • Part II (focuses on consultation skills for sustaining program with fidelity).
  • Participation in regular phone consultation with an Olweus Technical Assistance Consultant (OTAC) (12-15 hours of consultation over a 24-month period).
  • One complete set of OBPP program materials plus additional trainer/consultant materials for use when training BPCCs, educators and staff and program implementation guidance.
  • Access to the trainer-only section of the OBPP website and access to ongoing program communications.

The price does not include costs of travel, lodging or meals associated with either training session, or program materials needed for school-level implementation (committee and staff training). Those who have had training costs paid by their district or state usually provide training to BPCCs and school staff at no cost; this training and consultation may become a part of their job responsibilities.

Requirements for Certification, Trainer/Consultant Application Packets, scheduled TCC locations and upcoming dates are available at www.clemson.edu/olweus/trainer_certification.html.

Following Part 1 of the training, it usually takes an individual two years to achieve full certification. Certification of OBPP Trainers/Consultants lasts for three years. Trainers must then participate in a recertification protocol. Recertification cost is $325 for an additional three year period. Recertification is done on-line through Clemson University.

Benefits and Costs

Source: Washington State Institute for Public Policy
All benefit-cost ratios are the most recent estimates published by The Washington State Institute for Public Policy for Blueprint programs implemented in Washington State. These ratios are based on a) meta-analysis estimates of effect size and b) monetized benefits and calculated costs for programs as delivered in the State of Washington. Caution is recommended in applying these estimates of the benefit-cost ratio to any other state or local area. They are provided as an illustration of the benefit-cost ratio found in one specific state. When feasible, local costs and monetized benefits should be used to calculate expected local benefit-cost ratios. The formula for this calculation can be found on the WSIPP website.

Program Costs

Start-Up Costs

Initial Training and Technical Assistance

Initial phone consultation to determine readiness to implement Olweus Bullying Prevention Program (OBPP) is available at no cost through the OBPP training and technical assistance purveyor, Institute on Family and Neighborhood Life, Clemson University. Please call 864-656-6271. A "Readiness Checklist" is available at www.violencepreventionworks.org and an archived webinar overview of the OBPP Components is available at www.hazelden.org/webinars.

Once schools have decided to implement, there are two options for training and technical assistance:

Option 1 Costs:

Schools hire an OBPP Certified Trainer/Consultant to train the school building's Bullying Prevention Coordination Committee (BPCC]) or the School's Safe School Leadership Team and provide technical assistance for implementation.

Training costs are negotiated with a Certified Olweus Trainer directly. A single OBPP Certified Trainer/Consultant will set his/her own fees when contracting for training and consultation, but may charge no more than:

  • $3,000 for a 2-day training involving 1 or 2 schools' Bullying Prevention Coordinating Committees (BPCC).
  • An additional $250 per half-day of travel time for the trainer to travel to and from the training site.
  • Travel costs for the trainer, including airfare or mileage, lodging, meals, and local transportation, if needed.
  • Maximum of $125/hour for 12 to 24 months for telephone consultation for each school site. The fee is negotiated with the selected OBPP Trainer and includes the cost of phone calls. Consultation is required for schools to coach leadership teams to implement with fidelity.
  • If two trainers are present (required when 3 schools' BPCCs are being trained at one time), fees may increase accordingly, up to a maximum of $4,500 for the 2-day training. (No more than 3 schools may be trained at one time without prior permission from program leaders.) For more information, visit www.clemson.edu/olweus/training_staff.html#scope.
  • If there is a need for further training in following years, the trainer/consultant fees will be determined per agreed upon contract for the work required.

Option 2 Costs:

Schools invest in professional development to train their own Certified OBPP Trainer/Consultant and make ongoing training for school committees and staff a part of the professional's job duties. This builds in lower costs for additional school training and program sustainability.

The cost of participation in the Trainer/Consultant Certification Course (TCC) is $3,925, which includes:

  • 40 hours of training
    • Part I (prepares participants to train BPCCs (leadership teams)).
    • Part II (focuses on consultation skills for sustaining program with fidelity).
  • Participation in regular phone consultation with an Olweus Technical Assistance Consultant (OTAC) (12-15 hours of consultation over a 24-month period).
  • One complete set of OBPP program materials plus additional trainer/consultant materials for use when training BPCCs, educators and staff and program implementation guidance.
  • Access to the trainer-only section of the OBPP website and access to ongoing program communications.
  • The price does not include costs of travel, lodging or meals associated with either training session, or program materials needed for school-level implementation (committee and staff training).

Following Part 1 of the training, it usually takes an individual two years to achieve full certification as certification requires training of two Bullying Prevention Coordinating Committees and consultation with each committee for 12 to 18 months to guide implementation efforts. (For certification requirements: www.clemson.edu/olweus/trainer_certification.html) Full Certification of OBPP Trainers/Consultants lasts for three years. Trainers must then participate in a recertification protocol. Recertification cost is $325 for an additional three-year period. Recertification is done on-line through Clemson University.

Several State Departments of Education and/or Area Education Agencies have Certified OBPP Trainer/Consultants in their employ. Training may be available to schools in those areas free of charge. (Contact OBPP National Training Director, Jane Riese at jriese@clemson.edu, to see if this option exists for your area).

Curriculum and Materials

Costs below are standard costs for hard copy materials. Bulk discounts are available as well as electronic formats. Contact Hazelden Publishing for information on electronic options and exact price quotes. 1-800-328-9000.

Required Program Materials:

1. Olweus Bullying Questionnaire (OBQ) - Each student grades 3-12. $33.95 for 30 scannable without scanning service; $39.95 for 30 scannable with scanning service.

2. OBPP Schoolwide Guide - For Bullying Prevention Coordinating Committee members. $89.95

3. OBPP Teacher Guide - For classroom teachers and identified staff members. $59.00

Optional Recommended Program Materials: These materials were developed to fulfill schools' requests for additional support to implement with fidelity, but they are not required.

1. Class Meetings that Matter: A Year's Worth of Class Meeting Ideas for Grades K-5, Grades 6-8, and Grades 9-12. $79.00

2. Class Meeting and Individual Intervention DVD for elementary and middle school (high school version available Fall 2013). This DVD is used for BPCC, staff and parent training. $195.00

3. Bullying for Grades K-5 and Grades 6-8. A general video that introduces key concepts to students, teachers and parents. (Good for parent groups, class meetings, teacher discussion groups.) $119.00

4. Cyberbullying Curriculum for Grades 3-5 and Grades 6-12. $109.00

Licensing

None.

Other Start-Up Costs

Districts should plan on two full days of training for their BPCCs and one full day of training for educators and administrators. BPCCs will also need to train other school staff, i.e., playground supervisors, substitute teachers, lunchroom and custodial staff, and bus drivers.

Intervention Implementation Costs

Ongoing Curriculum and Materials

All required materials are purchased the first year. Some schools purchase optional recommended program materials the second year of implementation, others purchase all materials during year one. Materials are for educators and staff and do not require replacement.

Student surveys are administered each year - their costs vary depending whether scannable, on-line, or electronic subscription versions are utilized. A full survey analysis and report including graphs for presentations for each school is included in the survey price. Costs vary from $1.28 per student for the scannable survey to a few cents per student in large districts using the on-line survey in the electronic subscription.

Staffing

Qualifications: Educators, counselors, staff, and administrators are the primary implementers of the program and it is incorporated into the regular school day.

Ratios: The teacher to student ratio reflects the ratios in the school.

Time to Deliver Intervention: All adults in the school will be aware of bullying behavior and use the program's principles in daily interaction with students and other adults. Bullying prevention and intervention become a regular part of the safe school culture. The class meeting is a specific part of the program for facilitated discussions around positive behavior expectations, social emotional learning opportunities, community-building, as well as how to handle bullying situations. Recommended times for class meetings are 15-20 minutes for elementary students and 20-40 minutes for middle school students, once a week; one full class period for high school students twice a month.

Other Implementation Costs

Support of BPCC activities that are part of the program such as parent training and a schoolwide event to launch the program.

Implementation Support and Fidelity Monitoring Costs

Ongoing Training and Technical Assistance

Technical assistance is provided by the Certified OBPP Trainer/Consultant as a part of the training/consultation contract established prior to the BPCC training. Suggested time is one hour per month for 12-24 months. Additional consultation may be purchased as arranged with the trainer or with Clemson University.

Resources must be devoted to ongoing administrator, educator, and staff development each year for new teachers, administrators, substitute teachers, staff, and new parent training. Training in following years may be conducted by a district sponsored Certified Trainer, or trained members of the school's BPCC at no additional cost, or by contract with a Certified OBPP Trainer/Consultant.

Fidelity Monitoring and Evaluation

This is the responsibility of the BPCC in each school. Fidelity checklists and evaluation protocols are offered with the program.

Ongoing License Fees

None at this time.

Other Implementation Support and Fidelity Monitoring Costs

No information is available

Other Cost Considerations

Electronic materials represent large costs savings for schools. (All program materials are distributed through Hazelden Publishing. Hazelden can provide detailed cost estimates based on local needs. Call 1-800-328-9000.) It is typically more cost efficient for districts with three or more buildings to have local staff trained to become Certified OBPP Trainer/Consultants.

Year One Cost Example

If a school district were to implement the Olweus Bullying Prevention Program in two school buildings (e.g., 1 middle school and 1 high school) with 20 teachers, 10 BPCC members, and 250 students per school, and use printed materials, the following maximum Year One costs could be anticipated.

Training 2-day - Using Contracted Trainer $3,000.00
Trainer Travel Time $500.00
Trainer Travel Expenses $1,000.00
Consultation - $125/hr x 1 hr/month x 12 months x 2 schools $3,000.00
Schoolwide Guide - $89.00 x 10 copies x 2 schools $1,780.00
Olweus Bullying Questionnaire - $1.28 x 250 students x 2 schools $640.00
Teacher Guide - $59.00 x 20 teachers x 2 schools $2,360.00
Optional: Class Meeting Individual Interventions DVDs - $175 x 2 schools $350.00
Optional: Class Meetings that Matter - $79.00 x 20 teachers x 2 schools $3,160.00
Optional: Bullying DVD - $119.00 x 2 schools $238.00
Total One Year Cost $16,028.00

In this example, a total of 500 students would receive the program in two schools (each school includes 10 BPCC members and 20 teachers). Using hard copy materials, the Year One cost per student would be $24.56. If the optional materials were included, the unit cost would be $32.06.

The Year One cost for a larger district implementing OBPP in ten schools (with 20,000 students, 120 BPCC members, and any number of teachers), including training and the purchase of the electronic materials subscription, would be $45,000 or $2.25 per student.

Funding Strategies

Funding Overview

With teachers implementing the OBPP during regular class time, the main categories of cost are initial training/consultation and materials. Sustainability costs may involve training each year for new school administrators, educators, staff and parents if conducted by someone outside the district. The BPCC (or overall School Climate Committee) may also need refresher trainings as members will change over the long term.

Grants, whether federal or from the foundation community, are good options for initial costs. Parent organizations within the schools have often funded bullying prevention training or materials for school professionals. Funding toolkits for writing grants for implementation of the OBPP are available on the OBPP website olweus.sites.clemson.edu/documents/OBPPFundingTookKit.pdf.

Allocating State or Local General Funds

State education funds as well as local school district funding should be considered for both training and materials. Some county and state mental health funds have been used to support OBPP implementation in schools.

Maximizing Federal Funds

Formula Funds: Title I funds can potentially support materials purchase, training and teacher salaries in schools that are operating schoolwide Title I programs. While OBPP is integrated into the curriculum, it must be shown to contribute to overall academic achievement.

Discretionary Grants: Federal discretionary grants from the Department of Education have been used to fund the initial training of BPCC members, administrators, teachers and staff.

Foundation Grants and Public-Private Partnerships

Foundations can be a good source of funds for initial training, student surveys and materials.

Generating New Revenue

Community fundraising through parent-teacher organizations or partnerships with local business and civic associations can assist with funding training and/or materials.

Data Sources

All information comes from the responses to a questionnaire submitted by the purveyor.

Evaluation Abstract

Program Developer/Owner

Dan Olweus, Ph.D.University of BergenUNI HealthPb 7800BergenNorwayolweus@uni.no

Program Outcomes

  • Bullying
  • Delinquency and Criminal Behavior
  • Prosocial with Peers
  • Truancy - School Attendance
  • Violent Victimization

Program Specifics

Program Type

  • Bullying Prevention
  • School - Environmental Strategies

Program Setting

  • School

Continuum of Intervention

  • Selective Prevention
  • Universal Prevention

Program Goals

A multi-level bullying prevention program designed to reduce and prevent school bullying in elementary, middle, and high schools.

Population Demographics

The sample in the initial evaluation is taken from 112 classes of 5th-8th graders in 42 schools in Bergen, Norway. It is assumed that the sample is representative of the population of 4th-7th graders in the country of Norway. Evaluations in other countries, including the U.S., have also targeted elementary, middle, and high school students.

Target Population

Age

  • Late Adolescence (15-18) - High School
  • Early Adolescence (12-14) - Middle School
  • Late Childhood (5-11) - K/Elementary

Gender

  • Both

Gender Specific Findings

  • Male
  • Female

Race/Ethnicity

  • All

Race/Ethnicity Specific Findings

  • White

Race/Ethnicity/Gender Details

The program targets youth of all races and ethnicities but was designed and originally implemented in northern Europe. The program has been evaluated a number of times; some evaluations give few specific details about the ethnicity of participants and the majority do not examine effects by race. Bauer et al. (2007), however, found program effects on relational and physical victimization for white youths only, relative to youths of other ethnicities, in an evaluation of program implementation in Seattle.

Study 13

Limber et al. (2018) found a significant program by sex interaction for students in grades three through five, such that boys reported greater reductions in the frequency with which they were bullied, relative to girls. In addition, White students in grades three through eight reported greater reductions in the frequency with which they were bullied, relative to Black and Hispanic students. White students in grades three through five also reported greater reductions in their own bullying of others, relative to Black and Hispanic Students.

Other Risk and Protective Factors

Risk and protective factors have been shown to differ for bullies, victims, and bully-victims.

Risk/Protective Factor Domain

  • Individual
  • School
  • Peer
  • Family

Risk/Protective Factors

Risk Factors

Individual: Bullies others*, Early initiation of antisocial behavior, Favorable attitudes towards antisocial behavior, Victim of bullying*

Peer: Interaction with antisocial peers

Protective Factors

Individual: Clear standards for behavior*, Problem solving skills, Prosocial behavior, Prosocial involvement, Refusal skills, Rewards for prosocial involvement, Skills for social interaction

Peer: Interaction with prosocial peers

Family: Parental involvement in education

School: Opportunities for prosocial involvement in education, Rewards for prosocial involvement in school


*Risk/Protective Factor was significantly impacted by the program

Brief Description of the Program

The Olweus Bullying Prevention Program is a multi-level, multi-component program designed to reduce and prevent school bullying in elementary, middle, and high schools. Secondary goals include increased awareness and knowledge about bullying, involvement of teachers and parents in bullying prevention, development of clear rules against bullying, and providing support and protection to victims. The program includes school-, classroom-, and individual-level components. The school-level components consist of an assessment of the nature and prevalence of bullying in the school, the formation of a committee to coordinate the prevention program, and development of a system ensuring adult supervision of students outside of the classroom. Classroom components include defining and enforcing rules against bullying, discussions and activities to reinforce anti-bullying values and norms and active parental involvement in the program. Individual components intervene with students with a history of bullying and/or victimization.

Description of the Program

The Olweus Bullying Prevention Program targets the problem of bullying at three levels: the school, the classroom, and the individual. Designed for elementary, middle, and high schools, the program addresses the problem of bullying with multiple strategies at each level. At the school level, students are given an anonymous questionnaire (25-45 minutes long) to assess the nature and prevalence of bullying at the school. The survey is administered in spring of the school year prior to program implementation. Secondly, the school administration convenes a conference day, during which program consultants and school staff discuss findings from the student questionnaire, familiarize themselves with the program and its effects (through discussions with program consultants, handbooks and videos), form a Bullying Prevention Coordinating Committee, and plan for program implementation. The coordinating committee includes representatives from all constituencies involved with the school (i.e., administration, teachers, counselors, health professionals, parents, and students). The school component also involves increased adult supervision of school areas that are frequently the setting for bullying (i.e., the playground, cafeteria, and restrooms).

The classroom-level component involves establishing clear and consistently enforced rules against bullying, along with regular class discussions and activities designed to reinforce rules and anti-bullying values and norms. Discussions and activities also present the harm caused by bullying and strategies for preventing it. The program encourages parental involvement through meetings and discussion of the problem and efforts to address it.

Individual-level components include interventions with students who bully, students who are bullied, and their parents. Interventions are designed to ensure the cessation of the bullying behavior and to provide support to victims.

Theoretical Rationale

For both the student who bullies and the student who is bullied, the problem of bullying is best addressed through a systematic restructuring of the social environment. Bullying behavior is stopped and redirected by eliminating the opportunities and reward structures for anti-social behavior and encouraging and rewarding pro-social behavior.

Theoretical Orientation

  • Person - Environment

Brief Evaluation Methodology

All studies have used quasi-experimental designs. The original Norway study and several subsequent replications used comparisons of adjacent age-cohorts. All but the Toronto, Ireland, and most recent Oslo studies had comparison groups, but none used random assignment to treatment and comparison groups. The primary sources of data for evaluations presented were self-report data using the Bully/Victim Questionnaire (or a modified version of it) and teacher surveys.

Study 13

Limber et al. (2018) evaluated the program in a quasi-experimental extended age cohort design in which posttest results for one cohort were compared to pretest results for the previous cohort for students in the same grade and same school. Bullying and being bullied by others, as well as two risk and protective factors for bullying, were measured at baseline and for up to three consecutive years following the beginning of the intervention, though most schools only completed assessments at baseline and two years after baseline.

Outcomes (Brief, over all studies)

In the original Norway study and the South Carolina replication, there were reductions in self-reported bullying and antisocial behaviors (theft, vandalism, and truancy). Only the Norway study demonstrated reductions in self-reported victimization and improved school climate, as well as teacher and peer reports of bully-victim problems. The outcomes in Norway were found in the first follow-up (8 months after baseline) as well as the second follow-up (20 months after baseline). South Carolina outcomes were significant only after one year of the program and were not found after two program years. An English study also showed significant decreases in self-reported frequency of bullying. A second U.S. study in Seattle showed no overall effects on physical or relational victimization; however, both types of bullying victimization were significantly reduced among white intervention students, relative to white controls.

In Pennsylvania, there were small but significant improvements over time in all 14 bullying outcomes, including a 13% decrease in the likelihood of being bullied and a 29% decrease in the likelihood of bullying others.

Other studies were too weak to merit mention of findings.

Study 13

Two years after baseline, Limber et al. (2018) found significant reductions in the frequency of students being bullied and students bullying others, both when examining the percentage of students who reported being bullied or bullying others two to three times per month or more, and on composite scores representing the frequency with which students experienced or engaged in nine different types of bullying. These effects were maintained three years after baseline in the subgroup of schools that completed year three assessments, and additional improvements were found in risk and protective factors for bullying (increased empathy for a bullied peer, reductions in willingness to join in bullying a disliked peer) across the three years in this subgroup.

Outcomes

  • Reductions in self-reported bullying are mixed across multiple evaluations, but generally positive.
  • Reductions in self-reported victimization are mixed across multiple evaluations.
  • Decreases in other forms of delinquency and anti-social behavior, such as theft, vandalism and truancy found in the original Norway study and South Carolina replication.
  • Improvements in positive social relationships and school climate found in Norway study.
  • In Pennsylvania, improvements in all 14 bullying outcomes, including a 13% decrease in the likelihood of being bullied and a 29% decrease in the likelihood of bullying others.

Study 13

Two years after baseline, Limber et al. (2018) found significant reductions in:

  • Frequency of students being bullied
  • Frequency of students bullying others

Three years after baseline, for a subgroup of schools who completed year three assessments, Limber et al. (2018) found significant reductions in:

  • Frequency of students being bullied
  • Frequency of students bullying others

Risk and Protective Factors

  • Increased empathy for bullied peers
  • Decreased willingness to join in bullying a disliked peer

Mediating Effects

Study 13: Limber et al. (2018) did not examine mediating effects.

Effect Size

Study 13: Limber et al. (2018) found effect sizes ranging from d= 0.31 (reductions in the frequency with which third grade students bullied others from baseline to two years after baseline) to d= 2.56 (increases in 9th-11thgrade students' perceptions that their primary teacher had increased his or her efforts to address bullying from baseline to three years after baseline).

Generalizability

The program has been evaluated in multiple settings with mixed results. The first U.S. replication in South Carolina demonstrated mixed results. The population was predominately African-American, poor and lived in areas with high rates of delinquency, suggesting that the program may not be generalizable to populations similar to the one in this study. The 2007 Seattle-based replication furthers this notion, finding that physical and relational bullying victimization was significantly reduced for White treatment youth, versus youth of other ethnicities.

Other samples used in program evaluations have been representative of the general population from which they came. All but the Norway original evaluation indicated these were ethnically diverse populations. Students' reports of bullying and victimization were significantly reduced and were generally consistent across gender and grade subgroups. Reductions in other types of delinquency and anti-social behavior, such as school misbehavior and vandalism, were also consistent across studies, where measured, regardless of gender or grade, indicating that the program was equally effective for boys and girls of various ages (although one small study in one Catholic middle school showed no positive results for boys).

Study 13: Limber et al. (2018) can generalize their sample to other southern and central North Carolina students in grades 3-11.

Potential Limitations

When all of the evaluation studies are considered, most are weak methodologically. The program has demonstrated some positive results in quasi-experimental studies using an age cohort design, although there are some mixed findings across studies. The studies lacked random assignment of schools to treatment and control samples.

Study 13

Limber et al. (2018) had the following limitations:

  • QED with non-random assignment and limited matching (by grade and within school)
  • No tests for baseline equivalence
  • No baseline outcome controls
  • Tests for differential attrition are incomplete

Endorsements

Blueprints: Promising

Peer Implementation Sites

Stark County, Ohio
Maureen Capellas
Olweus Bullying Prevention Program Coordinator Stark County ESC
2100 38th Street NW
Canton, OH 44709
Phone: (330) 492-8136 ext.1585
Fax: (330) 491-9731
maureen.capellas@email.sparcc.org
http://www.starkspeaksupforkindness.org

Everardo Marquez, Principal
Sherwood Elementary School
110 S. Wood St. Salinas, CA, 93905
emarquez@salinascity.k12.ca.us
831-753-5650
Fax # 831 751-3616

Program Information Contact

Jan Urbanski, Ed.D.
Director of Safe and Humane Schools
Institute on Family & Neighborhood Life
Clemson University
2037 Barre Hall
Clemson, SC 29634
(864) 656-1836
Email: jurbans@clemson.edu
Website: www.clemson.edu/olweus

References

Study 1

Olweus, D. (1992). Bullying among school children: Intervention and prevention. In R. D. Peters, R. J. McMahon, & V. L. Quinsey (Eds.), Aggression and violence throughout the life span (pp.100-125). Newbury Park, CA: Sage Publications.

Olweus, D. (1993). Bullying at school: What we know and what we can do. Oxford, England: Blackwell.

Certified Olweus, D., & Alsaker, F. D. (1991). Assessing change in a cohort-longitudinal study with hierarchical data. In D. Magnusson, L.R. Bergman, G. Rudinger, & B. Torestad (Eds.), Problems and methods in longitudinal research: Stability and change (pp. 107-132). New York, NY: Cambridge University Press.

Study 2

Melton, G. B., Limber, S. P., Cunningham, P., Osgood, D. W., Chambers, J., Flerx, V., . . . Nation, M. (1998). Violence among rural youth. Final report to the Office of Juvenile Justice and Delinquency Prevention. Washington, DC: Office of Juvenile Justice and Delinquency Prevention.

Study 3

Eslea, M. (1998). The long-term effectiveness of anti-bullying work in primary schools. Educational Research, 40, 203-218.

Smith, P. K. (1997). Bullying in schools: The UK experience and the Sheffield anti-bullying project. The Irish Journal of Psychology, 18, 191-201.

Whitney, I., & Smith, P. K. (1993). A survey of the nature and extent of bullying in junior/middle and secondary schools. Education Research, 35, 3-25.

Whitney, I., Rivers, I., Smith, P. K., & Sharp, S. (1994). The Sheffield Project: Methodology and findings. In P.K. Smith & S. Sharp (Eds.), School bullying: Insights and perspectives (pp. 20-56). London, England: Routledge.

Study 4

Pepler, D. J., Craig, W. M., Ziegler, S., & Charach, A. (1994). An evaluation of an anti-bullying intervention in Toronto schools. Canadian Journal of Community, 13, 95-110.

Study 5

O'Moore, A. M., & Minton, S. J. (2005). Evaluation of the effectiveness of an anti-bullying programme in primary schools. Aggressive Behavior, 31, 609-622.

Study 6

Olweus, D. (2005). New positive results with the Olweus Bullying Prevention Program in 37 Oslo schools. The HEMIL-Center, University of Bergen.

Study 7

Bauer, N. S., Lozano, P., & Rivara, F. P. (2007). The effectiveness of the Olweus Bullying Prevention Program in public middle schools: A controlled trial. Journal of Adolescent Health, 40(3), 266-274.

Study 8

Bowllan, N. M. (2011). Implementation and evaluation of a comprehensive, school-wide bullying prevention program in an urban/suburban middle school. Journal of School Health, 81(4), 167-173.

Study 9

Certified Schroeder, B. A., Messina, A., Schroeder, D., Good, K., Barto, S., Saylor, J., & Masiello, M. (2011). The implementation of a statewide bullying prevention program: Preliminary findings from the field and the importance of coalitions. Health Promotion Practice. 21 March 2011, epub.

Study 10

Certified Limber, S. P., Olweus, D., Massiello, M., Molnar-Main, S., & Moore, D. (2012). Evaluation of the Olweus Bullying Prevention Program in a large scale study in Pennsylvania. Unpublished report.

Study 11

Amundsen, E. J., & Ravndal, E. (2010). Does successful school-based prevention of bullying influence substance use among 13- to 16-year-olds? Drugs: Education, Prevention and Policy, 17(1), 42-54.

Olweus, D. (1987). Schoolyard bullying-grounds for intervention. School Safety, 4, 4-14.

Study 12

Yaakub, N. F., Haron, F., & Leong, G. C. (2010). Examining the efficacy of the Olweus prevention programme in reducing bullying: The Malaysian experience. Procedia Social and Behavioral Sciences, 5. http://www.sciencedirect.com, 595-598

Study 13

Limber, S. P., Olweus, D., Wang, W., Masiello, M., & Breivik, K. (2018). Evaluation of the Olweus Bullying Prevention Program: A large scale study of US students in grades 3-11. Journal of School Psychology, 69,56-72.

Study 1

Evaluation Methodology

Design: The Olweus Bullying Prevention Program in Norway was the result of a state mandated effort to combat serious bullying in Norwegian schools and classrooms. The catalyst for the mandate was the suicide deaths of three Norwegian school children with a history of victimization. The gravity of the tragedies, the media attention and government financing and coordination of the program fostered a great deal of support by teachers, parents and the general public.

Because the Bullying Prevention Program was a nationwide anti-bullying program, random selection of treatment schools was not possible. The evaluation was a longitudinal, quasi-experimental design with time-lagged contrasts of four adjacent grade cohorts. In this design, the grade 5 cohort at T1 was compared with the T2 data for the grade 4 cohort, which at that time had reached the same age as the baseline group. The same kind of comparisons were made between the grade 6 cohort at T1 and the grade 5 cohort at T2 and between the grade 7 cohort at T1 and the grade 6 cohort at T2. Only two of the cohorts could be used to assess T1 and T3 effects. Grade 6 and grade 7 cohorts were contrasted with data collected at T3 on the grade 4 and grade 5 cohorts, respectively.

All participants were administered a Bully/Victim questionnaire, assessing the nature and prevalence of bullying in their school, in the spring of the school year prior to program implementation (May 1983). Follow-up questionnaires were administered in the spring of the following two school years, i.e. after eight months of treatment (May 1984) and twenty months of treatment (May 1985).

Sample: The sample included 2,500 5th-8th grade students in 112 classrooms in 42 schools in Bergen, Norway. Each cohort consisted of between 500 and 700 students. Classes were the sampling unit used in analyses.

Measures: Measures taken from the Bully/Victim Questionnaire included (1) items on the questionnaire regarding the frequency of bullying incidents during the semester, (2) students' attitudes toward bullying and (3) teacher responses to bullying. Students were also given a 23 item questionnaire regarding self-reported anti-social behaviors, items assessing school climate. Additional measurements included teacher ratings of classroom bullying behaviors and a four-dimensional measure of classroom climate.

Analysis: Analysis of variance (ANOVA) was used to capture the effects of students within classes within schools within cohorts. To determine the intervention effects controlling for the cohort effects, comparisons were made for age/grade equivalent cohorts. In other words, separate analyses for each combination of age/grade cohorts were conducted comparing Time 1 (T1) to Time 2 (T2) and Time 1 to Time 3 (T3).

Outcomes

Bergen, Norway (Olweus & Alsaker, 1991):
Students' reports of bullying and victimization were significantly reduced, in most cases by 50% or more. These reductions were generally consistent across gender and grade subgroups. Some program effects were significantly stronger at the second follow-up (twenty months post baseline) compared to the first follow-up (eight months post baseline). Teacher and peer reports also indicated a decrease in bully/victim problems. Students reported an improvement in order and discipline, positive social relationships and attitudes toward school, while also indicating reductions in anti-social behavior, such as theft, vandalism and truancy, all of which indicate an improvement in overall school climate. Students reported an increase in their satisfaction with school.

Additional analyses indicated a cohort effect in bully/victim problems among the four grades included in the evaluation. However, when the cohort effect was controlled for, the intervention effect from baseline to follow-ups remained significant. Also, classes that received stronger implementation of integral program components showed greater reductions in bully/victim behaviors.

Study 2

Evaluation Methodology

Design: The evaluation was a quasi-experimental design with non-random assignment of treatment and comparison sites in matched pairs. The evaluation sample was comprised of 4th, 5th and 6th grade students in six non metropolitan districts in South Carolina. The districts were divided and matched into pairs based on location and student demographics. Districts were predominately African-American, poor and located in counties with high juvenile delinquency rates. Baseline data was collected in 1995 (Y1), when participants were in grades 4-6. Follow-up data was collected in March of 1996 (Y2) and 1997 (Y3). All schools in one district in each pair received the program in Y1, after baseline data collection. These schools formed group A. Schools in the other district in each pair received the program one year later, at Y2. These schools formed group B. Group B served as a control group for Y1. There were 11 group A schools in Y1 and 28 group B schools. In Y2, all group A schools continued in the evaluation, and 7 group B schools began the program. Data was collected from 6,389 students in Y1, 6,263 students in Y2 and 4,928 in Y3.

Sample Characteristics: Data was collected from 6,389 students in Y1, when students were in grades 4-6. In Y2, when the sample was in grades 5-7, 6,263 students were surveyed. In Y3, when students were in grades 6-8, there were 4,928 students in the sample. Schools in the sample were predominately African American (ranging from 46-95%), poor and located in counties with high juvenile arrests rates in the state (top 15% of all counties in S.C.).

Measures: The evaluation used the original Bully/Victim Questionnaire developed by Olweus for the Norway program (see primary study). Of the twelve scales, there were measures of bullying frequency, victimization frequency, frequency of bullying of teachers, and attitudes about bullying. Eight scales measured self-reported anti-social behavior (i.e. theft, vandalism and truancy), violence, delinquency, school misbehavior, school sanctions, group delinquency and substance use.

Analyses: Hierarchical linear modeling (HLM) was used to estimate program effects. Gender and grade were included as Level 1 variables, as was overall program impact. This variable compares scores from baseline to scores after each group has experienced one year of the intervention. Also at Level 1, the time variable was coded to distinguish the effect of the second year of the program (time 3). A Level 2 variable was included to provide a test of group differences at each point of measurement (group). Analysis of covariance (ANCOVA) was used to determine group differences after one year of the program, at T2.

Outcomes
Baseline Equivalence and Differential Attrition: Group A had higher rates bullying peers and bullying teachers, but Group A was equivalent to Group B on measures of victimization and attitudes toward bullying.

Posttest: Results of HLM demonstrated no significant differences between Group A and Group B in bullying peers, victimization, anti-social behavior, attitudes toward bullying or bullying teachers. There were no overall program effects (combining the effects for both years for group A and the second year group B) or year 2 effects.

ANCOVA, used to determine group differences after one year of the program, indicated that Group A after one year of the program, compared to Group B, had reduced rates of bullying and anti-social behaviors, i.e. theft, vandalism and truancy, but no differences on measures of victimization and bullying teachers.

HLM analyses indicated the program demonstrated positive results on measures of bullying and anti-social behavior, i.e. theft, vandalism and truancy, for group A after one year of treatment, but these differences were not sustained, as indicated by follow-up surveys in Y3. Group B, who received delayed program implementation, demonstrated no program effect.

Brief Bulleted Outcomes:

  • Positive outcomes for intervention youth on reductions in bullying and anti-social behavior after one year of participation, compared to control condition.

Limitations: There were baseline differences between the two groups. Differential attrition is not reported, and there were three schools that dropped from the control group by year two. The outcomes were mixed, with nothing found using HLM analysis for examining the effects of treatment over both years, or for year two alone. The ANCOVA at the end of year one found effects of treatment on frequency of bullying others and antisocial behavior.

Study 3

Evaluation Methodology

Design: The evaluation was a quasi-experimental design with adjacent age-cohorts. Baseline data was collected in November of 1990 and schools began program implementation in 1991. The first steps of program implementation, inviting schools to participate in the program, began in February of 1991. Teacher training workshops took place in spring and summer and program implementation and monitoring began in September of 1991. Follow-up data was collected in November of 1992, after a full year of treatment.

The participants were 8 to 11 years old (enrolled in junior/middle schools, also referred to in the article as primary schools) and 11 to 16 years old (enrolled in secondary schools). A total of 6,758 students from twenty four schools participated in surveying at baseline (2,623 from 17 primary schools and 4,135 from 7 secondary schools). One primary school and three secondary schools declined program participation, but agreed to be surveyed at follow-up. These four schools serve as comparison sites, who did little or nothing to address the problem of bullying.

Implementation and program development did not proceed along the optimal timeline. At the time of the follow-up survey only 8 primary schools and 4 secondary schools had made the expected progress through stages of development. In order to ensure comparability between baseline and follow-up, researchers omitted certain classes from each year so that the numbers of classes and pupils were equal at both survey times.

Sample Characteristics: The baseline sample was comprised of 6,758 students (1,271 boys and 1,352 girls) from 24 schools. This included 17 primary schools, with 112 classes of 2,623 students ages 8-11 and 7 secondary schools, with 211 classes of 4,135 students ages 11-16. The schools represented all areas of the city.

Measures: The student self-report survey was a modified version of the original Olweus Bully/Victim Questionnaire. At the follow-up survey, students were asked if they perceived the school had taken any action to stop bullying and if there had been any change as a result of that action.

Analyses: Change in means between the first year (T1) and second year (T2) surveys were compared using single-sample t-tests and probabilities for percentage difference scores.

Outcomes
Baseline Equivalence and Differential Attrition: After the researchers made adjustments to ensure comparability between baseline and follow-up, there were 2,212 primary school students and 4,256 secondary school students in treatment schools, and 99 primary students and 1,742 secondary students in comparison schools. The total number of students aged 8-16 years was 8,309.

Posttest: Most individual treatment schools indicated increases in the number of students who did not bully others, a decrease in the frequency of bullying others and a decrease in the number of peers who bully others. The only significant result across all schools was a decrease in students' self-reported frequency of bullying other students (a 12% decrease). In most schools, there was an increase in the number of students who reported spending break time alone. There was also an increase in the number of students who indicated they would not join in bullying. A larger percentage of students were reporting to adults that they were being bullied and that an adult had talked to them if they had bullied others. However, there were no differences in their perception of teachers stopping bullying. For most measures, changes were better in treatment schools than in comparison schools. Among comparison schools, one primary school and one secondary school outcome measure indicated bullying problems had worsened between baseline and follow-up.

Long-Term (Eslea, M., 1998).
This study is a follow-up study conducted in late 1993 after 2 years of intervention in the Sheffield evaluation. The follow-up contained 4 schools from the original sample and was conducted one year after the second survey (the last survey) of the Sheffield students. Of the original primary schools, 11 head teachers agreed to interviews, and four of the schools subsequently took part in a third survey. The surveys were done on a whole-school basis, with 657 children, aged 7-11, participating. Surveys were administered at the same time of the year as those in the earlier Sheffield evaluation. The results for these four schools were compared with their previous results.

Two of the schools showed a decline in students who reported being bullied sometimes or more since the start of the project. One school experienced increases over time in students who reported being bullied, and one school experienced a decline initially but then experienced an increase. Overall, boys experienced a decline in being bullied over time, while girls first experienced a decline and then an increase. By the third survey, girls were significantly more likely to report being bullied than were boys. The results were similar with regard to bullying others, with two schools showing a decline, one school an increase and one school a decrease and then an increase. After two years of intervention, there was little to no change in the percentage of students who told someone at home or at school about bullying. There were also no overall differences between 1992 and 1993 of the anti-bullying effort made by the schools and the effectiveness of that work as perceived by students, although individual schools did show some relationship between effort and effectiveness.

Brief Bulleted Outcomes:

  • Significant decrease in students' self-reported frequency of bullying other students.
  • Positive outcomes on measures of numbers of students and peers not bullying others, frequency of bullying, and students indicating they would not join in bullying.
  • Increases in the percentage of students reporting bullying incidents to adults and adults talking to students who bullied others.
  • Mixed long-term results in sustainment of program effects.

Study 4

Evaluation Methodology

Design: The evaluation was a pre-test post-test design. Schools participating in the program and evaluation were self-selected due to their interest in and commitment to bullying prevention efforts. A total of 1,052 students from four treatment schools (three K-8 schools and one grades 7-8) participated in the evaluation at baseline. Eleven fewer students (1041) participated in the follow-up survey. Baseline surveys were administered before program implementation, and follow-up surveys were administered 18 months post baseline. Supplementally, teachers were asked to complete a qualitative survey on classroom activities. Seventy-four percent completed surveys. Eight team leaders were interviewed.

Sample Characteristics: The sample included 1,052 students from four treatment schools (three K-8 schools and one grades 7-8). All schools were in urban areas with diverse ethnic populations. All students in grade 3-8 in all four schools were surveyed. Eleven fewer students (1,041) completed the follow-up questionnaire due to school mobility, baseline and follow-up samples were not paired. Seventy-eight teachers (74%) completed a Classroom Activities Questionnaire and eight team leaders participated in interviews.

Measures: The 40-item student self-report questionnaire on bullying and victimization was closely based on the original Bully/Victim Questionnaire developed by Olweus (described in primary study). The reference period was two months.

The Classroom Activities Questionnaire asked teachers about implementation of program components in their classrooms. The team leader interviews were based on school leaders' perception of program implementation and success indicated by observed change in attitudes and behaviors among students and staff.

Analyses: For the student bully/victim questionnaire, the differences in frequencies of behaviors between baseline and follow-up were calculated. A z -test was used to determine significant differences in the distribution of responses between baseline and follow-up. Teacher Classroom Activities Questionnaire and school leader interview data were qualitatively analyzed.

Outcomes
Posttest: According to students self-reports, there were significant increases in teacher interventions and teachers talking to bullies about their behavior. There was no difference in the frequency of victims talking to teachers, nor was there change in parents talking to their children. Students reported being less likely to engage in bullying (17% reduction). The number of children reporting that they or their peers had intervened in bullying and those who reported feeling uncomfortable observing bullying did not change. There was a small but significant increase in the percent of children who self-reported they had bullied other students (2% increase over the school term, and 5% increase in the past five days). A smaller percent reported being victimized in the past five days (18% decrease), but there was no change over the school term. Racially motivated bullying increased. More children reported spending recess alone.

Results were mixed. Bullying behaviors increased over time, although students reported they were less likely to join in bullying behaviors. Victimization decreased on one measure, but not another. Teachers, but not parents, were more likely to talk to bullies about their behavior, but neither were more likely to speak to victims about their difficulties.

Brief Bulleted Outcomes:

  • Significant increases in student-reported teacher intervention in bullying and talking to bullies about their behavior.
  • 17% reduction in student likelihood of engaging in bullying.

Study 5

Evaluation Methodology

Design: This program implementation was evaluated using a pre-post design. Of the 100 schools that were invited to participate, 42 (42%) accepted. Third and fourth class (or 3rd and 4th grade) students completed pre-test assessments at the beginning of the school year and posttest assessment at the beginning of their next school year. Only 22 schools were included in the analysis (52% of those involved, 22% of those invited) because of problems in the remaining schools with administration of the questionnaires to the correct class groupings. Only 22 schools had matching pretest (n = 527) and posttest (n = 520) data. A teacher questionnaire was also implemented to 126 teachers at pretest and 83 at posttest (66% retention rate).

Sample Characteristics: This report provides no specific information on the composition of the sample.

Measures: The student questionnaire was a modified version of the Olweus Bully/Victim Questionnaire. The teacher questionnaire assessed teacher's awareness of, and response to, bullying.

Analysis: Results were generated through chi-square tests with post-hoc analyses implemented to assess the effect of age on program effects.

Outcomes
Baseline Equivalence and Differential Equivalence: There were no differences between student reports at pre- or post-test based on age. The two grades of students were similar on their experiences being bullied and their experiences bullying others. The pre- and post-test teacher samples were also closely matched on gender, age, experience teaching, and employment positions held.

Posttest:
Student Reports: There were significant reductions in self-reported victimization. The percentage of students who were bullied once a week or more (7.2%) significantly declined (to 3.6%), as did the percentage of those who reported being bullied in the past five days. There was only a marginally significant decline in self-reported incidents of bullying in the past school term, but those who reported bullying others once a week or more in the past school term significantly declined from pre- to posttest, as did self-reported bullying in the past five days. The program had no effect on the reporting of bullying or on students' perceptions of the willingness of teachers or peers to try to stop bullying. Students were, however, significantly more likely to report that they would try to stop bullying if they saw it occurring. Finally, significantly fewer students reported at posttest that if they saw a student being bullied, they would do nothing.

Teacher Reports: Tests of significance were not performed or were not reported in this publication on the teacher-report data. The majority of teachers felt the school was a safe place for young people who find it hard to defend themselves and claimed that they were interested in stopping bullying, "always" attempted to stop bulling (96% at pretest, 97.6% at posttest) - or usually did, and believed that bullying is an issue for all staff to handle. Support for the notion that more rigorous monitoring of school bullying was needed dropped from 67.5% to 51.8% at posttest.

Brief Bulleted Outcomes:

  • Significant reductions in students reporting having been bullied in the past week or past five days.
  • Significant reductions in student self-reported bullying in the past week or past five days.
  • Significant increases in student intentions to try to stop bullying if they saw it occurring, as well as significant reductions in students reporting that they would do nothing if witness to bullying.

Limitations: Results are incredibly limited in generalizability because of the weak methodological design, which included no comparison or control group, high levels of attrition. While there were largely positive and significant results, this study is greatly limited by the pre-posttest design, which lacks scientific methodological rigor.

Study 6

Evaluation Methodology
This evaluation reports on program implementation in 37 Oslo schools who started the program in 2001 or 2002. There were 24 primary schools, 6 lower secondary schools, and 7 combined schools (grades 4 through 10) with a total of about 8,000 students. All schools carried out assessments at pre-test and one year later (after 8 months of the program) and about half of them completed a third assessment, two years after the first (after 20 months of the program). Outcomes included being bullied, defined as being bullying two or three times a month or more, and bullying other students, defined as bullying others two or three times a month or more.

Outcomes
Posttest
: Before starting the program, the general prevalence of being bullied was 14% for primary schools and 7% for lower secondary schools. The general prevalence of bullying another student was 6-7% at both primary and lower secondary schools. Eight months after the program began, data from the primary schools and combined schools were combined due to similarity. For these schools, there were significant reductions of 30-40% in the prevalence of being bullied and of 30-45% in the prevalence of bullying others. For lower secondary schools, there were no changes for being bullied and only a 5% reduction in bullying others. Twenty months after program implementation began, reductions continued in all schools, with 30-35% reductions (from baseline) in bullying others in lower secondary schools. In primary schools, there were 60% reductions from baseline in bullying others.

Additional analyses indicate that results for boys and girls were quite similar. The positive results also applied to students involved in bullying that took place once a week or more, students involved in relational bullying (exclusion, ostracism, social isolation), and for bullying in the form of sexual harassment.

Brief Bulleted Outcomes:

  • Significant reductions (between 30-40%) in bullying victimization among primary and combined school students at posttest.
  • Significant reductions (between 30-45%) in bullying perpetration among primary and combined school students at posttest.
  • At long-term follow-up (20 months), program effects on bullying perpetration were sustained in primary schools (60% reductions from baseline).
  • At long-term follow-up (20 months), reductions (between 30-35%) in bullying perpetration among lower secondary schools.

Limitations: The results of this study were greatly limited by the weak methodological design in which no control or comparison group is utilized.

Study 7

Evaluation Methodology

Design: This publication reports on a quasi-experimental evaluation of program implementation in Seattle middle schools (grades 6 through 8) in the 2003-2004 academic year. Ten schools were followed after the state of Washington mandated that middle schools implement anti-bullying policies. Schools were given the freedom to implement the program or policy of their choice. Seven schools chose to implement the OBPP, while three utilized less formal activities. Students completed assessments in the spring of 2003 (semester before program implementation) and again in the spring of 2005 (one year after program implementation completed). There were 4,959 intervention students and 1,559 comparison students.

Sample Characteristics: 51% of intervention youth and 50% of comparison youth were female. The intervention group was 12% Black, 7% Hispanic/Latino, 24% Asian, 40% White, 2% Native American, and 9% other, while the comparison group was 28% Black, 7% Hispanic, 25% Asian, 23% White, 1% Native American, and 10% other. Gradewise, 34% and 37% of treatment and comparison youth, respectively, were in the 6th grade, while 33% each were in 7th grade and 32% and 29% respectively were in 8th grade. In each group, 44-47% made mostly As, 47-48% made mostly Bs/Cs, and 6-8% made mostly D/Fs.

Measures: Questions from the Revised Olweus Bully/Victim Questionnaire were used to measure relational and physical bullying victimization (4 items), attitudes and feelings towards bullying (1 item), perceptions of others' readiness to intervene (2 items), perceptions of safety of school (1 item), perceptions of school supports (7 items), and school engagement (1 item).

Analysis: Student-level data were aggregated by school, and school-level data were used in analysis. Poisson regression was performed, controlling for several school-level variables, including school size, percent of students eligible for free/reduced lunch, percent of students meeting state standards for a reading achievement test. Analyses also controlled for baseline frequencies of victimization, age, gender, and ethnicity.

Outcomes
Baseline Equivalence and Differential Attrition: Groups were equivalent at baseline on gender, grade level, achievement, and school-level characteristics. The comparison group had significantly more Black students than intervention schools, while intervention schools had significantly more White students than comparison schools. Baseline frequencies of both relational and physical victimization were lower for the intervention group than the comparison group, but it is not specifically stated whether this difference was statistically significant. There were significant differences between groups, however, on pro-victim attitudes and desire to help others at baseline.

Because interview assessments did not contain personal information, it is not known if the students who completed baseline also completed follow-up assessments. Retention rates, in terms of number of assessments completed at baseline versus those completed at follow-up, varied depending on outcome measure (physical victimization, relational victimization, and attitudes). For youth in intervention schools, these retention rates ranged between 89.1% and 97.7%, while they ranged from 92.9% to over 100% for comparison youth.

Posttest: At one-year posttest (though one of the intervention schools continued program implementation through the academic year after post-test), there were no program effects on relational or physical victimization. When stratified by race, however, White students in intervention schools reported significantly less relational and physical victimization, compared to White students in comparison schools. There were no effects for youths of other races and no effects were seen when examined by gender or grade level. There were also no overall program effects on student attitudes/feelings towards intervening in bullying situations, school engagement, perceptions of school supports, or perceptions of school safety. Sixth grade intervention youth, however, were significantly more likely to feel sorry for bullied students and want to help them, compared to 6th grade youth in comparison schools. Finally, while intervention youth overall were significantly more likely to perceive other students as actively intervening in bullying incidents than comparison youth, there were no program effects on student perceptions of teachers' or other adults' readiness to intervene.

Brief Bulleted Outcomes:

  • Significant program effects by race: White intervention students reported significantly less relational and physical victimization, compared to comparison students.
  • Significant program effects on secondary measures by grade: 6th grade intervention students reported being significantly more likely to feel sorry for bullied students and want to help them, compared to comparison students in 6th grade.
  • Significant program effects on perceptions of other students actively intervening in bullying incidents, compared to comparison youth reports.

Limitations: The evidence of effect for this program, as used in a multi-ethnic, urban area in the United States, is problematized primarily by the lack of consistent effects throughout the entire sample. There were no overall effects. There were some effects, across ethnicity and age. Additionally, there was no measurement of bullying behaviors.

Study 8

Evaluation Methodology

Design: The study utilized a quasi-experimental design with time-lagged contrasts between age-equivalent groups. Students who received one year of the program were compared with children of the same grade level who did not receive the program. A cohort of 158 7th and 8th grade students in one small Catholic middle school in the northeastern U.S. served as the pre-OBPP baseline group, and a second cohort of 112 students in 7th and 8th grades served as the post OBPP group. Intervention began in February, 2007. Baseline assessments were administered in February 2007 to the cohort of 158 students and 17 teachers. The Time 2 assessment occurred in March, 2008 and was administered to the cohort of 112 students and 10 teachers. The baseline cohort included 59 7th graders and 99 8th graders. The post assessment included 49 7th graders and 62 8th graders (these 8th graders included 7th graders from the pre-OBPP baseline cohort).

Sample Characteristics: The pretest cohort included 45.6% female, 38% Black/Multiracial, 7.6% Hispanic, 51% Caucasian, and 3.4% Native American/Asian. The posttest cohort included 44.6% female, 48% Black/Multiracial, 10.5% Hispanic, 35.5% Caucasian, and 5.7% Native American/Asian.

Measures: The Revised Olweus Bully/Victim Questionnaire and the Teacher Questionnaire were used. Reliability and validity of the Olweus Bully/Victim Questionnaire has been previously validated. The Teacher Questionnaire included 29 items that addressed perceptions on prevalence of bullying, locations of bullying, types of bullying, students' reports of bullying, and teachers' and administrations' response to bullying. To be classified as a bully or a victim of bullying, one had to report having bullied or been bullied "2 or 3 times a month" to "several times per week."

Analysis: Pearson chi-square test for independence was used to evaluate statistical significance for items on the Olweus Bully/Victim Questionnaire. A phi coefficient was used to measure the effect size for a chi-square test for independence. Based on the small teacher sample, a Fisher's Exact Test was used to evaluate statistical significance on the teacher interview items.

Outcomes
Posttest: Results were quite mixed on the Olweus Bully/Victim Questionnaire for females. There was a 34.4% decrease in reports of being excluded and 31.1% decrease in reports of being bullied among 7th grade girls. However, contrary to expectations, 8th grade girls were more likely to be physically bullied (20% increase), at a higher frequency (25% increase), and taken part in bullying (35.6% increase). Indirect verbal bullying did decrease among 8th grade girls (35% decrease). There were no significant findings on these variables for 7th or 8th grade males, and the article suggests some negative trends on certain prevalence measures.

Teacher reports indicated significant and positive improvements in their ability to identify, manage, and report bullying incidents.

Brief Bulleted Outcomes

  • Significant outcomes for females only, and by grade, where 7th grade girls reported decreases in being excluded and bullied (34.4% and 31.1%, respectively), and 8th grade girls reported a 35% decrease in indirect verbal bullying.

Limitations
This evaluation occurred in one school only, and used time-lagged, age-equivalent contrasts. There were relatively few positive outcomes and those were only for 7th grade girls. Most of the outcomes were in the wrong direction for 8th grade girls, and there were no significant outcomes for boys, also with some negative outcomes. This study can lend no evidence in support of the OBPP.

Study 9

Evaluation Methodology
Design: Through the Pennsylvania Department of Education (PA CARES) and Windber Research Institute (HALT!), the OBPP was provided to 100,000 children in more than 70 school buildings in grades K-12. District-wide implementation became a part of HALT!, and received intensive support and onsite consultations, up to 8 hours per week per public school district as needed (3-7 buildings per district). Districts that chose to implement at the building-level became a part of PA CARES, receiving up to 12 hours of certified trainer support during the first year of implementation.

An age-cohorts design was used to analyze program effects. Data from two equivalent age cohorts of students were compared at two or more points in time. There were a total of 107 schools and 54,128 student responses across both 2007 and 2008. PA CARES conducted their baseline assessment in 2008. (It is assumed that a Time 2 assessment occurred in 2009, but the time period is not clear from the report.) HALT! conducted assessments in 2007 (pre) and 2008 (Time 2) and 2009 (Time 3). The HALT! 2007 schools only included elementary and high schools (no middle schools), although middle schools were included in HALT! 2008 surveys.

Measures: The Olweus Bullying Questionnaire was administered to students in grades 3 to 12 prior to the start of the program and at the end of Year 1 and Year 2. The Teacher Questionnaire was also administered to teachers in HALT! schools.

Analysis: A measure of relative change was used to calculate the difference in percentages between baseline assessment (2007) and the assessment after one and two years of the program (2008 and 2009).

Outcomes
Posttest: Overall, after one to two years of program implementation, and across age groups and cohorts, intervention students reported a decrease in bullying others. However, self-reports of being bullied showed mixed findings with some age groups and cohorts showing decreases and others showing no positive program effects. Positive changes were observed across most age groups and cohorts in students' perceptions that adults in the school were actively working to address bullying. Across most groups, increases were observed in the percentage of students who indicated they would try to help a bullied student, and decreases in the percentage of students who would passively stand by or join in the bullying.

Brief Bulleted Outcomes:

  • Decreases in reports of students bullying others.
  • Increases in the perceptions of students who felt that adults in the school were actively trying to address bullying.
  • Increases in the percentage of students who indicated they would try to help a bullied student.
  • Decreases in the percentage of students who felt they would be a passive bystander during a bullying incident.

Limitations: The sample was large, however, it appears that student assessments only came from approximately half of the eligible student body. There is an absence of Time 1 middle schools from HALT!, but middle schools are included at Time 2. PA CARE schools were not assessed at the same baseline period as the HALT! schools (in 2007), but rather PA CARE schools first assessment was in 2008. There is no description of the baseline equivalence of the cohorts being compared over time.

Study 10

Evaluation Methodology

Design: This quasi-experimental study examined 194 elementary, middle, and high schools in Pennsylvania, all of which implemented the program. One cohort of schools completed the baseline assessment and began the program in 2008; another cohort completed the baseline assessment and began the program in 2009. With the program lasting at least one and one-half school years, the posttest assessments followed two years after baseline assessments. The authors refer to Time 1 and Time 3 assessments.

Based on the selection cohort design, the study did not follow students over time but compared the postintervention outcomes for each grade to the preintervention outcomes for the same grade two years earlier. For example, postintervention 6th graders at Time 3 (who were preintervention 4th graders at Time 1) were compared to preintervention 6th graders at Time 1. The design thus compared different cohorts but the same grades.

A total of 70,531 students completed baseline assessments (Time 1), and 63,843 students completed posttest assessments (Time 3). There is no other information on participation rates or the population of students in the schools from which the analysis sample was obtained.

Measures: Subjects completed the anonymous 40-question Olweus Bullying Questionnaire on "students' self-reports of bullying others, being bullied, their own actions when they witness bullying, their attitudes about bullying, and their perceptions of others' actions (e.g., other students and teachers) to address bullying." With most measures dichotomized to indicate bullying 2-3 times a month or more, the questionnaire produced 14 outcomes for analysis. Previous studies showed that the items have high reliability, construct validity, and criterion validity.

Sample Characteristics: At Time 1, the sample included 48.9% girls and 50.5% boys. Students were distributed across grades as follows: grade 3 (11.0%), grade 4 (11.4%), grade 5 (12.2%), grade 6 (15.7%), grade 7 (16.0%), grade 8 (15.9%), grade 9 (5.2%), grade 10 (4.4%), grade 11 (4.1%), and grade 12 (2.3%). The race/ethnicity of the sample was 58.6% white, 4.6% Black/African American, 3.8% Hispanic or Latino, 7.3% other race/ethnicity, 8.6% multi-racial, and 17.2% missing.

Analysis: The analysis combined the schools entering the program in 2008 with the schools entering in 2009. It tested for changes due to the intervention by examining differences over two years between two different cohorts that were in the same grade at preintervention (Time 1) and postintervention (Time 3).

First, the descriptive analysis reported mean probabilities at Time 1 and Time 3 (for dichotomous outcomes. From the probabilities, it calculated the absolute change (Time 1 - Time 3), the relative change (Time 1 - Time 3 / Time 1), and odds ratios (Time 1 odds / Time 3 odds). For continuous outcomes, the analysis used mean values at Time 1 and Time 3.

Second, the inferential analysis used multilevel logistic regression and linear regression to test for the significance of time and the time interactions. The multilevel models adjust for clustering by school.

To supplement the analysis of program effects, an additional analysis checked for history effects in the absence of the program. It compared preintervention outcomes in 2008 to preintervention outcomes of students in the same grade in 2009. If these 2008 and 2009 outcomes differ little, it suggests that changes over time in the absence of the program have limited importance.

Outcomes

Baseline Equivalence and Differential Attrition: With the cohort-based design, the study cannot perform typical tests of baseline equivalence. And with the anonymous questionnaire, the study cannot follow attrition of individual subjects from pretest to posttest. The numbers show 9.5% fewer respondents at Time 3, but comparisons of the gender, race, and ethnic makeup of the Time 1 and Time 3 samples were not provided.

Posttest: All 14 outcomes showed statistically significant improvement from Time 1 to Time 3. With more than 60,000 subjects at each time point, statistical significance provides only a minimal standard. Effects sizes for the improvement over time are all small (though applicable to a large population). The largest odds ratio equals 1.54. Using the formula from Lipsey and Wilson (d = ln(OR)/1.81), this odds ratio is equivalent to .23. The smallest odds ratio is 1.18 (d = .09) and the mean odds ratio is 1.32 (d = .15). The two effect sizes for continuous measures are .09 and .11.

Tests for the interaction of time and student sociodemographic characteristics found many significant results. Despite some differences in program effects, however, nearly all the cohort, gender, race/ethnicity, or grade groups showed significant improvement over time.

As a check on the cohort design, the authors say the following:

To assess possible history effects (i.e., whether observed reductions might be attributed to historical effects and not to the program itself), baseline measurements were compared on key variables at 2008 and 2009. Using Satterhwaite approximation (to correct for unequal sample sizes within schools), the Bonerroni adjustment for multiple comparisons, and a robust estimation (which is recommended when distributional assumptions are not fully met), only one (being socially excluded, adjusted significance = .003) was statistically significant, suggesting that there were no significant improvements in scores on key bullying variables in the absence of the intervention.

Brief Bullets

  • The intervention produced small but significant improvements for 14 bullying outcomes, while over-time comparisons across cohorts before the intervention showed only one significant difference.
  • The intervention affected most cohort, gender, race/ethnic, and grade groups, although they were often stronger for some subgroups than others.

Limitations

  • Description of the sample offered no information on the size of the student population from which the analysis sample was obtained.
  • No figures on gender, race, and ethnic differences between the Time 1 and Time 3 samples were provided.

Study 11

Evaluation Methodology

Design: The study was conducted in grades 7 to 9 in six Oslo schools in 2001, and the classes were followed up to grade 10. There were four treatment and two control schools. Three cohorts were followed: those starting grade 7 in 1999 were followed from grade 9 through 10; those starting grade 7 in 2000 were followed from grade 8 through 10; and those starting grade 7 in 2001 were followed from then through grade 10. The design thus included two, three or four interviews with the individual subjects, depending on their grade level at baseline. Data collections occurred at the turn of each year. The schools were not chosen randomly, but were stratified on the basis of geographical variation in alcohol drinking among young people in the city. There was no discussion of student turnover. Response rates ranged from 86-90% over the four years.

Sample Characteristics: No information was provided on the sample or differences between the treatment and control schools.

Measures: There were questions on the use and frequency of tobacco, and alcohol over the preceding 12 months (yes/no, number of times), whether the student had been intoxicated in the past 12 months, and use of illegal substances in the past 12 months (cannabis, amphetamine, ecstasy, GHB, LSD, cocaine, heroin).

Analysis: Logistic regression for binary measures and linear regression on aggregated outcomes (proportions in grades) were reported. Multi-level random models were applied for each outcome. In the multi-level models, the measurements for individuals over grades were designated at the lowest level, cohorts at the second level, and schools at the third level.

Outcomes

Posttest: In logistic models, 3 of 9 measures were significant. Control schools had a higher increase in cannabis use and drunkenness (6 times or more and 11 times or more) than OBPP schools. Smoking was marginally significant, with higher increase in control schools. There were no significant differences using aggregated models (fixed and random).

Brief Bulleted Outcomes:

  • Significant differences in levels of cannabis use, smoking, and drunkenness between groups, with levels much higher among control group.

Limitations

  • There were a few significant outcomes on substance use measures favoring the intervention schools, but most effects were non-significant. However, there is not etiological reason for a program designed to prevent bullying behaviors to also prevent or reduce substance use. With regard to study design, there was no information provided on the sample characteristics, baseline equivalency, or attrition.

Study 12

Evaluation Methodology
The study was quasi-experimental with data from six secondary schools (three experimental and three control) in the Federal Territory of Kuala Lumpur. In both conditions there was an all boys' school, and all girls' school, and a co-educational school. A bullying questionnaire with 24 items was used to assess physical, verbal, and relational bullying. The Cronbach test for reliability showed acceptable levels.

The report is extremely brief, with no information provided on sample selection or characteristics, baseline equivalency, attrition, or type of analyses. It appears that the analysis involved examining the trend from pretest to posttest within both conditions.

Outcomes
Posttest: Contrary to expectation, there was an upward trend in the three types of bullying (physical, verbal and relational) in the experimental schools as a whole. However, in the all girls' school, there were significant downward trends for the three types of bullying after one year. The trend was upward at the boys' and co-educational schools.

Being a victim of bullying was on a downward trend at the all girls' school on all types of bullying victimization (physical, verbal, relational, signal, and extortion). Victimization was higher at the boys' school, and there were mixed findings at the co-educational school.

Brief Bulleted Outcomes:

  • Significant decreases in physical, verbal, and relational bullying among treatment participants at an all-girls school.

Limitations: The study has a weak design with no information on how the schools were selected, baseline equivalency, attrition, analysis, etc. The program only worked in the all girls' school. It is unclear which elements of the program were delivered. The brief article mentions classroom intervention programs, but it is not clear what they were comprised of. The classroom intervention showed more positive effects, but they have not been reported in this writeup because of the ambiguity of the intervention.

Study 13

Evaluation Methodology

Design:

Recruitment: Participants were drawn from schools that were involved in a wide-scale effort to implement the intervention in various elementary, middle, and high schools in 49 counties in southern and central Pennsylvania. Of the 230 schools who implemented the intervention, 210 were included in the evaluation based on having completed the assessment at baseline and two years later. The initial sample included 70,998 students in grades 3-11 from 210 schools. These students represented three different cohorts that began implementing the program in 2008, 2009, and 2010.

Assignment:

The study employed an extended age cohort design, in which posttest results for one cohort are compared to pretest results for the previous cohort when they are in the same grade and same school. For example, to evaluate the effect of the program after one year, students in grade 7 at baseline and grade 8 at posttest were compared with students from the same school who were in grade 6 at baseline and grade 7 at posttest. This design controls for "possible maturational or age-related differences between the comparison groups."

Attrition:

Schools were asked to complete assessments at baseline and annually. In Limber et al. (2018), two sets of analyses were reported: one following 210 schools (n=70,998 students) across two years, and another following a subgroup of 95 schools (n=31,620 students) that completed the assessment across three years. In the two-year sample, of the 70,998 students who completed the baseline assessment, 67,734 completed the Time 2 assessment (i.e., two years after baseline), resulting in an overall attrition rate of 4.6% from baseline to the two-year follow-up. In the three-year subgroup, of the 31,620 students who completed the baseline assessment, 29,814 completed the Time 3 assessment, resulting in an attrition rate of 5.7% for this subgroup.

Sample:

The full sample was 49% female, 50.5% male, and 0.4% unknown. Approximately 12% of the sample was in third grade, 12.1% was in fourth grade, 12.9% was in fifth grade, 16.1% was in sixth grade, 16.9% was in seventh grade, 16.8% was in eighth grade, 4.8% was in ninth grade, 4% was in tenth grade, and 4.3% was in eleventh grade. Sixty two percent identified as White, 5% as Black, 4.5% as Hispanic, 11.5% as Other, and 17% did not report their race. For the subgroup of students whose schools administered the Time 3 (i.e., three years after baseline) assessment, 49.2% were female, 50.4% were male, and .4% did not report their gender. Approximately 14% of the subgroup sample was in third grade, 13.9% was in fourth grade, 14.1% was in fifth grade, 14.2% was in sixth grade, 13.1% was in seventh grade, 13.2% was in eighth grade, 6.3% was in ninth grade, 4.9% was in tenth grade, and 6.1% was in eleventh grade. Sixty one percent of the subgroup sample identified as White, 4.5% as Black, 3.2% as Hispanic, 11.2% as Other, and 20.1% did not report their race.

Measures:

Participants reported on being bullied, bullying others, reactions to bullying, and perceptions of teachers' actions to address bullying using the Olweus Bullying Scale, which was previously self-developed by the researchers and has been used and validated in their prior work. Four outcomes were computed for analysis for the full sample: bullying and victimization (being bullied) when examining the percentage of students who reported being bullied or bullying others two to three times per week or more, and frequency with which students experienced or engaged in nine different types of bullying. Five outcomes were computed for analysis for the subgroup sample: frequency with which students experienced or engaged in nine different types of bullying, empathy for a bullied peer, willingness to join in bullying a disliked peer, and perceptions of class teachers' actions to address bulling. Subscale reliabilities in the current sample ranged from α = .85 (frequency with which students experienced nine different types of bullying two years after baseline) to .89 (frequency with which students engaged in nine different types of bullying three years after baseline).

Analysis:

All program effects were tested separately within grades or clusters of grades (3-5, 6-8, and 9-11, roughly corresponding to elementary, middle, and high school levels). The researchers used multilevel modeling to test for intervention effects while accounting for clustering of students within schools. Program effects were based on school-aggregated outcome variables. Program year (e.g., baseline, one year after program start, etc.) was treated as a within-school treatment indicator. Outcome variables were treated as continuous, with the exception of the two dichotomous questions regarding being bullied/bullying others "2-3 times a month or more (coded 1) versus "less than 2-3 times a month" (coded 0), which were analyzed with multilevel logistic regression using a logit link. The MLR estimator was used in all multilevel analyses except for analyses of possible interaction effects between program and sex with categorical outcome variables. In these cases, the Bayes estimator was used because it is less computationally demanding than MLR. When a significant program x sex interaction effect was found, analyses were rerun on boys and girls separately using the MLR estimator.

Intent-to-Treat: Twelfth grade students were omitted from the analyses "because their numbers were much lower than those in other high school grades (approximately 50% of the numbers of students in grades 9-11)," (pg. 60). In the full sample, complete baseline and posttest data were available for 210 of the initial 230 schools. The remaining 20 schools (n=5980 students, or 7.8% of the total) were dropped. In the subsample of schools with assessments across four consecutive years, complete baseline and follow-up data were available for 95 of the initial 116 schools that had taken the survey up to four consecutive years. The remaining 21 schools (n=8001 students, or 20.2% of the total) were dropped.

Outcomes

Implementation Fidelity:

Successful standard implementation of the intervention was determined by the minimum criterion that a given school had conducted the Olweus Bullying Questionnaire at baseline and two years later, as the surveys "are important components of the program and thereby provide a rough indication that the program had been introduced in the school in a standardized way." In addition, students' perceptions of the degree to which their class teacher actively addressed bullying were assessed. Among all students, there were significant increases in students' perceptions that their primary teacher had increased his or her efforts to address bullying between baseline and Time 1, between baseline and Time 2, and between baseline and Time 3. In addition, among students in grades 3-5, there were significant increases in students' perceptions that their primary teacher had increased his or her efforts to address bullying between Time 1 and Time 3 and between Time 2 and Time 3.

Baseline Equivalence:

No tests of baseline equivalence were reported.

Differential Attrition:

Tests of differential attrition were not reported at the student level. In both the two-year and three-year samples, schools that were dropped for missing data had "somewhat smaller" levels of bullying and being bullied at baseline relative to the remaining schools but were similar on ethnicity and gender. Statistical tests for these differences were not reported.

Posttest:

The program continues indefinitely so no true posttest assessments are available. In the two-year sample, two years after baseline, all grades showed reductions in victimization (when examining the percentage of students who reported being bullied two to three times per month or more), and these changes over time were significant for all but three grades (8th, 10th, and 11thgrades). All but one grade (11thgrade) showed reductions in the frequency with which students experienced nine different types of bullying. Further, all grades showed significant reductions in the percentage of students who reported bullying others two to three times per month or more. All but one grade (3rdgrade) showed reductions in the frequency with which students engaged in nine different types of bullying.

Long-Term:

In the three-year subgroup, among students in grades 3-5 and 6-8, there were significant reductions in bullying and being bullied across the three years. These effects were significant at the first posttest assessment and maintained through the second posttest assessment. Among students in grades 9-11, there were significant reductions in bullying and being bullied across the three years. These effects were significant between baseline and Time 3, between Time 1 and Time 3, and between Time 2 and Time 3. For risk and protective factors, among students in grades 3-5 and 6-8, there were increases in expressions of empathy for a bullied peer across the three years, which were significant at all timepoints. Among students in grades 9-11, there were significant increases in expressions of empathy for a bullied peer between baseline and Time 3, Time 1 and Time 3, and Time 2 and Time 3. For students in grades 3-5, 6-8, and 9-11, there were reductions in students' willingness to join in bullying a disliked peer across the three years, which were significant at all timepoints.