Author: Tom

Welcome to the Blueprints Bulletin

Adaptation 

We at Blueprints are frequently asked to provide guidance on maintaining fidelity in adaptations of evidence-based interventions. The Blueprints registry currently lists 106 preventive interventions that have been evaluated using rigorous scientific standards and demonstrated to improve youth behavioral outcomes.  
 
Implementing Blueprints-certified interventions with fidelity (or as intended) is important to yield these positive outcomes, as poor implementation fidelity can compromise program mechanisms and undermine their effectiveness.
 
Adhering to fidelity guidelines, however, is challenging when interventions are adapted to the realities of the environment. Some adjustments or adaptations are necessary when interventions are implemented in natural settings.
 
Fidelity and adaptation are often at odds; simultaneously adhering to fidelity guidelines while adapting to realities of the implementation context is a delicate balance. So how do we specify that balance between flexibility and fidelity? This is a topic of much debate.

Advantages and Disadvantages of Adaptation
 
Thoughtful and deliberate adaptation to the delivery of a Blueprints-certified intervention to improve its fit in each context can lead to better engagement, acceptability, and outcomes.
 
However, modifications that remove key elements – or core components – of an intervention may be less effective. An evidence-based program’s core components are the essential activities deemed necessary to produce desired outcomes. The core components are the features that define an effective program.
 
Similarly, reactive adaptations can lead to program drift, resulting in lower impact on outcomes and rendering an evidence-based intervention ineffective.
 
Adaptations, especially untested, threaten to weaken evidence-based interventions, undermine results, and thereby erode confidence in scientific claims that Blueprints-certified interventions work.

Research-Informed Adaptation Strategies

What are “best practices” for making adaptations to a Blueprints-certified program?
 
Using a traffic light as an analogy, Balis and colleagues (2021) assign a color for making changes to an evidence-based intervention:

• Tailoring language or pictures (green light/low risk).
• Adding/substituting activities or session sequence (yellow light/medium risk).
• Deleting lessons and decreasing session length (red light/high risk).

The National Institutes of Health developed a series of Practice Tools to understand, plan for, and implement evidence-based interventions into routine health care and public health settings.
 
Included in this series is a workbook, titled Implementation Science at a Glance. The picture below can be found on page 12 of this guide – it offers some concrete guidance on how to use the “traffic light” analogy to balance fidelity and adaptation.

Video Trainings on Best Practices for Adaptation of Evidence-Based Programs
 
Below are a series of pre-recorded workshops developed by the Prevention Research Center (PRC) at Colorado State University (CSU). Click on each title to view the workshop.
 
Learning Workshop I: Orientation to the PRC’s Implementation Toolbox
 
Join CSU’s PRC Research-to-Practice team to learn about their new Implementation Toolbox website. During this interactive tutorial, we will walk you through the website, introduce you to new tools and resources for program implementation and adaptation, and describe different navigation methods.
 
Learning Workshop II: Roadblocks, Potholes, & Solutions for High-Impact EBP Implementation
 
Join CSU’s PRC Research to Practice team as we delve into the topic of maintaining fidelity to evidence-based program (EBP) implementation. This workshop will describe what implementation fidelity is, best practice methods for maintaining fidelity to EBPs, and discuss common implementation barriers and potential solutions for overcoming them. Additionally, available resources through the PRC’s Implementation Toolbox will be described.
 
Learning Workshop III – Adaptations Part I: Best Practices for Adapting EBPs
 
Join the PRC’s Research to Practice Team as we dive into best practices for modifying programs in part one of a two-part Learning Workshop series on program adaptations. In this video you will learn what adaptations are, common causes that lead to them, and best practices for deciding upon and implementing program changes. Specifically, the Traffic Light Model for Adaptations is discussed and considered through a series of activities provided throughout the presentation.
 
Learning Workshop IV – Adaptations Part II: Challenges in Adapting for Culture
 
Join the PRC’s Research to Practice Team as we dive into best practices for culturally adapting evidence-based programs in part two of a two-part Learning Workshop series. In this video, you will learn what cultural adaptations are, important definitions to keep in mind, and best practices to follow as you adapt your programs to meet your local community’s needs and strengths.

Conclusion

Please send us research-informed strategies you have used to balance fidelity with adaptation to a Blueprints-certified program. We can be reached at: blueprints@colorado.edu.
 
And as always, thank you for your continued interest in and support of Blueprints.
 
Sincerely,

Pamela Buckley, PhD
PI, Blueprints Initiative
Pamela.Buckley@Colorado.edu

Karl G. Hill, PhD
Board Chair and Co-PI
Karl.Hill@Colorado.edu

Blueprints for Healthy Youth Development is developed and managed by the University of Colorado Boulder, Institute of Behavioral Science, with current support from Arnold Ventures and former funding from the Annie E. Casey Foundation. Each intervention included in the Blueprints database has been reviewed carefully by an independent advisory panel that looked at research on the intervention’s impact, practical focus and potential for implementation in public systems.

Featured Model Program
Blues Program

Blueprints Certified: 2015

Ages Served: Late Adolescence (15-18) – High School

Program Outcomes: Alcohol, Depression, Illicit Drugs, Marijuana/Cannabis

Goal and Target Population: A school-based group intervention that aims to reduce negative cognition and increase engagement in pleasant activities to prevent the onset and persistence of depression in high school students exhibiting depressive symptoms.

Learn more about Blues Program

Featured Promising Program
RealTeen

Blueprints Certified: 2022

Ages Served: Early Adolescence (12-14) – Middle School

Program Outcomes: Illicit Drugs, Marijuana/Cannabis, Tobacco

Goal and Target Population: An internet-based, gender-specific drug abuse prevention program intended to reduce substance use among early adolescent girls through improving personal, social, and drug refusal skills.

Learn more about RealTeen

© 2023 Blueprints for Healthy Youth Development, Regents of the University of Colorado. All rights reserved. 

Our mailing address is:
University of Colorado Boulder | Institute of Behavioral Science
483 UCB, Boulder, CO 80309

Welcome to the Blueprints Bulletin

Leadership Letter: 

For more than 25 years Blueprints has served as a resource for governmental agencies, foundations, community organizations and practitioners seeking to make informed decisions about their investments in preventive interventions. Since 1996, Blueprints has carefully reviewed the evidence for over 3,000 studies evaluating 1,500 interventions and thus serves as an excellent, internationally recognized resource on what programs have been shown to work in rigorous evaluations across a wide range of policy areas.
 
Often, we are asked: How does Blueprints measure its success?
 
We do this through a range of markers, examples of which are highlighted below.
 
Blueprints’ Electronic Footprint
 
For pure dissemination information, we track google analytics to determine user interactions with the website. On average, Blueprints receives around 17,500 unique pageviews and 9,000 sessions a month, which translates to nearly 600 unique pageviews and 300 sessions per day. We are also active on social media, including:

If you don’t already, please sign up to follow us on any of these social media platforms!

Community Prevention Strategies
 
Another method for measuring success involves documenting the use of Blueprints in community prevention strategies now in wide use. For example, Communities That Care (CTC), is a prevention system designed to reduce levels of adolescent delinquency and substance use through the selection and use of effective preventive interventions tailored to a community’s specific profile of risk and protection. CTC trains community members in how to select and expand their use of tested and effective interventions specifically using the Blueprints website. CTC is currently successfully operational in over 130 communities in the United States, such as in Colorado where the Colorado Department of Public Health and Environment (CDPHE) adopted CTC in 2016 as a statewide prevention model, and CDPHE presently funds 30 CTC communities across the state. Internationally, CTC is operating in dozens of communities around the world (including Germany, Sweden, Denmark, The Netherlands, the United Kingdom, Croatia, Austria, Switzerland, Canada, Mexico, Colombia, Chile, Panama and Australia) to promote healthy development and reduce risk behavior.
 
An additional example is Evidence2Success®, which  is an initiative of the Annie E. Casey Foundation that brings together public-system leaders and community members to understand how children are doing with the help of data; select evidence-based programs, including those from Blueprints to enhance strengths and address needs; and develop financing and action plans to support the ongoing use of those proven programs. The initiative began with a pilot site in Providence, RI in 2012 and has since expanded to Mobile, Alabama; Selma, Alabama; Kearns Township, Salt Lake County, Utah; Memphis, Tennessee; and Miami, Florida. As part of the Evidence2Success framework, public systems and schools commit up front to redirecting a portion of their combined resources for children. These partners work with a team of finance and administration professionals to identify funding sources within their agency budgets and coordinate funding to invest in programs that respond to the outcomes prioritized by the partnership. In recent years, the Foundation has carefully tracked the resources leveraged by sites in support of Blueprints programs and supporting infrastructure. Since 2017, sites have leveraged more than $6 million across child welfare, juvenile justice, school district, public health, social service, city, county, and substance abuse and mental health systems. The addition of related activities, such as clinical services and trauma trainings in Providence, raises the total amount leveraged to $15 million.

References in Evidence-Based Funding Guidelines
 
We track references to Blueprints in guides used to determine grant eligibility or that require evidence to support funding decisions. For example, the Pew Charitable Trusts cited Blueprints by name in a 2022 report titled How Nongovernmental Groups Can Support States in Evidence-Based Policymaking (see page 11). In addition, Blueprints is listed as a reference within the Evidence-Based Practices Resource Center sponsored by the Substance Abuse and Mental Health Services Administration (SAMHSA) housed within the US Department of Health & Human Services, and Blueprints is referenced in several reports prepared for SAMHSA such as:

• Preventing Marijuana Use Among Youth. SAMHSA Publication No. PEP21-06-01-001. Rockville, MD: National Mental Health and Substance Use Policy Laboratory. Substance Abuse and Mental Health Services Administration, 2021.
• Selecting Best-fit Programs and Practices: Guidance for Substance Misuse Prevention Practitioners. Substance Abuse and Mental Health Services Administration, 2018.

Historically, Blueprints has been listed as a resource for locating evidence-based interventions in several handbooks developed to support funding mandates across the United States. Recent examples include:

• South Carolina’s Guide to Approved Uses for Investing Opioid Settlement Funds (2022)
• Montana’s “Guide to Evidence-Based Substance Prevention” (2021)
• Nevada’s “Evidence-Based Practices, Programs, and Policies Manual” (2021)
• Washington state’s “Reporting Guide for Research and Evidence-Based Practices in Children’s Mental Health” (2021) and “Prevention Tools What Works, What Doesn’t” (2019)
• Ohio’s Evidence resource page: Children, youth and families, Supports for multi-system youth (2021)
• West Virginia’s Guide to “Preventing Substance Use Disorder: What Works and What Doesn’t” (2020)
• Nevada’s “Evidence-Based Work Group Policy and Guidelines” (2019)

Informing Standards of Research on Evidence-Based Programs
 
We monitor documents referencing Blueprints standards of evidence, highlighting the design, methods, and analysis issues Blueprints considers when determining the credibility of causal claims of program effectiveness. For example, the United Nations Office on Drugs and Crime and World Health Organization recently suggested that national standards globally enforce a requirement of implementing evidence-based strategies only by utilizing registries such as Blueprints, citing the registry by name (United Nations Office on Drugs and Crime & World Health Organization, 2018, see page 42). Meanwhile, A Guide to Writing High-Quality Evaluation Reports in Child Welfare distributed by the Office of Planning, Research, and Evaluation at the Administration for Children and Families (ACF) and the Urban Institute discussed key design elements for rigorous impact evaluations by analyzing the study rating criteria used at Blueprints (among other registries) to create an invaluable resource for anyone in the planning and design stages of impact evaluation in child welfare.

Media Op-Eds Highlighting Blueprints

We identify when Blueprints is mentioned in national and local media pieces, such as a New York Times op-ed on reducing school violence (Keels, 2018), several op-eds in local and national news media outlets advocating for a prevention strategy in dealing with the opioid surge (Riggs, 2022, Richter & Fishbein, 2022), and a discussion in The Hill of interventions such as those listed on the Blueprints registry that are effectively alleviating economic hardship (Baron, 2022).

Scientific Impact

And finally, we examine success in terms of scientific impact. Blueprints’ founder, Dr. Del Elliott, is a leading advocate for evidence-based prevention. His work has been cited over 27,000 times in scientific literature. In addition, we are aware of 48 different academic papers published since 2015 that discuss and/or reference Blueprints, and these publications have been cited (according to Google analytics) more than 3,500 times. We also maintain a publications page on our website listing academic papers written by Blueprints board and staff highlighting the registry’s work.

Conclusion
 
Launched in 1996, Blueprints recently celebrated its 25^th anniversary. In preparing for next year and future work, we frequently reflect on our purpose and impact.
 
If you are aware of initiatives, funding mandates, frameworks, media pieces, academic papers or any other efforts that cite Blueprints as a resource for guiding decisions about investments in preventive interventions, please send this information to us at blueprints@colorado.edu.
 
And as always, thank you for your continued interest in and support of Blueprints.
 
Happy Holidays!

Sincerely,

Pamela Buckley, PhD
PI, Blueprints Initiative
Pamela.Buckley@Colorado.edu

Karl G. Hill, PhD
Board Chair and Co-PI
Karl.Hill@Colorado.edu

Blueprints for Healthy Youth Development is developed and managed by the University of Colorado Boulder, Institute of Behavioral Science, with current support from Arnold Ventures and former funding from the Annie E. Casey Foundation. Each intervention included in the Blueprints database has been reviewed carefully by an independent advisory panel that looked at research on the intervention’s impact, practical focus and potential for implementation in public systems.

2022 in Review
By the numbers
 
Below is a summary of our year in review. 
 
Interventions Reviewed & Certified in the Past Year
 

Certified in 2022  
Growth Mindset for 9th Graders (Promising)
Added: Jan. 18, 2022
Learn more
 
Transitional Jobs Program – RecycleForce (Promising)
Added: Feb. 25, 2022
Learn more
 
Bounce Back Newham (Promising)
Added: July 27, 2022
Learn more
 
Fostering Healthy Futures for Preteens (Promising)
Added: August 3, 2022
Learn more
 
RealTeen (Promising)
Added: December 7, 2022
Learn more

Featured Model Program
Family Foundations

Blueprints Certified: 2022 (Moved from Promising to Model)

Ages Served: Infant (0-2)

Program Outcomes: Antisocial-aggressive Behavior, Anxiety, Close Relationships with Parents, Depression, Externalizing, Internalizing

Goal and Target Population: A universal prevention program designed to improve birth outcomes, reduce family aggression, enhance child mental and behavioral health, and enhance parent mental and physical health through promoting coparenting quality among couples at the transition to parenthood.

Learn more about Family Foundations

Featured Promising Program
YouthBuild

Blueprints Certified: 2021

Ages Served: Late Adolescence (15-18) – High School, Early Adulthood (19-24)

Program Outcomes: Dropout/High School Graduation, Employment

Goal and Target Population: A full-time, comprehensive program aiming to help young adults between the ages of 16 and 24 who lack a high school diploma and may have experienced other systemic barriers by building skillsets and mindsets that lead to lifelong learning, livelihood, and leadership.

Learn more about YouthBuild

© 2022 Blueprints for Healthy Youth Development, Regents of the University of Colorado. All rights reserved. 

Our mailing address is:
University of Colorado Boulder | Institute of Behavioral Science
483 UCB, Boulder, CO 80309

Welcome to the Blueprints Bulletin

Leadership Letter: 

Several opioid settlements with pharmaceutical companies have been determined with more to come. Many of these resources are being distributed by states with the aim of addressing education, economic, social, and health hardships and creating sustainable innovations to address the opioid epidemic. This month’s newsletter highlights the role of prevention – most notably, scaling evidence-based preventive interventions listed on the Blueprint registry – as one approach embedded within a larger prevention strategy for guiding the effective use of these settlement funds.
 
The Power of Prevention
 
With the current opioid addiction and overdose crisis, many countries around the world have been ramping up treatment and harm-reduction services to mitigate the negative consequences associated with drug use. What is also needed, however, is prevention!
 
Dr. Diana Fishbein of the National Prevention Science Coalition to Improve Lives wrote an op-ed titled “Prevention is our best – and most underrated – weapon against opioids,” emphasizing the critical role for prevention in the battle against the opioid epidemic, which has only been exacerbated by the COVID-19 pandemic.
 
Over the past 30 years, prevention research shows that systematically addressing the root causes of behavioral problems among vulnerable populations and promoting protective and supportive environments will steadily divert trajectories away from substance use disorders, including opioid use disorders, later in life. The same risk and protective factors in child and adolescent families, schools, peer groups and neighborhoods that affect smoking, alcohol use and cannabis use are also predictive of advancing to opioid use.
 
Research on periods of vulnerability during youth development and the kinds of social and environmental factors that increase risks associated with substance use have led to the design and testing of numerous evidence-based prevention interventions that have been shown to reduce risk factors as well as increase protective factors. These evidence-based programs have multiple benefits – with some showing decreased or delayed drug experimentation in adolescence and young adulthood.
 
In this 15-minute talk, prevention scientist Dr. Phillip Graham provides a powerful and compelling case explaining the role of evidence-based individual, family, school, and community-level preventive interventions in the prevention of youth substance use and opioid misuse.
 
Opioid Settlement Funds
 
The opioid settlement funds offer a pressing opportunity to promote dissemination and scaling of effective prevention programs across communities and social systems. Settlement funds could save lives and abate lifelong harms from substance use disorders (including opioids) if they are allocated to the most effective preventive interventions targeting youth, families, and/or schools.
 
For this reason, states and jurisdictions should fund initiatives demonstrated by research to work and not fund programs shown not to work.

Investments in early intervention preventive approaches among youth and families will result in significantly lower long-term rates of substance use disorders. 

Blueprints-Certified Preventive Interventions
 
Primary prevention programs listed on the Blueprints registry are designed to improve child development, support families, and enhance school experiences. These programs are developmentally appropriate and many have been shown to either prevent the initiation of substance use or escalation of use.
 
Several organizations have published guidance on how to spend newly allocated opioid settlement funds on substance use prevention and mental health promotion that specifically reference the Blueprints registry. For example: 

• This opioid policy briefing from the National Prevention Science Coalition recommends the Blueprints registry as part of a Strategy for Preventing Opioid Use Disorders in Communities. The brief reviews current challenges regarding opioid use disorder and provides recommendations and strategies for prevention.

• Blueprints was referenced in a news piece titled: How To Invest Opioid Settlement And Federal Funding To Prevent Substance Use And Promote Youth Mental Health. Specifically, the news article said: “Too often, programs and interventions are not sustained on a long-term basis because of resource limitations and a lack of infrastructure. To ensure sustainability, recipients of funds should use established evidence-informed programs whenever possible and appropriate, as designated by reputable lists such as Blueprints for Healthy Youth Development.”

• John Hopkins Bloomberg School of Public Health released several “Principles for the Use of Funds from the Opioid Litigation,” including to Invest in Youth Prevention using compilations of effective youth primary prevention interventions, such as those listed on the Blueprints registry. 

Blueprints-certified programs can be effectively scaled when integrated into a comprehensive service delivery system based on an assessment of need, delivered at the community level, and supported by a monitoring and evaluation data infrastructure.
 
This previous newsletter highlights examples of effective frameworks for broad dissemination of Blueprints-certified programs.
 
Scaling of evidence-based programs, such as those listed on the Blueprints registry, is also part of a comprehensive national strategy to prevent opioid use disorder launched by the National Prevention Science Coalition to Improve Lives.
 
Conclusion
 
In sum, we believe that effective allocation of state opioid settlement funds toward prevention can and should be used to establish the infrastructure needed for a comprehensive approach to preventing youth substance use and mental health problems.
 
If you are aware of initiatives that use opioid settlement funds for large-scale dissemination of evidence-based preventive interventions listed as Model/Model Plus or Promising on the Blueprints registry, we are interested in profiling your work. Please contact us at blueprints@colorado.edu.
 
And as always, thank you for your continued interest in and support of Blueprints.

Sincerely,

Pamela Buckley, PhD
PI, Blueprints Initiative
Pamela.Buckley@Colorado.edu

Karl G. Hill, PhD
Board Chair and Co-PI
Karl.Hill@Colorado.edu

Blueprints for Healthy Youth Development is developed and managed by the University of Colorado Boulder, Institute of Behavioral Science, with current support from Arnold Ventures and former funding from the Annie E. Casey Foundation. Each intervention included in the Blueprints database has been reviewed carefully by an independent advisory panel that looked at research on the intervention’s impact, practical focus and potential for implementation in public systems.

Blueprints News and Resources 
Opioid Response Resources
 
In lieu of featuring one Model/Model Plus and one Promising program as we have historically done in each newsletter, we are highlighting two Blueprints-certified programs with high-quality evaluation studies demonstrating an impact on reduce of opioid use. These include:
 
Model/Model Plus Programs

Project Towards No Drug Abuse (classroom-based drug prevention program designed for at-risk youth that aims to prevent teen drinking, smoking, marijuana, and other hard drug use).
 
Study 1 – Sussman et al. (1998), Simon et al. (2002)

• Measured cigarettes, alcohol, marijuana, cocaine/crack, hallucinogens, stimulants, inhalants, and other drugs (depressants, PCP, steroids, heroin, etc.).
• Findings showed reductions in hard drug use (including opioids) prevalence rates at 1-year and 4-5-year follow-ups.

Study 2 – Dent et al. (2001)

• Measured cigarettes, alcohol, marijuana, cocaine/crack, hallucinogens, stimulants, inhalants, and other drugs (depressants, PCP, steroids, heroin, etc.).
• Findings showed reductions in hard drug use (including opioids) prevalence rates at 1-year follow-up.

Study 3 – Sussman, Dent & Stacy (2002), Sussman, Dent et al. (2002), Sussman et al. (2003)

• Measured cigarettes, alcohol, marijuana, cocaine/crack, hallucinogens, stimulants, inhalants, and other drugs (depressants, PCP, steroids, heroin, etc.).
• Findings showed reduced problem behavior rates (i.e., hard drug use, alcohol use, tobacco, and marijuana use) at 1-year follow-up, and reduced hard drug use at 2-year follow-up.

Study 4 – Sun et al. (2008)

• Measured cigarettes, alcohol, marijuana, cocaine/crack, hallucinogens, stimulants, inhalants, and other drugs (depressants, PCP, steroids, heroin, etc.).
• Findings showed reductions in hard drug use (including opioids) frequency at 1-year follow-up. 

Promising Programs

PROSPER (delivery system that attempts to foster implementation of evidence-based youth and family interventions, complete with ongoing needs assessments, monitoring of implementation quality and partnership functions, and evaluation of intervention outcomes).
 
Study 1 – Spoth et al. (2007, 2015), Redmond et al. (2009)

• Measured “Illicit Substance Index,” including marijuana, methamphetamines, ecstasy, opioids, and medications prescribed for someone else, as well as alcohol and cigarettes.
• Findings showed lower rates of lifetime use of gateway drugs (alcohol, cigarette, marijuana) and illicit drugs (marijuana, meth, ecstasy, prescription misuse, opioids), and lower rates of marijuana and inhalant use in the past year.

© 2022 Blueprints for Healthy Youth Development, Regents of the University of Colorado. All rights reserved. 

Our mailing address is:
University of Colorado Boulder | Institute of Behavioral Science
483 UCB, Boulder, CO 80309

Welcome to the Blueprints Bulletin

Leadership Letter:

Blueprints is a privately funded nonprofit organization housed in the Prevention Science Program within the Institute of Behavioral Science at the University of Colorado Boulder.
 
We are the longest-standing registry of evidence-based interventions and internationally recognized for our work in the field.
 
In 2018, the United Nations Office on Drugs and Crime and World Health Organization suggested that national standards globally enforce a requirement of implementing evidence-based strategies only by utilizing registries such as Blueprints, citing us by name (United Nations Office on Drugs and Crime and the World Health Organization, 2018, p. 42).
 
Funding History
 
Blueprints began in 1996 as an initiative of the State of Colorado, with initial funding from the Colorado Division of Criminal Justice, the Centers for Disease Control and Prevention, and the Pennsylvania Commission on Crime and Delinquency.
 
Blueprints was later supported (starting in 2010) by the Annie E. Casey Foundation (AECF) to expand the registry’s focus beyond violence prevention to include positive behavioral outcomes.
 
Following AECF, Arnold Ventures has funded Blueprints core operations, which include:

1. Conducting Blueprints reviews (see Blueprints review process).
2. Providing a detailed account on the Blueprints website of each certified intervention (i.e., those that receive a rating of Promising, Model or Model Plus).
3. Support of other communication activities designed to promote the Blueprints registry as a resource for scientifically proven and scalable interventions (e.g., through social media, a quarterly e-newsletter, presentations, etc.).

With support from Arnold Ventures, Blueprints also lists information on all interventions we review and has one page on our website for identifying certified interventions (i.e., those rated as Promising, Model or Model Plus) and a separate page for non-certified interventions to reduce confusion and increase transparency of the Blueprints review process.
 
Continuation Grant
 
Arnold Ventures recently awarded Blueprints a two-year grant (starting July 1, 2022) to support our core operations. In addition, Arnold Ventures is providing funding for Blueprints to:

• Communicate important findings on effective interventions to the practitioner, policy, funder, and research communities by extending and strengthening the Blueprints rating system to incorporate meta-analysis into its thorough and reflective review process, thus assisting the Blueprints Advisory Board in making recommendations for Model and Model Plus interventions that are ready for scale.
• Increase the use of rigorous evidence through partnerships and nonpartisan educational efforts focused on promoting the use of evidence about what works to increase the effectiveness of U.S. social spending and improve people’s lives.

We are deeply grateful to all our funders, past and present, for their investment in Blueprints – demonstrating their commitment to promoting a strong scientific standard of evidence for identifying effective prevention interventions that provide a high probability of success when taken to scale in communities.
 
Part of Arnold Ventures’ mission is to invest in evidence-based solutions that maximize opportunity and minimize injustice. With this continuation grant to Blueprints, Arnold Ventures adds to its portfolio of strategies designed to increase the effectiveness of social spending using rigorous evidence about what works. Learn more about Arnold Ventures’ work in correcting systemic failures through evidence-based solutions.
 
And, as always, thank you to all of you for your continued interest in and support of Blueprints!
 
Sincerely,

Pamela Buckley, PhD
PI, Blueprints Initiative
Pamela.Buckley@Colorado.edu

Karl G. Hill, PhD
Board Chair and Co-PI
Karl.Hill@Colorado.edu

Blueprints for Healthy Youth Development is housed at the University of Colorado Boulder, Institute of Behavioral Science, with current support from Arnold Ventures. 

© 2022 Blueprints for Healthy Youth Development, Regents of the University of Colorado. All rights reserved. 

Our mailing address is:
University of Colorado Boulder | Institute of Behavioral Science
483 UCB, Boulder, CO 80309

Welcome to the Blueprints Bulletin

View this email in your browser

Leadership Letter:

We are devoting this newsletter to acknowledging our Blueprints Advisory Board members, both past and present.
 
Launched in 1996, Blueprints is the longest-standing online clearinghouse of evidence-based interventions. Throughout most of our tenure, our review process has been supported by a distinguished Advisory Board of experts in prevention science content areas, evaluation methodology, intervention design and testing, and statistical methods. Each certified intervention listed on the Blueprints registry has been reviewed carefully by this independent advisory panel that looks at research on the intervention’s impact, practical focus, and potential for implementation in public systems.
 
We want to welcome our new Advisory Board members, who include:

•  Elizabeth Stuart, PhD, Bloomberg Professor of American Health, Associate Dean for Education and Professor in the Department of Mental Health, with joint appointments in the departments of Biostatistics and Health Policy and Management at Johns Hopkins University.
• Elizabeth Tipton, PhD, Associate Professor of Statistics, and Faculty Fellow in the Institute for Policy Research at Northwestern University.

These individuals will be joining an accomplished group of scholars who currently sit on the Blueprints Advisory Board, including:

• Abigail Fagan, PhD, Professor, Department of Sociology, Criminology & Law, University of Florida.
• Frances Gardner, DPhil, Professor of Child and Family Psychology, Fellow of Wolfson College Centre for Evidence-Based Intervention, Centre for Evidence-Based Intervention, Department of Social Policy & Intervention, Oxford University.
• Larry V. Hedges, PhD, Board of Trustees Professor of Statistics and Social Policy and Faculty Fellow, Institute for Policy Research at Northwestern University.
• Karl G. Hill, PhD, Professor, Department of Psychology and Neuroscience and Director of the Prevention Science Program in the Institute of Behavioral Science at the University of Colorado Boulder.
• Velma McBride Murry, PhD, Professor, Departments of Health Policy and Human and Organizational Development, Vanderbilt University Medical Center and Vanderbilt University.
• Patrick H. Tolan, PhD, the Charles S. Robb Professor of Education at the University of Virginia in the School of Education and Human Development and Professor in the Department of Psychiatry and Neurobehavioral Sciences in the School of Medicine.

 We also want to thank members of the Blueprints Advisory Board who recently retired, including:

• Thomas Cook, PhD, Professor Emeritus of Sociology at Northwestern University and Emeritus Fellow at their Institute for Policy Research, where he was formerly the Joan and Sarepta Harrison Chair of Ethics and Justice. 
• Delbert Elliott, PhD, Blueprints founder and Distinguished Professor Emeritus of Sociology, research professor in the Institute of Behavioral Science at the University of Colorado Boulder and founding director of the Center for the Study and Prevention of Violence.
• Denise Gottfredson, PhD, Professor Emeritus at the University of Maryland Department of Criminal Justice and Criminology.
• David Hawkins, PhD, the Endowed Professor of Prevention Emeritus in the School of Social Work at the University of Washington and Founding Director of the Social Development Research Group.

 The Blueprints Review Process
 
The Blueprints website provides information about our review process, which is illustrated below.  
 

 
We thank and acknowledge members of the Blueprints Advisory Board whose collective knowledge, critical thinking and analysis increases the credibility of the ratings and interventions we certify as “Promising,” “Model,” and “Model Plus” on the Blueprints registry.
 
Nominate an Intervention
 
If you would like us to review a study of a prevention program that serves youth (ages 0-25), families and/or communities, please send all evaluation articles/papers to: blueprints@colorado.edu.
 
More information on nominating an intervention for Blueprints review can be found on our website.
 
As always, thank you for your continued interest in and support of Blueprints.
 
Sincerely,

Pamela Buckley, PhD
Principal Investigator
Institute of Behavioral Science
University of Colorado Boulder
Pamela.Buckley@Colorado.edu

Karl G. Hill, PhD
Co-Principal Investigator
Chair of the Blueprints Board
Institute of Behavioral Science
University of Colorado Boulder
Karl.Hill@Colorado.edu

Blueprints for Healthy Youth Development is housed at the University of Colorado Boulder, Institute of Behavioral Science, with current support from Arnold Ventures. 

© 2022 Blueprints for Healthy Youth Development, Regents of the University of Colorado. All rights reserved. 

Our mailing address is:
University of Colorado Boulder | Institute of Behavioral Science
483 UCB, Boulder, CO 80309

Welcome to the Blueprints Bulletin

Leadership Letter: 

In our last newsletter, we explained how we apply our standards of evidence (Steeger et al., 2021) in rating interventions with an evidence base that is strong enough to be recommended by Blueprints for large-scale adoption (please click here for a summary of our review process).
 
Obtaining desired outcomes from scaling up Blueprints-certified interventions, however, requires multiple layers of support, including (for example): (1) adequate and appropriately structured funding; (2) partnerships with stakeholders at every level to ensure necessary support for program implementation and continuing service provision; and (3) community engagement to support local service delivery.
 
Several effective frameworks for broad dissemination have been established to ensure the impact of Blueprints-certified programs on a large scale.
 
The focus of this newsletter is highlighting examples of these efforts, with the goal of providing a “blueprint” for national, state, and/or local initiatives that might want to adopt a similar approach.
 
Statewide Public/Private Initiatives to Implement a Portfolio of Blueprints-Certified Interventions
 
In Colorado (home to the Prevention Science Program at the Institute of Behavioral Science at CU Boulder, which houses Blueprints), examples include:

1. Invest in Kids (IIK), a nonprofit organization that for over two decades has worked alongside Colorado communities to adopt, implement, and successfully scale proven programs that have the greatest long-term impact on young children and families experiencing poverty. These Blueprints programs, which IIK sustains via federal, state, and local grants and investments from private foundations and individual donors, include: 1) Nurse-Family Partnership® (Model), a relationship-based program, delivered in the home, that partners highly-trained registered nurses with first-time mothers and their babies and is designed to improve prenatal and child rearing practices and parent self-sufficiency through the child’s second birthday; 2) The Incredible Years® (Promising), a prevention program designed to increase a child’s success at school and at home by promoting positive parent, teacher, and child relationships; and most recently, 3) Child First® (Promising), a home-based two-generation mental health-focused program which works to heal and protect young children and their families from the devastating effects of chronic stress and trauma.
2. Colorado Equitable Economic Mobility Initiative (CEEMI), a newly formed independent nonprofit  collaborating with public and private partners to create a more equitable economic ecosystem by blending public, private, and philanthropic funds to enroll Coloradans facing poverty in effective employment programs. CEEMI is working to scale up two Blueprints-certified programs: 1) Accelerated Study in Associate Programs (Model Plus), a post-secondary program addressing potential barriers to academic success and promoting credit accumulation and associate degree completion through comprehensive advisement and career and tutoring services provided by dedicated advisers; and 2) Year Up (Promising), a training and internship program that helps young people with limited post-secondary education get high-quality jobs by learning to work with technology, developing employment skills, and obtaining internships. Year Up is also one of the workforce programs backed by strong evidence of effectiveness identified by the National Conference of State Legislatures, and their new Center for Results-Driven Governing, for policymakers to consider as states leverage the American Rescue Plan (ARP) in seeking ways to re-employ workers and build a workforce that will help lead their economic recovery in this post-coronavirus economy.  
3. Safe Communities Safe Schools (SCSS), an initiative within the Center for the Study and Prevention of Violence at the University of Colorado Boulder that follows a research-based model to build a school’s unique capacity to reduce violence by promoting social, emotional, and physical safety. Included in the SCSS Model is support that allows schools to (1) measure and strengthen their readiness to adopt evidence-based interventions, including those listed on the Blueprints registry, (2) build high-functioning multi-disciplinary teams to champion the work, and (3) receive tailored tools, processes, and technical assistance to support high-quality implementation.
4. Communities That Care (CTC), a framework which aims to reduce adolescent substance misuse, violence, and depression by implementing from a menu of strategies particular programs, practices and/or policies—including Blueprints-certified interventions—that have demonstrated effectiveness in reducing risk factors and enhancing protective factors. The Colorado Department of Public Health and Environment (CDPHE) adopted CTC in 2016 as a statewide prevention model. CDPHE currently funds 30 CTC communities across the state.

National Philanthropic Foundation/Community Partnerships Helping Scale Blueprints-Certified Interventions
 
Two examples of efforts to scale evidence-based interventions, including many on the Blueprints registry, funded by national philanthropic organizations include:

1. Evidence2Success®, which  is an initiative of the Annie E. Casey Foundation that brings together public-system leaders and community members to understand how children are doing with the help of data; select evidence-based programs, including those from Blueprints to enhance strengths and address needs; and develop financing and action plans to support the ongoing use of those proven programs. The initiative began with a pilot site in Providence, RI in 2012 and has since expanded to Mobile, Alabama; Selma, Alabama; Kearns Township, Salt Lake County, Utah; Memphis, Tennessee; and Miami, Florida. While scale is sometimes more quickly achieved through a focus on system-wide reform and legislative change, Evidence2Success is built at the intersection of place-based change and system change – with a strong value on system change informed by community input and leadership. As part of the Evidence2Success framework, public systems and schools commit up front to redirecting a portion of their combined resources for children. These partners work with a team of finance and administration professionals to identify funding sources within their agency budgets and coordinate funding to invest in programs that respond to the outcomes prioritized by the partnership. In recent years, the Foundation has carefully tracked the resources leveraged by sites in support of Blueprints programs and supporting infrastructure. Since 2017, sites have leveraged more than $6 million across child welfare, juvenile justice, school district, public health, social service, city, county, and substance abuse and mental health systems. The addition of related activities, such as clinical services and trauma trainings in Providence, raises the total amount leveraged to $15 million.
2. Arnold Ventures’ Moving the Needle initiative, which seeks to spur expanded implementation of programs with credible evidence of meaningful positive effects in order to make significant headway against U.S. social problems. Specifically, the initiative is designed to encourage state or local jurisdictions, or other entities, to: (1) Adopt social programs shown in well-conducted randomized controlled trials (RCTs) to produce large, sustained effects on important life outcomes (see attachment 1 in the RFP for the list of eligible programs; many are listed on the Blueprints registry); (2) Implement these programs on a sizable scale with close adherence to their key features; and (3) Determine, through a replication RCT, whether the large effects found in prior research are successfully reproduced so as to move the needle on important social problems. Arnold Ventures (AV) has provided more than $38 million in funding over the past five years to help scale up and conduct replication trials of these interventions. Click here to listen to a 10-minute interview with David Anderson, VP of AV’s Evidence-Based Policy Initiative, for insights about building credible evidence in social policy.

Reallocation of Criminal Justice Dollars to Prevention
 
The Virginia Department of Juvenile Justice (Virginia DJJ) built a more robust continuum of services, with a focus on implementing evidence-based and evidence-informed programs and services statewide, thereby reducing the overuse of juvenile correctional centers and developing alternatives to incarceration. The Blueprints registry is identified within the Virginia DJJ’s guidance document as a resource in selecting effective prevention programs ready for large-scale adoption. As a result of Virginia DJJ’s commitment to evidence-based programs and high-quality implementation, youth and families who are involved with the juvenile justice system anywhere in the Commonwealth now have access to one or more Model/Model Plus programs. Click here for a summary report on Virginia DJJ’s Transformation over the past five years.
 
Conclusion
 
Attempts to scale inconclusive, ineffective, or harmful programs are a waste of time, money, and opportunity. As such, we advocate for identifying and implementing interventions that are known to work, that can be implemented at scale with fidelity, and that are cost effective. Blueprints provides a trusted source of accessible information on experimentally proven interventions recommended for large-scale implementation that meet a high evidentiary standard based upon a rigorous systematic review of all the research evidence. This newsletter provides just a few models for scaling social programs that produce important improvements in people’s lives.
 
If you are aware of additional models or frameworks for large-scale dissemination of evidence-based interventions, including those on the Blueprints registry, we would be interested in profiling your work. Please contact us at blueprints@colorado.edu.
 
And as always, thank you for your continued interest in and support of Blueprints.
 
Happy Holidays!

Sincerely,

Pamela Buckley, PhD
Principal Investigator
Institute of Behavioral Science
University of Colorado Boulder
Pamela.Buckley@Colorado.edu

Karl G. Hill, PhD
Co-Principal Investigator
Chair of the Blueprints Board
Institute of Behavioral Science
University of Colorado Boulder
Karl.Hill@Colorado.edu

Blueprints for Healthy Youth Development is hosted by the University of Colorado Boulder, Institute of Behavioral Science, with current support from Arnold Ventures and former funding from the Annie E. Casey Foundation. Each intervention included in the Blueprints database has been reviewed carefully by an independent advisory panel that looked at research on the intervention’s impact, practical focus and potential for implementation in public systems.

2021 in Review
By the numbers
 
Below is a summary of our year in review. 
 
Interventions Reviewed & Certified in the Past Year
 
•  Number of Unique Interventions Reviewed = 47
•  Number of Individuals Articles or Reports Reviewed = 191
•  Number of Interventions Certified = 5
 
Coping Power Universal (Promising)
Added: Feb. 18, 2021
Learn more
 
YouthBuild (Promising)
Added: Mar. 24, 2021
Learn more
 
EveryDay Intervention (Promising)
Added: Jun. 4, 2021
Learn more
 
Cannabis eCHECKUP TO GO (Promising)
Added Jul. 7, 2021
Learn more
 
ASSISTments (Promising)
Added Nov. 22, 2021
Learn more

© 2021 Blueprints for Healthy Youth Development, Regents of the University of Colorado. All rights reserved. 

Our mailing address is:
University of Colorado Boulder | Institute of Behavioral Science
483 UCB, Boulder, CO 80309

Welcome to the Blueprints Bulletin

Leadership Letter: 

We are excited to share with you an important milestone Blueprints recently reached. But first, a quick summary of our history will help frame the discussion.
 
Blueprints History

The Blueprints for Healthy Youth Development mission is to provide a comprehensive registry of scientifically proven and scalable interventions that prevent or reduce the likelihood of antisocial behavior and promote a healthy course of youth development and adult maturity. Those interventions are rated as either Promising or Model/Model Plus.
 
Originally started as Blueprints for Violence Prevention, Blueprints was one of the earliest efforts to establish a clear scientific standard for evaluating the evidence of an intervention’s effectiveness, implementing a rigorous expert review process and certifying those interventions that met this standard. Launched in 1996 by internationally renowned researcher Dr. Delbert S. Elliott (who also founded the Center for the Study and Prevention of Violence in 1992), the early years of Blueprints focused on identifying programs that were effective in addressing violence and drug use outcomes. Since its inception, Blueprints has expanded its scope to include mental and physical health, self-regulation, educational achievement, and other positive developmental outcomes.

Celebrating an Important Milestone: 100 Certified Interventions!

In August 2021, Blueprints reached an important milestone by certifying its 100^th intervention!
 
Of these, 18 are rated Model/Model Plus and 82 are Promising.
 
The Blueprints website provides an interactive search feature to identify Blueprints-certified interventions based on specific criteria that allows users to browse through a wide range of interventions that match those criteria.
 
Model and Model Plus interventions are listed separately from Promising interventions.
 
This is because Model and Model Plus interventions have demonstrated efficacy for changing outcomes over time and are recommended by Blueprints for large-scale implementation.
 
Promising interventions show promise of efficacy but require follow-up research before being recommended for large-scale adoption.
 
How does an Intervention receive a Rating of Model or Model Plus?

The Blueprints registry standard for a Model rating requires: 

1. A theoretical rationale/logic model
2. One or more high quality randomized control trials
3. An experimental replication
4. Sustainability of effects for at least one-year post intervention
5. No evidence of negative or harmful effects
6. The organizational capacity to provide materials, training, and information for new users to adopt and implement the model intervention with fidelity

Blueprints also has a Model Plus rating, which requires at least one trial by an evaluation team independent of the program developer and his or her colleagues. In sum, the logic is that to consider any program as ready to be relied on in practice or for scaling up, there should be a clearly specified theoretical rationale, reliable evidence of positive, sustained impact based on a sound experimental research design, and at least one replication of positive findings.
 
A high-quality quasi-experimental design (QED) can meet Blueprints standards, but to earn a Model or Model Plus rating, effective interventions must have demonstrated positive effects in at least one high-quality RCT.
 
The Blueprints website provides more information about our standards and review process.

Testing Promising Interventions for Scale-up
 
While most certified interventions listed on the Blueprints website are rated Promising (82 out of 100), several are currently being tested for replication.
 
Under the Research Grants Focused on Systematic Replication (84.305R) competition, the Institute of Education Sciences (IES), which is the research arm of the US Department of Education, will support systematic replication studies of interventions that have produced beneficial effects on education outcomes in one or more rigorous causal-impact studies. We are aware of two replications of Blueprints Promising programs that IES is currently funding:

• Targeted Reading Instruction (TRI, formerly called Targeted Reading Intervention) is a teacher professional development program designed to help classroom teachers acquire key diagnostic reading instruction strategies differentiated to meet the exact needs of individual students who are struggling with early reading. Blueprints certified the program based on an RCT conducted by Amendum et al. (2011). Dr. Robin Wisniewski from RTI International received a 5-year grant (07/01/2021 – 06/30/2026) that will follow first grade students and teachers for an additional 2 years beyond the intervention to test TRI’s long-term impacts on reading achievement.
• Whole Number Foundations Level K (Roots) is a 50-session small group mathematics intervention delivered as a pull-out program during the regular school day. The aim is to help strengthen whole number concepts and operations skills in students at risk for developing long-term mathematics difficulties. Blueprints certified the intervention based on an RCT conducted by Clarke et al. (2016). Dr. Ben Clarke from the University of Oregon received a 5-year grant (07/01/2020 – 06/30/2025) to conduct a replication study of ROOTS for students at risk for mathematics learning disabilities across school types and student populations intentionally selected to differ from previous efficacy studies.

Arnold Ventures, a philanthropy dedicated to tackling some of the most pressing problems in the United States, has provided support for RCTs of programs across the spectrum of social policy whose prior evidence shows potential for sizable effects on education, earnings, crime, and other important outcomes. Not only has Arnold Ventures provided support for the Blueprints registry, but it has also supported replications of several interventions rated as Promising on Blueprints:

• Big Brothers Big Sisters of America (BBBSA) is a community mentoring program that matches a volunteer adult mentor to an at-risk child or adolescent to delay or reduce antisocial behaviors and improve academic success. The program was certified based on an RCT conducted by Grossman & Tierney (1998). Dr. David DuBois from the University of Illinois at Chicago and Dr. Carla Herrera (an independent consultant) are conducting a multi-site RCT with a sample of 20 BBBSA agencies nationwide (the grant term runs from 2016-2024).
• Career Academies, which are school-based programs designed to provide students with the credentials and skills needed to make successful transitions to post-secondary education and careers, was certified by Blueprints based on a multi-site RCT conducted in a diverse set of urban and small-city high schools in California, Texas, and several cities on the east coast (Kemple & Snipes, 2000; Kemple, 2004; Kemple & Wilner, 2008). MDRC is conducting a replication of Career Academies in California high schools. The term of the grant is from 2017-2032.
• Child First, a two-generation home visitation program which works to heal and protect young children and their families from the devastating effects of chronic stress and trauma, was certified based on an RCT conducted by Lowell et al. (2011). With support from Arnold Ventures and The Duke Endowment, MDRC is leading an experimental study that aims to replicate the findings from the earlier study in multiple locations across Connecticut and North Carolina.
• EAAA (Enhanced Assess, Acknowledge, Act) Sexual Assault Resistance Education is a four-unit program to help first-year college women resist acquaintance sexual assault by providing them with information and resistance training. The intervention was certified based on an RCT conducted at three universities in Canada by Senn et al. (2015). Dr. Koss from the University of Arizona is leading a multi-site RCT that includes a one-year follow-up across five U.S. universities.
• Rochester Forensic Assertive Community Treatment (R-FACT) is an outpatient treatment program to reduce recidivism and promote recovery among justice-involved adults with a serious mental illness that was certified based on an RCT conducted in New York by Lamberti et al. (2017). The University of Rochester is conducting an RCT to evaluate the R-FACT model in Minnesota. The grant runs from 2018 to 2025.
• Year Up is a training and internship program that helps young people with limited post-secondary education get high-quality jobs by learning to work with technology, developing employment skills, and obtaining internships. The program was certified based on an RCT conducted by Fein & Hamadyk (2018) in eight sites across the US. Dr. David Fein of Abt Associates is leading long-term follow-ups to two RCTs of adaptations of the Year Up model, and the grant runs from 2019-2022.

Nominate an Intervention
 
If you are aware of an intervention Blueprints has rated as Promising that is being replicated and you would like us to review the study, please send all evaluation articles/papers to: blueprints@colorado.edu.
 
And if you are a funding agency awarding grants to replicate intervention programs designed to promote healthy youth development, please contact us as well. We would be interested in profiling your work and any interventions from the Blueprints registry you are funding to replicate.
 
More information on nominating an intervention for Blueprints review can be found on our website by clicking here.
 
As always, thank you for your continued interest in and support of Blueprints.

Sincerely,

Pamela Buckley, PhD
Principal Investigator
Institute of Behavioral Science
University of Colorado Boulder
Pamela.Buckley@Colorado.edu

Karl G. Hill, PhD
Co-Principal Investigator
Chair of the Blueprints Board
Institute of Behavioral Science
University of Colorado Boulder
Karl.Hill@Colorado.edu

Blueprints for Healthy Youth Development is hosted by the University of Colorado Boulder, Institute of Behavioral Science, with current support from Arnold Ventures and former funding from the Annie E. Casey Foundation. Each intervention included in the Blueprints database has been reviewed carefully by an independent advisory panel that looked at research on the intervention’s impact, practical focus and potential for implementation in public systems.

© 2021 Blueprints for Healthy Youth Development, Regents of the University of Colorado. All rights reserved. 

Our mailing address is:
University of Colorado Boulder | Institute of Behavioral Science
483 UCB, Boulder, CO 80309

Announcement: 

We are excited to announce that Dr. Pamela Buckley is the new Principal Investigator (PI) of Blueprints for Healthy Youth Development. Dr. Buckley joined the Blueprints team in 2016 and has been serving as Co-PI and Director. Dr. Karl Hill is still involved in the vision and scientific integrity of Blueprints and will assume the role of Co-PI and will remain Chair of the Blueprints Advisory Board. Read more about Drs. Buckley and Hill’s experience here.

Leadership Letter: 

This newsletter features results from an online survey conducted in collaboration with our friends at Evidence-Based Associates on Model/Model Plus and Promising Programs’ COVID-19 (COVID) response.
 
In May 2020, Blueprints self-funded a survey that was administered to contacts listed for the Blueprints certified programs on how evidence-based interventions had responded to, and begun to plan for, the aftermath of the COVID pandemic. The 2020 survey contained eight questions and 58 of the 94 programs surveyed responded, for a 62 percent response rate. 
 
This one-year follow-up, administered between May and June 2021, had a 58% response rate (57 of the 99 certified programs responded).
 
Results show most programs have experienced a small (or no) impact in delivering services as a direct result of COVID. However, a wide range of modifications to service delivery and implementation strategies were reported – findings that remained relatively stable over time.

Go here to review the May 2021 findings. 
 
Impact of COVID-19 on dissemination/implementation 

Most programs reported either no significant impact in service delivery from COVID, or a small impact causing some difficulties in maintaining dissemination but overall stability. In addition, more than one-quarter reported a positive impact, with COVID providing new opportunities for development. There was, however, a small percentage of programs that experienced a great impact, leading to discontinuation or serious difficulties in sustaining dissemination of their Blueprints-certified program.
 
Status of the dissemination/implementation 

The data show the majority of programs that completed our survey had received requests for changes to the delivery modality and training/support of their intervention. These findings were relatively consistent across the initial (May 2020) and follow-up (May 2021) surveys. As of May 2020, just under half reported experiencing new requests for the adoption of their model – a number that increased by ten percentage points, as measured by the May 2021 responses. Meanwhile, none had discontinued implementation in the wake of COVID as of May 2020 (a percentage that slightly increased in May 2021), but some reported they temporarily suspended their program (a percentage that slightly decreased from the initial to the follow-up survey).
 
Modifications to service delivery 

We conducted our May 2020 survey as many states were experiencing varying levels of stay-at-home orders, schools had been abruptly closed, and most business and childcare facilities were either shut down or beginning to phase in opening plans. As a result, programs were quickly faced with important decisions to ensure the safe continuity of programming while following public health guidelines. Just under one-quarter had not modified the intervention or its delivery due to COVID at the time of our May 2020 survey. The majority, however, provided online resources, turned to tele-sessions and/or video conferencing, and/or offered online training or lessons to support service delivery. A small percentage started a blog as a forum for conversation.
 
By May 2021, as the United States and some countries around the world had begun to ease restrictions with the availability of the COVID-19 vaccine, most programs continued to modify their service delivery by providing online resources and training and turning to tele-sessions/video conferencing – however, these numbers consistently increased by roughly 10 percentage points. Meanwhile, those blogging to offer a forum for conversation remained a small percentage (and this number dropped in half from May 2020 to May 2021).
 
In a new question posed with the follow-up survey, we found that an overwhelming majority had implemented changes to their delivery modality. An even greater majority also had implemented changes to their training/support model. 
 
Impact of modified service delivery on intervention outcomes 

More than one-third of survey respondents are either collecting data or plan to collect data soon on the relationship between modifications made due to COVID and intervention outcomes. This finding was consistent across both surveys. However, just over one-quarter reported in May 2020 that they do not have the resources to collect these data – a number that increased to over one-third in May 2021.
 
What was learned about implementing evidence-based interventions in the context of COVID?

We included an open-ended item asking about lessons learned. Responses are summarized here:

• The pandemic highlighted the importance of evidence-based programs for youth as needs increased (e.g. increased levels of behavioral and mental health problems) and virtual programming revealed the basic needs of families supporting these youths (e.g. food insecurity, housing, mental and basic health resources).
• Though developing virtual formats while maintaining fidelity to essential program elements is a lot of work, many programs found various levels of success in providing online/virtual training and/or implementation.
• There is a need for in-person/online hybrid models. Virtual delivery encounters challenges – e.g., lack of access to or difficulties among participants with incorporating technology, inability to fully conduct interactive program activities, and lower levels of engagement and community-building. Virtual delivery also provides opportunities – e. g., increased engagement by removing barriers to access, reaching a wider audience, and reducing costs of training and delivery.
• For school-based programs, hard copy curricula must take advantage of the modern classroom and technology so that schools will continue to adopt these programs. Teachers have learned to use technology more effectively, and will continue to use technology platforms to assist with teaching.
• Flexibility is key for any program.

On March 11, 2020, the coronavirus outbreak was declared a pandemic by the World Health Organization. Since then, COVID has touched all aspects of life, from health care and mental health services to education, the environment, and the economy. Blueprints seeks to assist evidence-based interventions in sharing our survey data so that we can work together in supporting families and communities through the complexities of these challenging times. We hope the resources provided by survey respondents and shared in the next section of this newsletter offer helpful tips and ideas.
 
Sincerely,

Pamela Buckley, PhD
Principal Investigator
Institute of Behavioral Science
University of Colorado Boulder
Pamela.Buckley@Colorado.edu

Karl G. Hill, PhD
Co-Principal Investigator
Chair of the Blueprints Board
Institute of Behavioral Science
University of Colorado Boulder
Karl.Hill@Colorado.edu

Blueprints for Healthy Youth Development is hosted by the University of Colorado Boulder, Institute of Behavioral Science, with current funding from Arnold Ventures and former funding from the Annie E. Casey Foundation. Each intervention included in the Blueprints database has been reviewed carefully by an independent advisory panel that looked at research on the intervention’s impact, practical focus and potential for implementation in public systems.

© 2021 Blueprints for Healthy Youth Development, Regents of the University of Colorado. All rights reserved. 

Our mailing address is:
University of Colorado Boulder | Institute of Behavioral Science
483 UCB, Boulder, CO 80309

Leadership Letter: 

The Importance of Dissemination Readiness  
 
Did you know that Blueprints for Healthy Youth Development lists dissemination readiness as one of four formal requirements for meeting its standard of evidence? This means that, before an intervention is listed as Blueprints-certified, we verify that there is a clear description of its activities and explicit implementation materials or procedures. Blueprints’ other standards are intervention specificity, evaluation quality, and intervention impact; the dissemination readiness dimension is evaluated only for interventions that satisfy these first three requirements.
 
According to Blueprints’ dissemination readiness criteria, well-evaluated interventions must have materials or instructions that specify the intervention content and guide the implementation of the intervention, such as manuals, training, and technical assistance, and (where available) specify costs associated with implementation (such as those for start-up, implementation, and support) and staff resources (for example, staff qualifications and time commitments) needed to deliver the intervention with fidelity.
 
Thus, interventions that meet the Blueprints standard of evidence must be available for use.
 
Blueprints certifies programs that meet all four of its standards (intervention specificity, evaluation quality, intervention impact, and dissemination readiness) as Model/Model Plus or Promising.
 
We recently conducted a study to determine whether online clearinghouses rate and provide information about the degree to which well-evaluated programs can be well-implemented. Our goal was to fill a knowledge gap by examining if and how the dissemination readiness of evidence-based interventions has been defined, evaluated, and communicated to potential users. To do so, we compared 11 different U.S.-based online private and federally funded clearinghouses on (1) the degree to which they use dissemination readiness as a criterion for inclusion/exclusion of evidence-based interventions, and (2) the extent of information and support they provide about dissemination readiness to facilitate real-world implementation.
 
We found wide variability, with few having standards about dissemination readiness or making information about implementation of evidence-based interventions easily accessible to users. Across the 11 clearinghouses we examined, only Blueprints has both a requirement for dissemination readiness and provides relatively complete information relevant to dissemination readiness.
 
Our findings indicate the need for online clearinghouses to (1) do more to assess dissemination readiness and (2) offer more complete information on dissemination readiness and implementation support to users. Doing so, however, requires a commitment from funders (who have finite resources) to invest in the dissemination readiness of evidence-based interventions. That is, funding is needed for developers to provide robust dissemination materials and time is needed for clearinghouse staff to collect this information. We therefore want to recognize the Annie E. Casey Foundation, which provided funding to Blueprints to develop the tools we currently use to gather information on dissemination readiness and historically paid contractors to collect this information as part of its Evidence2Success initiative.
 
Why Conduct this Study?
 
This is important work to share because we want to convey to users our process for listing on our website interventions rated as Model/Model Plus or Promising. If an intervention does not meet most or any of Blueprints dissemination readiness criteria but has a high-quality design, that intervention will receive a “not dissemination ready” rating – meaning it has met criteria for evaluation quality (as determined by the Blueprints Advisory Board) but has not yet met the dissemination readiness criteria. This could be one reason why certain interventions are identified with an effective rating on other clearinghouses but not listed as a Blueprints-certified intervention on our website.
 
If you are interested in reading more, the full study was published in Evaluation Review, a peer-reviewed academic journal that aims to advance the practice of evaluation and to publish the results of high-quality evaluations. The abstract and full text have been published as an open-access article and can be accessed through the publications page of the Blueprints website as well as on PubMed.gov.
 
Recognizing These Challenging Times 
 
In closing, we cannot ignore that many communities, families, and individuals continue to struggle. With a global pandemic, protests and riots in parts of the United States and world, divisive elections, online learning and remote work, and – for some parts of the US – extreme weather conditions, this past year will go down as one of the most devastating in modern history. As we all continue to cope with and adapt to these realities, Blueprints will persist in working on our mission of promoting effective preventive interventions and helping governmental agencies, foundations, and practitioners use strong evidence to make informed decisions when identifying interventions that provide a high probability of success when taken to scale in communities. It is this commitment that helps us persevere through these challenging times. We thank each one of you for your part in our community and for your involvement in this important effort.
 
Sincerely,

Pamela Buckley, PhD
Director and Co-Principal Investigator
Blueprints Initiative
Institute of Behavioral Science
University of Colorado Boulder
Pamela.Buckley@Colorado.edu

Karl G. Hill, PhD
Principal Investigator
Blueprints Initiative
Institute of Behavioral Science
University of Colorado Boulder
Karl.Hill@Colorado.edu

Reference: Buckley, P. R., Fagan, A., Pampel, F. & Hill, K. (2020). Making evidence-based interventions relevant for users: A comparison of requirements for dissemination readiness across program registries. Evaluation Review, 44(1), 51-83. https://doi.org/10.1177/0193841X20933776

Blueprints for Healthy Youth Development is hosted by the University of Colorado Boulder, Institute of Behavioral Science, with current funding from Arnold Ventures and former funding from the Annie E. Casey Foundation. Each intervention included in the Blueprints database has been reviewed carefully by an independent advisory panel that looked at research on the intervention’s impact, practical focus and potential for implementation in public systems.

• Child First (a Blueprints promising program) is a home visitation program that works to heal and protect young children and their families from the devastating effects of chronic stress and trauma. With a $700,000 grant awarded in 2018, The Duke Endowment is supporting a randomized controlled trial of Child First in eastern North Carolina. Arnold Ventures is providing funding to study Child First in Connecticut. MDRC is conducting research in both states and Meghan McCormick (an MDRC research associate) explains more about evaluating Child First and about finding a silver lining in COVID-19. Go here to read the full interview.
• Big Brothers Big Sisters of America (BBBSA) is a Blueprints promising program that matches an adult volunteer mentor with an at-risk child or adolescent to delay or reduce antisocial behaviors and improve academic success. Emily Johnson, CEO of BBBS of Southwest Idaho, says their chapter is matching kids with adult mentors based on compatibility. Go here to learn more about the procedures BBBS of Southwest Idaho is following to help mentors and mentees interact while ensuring the safe continuity of programming during the COVID-19 outbreak.
• The Communities That Care (CTC) chapter of West Chester (a township in Pennsylvania) is providing resources for parents to help them deal with the challenges the COVID-19 pandemic has presented. CTC (a Blueprints promising program) is a prevention system designed to reduce levels of adolescent delinquency and substance use through the selection and use of effective preventative interventions. The CTC chapter of West Chester has dedicated its 2021 Parent Speaker series to the topic of emotional health to keep parents connected, and to offer concrete solutions concerning today’s parenting challenges. To learn more about the speaker series and the efforts being made by the West Chester CTC chapter, go here.
• The Pace Center for Girls in Alachua (a county in Florida) recently received a $5,000 Golden Grant award from the North Florida McDonald’s Operators Association. Pace Center for Girls (a Blueprints Promising program) is a set of gender-responsive prevention and early intervention programs for girls with multiple risk factors for juvenile justice system involvement, which uses a holistic approach to re-engage girls with learning, improve academic performance, and address underlying trauma. Natalya Bannister, the executive director for Pace Center for Girls Alachua, says the program was vulnerable due to the COVID-19 pandemic and that the funds will go towards supporting their reading initiatives. Go here to learn more.

© 2021 Blueprints for Healthy Youth Development, Regents of the University of Colorado. All rights reserved. 

Our mailing address is:
University of Colorado Boulder | Institute of Behavioral Science
483 UCB, Boulder, CO 80309