Creating a Blueprint for Enhancing School Supports in North Dakota

A case study from McREL and the Region 11 Comprehensive Center

When the North Dakota Department of Public Instruction (NDDPI) in 2019 began identifying schools for comprehensive and targeted support, per the Every Student Succeeds Act (ESSA) requirements, the department’s Office of Education Improvement and Support took the reins. They worked with all the state’s Targeted Support and Improvement (TSI) schools and districts, which are selected on an annual basis and are considered consistently underperforming in the bottom 10 percent, and the state’s Comprehensive Support and Improvement (CSI) schools and districts, which are selected every three years and have the highest proportion of struggling students.

As a variety of improvement efforts got underway at individual schools and districts, NDDPI knew that it needed an overarching plan and uniform framework to set statewide expectations and monitor if the work was resulting in real change.

Recognizing these challenges, the NDDPI and its Office of Education Improvement and Support reached out to the Region 11 Comprehensive Center (R11CC), a support center funded by the U.S. Department of Education and operated by McREL, a nonprofit education research and service organization.

“Working with Region 11, we wanted to do a deeper dive into what our schools truly need,” said Rachel Tabler, assistant director with the Office of Education Improvement and Support. “We wanted to do some root cause analysis of why some schools reappear on this list. Do we need to be engaging with them differently? Do we need a different process in order to help them exit from either TSI or CSI?”

With their high-leverage problem identified, a goal of supporting and monitoring continuous improvement in schools using evidence-based practices came into focus. The NDDPI team knew that whatever was created needed to become embedded into the fabric of their systems and processes, scaled statewide to all TSI and CSI schools, and touch every aspect of the state’s K–12 school ecosystem.

The work began by creating a partnership and identifying teams to drive the project.

A capacity building, school-level approach for sustainable improvement

The NDDPI and the R11CC leadership believed the key to the project would be empowering school teams to do their own problem solving, which in previous efforts wasn’t always the expectation. “For many years leading up to this work, schools were conditioned to have outside vender groups come in and basically do improvement to them versus improvement alongside them,” said Ben Cronkright, the R11CC’s co-lead for North Dakota. “So from the beginning we wanted to create a model that would support and build the capacity of school teams to lead their improvement efforts.”

Teams from several schools and districts were brought into the work as early adopters, and they helped identify ways to keep local districts involved in school improvement efforts, rather than letting the state or outside consultants take the lead. “You have to have skin in the game,” explained Joe Simpson, a McREL school improvement leader and co-director of the R11CC. “Their work and input proved critical in crafting and continuously improving the statewide efforts.”

Partnering with additional organizations for greater success

Next, leveraging existing relationships and partnerships developed by Amanda Peterson’s work as director of the Office of Education Improvement and Support, the North Dakota University System was invited to join the effort by supporting data collection and data use. With the addition of the university system’s expertise, led by Dr. Greg Carlson, the state’s capacity for, clarity of, and transparency with data collection, analysis, and literacy was greatly expanded.

“The majority of schools in our state didn’t really know why they were being identified,” said Peterson. “They didn’t know the data that went into it, and we didn’t have really good tools for them to even look at it other than at the very global scale. They didn’t know what the formulas were. That has completely shifted. They now understand their data. And if you understand your data, you have a better opportunity to influence what can change that data.”

About a year into the work, the NDDPI and R11CC team realized they were missing an important capacity in the partnership. “We came to the conclusion that we needed more boots on the ground, something our state’s Regional Education Association could provide,” said Tabler. “They’re regionally located. They’re close to the schools. They know the communities, and many of them already work very closely in those schools.”

The North Dakota Regional Education Association (NDREA) supports seven education service agencies across the state. Within the R11CC partnership, NDREA’s leaders and regional liaisons operationalized and scaled the work to bring it directly into every TSI and CSI school through coaching on the improvement science framework and use of evidence-based practices.
Once NDREA joined the partnership, “we really had alignment across all parts of the K–12 education system that set it up to do what it was intended to do—to support each school team with implementing their change and improvements well,” said Cronkright.

Structures: Guidance and implementation teams, and a common framework for improvement

While each entity plays its own role within the partnership, representatives also participated in three collaborative cross-agency teams that guided the work: the guidance team, the implementation team, and the core team.

The guidance team, made of representatives across NDDPI and the R11CC, selected the Carnegie Foundation’s Continuous Improvement Framework as the improvement theory of action for approaching all TSI/CSI work. “The real underpinning of that decision was to get everyone to use a common problem-solving framework and professional language. It helps pull everybody in to being responsible to help improve these districts and schools,” said Simpson.

The implementation team, through working strategy sessions, identify the technical assistance plans, which are important for developing and maintaining project timelines. “The outcomes of those meetings helped us get that much more clarity and definition on the project management plan that was coupled with our logic model,” said Cronkright.

And then there’s the core team, which meets weekly and includes key decision makers from all of the partners. The team focuses on facilitation and continuous improvement, helping define the project’s direction and then stopping periodically to evaluate and assess if they’re meeting their goals, said Cronkright.

Supporting and sustaining continuous improvement

From all these meetings and technical assistance work came one of the largest lifts to date, the creation of the North Dakota School Renewal Guide. Now in its third edition, the guide distills the methodology of improvement science and outlines three phases for all TSI and CSI school renewal teams to follow to identify the problem they’re trying to solve, consider the change practices they can introduce, and create measurements to check if the practice is resulting in desired changes. Each edition has captured updated continuous improvement resources and insights learned from NDDPI, the regional agencies, and the TSI and CSI schools and districts using the guidebook.

“The R11CC helped us create a product that we’re really proud of,” said Peterson. “It’s an actual tool that we’re able to use. They were able to do some of the heavy lifting, but we were able to very much inform the adjustments and the changes that we knew would be relevant to our state and our work. We personalized it, yet still hold true to the improvement science approach.”
In tandem with creating the guide, the R11CC began training all the partners and teams who would promote its use statewide and help educators use the guide. “If you’re really going to make a significant systemic improvement change, you have to educate the department and key players on what we’re trying to do,” said Simpson.

This technical assistance produced multiple pathways for understanding and using improvement science tools and methods, including professional learning across the agencies and teams, as well as the creation of a short video on the plan-do-study-act (PDSA) cycle of improvement science. The R11CC worked closely with regional agency leadership and liaisons to ensure they understand the framework and the state’s expectations. “Really, it was a lot of support on coaching them through the process so that they could help coach our schools through the process,” said Tabler.

The R11CC team also promoted the use of communities of practice to ensure trainings weren’t simply delivered once and then forgotten. “The whole idea of it is building on each technical assistance activity so it wasn’t a one-and-done thing,” said Cronkright.

Measuring progress and moving forward

The U.S. Department of Education’s comprehensive center contracts are five-year agreements. As the current R11CC project enters its fifth and final year, the partnership work is becoming more focused on efforts to ensure long-term sustainability. “Our R11CC goal from the onset has been to build local capacity and essentially work ourselves out of the work,” said Cronkright.
To do this, the teams are focusing on how to best support the regional education agency liaisons, who have become far more intentional as they’ve grown in confidence, according to Peterson, as well as ensuring calibration and data transparency and continuing to build trusting, supporting relationships across the partnering agencies and schools.

One major task left on partnership’s to-do list is reducing variability in how schools are measuring short-cycle improvement and ensuring that the body of evidence being collected is consistent and accurate across the state. Within the guidebook, the NDDPI team created a rubric/progression scale. This year, the team is working on a calibration process for that rubric to ensure inter-rater reliability across the regional liaisons and the NDDPI reps, who use the rubric every time they visit a school. This rubric helps track each school team’s progress on their improvement project, so the data isn’t just reflective of interim assessment data, but also tracks the school’s ability to implement something well, said Cronkright.

“The calibration process should help make the rubric a valid and reliable measure aligned with how the state defines progress for CSI and TSI schools,” he said. “Yes, it’s about student outcomes, but it’s also looking at how well the school team is using evidence-based strategies.”

In addition to understanding their data, the rubrics will contribute to a more thorough body of evidence for schools to exit CSI/TSI status. Peterson explains that even if one year’s worth of work isn’t quite enough to get a school out of status, the collected evidence will help the school see that they’re heading in the right direction.

“It’s not a secret about what gets you in or out of CSI or TSI status,” Peterson said. To the NDDPI team, this transparency and body of evidence is helpful for relationship and trust building.

“I really like where we are at with our schools. I feel like we know more about what the schools are doing than we’ve ever known,” said Tabler. Between the improvement office staff and the regional liaisons, crucial conversations happen regularly and there is a clearer understanding of the work being done and the requests being made, and there is respect among the team for each other as they’re all working toward the same goal, she said.

Some schools that have used this approach have exited CSI/TSI status during the course of the partnership, which is a success worth celebrating. Partnership members also point to the state model itself as another success story.

“It’s a combination of a lot of things, but the creation and use of the state guidebook coupled with the regional coaching is promoting systems change that obviously impacts the schools’ abilities to improve outcomes for their students,” Cronkright said.

The model sets up a process that can be repeated year in and year out, starting the moment a school is identified and continuing all the way to when the school exits status. Each district and school is responsible for its own work, but the whole collective body pulling together is a major component of its success, Simpson said.

“This partnership has created a real, working prototype that other rural states can now use to enact their own systemic changes to support their CSI and TSI districts and schools,” said McREL’s CEO Bryan Goodwin. “This could be a blueprint for statewide school improvement.”

The partnership’s advice for other state agencies undertaking similar initiatives 

North Dakota’s work has benefited from several important elements. The NDDPI and core team leadership has stayed generally consistent throughout the project’s life cycle, and they’ve been open to adaptability and transparency. And many of the agency relationships that make up the partnership were already established before this project began. These factors have supported the project’s success.

For any agency looking to rework its approach to school support, consider the following:

■ Identify a common framework for continuous improvement and couple it with an approach that is intentional about the design of cross-agency teams who will support the use of the framework.

■ Use aligned approaches and resources across agencies.

■ Define measures beyond interim and state-level assessments to provide actionable data that can be collected and reviewed in a rapid and disciplined manner.

■ Be prepared to devote time to the project. In the first four years of this project, the core team met over 200 times.

■ Build strong, trusting relationships by honoring the expertise and contributions brought by team members and agencies.

■ Embrace symbiotic relationships with your schools. Take the time to learn from your school partnerships and be willing to adapt your own processes based on their feedback.

■ Create a consistent feedback loop across the state agencies, regional partners, and TSI/CSI districts and schools.

■ Take a personalized approach to the supports offered to each school, recognizing their unique context and entry points, and respect the work they’re already doing and the in-house expertise they already have.