I sometimes feel like American education is under a massive collective delusion. You might have heard the phrase "High Quality Instructional Materials," often abbreviated as HQIM. That phrase does not mean what you might think it means.
A bit of history. EdReports is a website for curriculum reviews, founded in 2015. The Common Core standards were relatively new. At the time, many curriculum companies stuck a "Common Core Aligned" sticker on their existing materials while making few actual changes. There was a need to figure out which curricula were in fact Common Core aligned. EdReports filled that gap. That was the original goal: help districts understand whether curricula were truly aligned to the standards.
Somehow, EdReports is still the only major organization reviewing curricula. And somewhere along the line, principals and superintendents all decided that if EdReports gives a curriculum all green (their best rating), then they must be High Quality Instructional Materials. That's what the phrase means to most people. That's all. A perfect score on EdReports is necessary and sufficient to be called High Quality Instructional Materials.1
Plenty of other writers have pointed out issues with the idea of High Quality Instructional Materials. My goal is to give a specific example from a real curriculum to illustrate what this stuff can look like in practice.
An Example
Below is an example from a curriculum that got a perfect score on EdReports.
This is the culmination of a two-lesson sequence on multiplying rational numbers, and it is the first time students have seen multiplication with negative numbers. The two lessons include a few activities introducing the concept to students and doing a quick check for understanding. This curriculum offers teachers some choice so there are multiple ways the teacher can introduce the content. Then, there are two pages of practice.
(I won't reproduce the exact curriculum here or name the program because I don't want to be sued for a copyright violation. If you would like to see it or learn more, feel free to email me.)
The first page begins with these four problems:
1. -8(7)
2. 5(-13)
3. -3.4(3)
4. 4.8(-6)
Then there are four word problems. The word problems are fine; a bit contrived, but so are most word problems. I won't focus on them here.
The second page begins with these four problems:
9. -4(-9)
10. -8(-12)
11. -4.1(5)
12. -12.6(-5)
Then there are three word problems; again, a bit contrived, but fine.
That's the entire set of practice problems for this topic. There's a lesson on division, and then a lesson focused on combining multiple operations, like -5(-8) - 7, though there are only six total multiplications in the lesson. Then there are a few more multiplication problems in the review questions at the end of the unit.
My claim is that, at a very basic level, you cannot call this an example of high-quality curriculum.
So what are the issues?
Not enough practice
It's hard to judge how much practice students need. Curriculum designers want to avoid too much repetitive practice, while also providing enough for students to have a solid understanding of the topic. There is no perfect number. But this is absolutely not enough practice. Multiplying with negatives is an important skill, and most students will need more than 15 total problems.
The numbers are poorly chosen
8 × 7 is typically one of the last multiplication facts that students commit to memory. Very few students have 5 × 13 committed to memory. This matters because, when students are learning a new topic, they should focus on the concept that's new: in this case, whether the answer is positive or negative. When I teach this topic, I ask a ton of questions like -3 × 2, or -4 × (-10). I use easy numbers that students can multiply without having to think, so they can focus their thinking on whether the answer is positive or negative. This curriculum does the exact opposite. Is there a place for questions with bigger numbers and decimals? Absolutely. But not at the beginning of practice, where decimals and bigger numbers pull student attention away from the new concept.
The notation is repetitive
Every question is written in the form 2(-3). That's great notation — it's important students understand what those parentheses mean. But they should also see other notation, like 2 × (-3), and 2·(-3), and (2)(-3). There should be a few problems like “find 3x if x = -5.” Variety helps students use different notation flexibly in the future.
Students can answer questions correctly without understanding the core concept
This is maybe the most important one. On the eight problems on the first page, every problem involves multiplying a positive and a negative, with a negative answer. Students can avoid understanding the concept entirely, and just say "oh the answer must be negative." Then on the second page, every question involves two negatives and a positive answer. A bit of blocked practice is fine at the start, but it should transition as quickly as possible to interleaved practice mixing different problem types together.
The practice isn't very ambitious
There are a bunch of problem types that aren’t present. There are no fractions. There are no problems multiplying more than two numbers, like (-2)(-2)(-2). Those are important problems as students become comfortable with the basics of multiplying two numbers, so they can extend their knowledge to different contexts. For some reason, fractions and more complex problem types appear in the unit review but not in the actual lessons.
The Delusion
These are not high quality instructional materials. Learning to multiply negative numbers is a skill that will come up over and over again, both later in 7th grade math and in future years. It’s worth learning. These materials come up short. The reality is most school leaders making decisions about curriculum don’t have the time to pick through curricula with this level of detail. Many don’t have the expertise to know what they should be looking for in a math curriculum. I understand why they want to farm out some of that decision-making to an organization like EdReports. It’s a shame that EdReports doesn’t do a good job, and “High Quality Instructional Materials” has become synonymous with a high score from EdReports all the same.
As you read this, somewhere there is a principal or math curriculum lead in a meeting with a superintendent or a school board. They’re getting up on their high horse, talking about how important High Quality Instructional Materials are. Maybe they’re talking about how they’ll require teachers to follow their new curriculum with fidelity. And then they go and adopt a curriculum like the one I described above.
A Few Notes
This is one lesson. You might wonder if I’m cherry-picking. This lesson is particularly poor compared to others from this curriculum. Most lessons do a better job. But it’s not uniquely bad — there are plenty more with similar problems. And multiplying rational numbers is a particularly important concept, so I think it's a fair example to use. If the lesson on complementary angles doesn’t do a great job structuring interleaved practice, life will go on. But you have to get multiplying rational numbers right. If this curriculum can get a perfect score despite lessons like the one I described, something is wrong.
There are good curricula out there. Not all programs rated by EdReports are as poor as what I'm describing. The issue is that it's very hard to tell with the kind of cursory glance most schools have the resources to give. And the cult of High Quality Instructional Materials means that people assume, since it has a good score from EdReports, it must be good.
Ironically, I helped choose this curriculum. Teachers in my district were given two choices for middle school math. We looked at this program and one more. The other was worse, and it also got a perfect score from EdReports.
I use the JUMP materials (which are not perfect, but I like a lot) to supplement my primary curriculum. They aren’t currently rated on EdReports.2 Does that mean I’m using Low Quality Instructional Materials to compensate for the shortcomings of High Quality Instructional Materials?
If I were to generalize broadly, the most common issue with “High Quality Instructional Materials” in math is practice. Practice is important, but it’s not evaluated at all by EdReports. I’ve seen and worked with a number of highly-rated curricula, and a lack of practice or poorly-structured practice are common issues.
I could name many more issues. Too many representations without enough sustained effort on each individual representation. All sorts of fluff about mindset and social-emotional learning that feels like it was thrown together at the last minute. A lack of spaced practice. Little attention paid to prior knowledge. Cluttered, hard-to-use handouts. Clunky digital interfaces. All in highly-rated curricula. My goal with this post isn’t to write an exhaustive critique, just to give one specific example of one way a highly-rated curriculum can fall short.
The moral of the story is that we have a desperate need in American education for someone to compete with EdReports, offer a serious definition of high-quality curriculum, and give decision-makers another source of information.
I realize some people define “High Quality Instructional Materials” more rigorously. To the vast majority of US educators, that phrase refers to materials that get all green on EdReports, no matter what some would like to claim.
I spent a while looking and couldn’t find any reports on JUMP, though I do remember them being rated in the past and getting a yellow or “partially meets expectations” score. Weird.