Measure What Matters Most

 

The Roku Express, $29.95. This quality device allows you to stream all kinds of content to your TV for a very reasonable price. The genius of the free market is that it forces companies to constantly measure what matters most: Does my product have just the right blend of quality and value price to make everyone need one?

“If you can’t measure it, you can’t improve it.” I don’t know when I first heard this quote from management thinker Peter Drucker, but I’ve believed it all of my adult life. I was a salesman at Pepsi before I became a teacher and I remember that at the beginning of each quarter, our manager posted a chart that showed our projected sales goal for that quarter, our average weekly sales, and how many cases of soda we still needed to sell until we reached our goal. We were slaves to those numbers! When route changes came up, the ones that received the best stores were the ones who most consistently hit their numbers, and you always sold more product to stores than they needed in the final few weeks so you could make the goal. Our plant manager had a pretty hands off technique of managing, but he was religious about our numbers and our facility was one of the most profitable Pepsi facilities in the nation. 

I learned a couple of things from this experience that stayed with me from all those years ago until now: First, if you want something to happen, you need to measure it. If you want your pass rate to go up, you need to track the data each term and see what is happening. If you want your test or essay scores to improve, measure the averages. You want to see more kids taking advanced courses in your school? Measure. The second thing that I learned, even more important than the first, is that you should measure what matters most so that the right thing improves. What if my manager at Pepsi had been religious about measuring how employees were about showing up on time? I’m sure that more people would have showed up exactly at 6 am rather than sometime between 6 to 6:30, but would it increase sales? 

Change happens when you measure something because it makes you aware of the actual data and allows you to make accurate judgement calls concerning whatever it is that you are measuring. Hence the quote: “If you can’t measure it, you can’t improve it.” The reverse is true as well, though: If you don’t pay attention to the data because you aren’t measuring it, you may miss important things. That’s why measuring what matters most is even more important than the idea of measuring things in the first place. Specifically, if you don’t measure what matters most, you will still see change, but perhaps not in the best places. Magic happens when organizations measure what matters most religiously and make that data available to their stakeholders so they can make good decisions. 

Someone involved in the passage of NCLB and then later ESSA must have been a believer in Druker’s proverb too, because since 2001 we’ve measured performance on tests tied to the standards, and now at the high school level we measure graduation rate too. We’ve put a lot of energy into measuring the tests in particular so it’s worthwhile to really consider how that measurement has affected education. The design of NCLB was to enforce the teaching of the various state standards in education, and we judge education systems’ effectiveness in doing this by how well students do on tests based on these standards. Did it work? 

The most perceivable change to education that standardized testing has brought is that as a nation, we are probably aligning instruction to our state standards better than we ever have before. One of the oldest problems in education is that a major factor in how students are educated is based on whose classroom they are in. While one teacher dutifully tries to cover the standards her state and district created for her subject, another devotes lots of time to personal stories and likes to play his guitar for the kids. We’ve all had both types of teachers. Repeated use of standards based exams has increased focus on standards. Since 2001, states, districts, schools, and teachers have devoted a massive amount of energy into aligning curriculum and practice at a classroom level with the standards. The conversations and discussions that this practice produces bring out the best in those that take part in them and without a doubt bring about better understanding of the practice of teaching. And without measuring this, undoubtedly, these conversations would happen less or not at all.

When I consider with an open mind that the state tests have aligned curriculum to state standards as never before, I realize that this is actually a good thing that has improved education, so it passes the first Drucker test. Now for the second test: Are we measuring what matters most? When you consider what students want and need from their education you begin to see the weaknesses of only measuring one way. If a student can’t pass the ISAT to save her life but is enrolled in Honors/Dual Credit courses and earns a high GPA do we have a problem? If a Principal wants to raise his school’s ISAT English scores how does he motivate his 11th and 12th grade English teachers to help improve student performance on the 10th grade test? Should a school that sees enrollment in advanced courses increase year after year be shut down if test scores don’t go up? 

The point here is that the use of exams tied to state standards has improved education. If you don’t agree with the word ‘improved’, it’s undeniable that it’s changed the practice of education, but are we measuring what matters the most? NCLB brought us the focus on state standards, but now we are under the Every Student Succeeds Act (ESSA). ESSA is similar to NCLB but differs in that it allows states to create their own standards to measure education as long as they keep testing. What that means is that states can consider what they feel matters most in education and measure those as well. I’ve held the Drucker proverb as truth my entire adult life and I well know the importance of measuring the right things because not measuring something may cause you to underestimate its importance or impact. 

Educators often come out against testing because they realize that it’s a clumsy tool that doesn’t tell the whole story or measure what is most important to focus on for students. If tests don’t tell the whole story, maybe the solution is to realize that and fill in the gaps with additional measures that measure other important indicators. The law allows us to do this, and the best way to tell the whole story of what does happen in schools is to look at them in several different ways rather than one way.