Percent Proficient: A Flawed Metric

 

Boston Red Sox pitchers 1916. Rating schools on % Proficient is like rating baseball teams on the amount of players that never sit out games. It’s a way to look at the team but it ignores a really glaring piece of data: How did the team do when it played against others?

Right now one of the biggest problems in education is that we can’t see the forest for the trees when it comes to student performance. What this means is that the way we look at performance is causing more harm than good when it comes to how we do school. If you’ll remember, all students at certain grade levels take tests like the ISAT, and then these scores are used to determine the efficacy of teachers and schools. Although there are problems with just one test or any metric by itself ‘defining’ schools and teachers, another big problem is how we look at test performance itself. Right now, the school with the highest percentage of students ‘proficient’ on the ISAT is deemed the best, but this isn’t the only way to look at performance and in fact this actually hinders how educators deal with students. 

The % Proficient System: ‘Proficiency’ is when students hit a certain cutoff score (for ISAT ELA and math it’s a score of 2500 or better on a 3000 pt scale); all students above 2500 positively affect a school’s ISAT rating and all students below 2500 negatively affect the rating. Although no one in schools is allowed to see the ISATs and have no study materials to prepare students for them, we do have “ISAT Interim” (practice) exams that we can give to students that will give us a good idea of whether the student is proficient or not before they take the test that counts. So when the % of proficiency is the most important thing, you do nothing with all students who look like they’ll pass the ISAT and you isolate the group that won’t pass the ISAT and either increase their exposure to the ISAT interim or give them some kind of remedial instruction in hopes to bring them up to speed before test day.

Important: remember that the % of proficient students is a school’s test score? It’s not about how kids across the board do on the exams, it’s about how few kids don’t hit that 2500 score. This means that rather than focusing on skills, you increasingly focus on individuals: student is doing well, do nothing with her; student is doing poorly, have him spend more time with ISAT practice and remedial courses, and at the high school level, maybe transfer him to an alternative school if necessary. Under the proficiency system, educators don’t care about kids who pass tests because they don’t get dinged for those students. Under this system, the focus goes only to those not passing because each student not proficient is deemed a problem. 

NCLB and ESSA, the federal acts that support our testing regimen may require us to look at it this way, (and that's one of the biggest reasons why these acts keep running into so many problems) but this is not the only way to look at exams. Remember how each school is given a ‘score’ for their exam performance? What if we used ‘Average Test Score’ instead of ‘% Proficient’? What is the difference and how does that change things?

Average Test Score System: Under this system, you total up all of a school’s test takers’ scores and divide by individuals to produce a number that is given to a school as their definitive test score. Remember that we created these tests to judge schools’ efficacy? In that case, look at the school’s average vs the state average; schools at or above state pass and those below fail this measure. In other words, you can rate schools this way as easily as the % proficient method does, but how schools try to raise their scores is different and better. 

If your average school score is important, the huge shift is that rather than looking at the individual you look at the skills assessed in the test. What parts of the test are students scoring highest on, what parts are they scoring poorly on? You break down the tests into the various skills covered, figure out where the problems are, and give your teaching staff materials to supplement curriculum in these areas. The reason you do it this way is now ALL student test scores count toward the average, both those doing poorly and those doing well. Many books on management tell supervisors to focus on their highest performers rather than lowest because this is actually where the most growth occurs. To sum up, under the average score system, we care about the performance of all students rather than just those who are failing the exams. It also takes pressure off both students and teachers in scenarios where students just don’t do well on the exams. Winston Churchill couldn't pass a test to save his life and he did OK in the end.

Education has been in the ‘% Proficient’ mentality for the last 20 years and the net effect of this is that nobody cares about proficient or advanced students. You eventually come to realize as a teacher that good teaching affects students differently depending on where they are: Sometimes a good teacher transforms an F student to a D student...that’s growth. Other times you take a C student and transform them into an A-B student. And other times you move a student who earned A’s and B’s in regular ed courses into advanced, college prep courses...that too is growth, even if they now earn C’s in advanced courses. In other words, our job as educators is to challenge all students to move from wherever we find them to a higher place. ‘% Proficient’ mentality works directly against that because it forces us to focus only on those students not attaining the cutoff score. 

The 'Percent Proficient' idea has been a cancer to education for the same reasons the ‘Super Teacher’ idea has: it causes us to focus only on one group of students rather than all students just as the Super Teacher idea causes us to focus on teachers rather than looking at what happens in schools holistically and scrutinizing all educators rather than just teachers. If you want something improved you measure it, but deciding what to measure and how to do it are critical questions that will absolutely affect how successful you are in improving something.