Educators debate validity of MAP testing
The set of exams now under attack in Seattle was originally developed in the ’70s by an informal group of school administrators from Washington and Oregon, an attempt to improve upon the tests of their day.
Seattle Times education reporter
Four decades ago at the Jolly Roger restaurant in Chehalis, a group of school administrators dreamed up the Measures of Academic Progress (MAP), the set of exams that is now under attack in Seattle.
The group met in Chehalis because it sits halfway between Seattle and Portland, where most of them worked. They chose the Jolly Roger because they liked its giant cinnamon rolls.
They were mostly research and testing directors, not teachers. They were dissatisfied with the standardized tests of their time, national exams that students took whether they lived in Connecticut or California.
So they created a new set of tests they hoped would provide better information about how all their students were doing — exams measured academic growth, rather than just overall achievement.
They focused the questions on material they knew their school districts were teaching and they made sure the results could be available within days, not the weeks or months it took for national tests. At first, working with teachers, they wrote all the items themselves.
Yet now the latest version of the MAP tests, designed to be more useful, is being boycotted by Seattle teachers who say the tests are useless, sparking a debate that has resonated across the country.
The MAP itself has evolved over the years from a set of paper-and-pencil tests used by a handful of places to an online exam now used in more than 5,000 school districts across the nation. It has become a big business for the nonprofit Northwest Evaluation Association (NWEA), which reported $84 million in revenues in 2012.
The testing landscape in American education has changed even more dramatically, with the MAP now just one of many exams in an increasingly crowded and contentious arena.
In Seattle — where students at some grade levels take nine district- or state-required tests each year — some of the protesting teachers hope the boycott will fuel a national revolt over what they consider excessive testing and the growing use of test scores to evaluate not just students but schools, principals and teachers.
Others think the boycott, which started at Garfield High in January, is calculated to undermine Seattle’s new teacher-evaluation system, although the protesting teachers say that’s simply not true. The new system, for the first time, includes scores from state and MAP tests as one measure of whether teachers are doing a good job.
An advisory-task force appointed by Superintendent José Banda is now weighing whether Seattle should keep using the MAP and, if so, how.
What’s happening in Seattle “is new territory,” said Peter Hendrickson, a retired testing director who joined the Jolly Roger group in the mid-’80s, when they’d moved to Kelso, and served for nine years on the NWEA board.
“It may be a bellwether for what we will see happening in other states, when student-test scores are part of evaluations.”
Seattle appears to be one of the few large, Puget Sound school districts that use the MAP as an interim test that’s given more frequently than the one-a-year state exams.
Highline is another, but many other districts use other tests or have developed their own.
Some districts prefer such tests to be very closely tied to their curriculum, such as Edmonds, where student progress is monitored with teacher-developed tests or activities, or with exams that come with the curriculum the district has purchased.
That’s based on the philosophy that the closer assessments are tied to classroom instruction, the more useful they are for teachers, said Nancy Katims, the district’s director of assessment, research and evaluation.
The MAP, in contrast, is not even tied to one grade level. One of its strengths — and, to critics, its weaknesses — is that it’s designed to measure students’ academic level, independent of their age or year in school. NWEA says the MAP used in Washington state is aligned to the state’s overall learning objectives, but Cathy Taylor, a testing expert at the University of Washington, says it’s only partially aligned.
Seattle starting using the MAP five years ago because it wanted to better monitor student progress throughout the district.
From the start, some teachers raised concerns about adding more standardized tests to the state exams students were already taking. At most grade levels, the state already required two tests a year, one in reading and one in math, and sometimes a third in writing or science.
Back then the MAP added six more tests to that schedule — with reading and math exams in the fall, winter and spring.
Now schools have the option to give it just twice, and the district has relaxed some other requirements, too, especially for kindergartners and, most recently, for ninth-graders.
Some teachers embrace the MAP, saying it helps them know when they’re not reaching students who are behind, or ahead.
There’s no point teaching something that students aren’t ready for, or something they already know, said Bill Harris, a math teacher at Whitman Middle School.
Some also say that the test’s focus on academic growth helps inspire students who have never passed a state exam — with the MAP, at least they can see they’re making progress. And they say it helps ensure students don’t get stuck in remedial classes if their scores show they don’t need them anymore.
But others are lukewarm at best, in part because the MAP results take training to interpret, and while they point to possible areas of concern, it’s not the type of exam that can tell teachers what specific questions their students answered wrong.
They “tell you where a kid may be,” said Eric Anderson, the district’s director of research, assessment and evaluation.
The protesting teachers — roughly 100 of them in four different schools — bluntly say the MAP is not worth the time and energy it takes to give.
They don’t trust the results, saying many students don’t take the exams seriously and, at least for high school, the tests’ margin of error is as great as the number of points students are expected to gain from one testing period to another. They also say many questions focus on low-level skills, such as the names of poetry-rhyming schemes.
At a recent task-force meeting, Garfield senior Joel Thomson said his reading MAP score dropped in ninth grade, even though he felt he learned more from his teacher that year than he had in any English class before.
The district’s decision in 2009 to use the MAP in all schools also remains a sore point for some because of its link to then-superintendent Maria Goodloe-Johnson, who held an unpaid position on the NWEA board.
Goodloe-Johnson announced her board appointment publicly but failed to mention it as the school board voted on the MAP contract. A district ethics investigation cleared her of wrongdoing, but the state auditor’s office criticized her for failing to disclose her relationship before that vote.
The district has continued to use the MAP, at a cost of about $500,000 a year.
Beyond its function as a tool for teachers and principals, the MAP is increasingly used for other purposes as well.
Most significantly, the district and the teachers union agreed that scores from the MAP and state tests would be used to rate teachers on the progress their students make on those exams. A low rating isn’t factored directly into a teacher’s overall evaluation, but it means he will be scrutinized more closely.
That represents one of the expanding uses of test scores, in contrast to even 20 years ago, when few outside district offices cared much about them. Now, parents, lawmakers, philanthropists and even the federal government pore over them.
One issue is that the MAP wasn’t designed to measure teaching skill — and the Northwest Evaluation Association warns that MAP scores shouldn’t be used as the determining factor in evaluating teachers.
The association also discourages districts from using MAP scores alone to exclude students from advanced classes or other programs.
If a student has worked hard and received a good grade and a teacher recommendation “the kid should be in the class,” said John Cronin, who directs an NWEA-affiliated research center.
The next MAP-testing period is scheduled to start in late April, and it’s not yet clear whether the boycott will continue. Banda said Friday he hopes it doesn’t.
He also announced that no teachers will face discipline for the winter protest, saying none of them were directly responsible for administering the exams. At Garfield in particular, that might be because administrators stepped in to do so.
The advisory-task force is scheduled to give its recommendations in early May. It isn’t charged with deciding whether the MAP should play a part in evaluating teachers — that’s to be decided in the upcoming teacher-contract talks. But its members will recommend whether the district should continue use the MAP, and how.
Hendrickson, the retired test director who once served on the NWEA board, said the key question isn’t whether a particular test is good or bad, but whether it can support the purposes for which it’s being used, and how it fits into all the testing a district is doing.
“It’s important for stakeholders,” he said, “and that includes kids and teachers and parents and school administrators and the public at large, to think hard about what we want kids to learn, how we’re measuring that, and whether we’re getting the information we need to help teachers do the very best jobs they can.”
Linda Shaw: 206-464-2359 or firstname.lastname@example.org. On Twitter @LShawST