Standardized Tests
Newsweek Jukes the Stats for Arne Duncan
Published May 06, 2009 @ 08:28AM PT
A lie gets halfway around the world before the truth has a chance to get its pants on.
--Winston Churchill
A couple of weeks ago it was Thomas Friedman practicing stenography instead of journalism in his op-ed on the McKinsey report; now it's Newsweek. In this week's feature article on Chicago Public Schools CEO Ron Huberman, who replaced Arne Duncan when Duncan followed Obama to take the Education Secretary position, Newsweek characterizes Duncan's stint in Chicago as follows:
Backers of mayoral control point to successes in Chicago, where 64 percent of the students met or exceeded state standards on achievement tests in 2008, compared to 36 percent in 2000. Under Duncan's leadership, test scores improved overall, and the city revamped dozens of schools, typically dismissing administrators, teachers and staff in underperforming schools, and starting over from scratch.
Chicago's Catalyst Notebook blog offers some much-needed complication to Newsweek's wholesale swallowing of Duncan's record there:
As CEO of Chicago Public Schools, Arne Duncan boasted about ISAT [i.e., Illinois Standards Achievement Test - the "state standards" assessment mentioned in Newsweek above] gains, even though the district showed poorly on the National Assessment of Educational Progress, known as the nation’s report card.
[...] According to the most recent scores, only 16 percent of Chicago’s 8th graders were proficient in reading and math on the NAEP; while more than 70 percent of 8th graders met reading and math standards on the ISAT.
So: Duncan's test scores in Chicago rose on the state tests, but stayed dismal on NAEP. Too bad Newsweek couldn't see here the same thing Diane Ravitch saw with NYC schools chancellor Joel Klein's similar boasting about New York state test performances:
On the federal National Assessment of Educational Progress [NAEP] — widely acknowledged as the gold standard of the testing industry — New York City showed almost no academic improvement between 2003, when the mayor’s reforms were introduced, and 2007. There were no significant gains for New York City’s students — black, Hispanic, white, Asian or lower-income — in fourth-grade reading, eighth-grade reading or eighth-grade mathematics. In fourth-grade math, pupils showed significant gains (although the validity of this is suspect because an unusually large proportion — 25 percent — of students were given extra time and help). The federal test reported no narrowing of the achievement gap between white students and minority students.
The city’s Department of Education belittles the federal test scores and focuses on the assessments given by New York State. And, indeed, the state scores have soared in recent years, not only in the city but also across New York state However, the statewide scores on the N.A.E.P. are as flat as New York City’s. Our state tests are, unfortunately, exemplars of grade inflation.
Brought to you by the "juking the stats" department, compliments of The Wire.
--screenshot from The Wire, HBO
What "The Wire" Teaches Us About Education
Published April 30, 2009 @ 09:12AM PT
I'm one of those non-TV-watchers who discovers great shows years later than most people. Case in point: The Simpsons. I didn't discover that show's brilliance until around 2003. It took word of mouth through trusted friends to lead me to those waters.
I'd never watched HBO's The Wire, either, until last week. This time the word of mouth was not through a friend, but through a post by change.org contributor Sharon Higgins on her always-excellent Perimeter Primate blog. On "Oligarchs, Crime, the Underclass, Neglected Schools, and more," Sharon wrote:
Watch a new interview with David Simon, former Baltimore Sun journalist and creator-producer-writer of The Wire, on Bill Moyer's Journal. He discusses a variety of things such as America’s abandoned underclass, our current oligarchy, and the high level of national apathy. In the mix, he talks about inner-city education issues and crime.
Let’s just say...he gets it.
I followed Sharon's post to the Moyers (must-watch) interview, and followed that with a week-long marathon watching every season of The Wire. The short version: not only do Simon and his co-writer Ed Burns, a former Baltimore police detective and Baltimore city public school teacher, "get it." They deliver it. In my dreams, I'd teach an entire semester-long course using The Wire as the leaping-off point to explore politics and government, poverty, the war on drugs and criminal justice, white-collar crime, the contemporary labor movement, education politics, human trafficking, LGBT issues, mainstream journalism, and more. For my money, it would be time as well-spent as spending the same number of hours reading War and Peace in an English course. The entire five seasons form more of a novel than a series of short stories, unfolding and complicating the plot through over 50 hour-long episodes. Walking students through it would lead them, I'm convinced, to wanting to understand the complexity of all these issues more.
Here's what it has to say about the politics of high-stakes state achievement tests (it's in Season 4, I think). As you watch/read the fictional Baltimore mayor meeting with his campaign managers, tell me which mayors or other edu-politicians come to mind:
You can buy individual episodes of The Wire for $1.99 on iTunes. What a world.
A Foundation of Bubbles: Deconstructing the McKinsey Report, Part 1
Published April 29, 2009 @ 06:11AM PT
The report by McKinsey and Company, "The Economic Impact of the Achievement Gap in American Schools," has generated a lot of unquestioning fanfare in the media. Arne Duncan joined NYC schools chancellor Joel Klein in a conference to launch the report, and seized on it as "proof" that we need "radical" reform in our schools.
Klein promptly highlights Duncan's speech at the event in a video on his Education Equality Project website. The makers of the report, we've already noted, are also on Klein's payroll at NYC public schools. (We also noted McKinsey was previously on Enron's payroll.)
The report itself isn't remarkable in identifying the achievement gap. That's old news. What's new in the report is its claim that the gap is causing "the equivalent of a permanent, deep recession in terms of the gap between actual and potential output in the economy" (18).
That's an argument sensational enough to make headlines, and put teachers and principals right up there next to bankers and financiers as the culprits behind America's economic woes. There's nothing like a manufactured recession to divert us from a real one, and to divert the populist anger from that real one toward the invented one.
This is the first in a series of posts that will look more skeptically at the report than the mainstream press and the Duncan-Klein camp has.
1. A Foundation of Bubbles
The McKinsey Report itself states, "In this analysis, we focus mainly on 'achievement,' which reflects the mastery of particular cognitive skills or concepts as measured through standardized tests" (Footnote 1, p. 5). Thus the crisis this report alleges stands or falls on our willingness to accept that student performance on standardized tests is an accurate measure of student value in the workforce; it further rests on our willingness to accept the notion that the primary purpose of education is to create not citizens, and not well-rounded characters, but instead to create workers able to benefit an economy that more and more does not serve the interests of the working rank and file. Think Wal-Mart.
Nobody is saying that reading, math, and future employment based on proficiency in (and by no means mastery of) these two skills are unimportant. What should be said, though, is that other traits like creativity, global awareness, the ability to learn independently instead of needing to be taught, to work well with others, to innovate, on and on, surely also benefit our economy. If we accept that, then we should have no problem accepting that the current math-and-reading standardized test fixation carries an opportunity cost for every minute taken from broader studies in order to deliver test-prep classes to "juke the stats*."
We note a related irony on page 7 of the report, which describes the Program of International Student Assessment (PISA) test upon which the study's international comparisons are based:
PISA is a respected international comparison of 15-year-olds by the OECD that measures "real-world" (applied) learning and problem-solving ability. In 2006 the United States ranked 25th out of 30 nations in math and 24th of 30 in science.
The irony? The high-stakes testing regime of NCLB has generated account after account of schools narrowing curriculum in order to focus on test-taking skills and knowing (or correctly guessing) the right answer on state tests. It's no surprise that this would produce low scores on the PISA test (which is unrelated to NCLB). PISA, as the report states, tests "real-world" application of mathematical and scientific thinking. NCLB test-prep sessions focus on the opposite of applied knowledge. I'd love to see a breakdown of how schools that performed well on NCLB-mandated state tests performed on PISA. My hunch is we'd see a picture of schools great at finding the right bubble, but horrible at applying learning and solving real-world problems.
Related: Re: "Juking the stats," if you haven't seen the Bill Moyers interview with David Simon, co-writer of the HBO series The Wire, by all means watch it. A snippet:
DAVID SIMON: Well, and facts-- one of the themes of THE WIRE really was that statistics will always lie. That I mean statistics can be made to say anything.
BILL MOYERS: Yes, one of my favorite scenes, in Season Four, we get to see the struggling public school system in Baltimore, through the eyes of a former cop who's become a schoolteacher. In this telling scene, he realizes that state testing in the schools is little more than a trick he learned on the police force. It's called "juking the stats." Take a look.
[...]
ASSISTANT PRINCIPAL: So for the time being, all teachers will devote class time to teaching language arts sample questions. Now if you turn to page eleven, please, I have some things I want to go over with you.
ROLAND "PREZ" PRYZBYLEWSKI: I don't get it, all this so we score higher on the state tests? If we're teaching the kids the test questions, what is it assessing in them?
TEACHER: Nothing, it assesses us. The test scores go up, they can say the schools are improving. The scores stay down, they can't.
PREZ: Juking the stats.
TEACHER: Excuse me?
PREZ: Making robberies into larcenies, making rapes disappear. You juke the stats, and major become colonels. I've been here before.
TEACHER: Wherever you go, there you are.
[...]
DAVID SIMON: You show me anything that depicts institutional progress in America, school test scores, crime stats, arrest reports, arrest stats, anything that a politician can run on, anything that somebody can get a promotion on. And as soon as you invent that statistical category, 50 people in that institution will be at work trying to figure out a way to make it look as if progress is actually occurring when actually no progress is. And this comes down to Wall Street. I mean, our entire economic structure fell behind the idea that these mortgage-based securities were actually valuable. And they had absolutely no value. They were toxic. And yet, they were being traded and being hurled about, because somebody could make some short-term profit. In the same way that a police commissioner or a deputy commissioner can get promoted, and a major can become a colonel, and an assistant school superintendent can become a school superintendent, if they make it look like the kids are learning, and that they're solving crime. And that was a front row seat for me as a reporter. Getting to figure out how the crime stats actually didn't represent anything, once they got done with them.
Image by Potatojunkie
Core Knowledge and Class Size
Published April 28, 2009 @ 03:14AM PT
I've been meaning to point this out ever since reading the Core Knowledge blog announce that the Carl C. Icahn Charter School is NYC's "toughest charter to get into."
Robert at the Core Knowledge blog points out:
The school had spots for less than 3% of its 868 applicants, the Daily News reports. On last year’s state ELA test, 85.1% of students were proficient, more than double the rate of the surrounding district–as good an argument for the efficacy of a content-rich curriculum on reading achievement as one could want. Math proficiency is even higher–over 97%.
Robert also points out that the school uses the Core Knowledge curriculum, and implies that curriculum is a factor in much of the school's success at those test scores (and has the school taken the more respected NAEP tests, instead of the easier NY state tests?).
I'm not going to dispute that possibility. I can get behind Core Knowledge in this respect, at least: if I understand it correctly, it pushes content-rich reading in a coherent, historically-grounded framework, instead of pushing scripted lessons and test-prep "reading" instruction. (I've already written about what I can't so easily get behind with CK.)
But there's another factor of Icahn charter that separates it from NYC public schools: class size.
From the NYC Public School Parents blog:
All classes at the school are capped at 18, according to its website and an article in the NY Sun. Classes run to 4 PM, with Saturday help for any child who needs it.
And yet this administration, which promotes charter schools at every opportunity, allowed class size to rise in our regular public schools in all grades this year but 4th – despite $150 million in state aid that was targeted specifically to reducing class size. More than 66,000 students-- or about one quarter of all NYC public school children in grades K-3 are now in classes of 25 or more– an increase of more than 11, 000 students compared to last year. There are nearly 14,000 students in grades 1-3 in classes over 28 – a 36% jump.
The size of Kindergarten classes increased so much that average class size is now as large as in 2002 – when the mayor was first elected. Next year will likely be worse – with hundreds of parents on waiting lists for their zoned neighborhood schools. See articles about waiting lists in Chelsea, Upper East side, and Greenwich village – even after increasing class size to 25 – the union contractual maximum -- in all these neighborhood schools.
The administration says it will provide 100,000 seats for charter school students by 2012 – though there are only 25,000 new seats in the entire proposed five year capital plan. This means that they are planning to take at least 75,000 seats from our already overcrowded regular public schools – with more closing of neighborhood schools to make way for charters, and higher class sizes for those kids sent elsewhere.
Everything's complicated.
Update: This blog has an interesting comparison/contrast of the Icahn school and other big-brand charters like KIPP and Green Dot. Especially noteworthy is that the teachers at Icahn aren't Teach For America naifs - and don't seem worked to the bone like KIPP's TFA-ers - but instead are professional teachers from NYC schools. Also noteworthy: the school days, weeks, and year aren't radically longer, as Arne Duncan is convinced they should be (and as they are with KIPP):
a) Nearly every charter school I've seen has a young leader (usually a TFA alum), whereas Litt is a grizzled veteran.
b) There's only a slightly extended school day -- 8:30-4 -- and no Saturdays or summer school (though maybe school started a week early in August?).
c) There are no TFA teachers on staff -- most of the teachers appear to have been recruiting from the NYC public schools (see below for one teacher's story).
d) There were few posters with slogans on the wall ("Work Hard. Be Nice.", "Climbing the Mountain to College", etc.).
e) I didn't observe any of the teaching techniques that involved all the students chanting vocabulary words, doing multiplication with "oom-pop-drop" and the like.
I wonder how the compensation and benefits work here, compared to traditional public schools?
How to Write Timed Essays That aren't Crap
Published April 27, 2009 @ 11:28AM PT
For the record, I despise the timed essay tests students have to write for such high-stakes tests as the AP English exams and the SAT, but like death, taxes, and acne, they're a fact of life for teachers and students.
The worst thing about the tests is that they promote the opposite of writing, since they tempt teachers and tutors to train students in such abominations as the five-paragraph essay. You know, upside-down triangle intro (trite generalization narrowing down to cookie-cutter thesis sentence), followed by three body paragraphs (three rectangles, each with topic sentence first, supporting examples after, then conclusion and transition), and then wiped clean with a snorer of a right-side-up triangular conclusion ("restate your thesis, then generalize out to close").
"Then flush," we should all add, if honest.
Because that kind of writing is crap. Nowhere does it exist except in classrooms, AP exams, and SATs. Most horribly, students get the idea that this mechanical form is good essay writing generally, even for take-home papers. To me, it's the job of the high school teacher to unteach the mechanical form, and grow students into the organic approach.
Better still is the challenge of teaching students to write organically on those damned timed essays themselves. Anybody who thinks a mechanical five-paragraph essay is going to stand out in a two-foot stack of five-paragraph essays on an AP or SAT scorer's desk, and gain the highest marks, should go to rehab and dry up. Or maybe read real essays beyond school.
So let me share with you a way to teach writing that was never possible before about two years ago - a way that allows the students to literally watch and hear their teacher read an AP Exam prompt, read the exam poem cold, and then write the exam. All under test conditions, within the 40 minute time limit. (The same thing could be done with history or other subjects - anything with an essay prompt.)
All it takes to do this is a computer microphone and an internet connection to such free screencast sites as screencast-o-matic.com.
Here's the first fifteen minutes of a lesson I gave to my AP Lit students last year. Since AP exams are coming up soon, it might come in handy for some classrooms or, better still, prompt teachers to make their own. Students need teachers to put themselves out there as writers, instead of adults who only talk about how writers should write.
From an old post, the background:
A few days ago, I had my AP Literature students do a timed writing of an old AP Lit essay question under exam conditions – 40 minutes to read a challenging poem and write an essay that could make or break their opportunity to get college credit for our course.
Many students had a hard time with it. Many didn’t manage to write more than half a page, hand-written – two small paragraphs – for the assignment. (The poetry essays are apparently always what they do worst on in the real AP exam, which is why we’re starting the year with six weeks of poetry.) So I did the assignment myself, with headphones and mic on, talking through each stage of my own approach to taking timed essay exams on poetry.
Here’s how it looks (but you really should take a glimpse at the AP Lit channel on Screencast-o-matic itself, because it allows comments, time-stamped notes, downloads, and more. It’s a great tool.)
(Click here for a larger, clearer version. It's a beautiful poem, by the way.)
Part 1: Attacking the question, annotating the poem:
Here are the second and third parts, in which students get inside this writer's head as he talks through and composes his essay. They hear the thoughts, they see the sentences form, all in real time. (Don't tell me technology can't improve instruction.)
Note the student feedback under part three. They suggest, among other things, that filming the process being hand-written instead of typed might be more effective. I may try that next fall. (Or I might decide they were at their whiniest adolescent worst and just need to suck it up.)
Finally, this isn't supposed to be entertainment. Asking students to sit quietly at home, listening to the teacher talking through his ideas as he reads the prompt and writes the essay for 40 minutes is asking a lot. But making them do it at least once, and reflect on what they learned about real, organic writing - and more importantly, about spending a full ten minutes or more thinking and annotating before beginning the essay at all - that's an investment most of my students said they thought paid off.
Image by Arthae
Arne Duncan to 8-Year-Old Woodrow Wilson: "No College for You"
Published April 26, 2009 @ 09:43AM PT
Connect the dots:
From CNN's (poorly titled) "10 Homeschooled Celebrities":
Woodrow Wilson studied under his dad, one of the founders of the Southern Presbyterian Church in the United States (PCUS).
He didn't learn to read until he was about 12. He took a few classes at a school in Augusta, Georgia, to supplement his father's teachings, and ended up spending a year at Davidson College before transferring to Princeton.
From an interview with Secretary of Education Arne Duncan (which I swear I've seen, but need help to cite), on the value of standards and standardized test data:
[We have to be honest enough to] look a second grader in the eye and tell them if they’re on track to get into a good college or not.
What Thomas Friedman Doesn't Say
Published April 22, 2009 @ 04:15PM PT
In this week's NYTimes op-ed, "Swimming Without a Suit," Thomas Friedman pulls an interesting move by connecting the Wall Street meltdown to - what else - America's "decline in education." When Friedman says "that's the conclusion [he] drew from a new study by the consulting firm McKinsey, entitled 'The Economic Impact of the Achievement Gap in America’s Schools'," he overstates the case. He's not drawing conclusions: he's summarizing those of the report. And he does so without the least sign of skepticism or analysis. Far more interesting than anything in Friedman's piece is what's not in it.
It's a prime example of what David Sirota noted about Friedman back in 2006:
Tom Friedman parrots the propaganda of Big Money, using his column to legitimize some of the worst, most working-class-persecuting policies this country has seen in the last century - all while bragging on television that he doesn't even bother read the details of the policies he advocates for.
Is the McKinsey report "propaganda of big money"? Judge for yourself. It claims to be neutral, but its steering committee consists of (billionaires Eli Broad and Mayor Bloomberg protege) Joel Klein, Klein's factually-challenged Education Equality Project (along with it's $.5 million mouthpiece Al Sharpton), and the anti-teacher-union Bill Gates, among others.
I find it hard to believe any such committee will steer any study into "neutral" ground - and McKinsey is a business consultancy firm, anyway. Friedman doesn't bother to share any of this info about the report's origins. He just passes it to Times readers an unchewed, undigested whole.
Friedman starts off with "a quick review":
In the 1950s and 1960s, the U.S. dominated the world in K-12 education. We also dominated economically.
Friedman doesn't steer us into other comparisons of 1950s and '60s America with today's, so let's add this data from Robert Borosage to the mix:
The Institute for Policy Studies details the staggering contrast to the Eisenhower years. In 1955, the top 400 taxpayers averaged about $12.3 million in income (2006 dollars) and paid, after exploiting every loophole imaginable, 51.2% of that in federal income tax. A half century later, the richest 400 average a breath-taking $263.3 million in income each, and pay a mere 17.2% of that in federal income taxes. (A lower tax rate than paid by most of their secretaries).
If those 400 taxpayers had paid at the same rate in 2006 as a half century earlier, the federal treasury would have collected $35.9 billion more in revenue, or enough to double the energy and transportation budget combined. No wonder Ike, clearly a stealth "socialist", could afford to build the interstate transport system.
Call me crazy, but I suspect we could make a connection between the wealth gap and the achievement gap in those halcyon days to the "decline of education" Friedman is wringing his hands about here. (But Friedman is married to a member of a family worth over $2.5 billion dollars - pre-Wall Street decline, anyway - so that wouldn't be in his interest. This is not to attack his wealth, mind you. It's just to identify his position in America's class structure, and his relevant blind spots.)
Friedman then trots out the McKinsey report's standard fare about America's ranking on international test scores:
For instance, in the 2006 Program for International Student Assessment that measured the applied learning and problem-solving skills of 15-year-olds in 30 industrialized countries, the U.S. ranked 25th out of the 30 in math and 24th in science. That put our average youth on par with those from Portugal and the Slovak Republic, “rather than with students in countries that are more relevant competitors for service-sector and high-value jobs, like Canada, the Netherlands, Korea, and Australia,” McKinsey noted.
Education scholar Gerald Bracey's critiques of such studies are as available to Friedman as they are to me, so it's anybody's question why he parrots McKinsey instead of complicating it. Here's Bracey (emphasis added):
As usual in these comparisons, Americans in low-poverty schools look very good, even in mathematics. They would be ranked third in the 4th grade (among 36 nations), 6th in the 8th grade (among 47 nations). This is important because while other developed nations have poor children, the U.S. has a much higher proportion and a much weaker safety net. When UNICEF studied poverty in 22 wealthy nations, the U. S. ranked 21st.
Friedman's fixation on test score rankings divorced from poverty rankings needs fixing. But it's standard in education punditry today. We ignore poverty, and instead only focus on schools and teachers. Friedman follows that script with his next paragraph, in which he summarizes the McKinsey report on American high schools:
Actually, our fourth-graders compare well on such global tests with, say, Singapore. But our high school kids really lag, which means that “the longer American children are in school, the worse they perform compared to their international peers,” said McKinsey.
Again, Gerald Bracey complicates the Singapore meme:
Who cares if Singapore, with about the same population as the Washington Metro Area, and Hong Kong, with about twice that number, score high? There aren't many people there. (And, as journalist Fareed Zakariya found out, the Singapore kids fade as they become adults.)....
When Zakariya asked the Singapore Minister of Education why his high-flying students faded in after-school years, the Minister cited creativity, ambition, and a willingness to challenge existing knowledge, all of which he thought American excelled in. But, as Bob Sternberg of Tufts University has pointed out, our obsession with standardized testing has produced one of the best instruments in the nation's history for stifling creativity."
Still, there is a problem with education for many of the nation's least privileged children. So how do Friedman's solutions look? More on that in Part Two.
Image by Charles Haynes
