The Narrow, Myopic Vision of the Solidarity Caucus

7/14 postscript on doublethink below.
A novice United Federation of Teachers (UFT) caucus, Solidarity, is making a premature run for the 2016 union election. Much attention has been given to related personal controversies. This article takes the opportunity to look at the caucus, its origins and its campaign program or platform.

The caucus had in its genesis an intense opposition to both the MORE Caucus and the Unity Caucus statements of empathy for Eric Garner, who was killed in a chokehold by New York Police Department officers. In the nearly one year since, deadly police brutality, particularly against people of color, has become the civil rights issue of the nation. People from across the political spectrum were shocked and quite disappointed when a grand jury issued no indictment against any of the officials involved in killing Garner. This is important to recall when considering the character and principles of the caucus.

In its founding it insisted on staying narrowly focusing on teacher issues and on eschewing any attention to any other concerns beyond teachers' issues. Yet, while it appropriates the images of Albert Shanker, Unity Caucus leader of the UFT, it ignores that more largely, the labor union movement has taken stances of solidarity with civil rights activists and anti-war activists. It ignores that in 1957 the American Federation of Teachers dismissed locals that refused to desegregate. It ignores that the United Auto Workers, the AFT and other unions marched in solidarity in the 1963 Civil Rights March on Washington.  It ignores that strikes have had their greatest popularity when they had reached out to the community. Witness the work that the Lawrence 1912 Bread and Roses textile strikers did in reaching out to the community prior to their strike. Witness the parallel work that the Chicago Teachers Union has done when it took a year of community outreach in preparing for its 2012 strike. The CTU won widespread parent support after making sure to have a two-way dialog to understand the concerns of parents and students, instead of simply pursuing an employee-centered contract and strike campaign. Teacher-community partnership continues to this day in other examples such as St. Paul, Minnesota

Hypocritically, in the quest for allies for its caucus once it had an election in mind, it opportunistically parted from the caucus' founding principle to eschew larger issues or the community and solicited support from a community anti-high-stakes testing group, Change the Stakes.

Now, let us look at the Solidarity Caucus' platform and what is missing

While it gives attention to the special needs of Absent Teacher Reserve teachers (ATRs) where others are silent, this is almost where its issue attention ends. It does not look to the experiences of the regularly assigned teachers. One wonders if the authors even consulted regular classroom teachers. There is no specific discussion about the Danielson Framework, the New York State-dictated Measures of Student Learning (MOSLs) or other aspects of New York City's Advance teacher evaluation system. There is no attention to structural dynamics such as mayoral control, which determine the parameters of what the City Department of Education can do. There is no attention to the Common Core State Standards, which are alienating teachers from their profession and are alienating students from learning. There is no attention to the parent-student solidarity movement which is so essential for teachers who are suffering under the over-testing regimen, the opt-out movement. 

The caucus platform includes complaints about the use of tests in value-added metrics to evaluate teachers. However, the platform does not recognize how the testing emphasis is directly linked to other larger issues, some of which are issues that lead directly to the excessing of teachers into the ATR category: the DOE's testing focus narrows the latitude of what teachers can do in the classroom, the testing focus on core academic subjects crowds out attention to non-core subjects such as foreign language, the arts or physical education. Moreover, the platform studiously ignores larger social-economic-political realities. It ignores that there is a corporatist campaign by politicians in both major parties and an economic privatization campaign by privatizers that capitalize by the test-based stack ranking (rank ordering) of teachers and students and overall privatization of curricula, tests and schools.

Additionally, the caucus program's focus on the teacher involves an ignoring of the effect of the high-stakes tests on students themselves. It ignores the emotional stresses upon students and the attendant alienation from the schooling process in this overall climate of high-stakes testing and driving non-tested subjects out of the curriculum. Just as a decent doctor would care about a patient personally and their living and working conditions and how those impacted the person's health, one would hope that teachers would care about their student charges and the factors that impacted on their life. Just as one commenter on a blog in reaction to the formation of the caucus by saying that he cared about the living conditions of the students, I would echo those points. Teachers are choking under the attacks by corporatist politicians and by privatizatizers, and more rank and file teachers are recognizing this. Yet, Solidarity has made a big conceptual error to ignore this larger context in their program. In this sense, it is mirroring the conceptual errors that conventional, mainstream unions or caucuses, that Michael Mulgrew and the Unity Caucus makes.

Teachers ought to consider the larger social conditions impacting on students. As emphasized previously on this blog, income disparities are extreme in this nation and within the nation, income inequality is more extreme in New York city and state. (See this posting re inequality in the city; and regarding poverty impact on learning see introductory comments at this blog posting.) The stresses of race and class inequality impact on students' lives and negatively impact their educational performance. From issues of police mistreatment of youths of color to profound personal emotional stresses of poverty, these experiences inform the lived reality of the children that we teach in our classrooms. If we wish our students to have the best educational experience, we as teachers should not only attend to our own craft, our own performance, but we should also be concerned with recognizing the social inequality; we should work to change that inequality. This is why I am a social justice education unionist.

POSTSCRIPT:
There are many aspects about the organizing tactics of the key personalities in the caucus that are at once deceptive and deeply disturbing.  Most recently, a social media page connected with the group has been created. A Solidarity page headline boasts an army of 10,000. It turns out that the organizers of the page snatched up 10,000 names and made them --unwittingly, mind you-- into "supporters." This is putschist organizing at its basest.
New exploitative techniques in political deception on Facebook:
Just when you thought it could not get worse. The leader of Solidarity has taken the names of a few dozen well-known opponents of Michael Mulgrew and posted their names in Facebook postings. The risk is that the uninitiated would think that these persons endorse the Solidarity cult.
Doublethink alert:
The Solidarity leader ignores that the New Action caucus supports Michael Mulgrew in every election in NAC's quid pro quo for executive board seats, and he cries foul when Mulgrew and Unity opponents challenge his desire to ally with NAC, questioning whether they are truly an opposition caucus. Yet, Solidarity claims to oppose Mulgrew and Unity. Thus, this is a classic example of doublethink: the ability to hold two contradictory beliefs at the same time, especially used in political indoctrination. If you cannot be clear how Solidarity leaders think, can you be clear where their true convictions lie?

Scientific American: "Learning Styles" Could Be Invalid Theory

Incorporating "learning styles" is conventional in teaching techniques these days, but does it hold up under empirical scrutiny? And if it is shaky, should your supervisor expect you to incorporate attention to multiple learning styles (with differentiation six or so times over)? Furthermore, if this is dubious then this throws into question one of the Charlotte Danielson Framework tenets. Let's hope that NYSUT attorneys attend to contrasts between theory and empirically demonstrable reality.

Is Teaching to a Student’s “Learning Style” a Bogus Idea?

Many researchers have suggested that differences in students’ learning styles may be as important as ability, but empirical evidence is thin

By Sophie Guterl, Scientific American, September 20, 2013

Ken Gibson was an advanced reader in elementary school and easily retained much of what he read. But when the teacher would stand him up in front of the class to read a report out loud, he floundered. His classmates, he noticed, also had their inconsistencies. Some relished oral presentations but took forever to read a passage on their own; others had a hard time following lectures. Gibson now explains these discrepancies as “learning styles” that differ from one student to the next. He founded a company, LearningRx, on the premise that these styles make a difference in how students learn.

The idea that learning styles vary among students has taken off in recent years. Many teachers, parents and students are adamant that they learn best visually or by hearing a lesson or by reading, and so forth. And some educators have advocated teaching methods that take advantage of differences in the way students learn. But some psychologists take issue with the idea that learning style makes any significant difference in the classroom.

There is no shortage of ideas in the professional literature. David Kolb of Case Western Reserve University posits that personality divides learners into categories based on how actively or observationally they learn and whether they thrive on abstract concepts or concrete ones. Another conjecture holds that sequential learners understand information best when it is presented one step at a time whereas holistic learners benefit more from seeing the big picture. Psychologists have published at least 71 different hypotheseson learning styles.

Frank Coffield, professor of education at the University of London, set out to find commonalities among the many disparate ideas about learning style using a sample comprising 13 models. The findings, published in 2004, found that only three tests for learning styles met their criteria for both validity and reliability, meaning that the tests both measured what they intended to measure and yielded consistent results. Among the many competing ideas, Coffield and his colleagues found no sign pointing to an overarching model of learning styles.

 . . . .

Daniel Willingham, a professor of cognitive psychology at the University of Virginia and outspoken skeptic of learning styles, argues that Gibson and other cognitive psychologists are mistaken to equate cognitive strengths with learning styles. The two, Willingham says, are different: Whereas cognitive ability clearly affects the ability to learn, an individual’s style doesn’t. “You can have two basketball players, for example, with a different style. One is very conservative whereas the other is a real risk-taker and likes to take crazy shots and so forth, but they might be equivalent in ability.”

As Willingham points out, the idea that ability affects performance in the classroom is not particularly surprising. The more interesting question is whether learning styles, as opposed to abilities, make a difference in the classroom. Would educators be more effective if they identified their students’ individual styles and catered their lessons to them?

 . . . .

Harold Pashler of the University of California, San Diego, and his colleagues searched the research literature for exactly this kind of empirical evidence. They couldn't find any. One study they reviewed compared participants’ scores on the Verbalizer–Visualizer Questionnaire, a fifteen-item survey of true-or-false questions evaluating whether someone prefers auditory or optical information, with their scores on memory tests after presenting words via either pictures or verbal reading. On average, participants performed better on the free-recall test when they were shown images, regardless of their preferences.

Some studies claimed to have demonstrated the effectiveness of teaching to learning styles, although they had small sample sizes, selectively reported data or were methodologically flawed. Those that were methodologically sound found no relationship between learning styles and performance on assessments. Willingham and Pashler believe that learning styles is a myth perpetuated merely by sloppy research and confirmation bias.

 . . . .

[Read more at the original Scientific American website: http://www.scientificamerican.com/article.cfm?id=is-teaching-to-a-students-learning-style-a-bogus-idea&WT.mc_id=SA_CAT_SP_20130923]

What will real regime change look like at the DOE?

*Will de Blasio battle the toxic character of the NYC Department of Education or will he peddle old Bloomberg wine in a new "progressive" bottle?

New York City mayor-elect Bill de Blasio will face a teaching force that is the most demoralized and is eager for qualitative change. The NYC Department of Education under Bloomberg pursued a program of destruction of education and the teaching profession. Just look at the names of some of the blogs in this city, Assailed Teacher, Under Assault, Pissed Off Teacher, Exasperated teacher. The sheer volume of New York City oriented blogs alone (see the sidebar of this blog) suggests a definitely high degree of weariness.

For the good of education, and out of humanity to the people laboring in the schools, great changes are need to reverse and correct the damage. And the battle-scarred teachers will be extra-attentive to the quality of changes that the new mayor will make. There is a tall order of tasks for that he will have to attend to.

I would want to say that I am cautiously optimistic. I am not. I am guarded. The hints in the last couple of weeks have been discouraging. First was his identification to the business community that he is a fiscal conservative. Then this week we have his characterization of ruthlessly anti-labor Chicago mayor Rahm Emanuel as a great mayor. Union activists are well aware of the latter's record. Such a characterization of Emanuel sound like fighting words targeted against the New York's municipal workers. De Blasio now has a negative reputation to undo. So far, it looks like we, the democratic left, have been thoroughly duped.

The Bloomberg record and the challenge of regime change
In schools, the break-up of large schools meant that students no longer enjoy the opportunities for English electives other than the test-prep treadmill. Schools no longer offer foreign language choices other than Spanish. (Actually, in the recent era, even Spanish has been scaled back as an option in the middle schools.) It meant the loss of arts choices. It meant the closure of school libraries. It meant the drastic reduction of guidance counselors, social workers, school psychologists. Envision the thousands of youths harmed by being deprived by these staff eliminations. On many points it has meant the city violating state law in pursuit of these policies.

It meant a racist and classist policy, as these changes were pursued first and more thoroughly in minority and low-income neighborhoods. Truth be told, the break-up of schools into different "learning communities" within one building was pioneered in the 1990s, before Bloomberg, such as with the closure of Julia Richman High School in 1993 during David Dinkins and Erasmus High School in 1994 during Rudy Giuliani, when the schools were run by the Board of Education.

It meant the loss of departmental chair assistant principals or department coordinators to guide curricula in the middle and high schools. Now, in the smaller schools, there are one or two assistant assistant principals. Good luck having an observation with someone that has taught or understands your subject.

Beyond the school-break-up trend, the Department of Education changes meant the replacement of the schools' leaders, professional, seasoned educators, with businesspeople, utterly unfamiliar and unconcerned with issues of pedagogy. Having businesspeople run schools has made as much sense as car manufacturers running farms. Their singular concern with metrics and the bottom line has often led to schools ill-serving authentic educational goals. Again, fat chance on getting an empathetic observation, as the private sector originating administrator weighs in on what good teaching looks like.

It meant replacing the traditional career trajectory of seasoned educator moving onto AP, moving onto principal, moving onto superintendent. These people knew what it was like to teach in the classroom, to get to know students as they watched students progress through the grades in the school. Now, you get people that stick around for two years at best in a position before getting kicked upstairs.

It meant replacing a public institution's leadership with private, unaccountable, non-public bid (or no-bid) "networks" that have questionable practicality, as non-educators or people with fleeting classroom experience (and little or no content familiarity) inform teachers about proper teaching. Regime change will involve terminating this privatization boondoggle staple of the Bloomberg years. This was just one example of how the NYC DOE has been following the Broad Virus program.

The system, from top to bottom is poisoned and corroded. The institutional damage needs a complete rapid turnover to past practices. Even by establishment metrics there is no quantitative or qualitative evidence that the changes of the Bloomberg mayoral control years have improved schools, the teaching or learning environment.

This system has been one of repression. Edward Stancik passed away, but his spirit has continued, with an alphabet soup of investigation agencies. Real regime change and an unoppressive teaching climate will require the end of a no due process system for teachers. And as blogger Chaz has said, administrators can do all sorts of unseemly things and remain in the system, barely punished. Read his blog, Chaz's School Daze, for dozens of illustrations of such scandalously double-standard treatment.

The next mayor would do well to study this pattern of destruction and undo each and everyone of these destructive policies.

Alas, merely a change at the top, in and of itself, will not create changes. The mayor shall be gone, but principals will remain. Outgoing New York City Michael Bloomberg made changes that will last long. Even if broken schools re-unite, there is the issue of his thorough stamp of appointment of Leadership Academy principals and assistant principals. These new style principals are too often not professional educators (or have a token two years exposure to teaching). Their orientation is often decisively anti-union, creating a hostile work environment, routinely violating contract provisions and even standard labor law. Even a temporary wage-earning employee in Macy's has a duty-free lunch, as it is state law. Not so in many New York City schools. Many of the new style principals flagrantly ignore that and insist that teachers attend meetings or work in "inquiry groups." Really, many teachers work during lunch on their own choice, but that should be the teacher's choice.

Finally, all of the above is really an indictment of the leadership the United Federation of Teachers (UFT), the New York City teachers union. The union's Unity leadership has not recognized the above transformation, it has not frontally challenged these violations. It has failed to recognize or challenge the not just the whole set of patterns violations but many of the individual violations.

Teachers in touch with the chapter leaders would have heard from president Michael Mulgrew and his statements at the Thursday, September 12 chapter leaders meeting at the Brooklyn UFT headquarters. (If your CL did not debrief you, read this report here. Added commentary is at Perdido St. School. Other commentary on the UFT leaders' defense of Advance APPR at NYC Educator, and this post has an important discussion of how Danielson will turn NYC teachers into factory workers: Assailed Teacher. Many teachers have not had a proper explanation of the new evaluation system, and would do well to read the thorough analyses from the MORE site, here and here, of what is involved. The uninformed veterans and new teachers are in for a shock.

President Mulgrew's mood and stance has been that the new Advance APPR teacher evaluation system is the greatest thing to have happened. The UFT, when speaking of observation patterns has often said if you see any violation issues, such as principal-imposed lesson plan formats let us know. This is outrageous. If they were really in touch with their chapters and were mobilizing their chapter leaders, they would know that these sorts of violations have been going on for a while. Witness many principals' declaration that the Danielson Framework was official and that they were implementing it, before Walcott made the official announcement. (A succinct overview of Danielson developments approaching late January 2013 can be read in this Ed Notes post.)

The naivete that Mulgrew evinces, most prominently in his emphasis that many properly behaving principals will follow the Advance program fairly. Many principals will, but hundreds will not. Mulgrew and the whole (Unity caucus) leadership's naive acceptance of this program, and their ignorance of rampant contract violations are expressions of a union leadership far removed from the classroom. Their naivete is predicted on naivete and ignorance of teaching conditions.

This lack of knowledge of how the Danielson Framework of observation will work in real world contemporary conditions is a case in point. For instance, Danielson, Mulgrew, and others are failing to consider factors that affect smooth teacher performance. In broken up schools in particular, space is at a premium. Whereby a teacher in pre-Bloomberg, pre-broken-up schools would have had their own classroom, that they designed and that they had arranged their materials in; the conditions are far worse in the split up school format. The teacher moves room to room, either keeping multiple sets of supplies or moving them. And teachers are to be judged on classroom environment in these conditions?

Their view of the classroom is from the halcyon pre-Bloomberg years, when the conditions, while imperfect, were far more often in keeping in keeping in with the contract. The more frequent condition these years are schools with no chapter leader or no teachers lounge which is generally off-limits to administrators. There has been such turnover in the schools in the last six years that the night and day contrast of teaching conditions between then and now is imperceptable to maybe half the city teaching corp today.

Teachers' roles in effecting change
If the NYC teachers reading this are disturbed by the issues above, they should come to the Movement of Rank and File Educators, MORE, caucus (democratic alternative UFT caucus) events. A prime campaign is a moratorium campaign against the evaluation system.

Finally, Bill de Blasio has spoken of two cities. In the United States there is tale of two standards: a standard against urban schools with a large percentage of impoverished residents and suburbs. The latter have democratic-republican-driven school policy, the former do not. From a democratic-republican perspective, real regime change will require the city and the state allowing New York City residents to have the right to democratically organize their school system, just as suburban schools do. In short, the city must have a popularly elected Board of Education. Right now, this is a separate but unequal standard for municipal democracy, biased against New York City. The legal myth of the Department of Education must end with the Bloomberg regime. Or will de Blasio perpetuate the bias against urban schools having popularly controlled school systems? Will we have real regime change? Or a Bloomberg fourth term, under a Democratic face? Many of us one-time partisans of Democrats are cynical because of the terrible betrayals and oppression coming from Democrats in recent years.

If de Blasio fails to make a 180 degree turn from Bloomberg's record and his own recent rhetoric and steer the school system away from the disastrous policies of the Bloomberg decade, he will put himself in the position of being challenged from the left in 2017. --Perhaps from Letitia James (incoming public advocate), the woman who ought to be mayor.

Perdido Street School put the matter most astutely yesterday:

Tish James, responding to questions of what happens if de Blasio governs as a corporatist like Andrew Cuomo:

 “I don't think he's going to have much wiggle room,” says New York City councilwoman Letitia James, who will replace de Blasio as public advocate come January. “I'm going to make sure that he stays focused on a progressive agenda.”

Teachers, Feel Oppressed By Danielson, APPR? HBS Researcher Links Modern Micro-Management Policies to U.S. Slavery Practices

Forbes magazine --of all places!-- has published a story that reports on a Harvard Business School researcher that has produced a story on modern business management practices and slavery. While the story does not raise issues of applicability to teachers, we can clearly read penchant for micro-management as having clear applicability to the experience of school teachers in the United States today. The micro-management problem is particularly acute in public schools as teachers are held accountable to test scores, when attention to test scores ignores the influence of external social factors or school administration mismanagement.

This study inherently raises parallels with modern tools against teachers, such as Charlotte Danielson's Danielson Framework (which assumes compliant, attentive, intellectually precocious students shall be the norm, evidently, showing little awareness of real world living and working conditions -notice how she has never publicly divulged the specific grade levels at which she taught, her tenure or service or the district in which she worked). Additionally, the HBS researcher's report resonates with Bill Gates/Microsoft-informed stack ranking that enables Jack Welch/GE-style yearly 7 or 10 percent workforce shedding. 

Incidentally, Aaron Pallas of Columbia University's Teachers College reports in "Ratings madness: Are there really no highly effective elementary or middle-school teachers in Syracuse?" that no elementary or middle school teacher in Syracuse public schools rated highly effective, as indicated in that school system's cumulative rating report. Pallas looked carefully and found that the culprit lay in the school-wide local measures. They dragged down the rating for individual teachers. Can we really anticipate that the Annual Professional Performance Review, APPR, implementation will be fair and appropriate? 

Katie Johnston, in Forbes, January 16, 2013

The Messy Link Between Slave Owners And Modern Management

Caitlin C. Rosenthal didn’t intend to write a book about slavery. She set out to tackle something much more mundane: the history of business practices. But when she started researching account books from the mid-1800s, a period of major economic development during the rise of industrialization in the United States, Rosenthal stumbled across an unexpected source of innovation.

Rosenthal, a Harvard-Newcomen Fellow in business history at Harvard Business School, found that southern plantation owners kept complex and meticulous records, measuring the productivity of their slaves and carefully monitoring their profits—often using even more sophisticated methods than manufacturers in the North. Several of the slave owners’ practices, such as incentivizing workers (in this case, to get them to pick more cotton) and depreciating their worth through the years, are widely used in business management today.
As fascinating as her findings were, Rosenthal had some misgivings about their implications. She didn’t want to be perceived as saying something positive about slavery. On the contrary, she sees her research as a critique of capitalism—one that could broaden the understanding of today’s business practices.

The work is part of her current book project, “From Slavery to Scientific Management: Capitalism and Control in America, 1754-1911,” and the forthcoming edited collection Slavery’s Capitalism.

The evolution of modern management is usually associated with good old-fashioned intelligence and ingenuity—”a glorious parade of inventions that goes from textile looms to the computer,” Rosenthal says. But in reality, it’s much messier than that. Capitalism is not just about the free market; it was also built on the backs of slaves who were literally the opposite of free.

“It’s a much bigger, more powerful question to ask, If today we are using management techniques that were also used on slave plantations,” she says, “how much more careful do we need to be? How much more do we need to think about our responsibility to people?”

The next part raises the topic of absentee ownership; this brings to mind the way that current policy is set by people that do not live in the affected districts, by people that do not send their children to public schools (Bill Gates, for one, on both counts)

Absentee ownership

According to Rosenthal, the history of detailed record-keeping on plantations goes back to at least the 1750s in Jamaica and Barbados. When wealthy slave owners in the West Indies started leaving others in charge of their plantations, she found, they asked for regular reports about how their businesses were faring. Some historians see this rise in absentee ownership as a sign of decline, but it is also among the first instances of the separation of ownership and management, Rosenthal says—a landmark in the history of capitalism.

Slave owners were able to collect data on their workforce in ways that other business owners couldn’t because they had complete control over their workers. They didn’t have to worry about turnover or recruiting new workers, and they could experiment with different tactics—moving workers around and demanding higher levels of output, even monitoring what they ate and how long new mothers breastfed their babies. And the slaves had no recourse.

“If you tried to do this with a northern laborer,” Rosenthal says, “they’d just quit.”

Here we find ourselves thinking of the emphasis of metrics over all things student and teacher-related. And on competition and pitting one against the other, we can think of merit pay. On collective punishment, we can think of New York City's mad Measures of Student Learning, which punish gym teachers on the basis of an entire science department's test scores. Has anyone ever considered that many teachers demeaned under the new metrics will "just quit"?

Looking forward

This led owners to experiment with ways of increasing the pace of labor, Rosenthal explains, such as holding contests with small cash prizes for those who picked the most cotton, and then requiring the winners to pick that much cotton from there on out. Slave narratives describe how others used the data to calculate punishment, meting out whippings according to how many pounds each picker fell short.
Similar incentive plans reappeared in early twentieth-century factories, with managers dangling the promise of cash rewards if their workers reached certain production levels.

Planters also used group incentives to encourage honesty, doling out a barrel of corn to each hand with the caveat that if anything was stolen from the farm and no one turned in the thief, double the value of that corn would be deducted from each of their Christmas awards. Collective penalties would later be adopted by salesmen and companies like Singer Sewing Company to encourage workers to police one another.

 . . .

These account books played a role in reducing slaves to “human capital,” Rosenthal says, allowing owners who were removed from day-to-day operations to see their slaves as assets, as interchangeable units of production in a ledger, instead of as people.

 Again, on human capital, we can think of the objectifying conception that education policy rulers have of students.

MOSL, Collective Punishment in the NYC Teacher Evaluation System -Portelos Got the Eval Proposals

*Postscript: the DOE, UFT Evaluation Proposals, acquired, via FOIL! -scroll to end.

Queens high school teacher Arthur Goldstein writes, posted at Diane Ravitch's blog, as "Our Thoroughly Insane Teacher Evaluation System," of the absurdity of the Student Learning Objectives, otherwise in New York City called Measures of Student Learning (MOSL). He wrote of how some teachers will be evaluated by the test scores of students in areas they do not teach. [Notice how the NYC Department of Education and the union, the United Federation of Teachers (UFT), have been more recently using the official name, "Advance." --Probably, among other reasons, to reduce the number of people using the politically charged phrase, "King's imposed teacher evaluation system."]

 For state measures, some were mandated to reflect individual classes of teachers. In those cases, we opted to have department results reflect the local measures. In other words, your department Regents results could be the local measures. This would reflect not only the exams in areas you teach, but those given by your entire department. For example, if you teach algebra, the results in geometry and trigonometry will also be part of your local measures.

 . . .
If your subject, like music, art, or physical education, does not have a state assessment, your evaluation would be based on schoolwide state tests. Your local evaluation would be based on the lowest third of schoolwide state tests.

Make no mistake, this MOSL plan is collective punishment. Of course, low performance is not always attributable entirely to a teacher. The test can be poorly written. The school administration can be slipshod. The student and the community could face very difficult social circumstances.
Yet, no matter. Any teacher outside of the state-measured area will be punished for the so-called mistakes of the other teachers.

The nasty irony is that teaching to the test to ensure good test scores will not work. Yet, administrators insisting on teaching to the Common Core will want instead Danielson-compliant, Common Core-compliant lessons. Heads, you lose, tails, you lose.
A very appropriate post at Ravitch's blog in June, 2013: Carol Burris: John King’s Absurd Educator Evaluation Plan.

As Perdido Street has been chronicalling, Mulgrew has been defending APPR since May 30, before the imposed evaluation deal, defending every bit of it from Value-Added Modeling, VAM to the SLOs/MOSL. [See also his more recent post on how the new "Advance" system turns teachers into factory workers.]
And surreal that in late August and the first week of September, the UFT has been frantically trying to repeat the myths, that Common Core is a necessary and imperative curriculum, that the Advance APPR system is the best way to evaluate teachers. Most recently, at the chapter leader meeting at its Brooklyn headquarters president Michael Mulgrew went through great effort to tout NYSED Commissioner John King's imposed system as just wonderful, and that it the new system will get principals to be more fair.

The story has seeped into one mainstream education news site, Gotham Schools, September 16, 2013, which provided news that illustrates how teachers in non-tested subjects will be subjected to the effects of the test, "Instead of telling teachers apart, new evals lump some together."

Note the details in Gotham Schools' photo caption: Teachers attended training sessions about the city’s new teacher evaluation system over the summer, but some features of the system – including how teachers whose students don’t take state tests would have their student growth measured — were not decided until the school year began last week. [How ludicrous and sneaky that the DOE only announced or determined the procedure until last week. And the UFT still promotes this system, on principle? See important note on the MORE caucus at the end of this post.]

[A]cross the city this year, thousands of teachers will be rated in large part based on test scores of subjects and students that they do not teach.

[T]he scenarios are examples of how schools have tried to comply with a new teacher evaluation system that must factor student performance into final ratings. They also represent how the original purpose of the evaluations, to differentiate teachers’ effectiveness, has been squeezed by restrictive state laws, limited resources, and a tight timeline for implementation.

“It’s insane to me that 40 percent of my evaluation is going to be based on someone else’s work,” said Jason Zanitsch, a high school drama teacher who will share the same “student growth” score with colleagues in his school this year.

An incomplete evaluation system, implemented rapidly

Sixty percent of teachers’ ratings this year will come from observations by administrators. The state’s evaluation law mandates that the remaining 40 percent come from a combination of state tests and assessments chosen by each district, whose scores are all crunched to determine student growth.

But neither kind of test exists for Zanitsch and other drama teachers, at least this year. They are among the thousands of city teachers for whom the state has not approved any way to measure student learning. They include librarians, 5,000 physical education and arts teachers, and others who teach foreign languages, health, and career education.

New York City principals had until the first day of school last week to choose from a menu of limited options, first made available in early August ["Still no word from city on final details of teacher evaluation plan", August 1, 2013], for evaluating their teachers on student growth. Principals and teachers told GothamSchools that their schools have picked a ”default” option in which all teachers — even core subject teachers — will receive the same score cobbled together from all of the state tests taken in the school.

“What we are advising most of our schools and principals this year is since the principal’s rating is based on how their school collectively is doing, just take the default, especially since it means the minimum of extra work and testing for everyone,” said a person who works in a network with many high schools.

The arrangement has drawn a lawsuit in Florida ["Florida, Teacher Lawsuit Could Spread to Other States", Governing.com May 14, 2013] and criticism from dozens of city principals who last week pledged not to help execute it ["First Day of School Means Start of Controversial New Teacher Eval System", ]. But in lieu of state-approved assessments for all subjects, officials say rating teachers by their colleagues’ scores is the best option available until more credible alternatives can be developed.

This will no doubt drive teachers away from serving in challenging schools. And so, the schools will get further labeled as failing.

Collective punishment. We can call John King and his local collaborators the 21st century barbarians.

New York City teachers, band together: The MORE caucus of the UFT is pursuing a campaign for a moratorium on the use of the teacher evaluation system --text and online petition. This writer has signed. Will you? Visit the MORE site for the analyses of the new Advance teacher evaluation program. Download the hard copy of the moratorium petition and collect signatures of colleagues. Share with chapter leaders and delegates. Keep posted to the MORE site, for upcoming events (such as its first general meeting, tomorrow, Sat. 9/21, 12 - 3 pm, 224 W St., 14th Floor) to learn more about MORE and to get plugged into the moratorium campaign.

A commenter on the above blogpost at Diane Ravitch's site (Our Thoroughly Insane Teacher Evaluation System) has this suggestion on positive action teachers and supporters outside the classroom can take:

LISTEN to me! I know there is something at least pro-active you can do. Contact Carol Burris or Sean Feeney of New York Principals. They are collecting data and anecdotal data of teachers in NY State who have just received their evaluations for the first time.
Yours is a CLASSIC TEXTBOOK example of the hideous lies and injustices of this new junk science system. Write down your situation and literally send it to John King, every member of the Board of Regents, and Merryl Tisch. Send it to EVERY newspaper, local and national, hardcopy and online. Send it to your local and even the stinky NYSUT union. 

Carol and Sean can be reached at:
burriscarol@gmail.com
scfeeney@newyorkprincipals.org
I am getting my evaluation results this Tuesday, and I would not at all be surprised if I face this same situation as you.
Don’t despair!!!!!!

Postscript: Francesco Portleos secured the evaluation proposals by Freedom of Information Law requests, shared at his blog, Protect Portelos:

FOILed NYCDOE and UFT Teacher Evaluation Proposals Acquired. Let’s Compare.
What did the UFT Want? What did the NYC DOE want? What did Mayor Bloomberg want?
…and how did those topics differ from what NYSED Commissioner Dr. John King wanted in the end? 

I FOILed those May 8th proposals on May 22, 2013. Although there was some delay and loss in communication, the FOIL officer at NYSED, just emailed me both proposals. I withdrew my appeal. 

I quickly browsed through them, but then I realized “Wait…my termination hearing continues on Monday at 51 Chamber Street, 7th Fl at 10AM. I have to prepare for that over this right now.” [The entrance to use is 49 Chambers St., right next door to the parking lot (where the crew for "Law and Order" park, and tape the show). Read more on Monday, September 23's hearing here.] 

Therefore I am sharing them with you all, so you can read, dissect and compare. Enjoy and I look forward to your analysis.
UFT Evaluation Proposals
NYC DOE Evaluation Proposals

Would be a shame to see how much the DOE, UFT had coincided on the nasty elements.

International Studies of Teacher Evaluation: Student Tests Seldom Cited, Portfolios Carry More Weight

Big News: U.S. student test performance emphasis in teacher evaluations is a global anomaly. This is a devastating retort to the edureform/privatizing industry, their media supporters, their political supporters and acquiescing teacher trade unions.

A number of comparative internationally studies have been available in the last five years, delving into the different ways that other countries handle teacher evaluation. Diverse qualitative measures abound in these surveys; student test scores are rarely used for teacher advancement or termination conditions.

ch (International comparisons references: Laura Figazzalo, "THE USE AND MISUSE OF TEACHER APPRAISAL: An overview of cases in the developed world," Education International [the international federation of teachers' unions], 2013; Marlène Isoré, Teacher Evaluation: Current Practices in OECD Countries and a Literature Review, OECD Working Paper, 2009; Tim Walker, "How Do High-Performing Nations Evaluate Teachers?," NEA Today, March 25, 2013. Link for Isoré is difficult. You may access the initial page [http://www.oecd-ilibrary.org/education/teacher-evaluation-current-practices-in-oecd-countries-and-a-literature-review_223283631428] and the pdf link: http://www.oecd-ilibrary.org/docserver/download/5ksf76jc5phd.pdf?expires=1372295146&id=id&accname=guest&checksum=4C3FAA149C84B4E2B2FC0000C187934F)

The UFT, the AFT and the larger ramifications of these studies
These studies --and the conclusion that high-performing countries do perform well in spite of no test-score link-in on teacher promotion or firing-- indicate that this country's teacher unions, by and large have not performed their job in looking beyond our borders at other countries at how they evaluate teachers. For all the current talk of international benchmarks and keeping up with the Finnish or Singaporeans, facts and logic indicate that unions have done a terrible job of researching or of doing some serious self-analysis or comparison. Worse, in the face of all the teacher bashing Michael Mulgrew's United Federation of Teachers (UFT) or Randi Weingarten's American Federation of Teachers (AFT) have not taken the opportunity to refute the line of the reformers that teachers can be held exclusively responsible for test results, they have cooperated with evaluation schemes that rest on reference to test scores. They have the resources to research these matters. The Chicago Teachers Union, to their credit, referenced Isoré's 2009 Organisation for Economic Co-operation and Development (OECD) study in one of their statements on teacher evaluation, "Teacher Evaluation or Teacher Collaboration?" Incredibly, the UFT's Mulgrew is promoting the state-imposed teacher evaluation system as a long-needed model of reform.

Major takeaways

I) High-stakes tests and student performance.
A common current among these studies: reference to test scores are made in some countries, but not the top flying countries in global comparisons. Finland has no national standardized test; and Singapore does not use test scores to measure teacher performance. The report authors cited few specific countries, China, the U.K. and the U.S. being exceptions, that use student test scores.
Indeed, Figazzolo's study indicates that the consistently top-performing nations on international tests do not evaluate teachers on the basis of test scores. High flyers, no test incorporation: The following countries are countries that are consistently higher performing in the PISA and TIMSS international comparative tests of students, yet, their teacher evaluation systems do not cite test scores: Austria, Australia, Belgium, Canada, Finland, Japan, Korea, the Netherlands, New Zealand, Slovakia, Sweden, Switzerland. (References: PISA test result rankings, and TIMSS test result rankings, Laura Figazzolo's 2013 study.)
Marlene Isoré's OECD survey provides a number of reasons why test score reliance is avoided. (See summation paragraph, no. 65, and component paragraphs, 60 to 64.)

II) Student portfolios
Student class portfolios carry greater weight than they do in the U.S. (Isoré)
It is interesting: in New York City it is the alternative school where we often find student portfolios as traditionally trumping the state-wide standardized test. In several countries in Isore's OECD student student portfolios carry greater weight in teacher assessment than in the U.S.

III) Evaluations by staff other than school principals
In many countries peers (other teachers) are evaluators. The evaluations are meant in a supportive, rather than competitive manner. (Figazzolo; Isoré)

IV) The overall climate is less hostile and more professional

The tenor in public discourse is hostile to teachers. The working conditions, worsening by the year, are making the job status less professional and marked by deskilling.

Linda Darling-Hammond has noted the greater respect the high-performing nations afford their instructors.

They enter a well-paid profession – in Singapore earning as much as beginning doctors -- where they are supported by mentor teachers and have 15 or more hours a week to work and learn together – engaging in shared planning, action research, lesson study, and observations in each other’s classrooms.

VI) More qualitative assessment

This comes with reports of greater Charlotte Danielson reach than we realized. Isoré reports that Danielson or Danielson-influenced evaluation schemes are used in Chile and in Quebec, Canada.

* * *

Laura Figazzalo, "THE USE AND MISUSE OF TEACHER APPRAISAL: An overview of cases in the developed world," Education International [the international federation of teachers' unions], 2013

A review of the technical evidence leads Baker et al (2010) and other sources (Burris, 2012; Strauss, 2012) to conclude that, although standardised test scores of students are one tool school leaders can use to make judgments about teacher effectiveness, such scores can only be a part of an overall comprehensive evaluation. Any sound evaluation has to necessarily involve a balancing of all relevant factors in order to provide a more accurate view of what teachers do in the classroom and their contribution to student learning. In addition, binding teacher evaluation and sanctions to test score results can discourage teachers from wanting to work in schools with the neediest students, while the large, unpredictable variation in the results and their perceived unfairness can undermine teacher morale (Baker et al, 2010). For instance, teachers show lower gains when they have large numbers of new English-learners and students with disabilities than when they teach other students. This is true even when statistical methods are used to “control” for student characteristics (Darling-Hammond, 2012). Surveys have found that teacher attrition and demoralisation have been associated with test-based accountability efforts, particularly in high-need schools.
The use of VAMS is also associated with a narrowing of the curriculum; a de facto curriculum whose subject matter is defined by what is tested. Teachers who rate highest on the low-level multiple-choice tests currently in use are often not those who raise higher scores in assessments of more-challenging learning (Darling-Hammond, 2012). Some believe that the pressure to teach “fill-in-the-bubble tests” will further reduce the focus on research, writing, and complex problem-solving; areas which students will need competence in to compete with their peers in high-achieving countries (Darling-Hammond, 2012). Finally, as far as merit-pay systems are concerned, tying teacher evaluation and remuneration to test results is problematic on numerous levels, not least because it reinforces a competitive spirit that undermines teacher collegiality and teamwork (Froese-Germain, 2011).

2009 OECD Working Paper, Teacher Evaluation: Current Practices in OECD Countries and a Literature Review
By Marlène Isoré

65. As a consequence, despite the attractiveness of the idea, there are numerous caveats against the use of student scores to evaluate teachers. In particular, there is a wide consensus in the literature around two specific directions: student outcomes should not be used as the sole measurement of teacher performance, and student outcomes should not be naively used for career decisions concerning the teacher, including the link to pay, because this incorporates a substantial risk to punish or reward teachers for results beyond their control (Kane and Staiger, 2002; Kupermintz, 2002; McCaffrey et al., 2003; CAESL, 2004; Raudenbush, 2004; Braun, 2005; Ingvarson, Kleinhenz and Wilkinson, 2007; Rowley and Ingvarson, 2007). These rejections from teachers and scholars have materialized, for instance, in the New York State’s legislature decision to ban the use of test scores in evaluating teachers in April 2008.

* * * * *

DELVING FURTHER INTO REASONS WHY USING STUDENT TEST SCORE ACHIEVEMENT IS A DUBIOUS MEASURES:
60. Student learning outcomes is an appealing measure to assess teaching performance, since the ultimate goal of teaching is to improve student learning. Not surprisingly, much research has focused on the use of student achievement as measured by standardised tests to evaluate teachers. For instance, Leigh (2007) recently examined the test scores in literacy and numeracy of three cohorts of students, and concluded that the changes in the relative positions of classes of students provided a basis for the identification of effective and ineffective teachers. Braun (2005) argues that considering student scores is a promising approach for two reasons: first, it moves the discussion about teacher quality towards student learning as the primary goal of teaching, and second, it introduces a quantitative – and thus, objective and fair – measurement of teacher performance. In this respect, the development of “value-added” models represents significant progress relative to methods based on the absolute proportion of students meeting a given achievement level. “Value-added” models are designed to control for the individual students’ previous test scores, and therefore have the potential to identify the contribution an individual teacher made to students’ achievement.

61. In Florida, the “Special Teachers are Rewarded” (STAR) scheme links salary or bonus awards for individual teachers to value-added measures of st udent learning (Ingvarson, Kleinhenz and Wilkinson, 2007). Nevertheless, this type of link between a direct measure of performance and pay remains extremely rare, given the numerous statistical and theoretical challenges associated with the use of these methods. Indeed, Braun (2005) emphasises the marked contrast between the enthusiasm of those who would like to use such measurements, mainly policymakers, and the reservations expressed by the researchers who have studied their technical characteristics.

62. Using student achievement on standardised tests to evaluate teacher performance presents numerous statistical challenges. Most authors (L ockwood, Louis and McCaffrey, 2002; Kupermintz, 2003; Braun, 2005; Aaronson, Barrow and Sander, 2007; Goe, 2007) are not convinced that the current generation of value-added models is sufficiently valid and reliable to be used for fairly evaluating individual teachers’ effectiveness. Statistical limitations first refer to the noticeable lack of reliable data, mainly due to the fact that individual students rarely take annual standardised tests. Rowley and Ingvarson (2007) criticise Leigh (2007)’s methodology, which consists of creating a hypothetic test score in the missing data year at the midpoint of two available test results, arguing that it does not allow to fairly attribute the students’ success to the different teachers involved. Second, when data are available, sampling variations can cause imprecision in test score measures; this problem is particularly striking in elementary EDU/WKP(2009)2 schools, where the limited number of students per classroom creates large idiosyncrasies of the particular sample of students being tested (Kane and Staiger, 2002).

63. Broader methodological criticisms stress that value-added models, whatever their degree of sophistication, can neither fully integrate all factors influencing student achievement scores – qualitative by nature – nor reflect all student learning outcomes. Family background and support, school attendance, peer and classroom climate, school policies, availability of adequate materials, and children effects influence student learning (CAESL, 2004; Ingvarson, Kleinhenz and Wilkinson, 2007; Goe, 2007, Weingarten, 2007). Specific factors at the time of the test – “a dog barking in the playground, a severe flu season, a disruptive student in a class” – can also affect one student’s results independently from his teacher’s contribution (Kane and Staiger, 2002). Moreover, good teachers are likely to have an impact on children’s achievement during several years after having taught to them; and conversely, after several years of ineffective teachers, students may never be able to catch up academically. These teacher ‘cumulative effects’ cannot be accurately measured at discrete points in time (Hanushek, 1986; Sanders and Rivers, 1996; CAESL, 2004). Finally, teaching impact on students is not restricted to areas assessed through student standardised tests, – generally limited to reading and numeracy –, but also include transfer of psychological, civic and lifelong learning skills (Margo et al., 2008). While Xin, Xu and Tatsuoka (2004) tried to decompose single test scores into several categories of cognitive abilities in four countries (Japan, Korea, the Netherlands and the United States), they found that teachers’ attributes used in pay decisions have no consistent positive impact on any type of cognitive skills, despite their attention to controlling for individual and family background. These are sources of skepticism for using such statistical methods.

64. Theoretical limitations also need to be considered. First, a statistical correlation is not a causal relationship: the fact that teachers matter for student learning does not necessarily indicate that student learning is the result of good teaching. Second, the standardised tests used to differentiate students are not specifically designed for the purpose of assessing teachers. Following Popham (1997), Goe (2007) argues that they were not engineered to be particularly sensitive to small variations in instruction or to sort out teacher contributions to student learning. Thus they do not provide a solid basis on which to hold teachers accountable for their performance. Third, using student tests scores to evaluate teachers may induce unexpected distortions and constrictions in teacher behaviour towards the sole achievement on standardised tests. High-stakes incentive schemes based on standardised tests can incite teachers to concentrate exclusively on teaching areas assessed in the tests – therefore reducing the curriculum to the basic skills generally tested – (Jacob and Lefgren, 2005, Weingarten, 2007), incite teachers to concentrate on the specific students who are close to passing mark at the expense of children who are behind or ahead (Weingarten, 2007), and even provoke serious cases of teacher cheating on standardised tests (Jacob and Levitt, 2003; Jacob, 2005). Furthermore, test results may identify teachers who are ineffective or should professionally develop but do neither permit to fairly discriminate between the wide range of effective teachers nor identify which professional development activities should be established in order to improve their performance (Braun, 2005). Finally, it may lead to holding teachers responsible for the whole student performance whereas one should instead recognise that successful teaching is a shared responsibility between governments, schools and the teaching profession (Ingvarson et al., 2007).

* * * * *

84. Measuring the effect of teacher evaluation faces a number of challenges. First, it needs to control for the broad set of qualitative variables which are likely to influence student learning. These variables encompass teacher characteristics (e.g. age, gender), teacher education and experience, students’ family factors (e.g. parents’ background, parents’ support), school factors (e.g. school policies, school incentives, peer and classroom effects) and student factors (e.g. motivation, cognitive abilities, cumulative experience). The complex realities of education prevent researchers from accurately assimilating these factors as traditional inputs into production functions (Hanushek, 1986). Second, because of its qualitative and heterogeneous nature, the output itself – student learning – is not a traditionally measurable ‘end product’, and this makes the decomposition between different factor contributions even more difficult (Hanushek, 1986; Ingvarson et al., 2007). This does not mean that doing any quantitative study in education is vain but rather than it requires particular attention to analytical issues or potential misinterpretations of the results. A particular focus should be placed on the fact that each factor omission or measurement problem – including lack of data – creates a potential quantitative bias in the estimated relationship between teacher quality and student achievement (Xin, Xu and Tatsuoka, 2004).

85. As a consequence, the empirical literature that primarily indicated that teacher evaluation may have an important role in student learning came from a process of elimination. By contradicting or restricting the respective roles of individual teachers’ apparent features (whether characteristics, education, experience, or financial incentives), numerous studies concluded that it was teacher practices – and, by extrapolation, evaluation of these practices – that indeed matter. The first influential contribution was Hanushek’s distinction between observable aspects of teachers, such as teacher background, gender, or race, and teachers’ unquantifiable “skills” (Hanushek, 1986, 1992). According to Hanushek, if the previous literature has found no significant impact of teacher quality on student achievement, it was because it concentrated on observable attributes of teachers – teacher’s holding of a master degree for example – while teacher quality was instead related to their “skills” or “idiosyncratic choices of teaching and methods” (such as classroom management, methods of presenting abstract ideas, communication skills, and so forth), i.e. their practice.

* * * * *

A UNITED STATES EVALUATION SAMPLE, ALONG WITH SOME GLOBAL EXAMPLES OF TEACHER EVALUATION:

ANNEX 2: EXAMPLES OF TEACHER EVALUATION SYSTEMS IN OECD COUNTRIES
1. Teacher evaluation for summative purposes with links to pay: The US District of Cincinnati [Milanowski, 2004]
Context: Cincinnati is a large urban district with 48,000 students and 3,000 teachers in more than 70 schools and programmes. Its average level of student achievement is low compared to the surrounding suburban districts. Cincinnati has also had a history of school reform activity, including the introduction of new whole-school designs, school-based budgeting, and teams to run schools and deliver instruction. The union-management relationship has generally been positive. Like many other urban districts, state accountability programmes and public expectations have put pressure on the district to raise student outcomes.

Implementation: In response to the obsolescence of the existing teacher performance evaluation system, and ambitious goals for improving student achievement, the District designed a knowledge- and skill-based pay system and a new teacher evaluation system during the 1998-1999 school year. The assessment system was piloted in the 1999-2000 school year and is used for teacher evaluation district wide since the 2000-01 school year.

Criteria: The assessment system is based on a set of teaching standards derived from the Framework for Teaching (Danielson, 1996). Seventeen performance standards are grouped into four domains: (i) planning and preparation; (ii) creating an environment for learning; (iii) teaching for learning; and (iv) professionalism. For each standard, a set of behaviourally anchored rating scales called rubrics describe four levels of performance: unsatisfactory, basic, proficient, and distinguished. Instruments: Teachers are evaluated using the rubrics based on two major sources of evidence: six classroom observations and a portfolio prepared by the teacher. The portfolio includes artifacts such as lesson and unit plans, attendance records, student work, family contact logs, and documentation of professional development activities.

Evaluators: Four classroom observations are made by a teacher evaluator hired from the ranks of the teaching force and released from classroom teaching for three years. Principals and assistant principals do the other two observations.

Aggregation of scores: Based on summaries of the six observations, teacher evaluators make a final summative rating on each of the standards in domains (ii) and (iii), whereas principals and assistant principals rate teachers on the standards in domains (i) and (v), primarily based on the teacher portfolio. Standards-level ratings are then aggregated to a domain-level score for each of the four domains. Scope and frequency of the evaluation: The full assessment system is used for a comprehensive evaluation of teachers in their first and third years and every five years thereafter. A less intensive assessment is done in all other years, conducted only by principals and assistant principals and based on more limited evidence. The annual assessment is intended to be both an opportunity for teacher professional development and an evaluation for accountability purposes. EDU/WKP(2009)2

Training on the evaluation process: Both teachers and evaluators receive considerable training on the new system. Evaluators are trained using a calibration process that involves rating taped lessons using the rubrics and then comparing ratings with expert judges and discussing differences. To ensure consistency among evaluators, the district eventually requires that all evaluators, including principals, meet a standard of agreement with a set of references or expert evaluators in rating videotaped lessons. Since the 2001-02 school year, only those who meet the standards are allowed to evaluate. Direct consequences: For beginning teachers (those evaluated in their first and third years), the consequence of a poor comprehensive evaluation could be the termination of the contract. For tenured teachers, consequences of a positive evaluation could include eligibility to become a lead teacher. A poor evaluation could lead to placement in the peer assistance programme and to the eventual termination of the contract.

Link to pay: The performance evaluation system was designed in part to provide the foundation for the knowledge- and skill-based pay system. This system defines career levels for teachers with pay differentiated by level. The new pay system was originally scheduled to come into effect in the 2002-03 school year, resulting in relatively high stakes evaluations for the district’s teachers. However, the link between the evaluation system and pay was rejected by teachers in a special election held in May 2002.

2. Teacher evaluation for formative purposes and as part of broader school policies
2a. Finland [UNESCO, 2007]
Context: In Finland, school teachers have positions comparable to national or municipal public servants. However, school leaders are in charge of teacher selection – once the required license is obtained – and in charge of all the policies that are considered as necessary to the enhancement of teaching quality, among which teacher evaluation. Finland is a paradigmatic case where the former system of ‘teachers and schools inspection and supervision’ was removed in 1990 but not replaced by another similar external system. As a consequence, teacher evaluation currently goes hand in hand with other policies within each particular school.

Methods / Evaluators: The Finnish scheme of teacher evaluation is characterised by the very high level of confidence placed in school and teacher competencies and professionalism as a basis to improve teaching quality. Thus, teacher self-evaluation is considered as a prime means of professional optimisation. School leaders also have a crucial role in engaging teachers in self-reflection about their own practice, and in developing a culture of evaluation alongside ambitious goals, according to the school context and challenges. The majority of schools have implemented annual discussions between school leaders and teachers to evaluate the fulfillment of the personal objectives set up during the previous year and to establish further personal objectives that correspond both to the analysis of the teacher and the needs of the school.
2b. England [Ofsted, 2006; TDA, 2007]
Context: The English system was originally designed with summative purposes, aiming at evaluating teachers’ performance, and providing them with opportunities to access a higher career stage and the corresponding pay scale. However, numerous concerns about the fairness of the process and the potential perverse impacts of the procedure on teacher performance itself were addressed (Kleinhenz and Ingvarson, 2004). Hence, the recent developments of the system – including new professional standards from September 2007 – indicate an increased formative approach, embodied by a willingness to reinforce the link between the teacher appraisal system and teacher professional development needs relative to the EDU/WKP(2009)2 school goals. More generally, the system, completed within a wider framework for the whole school workforce, aims to improve school leadership and to be an integral part of the school’s broader policies. Scope/Methods: The evaluation is differentiated according to the career stage of the teacher being evaluated. Five professional stages are identified: (i) the award of the Qualified Teacher Status (Q); (ii) teachers on the main scale (Core) (C); (iii) teachers on the upper pay scale (Post Threshold Teachers) (P); (iv) Excellent Teachers (E); and (v) Advanced skills Teachers (A).

Criteria: At each stage, teaching professional standards encompass three domains. The first one refers to the teacher’s professional attributes, including relationships with children and young people; attitude vis-à- vis the framework and the implementation of new school policies; communicating and working with others; and professional development activities. The second domain is composed of the teacher’s professional knowledge and understanding, including knowledge on teaching and learning; understanding of assessing and monitoring; subjects and curriculum knowledge; literacy, numeracy and ICT skills; understanding the factors affecting the achievement of diversified student groups; and knowledge on student health and well-being. The last domain refers to the teacher’s professional skills, including planning, teaching, assessing, monitoring, giving feedback competencies; ability to review and adapt teaching and learning; ability to create a learning environment; capacities to develop team working and collaboration. All of these standards are statements of good teaching which do not replace the professional duties and responsibilities of teachers.

Consequences on teacher professional growth and links to school expectations and policies: The standards support teachers in identifying their professional development needs. Where teachers wish to progress to the next career stage, the next level of the framework provides a reference point for all teachers when considering future development. Whilst not all teachers necessarily want to move to the next career stage, the standards also support teachers in identifying ways to broaden and deepen their expertise within their current career stages. These frameworks are a basis for professional responsibility and contractual engagement to engage all teachers in effective, sustained and relevant professional development throughout their careers. They provide a continuum of expectations about the level of engagement in professional development that provides clarity and appropriate differentiation for each career stage. They also set expectations about the contribution teachers make to others, taking account of their levels of skills, expertise and experience, their role within the school, and reflecting on their use of up-to-date subject knowledge and pedagogy. In all these cases, performance management is the key process that provides the context for regular discussions about teachers’ career aspirations and their future development, within or beyond their current career stage.

For further information:
• Training and Development Agency for Schools (TDA): http://www.tda.gov.uk/teachers/professionalstandards.aspx and http://www.tda.gov.uk/teachers/continuingprofessionaldevelopment.aspx
• Office for Standards in Education (Ofsted): http://www.ofsted.gov.uk
3. Conciliating the summative and formative purposes in a comprehensive approach: Chile [Avalos and Assael, 2006]
Context: The historical context of the Chilean educational system has doubtlessly played a critical role in understanding the necessity for a comprehensive and conciliating teacher evaluation scheme. In 1980, the military government [1973-1990] transferred the management of schools to the municipal authorities, which also implied a change of status of teachers from public servants to salaries employees of EDU/WKP(2009)2
municipalities. At the end of the dictatorial regime, a major concern was that teachers’ conditions did not evolve in line with those for public servants, which had an enormous impact on how teachers perceived and valued themselves, as well as on public opinion. In the 1990s the teaching profession suffered from a dramatic deterioration of the quality of applicants to teaching and from worsened working conditions. At the same time, evidence of unsatisfactory student learning results put a strong pressure on the government to include a clause in the new Teacher Statute (1991) that required a yearly evaluation of teachers. But while teachers continued to make their case for improved salaries and working conditions, they rejected the implementation of the evaluation system. This was followed by a long period of discussions and negotiations on the teacher evaluation model to be implemented.
Design and implementation of the system: The system was enacted by law in August 2004, that is, some seven years after the initial discussions. The system is directed toward the improvement of teaching and learning outcomes. It is designed to stimulate teachers to further their own improvement through the learning of their strengths and weaknesses. It is based on explicit criteria of what is evaluated, but without prescribing a model of teaching. It rests on the articulation of its different elements: criteria sanctioned by the teaching workforce, an independent management structure, especially prepared evaluators, and a coordinated set of procedures to gather the evidence required by the criteria.
Key actors in the system: The Centre for In-service Training located in the Ministry of Education (Centro de Perfeccionamiento, Experimentación e Investigación Pedagógica) manages the system. A consultative committee composed of academics and representatives from the Teachers’ Union, the Chilean Association of Municipalities and the Ministry of Education, monitors and provides advice on the process. A university centre is contracted to implement the process: production and revision of instruments, selection and preparation of evaluators and scorers, and analysis of evidence gathered from each evaluation process. The application process itself is decentralised so that in every district there is a committee that is directly responsible for organising the evaluation procedures. The evidence gathered is processed at the district level and sent to the central processing unit at the university, together with contextual information that can help interpret results. This central form of processing the evidence follows a request by teachers with the purpose of greater objectiveness.

Criteria: The Ministry of Education took the lead in defining the assessment criteria, producing a set of standards based on the work done earlier for the initial teacher education standards and on Danielson’s Framework for Teaching. The result is a framework for competent teaching formulated in four teaching domains (planning, learning environment, professionalism and teaching strategies for the learning of all students) and twenty criteria/standards. The framework was the subject of wide consultations among teachers until an agreement was reached. The criteria are linked to four levels of quality/performance: ‘unsatisfactory’, ‘basic’, ‘competent’ and ‘excellent’.

Instruments: The evidence used to evaluate the teachers, structured around the Framework, includes four sources: (i) a portfolio with samples of teachers’ work and a video of one of their lessons; (ii) a structured self-evaluation form; (iii) a structured interview with a peer evaluator; and (iv) a report from the school management and pedagogic authorities. The evaluation takes place every four years. Training of evaluators: The peer evaluators are specifically prepared for their task and must pass a test to be accredited. Although they should be familiar with the context in which the evaluated teacher is based (e.g. socio-economic and working conditions) they may not be teachers in the same school.
Consequences of the evaluation: One of the main challenges that needed to be addressed during the negotiation process referred to the potential implications for the individual teacher evaluated. It was agreed that teachers rated as being at a ‘basic’ level are provided with specific professional development opportunities in order to improve. Teachers rated as performing ‘unsatisfactorily’ are also provided with EDU/WKP(2009)2
professional development opportunities, but are evaluated again one year later; if the teacher fails to perform satisfactorily in two consecutive evaluations, he or she is dismissed. By contrast, teachers assessed as ‘competent’ or ‘exceptionally competent’ are given priority in promotion opportunities and in professional development activities of their interest. They may also apply for a salary bonus provided that they take a test on curricular and pedagogical knowledge. The system has both summative and formative elements instead of being primarily dedicated to one of the purposes, which is the result of the negotiation process which had taken the multiple stakeholders’ interests into account. For instance, the summative elements neither include a link between teacher’s performance and student results (something the union strongly opposed) nor a link to the career ladder. The link to professional development is emphasised and differentiated on the basis of the teacher’s level of performance.
For further information: Chile’s laws on the Teaching Statute: Ley N°3.500; Ley N°19.070; Ley N° 19.933; Ley N° 19.961.

4. Teacher evaluation stemming from bureaucratic procedures: France [Haut Conseil de l’évaluation de l’école, 2003; Pochard, 2008] Context: French teachers are classified in three distinct categories according to their education and initial certification: primary education teachers (professeurs des écoles), secondary education teachers with a regular certification (enseignants certifiés), and secondary education teachers with a higher level of certification (enseignants agrégés). All teachers are public servants but are placed in one of these three career tracks. These differ in terms of conditions and hours of work, administrative pay scale, and teaching practice (multitask primary education teachers vs. subject-specialised secondary education teachers). France does not generally suffer from teacher shortages and examinations to enter the profession continue to be selective. However, France has concerns regarding the societal status of teaching, and the skills necessary to respond to school needs. The current teacher evaluation system is often described as ‘not very fair’, ‘not very efficient’, and ‘generating malaise and sometimes suffering’ for both evaluated teachers and evaluators, because it is based on administrative procedures rather than a comprehensive scheme with a clear improvement purpose.
Periodicity of evaluation/evaluators: Teacher evaluation is supposed to be undertaken on a regular basis, as an integral part of the work and duties of the teacher. Primary education teachers are evaluated by a teaching inspector (inspecteur), while secondary level teachers are evaluated by a panel composed of an inspector – who defines 60 % of the final score – and the school principal – responsible for the other 40 %. However, the intended frequent evaluations often fall short of expectations. First, the frequency of evaluations is not legally fixed, and is arbitrarily determined by the inspectors’ availability. This is a cause for concern regarding the fairness of the system – because teachers working under the same rules receive feedback at diverse intervals – as well as regarding its efficacy – the average interval between two evaluations being 3-4 years in primary education and 6-7 years in secondary education, deemed much too long. Moreover, the workload is such that concerns might be raised regarding the value of the feedback. An inspector takes responsibility for between 350 and 400 teachers, which is excessive for the feedback to be effective in improving teachers’ practices. As a consequence, the inspectors themselves report malaise and frustration associated with the evaluation process, mainly because they feel that they have little impact on teaching practices and cannot develop their competences and skills for teaching enhancement. Their role is sometimes de facto restricted to control the abuses within the profession.
Instruments: Evidence on the teacher’s practice is gathered through the observation of a teaching session, followed by an interview with the teacher. Criticisms of this approach include: (i) the fact that a single classroom observation might not be enough to forge a fair and accurate view of the teacher’s abilities and knowledge; and (ii) in the interview teachers focus on reacting to the inspectors’ criticisms instead of EDU/WKP(2009)2 discussing their particular needs for improvement. The whole procedure does not seem to give much room for self-evaluation and teachers’ reflection on their own practice and performance.
Criteria: Both ‘pedagogical’ and ‘administrative’ aspects are observed and rated but with no reference to a framework which defines what ‘good’ teaching is. Concerns are numerous. The nature of the different ‘pedagogical’ skills assessed, as well as their weight in the overall appreciation of the teacher, remains largely at the discretion of each inspector. This reinforces subjective appraisals, unpredictable and random results, at the expense of fairness and accuracy in the process. Teachers report not knowing how and on what criteria they are evaluated. The most objective and understood criteria used to evaluate teachers are the ‘administrative’ ones such as punctuality and attendance. As a result, the rating obtained by a teacher often remains primarily determined by their certification rating (i.e. result of entrance examination). Consequences: The consequences of the teacher’s evaluation on the career are limited, except in cases of serious misconduct. Teachers’ salaries are determined by a single salary scale in which progression depends on years of service and the initial qualifications and entrance examination. Commitment to work is rarely recognised and valued, as well as merit, outstanding performance, or initiatives seeking to improve student learning. In addition, there is no link to professional development activities, the latter being very limited and disconnected from teachers’ identified weaknesses. The evaluation process does not provide opportunities for self-reflection on teaching practices or for peer mutual learning, and entails little advice and coaching.

For further information:
• Haut Conseil de l’évaluation de l’école (2003): http://cisad.adc.education.fr/hcee/ documents/rapport_annuel_2003.pdf
• Rapport des inspections générales: http://lesrapports.ladocumentationfrancaise.fr/BRP/054004446/0000.pdf
• Ministry of Education: http://www.education.gouv.fr/cid263/l-evaluation-des-personnels.html
About other teacher evaluation systems:
• The Canadian Province of Alberta: http://www.education.alberta.ca/department/policy/k12manual/section2/teacher.aspx • The US State of Iowa: http://www.iowa.gov/educate/content/view/1450/1617

* * * * *

REFERENCES
Aaronson, D.; Barrow, L. and Sander, W. (2007) “Teachers and Student Achievement in the Chicago Public High Schools”, Journal of Labor Economics, Vol. 25, No. 1, pp 95-135.
American Federation of Teachers (2001) “Beginning Teacher Induction: The Essential Bridge”, Educational Issues Policy Brief No. 13, AFT, 2001.
American Federation of Teachers and National Education Association (2008) A Guide to Understanding National Board Certification, AFT and NEA, Washington, DC.
Anderson, L. and Pellicer, L. (2001) Teacher Peer Assistance and Review, Corwin Press.
Avalos, B. and Assael, J. (2006) “Moving from resistance to agreement: The case of the Chilean teacher performance evaluation”, International Journal of Educational Research, Vol. 45, No. 4-5, pp 254-266.
Beck, R.; Livne, N. and Bear, S. (2005) “Teachers’ self-assessment of the effects of formative and summative electronic portfolios on professional development”, European Journal of Teacher Education, Vol. 28, No. 3, pp 221-244.
Bolino, M. and Turnley, W. (2003) “Counternormative impression management, likeability, and performance ratings: the use of intimidation in an organizational setting”, Journal of Organizational Behavior, Vol. 24, No. 2, pp 237-250.
Bond, L.; Smith, T.; Baker, W. and Hattie, J. (2000) “The Certification System of the National Board for Professional Teaching Standards: A Construct and Consequential Validity Study”, NBPTS, 2000.
Borman, G. and Kimball, S. (2005) “Teacher Quality and Educational Equality: Do Teachers with Higher Standards-Based Evaluation Ratings Close Student Achievement Gaps?”, The Elementary School Journal, Vol. 106, No. 1, pp 3-20.
Braun, H. (2005) “Using Student Progress to Evaluate Teachers: A Primer on Value-Added Models”, Educational Testing Service (ETS), 2005.
Casson, H. Jr (2007) “Reducing Teacher Moral Hazard in the U.S. Elementary and Secondary Educational System through Merit-pay: An Application of the Principal – Agency Theory”, Forum for Social Economics, Vol. 36, No. 2, pp 87-95.
Campbell, D.; Melenzyer, B.; Nettles, D. and Wyman, R. (2000) Portfolio and performance assessment in teacher education, Needham Heights, MA, Allyn & Bacon.
Cavalluzzo, L. (2004) “Is National Board Certification An Effective Signal of Teacher Quality?”, The CNA Corporation, Alexandria, Virginia, 2004. EDU/WKP(2009)2
 40
Center for Assessment and Evaluation of Student Learning (2004) “Using Student Tests to Measure Teacher Quality”, CAESL Assessment Brief No. 9.
Center for Teaching Quality (2008) “Measuring What Matters: The Effects of National Board Certification on Advancing 21st Century Teaching and Learning”, CTQ, 2008.
Cohen, C. and King Rice, J. (2005) “National Board Certification as Professional Development: Design and Cost”, NBPTS, 2005.
Corcoran, T. (2007) “Teaching Matters: How State and Local Policymakers Can Improve the Quality of Teachers and Teaching”, Consortium for Policy Research in Education (CPRE) Policy Briefs RB-48.
Danielson, C. (2001) “New Trends in Teacher Evaluation”, Educational Leadership, Vol. 58, No. 5, pp 12-15.
Danielson, C. (1996, 2007) Enhancing Professional Practice: A Framework for Teaching, 1st and 2nd editions, Association for Supervision and Curriculum Development (ASCD), Alexandria, Virginia.
Danielson, C. and McGreal, T. (2000) Teacher Evaluation to Enhance Professional Practice, Association for Supervision and Curriculum Development (ASCD), Alexandria, Virginia.
Darling, L. (2001) “Portfolio as practice: the narratives of emerging teachers”, Teaching and Teacher Education, Vol. 17, pp 107-121.
Darling-Hammond, L.; Pecheone, R. and Stansbury, K. (2004) “Beginning Teacher Quality: What Matters for Student Learning?”, Research proposal from Standford University to the Carnegie Corporation of New York, available at www.pacttpa.org/_files/Publications_and_Presentations/Carnegie_grant_proposal.doc.
Day, C. and Gu, Q. (2007) “Variations in the conditions for teachers’ professional learning and development: sustaining commitment and effectiveness over a career”, Oxford Review of Education, Vol. 33, No. 4, pp 423-443.
Department of Education, Science and Training (2007) “Performance-based rewards for teachers”, DEST Research Papers, 2007.
Elmore, R. (2000) Building a New Structure for School Leadership, Albert Shanker Institute, Winter 2000.
Figlio, D. and Kenny, L. (2007), “Individual teacher incentives and student performance”, Journal of Public Economics, Vol. 91, No. 5-6, pp 901-914.
Freund, M.; Kane Russell, V. and Kavulic, C. (2005) “A Study of the Role of Mentoring in Achieving Certification by the National Board for Professional Teaching Standards”, NBPTS, 2005.
Goe, L. (2007) “The Link Between Teacher Quality and Student Outcomes: A Research Synthesis”, National Comprehensive Center for Teacher Quality, 2007.
Goldhaber, D. and Anthony, E. (2007) “Can Teacher Quality Be Effectively Assessed? National Board Certification As a Signal of Effective Teaching”, The Review of Economics and Statistics, Vol. 89, No. 1, pp 134-150. EDU/WKP(2009)2  41
Greenfield, W. (1995) “Toward a Theory of School Administration: the Centrality of Leadership”, Educational Administration Quraterly, Vol. 31, No. 1, pp 61-85.
Halverson, R.; Kelley, C. and Kimball, S (2004) “Implementing Teacher Evaluation Systems: How Principals Make Sense of Complex Artifacts to Shape Local Instructional Practice” in Educational Administration, Policy and Reform: Research and Measurement Research and Theory in Educational Administration, Vol. 3, W.K. Hoy and C.G. Miskel (Eds.) Greenwish, CT: Information Age Press.
Hanushek, E. (2004) “Does School Accountability Lead to Improved Student Performance?”, NBER Working Papers n°10591.
Hanushek, E. (1992) “The Trade-Off between Child Quantity and Quality”, Journal of Political Economy, Vol. 100, No. 1, pp 84-117.
Hanushek, E. (1986) “The Economics of Schooling: Production and Efficiency in Public Schools”, Journal of Economic Literature, Vol. 24, No. 3, pp 1141-1177.
Hanushek, E.; Kain, J.; O’Brien, D. and Rivkin, S. (2005) “The Market for Teacher Quality”, NBER Working Papers n°11154.
Harris, D. and Sass, T. (2007) “The Effects of NBPTS-Certified Teachers on Student Achievement”, Center for Analysis of Longitudinal Data in Education Research (CALDER), Working Paper No. 4.
Haut Conseil de l’évaluation de l’école (2003) Rapport annuel, HCéé, 2003.
Heneman, H. and Milanowski, A. (2003) “Continuing Assessment of Teacher Reactions to a StandardsBased Teacher Evaluation System”, Journal of Personnel evaluation in Education, Vol. 17, No. 2, pp 173-195.
Heneman, H.; Milanowski, A. and Kimball, S. (2007) “Teacher Performance Pay: Synthesis of Plans, Research, and Guidelines for Practice”, Consortium for Policy Research in Education (CPRE) Policy Briefs RB-46.
Heneman, H.; Milanowski, A.; Kimball, S. and Odden, A. (2006) “Standards-Based Teacher Evaluation as a Foundation for Knowledge- and Skill-Based Pay”, Consortium for Policy Research in Education (CPRE) Policy Briefs RB-45.
Hess, F. and West, M. (2006) “A Better Bargain: Overhauling Teacher Collective Bargaining for the 21st Century”, Cambridge, MA: Program on Education Policy and Governance, Harvard University.
Holland, P. (2005) “The Case for Expanding Standards for Teacher Evaluation to Include an Instructional Supervision Perspective”, Journal of Personnel Evaluation in Education, Vol. 18, No. 1, pp 67-77.
Ingvarson, L.; Kleinhenz, E. and Wilkinson, J. (2007) Research on Performance Pay for Teachers, Australian Council for Educational Research (ACER), 2007.
Interstate New Teacher Assessment and Support Consortium (1992) “Model Standards for Beginning
Teacher Licensing, Assessment and Development: A Resource for State Dialogue”, INTASC,
Council of Chief State School Officers (CCSSO), 1992.EDU/WKP(2009)2  42
Jacob, B. (2004) “Accountability, Incentives and Behavior: The Impact of High-Stakes Testing in the
Chicago Public Schools”, Journal of Public Economics, Vol. 89, No. 5-6, pp 761-796.
Jacob, B. and Lefgren, L. (2008) “Can Principals Identify Effective Teachers? Evidence on Subjective Performance Evaluation in Education”, Journal of Labor Economics, Vol. 26, No. 1, pp 101-136.
Jacob, B. and Lefgren, L. (2005b) “What Do Parents Value in Education: an Empirical Investigation of Parents’ Revealed Preferences for Teachers”, NBER Working Paper n°11494.
Jacob, B. and Lefgren, L. (2005a) “Principals as Agents: Subjective Performance Measurement in Education”, NBER Working Papers n°11463.
Jacob, B. and Levitt, S. (2003) “Rotten Apples: An Investigation of the Prevalence and Predictors of Teacher Cheating”, The Quaterly Journal of Economics, Vol. 118, No. 3, pp 843-878.
Jacobs, C.; Martin, S. and Otieno, T. (2008) “A Science Lesson Plan Analysis Instrument for Formative and Summative Program Evaluation of a Teacher Education Program”, Science Teacher Education (Articles online in advance of print).
Jun, M.-K.; Anthony, R.; Achrazoglu, J. and Coghill-Behrends, W. (2007) “Using ePortfolio for the Assessment and Professional Development of Newly Hired Teachers”, TechTrends, Vol. 51, No. 4,
pp 45-50.
Kane, T. and Staiger, D. (2002) “Volatility in School Test Scores: Implications for Test-Based Accountability Systems”, Brooking Papers on Education Policy, Washington, DC.
Kennedy, M. (2005) Inside Teaching, Harvard University Press, London, England, 2005.
Kimball, S. (2002) “Analysis of Feedback, Enabling Conditions and Fairness Perceptions of Teachers in Three School Districts with New Standards-Based Evaluation Systems”, Journal of Personnel Evaluation in Education, Vol. 16, No. 4, pp 241-268.
Kimball, S.; Milanowski, T. and McKinney, S. (2007) “Implementation of Standards-Based Principal Evaluation in One School District: First Year Results From Randomized Trial”, Paper presented at the annual conference of the American Educational Research Association, available at http://cpre.wceruw.org/publications/KimballMilanowskiMcKinney.pdf.
Klecker, B. (2000) “Content validity of preservice teacher portfolios in a standards-based program”, Journal of Instructional Psychology, Vol. 27, No. 1, pp 35-38.
Kleinhenz, E. and Ingvarson, L. (2004) “Teacher Evaluation Uncoupled: A Discussion of Teacher Evaluation Policies and Practices in Australian States and Their Relation to Quality Teaching and Learning”, Research Papers in Education, Vol. 19, No. 1, pp 31-49.
Leigh, A. (2007) “Estimating Teacher Effectiveness From Two-Year Changes in Students’ Test Scores”, Research School of Social Sciences, Australian National University, available at http://econrsss.anu.edu.au/~aleigh/pdf/TQPanel.pdf.
Levin, J. (2003) “Relational incentive contracts”, American Economic Review, Vol. 93, No. 3, pp 835–57.
Lustick, D. and Sykes, G. (2006) “National Board Certification as Professional Development: What are Teachers Learning?”, Education Policy Analysis Archives, Vol. 14, No.5. EDU/WKP(2009)2  43
MacLeod, B. (2003) “Optimal contracting with subjective evaluation”, American Economic Review 93, No. 1, pp 216-240.
Mansvelder-Longayroux, D.; Beijaard, D. and Verloop, N. (2007) “The portfolio as a tool for stimulating reflection by student teachers”, Teaching and Teacher Education, Vol. 23, No. 1, pp 47-62.
Margo, J.; Benton, M.; Withers, K. and Sodha, S. (2008) Those Who Can?, Institute for Public Policy Research (IPPR) Publications, 2008.
Marshall, K. (2005) “It’s Time to Rethink Teacher Supervision and Evaluation”, Phi Delta Kappan, Vol.
86, No. 10, pp 727-735.
McColskey, W. and Stronge, J. (2005) “A Comparison of National Board Certified Teachers and nonNational Board Certified Teachers: Is there a difference in teacher effectiveness and student
achievement”, NBPTS, 2005.
Milanowski, A. (2007) “Performance Pay System Preferences of Students Preparing to Be Teachers”, American Education Finance Association, 2007.
Milanowski, A. (2004) “The Relationship Between Teacher Performance Evaluation Scores and Student Achievement: Evidence From Cincinatti”, Peabody Journal of Education, Vol. 79, No. 4, pp 33-53.
Milanowski, A. and Heneman, H. (2001) “Assessment of Teacher Reactions to a Standards-Based Teacher Evaluation System: A Pilot Study”, Journal of Personnel Evaluation in Education, Vol. 15, No. 3, pp 193-212.
Milanowski, A. and Kimball, S. (2003) “The Framework-Based Teacher Performance Assessment Systems in Cincinnati and Washoe”, CPRE Working Paper Series TC-03-07.
Ministerial Council on Education, Employment Training and Youth Affairs (2003) “A National Framework for Professional Standards for Teaching”, MCEETYA, Carlton South, Australia, 2003.
Mizala, A. and Romaguera, P. (2004) “School and teacher performance incentives: The Latin American experience”, International Journal of Educational Development, Vol. 24, No. 6, pp 739-754.
Muñoz, M. and Chang, F. (2007) “The Elusive Relationship Between Teacher Characteristics and Student Achievement Growth: A Longitudinal Multilevel Model for Change”, Journal of Personnel Evaluation in Education, Vol. 20, No. 3-4, pp 147-164.
Nabors Oláh, L.; Lawrence, N. and Riggan, M. (2008) “Learning to learn from benchmark assessment data: How teachers analyze results”, Paper presented at the Annual Meeting of the American Educational Research Association, New York, 2008, available at http://www.cpre.org/images/stories/cpre_pdfs/ aera2008_olah_lawrence_riggan.pdf .
National Board for Professional Teaching Standards (2007) “A Research Guide on National Board Certification of Teachers”, NBPTS, Arlington, VA, 2007.
Odden, A. and Kelley, C. (2002) Paying Teachers for What They Know and Do: New and Smarter Compensation Strategies to Improve Schools, Corwin Press, Thousand Oaks, California, 2002.
Office for Standards in Education (2006) “The logical chain: continuing professional development in effective schools”, OFSTED Publications n°2639, United Kingdom, 2006. EDU/WKP(2009)2  44
Organisation for Economic Co-Operation and Development (2008) Improving School Leadership, OECD, Paris, 2008.
Organisation for Economic Co-Operation and Development (2005) Teachers Matter: Attracting, Developing and Retaining Effective Teachers, OECD, Paris, 2005.
Ovando, M. and Ramirez, A Jr (2007) “Principals’ Instructional Leadership Within a Teacher Performance Appraisal System: Enhancing Students’ Academic Success”, Journal of Personnel Evaluation in Education, Vol. 20, No. 1-2, pp 85-110.
Pecheone, R. and Chung, R. (2006) “Evidence in Teacher Education: The Performance Assessment for California Teachers (PACT)”, Journal of Teacher Education, Vol. 57, No. 1, pp 22-36.
Peterson, K. (2000) Teacher Evaluation: A Comprehensive Guide to New Directions and Practices, 2nd edition, Thousand Oaks, CA: Corwin Press.
Peterson, K.; Wahlquist, C. and Bone, K. (2000) “Student Surveys for Teacher Evaluation”, Journal of Personnel Evaluation in Education, Vol. 14, No. 2, pp 135-153.
Peterson, K.; Wahlquist, C.; Esparza Brown, J. and Mukhopadhyay, S. (2003) “Parents Surveys for Teacher Evaluation”, Journal of Personnel Evaluation in Education, Vol. 17, No. 4, pp 317-330.
Peterson, K. (2006) “Teacher Pay Reform Challenges States”, Stateline.org: where policy and politics news click, available at http://www.stateline.org/live/ViewPage.action?siteNodeId= 136&languageId=1&contentId=93346.
Petty, T. (2002) “Identifying the Wants and Needs of North Carolina High School Mathematics Teachers for Job Success and Satisfaction”, NBPTS, 2002.
Ping Yan Chow, A.; King Por Wong, E.; Seeshing Yeung, A. and Wan Mo, K (2002) “Teachers’ Perceptions of Appraiser-Appraisee Relationships”, Journal of Personnel Evaluation in Education, Vol. 16, No. 2, pp 85-101.
Pochard, M. (2008) Livre vert sur l’évolution du métier d’enseignant, Rapport au ministre de l’Education nationale, La Documentation française, Collection des rapports officiels, 2008.
Popham, J. (1997) “Consequential validity: Right Concern – Wrong Concept”, Educational Measurement: Issues and Practice, Vol. 16, No. 2, pp 9-13.
Robinson, V. (2007) “School Leadership and Student Outcomes : Identifying What works and Why”, Australian Council for Educational Leaders, ACEL Monograph Series No. 41.
Sanders, W.; Ashton, J. and Wright, P. (2005) “Comparison of the Effects of NBPTS Certified Teachers with Other Teachers on the Rate of Student Academic Progress”, NBPTS, 2005.
Smith, T.; Gordon, B.; Colby, S. and Wang, J. (2005) “An Examination of the Relationship Between Depth of Student Learning and National Board Certification Status”, Office for Research on Teaching, Appalachian State University.
Stronge, J. and Tucker, P. (2003) Handbook on Teacher Evaluation: Assessing and Improving Performance, Eye On Education Publications, 2003. EDU/WKP(2009)2  45
Stronge, J.; Ward, T.; Tucker, P. and Hindman, J. (2007) “What is the Relationship Between Teacher Quality and Student Achievement? An Exploratory Study”, Journal of Personnel Evaluation in Education, Vol. 20, No. 3-4, pp 165-184.
Strudler, N. and Wertzel, K. (2008) “Costs and Benefits of Electronic Portfiolos in Teacher Education: Faculty Perspectives”, Journal of Computing in Teacher Education, Vol. 24, No. 4, pp 135-142.
Training and Development Agency for Schools (2007a) “Models Performance Management Policy for Schools”, TDA, United Kingdom, 2007.
Training and Development Agency for Schools (2007b) “Professional Standards for Teachers: Why Sit Still in Your Career?”, TDA, United Kingdom, 2007.
Tucker, P.; Stronge, J. and Gareis, C. (2002) Handbook on teacher portfolios for evaluation and professional development, Larchmont, NY, Eye on Education.
UNESCO (2007) Evaluación del Desempeño y Carrera Profesional Docente: Una panorámica de América y Europa, Oficina Regional de Educación para américa Latina y el Caribe, UNESCO Santiago, 2007.
Vandervoort, L.; Amrein-Beardsley, A. and Berliner, D. (2004) “National Board Certified Teachers and Their Students’ Achievement”, Education Policy Analysis Archives, Vol. 12, No. 46.
Weingarten, R. (2007) “Using Student Test Scores to Evaluate Teachers: Common Sense or Nonsense?”, United Federation of Teachers, available at http://www.uft.org/news/randi/ny_times/ UFT_WMM_Mar07_v32.pdf.
Wertzel, K. and Strudler, N. (2006) “Costs and Benefits of Electronic Portfolios in Teacher Education: Student Voices”, Journal of Computing in Teacher Education, Vol. 22, No. 3, pp 69-78.
Xin, T.; Xu, Z. and Tatsuoka, K. (2004) “Linkage Between Teacher Quality, Student Achievement, and Cognitive Skills: A Rule-Space Model”, Studies in Educational Evaluation, Vol. 30, pp 205-223.

* * *
Figazzolo's critique of value-added modeling and the use of standardized tests:

Value-added evaluation methods, where teacher effectiveness and compensation are increasingly being tied to student scores on standardised tests, have raised concerns among teachers, unions and practitioners in general. As Froese-Germain (2011) puts it, these methods originate from a highly charged climate of data-driven accountability, and are increasingly common across the US. For example, the Los Angeles Unified School District is among a growing number of US school districts using the results of standardised tests to determine the “value-added” outcomes produced by the teacher (the value-added measure of teacher performance is related to gains in test scores in the teacher’s class over time)5. Other stories are reported from Chicago, where some reformers, such as Chicago Mayor Rahm Emanuel, want as much as half of a teacher’s evaluation to be linked to student test scores6 (Strauss, 2012).
Rather than placing student results in context, these methods issue a comprehensive judgment purely based on data developed through standardised calculations. However, as Baker et al (2010) highlight, VAM estimates have proven to be unstable across statistical models, years, and teaching classes. Studies quoted in Baker et al (2010) prove in fact that a teacher who appears to be ineffective in one year may achieve dramatically different results the following year. VAM’s instability can result from differences in the characteristics of students assigned to given teachers in a particular year and from specific evaluation measures. Such factors include: small samples of students (made even less representative in schools serving disadvantaged students and which have high rates of student mobility), other influences on student learning both inside and outside school, and tests which are poorly lined up with the curriculum teachers are expected to cover, or which do not measure the full range of achievement of students in the class.
A number of non-teacher factors have been found to have strong influences on student learning gains. These include the influence of other teachers, tutors or instructional specialists; school conditions — such as the quality of curriculum materials, specialist or tutoring supports, class size, resources, learning environment; and other factors that affect learning.
A review of the technical evidence leads Baker et al (2010) and other sources (Burris, 2012; Strauss, 2012) to conclude that, although standardised test scores of students are one tool school leaders can use to make judgments about teacher effectiveness, such scores can only be a part of an overall comprehensive evaluation. Any sound evaluation has to necessarily involve a balancing of all relevant factors in order to provide a more accurate view of what teachers do in the classroom and their contribution to student learning.
In addition, binding teacher evaluation and sanctions to test score results can discourage teachers from wanting to work in schools with the neediest students, while the large, unpredictable variation in the results and their perceived unfairness can undermine teacher morale (Baker et al, 2010). For instance, teachers show lower gains when they have large numbers of new English-learners and students with disabilities than when they teach other students. This is true even when statistical methods are used to “control” for student characteristics (Darling-Hammond, 2012). Surveys have found that teacher attrition and demoralisation have been associated with test-based accountability efforts, particularly in high-need schools.
The use of VAMS is also associated with a narrowing of the curriculum; a de facto curriculum whose subject matter is defined by what is tested. Teachers who rate highest on the low-level multiple-choice tests currently in use are often not those who raise higher scores in assessments of more-challenging learning (Darling-Hammond, 2012). Some believe that the pressure to teach “fill-in-the-bubble tests” will further reduce the focus on research, writing, and complex problem-solving; areas which students will need competence in to compete with their peers in high-achieving countries (Darling-Hammond, 2012).
Finally, as far as merit-pay systems are concerned, tying teacher evaluation and remuneration to test results is problematic on numerous levels, not least because it reinforces a competitive spirit that undermines teacher collegiality and teamwork (Froese-Germain, 2011).

POSTSCRIPT: On a related note to the last point, one Nashville, Tennessee, U.S. study has concluded that merit pay alone does not raise student performance: Melanie Moran, "Vanderbilt News," September 21, 2010, "Teacher performance pay alone does not raise student test scores."