As CPS prepares to close a record number of schools, the fate of students and communities is in question.
Join the conversation
We encourage our readers to leave comments and engage in dialogue about our stories. But before you do, please check out our "rules of the road."
Recent Notebook Entries
Right Now On Notebook
By:
By:
Subscribe to catalyst-chicago.org by e-mail
Other Blogs
catalyst-chicago.org feeds
Current Issue
Teacher evaluation pilot shows promise
By: Sarah Karp / November 15, 2011
When using a new, more detailed teacher evaluation process, principals mostly got it right as their ratings lined up with value-added student test scores, according to a University of Chicago Consortium on School Research study released Tuesday.
While researchers say this is evidence that the new evaluation works, the Chicago Teachers Union’s Carol Caref says the results prove that test scores don’t need to be factored into teacher evaluations in order to make them accurate.
“If there is a strong correlation, then why not rely on teacher practice and principal observation?” she says.
The use of test scores, or growth in test scores, as part of a teacher's job evaluation is a hot issue in the education world. But in Illinois and other states, it's an issue that has been decided. The Performance Evaluation Reform Act, passed in 2010, requires that by 2013, test scores count for a significant portion of a teacher’s evaluation. The bill also requires a new evaluation process.
Researchers did not give teachers an overall mark. But using a formula from another urban district, about a third of the teachers in the pilot would be rated unsatisfactory, 42 percent would be satisfactory and a quarter would be excellent.
These percentages would be a big change from the current CPS teacher evaluation, in which fewer than 1 percent of teachers have been rated unsatisfactory and more than 90 percent rated superior or excellent.
The Consortium researchers studied the use of the Charlotte Danielson framework, a well-regarded evaluation tool, in about 100 schools over a two-year period. Last year, CPS also tested the Teaching and Learning Framework, an evaluation tool that is modeled on a tool used in Washington, D.C. public schools and was developed in the district by principals, teachers and CPS officials.
CEO Jean-Claude Brizard has said that he likes the Danielson framework and insiders say that may be the way his administration is leaning.
What ratings show
The Danielson framework is more detailed than the state's current checklist format, and includes paragraphs describing what teachers rated in each category should or should not be doing.
For example, a distinguished teacher—which is the highest rating—in the area of classroom management would involve students in determining the behavior standards for the classroom and monitoring them. A proficient teacher--the second highest rating—would make behavior standards clear and monitor them. An unsatisfactory teacher—the lowest rating—would establish no standards and respond to bad behavior by shaming the student.
“We are seeing a paradigm shift from a checklist to describing practice in a specific way,” says Sara Ray Stoelinga, who was one of the study’s authors, speaking at an Education Writers Association conference on Saturday.
Caref, who was surprised when the district began piloting its own framework, notes that the Danielson framework has been around for more than a decade and is well-vetted.
Consortium researchers did not give teachers in the pilot an overall rating, but instead focused on the various aspects of instruction and classroom management in which teachers were rated and analyzed whether their value-added test scores matched up.
Across almost all of the Danielson Framework components, teachers with the lowest ratings had the lowest value-added test scores and those scores increased as the teacher’s rating increased, according to the report.
But this pattern didn’t hold true in all instances. For reading and math scores, the pattern did not hold in the areas of “creating an environment of respect and rapport." For math scores, the patten did not hold true for “managing classroom procedures” and “organizing physical space." (Researchers note that few principals gave teachers low marks in these areas.)
Also, researchers found that the ratings and value-added test scores were more likely to align in the area of instruction than with classroom management.
The proof that the framework can identify good teaching is helpful for another reason: Students aren’t tested in subjects such as social studies and art, and these teachers don’t have classroom test scores on which they are to be measured.
Rather than create tests in these subjects, the validity of the framework means that it can be counted on to pinpoint who are good teachers, said University of Chicago Urban Education Institute Director Tim Knowles.
Another question researchers sought to answer is whether principals have the ability to properly evaluate teachers. The key finding is that principals reliably peg bad and middle teachers, but were more likely to peg teachers as distinguished, when outside observers gave them only proficient marks.
Almost 30 percent of principals were either more severe or more lenient than outside observers. Yet, the researchers suggest that the principals were more likely to be right than the outside observers. Those with distinguished ratings had higher value-added scores than those with proficient. “Maybe the principals know something the outside observers don’t,” Stoelinga said.
Still, when CPS adopts a new framework, some checks will have to be built in so teachers are not adversely affected by principals who are more severe than others.
“If principals are too severe, this could affect whether a teacher gets tenue or not,” she said.
The Consortium report includes an example of a principal who did not embrace the framework and it suggests that if principals aren’t trained properly, the framework would become too subjective.
While the new evaluation will play a big role in whether teachers get tenure and keep their jobs, researchers and experts note that its best use will be to help teachers improve.
“It gives teachers and principals a shared language about good practice,” Stoelinga said.
But whether principals will be able to help teachers improve with the evaluation depends on the training and support they are given. The researchers found that the conversations that principals and teachers had with the new framework were more reflective and based on concrete evidence. However, they note that the principals did most of the talking and that, for the most part, they didn’t ask teachers hard questions.
Promise, but caution, too
"Illinois' Senate Bill 7, passed in 2010 requires that by 2013, test scores count for a significant portion of a teacher’s evaluation."
SB7 was passed in the spring of 2011; Sarah Karp means to say that the Performance Evaluation Reform Act of 2010...yada yada yada.
While I agree the pilot shows "promise," it causes me no small amount of concern that it hinges on trained principals. Remember that the principals in the pilot schools were enthusiastic enough to volunteer for the pilot program. If the percentage of unsatisfactory principals is as high as the percentage of unsatisfactory teachers ("about a third"), then implementation is going to be a nightmare for some schools.
i agree
However, I hate to use it the old saying. You cant turn a sows ear into a silk purse. It's like we have invented 10 different ways to prove we have diabetes. How are we going to get to these elusive common cores.....with old books, crowded classrooms and an admin that is h-bent on destroying it's teachers morale...rather than giving them hope and new materials. If rahm and brizzard spent more time working and devolping their schools than greasing the tribune/sun times spin machine, we might actually be able to use these new standards to our advantage!
Relationship of Danielson Rubric to Teacher Ratings
There is nothing in the Danielson rubric itself that would necessitate 1/3 of teachers being rated as unsatisfactory. How ratings on each component of the rubric roll up to a summative evaluation rating of the teacher is a matter for negotiation. The Chicago Teachers Union would never agree to a process that labeled one-third of the teachers unsatisfactory.
Objectivity
If test scores correlate with subjective observations, why isn't it good to have test scores form part of the assessment? That way, a good test score can balance an unfairly negative subjective rating, and vice versa.
The problem with test scores
One problem with using test scores is that they do not give teachers feedback about how to strengthen instructional practices. Also, we know from 10 years of NCLB that using test scores for high stakes decisions leads to a narrowing of the curriculum and in some cases, to cheating.
The problem with testing data
I thought the report itself at pages 11-12 correctly presented the problem with using value added measurement when you have complex students who have multiple teachers, especially special education students. The Consortium report states:
"Due to data limitations, we may be underestimating the relationship between ratings and value-added indicators. While researchers and practitioners have highlighted issues with using value-added measures for evaluation purposes, we want to mention some problems that relate specifically to limitations in data systems and in identifying which students are actually taught by which teachers. Evaluating teachers based on the achievement of their students, as measured by standardized tests, means that districts must be able to identify which teachers teach which students. Elementary schools are relying more and more heavily on team teaching and arrangements that allow for more differentiation of instruction for students. These flex¬ible arrangements make it difficult to pinpoint—at least with the current data systems—which teacher is teaching each individual student and how to attribute student growth accordingly. While we do not know the rate at which elementary schools are using these strategies, we do know that they are common. And the problem of matching students to teachers is a bigger one in schools that use flexible teaching arrangements to a greater extent."
Rod Estvan
R.O.I. - Testing vs. professional developement
Testing costs money, and that money by and large leaves the municipalities where the testing occurs. The ramping up of testing, especially during times like these when education budgets are being slashed, means less money available for that which actually educates kids. Testing only identifies what happened in the past, and is incapable of giving any indication of why and how it happened. Understanding the why and how is the critical information obtainable only by observation which then defines what specific support is needed by individual teachers to improve. This seems a much better use of our time and money as a way to improve educational outcomes. Testing contributes nothing to teacher improvement or student knowledge. Used as an evaluation tool, it's therefore redundant and inferior to other methods. Based on the well known problems and limitations of VAM, testing as tool for high stakes decisions, even in part, is ill advised. At the very most we should allow it to be done for only as long as it takes to prove that focusing on supporting and improving teachers is all that's needed from the in school side of the equation. If we improve teaching in this way then any failure of student achievement to rise must be the result of out of school factors. The proposed use of VAM and test scores as high stakes decision tools and the resulting turnover of falsely blamed teachers means that the extent to which out of school factors affect learning will never be acknowledged. This is how the steady river of snake oil cash flowing out of our schools as they continue to decline will be defended.
Will the Board Bargain?
Not sure if Carol was responding to my comment or not, but I put "about a third" in quotations because that was what came from the article. I would certainly hope that no quota for finding unsatisfactory teachers be part of the evaluation process. I was really just trying to make the point that we should expect to have a number of unsatisfactory principals, as well.
But that's all immaterial if the Board doesn't bargain in good faith with the CTU. Under PERA there really is no incentive for the Board to negotiate with the Union. (Well, other than human decency.) Only in Chicago, if the Board and Union cannot agree on a process, the Board gets to impose it last, best offer, and teachers must live with it.
Have the negotiations begun yet? And if so, has the Board given any indication that it will bargain with the Union as a partner?
Save money
Why do we bother with a school board? Why don't replace them with a 10 dollar rubber stamp from Kinkos? BTW why didn't the unions address this at the sb7 meetings? Why is chicago the only city in the state where the Mayor can appoint ALL memebers plus it's ceo In the end shouldnt Rahm and Daley be 100% with any ISAT common core results?? And if they get two unsatisfactory years? They may never teach/lead again? Oh ....no that's just the teahers
Congratulations on having
Congratulations on having Hands down the most sophisticated blogs Ive come throughout in most time! Its just incredible how much you can detract from a thing mainly because of how visually beautiful it’s. Youve put collectively a very good blog space -great graphics, videos, layout. This really is undoubtedly a must-see weblog!
Thanks for your posting on
Thanks for your posting on this blog. From my own experience, often times softening upwards a photograph could provide the photography with a chunk of an inventive flare. Oftentimes however, that soft clouds isn't what precisely you had in mind and can frequently spoil a normally good image, especially if you anticipate enlarging this. regards!
Thanks for your posting on
Thanks for your posting on this blog. From my own experience, often times softening upwards a photograph could provide the photography with a chunk of an inventive flare. Oftentimes however, that soft clouds isn't what precisely you had in mind and can frequently spoil a normally good image, especially if you anticipate enlarging this. regards!
Add your comment