A Middle Path on Teacher Evaluation - Somewhere Between VAM and Irresponsibility

I AM DEEPLY CONCERNED ABOUT WHAT’S GOING ON IN NY - AND THEREFORE ELSEWHERE - WITH REGARDS TO TEACHER EVALUATION SYSTEMS AND RTTT.  THIS POST IS MEANT TO DEFINE A PATH FORWARD THAT INCLUDES STUDENT ACHIEVEMENT DATA BUT MOVES AWAY FROM OBFUSCATED AND HIGHLY QUESTIONABLE VAM MODELS OF “DATA-DRIVEN TEACHER EVALUATIONS.”  IF YOU WANT TO READ ABOUT THE CONTEXT FOR MY CONCERNS, PLEASE CLICK HERE AND FIND MY EARLIER POST.

I HOPE YOU’LL LEAVE SOME FEEDBACK AND CONTINUE THIS CONVERSATION.  WE CAN TALK HERE OR ON TWITTER.

As always, I want to start my thinking with questions that need answering instead of solutions I’m hoping will fit.  In this case, I’m wondering: What expectation(s) can schools actually and realistically have for teachers when it comes to students’ learning, including standardized testing results?

I know this sounds crazy to ask, and many of you are already thinking that I’m about to lower the bar and let teachers off the hook when students don’t achieve, but that’s not where I’m going. I am, though, very hesitant to back value-added models of evaluation that promote paranoia and professional doubt and suffer from flawed formulas, flawed philosophies, myopic accountability plans, and deep obfuscation.  I also believe proactive and reactive action, not accountability, ought to be policy’s goal.  Are we - as a culture and a profession - looking to blame people for a problem or prompt people to find a means of moving towards its solution(s)?  

So, I need to hone the next phase of my initial question… Can schools ask teachers for:

1) an analysis of what the data from required standardized test(s) tell them about their students?

2) evidence of their ability to formatively and summatively assess their students so that they will create data for things they want to know about their students that aren’t portrayed by the standardized tests?

3) evidence that their curriculum and pedagogy continually reviewed and revised so that they are appropriately designed to move students from their current achievement levels to proficiency and mastery of the desired standards for the course?

Why do I like these questions?

1) They reinforce the idea that teachers’ work must be steeped in students’ learning.  There is a non-negotiable connection between data, curriculum, and pedagogy.

2) They allow for teachers’ growth over the course of time instead of unrealistically expecting them to be experts at helping each students in each course as soon as they meet.  Not doing this would be akin to telling doctors that they’re no good if they have to “run a test” to see what’s going on.  

3) They allow for the application of data from local and standardized testing data without suggesting that one is more important than the other.  There is no unilateral approach to improving teaching and learning.  It’s why testing-based, standards based, and technology based reforms will all struggle until they include a bigger picture.  

4) They recognize that it’s teaching appropriately that moves students forward, not “holding high expectations.”  Without great teaching, holding high expectations is nothing but a bunch of soundbites.  

5) They force conversations between evaluators and teachers and between teachers and students.  Currently, VAM models can be - in fact, I believe almost all are - enforced by a computer in a room somewhere.  Students get matched with a teacher.  Scores go in.  Teacher gets a grade in the mail.  This model pushes teachers to get to know their students and their needs as well as administrators to deeply get to know their teachers’ work.  It truly couldn’t be done remotely.

6) They provide a ton of room for a lot of supportive feedback loops and collaboration along the way.  There are no surprises when you’re not hoping to get all of your answers from a one-time event like a major exam.  

7) They support the very real idea that teachers are going to struggle and potentially even fail at times.  It’s knowing how to recover and who to ask for help that’ll ultimately make them great at their work.  If we’re being honest, we all know that that’s a truth for anyone.  

Do I, then, believe that there’s a place for students’ actual scores in a teacher’s evaluation?  Until I see some serious research about this working well at an aggregated level, I’m just going to say that even if it’s possible, it doesn’t seem to be worth the infighting and political capital it’ll take. It’s not worth the divisive and desperate culture that’s manifesting in our schools today.  I also know that I do fully believe that if we get teachers working towards purposeful and researched classroom practices, the achievement we want will happen.  

How can we move to this?  It’s simple, I think.  Let’s take out the pieces of states’ evaluation systems (in NY, it’s worth 40%) that account for testing scores, and instead create a rubric around the three questions of data-based practice I’ve outlined above.  We’ll add it to whatever observation rubric is in use and move forward from there.  

The Hot Mess of RTTT, Teacher Evaluations and Student Testing Data In NY

THIS POST IS MY TAKE ON THE CONTEXT AROUND TEACHER EVALUATION AND STUDENT ASSESSMENT DATA IN NEW YORK (AND OTHER STATES).  IF YOU WANT TO SKIP THIS AND GO RIGHT TO READING MY MIDDLE-PATH PLAN ON MOVING FORWARD, HERE IT IS.

I’ve started this post five times this morning.  Because it’s about Race to the Top, teacher evaluation systems, and New York education, I have a lot I’d like to say, and I wouldn’t even be able to scratch the surface.  We’ll have to start in the present, then, and assume that if you’re reading this, you’re probably aware of the history of our Federal government’s intervention into public education that includes No Child Left Behind and Race to the Top.  Truthfully, I hope someone is putting together the objective history of these last bunch of years.  I don’t believe the debate or the stances are anything new, but the time period between A Nation at Risk and 2020 will make for an extremely interesting case study, one in which all of the arguments and the attempted fixes are elevated to an extreme at which our modern world is becoming far too talented.  

I want to focus on a tiny piece of this milieu: teacher evaluation and student-testing data.  I’m talking about the idea of using data on students’ performance on standardized tests as a large piece of a teacher’s overall evaluation.  In order to measure this, “experts” have created a method called VAM (Value-Added Measurement) to ostensibly assess how valuable a student’s time in any particular teacher’s class has been.  If you want more background on it, try this, this, this, and/or this article/feature/blog.  This is statistics and algorithms at a whole new level; it’s highly controversial, and one of the two pieces that have brought legislators and the Board of Regents in NY to a deeply unhealthy level of uncertainty and infighting.  

I believe, when it comes to implementing top-down initiatives, that I should work my hardest to change towards what good for students and take a “path of least compliance” with anything else.  This means that I know I work for people and for a system, but if my team(s) and I don’t see how something’s not beneficial to students, I’m not going to push to be ahead of the curve on it. I’d rather see how things will play out.  NY, however, opted to be an early adopter of not only the Common Core Standards, of which I’m a supporter, but also the idea that grades 3-8 testing must be “aligned” to the Common Core.  If this isn’t making sense, realize that all RTTT states are in a transition between their own testing systems from NCLB and either the PARCC or the SB consortiums’ tests, which have release dates that are coming soon but haven’t yet hit.  While there are many differences that are being expected from these new tests, the most important is supposedly going to be their ability to accurately track how much a student is learning with a given amount of time in a particular teacher’s room, VAM. NYs desire to be out in front meant that it’s own tests were “transitioned” to a version that more “aligned” with the  CCs increased “rigor.”  These would better measure what students “actually” knew and could do, and that would allow them to best be used to assess what is being learned in the teachers’ classrooms.  

This plan has been unravelling for a few years now for the following reasons, in no particular order:

1)   Parents, teachers and students are very upset about the amount of testing many of states’ plans involve.  This includes the states’ tests and, in NY, the assessments being used as “local measures” of student performance.  While few people are listening to teachers’ opinions, the parents’ and students’ voices are disrupting the process.  The “opt out movement” (see this as a general version of grass root action) is generating great amounts of anti-testing press and threatening schools’ ability to have enough students taking the tests to meet AYP (Adequate Yearly Progress) requirements.

2) The tests themselves have issues.  From poor questions to marketing within test questions and a lexile level of questions that seem absurdly inappropriate for kids, there have just been too many questions about the tests themselves for most people to feel okay about it.  Let’s also add the truth that many of the big test makers (in NY, it’s Pearson) is also profiteering by selling it’s curriculum and workbooks to schools as a way to prepare for its tests.  

3) You can’t sell failing grades to parents with a smile and expect to walk away.  It’s truly unbelievable to me that those in charge of NYs education plans - the governor included - haven’t done a better job predicting and handling public backlash over all of this.  When the tests became more “rigorous,” the grades were expected to drop.  This makes sense, but NY went ahead and opted to count these exams for all sorts of things, including teacher evaluations, instead of finding a way to pilot new questions.  They then went about insulting parents and dismissing all concerns.  They’re taking a “we don’t need your hearts and minds behind this since we’re more powerful than you are” approach that’s now haunting them.  They simply could have said: “We’re not going to have cut-scores that define student achievement this year.  It’s a trial. We’re going to analyze our data to figure out where our students are and how where we have to improve our efforts so that we’re all ready for PARCC.”  It just sounds better than: “Many more of you will fail this year than last because your teachers aren’t that good and you aren’t as smart as you’ve always thought you were.” Duh.  

4) Although both PARCC and SB claim that their tests are going to be able to measure students abilities with higher-order thinking and the upper reaches of Bloom’s Taxonomy, nobody has shown how our current tests are doing that, so - at least for now - very few people are interested enough in what the tests can actually tell us.

5) Increasing the rigor of a test as new curricula is being written around standards that have just somewhat recently been released isn’t going to give accurate measures of a student’s learning in a class.  Rushing the curriculum writing and the testing rigor only makes VAM less believable.  Teachers unions now held the moral high ground on what’s good for students (alongside parents, which made their case stronger than ever) and the professional validity to publicly denounce VAM and therefore the overall APPR (Annual Professional Performance Review) program that came along with it.  

Whew, we’re almost caught up.  Just this month, NY realized it has to bail, at least a bit.  If you read this, you’ll rejoice that NY is going to follow Massachusetts’ lead and take some time to consider their initial plan and adjust things as needed before students and teachers are held accountable for the Common Core curricula to which schools are still transitioning.  This leaves a ton of questions on the table, but it means that fewer people (students and adults) will be crying at night over the attached judgements and career-defining evaluations the way they were last year.  Questions I have:

1) Is there now a plan to ensure schools are moving forward with curriculum transitions?

2) Is there now a plan to gauge test validity and reliability?

3) What does this delay do to NYs relationship with RTTT funds?

4) What are this year’s exams going to look like?

5) Is PARCC going to become hesitant to go ahead with its plan for rigorous assessments?

6) What exactly will teacher evaluations look like since 20% of NYs were based on these assessments?

And finally, there’s this article, in which we learn that NYs teacher evaluation system is to somehow move forward, as is.  What a hot mess.  

MY NEXT POST ELABORATES ON HOW NEW YORK - AND ALL STATES - CAN MOVE PRODUCTIVELY FORWARD WITH THE CONNECTION BETWEEN STUDENT ASSESSMENT AND TEACHER EVALUATION.

Where All Students Learn

We all have talking a lot about the need to address all of our students' needs. It's our mandate, so I'm hoping your school has embraced the call. It's bigger than new programs...bigger than technology integration and standardized testing. It's even bigger than - I think - the achievement gap issue in that we're talking about the needs of all of our students in all of our schools.

None of these are easy to do, and each requires a number of posts and plans for bringing it to fruition, so I don't mean to sound overly casual here. These five pieces, though, represent what I feel are non-negotiable pieces to making it happen.

As always, your feedback is deeply appreciated.

1) Adapting the Right Mindset

Educating teenagers is extremely difficult work that has to start with the right attitude.  Simply put, everyone’s “hearts and minds” have to be in a place that says all children can learn and that it’s our job to support them through the necessary processes.  Every school needs to somehow include Carol Dweck’s Mindset (or some parallel research) into their teachers’ and students’ work as it details the truth that all people can learn more than they currently know and become better than they currently are.  It’s not the overly optimistic hype that says anyone can be a surgeon or a physicist, but instead, it’s the very strong reality that who we are today is not who we need to be tomorrow.  Being able to meet the needs of all students means that a school has to believe that all teachers and learners can and must grow.  If we don’t believe in everyone’s potential, our school culture will reflect that; the limits and negativity will be palpable in the conversations and interactions between teachers, students, administrators, and staff; people will continue to struggle as a result.

2) Clearly Defining Quality Benchmark Standards  

Once we believe all students can grow and achieve, schools will embrace the process involved in identifying, accepting, and teaching to high and interesting benchmark standards for all students.  It’s important to have these standards defined in ways that are understood by the students and addressable in our assessment systems because that’s how students, teachers, and families are going to know what students are learning and what they’re having a hard time understanding.  As we are better able to understand exactly where students are struggling or not, schools will become increasingly better at helping them take steps forward.

3) Building Relationships, Allowing for Choice and Voice

A tremendous piece in educating teenagers and engaging anyone is showing them respect. Part of our best practices ought to be finding ways to include students’ voice and choices in their education.  If we’re actually looking to meet the needs of all learners, we really need to talk to them and build relationships with them, which will include considering what information would interest them, how best to engage them, and what chances can we give them to show what they know.  Education should be something done with students, not to students.  

4) Finding and Supporting Solid Tier-One Strategies for All Classrooms

As I said earlier, teaching teenagers is extremely difficult, nuanced work when it’s done right.  It’s a bigger task than ever to find ways to appropriately use assessment data and differentiate lessons while properly addressing core content and skills.  We must have opportunities to continually research, try, share, observe, reflect, and improve upon classroom practices.  Teachers have to feel safe when they find something new that may work.  Setting up a mentoring system that includes 1:1 time, collegial intravisitations, critical friends groups, grade-level professional learning communities, and vertical departments codifies a culture of learning and sharing what works best for students, which means that fewer students will need further interventions to find success.  

5) Creating and Using a 360 Support System

Learners struggle for a variety of reasons, and schools who want to meet all learners’ needs must have not only a wide range of academic, social, and emotional supports ready to help students, but also dedicate time to helping all adults in the building serve as that first set of eyes and ears who will notice when students need help.  I actually believe that addressing students’ emotional needs can pay tremendous learning dividends by making students know that the school’s adults care about them and can most often happen casually by building relationships in classes.  Using frameworks such as UDL and UbD will dovetail with this relationship-building philosophy by solidifying a purposeful and differentiated academic system for teaching courses’ content and skills.

Modernizing Schools' Work

My Professional Development Philosophy:

I like to plan PD in ways that afford faculty members as much input and freedom as possible.  This “teacher-centered” approach not only models best practices for classrooms’ “student-centered” work, but also recognizes my belief that a system must be led through a democratic, collegial and trusting culture.  I may be able to set our priorities and bring in initiatives, but If I don’t trust my faculty with a voice, I’d be establishing an adversarial, top-down culture, and I don’t believe that’s how schools operate optimally.

The Process Starts With a Question:

I believe that best practices for faculty learning always start with a question instead of a mandate in order to safely engage everyone as thinkers and participants.  An example of a question that could drive a faculty’s modernization would be: “How has the world changed since the turn of the century in ways that will be important to our students’ futures as citizens and professionals?” Answering this would be a starting point for an ever-evolving, organic list that we can continue to debate, organize, and consider this list as time goes on.  Obviously, it’s going to push us to consider how we’re going to include them in our curricula.  

Let’s Compare Our Own Thinking to Work from State and National Organizations:

Certainly, there are pieces of this work being mandated by Massachusetts, such as the Curriculum Frameworks for Math and ELA, but there are also a number of groups and professional leaders that may have ideas that should be included in our thinking.  Examples of these optional sources for ideas include the NextGen Science Standards, C3 Social Studies Standards, P21, ISTE and people like Tony Wagner of Harvard, Thom Whitby, who is well known for innovative perspectives on what’s coming next in education and as a founder of #edchat on Twitter, and someone like @johnfalino1, who is leading his school's modernization process the right way and often posts the faculty's processes on his blog.  While all of these respected sources should be considered, I’d imagine that a faculty may know of even more sources.

Using Our Updated Student-Learning Goals to Guide Pedagogical Change:

Pedagogical choices are best when considered as a function of desired learning outcomes.  We need to ask: “What are the classroom practices that will best help our students master ____?”  There may be internal success and someone who can lead the learning on this, but it also may require research teams.  In some instances, this will mean deciding when to include certain new or different information while other pieces such as metacognition, technological facility and multi-disciplinary literacy will take deeper consideration since they need to be woven throughout various coursework.  Our student-learning goals ought to lead to PD that is individualized for each teacher.  

Assessing What We Believe is Important:

I want a lot of varied data to inform a school’s work.  I understand that assessment has received a lot of negative publicity since the inception of NCLB and RTTT, but that doesn’t mean that standardized-testing data doesn’t tell us anything.  Beyond it, we also must create assessment systems that give us the information we feel is missing from state and nationalized exam scores.  We must have a complete assessment system that tells us whether or not our students are appropriately progressing towards our desired achievement goals.  

There is a tremendous amount of research and innovation in this area that ought to be explored.  We must look at our use of formative and summative assessments, the commonality and quality of our rubrics, transitioning to standards-based-grading strategies and even considering including student portfolios, where they would get to write reflections explaining their growth and choose pieces of work where they feel that proficiency/mastery is most evident.  

In addition to student assessment, we’d also need to ensure that our teacher observation and evaluation system is aligned with modern best practices.  Teachers must be accountable for including the school’s goals in their curriculum, for showing an appropriate level of growth in the goals they have set, and for understanding how their students’ achievement levels appropriately drive them to adjust their coursework and their own goals as is necessary.  

I love and believe in this plan because it’s focused and detailed while leaving a lot of room for individualization and flexibility.  It ensures that teachers and the whole school are set up to help each other move forward on behalf of students’ needs without being prescriptive.  It leaves plenty of room for local ownership as well as state and federal government mandates.  It means that teachers don’t need to be disruptive if they are willing to be involved.

A New Piece for My Work-Flow Strategy

Yes, I've added a fourth email.

I think I’m a pretty organized guy, and I’m really proud of my ability to share great resources with but I know that I’m a mere amateur compared to some of my PLN when it comes to content organization and curation.  I’m working on a post that talks about my overall Chrome-based productivity life, but today, I did something that made me stop and think - I created a fourth email address to help organize my world.

I’m thinking about this not only for me, but also for the kids in our schools, and mine at home, who are growing up in a digital world.  I have set up emails for my own kids that they don’t use so that I can send them things I want them to have in the future, but if your students are like mine, they have email and use it randomly and rather haphazardly, to be honest.  I’m seeing this as one of those pieces that we may be missing if we mistakenly believe that our “digital natives” are strong, reflective “digital citizens.” Even if students claim to no longer use email, I’m not sure I’m seeing it go away.  Yes, there are other ways to message people and share files, but I’m still going to bet on email’s remaining a relevant part of the mix.  

What four emails do I now have:

1) This is my junk file.  My brother, who’s very involved in an academic arm of internet privacy and safety groups, once wisely said: “If something is being given to you for free, it’s likely that you are the product being sold.”  I sign up for a lot of apps and other resources, and I’m almost 100% against using my FaceBook account to do so.  Twitter and LinkedIn sometimes get used, but those functions are specific to those sites, so I accept the trade-off.  What this means is that I’m giving out my email a lot, which I’m assuming means that it’s getting sold a lot.  I have an email account that they can stuff all they’d like.  I scroll through it and double check for anything interesting, but it’s 99.5% junk.

2) My personal mail.  This is where I talk to my friends and others whom I actually want to talk to.  It’s also my gmail, so Google + and Google Drive run with it.  Yes, I know that Google may just be out there selling my existence, but I don’t get any junk mail there, so I’m happy.

3) My professional account.  Work life these days is flooded with email.  Usually, most of us will have an email assigned to us at work.  In NYC (and I assume a lot of other places are also this way), DOE employees’ emails were so limited in function and memory and access, that nobody used them, meaning that I actually had two professional emails (one official one for anything confidential and the other for communicating with staff on a regular basis).

4) This is my new one.  It’s for “resources,” and I’m the only one who has it.  I am currently piecing together a post about my strategy for work flow and long-term content curation, but I’m swimming in content and I just can’t get to it all in the moment to figure where I want it to live, so I created an easy place to send things until I have time to figure it out.  I used to “favorite” tweets and go back to that list, but there are just too many to make that realistic.  I also now use “favorite” to also mean: “I agree so much that I’m going to share with other but don’t feel the need to reply that sentiment to you.”  While I used to open the link and then email it to my personal account, I now email the whole tweet to this new account so that I have the link / information and the source so that I can go back and thank whomever shared it. This is also a place where I can pull things from Feedly and Facebook and emails and general web browsing.

I’m curious to know if our middle school kids are being taught about having separate “registration,” “personal,” and “school” accounts.  Maybe at one point these were all tough to check, but iPads mail functions put them all together, so it’s easy.  Maybe at one point, email will become fully moot, but until then, I’m happy with my strategy.