State and District Perspectives on Educator Evaluation, Assessment, and Effectiveness Management: Integrating Policy, Practice, and Partnerships.

Slide1State and District Perspectives:Putting Policy into Practice Educator Evaluation and Assessment Mike

Slide2policy & practice integration: how it all fits together:Mike

Slide3Measures of educator effectiveness and studentpostsecondary and workforce readiness Effectiveness Management Increase and support the effectiveness of all educators Talent Pipeline Attract and develop the best educators Strategic Partnerships/ Committees Strategic Partnerships/ Committees CDE’s Educator Effectiveness  Unit Policy, Metrics and Monitoring Recruitment Educator Preparation Licensure Hiring/ Placement Induction Professional Dev. Evaluation And Support  Retention Equity Initiatives Vision: Effective educators for every student and effective leaders for every school Mike

Slide4Senate Bill 10-191• A system to evaluate the effectiveness of licensed personnel to improve the quality of education. • Improve instruction. • Serve as a measurement of professional growth and continuous improvement. • Provide a basis for making decisions in the areas of hiring, compensation, promotion, assignment, professional development, earning and retaining non-probationary status, dismissal, and nonrenewal of contract. Mike

Slide5Critical Effects of S.B. 10-191• Requires statewide minimum standards for what it means to be an                   teacher or principal • Requires that all teachers and principals be evaluated at least           on the academic growth of their students • Prohibits              placement of teachers • Makes non-probationary status “effective” 50 % forced “portable” Mike

Slide6Critical Effects of S.B. 10-191• Requires             evaluation of all teachers and principals • Changes non-probationary status from one that is        based upon years of               to one that     is              based upon three consecutive years of     demonstrated • Provides that non-probationary status may be based upon two consecutive years of annual earned service earned effectiveness lost ineffectiveness Mike

Slide7Tensions• validity vs. reliability • all students vs. sampling • local scoring vs. outside scoring • summative vs. formative • holistic vs. analytic • stand-alone vs. embedded • one-year’s growth vs. differences in resources (instructional time, etc.) • mandate by edict vs. preparation through professional development Moving Through Tensions  Guiding principles to assist in making sense of the requirements through practical implementation

Slide8Principles of Implementation• Human judgment – Data should inform decisions, but human judgment will always be a part of the process – Processes and techniques are recommended to improve individual judgment and minimize errors and bias • Embodiment of continuous improvement by monitoring – Pilot and rollout intended to capture what works and what doesn’t – Changes in assessment practices and tools – Emerging research and best practices Mike

Slide9•Providing credible and meaningful feedback with: – Actionable information – Opportunities for improvement – Idea that this is a  process  and not an event • Involves all stakeholders in a collaborative process – Families, teachers, related service providers, administration, school board, etc. Mike Principles of Implementation

Slide10•Takes place within a larger, aligned and supportive system – All components of the system must serve to increase the number of educators and students who are successful • Turn and Talk – Why is it important to understand these principles? Mike Principles of Implementation

Slide11Definition of Principal EffectivenessEffective Principals in the state of Colorado are responsible for the collective success of their schools, including the learning, growth and achievement of both students and staff. As schools’ primary instructional leaders, effective  Principals enable critical discourse and data-driven reflection about curriculum, assessment, instruction, and student progress, and create structures to facilitate improvement. Effective Principals are adept at creating systems that maximize the utilization of resources and human capital, foster collaboration, and facilitate constructive change. By creating a common vision and articulating shared values, effective Principals lead and manage their schools in a manner that supports schools’ ability to promote equity and to continually improve their positive impact on students and families. Courtney

Slide12STATE COUNCIL FOR EDUCATOR EFFECTIVENESSFramework for System to Evaluate Principals Definition of Principal Effectiveness I. Strategy II. Instruction III. Culture V. Management IV. Human Resources VI. External Development VII. Student Growth 50% Professional Practice Standards 50% Student Growth Measures Weighting: How Much Does Each Standard Count Towards Overall Performance? Number and Percentage       Other Measures of Teachers Aligned with CDE Guidelines School Performance          Other Measures Framework                     Aligned with CDE                     Guidelines Weighting: Scoring Framework: How Do Measures of Quality Standards Result in a Determination of Individual Performance? Performance Standards Ineffective Partially Effective Effective Highly Effective Quality Standards Courtney

Slide13Principal Quality StandardsI:  Principals demonstrate strategic leadership. II:  Principals demonstrate instructional leadership. III:  Principals demonstrate school culture and equity leadership. IV:  Principals demonstrate human resource leadership. V:  Principals demonstrate managerial leadership. VI:  Principals demonstrate external development leadership. VII:  Principals demonstrate leadership around student academic growth.

Slide14Tensions• validity vs. reliability • all students vs. sampling • holistic vs. analytic • stand-alone vs. embedded • one-year’s growth vs. differences in resources (instructional time, etc.) • mandate by edict vs. preparation through professional development Moving Through Tensions Application guidance to assist and/or provide choices in making sense of the requirements through practical implementation

Slide15Application of Quality Standards• Each quality standard includes “elements” — which provide a more detailed description of the knowledge and skills needed for each standard. • All districts must base their evaluations on the full set of quality standards and associated elements or on their own locally developed standards that meet or exceed the state’s quality standards and elements. • Some districts are using their own locally developed standards after completing a crosswalk of their standards to the state’s quality standards and elements.  These districts must provide assurances that they are meeting all additional requirements of SB 10-191. Courtney

Slide16STATE COUNCIL FOR EDUCATOR EFFECTIVENESSFramework for System to Evaluate Principals Definition of Principal Effectiveness I. Strategy II. Instruction III. Culture V. Management IV. Human Resources VI. External Development VII. Student Growth 50% Professional Practice Standards 50% Student Growth Measures Weighting: How Much Does Each Standard Count Towards Overall Performance? Number and Percentage       Other Measures of Teachers Aligned with CDE Guidelines School Performance          Other Measures Framework                     Aligned with CDE                     Guidelines Weighting: Scoring Framework: How Do Measures of Quality Standards Result in a Determination of Individual Performance? Performance Standards Ineffective Partially Effective Effective Highly Effective Quality Standards Courtney

Slide17Principal Evaluations50% Professional Practice 50% Student Academic Growth I. Strategic leadership II. Instructional leadership III. School culture/equity leadership IV. Human resource leadership V. Managerial leadership VI. External development leadership I. Strategic leadership II. Instructional leadership III. School culture/equity leadership IV. Human resource leadership V. Managerial leadership VI. External development leadership Measured using multiple measures on multiple occasions, including tools that capture: (1) teacher input; (2) number and percentage of teachers with each; and (3) number and percentage of teachers who are improving in their performance, in comparison to the goals articulated in the principal’s professional performance plan. Measured using multiple measures on multiple occasions, including tools that capture: (1) teacher input; (2) number and percentage of teachers with each; and (3) number and percentage of teachers who are improving in their performance, in comparison to the goals articulated in the principal’s professional performance plan. VII. Leadership around student academic growth VII. Leadership around student academic growth Evaluated using the following: (1)data included in the school performance framework; and (2) at least one other measure of student academic growth. Evaluated using the following: (1)data included in the school performance framework; and (2) at least one other measure of student academic growth. Courtney

Slide18Components of the Principal RubricStandard I:  Principals Demonstrate Strategic Leadership Not Evident Partially Proficient Proficient Accomplished Exemplary a.   School Vision, Mission and Strategic Goals:   Principals develop the vision, mission, values, beliefs and goals of the school, collaboratively determining the processes used to establish these attributes, and facilitating their integration into the life of the school community. Vision, mission, values, beliefs and goals of school are:  Not evident or familiar to staff and other stakeholders.  Developed by school administrators working in relative isolation.  Not integrated into the life of the school community. Vision, mission, values, beliefs and strategic goals of school are:  Developed through a collaborative process with staff and other stakeholder groups.  Publicly available at the school.  Part of routine school communications with staff and other stakeholders.  Routinely updated. . . . and Establishes strategic goals for students and staff that are:  Focused on student achievement.  Based on the analysis of multiple sources of information.  Aligned with district priorities.  Measurable.  Rigorous.  Concrete. . . . and  Staff incorporate identified strategies in their instructional plans to assure that students achieve expected outcomes.   . . . and  Staff and other stakeholders take leadership roles in updating the school’s vision, mission, and strategic goals.  Staff members assume responsibility for implementing the school’s vision, mission, and strategic goals. Quality Standard Element of the standard Rating levels Professional Practices Courtney

Slide19examples of artifacts that may be used to support Rating:Evidence Provided by Artifact:  Unified Improvement Plan*  Teacher Feedback*  Parent Feedback  Student Feedback  Supervisor Feedback  Meeting agendas, minutes, and rosters  Quarterly Reports to SAC  Teacher Lesson Plans  Minutes of Planning Sessions  Teacher Turnover Rates  Emails and memos to staff  Descriptions of processes and procedures  ICAP  Parent newsletters  School vision, mission, and goals  Business and/or community resource agreements Ratings: (# points per rating at this level) NE (0) PP (1) P (2) A (3) Ex (4) Total Points 0 to 2 points  = Not Evident 3 to 6 points  = Partially Proficient 7 to 10 points  = Proficient 11 to 14 points = Accomplished 15 to 16 points  = Exemplary Overall Rating for Standard I = _________________ a. School Vision, Mission, and Strategic Goals a. School Improvement Plan a. Leading Change a. Distributive Leadership Total Points Evaluator Comments: Response of Principal/Assistant Principal Being Evaluated: *Artifact is required for all principals and assistant principals. Evidence provided by artifacts Examples of artifacts Rating Scale Rating points Ratings and point equivalents Evaluator’s comments Principal comment section Courtney

Slide20standard i:  principals demonstrate strategic leadershipNot Evident Partially Proficient Proficient Accomplished Exemplary a.   School Vision, Mission and Strategic Goals:   Principals develop the vision, mission, values, beliefs and goals of the school, collaboratively determining the processes used to establish these attributes, and facilitating their integration into the life of the school community. Vision, mission, values, beliefs and goals of school are:  Not evident or familiar to staff and other stakeholders.  Developed by school administrators working in relative isolation.  Not integrated into the life of the school community. Vision, mission, values, beliefs and strategic goals of school are:  Developed through a collaborative process with staff and other stakeholder groups.  Publicly available at the school.  Part of routine school communications with staff and other stakeholders.  Routinely updated. . . . and Establishes strategic goals for students and staff that are:  Focused on student achievement.  Based on the analysis of multiple sources of information.  Aligned with district priorities.  Measurable.  Rigorous.  Concrete. . . . and  Staff incorporate identified strategies in their instructional plans to assure that students achieve expected outcomes.   . . . and  Staff and other stakeholders take leadership roles in updating the school’s vision, mission, and strategic goals.  Staff members assume responsibility for implementing the school’s vision, mission, and strategic goals. Not Evident  describes practices of a principal who does not meet state performance standards and is not making progress toward meeting them. The focus of  Partially Proficient and Proficient levels  is what principals do on a day-to-day basis to achieve state performance standards and assure that students are achieving at expected levels. The focus of  Accomplished and Exemplary ratings shifts to the outcomes of the principal’s practices, including expectations for staff, students, parents and community members, as a result of practices exhibited under rating levels 2 and 3. Courtney

Slide21Definition of Teacher EffectivenessEffective Teachers in the state of Colorado have the knowledge, skills, and commitments needed to provide excellent and equitable learning opportunities and growth for all students. They strive to support growth and development, close achievement gaps and to prepare diverse student populations for postsecondary and workforce success. Effective Teachers facilitate mastery of content and skill development, and employ and adjust evidence-based strategies and approaches for students who are not achieving mastery and students who need acceleration. They also develop in students the skills, interests and abilities necessary to be lifelong learners, as well as for democratic and civic participation. Effective Teachers communicate high expectations to students and their families and utilize diverse strategies to engage them in a mutually supportive teaching and learning environment. Because effective Teachers understand that the work of ensuring meaningful learning opportunities for all students cannot happen in isolation, they engage in collaboration, continuous reflection, on- going learning and leadership within the profession. Turn and talk with a partner about what resonated with you. Dawn

Slide22STATE COUNCIL FOR EDUCATOR EFFECTIVENESSFramework for System to Evaluate Teachers Definition of Teacher Effectiveness I. Know Content 50% Professional Practice Standards 50% Student Growth Measures Weighting: How Much Does Each Standard Count Towards Overall Performance? Observations of           Other Measures Teaching              Aligned with              CDE Guidelines State       Other Assessments      Other Measures  Summative       for Non-tested               Aligned Assessments       Areas                     CDE Guidelines Match of test to teaching assignments Weighting: Scoring Framework: How Do Measures of Quality Standards Result in a Determination of Individual Performance? Performance Standards Ineffective Partially Effective Effective Highly Effective Quality Standards II. Establish Environment III. Facilitate Learning IV. Reflect on Practice V. Demonstrate Leadership VI. Student Growth Appeals Process Dawn

Slide23Teacher Quality StandardsI:  Teachers demonstrate mastery of and pedagogical expertise in the content they teach. II:  Teachers establish a safe, inclusive, and respectful learning environment for a diverse population of students. III:  Teachers plan and deliver effective instruction and create an environment that facilitates learning for their students. IV:  Teachers reflect on their practice. V: Teachers demonstrate leadership. VI:  Teachers take responsibility for student academic growth. Dawn

Slide24Tensions• validity vs. reliability • all students vs. sampling • holistic vs. analytic • stand-alone vs. embedded • one-year’s growth vs. differences in resources (instructional time, etc.) • mandate by edict vs. preparation through professional development Moving Through Tensions During the feedback timeline, considerations are collected to build guidance to assist and/or provide choices in making sense of the requirements through practical implementation

Slide25STATE COUNCIL FOR EDUCATOR EFFECTIVENESSFramework for System to Evaluate Teachers Definition of Teacher Effectiveness I. Know Content 50% Professional Practice Standards 50% Student Growth Measures Weighting: How Much Does Each Standard Count Towards Overall Performance? Observations of           Other Measures Teaching              Aligned with              CDE Guidelines State       Other Assessments      Other Measures  Summative       for Non-tested               Aligned Assessments       Areas                     CDE Guidelines Match of test to teaching assignments Weighting: Scoring Framework: How Do Measures of Quality Standards Result in a Determination of Individual Performance? Performance Standards Ineffective Partially Effective Effective Highly Effective Quality Standards II. Establish Environment III. Facilitate Learning IV. Reflect on Practice V. Demonstrate Leadership VI. Student Growth Appeals Process Dawn

Slide26Teacher Evaluations50% Professional Practice 50% Student Academic Growth I. Mastery of content II. Establish learning environment III. Facilitate learning IV. Reflect on practice V. Demonstrate leadership I. Mastery of content II. Establish learning environment III. Facilitate learning IV. Reflect on practice V. Demonstrate leadership Measured using multiple measures on multiple occasions, including: (1) observations; and (2) at least one of the following: student perception measures, where appropriate and feasible, peer feedback, feedback from parents or guardians, or review of teacher lesson plans or student work samples.  May include additional measures. Measured using multiple measures on multiple occasions, including: (1) observations; and (2) at least one of the following: student perception measures, where appropriate and feasible, peer feedback, feedback from parents or guardians, or review of teacher lesson plans or student work samples.  May include additional measures. VI. Responsibility for student academic growth VI. Responsibility for student academic growth Evaluated using the following: (1) a measure of individually-attributed growth, (2) a measure of collectively- attributed growth; (3) when available, statewide summative assessment results; and (4) for subjects with statewide summative assessment results available in two consecutive grades, results from the Colorado Growth Model. Evaluated using the following: (1) a measure of individually-attributed growth, (2) a measure of collectively- attributed growth; (3) when available, statewide summative assessment results; and (4) for subjects with statewide summative assessment results available in two consecutive grades, results from the Colorado Growth Model. Dawn

Slide27Quality Standard II: Teachers establish a safe, inclusive and respectful learning environment for a diverse population of students. Not Evident Partially Proficient Proficient (Meets State Standard) Accomplished Exemplary Element c:   Teachers engage students as individuals with unique interests and strengths. The teacher :  Has low-level expectations for some students.  Uses data for instructional decision making on an infrequent basis. The teacher:  Monitors students for level of participation.  Encourages students to share their interests.  Challenges students to expand and enhance their learning. . . . and The teacher:  Asks difficult questions of all students.  Scaffolds questions.  Gives wait time equitably.  Flexibly Groups students.  Assumes that all students will meet or exceed expectations. Modifies instruction to assure that all students:  Understand what is expected of them.  Are challenged to meet or exceed expectations.  Participate in classroom activities with a high level of frequency and quality.  Take responsibility for their work.  Have the opportunity to build on their interests and strengths. . . . and Students:  Actively participate in all classroom activities.  Monitor their own performance for frequency of participation. Seek opportunities to respond to difficult questions. . . . and Students:  Select challenging content and activities when given the choice in order to stretch their skills and abilities. Encourage fellow students to participate and challenge themselves. Quality Standard Element that aligns with standard Rating levels Professional Practices Components of the Teacher Rubric Dawn

Slide28Element e:  Teachers provide proactive, clear and constructive feedback to families about student progress and work collaboratively with the families and significant adults in the lives of their students. The teacher:  Collaborates with others infrequently or only as required in order to address identify, understand and address student learning needs.  Establishes a classroom environment that is not inviting to families and significant adults in the lives of students. The teacher:  Maintains appropriate and respectful relationships with students and their significant adults.  Routinely communicates with parents to better understand student needs.  Is sensitive to the diverse family structures in all communication to/about the student. . . . and The teacher:  Provides tools, materials, and guidance significant adults to help students meet education goals.  Maximizes the home/school connection by coordinating information from significant adults with colleagues who provide student services.  Seeks out services, strategies and resources to meet the diverse needs of students. . . . and Students:  Take advantage of support services made available to them.   In order to deepen their skills and knowledge. . . . and Students:  Seek out assistance such as tutoring, computer software, advice from specialists, and guidance from fellow students to address learning issues. Examples of Artifacts that may be used: Evidence of Performance: Evidence of performance provided by artifact:  Student Achievement Data  Student feedback  Parent feedback  Lesson plans/units of study  Feedback from walkthrough observations  Instructional activities schedules  Student journals/learning logs  Student work  Anecdotal records  Formative and summative assessments of student work Examples of Artifacts Evidence provided by artifact Comments (Required for Ratings of “Not Evident” or “Partially Proficient” and recommended for all rating levels).  Please indicate the element for which the comment applies if not for the standard as a whole. Comments of person being evaluated. (Optional) Evaluator comments Teacher comment section Dawn

Slide29Quality Standard II: Teachers establish a safe, inclusive and respectful learning environment for a diverse population of students. Not Evident Partially Proficient Proficient (Meets State Standard) Accomplished Exemplary Element d:   Teachers adapt their teaching for the benefit of all students, including those with special needs across a range of ability levels. The teacher   does not :  Design instruction to address individual student learning needs.  Collaborate with specialists, colleagues and parents to provide understand student needs. The teacher:  Designs instruction to address specific learning needs of some groups of students (e.g., ELL, LD, special needs, gifted and talented).  Challenges all students with the same frequency and depth and monitors the quality of participation. . . . and The teacher:  Solicits input from parents, colleagues, specialists, and others to understand students’ learning needs.  Implements individualized plans for the content and delivery of instruction.  Uses multiple strategies to teach and assess students.  Adapts instructional strategies to meet student needs. . . . and Students:  Actively participate in all classroom activities.  Articulate an awareness of their learning needs.  Reflect about their learning and make adjustments to accommodate their learning needs. . . . and Students:  Seek out ways to cope with learning differences and apply coping skills to classroom situations.  Share coping strategies and with fellow students. Not evident.    This describes practices of a teacher who does not meet state performance standards and is not making progress toward meeting them. Not evident.    This describes practices of a teacher who does not meet state performance standards and is not making progress toward meeting them. The focus of  Partially Proficient and Proficient  levels is what teachers do on a day to day basis to achieve state performance standards and assure that students are achieving at expected levels. The focus of  Partially Proficient and Proficient  levels is what teachers do on a day to day basis to achieve state performance standards and assure that students are achieving at expected levels. The focus of  Accomplished and Exemplary  ratings shifts to the impact of the teacher’s practices on student outcomes. The focus of  Accomplished and Exemplary  ratings shifts to the impact of the teacher’s practices on student outcomes. Dawn

Slide30Principal and Teacher PerformanceEvaluation Ratings After CDE develops the state model system and an evaluation scoring matrix, the State Board will adopt definitions for each rating. Highly Effective Effective Partially Effective Ineffective Dawn

Slide31district uses state scoring framework matrix to determine Performance Standardaggregate measures Aggregate professional practice scores into a single score on Quality Standards I-V Aggregate student growth measures into a single score on Quality Standard VI data collection procedures Standards I-V:  Must occur with enough frequency to create a credible body of evidence Standard VI:  Must occur with enough frequency to create a credible body of evidence weights On each Standard I-V districts may weight priority standards more Standard VI must count for at least 50% of total score measures Standards I-V:  use observation plus at least one other method Standard VI:  select multiple measures appropriate to teaching assignment Districts decide… Mike

Slide32Denver Public Schools and LEAPLEAP - Leading Effective Academic Practice Educator Effectiveness Pilot in DPS

Slide33LEAP Overview• The district and the DCTA have worked in collaboration with DPS teachers and school leaders to develop a new teacher performance assessment system. • Through their work on Design Teams, teachers and principals applied the guiding principles from the focus groups to develop recommendations for a meaningful system of observation, feedback, support and evaluation for teachers.

Slide34LEAP – Multiple Measures

Slide35The LEAP Framework• In an  ongoing effort  to ensure that the Framework for Effective Teaching is an educator developed and tested tool, a primary goal of this year's pilot was to give DPS educators the opportunity to use the Framework and then provide feedback to guide further refinements prior to the 2012-13 LEAP pilot. • The revised 2012-13 Framework encompasses the common  feedback themes  identified throughout the 2011-12 LEAP pilot. The full Framework for Effective Teaching Evidence Guide is available to download on the DPS LEAP website.

Slide36Peer ObservationThird-Party Feedback With First-Hand Knowledge • Peer Observation is the component of LEAP that provides teachers with the opportunity to engage in reflective conversations and receive honest, open feedback with a peer or colleague who has similar teaching experience. • The Peer Observer role is a new position to DPS but one that has been used effectively in school districts across the country for a number of years. Peer Observers are  fellow teachers  who have been hired specifically for this role because they are recognized for their experience and expertise in content, classroom instruction, student achievement, and best practices. 

Slide37Peer Observers• Peer Observers will be matched as closely as possible to the content or grade level of the teacher they are observing so they can provide feedback and support that is  specific and relevant . Peer Observers will provide a third-party, outside perspective combined with first hand experience with the realities of teaching.  • Music Peer Observer classroom video clip: http://youtu.be/H-67UxQsRDQ http://youtu.be/LnYuIsUYhCA

Slide38Music Appendix sampleMASTERFUL CONTENT DELIVERY • I.1 - Refer to indicator • I.2 - Students provide performance rationale (self and others)    - Students answer questions aligned to music • I.3 - Teacher uses music instructional methods to support the Standards (i.e. Orff, Kodaly, Suzuki, etc.)

Slide39Category 1:State- mandated common assessment Category 2: District- approved common assessments Category 3: English language acquisition Category 4: Teacher/team developed assessment Category 5: School-wide measure Purpose: - Accountability to state - Capture longitudinal growth Example: TCAP Purpose: - Capture incremental growth - Inform instruction Example: Interim assessments Purpose : - Account for collective responsibility - Capture multiple areas of growth Example: SPF Purpose: - Allow for flexibility in the demonstration of student achievement Example: Core curriculum Purpose: - Account for high number of ELLs Example: CELA Student Outcomes Components