Exit Dashboard to Main Site

Success Factor 2: Accurate & reliable measures of a teacher’s impact on student growth

Key Practice A

Key Practice A: Student growth measures included in annual teacher summative ratings

Success Criteria

  • There is a clearly communicated plan for how student growth measures affect a teacher’s annual summative ratings
  • Student growth measures are clearly defined and used for all teachers put forth for designation
  • Multiple demonstrations of student learning are used to measure student growth

Descriptive Indicators

  • The Teacher Incentive Allotment requires the inclusion of a student growth measure because it provides a richer, more comprehensive picture of student learning. A measure of absolute student performance indicates the performance of a student at one point in time while a student growth measure indicates the progress of a student over time
  • The measures used in a teacher’s evaluation should be clearly defined and communicated prior to the beginning of the school year

Best Practices

  • Comprise a teacher’s achievement score with multiple measures of student growth when other student growth measures are available
  • Student growth measures are recommended to be a minimum of 30% of the overall evaluation•Create teacher focus groups, by teacher category, to provide input and build buy-in
  • The available student growth measures will vary based on teaching assignment
  • Create a guidebook to provide transparency in rules/procedures, including rules and procedures around calculations, minimum number of scores required, adjustment to measures when data is not available

Lessons Learned

  • Remove subjective Student Growth measures, such as student portfolios, whenever possible•Refrain from changing measures once the school year has begun
  • Strengthen testing protocols for assessments that are included in evaluation
  • Transparency of calculations will be important to build trust; this includes all calculations (student growth, other components, and other calculation methodology)

Resources

Best In Class, Strategic Evaluation Toolkit

Toolkit includes an overview of how to leverage evaluation data, the components of a strategic evaluation system, how the system has been implemented in districts to date, and key takeaways from district leaders and researchers at multiple districts.

Comparison of Student Achievement and Growth Measures

Comparison of Student Achievement and Growth Measures across six evaluation systems, including Dallas, DC, Denver, Tennessee, Connecticut, and Hillsborough.

Growth Data: It Matters, and It’s Complicated

Data Quality Campaign’s brief provides an introduction to the different ways states will measure student growth in their ESSA accountability systems to evaluate school quality and support improvements.

Guidance on Student Growth in T-Tess

TEA developed guidebook offering guidance on student growth in T-Tess

Dallas ISD – TEI Rulebook

Rules and Procedures for Calculating Achievement Statistics, Evaluation Scores, and Effectiveness Levels for Dallas ISD’s Teacher Excellence Initiative.

Dallas ISD – Student Achievement Templates

Dallas ISD’s templates, reflecting the types and weights for student achievement measures included in Dallas Teacher Evaluation System, TEI.

Grand Prairie ISD

Grand Prairie ISD’s TIA website

SAISD

San Antonio ISD’s TIA website

Key Practice B

Key Practice B: Quality of student growth measures and their implementation

Success Criteria

  • District utilizes student growth measures such as:–Value-added measures based on STAAR or other normed, valid tests–Student learning objectives–Student portfolios–Pre- and post-tests
  • Assessments used to measure student growth are valid and reliable
  • Assessments used to measure student growth are implemented with fidelity, according to vetted testing protocols

Descriptive Indicators

  • The use of multiple assessments that can be used to measure student achievement growth, either during the year and/or year-over-year. Assessments could include state standardized assessments, Measures of Academic Progress (MAP), I-Station, ITBS, or any other standard assessment used district-wide. The assessments must go through a district process to ensure the validity and reliability of the testing instrument
  • There are different types of student growth measures that provide different types of information. Below you will find details on three of the most common student growth measures. The first two measures are objective measures based on common assessments while the last is a subjective measure, dependent on teacher and principal judgement:
    • (i) Student growth percentiles (SGPs): This is a measure that uses a student’s past performance to determine a student’s current performance compared to the student’s peers
    • (ii) Value-added model (VAM): This is a measure that determines the impact of an educator or school on student learning and controls for factors outside of a teacher’s control that influence student achievement
    • (iii) Student learning objectives (SLOs): This is a measure of student progress based on student growth goals set by teachers

Best Practices

  • Comprise a teacher’s achievement score with multiple measures of student growth whenever possible
  • Student growth measures are recommended to be a minimum of 30% of the overall evaluation
  • Create teacher focus groups, by teacher content area, to provide input and build buy-in
  • The available student growth measures will vary based on teaching assignment
  • Create a guidebook to provide transparency in rules/procedures, including rules and procedures around calculations, minimum number of scores required, adjustment to measures when data is not available

Lessons Learned

  • Remove subjective Student Growth measures, such as student portfolios
  • If including SLOs, consider using standard SLOs by grade/content area instead of individual set SLOs. In a district that has used individual created SLOs, over 80% of teachers will receive all or almost all of the student growth points
  • Refrain from changing measures once the school year has begun
  • Strengthen testing protocols for assessments that are included in evaluation to remove the appearance of or opportunity for manipulation of student outcomes
  • Transparency of calculations will be important to build trust
  • Districts who are interested in using VAM should ensure they appropriately train teachers to understand the measure and give educators time to familiarize themselves with the model

Resources

Best In Class, Strategic Evaluation Toolkit

Toolkit includes an overview of how to leverage evaluation data, the components of a strategic evaluation system, how the system has been implemented in districts to date, and key takeaways from district leaders and researchers at multiple districts.

Comparison of Student Achievement and Growth Measures

Comparison of Student Achievement and Growth Measures across six evaluation systems, including Dallas, DC, Denver, Tennessee, Connecticut, and Hillsborough.

Growth Data: It Matters, and It’s Complicated

Data Quality Campaign’s brief provides an introduction to the different ways states will measure student growth in their ESSA accountability systems to evaluate school quality and support improvements.

Student Growth Percentiles

Created by the Rand Corporation, Student Growth Percentiles 101 Using Relative Ranks in Student Test Scores to Help Measure Teaching Effectiveness

Targeting Growth: Using SLOs as a Measure of Educator Effectiveness

Created by Reform Support Network, an overview of Student Learning Objectives as a measure of teacher effectiveness.

Dallas ISD – TEI Rulebook

Rules and Procedures for Calculating Achievement Statistics, Evaluation Scores, and Effectiveness Levels for Dallas ISD’s Teacher Excellence Initiative.

Dallas ISD – Student Achievement Templates

Dallas ISD’s templates, reflecting the types and weights for student achievement measures included in Dallas Teacher Evaluation System, TEI.

Grand Prairie ISD

Grand Prairie ISD’s TIA website

SAISD

San Antonio ISD’s TIA website

Key Practice C

Key Practice C: District rules and processes ensure high levels of validity and reliability of student growth measures

Success Criteria

  • Campus and district leaders disaggregate and analyze student growth trends in multiple ways, including by student groups, teaching assignments, and schools, as well as historical student growth data, to ensure results are accurate and reliable
  • District has appropriate internal procedures to evaluate outlier trends in the data in order to detect and prevent irregularities

Descriptive Indicators

  • It is critical for a district to ensure high levels of validity and reliability of student growth for several reasons
    • Instill confidence of district teachers and leaders in the measures, that they are accurately capturing the contribution of the teachers
    • Meet data validation step required as a part of the Teacher Incentive Allotment. This step is completed by Texas Tech University; Texas Tech reviews a district data with four criteria which involve looking for expectedrelations AND similarity between district data results and statewide VAM results
      • (i) Agreement between observation and growth: Correlation between teacher observation ratings and teacher’s student growth rubric ratings
      • (ii) Agreement between district growth ratings and statewide VAM results: Correlation between teacher student growth rubric ratings and Value-Add Measure
      • (iii) Consistency of observations across campuses and assignments: Degree of inter-rater reliability for observation and growth ratings
      • (iv) Designation frequency is comparable to other districts with the same Domain 2A rating: Comparison of district designation percentage to overall statewide performance

Best Practices

  • School leadership & campus leadership & data teams review student growth data as available. At a minimum, we recommend reviewing trends after mid-year assessments. A full review should be completed annually
  • School leadership & campus leadership teams review student growth data and observation data to identify calibration/congruence issues. At a minimum, an annual review of the congruence data should be completed. It is recommended that leading indicator data, when available, such as classroom observation data and growth data from mid-year assessments be studied to identify potential areas of concern
  • Perform ongoing analysis of evaluation data can also help mitigate for potential bias in observations or in student growth measures such as portfolios and SLOs. At a minimum, we recommend a review to identify trends in teacher category and teacher demographics to ensure that there are not underlying systemic issues in the measures themselves or in the application of the measures

Lessons Learned

  • Transparency of calculations will be important to build trust•Provide resources (locally created or from the creator of the assessment) to understand calculation methodology and data reports

Resources

Dallas ISD – TEI Rulebook

Rules and Procedures for Calculating Achievement Statistics, Evaluation Scores, and Effectiveness Levels for Dallas ISD’s Teacher Excellence Initiative.