Educators slam flaws in Value Added Model

Editor's note: The letter below was sent from the administration and faculty of  the W.W. Lewis Middle School in Calcasieu Parish to members of the Board of Elementary and Secondary Education and officials in the Department of Education:

This is being sent with endorsement from Wayne Savoy, Superintendent of Education for Calcasieu Parish Schools. 

Attention BESE Board and LADOE Officials:

     We wish to address three major concerns that could have a negative effect on VAM scores for many 8^th grade teachers as well as student scores across the state.  The first issue effects 8^th grade teachers and the other two affect all teachers. 

     The first concern is with the construction of 8^th grade LEAP tests and how the current test affects teachers’ VAM scores.       

In middle school, Sciences and Social Studies are taught in the following grades:

                        Science                      Social Studies

        6^th grade    Physical Science          World History

7^th grade    Life Science                        American History

8^th grade    Earth Science             Louisiana History

6^th and 7^th grade students take an iLEAP test that covers only the material they were taught during that school year.  However, in 8^th grade the students take a test that is comprehensive for all three middle school years.  The Science LEAP test covers not only the Earth Science that 8^th grade students were taught, but Physical and Life Sciences from 6^th and 7^th grades as well.  We understand the need for a comprehensive exam, but to have one teacher’s VAM score tied to what other teachers have or have not taught makes the score invalid. 

     The Social Studies LEAP covers not only the Louisiana History that 8^th grade students were taught, but also World and American History from 6^th and 7^th grade.  On the LEAP test, students are then required to remember or reproduce information that they were taught 1-2 years earlier.  8^th grade teachers have so much material to cover from their own curriculum that they cannot possibly address 6^th and 7^th grade GLEs as well, yet the students are tested on that material. 

8^th grade Social Studies LEAP test:

Strands/ Categories		Items		% of Total points
GEOGRAPHY
A. The World in Spatial Terms
B. Places and Regions		15		25
C. Physical and Human Systems
D. Environment and Society
CIVICS
A. Structure and Purposes of Government
B. Foundations of the American Political System		12		20
C. International Relationships
D. Roles of the Citizen
ECONOMICS
A. Fundamental Economic Concepts
B. Individuals, Households, Businesses		9		15
and Governments
C. The Economy as a Whole
HISTORY
A. Historical Thinking Skills
B. United States History		24		40
C. World History
D. Louisiana History
TOTAL		60		100

A large percentage of this test comes from material the students were to have learned in grades 6-7, so the 8^th grade teacher gets a VAM score based on what two other teachers have done or not done in grades 6-7.  The VAM score is therefore invalid.  

     For Social Studies, even though much of the material for all three grades comes from the same strand and/or benchmark, the teaching would be specific for the area of history being studied.  For example; all three grade levels teach Foundations of the American Political System from the Civics strand.  However, it is from three different perspectives- World, American, and Louisiana.  While this is a key concept and must be explored by students from all three perspectives and in all three grade levels, an 8^th grade teacher whose focus is on Louisiana should not be held accountable for the same concept from a World view point that was taught two years earlier.

8^th grade Science LEAP test:

Strand	Multiple	Short	ECR	Score
	Choice	Answer	Items	Points
	(1 point)	(2 points)	(4 points)
1. Science as Inquiry	8	0		8
2. Physical Science	8	1		10
3. Life Science	8	1		10
4. Earth and Space Science	8	1		10
5. Science and the Environment	8	1		10
Comprehensive Science Task
1. Science as Inquiry		1		2
Dimension 1 (Questioning,
Planning, Doing and
Recording)
1. Science as Inquiry		2		4
Dimension 2 (Interpreting and Communicating)
2. Physical Science			1
3. Life Science			(in each of	4
4. Earth and Space Science			the four
5. Science and the Environment			strands)
Total Score Points	40	14	4	58

The LEAP test is approximately 24% Physical science (taught in 6^th grade), 17% Earth Science (taught in 8^th grade), 17% Life Science (taught in 7^th grade), 17% Environmental Science (taught in all grades) and 24% Inquiry.  This shows that at least 41% of the 8^th grade test is based on 6^th and 7^th grade material.  The 6^th and 7^th grade tests do not include Science task questions in which students are required to set up graphs and draw conclusions based on a given scenario.  This could be an issue if the information needed to complete the task came from another grade level’s material.  Neither of these graphs reflects the fact that a constructive response item could come from another grade level’s material.  

     It is difficult enough to expect an 8^th grade student to write on a topic that he or she was taught 2 years earlier, and invalid to have the score reflect on an 8^th grade teacher.  The student has already taken an iLEAP test on that material.  Shouldn’t that test be enough to determine Proficiency without it reflecting on an 8^th grade teacher?   

     The 8^th grade math and ELA LEAP tests are set up in a similar manner (as comprehensive middle school exams) but the results are not as potentially harmful to the teacher.  Both of those subjects have concepts that are built upon, unlike Science and Social Studies who have so many isolated GLEs.  The problem then is if 6-7^th grade teachers have not been successful in teaching the concepts, the 8^th grade teacher’s VAM score suffers and is therefore invalid.    

     In schools with a high student turnover rate, an 8^th grade teacher’s score could be affected by students who studied 6^th or 7^th grade Science or SS at another school, multiple schools, or even in another state.  

     Shouldn’t a teacher’s VAM score be based solely on what that teacher has done with a student?  How can the state rate a teacher “Ineffective” based on another teacher’s effectiveness in teaching 1-2 years earlier?

     One possible solution would be to break a student’s 8^th grade score into two categories- one for a comprehensive Science and Social Studies score and one score which reflects only what a student scored on material from his or her 8^th grade subject.  By doing this, the teacher’s VAM score could be a valid representation of only what that 8^th grade teacher was responsible for teaching.   

     Our second concern is in the form of a question.  Our current understanding is that a student’s test score history for several years is used to predict an iLEAP or LEAP score for that student.  Following this premise, if a student scored Unsat for 2-3 years in a subject, then his predicted score would be Unsat as well.  Our question is- If the student does score Unsat, as predicted, would that score still count against the teacher because he was not Proficient even though Unsat is the predicted score?  If so, how is that valid for a teacher’s VAM score?   

     Part A of our third concern is about the attendance factor for SLTs.  We were told to use 80% attendance as the factor by which we would determine which students would and would not be included in our SLT.  No one questioned that, and we just added it to our SLTs as directed.  Recently we realized what 80% attendance really means.

• 80% attendance is a student missing school one day a week, every week
• Could a teacher be effective if a student missed one day a week?
• Think about the repercussions on student learning if teachers missed one day a week every week
• 80% means a student could miss 36 days of school a year and still count towards their SLT
• 10 absences a year is the maximum- by the state's own standard
• 80% attendance means we have basically not excluded anyone from our SLT count
• 80% means we are still accountable for those who do not value education enough to attend school

Even if we increase the attendance factor to 90%, a student could potentially miss 18 days a year and still count against a teacher’s SLT and VAM score.  We do not believe students who miss 18-36 days a year can grasp the numerous concepts taught in six or more subjects.

Looking at it from this perspective, if teachers verify their CVR roster from 1 Oct until testing, that is about 175 actual school days minus about 30 days from the start of school until 1 Oct and about 20 days after testing.  That leaves roughly a 125 day window.  Using 125 days, 80% attendance still means a student could be absent 25 days and count against a teacher on their SLT.  Increasing the attendance rate to 90% and using the CVR window means a student could be absent 12.5 days.  This seems a little more realistic for SLTs and VAM scores, although still above the state’s maximum 10 day absentee policy.  

 

     Part B of the third concern is that we have been told that an alternate formula will be used to calculate predicted test scores for students with attendance problems.  This leads us to believe that the same 80% attendance variable will be used by the state for VAM teachers to determine which students’ scores are calculated with which formula.  The numbers in the above paragraphs prove that 80% attendance is not a valid number.  It removes few if any students from the equation used to determine teacher effectiveness in VAM scores or SLTs. 

     We respectfully request that the BESE Board and State Department of Education work to resolve these issues as soon as possible.  No one would want teachers to be rated ineffective because of invalid data.  With tenure and possible teacher compensation relying on teacher effectiveness and test scores, the consequences could be devastating to careers and school systems.