Modifying achievement test items

A theory-guided and data-based approach for better measurement of what students with disabilities know

Ryan Kettler, Stephen N. Elliott, Peter A. Beddow

Research output: Contribution to journalArticle

31 Citations (Scopus)

Abstract

Federal regulations allow up to 2% of the student population of a state to achieve proficiency for adequate yearly progress by taking an alternate assessment based on modified academic achievement standards (AA-MAS). Such tests are likely to be easier, but as long as a test is considered a valid measure of grade level content, it is allowable as an AA-MAS (U.S. Department of Education, 2007b). In this article, we examine procedures for developing, modifying, and evaluating items and tests using an evolving modification paradigm, as well as a classic reliability and validity framework. Theoretical influences, such as principles of universal design, cognitive load theory, and item development research, are discussed. The Test Accessibility and Modification Inventory, a tool that provides systematic and comprehensive guidance to help educators modify grade-level tests, is introduced. Cognitive lab methods and experimental field tests are then described, along with examples and key findings from each, relevant to AA-MASs. The article concludes with a discussion of precautions, lessons learned, and questions generated about the methods used to improve both access and test score validity for the students who are eligible for this new alternate assessment.

Original languageEnglish (US)
Pages (from-to)529-551
Number of pages23
JournalPeabody Journal of Education
Volume84
Issue number4
DOIs
StatePublished - Oct 1 2009

Fingerprint

achievement test
disability
Students
student
Reproducibility of Results
academic achievement
Education
Equipment and Supplies
Research
Population
school grade
educator
paradigm
regulation
education

All Science Journal Classification (ASJC) codes

  • Education
  • Developmental and Educational Psychology

Cite this

@article{b37b32945f124816a92c30fb4f7fd3bd,
title = "Modifying achievement test items: A theory-guided and data-based approach for better measurement of what students with disabilities know",
abstract = "Federal regulations allow up to 2{\%} of the student population of a state to achieve proficiency for adequate yearly progress by taking an alternate assessment based on modified academic achievement standards (AA-MAS). Such tests are likely to be easier, but as long as a test is considered a valid measure of grade level content, it is allowable as an AA-MAS (U.S. Department of Education, 2007b). In this article, we examine procedures for developing, modifying, and evaluating items and tests using an evolving modification paradigm, as well as a classic reliability and validity framework. Theoretical influences, such as principles of universal design, cognitive load theory, and item development research, are discussed. The Test Accessibility and Modification Inventory, a tool that provides systematic and comprehensive guidance to help educators modify grade-level tests, is introduced. Cognitive lab methods and experimental field tests are then described, along with examples and key findings from each, relevant to AA-MASs. The article concludes with a discussion of precautions, lessons learned, and questions generated about the methods used to improve both access and test score validity for the students who are eligible for this new alternate assessment.",
author = "Ryan Kettler and Elliott, {Stephen N.} and Beddow, {Peter A.}",
year = "2009",
month = "10",
day = "1",
doi = "https://doi.org/10.1080/01619560903240996",
language = "English (US)",
volume = "84",
pages = "529--551",
journal = "Peabody Journal of Education",
issn = "0161-956X",
publisher = "Taylor and Francis Ltd.",
number = "4",

}

Modifying achievement test items : A theory-guided and data-based approach for better measurement of what students with disabilities know. / Kettler, Ryan; Elliott, Stephen N.; Beddow, Peter A.

In: Peabody Journal of Education, Vol. 84, No. 4, 01.10.2009, p. 529-551.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Modifying achievement test items

T2 - A theory-guided and data-based approach for better measurement of what students with disabilities know

AU - Kettler, Ryan

AU - Elliott, Stephen N.

AU - Beddow, Peter A.

PY - 2009/10/1

Y1 - 2009/10/1

N2 - Federal regulations allow up to 2% of the student population of a state to achieve proficiency for adequate yearly progress by taking an alternate assessment based on modified academic achievement standards (AA-MAS). Such tests are likely to be easier, but as long as a test is considered a valid measure of grade level content, it is allowable as an AA-MAS (U.S. Department of Education, 2007b). In this article, we examine procedures for developing, modifying, and evaluating items and tests using an evolving modification paradigm, as well as a classic reliability and validity framework. Theoretical influences, such as principles of universal design, cognitive load theory, and item development research, are discussed. The Test Accessibility and Modification Inventory, a tool that provides systematic and comprehensive guidance to help educators modify grade-level tests, is introduced. Cognitive lab methods and experimental field tests are then described, along with examples and key findings from each, relevant to AA-MASs. The article concludes with a discussion of precautions, lessons learned, and questions generated about the methods used to improve both access and test score validity for the students who are eligible for this new alternate assessment.

AB - Federal regulations allow up to 2% of the student population of a state to achieve proficiency for adequate yearly progress by taking an alternate assessment based on modified academic achievement standards (AA-MAS). Such tests are likely to be easier, but as long as a test is considered a valid measure of grade level content, it is allowable as an AA-MAS (U.S. Department of Education, 2007b). In this article, we examine procedures for developing, modifying, and evaluating items and tests using an evolving modification paradigm, as well as a classic reliability and validity framework. Theoretical influences, such as principles of universal design, cognitive load theory, and item development research, are discussed. The Test Accessibility and Modification Inventory, a tool that provides systematic and comprehensive guidance to help educators modify grade-level tests, is introduced. Cognitive lab methods and experimental field tests are then described, along with examples and key findings from each, relevant to AA-MASs. The article concludes with a discussion of precautions, lessons learned, and questions generated about the methods used to improve both access and test score validity for the students who are eligible for this new alternate assessment.

UR - http://www.scopus.com/inward/record.url?scp=75249106343&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=75249106343&partnerID=8YFLogxK

U2 - https://doi.org/10.1080/01619560903240996

DO - https://doi.org/10.1080/01619560903240996

M3 - Article

VL - 84

SP - 529

EP - 551

JO - Peabody Journal of Education

JF - Peabody Journal of Education

SN - 0161-956X

IS - 4

ER -