Library

Assessing Measures of Mathematical Knowledge for Teaching: A Validity Argument Approach

Abstract

In assessing the utility of a test, two issues stand out: whether it provides information of interest to test consumers, and whether scores generated by the test assist in making good decisions. Validity addresses these two issues, making an assessment of test validity the single most important product provided by test developers. Unfortunately, despite its importance, test validation is almost universally viewed as the most unsatisfactory aspect of test development. As Messick (1988) noted, there has been a consistent disjunction between validity conceptualization and validation practice. To start, the proliferation of many different kinds of validity evidence without clear prioritization presents test consumers with an enormous task, that of sifting through various methods, approaches, and empirical work to determine the usability of a test. At the same time, some test developers use evidence (and methods) selectively, choosing convenient means for test validation, and convenient results for reporting. Kane (2001, 2004a) developed an argument-based approach to validity as a means of addressing these difficulties. His approach consists of two stages, the Formative Stage and the Summative Stage. In this set of papers, we use Kane's approach to validate a measure of teachers' mathematical knowledge. We do so not only to learn about the measure itself, but also to assess the promise of and problems with the argument-based approach to validity. In doing so, we attend closely to issues raised in the lead author's and others' responses to Kane's original work.