An investigation of a hierarchy of question levels based upon antecedent conditions as estimated by teachers in the classroom

Date

1971

Journal Title

Journal ISSN

Volume Title

Publisher

Abstract

Purpose of the Study. The purpose of this study was to derive estimates, through correlation coefficients, of the characteristics of objectivity, reliability, validity, and generalizability of a hierarchy of question levels based upon the teacher's estimate of the degree of student experience over test items. Method. The characteristic, objectivity, defined in the study as the degree to which a group of eight teachers, two in science, three in American history, and three in mathematics, could agree to the classification of a set of hypothetical test items, was estimated through the use of a coefficient of concordance. The characteristic, reliability, defined in the study as the degree to which the test group of teachers could reclassify a set of their own test items that they had constructed into the original hierarchy levels to which they had been designed, was estimated through the use of a Kendall's tau correlation coefficient for each teacher. The characteristic, validity, defined in the study as the degree to which the passing or failing of a student on a test constructed by one of the teachers in the test group could be predicted through the use of the hierarchy and an associated set of regression curves, was estimated through the use of a phi coefficient. The regression curves were constructed to relate the IQ's of the students to an estimate of the proportion of items at each hierarchy level answered correctly by other students with corresponding IQ scores. The characteristic, generalizability, defined in the study as the degree to which all students who participated agreed that level I items were easiest, level II were of intermediate difficulty, and level III were most difficult, was estimated through the use of a coefficient of concordance. In effect, each student was a judge as to the relative difficulty of the three hierarchy levels as he responded with more correct answers for one type of item than for the other type. [...]

Description

Keywords

Examinations--Design and construction

Citation