The estimation of the reliability of composite measures by optimized splits

The primary purpose of the study was to compare maximized split-half reliability coefficients, the traditional Kuder-Richardson 20 or Alpha coefficient, and Guttman's coefficient. A related objective was to evaluate the effectiveness of a computer program which maximizes split-half coefficients. A number of hypothetical 10-item and 40-item tests were constructed from the total of 100 items which make up Forms S and T of the Test of Chemical Comprehension. Responses of 380 individuals who had taken both forms of the test provided the data for the study. Sampling errors were introduced by repeated divisions of the 380 individuals into two complementary samples of 190 each. The effects of sampling error on the three types of reliability coefficients were investigated for each item set by comparing original sample and cross-validated coefficients with the coefficient obtained in the total group. The major findings were: (a) The computer program produced split-half coefficients which were only slightly smaller than the largest of all possible coefficients, but did not produce halves which had equal means and variances. (b) Maximized split-half coefficients exceeded both and Alpha coefficients by average amounts ranging from .05 to .20. Larger differences between the coefficients were found in less homogeneous sets of items. (c) Sample L[lowered 2] and Alpha coefficients underestimated and sample maximized split-half coefficients overestimated the largest reliability coefficient in the total group. It was concluded that the population reliability may be estimated more accurately by applying cross-validation procedures to maximized split-half coefficients than by computing the other coefficients, but that additional research with an expanded sampling design is needed to adequately test this possibility.