Issues of construct validation and demand characteristics in the role play assessment of heterosexual social skill and anxiety

This study investigated the construct validity and the demand characteristics inherent in role play assessment of heterosexual social skill and anxiety. Hypotheses included: 1.) construct validation would be more appropriate than criterion validation in assessing the external validity of role plays, 2.) unobtrusive self reports would demonstrate greater validity than behavioral ratings of role plays, 3.) subjects treated as coinvestigators would experience fewer demand characteristics' and produce data with greater construct validity than subjects treated as traditional objects of investigation, 4.) subjects would be able to fake role play performances as easily as subjects are presumed to be able to fake self reports, and 5.) subject mortality would be reduced among coinvestigators compared to objects of investigation. Ninety-one subjects participated in two dating-interaction role plays. For the first role play, half of the subjects were informed of the nature and purpose of the experiment (coinvestigators), and half were kept naive (objects of investigation) . During the second role play, subjects were instructed to fake their performances: half were to fake good; half were to fake bad. Three unobtrusive methods of measurement were also employed—a self-report questionnaire, two significant other report questionnaires, and several waiting room behavioral measures. Correlational statistics, multivariate analysis of variance, and confirmatory factor analysis within and between groups were used to analyze these data. Results suggested that 1.) a multi- operational or construct validational approach was superior to the mono-operational or criterion validational approach, 2.) significant other and self reports were more externally valid than behavioral measures of the role play and waiting room situations, 3.) treatment as a coinvestigator produced different demands and different validities than did treatment as object of investigation, but these differences did not unilaterally favor one treatment over the other, 4.) subjects were able to fake bad (but not good) role play performances, and 5.) subject mortality was not significantly affected in the predicted direction.