A comparison of new versus traditional methods of analyzing weight loss data : results of a Monte Carlo data simulation study and actual data analyses

The present dissertation study compared the use of the "best" traditional method of analyzing weight loss data from a pretest-posttest control group design (ANCOVA on posttreatment weight covarying pretreatment weight) with two, previously uninvestigated alternatives (ANCOVA on percent weight loss covarying pretreatment weight and ANCOVA on the log of posttreatment weight covarying the log pretreatment weight). As part of the evaluation, a Monte Carlo study was conducted to determine the alpha level performance and relative loss in power when data were analyzed by methods inconsistent with the underlying data generator model. All three methods were robust under the conditions varied and reported. Power differences between the methods amounting to any practical significance did not emerge until a large effect size was employed. Regardless of the model used to generate the data, all three methods demonstrated acceptable power to detect the treatment effect and affirm homogeneity of regression. Despite the general implication from the simulated data that it does not greatly matter which method is used to analyze the data, analyses of the actual data suggested the opposite. The logarithmic data transformation involved in the Method 3 analysis was clearly the best approach. Evidence of the appropriateness of Method 3 was the observation that none of the crucial statistical assumptions was violated using this method. In contrast, use of the traditional ANCOVA violated the homogeneity of regression and homoscedasticity assumptions. Method 2, which involved a transformation of posttreatment weight to percent weight loss, was the second best data analytic approach for the actual data. Only the homoscedasticity assumption was violated using this approach. Discrepant results emanating from the actual data analyses relative to the simulated analyses can be reconciled in view of the fact that the former data were much more heterogeneous and had a much larger sample size (N = 352 as opposed to N = 20 or 60). The heterogeneity and large sample size of the actual data accentuated power differences between the methods. Regarding heterogeneity, the ratio of the heaviest person's pretreatment weight to the lightest person's pretreatment weight for the actual data was greater than 2/1. Emerson and Stoto (1982) recommend considering a data transformation when the maximum value/minimum value ratio is greater than two as was the case here. Future studies evaluating methods for analyzing weight loss data should attempt to replicate the present finding that actual weight loss data are best analyzed by the multiplicative methods, especially Method 3. Preferably, actual data from a randomized pretest-posttest control group design can be used to eliminate interpretation problems that arise with nonrandomization (Lord, 1967). Of note is that the actual data from the present study were from nonrandomized groups. Finally, additional Monte Carlo studies are indicated using larger sample sizes and more heterogeneous groups.