Randomized versus nonrandomized studies of alcohol treatment: Participants, methodological features and posttreatment functioning

Objective: Randomized trials are widely acknowledged to be the most rigorous method to estimate the comparative efficacy of treatments. Concerns have been raised, however, about the generalizability of findings from randomized trials of alcohol treatments to actual treatment circumstances, and about the internal validity of randomized trials. The purpose of this review was to compare the participants, methodological features and posttreatment functioning in randomized and nonrandomized studies of alcohol treatment. Method: Using systematic literature search techniques, 232 randomized and 92 nonrandomized trials published between 1970 and 1998 were identified. Each investigation was scored on a methodological quality scale, participant characteristics were recorded and participants' follow-up abstinence and improvement rates were calculated. Results: Randomized investigations were more likely to use participant selection criteria, to use recognized diagnostic criteria to characterize participants and to stringently implement treatment and assess outcomes. Nonrandomized investigations were more likely to assess outcomes in higher proportions of participants over longer follow-up periods and to have greater statistical power to detect treatment effects. Abstinence and improvement rates following active treatment were similar for the two types of design (proportion abstinent: 0.35 and 0.39, and proportion improved: 0.52 and 0.54, for randomized and nonrandomized trials, respectively), even when differences in study features were controlled. Conclusions: The contrasting strengths and weaknesses of randomized and nonrandomized studies suggest that they should be considered as complementary forms of treatment evaluation, in the alcohol treatment field and perhaps more generally.