I. PRE-EXPERIMENTAL OR FAULTY DESIGNS

One group of subjects gets one treatment. There may be a pre- and post-test or just a post-test. May eliminate chance, otherwise eliminates no alternative hypotheses.

Example: Research participants who receive the new form of therapy are tested afterward (post-test only), or participants are measured before and after the therapy.

This may be useful in showing that there is some reason to believe the new therapy works, but from this design, we cannot draw any conclusions about why there is improvement. It should be considered a pilot test at best and followed up with a better research design.

II. QUASI-EXPERIMENTAL DESIGNS

These designs eliminate some, but not all, alternative hypotheses. They are especially useful in applied settings where real-life constraints make it undesirable or impossible to control the research setting.

Example: These designs may be most applicable if the new therapy is being used in a mental health center or a private practice. Rather than compromise the needs of clients to eliminate alternative hypotheses, we would be willing to allow some alternative hypotheses. This is a choice of relevance and external validity over control and internal validity. Ideally, such a design would be paired with others to allow us to draw stronger conclusions.

A.

Non-equivalent groups or static groups design

Two groups receive different treatments, but are not randomly assigned or maltched to conditions. Eliminates history effects but not subject effects.

Example: Participants may be given the choice of which therapy to receive. The potential participants most likely to benefit from the new therapy are assigned to that condition. Intact (already-existing or static) group may also be used, for example, all the clients in an existing therapy group may be given the new therapy. If the group receiving the new therapy improves more than the control group, we can be somewhat more confident in the benefits of the new therapy.

B.

Time-series design

There is one group of research participants with several baseline measures, a treatment, and at least one more measurement. Eliminates subject effects but not history effects.

Example: One group of research participants is selected for the study. Their mental health is measured each month for several months. Then they are given the new therapy and measured again. If they improve after the therapy, but not before, we are more confident the new therapy helps.

This design is used most often to evaluate public policy changes which affect a large group of people. The dependent variable may be obtained from public records (say number of reported incidents of violence) before and after a change in public policy (say a community-wide program to reduce violence).

C.

Multiple time-series design

Two or more groups (not randomly assigned) receive several pre-treatment measures and at least one post-treatment measure. Can eliminate history and (most) subject effects. Thus it is considered a "strong quasi-experimental design."

Example: After taking a series of measurements of mental health at two different counseling centers, all clients at one center are given the new therapy. If those clients improve more than the clients at the "control" counselign center, we can be more confident of the new therapy.

In the example just above evaluating public policy changes, we could add data from a similar community which did not receive the violence reduction program. If we saw a decrease only in the community receiving the program and only after the program, we have more reason to believe the program was the cause of the reduction in violence.

III. TRUE EXPERIMENTAL DESIGNS

These designs attempt to eliminate most alternative hypotheses, especially those related to time (history, maturation, and regression) and those related to make-up of the groups (selection effects). Such control may be at the expense of making the situation too artificial.

A.

Randomized groups design, between-groups design

Each research participant is randomly assigned to one group and gets only one level of the indepndent variable. There may be pre-tests and post-tests or only post-tests. This design can eliminate selection, history, and maturation efects.

B.

Repeated measure design, within-subject design

Each research participant gets all levels of the IV. Treatment orders must be counterbalanced to eliminate order effects.

C.

Mixed model designs or complex designs

These designs combine randomized groups and repeated measdures designs. For instance, there may be two IVs, one measured between groups and one measured within groups.