The most highly regarded and rigorous experimental design in sociological research is the controlled experiment, where artificial conditions are established, the highest standards of internal validity prove that there is no other explanation for the cause and effect relationship, and the highest level of external validity means that the findings from the experiment can be generalized to the whole population, with no other outcomes or conditions where the cause and effect relationship do not apply.
This is generally impossible in the real world. Perfect internal and external validity, as well as perfect control of everything that goes on with people, programs, or social systems, would be rare, if they have ever been established. Even in the hard sciences, where cancer research is conducted under rigorous conditions, it is difficult to establish complete internal and external validity when the entire human organism is operating under both other explanations for the patient’s decline or remission (internal validity), and there are findings that cannot be generalized to all humans with that particular type of cancer (external validity).
As a result in the social sciences and other sciences, quasi-experimental design is the predominant method.The most important fact about all quasi-experimental designs is that they do not study populations or subjects that are randomly selected or randomly assigned. Random selection is the final standard for true experimental design of the highest standard.
Of the quasi-experimental designs, NONEQUIVALENT GROUPS DESIGN is the most common. In this design, two groups are identified and selected to insure that they are as similar as possible. Then each group is tested to see where the members of the group stand before the program or the variable that is to be changed is introduced. Then the program (a change in a set of causative variables) or change in a single causative variable is introduced to only one group (the test group). The other group (the control group) continues on in life with no introduction of a program or a causative variable. Finally, both groups are tested to see if the expected or desired change in the dependent variable occurred.
Nonequivalent groups design is fraught with potential problems and error, including bias in selecting the test group and the control group. There can be error in the testing. There can be “threats” to both the internal and external validity, as large groups of test subjects have other things going on in their lives that can distort the results, and they can have other reasons than the change in the causative variable for any changes that they undergo.
Let’s say that 200 students are tested for reading and writing. 100 students will be put under an after school program to improve reading and writing. They are the test group. The causative variable is the after school program.The cause and effect relationship is positive: increased tutoring will increase reading and writing scores.
100 students will not have any program. The expectation is that there will be no change in the students reading and writing test scores.
After the program is complete, the students are tested again to see how their reading and writing skills are.
Of the three hundred students, total, some will die or become ill. Many in the control group will improve without any exposure to the program, because they are already in school. Some will stay the same. Others will have great improvement or great reduction in their scores. It is at this point where the researcher will look at the data, factoring out intervening variables, using the mean or median scores, and using other analytical tools to identify and to eliminate as much internal and external threat as possible. Any bias in selecting the control and the test group should have been dealt with before the groups were selected.
The REGRESSION DISCONTINUITY DESIGN is a quasi-experimental design that begins with a threshold that is set for a co variate, such as prison recidivism. The test group is selected from communities where returns to prison are above the cutoff point. The control group is a community that has returns to prison that are below the cutoff. A program is administered to the test community see if it will reduce the returns to prison. Then both communities are measured again. Then the analysis determines if the program or some other intervening variables, such as death or relocation to other communities caused any positive change.
There are many other quasi experimental designs such as the Proxy Pretest, Non-Equivalent dependent variable, and Regression Point Displacement Design.
CITATIONS:
Regression Discontinuity
Non Equivalent Pretest/Post test Groups
Internal and External Validity
