In any experiment there is the possibility that extraneous variables, rather than the independent variable(s) are producing the change on the dependent variable (a confound or threat to internal validity). If the extraneous variable cannot be eliminated it must be constant across conditions (group equivalence by randomly assigning participants to groups). Each experiment has its own confounds, but most are examples of the Big 10:
1. History – Any of the many events that occur in the outside world other than the IV that occur before the measurement of the DV. For example, if you forgot to give one group a certain set of instructions or if a new technician or student took over the experiment in the middle.
2. Maturation – Any of the conditions internal to the individual that change as a function of the passage of time. For example, in a longitudinal study, people change in different ways. In an experiment, the participant could become bored, fatigued, or figure out the purpose of the experiment.
3. Instrumentation – Any changes that occur as a function of measuring the DV. Instruments are not perfect in that sometimes they change in some way through the course of an experiment. Calibration of the measurement device may change during the course of the experiment. For example, the sound used to generate a startle response may get louder or quieter during the experiment.
4. Testing or Practice Effect – Being exposed to the testing procedures, participants can learn and device different strategies for taking the test. Being exposed to a test once is sufficient to change on a second testing the way you take the test.
5. Statistical regression (regression toward the mean) – Any change that can be attributed to the tendency of extremely high or low scores to regress to the mean. Here is an article on why Tiger Woods has a difficult time being better than everyone all the time.
6. Selection – Any change due to the differential assignment procedures used in placing subjects in various groups. Whenever the participants are selected nonrandomly for participation in a group, there is always a chance that some unintentional bias may have occurred. For example, doing an experiment using existing groups may be affected by selection bias. This selection bias can interact with the treatment to amplify or decrease the treatment effect.
7. Mortality [attrition] – Any change due to the differential participant loss from the various groups. People may leave the experiment. If they leave or choose not to participate, it could be because of the experimental condition. This problem is often seen in treatment outcome studies. For example, a person may not like the side effects or think the procedures are too difficult in his or her condition. The reason why a particular participant dropped out is a form of self-selection. Thus, you may have experimental results which reflect only those participants who persevered or did not dropout for some reason.
8. Sequencing – Any change in the participant’s performance that can be attributed to the fact that the participant participated in more than one treatment condition. That is, the order of procedures can have an effect of outcome.
9. Participant Effects-
a. Demand Characteristics – Any bias produced by participants trying to be good participants and behave in a manner that helps the experimenter (helps achieve data in the expected direction). Ideally, in most experiments the intent is to have the participant blind to the purpose of the experiment and/or which condition they are in.
b. Hawthorne effect – Based on studies at the Hawthorne Works of the Western Electric Company. This effect is generally defined as the problem when participants’ knowledge that they are in an experiment modifies their behavior from what it would have been without the knowledge.
10. Experimenter Effects – Any effect related to the person conducting the study knowing about the study’s hypotheses. That is, the experimenter being aware of the research hypotheses may treat the participants in the different conditions differently. The cueing does not have to be conscious. There is the Clever Hans phenomenon, which is a form of unconscious cuing. The term refers to a horse who responded to questions requiring mathematical calculations by tapping his hoof. If asked what is the sum of 2 plus 2, the horse would tap his hoof four times. It was eventually discovered that the horse was responding to subtle physical cues from its owner. After the horse heard a question and started tapping, the owner would unconsciously give an almost imperceptible head movement, which was the horse’s cue to stop.