Rules that cannot be ignored about experimental evaluation design

Experimental designs are often considered the ‘gold standard’ of research designs because of the rigorous and robustness that provides. Before we move to understand basic rules of experimental designs, let’s have a look at how it really works and what it really means.

Experimental designs consists of treatment and control groups . The researchers that manipulate one or two independent variables (as treatments) which are subjected to random assignment to different treatment groups and accordingly, results of the treatment on the outcomes (dependent variables) are observed. Experimental research designs are best suited for explanatory research where the primary goal of the research design is to assess the cause-effect relationships as opposed to descriptive and exploratory research. What experimental designs bring to the table are high internal validity due to its ability to link the cause and the effect.

Three rules that one needs to keep in mind regarding the experimental designs are:

Randomization: The first principle of an experimental design is randomization, which is a random process of assigning treatments to the experimental units. The random process implies that every possible allotment of treatments has the same probability (eMATHZONE, n.a.). It is generally difficult to eliminate bias using only their expert judgment and hence the use of randomization in experiments is common practice. In a randomized experimental design, objects or individuals are randomly assigned (by chance) to an experimental group. Using randomization is the most reliable method of creating homogeneous treatment groups, without involving any potential biases or judgments (Course, 1997).

Replication: The second principle of an experimental design is replication, which is a repetition of the basic experiment. Although randomization helps to insure that treatment groups are as similar as possible, the results of a single experiment, applied to a small number of objects or subjects, should not be accepted without question. Randomly selecting two individuals from a group of four and applying a treatment with “great success” generally will not impress the public or convince anyone of the effectiveness of the treatment. To improve the significance of an experimental result, replication, the repetition of an experiment on a large group of subjects, is required.

Local Control: It has been observed that all extraneous sources of variation are not removed by randomization and replication. This necessitates a refinement of the experimental technique. In other words, we need to choose a design in such a manner that all extraneous sources of variation are brought under control. For this purpose, we make use of local control, a term referring to the amount of balancing, blocking and grouping of the experimental units. Balancing means that the treatments should he assigned to the experimental units in such a way that the result is a balanced arrangement of the treatments. Blocking means that like experimental units should be collected together to form a relatively homogeneous group. A block is also a replicate. The main purpose of the principle of local control is to increase the efficiency of an experimental design by decreasing the experimental error (eMATHZONE, n.a.).

[1] Treatment groups here refer to groups where a certain intervention is employed as opposed to control groups that receive no intervention (also known as receiving placebo in medical interventions)

Bibliography

Course, Y. (1997). Experimentation. http://www.stat.yale.edu/Courses/1997-98/101/expdes.htm: Yale.

eMATHZONE. (n.a.). Basic Principles of Experimental Design. https://www.emathzone.com/tutorials/basic-statistics/basic-principles-of-experimental-designs.html#ixzz78MGIijJa.

How to cite: Tomar, Manika. 2021. “Rules that cannot be ignored about experimental evaluation design.” DevInsights.