What makes a control experiment?

by Mia Breeze | Apr 18, 2024

Modern science was born out of experiments and future progress in science will continue to be made through experiments. The validity of science is therefore dependent of the validity of the experiments which underpin it. Understanding scientific experiments is crucial to comprehending science. One of the easiest ways to evaluate the results of an experiment is to compare it with the results of its control experiment.  Control experiments are also the easiest way to demonstrate flaws in the reasoning behind an experiment or in the methods.

Control experiments

The Collins English Dictionary defines control experiments as “An experiment designed to check or correct the results of other experiments by removing the variable or variables operating in that other experiment. The comparison obtained is an indication or measurement of the effect of the variables concerned.”

Notice that control experiments are defined by their function or purpose, namely, to determine the impact of certain variables on the results of the main experiment. There is no other criteria which disqualifies a control experiment from being one. As long as the experiment provides useful and comparative information in relation to the results of the main experiment by evaluating the effect of a variable on the results of the main experiment, that experiment amounts to a valid or proper control experiment.

How much clarity the control experiment does or does not provide regarding the main experiment is not an issue. If the experiment does nothing but rule out the possibility of confounding variables, it still amounts to a control experiment. Control experiments can therefore be tailored to test any experiment or any one step in an experiment. What form a control experiment takes depends on the setup of the main experiment or the step and variable being evaluated.

Control experiments should not be confused with a controlled variable in an experiment or a controlled experiment. A controlled experiment is one in which all the variables applicable to the main experiment, other than the independent and dependent variables, are held constant or standardized across the experiment. By ensuring the conditions of the experiment are the same, the impact of the variables on the results of the experiment are controlled and able to be accounted for.

Ensuring the same concentration of reagents are used, or ensuring the experiment takes place in an environment where the temperature and light are constant every time the experiment is run, are examples of controlled variables.

Perhaps the easiest way to distinguish between a controlled variable and a control experiment, is to keep in mind that a control experiment is an experiment carried out alongside the main one and provides the experimentor with results against which he or she can compare the results from the main experiment. A controlled variable does not provide the experimentor with any results nor is it an experiment on its own. A controlled variable standardises the impact of a variable within the main experiment.

There are few control experiments that are common among experiments. However, there is no definitive or exhaustive list of control experiments as they can take on different forms depending on the design of the experiment.

• Negative control – this creates what is refered to as the control group / controlled result of an experiment. It is achieved by conducting the same experiment under the same conditions except without the addition of independent variable. This step is generally not expected to produce a result and as such provides a baseline against which the results of the main experiment/ the experiment group can be compared.
• Positive control – this also creates a control group / controlled result, however, this group/ result is achieved by conducting the same experiment under the same conditions except with the addition of a different substance/thing in place of the independent variable. This step usually provides an example of a positive result against which the experimental group /results can be compared.

• Ethical control – in certain clinical trials it is considered unethical to have a proper control group as those carrying out the experiment are concerned that it would be unethical to withhold what is believed to be an effective drug / treatment from some patients. In these cases, and where possible, all patients are initially given the drug, and then, halfway through the clinical trial , the drug was stopped for a randomly selected half of them. The relapse rates of the two groups are then compared.

The following are examples of control experiments tailored for biology experiments by Harold Hillman:1

• Previous control – when a tissue is examined before and after an agent has been added, sometimes several. These are use often used by pharmacologists or can be used by microscopists.
• Control experiments with ischaemia, hypoxia or absence of substrate are used to determine whether one is studying a biosynthetic, rather than a simple chemical reaction. These are useful when doing experiments concerning biochemistry.
• Control experiments run with fixed or boiled preparations, with no likely biosynthetic properties but similar proteins to those in living tissue. These can be carried out when enzymes feature in an experiment.
• Control experiments in terms of which a temperature gradient is used to decide whether a reaction is metabolic or chemical.

Experiments and the Scientific Method

It is said that research amounts to scientific research when it adheres closely to the scientific method. Any beliefs or statements drawn from research that does not closely adhere to the scientific method is deemed to be pseudoscience or pseudoscientific.

The scientific method is generally set out as below:

If one considers the scientific method closely it is apparent that the scientific method can only be strictly adhered to in cases where the research is directed towards determining the cause-effect relationship of an observed phenomena. Any research that only demonstrates correlation can therefore be deemed pseudoscientific.

This is because the scientific method requires that all variables involved in an experiment (other than the independent and dependent variables) must be controlled and accounted for throughout the experiment. If variables cannot be controlled, then it cannot be said unequivocally that the independent variable is the root cause of the changes in the dependent variables (the observed phenomena).

By way of an example, current research is only able to show that there is a correlation between vaccinations and auto-immune diseases and no direct causal link has or can be demonstrated. This is because there are far too many variables to be accounted for (diet, home environment, inherited traits etc.) and it is likely to be considered unethical to do an experiment unequivocally demonstrating cause (force some people to get vaccinated / remain unvaccinated). As such, this research should be deemed pseudoscientific and, according to some, disregarded.

However, what we see in reality is that this type of research is not disregarded or ignored because even though it’s not establishing the cause it still provides valuable information. Moreover, there are in actual fact many examples of experiments which are technically pseudoscientific but are still valued by society as they nevertheless provide useful information.

Scientific experiments can be divided into two main categories: qualitative and quantitative.

Qualitative experiments are concerned with subjective phenomena, focusing on gaining insight and understanding about an individual’s perception of event and circumstances. Variables and results from this research can’t be numerically measured and generally involve people’s senses or feelings.

Even though Qualitative research can set out to determine the cause of the subjective phenomena studied, it does not do so objectively or according to the scientific method. In qualitative research, researchers do not study the predictive relationship between dependent and independent variables, variables are not strictly controlled, and control experiments are not necessary. Technically, all qualitative research is therefore pseudoscientific research.

Quantitative experiments, on the other hand, concern objective phenomena in terms of which things are measured or assigned a numeric value. For example, describing something as a square would be qualitative but measuring it and stating that it has four equal sides of 2 cm would be quantitative.

Quantitative experiments can be further broken down into four further sub-categories, descriptive research, correlational (statistical) research, quasi-experimental research and (true) experimental research.

Descriptive and correlational (statistical) research focuses on recognising trends and patterns in data (i.e correlation) but does not aim to prove causation of those trends or patterns. Examples are: a description of how the average 8 year old spends their summer holidays or the relationship between intelligence and self-esteem.

Similar to qualitative research, variables in these experiments are not manipulated or controlled. Instead, variables are identified, and their distribution and connections studied and described as they occur in a natural setting. Descriptive and statistical experiments can therefore only establish correlation and not causation.  This means that these experiments are technically also pseudoscientific.

Quasi- experimental research and (true) experimental research both attempt to establish the cause-effect relationship among a group of variables. These two types of experiments are very similar except for some key differences.

In (true) experimental research an effort is made to identify and impose control over all the variables except for one, the independent variable. The independent variable is manipulated to determine the causal effects on the dependent variables. When every variable in an experiment is identified and perfectly controlled, the results from such experiments can unequivocally demonstrate causal relationships. These are thus true scientific experiments and not pseudoscientific.

However, it is very often not possible to design an experiment in terms of which every variable is known and/or perfectly controlled. This is especially the case in the field of biology and medicine as the scientific method can be difficult to apply stringently to diverse, interconnected systems and large data sets.

In fact, many experiments attempting to determine causative relationships which are held out to be (true) experimental experiments are in actual fact quasi-experimental experiments.

In quasi-experimental research, the independent variable is identified and its effects on the dependent variables measured, however, the independent variable is generally not manipulated by the experimentor. In addition, all other variables are not always identified and/or controlled and groups are not randomly assigned. This could, as already mentioned, be because it is not possible to do so or because it is unethical to do so.

In other words, experiments are often quasi-experimental because there are factors limiting how much variables can be controlled. The result hereof is that the study, including the control experiments, has to be designed to adbide by these other factors in play. For example often non-equivalent control experiments are carried out for these type experiments. A non-equivalent control experiment is when the control experiment creating the control group/ result does not exactly align with the experimental group/ result.

Non-equivalent control experiments are resorted to in attempt to limit the impact of the lack of controlled variables. This is because when all the variables cannot be perfectly controlled the confounding of variables is very likely. When variables are confounded the relationship between two variables is distorted or confused.

A simple example of confounding variables would be people who take vaccinations may also take other pharmaceuticals (antibiotics, antidepressants etc) regularly, so pharmaceutical use is a confounding variable in the study of the association between vaccinations and auto-immune diseases. The increase in auto-immune disease could be due to the regular use of pharmaceuticals and not solely due to the vaccinations.

Therefore, when analyses and conclusions are made from the results of quasi-experimental experiments, determining causative relationships must be done carefully, as other variables, both known and unknown, could still affect the outcome. This is why control experiments are carried out in addition to having controlled variables, as they are crucial for establishing whether firm causation relationships can be made among the variables.

Pseudoscience and control experiments

Control experiments prevent us from having to reject all pseudoscientific research as useless out-right. This is because control experiments can confirm or reject the value of a particular pseudoscientific experiment by allowing us to narrow down the confounding variables.

This is especially important to understand when one realises that the majority of experiments do not adhere to the scientific method, are actually pseudoscientific in nature, but nevertheless provide useful information.

What this means is that while determining whether an experiment adheres to the scientific method is useful, it cannot be the sole factor when it comes to determining the value of an experiment. As I have tried to demonstrate above, applying that criteria would exclude most scientific research from being considered valuable. Moreover, we can find many examples of pseudoscientific experiments being considered valuable. This means, realistically speaking, that classifying something as pseudoscientific or not is not determinate or saying much at all.

When determing how much value should be attributed to a particular experiment what should be considered is not whether the experiements fit into any definition or model but rather to what extent the variables have been accounted for and controlled and to what extent control experiments have been employed the for those variables not controlled.

A simple example of control experiments adding value to a pseudoscientific experiment would be experiments carried out to determine the effects of the staining on cells observed under a light microscope.

Observing tissues and cells under a microscope does not follow the scientific method at all. Yet it is still considered a scientific experiment and the information derived from such experiments is valued.

It is recognised by cytologists that using stains to better see the cells under a light microscope at the very least shrinks those cells. Yet when taking measurements of stained cells, no account is made for shrinkage. Instead what is asserted is that the cells in living things are the same size as the stained cells under the microscope. As such, a variable hasn’t been taken into account and the results of these experiments have limited usefulness.

However, the value attributed to these results can increase once control experiments designed to establish the precise amount shrinkage a particular stain causes to a cells are carried out.2

The importance of control experiments cuts both ways. In the same way that a control experiment can save the results of a pseudoscientific experiment from being disregarded, a well-designed control experiment can demonstrate that a particular experiment proves nothing of value whatsoever. This is particularly useful when trying demonstrate the uselessness of an experiment to those who are unable to see or unwilling to admit the flaws inherent in a particular experiment.

A perfect example of this are the control experiments which have been carried out in relation to the cell culture procedures used to “isolate viruses”. The cell culture experiment is pseudoscientific for more than one reason but mainly because no independent variable or dependent variable is identifiable prior to the experiment. The independent variable (the virus) is from the beginning assumed to be in the sample and the dependent variables (the cytopathic effect ‘CPE’) which is not a a phenomenon observed in nature is used to confirm the presence of the independent variable.

Despite this being obvious, 90% of virologists will tell you there is nothing wrong with this pseudoscientific approach and that the experiment reliably detects the presence of a virus. After all, as already discussed, there are many experiments which are technically pseudoscientific which are still relied on, why should that fact matter to this one?

The only way therefore to counteract this inability or unwillingness to appreciate the flaws in the cell culture experiment is through control experiments, and particularly through control experiments designed to disprove the belief that the cell culture experiment detects the presence of virus.

This is why Stephan Lanka’s control experiments were so instrumental in demonstrating the fallibility of the cell culture experiment. Lanka’s control experiments showed that the experiment carried out under that same condition would produce the same results (CPE) without adding a sample or even when adding yeast as a sample. Comparing Lanka’s control experiment results with those of the main experiment demonstrated clearly that it could not be the “virus” assumed to be in the sample that was causing the dependent variables. The control experiments demonstrated that it was undoubtedly the experiemental protocol that was causing the observed changes.

This is also why the signatories of the “Settling the virus debate” challenge insisted on four different samples being used in the cell culture experiment.3 Those signatories were certain that each of the samples insisted upon would result in indistinguishable CPE to the CPE produced by “virus” infected samples.

In other words, the signatories knew that by running the experiment with those four different samples, the results thereof would act as control experiments in the context of that experiment. This is because the comparison between the results would indicate that the CPE produced was not specific to “viruses”.

This is also why the experiment carried out by Enders and Peebles et al.  in the foundational 1954 “virus” isolation paper with the “second agent obtained from an uninoculated kidney culture” amounts to a valid control experiment. Despite Enders not acknowledging it to be so and despite it not being a true negative control experiment, it nevertheless provided comparative results which informed the results of the main experiment.

The experiment carried out with the second agent acted as a control experiment in the context of that experiment. It demonstrated that the experiment was flawed because the control experiment and the experiment produced the same results – they were indistinguishable. This non-equivalent control didn’t pinpoint the exact flaws in the experiment but it certainly pointed to a major flaw. To disregard this experiment or exclude it from qualifying as a control experiment, completely ignores the role control experiments play in science.

Conclusion

When it comes to understanding experiments and ultimately science, it is crucial to realise that labelling experiments as pseudoscientific or control experiments as not valid, valid, not proper, or proper is not really helpful or informative. This is because determining whether an experiment or control experiments is useful or valuable must be determined on a case by case basis and by analyzing all the data gathered.

Strictly applying labels or definitions is unrealistic given that acceptable scientific research covers a broad spectrum of techniques and accounting for all variables is not always possible.

At the very least, those labels should not be determinative when it comes the amount of value that should be attributed to an experiments. If we were to go by labels and defitions alone there would be a lot of experiments that could not be relied upon or used to challenge other scientific findings.

For instance, are we to disregard the evidence showing vaccinations have a close correlation to autism and auto-immune diseases because such research is pseudoscientific? Are we to disregard the results from the experiment Enders carried out with the second agent because it technically does not fit into one of the recognised types of control experiments? Are Lanka’s experiments to be understood as something other than control experiments because they do not neatly fit under a recognized control experiment? The answer to these questions surely has to be no.

Note, it is not being proposed that the scientific method should be abondoned or that definitions should be ignored. Just that their application should not be definitive.

What should instead be determinative is to what extent the variables have been accounted for and controlled in the context of that particular experiement. To what extent confounding variables have been ruled out by control experiments. In other words, what lengths have been employed to ensure that the data is made as reliable as possible.

Excluding experiments, and especially control experiments, for any other reason is counter productive and greatly limits the amount of tools available to evaluate and challenge science.

Subscribe to dpl’s newsletter