Critical Thinking in Science: Important Concepts to Consider

Ruling out Rival Hypotheses

Usually the results of any single study are consistent with several different hypotheses. Additional research is often needed to decide which hypothesis is best supported. When looking at a pattern of results that has been reported from a study, it is important to ask, “are there any alternative hypotheses that could explain this pattern of data?”

That is, we should consider whether there are any other reasons why the researchers might have found the particular results that they found in their study. Maybe there was a confounding variable in an experiment that could offer a different explanation for the results, other than the one that the researchers have given. The rival hypotheses that are most important to acknowledge are those that could explain the specific pattern of results that has been found in the study. It is useful to consider how we could attempt to rule out these alternative hypotheses.

A correlation between two things (a statistical association) does not necessarily mean there is a cause and effect relationship between them. If a pattern of results was produced simply by measuring two different things and comparing them, we cannot say anything for sure about whether one of these things caused the other; all we can say is that the two things go together. When a causal claim (e.g., A causes B) is made from a correlation, it’s always important to ask whether the causal connection could be reversed (i.e., B causes A) or whether a third variable could explain the relationship (i.e., A and B do not cause each other; instead C causes A and B to go together). If there is more than one possible pattern of cause-and-effect that could result in a correlation, we cannot use that correlation as evidence that any one specific pattern is necessarily true.

Scientific claims must be capable of being disproved. In other words, we should be able to think of a way to test whether or not a claim is true; there should be data we can collect that tell us if our hypothesis is likely to be true or false. If the claim is made in such a way that there’s no good way to test it, the claim is not really scientific. In science, we should always be open to the possibility that our ideas are wrong. If there are no data that could convince us that our ideas are wrong, then our ideas are not properly scientific.

Correlation Vs Causation

Scientific findings must be capable of being duplicated following the same methodology. In other words, in science, other people must be able to follow our methods and should get similar results. In addition, the most reliable claims are those that have converging evidence for them. We can only really be confident in a claim if it has been tested in multiple different ways and all of them point to the same effect.

Science is, for the most part, a cumulative process, where new claims represent small advances over older ones. A claim that contradicts what we already know, or that seems to promise to completely explain or solve a complex problem in a new way, must have a lot of evidence to back it up. The bigger the claim, the more evidence must be provided.

If two hypotheses explain a phenomenon equally well, in science we generally prefer the simpler explanation. The simpler explanation is not necessarily correct, but we should start by using that explanation and only make a more complicated one when the simple explanation cannot account for our results. In other words, we shouldn’t make our explanations more complicated than necessary.

In science we prefer to base our conclusions on the results of systematic studies of lots of people (data), rather than on information about single individuals. An anecdote is a story about a single person. Personal beliefs are beliefs of a single person. Stories about single people or beliefs of a single person can be hard to verify, hard to generalize, and often fail to inform us about cause-and-effect relationships.

In general, statements that are backed up with data gathered from a study are to be preferred over statements that reflect the opinion of just a single person, or observations or experiences of just a single person’s life. Any single person’s observations may be based on an unrepresentative sample and may be influenced by biases (including a social desirability bias or biases in memory).

There are many technical words in science (e.g., neurotransmitter). These words have a precise meaning. However, pseudoscientific claims often use scientific sounding words that don’t really mean anything (e.g., neuropower). Technical jargon and scientific-sounding words can sound convincing but be essentially meaningless. We should be wary of claims that rely on confusing terminology, especially when those words sound like real scientific ones.

Science provides evidence that either supports or refutes certain ideas we have about the world. But ‘proving’ an idea is almost impossible because future research may show us that our existing ideas are incorrect, or at least only partially true. So, in science we generally avoid using words like “prove, proof, proven”, etc. Instead, we should say that a study “supports” an idea or provides evidence for or against it.