Evaluating Scientific Claims

Every day, we encounter claims that sound scientific.

You might see statements such as:

• "This drink improves concentration."
• "This material is completely biodegradable."
• "Plants grow faster when exposed to music."
• "This cleaning product kills 99% of bacteria."
• "Electric vehicles are better for the environment."
• "This diet gives you more energy."

Some claims are strongly supported by scientific evidence. Others are based on weak evidence, personal experiences, misleading statistics, or opinion.

Being scientifically literate means being able to ask:

What exactly is the claim, and what evidence supports it?

A claim should not be accepted simply because it sounds scientific. Good scientific thinking involves examining the evidence, methods, sources, reasoning, uncertainty, and alternative explanations.

What Is a Scientific Claim?

A claim is a statement that says something is true or describes a relationship between ideas or variables.

For example:

Increasing water temperature decreases the time required for sugar to dissolve.

This is a scientific claim because it can be investigated using observations and measurements.

Another claim might be:

Bean seeds germinate less successfully as salt concentration increases.

Again, this can be tested experimentally.

Scientific claims should be open to investigation.

The important question is not simply:

"Do I agree with this?"

Instead ask:

"What evidence supports this claim?"

---

Evidence and Opinion

Evidence consists of information that can be used to support or challenge a claim.

Scientific evidence may include:

• measurements
• experimental results
• observations
• repeated trials
• survey data
• field observations
• photographs or recordings
• graphs
• statistical analyses
• results from multiple studies

An opinion expresses a belief, preference, or judgment.

For example:

"Blue is the nicest colour."

This is an opinion.

Compare:

"In a survey of 500 students, 38% selected blue as their favourite colour."

This is evidence about what the surveyed students reported.

It does not prove that blue is objectively the "nicest" colour.

---

Evidence Is Not the Same as a Claim

Consider:

Claim: Plants grow faster under blue light.

That is something someone wants you to accept.

Now consider:

Evidence: Twenty bean plants grown under blue light had an average height of 18.4 cm after 21 days, compared with 14.1 cm for twenty plants grown under another specified light condition.

That is evidence that can be examined.

A strong scientific argument connects:

Claim → Evidence → Reasoning

The reasoning explains why the evidence supports the claim.

---

Claim, Evidence, Reasoning

This structure is often called CER.

Claim

What conclusion are you making?

Increasing temperature increased the reaction rate.

Evidence

What data support the claim?

At 20 °C the reaction took an average of 84 s, while at 50 °C it took an average of 31 s.

Reasoning

Why does the evidence support the claim?

A shorter reaction time indicates a faster reaction under the conditions tested. Therefore, the data show that the reaction rate increased as temperature increased.

A strong scientific argument needs all three parts.

---

Ask: Where Did the Evidence Come From?

Not all evidence has the same quality.

Suppose someone claims:

"Plants grow better when classical music is played."

Before accepting the claim, ask:

• How many plants were tested?
• What species were used?
• Was there a control group?
• Were light and water controlled?
• How was "better growth" measured?
• How long did the experiment last?
• Were trials repeated?
• Was sound volume controlled?
• Were the results statistically meaningful?
• Have other researchers obtained similar results?

The details of how evidence was produced matter enormously.

Look at Sample Size

Suppose a company claims:

"Students learn 50% faster using our study method."

Then you discover the test involved:

three students

That is weak evidence for making a broad claim about students in general.

A larger and appropriately selected sample can provide stronger evidence because unusual individuals have less influence on the overall result.

Compare:

3 participants

with:

3,000 participants

The larger study is not automatically correct, but its sample size may provide a stronger basis for generalization if the study is otherwise well designed.

---

A Large Sample Does Not Fix a Bad Experiment

Imagine a study involving 10,000 people.

That sounds impressive.

But suppose:

• participants were selected in a biased way
• there was no suitable comparison group
• measurements were unreliable
• important variables were uncontrolled

The large sample does not automatically make the conclusion trustworthy.

Evidence quality depends on more than how much data were collected.

It also depends on how the data were collected.

---

Look for a Control Group

A control group provides a comparison.

Suppose someone tests a new fertilizer on 20 plants.

After four weeks, the plants grow an average of 8 cm.

Does this prove the fertilizer increased growth?

No.

Perhaps the plants would have grown 8 cm without fertilizer.

A stronger investigation could compare:

Group A: plants receiving fertilizer

Group B: similar plants receiving no fertilizer

If other important conditions are controlled, the difference between the groups provides more useful evidence about the fertilizer's effect.

---

Look for Controlled Variables

A scientific claim may appear supported until you notice that several variables changed.

Suppose a student claims:

"Fertilizer X caused plants to grow taller."

But:

• plants receiving Fertilizer X were placed beside a sunny window
• plants without fertilizer were placed in a darker location

Now two factors changed:

fertilizer + light

You cannot confidently determine which factor caused the difference.

This is called a confounding variable or confounding factor.

---

Correlation and Causation

One of the most important skills in evaluating claims is distinguishing between correlation and causation.

A correlation means that two variables are related.

For example:

As temperature increases, ice cream sales increase.

You might also find:

As temperature increases, sunburn cases increase.

This does not mean:

Buying ice cream causes sunburn.

A third variable—warmer, sunnier weather—can influence both.

A relationship between two variables does not automatically prove that one caused the other.

Evidence for causation is stronger when researchers:

• manipulate the proposed cause
• control other important variables
• use suitable comparison groups
• repeat the investigation
• find consistent results
• identify a reasonable scientific mechanism

---

Look at the Actual Numbers

Claims may sound dramatic while describing very small changes.

For example:

"Product X increases plant growth by 50%!"

That sounds impressive.

But suppose untreated plants grew:

2 mm

and treated plants grew:

3 mm

The relative increase is indeed:

(3 − 2) ÷ 2 × 100 = 50%

But the absolute difference is only:

1 mm

Both pieces of information matter.

When evaluating numerical claims, ask:

What are the actual values?

---

Relative and Absolute Change

Consider a hypothetical situation in which a result changes from:

2 cases per 1,000

to:

1 case per 1,000

The relative reduction is:

(2 − 1) ÷ 2 × 100 = 50%

The absolute reduction is:

1 case per 1,000

Both statements describe the same data.

Neither calculation is necessarily wrong, but reporting only one can create a very different impression.

Good scientific communication provides enough context for the audience to understand the actual size of the effect.

---

Beware of Cherry-Picking

Cherry-picking occurs when someone selects only the evidence that supports a preferred claim while ignoring evidence that does not.

Imagine ten experiments investigating the same question.

Eight find little or no effect.

Two find a large effect.

If someone discusses only those two studies and ignores the other eight, the audience receives a misleading picture of the evidence.

A strong evaluation asks:

What does the overall body of evidence show?

not merely:

"Can I find one study supporting this claim?"

---

One Study Is Rarely the Whole Story

Scientific understanding usually develops through many investigations.

One study may:

• use a small sample
• produce an unusual result
• contain an unnoticed limitation
• apply only under certain conditions
• disagree with later research

Confidence generally increases when different researchers using appropriate methods obtain consistent results.

This is why replication and reproduction are important in science.

Compare Multiple Sources

Suppose you find three sources discussing the same scientific question.

Source A

A peer-reviewed scientific study with clearly described methods and data.

Source B

A university science article summarizing several studies and providing references.

Source C

An anonymous social media post claiming:

"Scientists have finally proved this!"

You should not treat all three sources as equally strong.

Consider:

• expertise
• evidence
• methodology
• references
• publication process
• possible conflicts of interest
• consistency with other evidence

Comparing multiple sources helps you build a more complete picture.

---

Source Reliability Matters

When examining scientific information, ask:

Who produced it?

Possible sources include:

• universities
• government science agencies
• scientific organizations
• peer-reviewed journals
• companies
• newspapers
• blogs
• influencers
• anonymous accounts

The source alone does not automatically determine whether a claim is true or false.

However, source information helps you judge:

• expertise
• accountability
• transparency
• possible bias
• whether evidence can be checked

A trustworthy source should make it possible to determine where its claims came from.

---

Check the Original Source

Scientific claims often travel through several layers.

For example:

Scientific study

↓

University press release

↓

News article

↓

Social media post

↓

Someone's comment

At every step, information can become simplified or exaggerated.

A headline might say:

"Scientists prove X causes Y!"

while the original research actually says:

"X was associated with Y in the population studied."

Those statements are very different.

Whenever a claim is important, try to trace it back toward the original evidence.

---

Watch for Misleading Graphs

Graphs can communicate evidence extremely well.

They can also create misleading impressions.

Suppose two values are:

98 and 100

A graph with a y-axis running from:

0 to 100

will show a small difference.

A graph with a y-axis running from:

97 to 100

can make the same difference appear enormous.

Neither scale is automatically wrong, but you need to examine the axes before interpreting the visual impression.

When examining a graph, check:

• axis labels
• units
• scale
• starting values
• intervals
• missing data
• sample size
• whether error bars are shown when relevant

Never judge a graph only by its shape.

---

Look for Uncertainty

Scientific evidence is rarely perfectly certain.

Good scientific communication may acknowledge:

• measurement uncertainty
• variation
• sample limitations
• confidence intervals
• error bars
• alternative explanations
• limitations of the experimental method

Be cautious when complicated scientific evidence is described using absolute phrases such as:

• "100% proven"
• "no doubt whatsoever"
• "guaranteed"
• "this settles the question forever"

Science usually communicates conclusions in proportion to the strength of the available evidence.

---

Weak Scientific Arguments

A scientific argument may be weak for many reasons.

Very small sample

"I tested two plants, so this fertilizer works for all plants."

No control group

"The plants grew after receiving fertilizer, therefore the fertilizer caused the growth."

Uncontrolled variables

"The warmer plants grew faster, but they also received more light."

Anecdotal evidence

"It worked for my friend, so it must work."

Cherry-picked evidence

Only supportive results are presented.

Overgeneralization

Results from one limited experiment are applied to every possible situation.

Correlation treated as causation

Two variables change together, so one is assumed to cause the other.

Unsupported mechanism

An explanation is presented without evidence.

Appeal to authority

"A famous person says it works."

Expertise can matter, but evidence remains essential.

---

Anecdotes Are Weak Scientific Evidence

An anecdote is a personal story or individual experience.

For example:

"My grandfather drank this tea every day and lived to 95, so the tea increases lifespan."

This does not establish causation.

Many other factors could have influenced his lifespan.

Anecdotes can be useful for:

• generating questions
• noticing possible patterns
• suggesting ideas for research

But controlled scientific evidence is needed to test the claim.

---

Identify Possible Bias

Bias occurs when some feature of a study, sample, measurement, or reporting process systematically influences the result.

Suppose a company tests whether customers like its product by surveying only people who already subscribe to its fan club.

That sample may not represent the wider population.

Other forms of bias can occur through:

• participant selection
• question wording
• measurement methods
• data exclusion
• publication choices
• researcher expectations

Bias does not necessarily mean deliberate dishonesty.

It can arise unintentionally.

---

Conflicts of Interest

Sometimes the person or organization producing research may benefit from a particular result.

For example, a company may fund research about its own product.

This does not automatically mean the research is false.

Instead, it is a reason to examine:

• methodology
• transparency
• data
• independent replication
• other research on the same question

A conflict of interest should increase scrutiny, not automatically determine your conclusion.

---

Consider Alternative Explanations

Good scientific reasoning asks:

Could something else explain these results?

Suppose students who sleep longer receive higher test scores.

Possible explanation:

More sleep improves academic performance.

But there could be other factors.

Perhaps students who sleep longer also:

• have more regular schedules
• experience less stress
• study differently
• have different workloads

The observed relationship is evidence worth investigating, but additional research is needed before establishing causation.

---

Does the Conclusion Match the Evidence?

Consider this evidence:

Ten bean plants receiving Fertilizer A grew an average of 2.1 cm more than ten untreated bean plants during a 14-day experiment.

Reasonable conclusion:

Under the conditions tested, plants receiving Fertilizer A showed greater average growth during the 14-day period.

Unreasonable conclusion:

Fertilizer A is the best fertilizer for every plant species.

The second claim extends far beyond:

• the sample
• the species
• the time period
• the experimental conditions

Good conclusions remain within the limits of the evidence.

---

Evidence Can Have Different Strengths

Not all evidence should be treated equally.

For a scientific question, evidence is generally stronger when it comes from investigations with features such as:

• appropriate sample size
• suitable control groups
• careful variable control
• reliable measurements
• repeated trials
• transparent methods
• appropriate statistical analysis
• independent replication
• consistency with other high-quality evidence

Evidence is generally weaker when it relies mainly on:

• anecdotes
• tiny samples
• uncontrolled observations
• vague measurements
• selective reporting
• unsupported assertions

The goal is not simply to count how many pieces of evidence exist.

You need to consider their quality.

---

Evidence Can Change Scientific Conclusions

Scientific conclusions are not supposed to remain unchanged regardless of new evidence.

Imagine early experiments suggest:

A causes B.

Later, larger and better-controlled studies show little relationship.

Scientists should reconsider the original claim.

Changing a conclusion when stronger evidence becomes available is not a weakness of science.

It is one of science's strengths.

Scientific knowledge is evidence-responsive.

---

A Useful Claim-Evaluation Checklist

When you encounter a scientific claim, ask:

What exactly is being claimed?

What evidence is provided?

Where did the evidence come from?

How large was the sample?

Was there a suitable comparison or control?

Were important variables controlled?

Were measurements repeated?

Are the statistics appropriate?

Are there anomalous results?

Does the graph represent the data fairly?

Are alternative explanations possible?

Does correlation appear to be confused with causation?

Do other reliable sources support the finding?

Are limitations acknowledged?

Does the conclusion go beyond the evidence?

This turns:

"Does this sound convincing?"

into:

"How strong is the evidence supporting this claim?"

---

Worked Example: Evaluating a Claim

Consider the claim:

"Listening to music makes plants grow faster."

A student reports that one plant exposed to music grew to:

22 cm

while one plant without music grew to:

17 cm

Is there evidence?

Yes.

Two plant heights were measured.

Is the evidence strong?

Not particularly.

The sample size is extremely small.

What else should we know?

• Were the plants the same species?
• Did they start at similar sizes?
• Did they receive equal water?
• Did they receive equal light?
• Were they kept at the same temperature?
• Was the music volume controlled?
• How long did the experiment last?
• Were several plants tested?
• Was the experiment repeated?

Reasonable conclusion

The result provides a reason to investigate further, but it is not sufficient evidence to conclude that music generally increases plant growth.

That conclusion matches the strength of the evidence.

---

Worked Example: Advertising Claim

An advertisement states:

"Our product improves performance by 40%."

Before accepting this claim, ask:

40% compared with what?

Was performance:

10 units → 14 units?

Was there a control group?

How many people were tested?

How was "performance" measured?

Was the study independently conducted?

Were all results reported?

Is 40% a relative or absolute change?

Without this information, the percentage alone tells you very little.

---

Common Mistakes

Accepting a claim because it sounds scientific

Scientific vocabulary is not scientific evidence.

Confusing opinion with evidence

Personal belief does not replace measurements or observations.

Trusting one study automatically

Compare findings with other reliable evidence.

Assuming a large sample guarantees good science

Experimental design and measurement quality still matter.

Ignoring the control group

Without an appropriate comparison, cause and effect can be difficult to establish.

Confusing correlation with causation

Variables changing together does not prove that one causes the other.

Looking only at percentages

Check the underlying numbers and absolute differences.

Ignoring anomalous results

Unexpected data may contain important information.

Trusting graphs without checking their axes

Graph design can exaggerate or hide differences.

Cherry-picking

Evaluate the overall evidence rather than selecting only supportive results.

Treating anecdotes as proof

Personal experiences can suggest questions but usually provide weak evidence for general scientific claims.

Ignoring conflicts of interest

Consider who produced or funded the research, while still evaluating the actual evidence.

Overgeneralizing

Do not extend conclusions beyond the populations and conditions that were investigated.

---

Key Terms

Claim — A statement presented as true and potentially supported by evidence.

Scientific claim — A claim about the natural world that can be investigated using scientific evidence.

Evidence — Observations, measurements, data, or research findings used to support or challenge a claim.

Opinion — A belief, judgment, or preference that may not be based on scientific evidence.

Reasoning — The logical scientific explanation connecting evidence to a claim.

Claim-Evidence-Reasoning (CER) — A structure connecting a conclusion with supporting evidence and scientific explanation.

Anecdote — A personal story or individual experience.

Sample — The group of observations, organisms, objects, or people investigated.

Sample size — The number of observations or individuals included in a study.

Control group — A comparison group that does not receive the experimental treatment or condition.

Confounding variable — An uncontrolled factor that could influence the observed result.

Correlation — A relationship in which two variables change together.

Causation — A relationship in which one factor produces a change in another.

Bias — A systematic influence that can affect how evidence is collected, analyzed, or reported.

Cherry-picking — Selecting supportive evidence while ignoring relevant evidence that does not support the preferred conclusion.

Conflict of interest — A situation in which personal or organizational interests could potentially influence research or its interpretation.

Replication — Repeating research to determine whether similar findings can be obtained.

Absolute change — The numerical difference between two values.

Relative change — A change expressed in relation to the original value, often as a percentage.

Statistical evidence — Numerical analysis used to describe or evaluate data.

Uncertainty — The degree of doubt associated with measurements or conclusions.

Overgeneralization — Extending a conclusion beyond the evidence or population actually investigated.

Scientific argument — A conclusion supported using scientific evidence and reasoning.

Scientific literacy — The ability to understand, evaluate, and use scientific information and evidence.

---

Key Takeaways

• Scientific claims should be evaluated using evidence, not simply accepted or rejected based on personal opinion.
• A scientific claim should be open to investigation.
• Evidence may include measurements, observations, experimental results, graphs, and statistical analyses.
• Strong scientific arguments connect Claim → Evidence → Reasoning.
• Ask where the evidence came from and how it was collected.
• Sample size affects how confidently results can be generalized.
• Large samples do not repair poor experimental design.
• Control groups provide important comparisons.
• Uncontrolled variables can create alternative explanations.
• Correlation does not automatically demonstrate causation.
• Examine the actual numbers behind percentages and dramatic claims.
• Relative and absolute changes can create very different impressions.
• One study rarely provides the entire scientific picture.
• Compare evidence from multiple reliable sources.
• Trace important claims toward their original evidence when possible.
• Examine graph axes, scales, units, and missing information.
• Be cautious when uncertainty and limitations are ignored.
• Anecdotes can generate scientific questions but generally provide weak evidence for broad conclusions.
• Cherry-picking creates a misleading picture of the evidence.
• Bias can influence how evidence is collected or reported.
• A conflict of interest does not automatically invalidate research, but it provides a reason for careful scrutiny.
• Consider alternative explanations for observed patterns.
• Strong conclusions remain within the limits of the evidence.
• Evidence quality matters more than simply counting the number of sources.
• Scientific conclusions should change when stronger evidence becomes available.

When evaluating a scientific claim, the most important question is not:

"Do I believe it?"

Ask instead:

"What evidence would justify this conclusion, and how well does the available evidence meet that standard?"

---

Check Your Understanding

1. What is the difference between a scientific claim and scientific evidence?

2. Explain the difference between evidence and opinion.

3. What are the three parts of Claim-Evidence-Reasoning?

4. Why does sample size matter when evaluating a scientific claim?

5. Explain why a study involving 10,000 participants could still provide poor evidence.

6. Why is a control group useful?

7. What is a confounding variable?

8. Explain the difference between correlation and causation.

9. Give an example of two variables that might be correlated without one causing the other.

10. A product reduces a value from 4% to 2%. Calculate the relative reduction and the absolute reduction.

11. Why can reporting only the relative change sometimes be misleading?

12. What is cherry-picking?

13. Why should scientific claims be compared with evidence from multiple sources?

14. Explain why an anecdote is usually weaker evidence than a controlled investigation.

15. What information should you check before trusting a graph?

16. Why should a scientific conclusion acknowledge the limitations of the evidence?

17. A company funds research about its own product. Does this automatically mean the research is incorrect? Explain.

18. A study finds that students who sleep longer receive higher test scores. Explain why this finding alone does not prove that additional sleep caused the higher scores.

19. A student tests one plant with fertilizer and one without fertilizer. The fertilized plant grows taller. Evaluate the strength of the claim that the fertilizer increases plant growth.

20. Challenge: You see the claim:

"Scientists prove that listening to music while studying improves grades by 30%."

Write at least six questions you would ask before deciding whether the evidence supports this claim.