Reliability and Statistical Thinking

Scientists rarely trust an important conclusion based on a single measurement.

Measurements naturally vary. A stopwatch may be started slightly earlier or later, plants may grow at different rates, instruments have limited precision, and individual samples may behave differently.

This is why scientists use repeated measurements, larger sample sizes, averages, ranges, graphs, and other statistical tools.

Statistics help scientists move from:

"This is what happened once."

to:

"This is the overall pattern shown by the evidence."

Statistical thinking does not mean using complicated mathematics. Even simple calculations can help you decide whether experimental evidence is consistent, reliable, and meaningful.

Why Repeat an Experiment?

Suppose you investigate how long a toy car takes to travel 1.0 m.

Your first measurement is:

2.36 s

Can you conclude that the travel time is exactly 2.36 s?

Probably not.

Repeat the measurement and you might obtain:

2.36 s

2.29 s

2.41 s

2.33 s

2.37 s

The measurements are similar, but not identical.

Small differences could result from:

• human reaction time
• slightly different releases
• measurement uncertainty
• changes in friction
• small environmental changes

Repeated trials help reveal this natural variation.

---

Reliability

Reliability refers to the consistency or dependability of experimental evidence.

If an investigation is repeated under similar conditions and produces similar results, we generally have greater confidence in the evidence.

For example:

Experiment A

4.2 s
4.1 s
4.3 s
4.2 s
4.1 s

Experiment B

2.8 s
5.9 s
3.7 s
6.1 s
2.4 s

Experiment A shows much greater consistency.

Experiment B contains much more variation.

This does not automatically mean Experiment B was conducted incorrectly. The system being investigated may naturally vary.

However, the greater spread means we should be more cautious about identifying a typical value from only a few measurements.

---

Repeating Is Different from Reproducing

There are two useful ways scientific findings can be checked.

Repeat measurements are additional measurements made under the same or very similar experimental conditions.

Reproduction occurs when an investigation is carried out again, often by another person or group, to see whether similar findings can be obtained.

If several groups independently obtain similar patterns, confidence in the finding increases.

This is one reason scientific knowledge depends on evidence that can be checked by others.

Finding a Typical Value

Repeated measurements often produce several slightly different values.

Scientists may therefore calculate a value that represents the data.

One of the most common is the mean, often called the average.

Mean = sum of all values ÷ number of values

Suppose the time measurements are:

4.2 s, 4.4 s, 4.1 s, 4.3 s, 4.0 s

First add the values:

4.2 + 4.4 + 4.1 + 4.3 + 4.0 = 21.0

There are five measurements.

Mean = 21.0 ÷ 5

Mean = 4.2 s

The mean gives us a useful single value representing the set of measurements.

---

Why Calculate a Mean?

Imagine measuring the height of five plants:

14 cm, 16 cm, 15 cm, 17 cm, 13 cm

No single plant perfectly represents the entire group.

The mean is:

Mean = (14 + 16 + 15 + 17 + 13) ÷ 5

Mean = 75 ÷ 5

Mean = 15 cm

Instead of describing the treatment using one individual plant, we can describe the group's average response.

This becomes particularly important in biological investigations because organisms naturally vary.

Mean, Median and Mode

Although people often use the word average to mean the mean, there are several ways to describe a typical value.

Mean

Add all values and divide by the number of values.

Data:

3, 4, 4, 5, 9

Mean = (3 + 4 + 4 + 5 + 9) ÷ 5

Mean = 25 ÷ 5

Mean = 5

---

Median

The median is the middle value when the data are arranged in order.

Data:

3, 4, 4, 5, 9

The middle value is:

Median = 4

If there is an even number of values, find the mean of the two middle values.

---

Mode

The mode is the value that occurs most frequently.

Data:

3, 4, 4, 5, 9

The most common value is:

Mode = 4

A data set may have:

• one mode
• more than one mode
• no mode

---

Which Average Should You Use?

The mean is extremely useful for many experimental measurements because it uses every value in the calculation.

However, it can be strongly affected by an unusual result.

Consider:

10, 11, 10, 12, 37

Mean = (10 + 11 + 10 + 12 + 37) ÷ 5

Mean = 80 ÷ 5

Mean = 16

But most measurements are around 10–12.

The value 37 has pulled the mean upward.

The median is:

Median = 11

In this situation, the median may describe the typical value better.

This is why scientists should not calculate statistics mechanically. They should look at the data and think about what the numbers mean.

---

Anomalous Results

An anomalous result is a value that appears noticeably different from the general pattern.

Suppose repeated reaction times are:

31.2 s

30.8 s

31.5 s

48.7 s

31.0 s

The value:

48.7 s

appears anomalous.

But you should not automatically delete it.

Instead ask:

• Was the value recorded correctly?
• Was the procedure followed correctly?
• Did the equipment malfunction?
• Did something unusual happen?
• Could the variation be real?
• Can the measurement be repeated?

Anomalous results should be investigated, not hidden.

---

Should an Anomalous Result Be Included in the Mean?

This depends on the evidence.

Suppose you timed a toy car five times:

2.41 s

2.38 s

2.44 s

8.73 s

2.40 s

You remember that during the fourth trial the car hit a dropped pencil.

There is a clear reason why that trial did not represent the intended experimental conditions.

You could justify excluding that measurement from the calculation and repeating the trial.

However, if you have no scientific reason for excluding a result, you should not simply remove it because it makes your data look less neat.

Any decision to exclude data should be recorded and justified.

---

The Range

The mean tells us about the centre of a data set.

But it does not tell us how spread out the values are.

A simple measure of spread is the range.

Range = highest value − lowest value

Consider:

4.1, 4.2, 4.3, 4.2, 4.2

Highest = 4.3

Lowest = 4.1

Range = 4.3 − 4.1

Range = 0.2 s

Now consider:

2.8, 3.7, 4.2, 5.1, 6.2

Highest = 6.2

Lowest = 2.8

Range = 6.2 − 2.8

Range = 3.4 s

The second data set has a much larger spread.

---

The Same Mean Can Hide Different Data

Consider two groups.

Group A

9, 10, 10, 10, 11

Mean = 10

Range = 11 − 9

Range = 2

Group B

2, 6, 10, 14, 18

Mean = 10

Range = 18 − 2

Range = 16

Both groups have exactly the same mean.

But their distributions are very different.

This demonstrates an important statistical idea:

A mean alone does not describe an entire data set.

Scientists often need information about both:

centre + spread

Percentage

Percentages can be useful when comparing groups of different sizes or describing proportions.

Suppose 18 of 20 seeds germinate.

Percentage germination = (number germinated ÷ total number) × 100

Percentage germination = (18 ÷ 20) × 100

Percentage germination = 90%

Now suppose another group has:

27 germinated out of 30 seeds.

Percentage germination = (27 ÷ 30) × 100

Percentage germination = 90%

Although the groups contained different numbers of seeds, percentages make their results easier to compare.

---

Percentage Change

Scientists may also calculate percentage change.

Percentage change = (change ÷ original value) × 100

Suppose average plant height increases from:

12 cm to 18 cm

Change = 18 − 12

Change = 6 cm

Percentage increase = (6 ÷ 12) × 100

Percentage increase = 50%

This tells us the size of the change relative to the starting value.

---

Larger Sample Sizes

Suppose you want to determine whether salt concentration affects seed germination.

At each concentration you use one seed.

At 0% salt, the seed germinates.

At 1% salt, the seed does not germinate.

Can you confidently conclude that 1% salt prevents germination?

No.

Perhaps the second seed:

• was damaged
• was old
• had poor genetics
• was already non-viable

One organism can behave unusually.

Now imagine using:

30 seeds at each concentration

The behaviour of one unusual seed has much less influence on the overall result.

Why Larger Samples Improve Reliability

Larger samples can help because they:

• better represent natural variation
• reduce the influence of unusual individuals
• produce more stable averages
• allow patterns to be identified more confidently
• provide stronger evidence for comparisons

This is especially important when investigating:

• plants
• animals
• microorganisms
• people
• natural materials

These systems often contain considerable natural variation.

However, larger sample size does not automatically fix a poorly designed experiment.

If the method is biased or important variables are uncontrolled, collecting more measurements using the same flawed method may simply produce more flawed data.

---

Sample Size and Repeated Measurements Are Related but Different

Suppose you investigate plant growth.

Using one plant and measuring it 20 times is not the same as using 20 plants.

Repeatedly measuring one plant can improve confidence in the measurement of that plant.

Using many plants helps you understand variation between individual plants.

The appropriate approach depends on the research question.

For example:

How tall is this particular plant?

Repeated measurements might help.

But:

How does fertilizer affect bean plant growth?

Several plants in each treatment would usually provide stronger evidence.

---

Graphs Help Reveal Patterns

Statistics are not only calculations.

Graphs are also powerful tools for statistical thinking.

A graph can help reveal:

• trends
• clusters
• differences
• relationships
• anomalies
• variation

Suppose you investigate how temperature affects dissolving time.

Looking at the table suggests a pattern.

A graph can make that pattern much easier to recognize:

As temperature increases, dissolving time decreases.

Look Beyond the Average

Suppose two fertilizers produce these plant heights:

Fertilizer A

18 cm
19 cm
20 cm
19 cm
19 cm

Mean = 19 cm

Fertilizer B

10 cm
14 cm
19 cm
24 cm
28 cm

Mean = 19 cm

The means are identical.

But Fertilizer A produced much more consistent results.

The range for A is:

20 − 18 = 2 cm

The range for B is:

28 − 10 = 18 cm

If you reported only the mean, you would miss this important difference.

---

Simple Statistics Work Together

Different statistics answer different questions.

Mean: What is the arithmetic average?

Median: What is the middle value?

Mode: What value occurs most often?

Range: How spread out are the values?

Percentage: What proportion of the total does something represent?

Percentage change: How large is a change relative to the starting value?

No single statistic tells you everything.

Good statistical thinking means selecting calculations that help answer the research question.

---

Statistical Thinking Is More Than Calculating

Suppose you calculate:

Mean = 15.7 cm

The calculation itself is mathematics.

Statistical thinking begins when you ask:

• How much did the measurements vary?
• Were there anomalous results?
• How large was the sample?
• Is the mean representative?
• Is the difference between groups large compared with the variation?
• Were enough trials completed?
• Could another variable explain the pattern?
• Is the conclusion supported by the data?

Statistics help you think about evidence, not merely calculate numbers.

---

Comparing Groups

Imagine an experiment comparing two growing conditions.

Condition A

12, 13, 12, 14, 13 cm

Mean = 12.8 cm

Condition B

18, 19, 17, 20, 18 cm

Mean = 18.4 cm

The difference between the means is:

18.4 − 12.8 = 5.6 cm

The individual measurements also show relatively little overlap.

This provides stronger evidence of a difference than if the groups were:

Condition A

10, 13, 16, 18, 20 cm

Condition B

12, 14, 17, 19, 21 cm

The averages may differ, but the large amount of variation makes the distinction between the groups less clear.

This is why you should consider variation as well as averages.

---

Correlation Does Not Automatically Mean Causation

Suppose your data show:

As temperature increased, plant growth decreased.

There is a relationship between the two variables.

But before claiming that temperature caused the change, ask whether other variables were controlled.

Perhaps the warmer plants also received:

• less water
• different light
• different soil
• different fertilizer

A statistical relationship alone does not automatically prove causation.

A well-controlled experiment provides stronger evidence for a causal explanation.

---

More Trials Can Reveal the Real Pattern

Imagine testing a pendulum at one length.

Your first measurement is:

Period = 1.7 s

Your second is:

Period = 1.5 s

Which is correct?

You do not yet know.

After ten measurements, you find most values are between:

1.58 s and 1.64 s

Now you have much stronger evidence about the likely period.

Repeating experiments helps distinguish:

a single result

from

a consistent pattern.

When Results Are Highly Variable

Sometimes repeated results vary considerably.

Do not simply keep averaging without thinking about why.

Investigate possible causes:

• Is the measuring technique inconsistent?
• Are important variables uncontrolled?
• Is the instrument appropriate?
• Is there natural variation?
• Is the sample size too small?
• Is the experimental system unstable?
• Are there anomalous values?

Large variation is itself information.

It may reveal something important about either the experiment or the system being studied.

---

Reliability Does Not Guarantee Accuracy

Suppose a miscalibrated balance measures the same object five times:

52.1 g

52.0 g

52.1 g

52.0 g

52.1 g

The measurements are highly consistent.

But the true mass is:

50.0 g

The results are repeatable but inaccurate because of a systematic error.

This means:

reliable-looking data do not automatically guarantee accurate data.

Statistical analysis should be combined with evaluation of:

• equipment
• calibration
• method
• variable control

---

How Many Trials Are Enough?

There is no single correct number for every investigation.

The appropriate number depends on:

• natural variation
• measurement precision
• experimental method
• available time
• sample type
• research question

For many simple school investigations, at least three repeated trials at each condition can be a useful starting point.

But three is not a magical number.

If results vary greatly, more trials may be needed.

Biological investigations may require much larger samples because organisms naturally vary.

Ask:

Do I have enough data to identify a reasonably consistent pattern?

---

Example: Evaluating Experimental Data

A student investigates how ramp height affects toy-car travel time.

10 cm ramp

2.83 s
2.79 s
2.81 s

Mean = 2.81 s

Range = 2.83 − 2.79

Range = 0.04 s

30 cm ramp

1.94 s
2.88 s
1.97 s

One value appears unusual:

2.88 s

The student should investigate what happened during that trial.

Perhaps:

• the car was released incorrectly
• it hit the side of the ramp
• the stopwatch was started late

Rather than simply ignoring the value, the student could repeat the trial and document what happened.

This is statistical thinking in practice.

---

Example: Sample Size

A student investigates fertilizer and plant growth.

First experiment

1 plant without fertilizer
1 plant with fertilizer

The fertilized plant grows taller.

Evidence quality:

Weak

One plant in each condition cannot represent the natural variation between plants very well.

Improved experiment

20 plants without fertilizer
20 plants with fertilizer

Plants are randomly assigned and kept under similar conditions.

The student compares:

• mean height
• median height
• range
• graphs of the results

This provides much stronger evidence.

Common Mistakes

Using only one measurement

One result tells you very little about variation.

Assuming repeated measurements should be identical

Real measurements usually contain some variation.

Calculating a mean without looking at the data

An anomalous result can strongly affect the mean.

Deleting inconvenient data

Do not remove results simply because they disagree with your prediction.

Assuming a large sample fixes everything

More data cannot repair a badly controlled or systematically biased experiment.

Reporting only averages

Averages may hide important differences in variation.

Confusing sample size with repeated measurement

Measuring one organism many times is not the same as studying many organisms.

Assuming correlation proves causation

Other variables may explain the observed relationship.

Believing reliable results must be accurate

Systematic error can produce highly consistent but inaccurate measurements.

Using statistics without interpretation

A calculation is useful only when you understand what it tells you about the evidence.

---

Key Terms

Reliability — The consistency or dependability of experimental evidence.

Repeat measurement — An additional measurement made under the same or very similar conditions.

Reproduction — Repeating an investigation, often independently, to determine whether similar findings can be obtained.

Trial — One complete performance of an experimental condition.

Data set — A collection of related measurements or observations.

Variation — Differences between measurements or individuals.

Mean — The sum of all values divided by the number of values.

Median — The middle value when data are arranged in numerical order.

Mode — The most frequently occurring value.

Average — A general term describing a representative or central value; often used informally to mean the arithmetic mean.

Range — The difference between the highest and lowest values.

Percentage — A proportion expressed out of 100.

Percentage change — The change in a value expressed relative to its original value as a percentage.

Sample — A group of objects, organisms, measurements, or observations studied.

Sample size — The number of observations or individuals included in a sample.

Anomalous result — A result that differs noticeably from the general pattern.

Outlier — A data value substantially different from most other values.

Trend — A general direction or pattern in data.

Spread — The amount of variation within a data set.

Statistic — A numerical value calculated from data to help describe or analyze them.

Statistical thinking — Using data, variation, patterns, and appropriate calculations to evaluate evidence.

Correlation — A relationship in which two variables change together in some way.

Causation — A relationship in which a change in one factor directly produces a change in another.

Random variation — Unpredictable differences between repeated measurements or individuals.

Systematic error — An error that consistently shifts measurements in a particular direction.

---

Key Takeaways

• Scientists repeat experiments because individual measurements naturally vary.
• Repeated trials help reveal the consistency of experimental evidence.
• More measurements help distinguish isolated results from genuine patterns.
• The mean is calculated by adding all values and dividing by the number of values.
• The median is the middle value when the data are ordered.
• The mode is the most frequently occurring value.
• The range is the difference between the highest and lowest values.
• Averages describe the centre of data, while measures such as range provide information about spread.
• Two data sets can have the same mean but very different variation.
• Anomalous results should be investigated rather than automatically removed.
• Data should only be excluded when there is a scientifically justified reason.
• Larger sample sizes reduce the influence of unusual individuals and better represent natural variation.
• Sample size is particularly important in biological investigations.
• Measuring one individual repeatedly is not equivalent to studying many individuals.
• Percentages help compare proportions.
• Percentage change shows the size of a change relative to the starting value.
• Graphs can reveal trends, relationships, variation, and anomalies.
• Do not interpret an average without examining the underlying data.
• Large variation can reduce confidence in comparisons.
• A statistical relationship does not automatically prove causation.
• More data cannot correct a systematically biased experimental method.
• Reliable results are not necessarily accurate.
• Statistics should help you evaluate evidence, not simply produce calculations.

Good statistical thinking asks:

What is the typical result?

How much do the results vary?

Are there unusual values?

Is the sample large enough?

Is the pattern consistent?

How strong is the evidence for my conclusion?

---

Check Your Understanding

1. Why is a single measurement usually less reliable than repeated measurements?

2. Calculate the mean:

8.2, 8.5, 8.1, 8.4, 8.3

3. Find the median:

4, 9, 6, 7, 5

4. Find the mode:

3, 4, 4, 5, 6, 4, 7

5. Calculate the range:

12.4, 13.1, 12.8, 14.0, 12.6

6. Why does the range provide information that the mean does not?

7. Consider:

10.1 s, 10.3 s, 10.2 s, 18.7 s, 10.1 s

Which result appears anomalous?

8. Why should the anomalous result not automatically be deleted?

9. A student discovers that the anomalous trial occurred when the experimental object hit another object. What should the student do?

10. Calculate the percentage germination if 24 of 30 seeds germinate.

11. Plant height increases from 10 cm to 15 cm. Calculate the percentage increase.

12. Why would 20 plants generally provide stronger evidence about plant growth than one plant?

13. Explain why measuring one plant 20 times is not equivalent to measuring 20 plants.

14. Two groups both have a mean plant height of 15 cm. Group A has a range of 2 cm and Group B has a range of 14 cm. What does this tell you?

15. Why might a larger sample be particularly important in biological investigations?

16. Explain why repeating measurements cannot automatically correct a systematic error.

17. What is the difference between correlation and causation?

18. A student finds that plants closer to a window are taller. Give one reason why this observation alone does not prove that increased light caused the difference.

19. Explain why scientists should examine both the average and the spread of experimental data.

20. Challenge: Two students perform the same investigation. Student A performs three trials and obtains very similar results. Student B performs 20 trials and obtains much greater variation. Explain what you would need to examine before deciding which student's evidence provides the stronger basis for a conclusion.