< Back DiffTest Page 2 of 8 Next >
Site highly rated by


Testing for Differences

Wine taster cartoon

Sensory analysis is the science of determining the attributes of products using the human senses, with the expert wine taster as a familiar example.


Though chemistry, physics or microbiology, say, can tell us about some ways that products differ, other important attributes such as pleasantness or the similarity of one odour to another, can be measured only by using human assessors. There are well developed methods for characterizing products in this way, but Difference Testing is a special branch of sensory analysis, addressing a particular kind of problem.

Difference testing

Difference testing is concerned with questions such as whether or not two batches of a product are noticeably different.  Probably no two batches are ever completely identical, given sufficiently sensitive analytical methods, but even real differences may not be noticed by consumers.  Here are some examples.

  • A company knows that the raw materials recently delivered for their food product are in some respects different from usual.  Is the consequent change in the finished product perceptible to consumers?
  • The Development Section have come up with a new recipe that has production advantages.  Do customers notice the difference?
  • After how much storage time has the product changed enough to be noticeably different?

Many types of difference test have been devised. Those in widespread use are described in detail in textbooks of sensory methods and in national and international standards.  All take the form of requiring someone called 'an assessor' (usually, a panel of several assessors) to carry out a task which can be performed perfectly by an assessor who detects the difference with certainty but will frequently result in errors if the assessor is unsure about the difference.

Results from difference testing are rarely clear-cut. They are usually statistical in nature, which means that answers always have some uncertainty. The aim of statistical analysis of difference tests is to obtain answers that are as precise as the data allow and whose uncertainty is quantified.

One tool for quantifying the uncertainty of a difference test is a test of statistical significance. This calculates a numerical probability, the significance level of the result. This is the probability of obtaining results showing at least as much detectability as was actually observed if, in truth, the difference is completely undetectable.  This probability can be calculated, since if the difference is totally undetectable, answers must be given at random and the probability of being correct on any one trial can be deduced from the nature of the test.