Confidence Bounds panel
It is this panel that gives DiffTest its unique ability to deal flexibly and easily with similarity testing. It also makes it possible to test the significance of differences nondirectionally, where this is appropriate. Extended information about confidence bounds and their uses can be reached by clicking the appropriate button on the left. This section is mainly confined to relatively brief explanations of the interpretation of the Confidence bounds panel in the DiffTest calculator. Once data have been obtained from a sensory difference test, we can estimate the probability of a single trial being correct. If this value exceeds the probability of being correct if nothing but chance has produced the result (that is, if the assessors were not systematically perceiving a difference) it constitutes evidence  though not necessarily persuasive evidence  that some difference was being perceived. The best estimate of the probability of a single trial being correct is the same as the obtained proportion of correct results. However, there is uncertainty about this estimate and the true probability may be either higher or lower than the estimate. Confidence bounds, constituting the two ends of a confidence interval, give a range of values around the best estimate within which the true answer may reasonably be expected to lie. The width of this interval depends on the amount of data  other things being equal, the more data the narrower the bounds.
Probability of a correct trialWindows in this row refer to estimates of the probability of a trial being correct. They show the best estimate surrounded by its current lower and upper bounds.
Discrimination indexWindows in this row refer to estimates of the Discrimination index. They show the best estimate surrounded by its current lower and upper bounds. The Discrimination index is supposed to indicate the proportion of trials that are correct for some reason other than chance but this is just a metaphor to simplify description of results. More detail about it is given in the glossary.
Best estimateThe Best estimate of the probability of a trial being correct is numerically equal to the proportion of correct trials but this estimate is always somewhat uncertain. For most purposes, it is sufficient to think of the true probability of correct trials lying between the lower bound and the upper bound with the specified level of confidence (95% confidence if the value entered in Set the bounds is 0.025).
Lower 95% boundThe best estimate is always somewhat uncertain but the lower bound puts a limit to the smallest values we consider plausible alternatives to this estimate. DiffTest calculates the lower bound as a number just far enough below the best estimate that if that were actually the true probability of a trial being correct, the observed frequencies would differ from it by an amount that was only just significant In the example illustrated here, the directional significance level is 0.025 (2.5%) since this was the number shown in the Set the bounds window. This produces the mostoften used confidence bounds of 95%. However, had it been, say, 0.005, the confidence bounds would have been labelled 99% instead. See the separate section (buttons on the left) on Confidence bounds for information about using the bounds to test the nondirectional significance of differences.
Upper 95% boundThe best estimate is always somewhat uncertain but the upper bound puts a limit to the largest values we consider plausible alternatives to this estimate. DiffTest calculates the upper bound as a number just far enough above the best estimate that if that were actually the true probability of a trial being correct, the observed frequencies would differ from it by an amount that was only just significant In the example illustrated here, the directional significance level is 0.025 (2.5%) since this was the number shown in the Set the bounds window. This produces the mostoften used confidence bounds of 95%. However, had it been, say, 0.005, the confidence bounds would have been labelled 99% instead. See the separate section (buttons on the left) on Confidence bounds for information about using the bounds to test the nondirectional significance of differences.
