The Australian Wine Research Institute > Services to Industry > Winemaking > Laboratory establishment and methods > Quality of data

Quality of data

Basic concepts

Scope

In discussing results obtained from laboratory analysis, there are several important concepts relating to the quality of the results that are often confused with one another. These concepts of accuracy, precision, error and uncertainty are defined and the differences between them described. Furthermore, some basic approaches to quality control are presented in order to provide assistance in ensuring systematic approaches to achieving better data from your laboratory.

Accuracy

Accuracy refers to the agreement between a measurement and the true or accepted value. Accuracy alone might or might not say something about the quality of the measuring instrument. The instrument might be of high quality and still disagree with the true value.

Examples:

  • If you used a balance to find the mass of a known standard 100.00 g, and you obtained a reading of 80.25 g, your measurement would not be very accurate
  • A stopped clock is accurate at least once each day.

An important distinction between accuracy and precision is that accuracy can be determined by only one measurement, while precision can only be determined with multiple measurements.

Precision

Precision refers to the repeatability of measurement and describes how close together a group of measurements actually are to each other. Precision has little to do with the true or accepted value of a measurement, so it is quite possible to be very precise and totally inaccurate.

Example:
If each day for several years a clock reads exactly 11:47 am when the sun is at the zenith, this clock is very precise because it is giving a highly repeatable result. Since there are more than thirty million seconds in a year this device is more precise than one part in one million.

A dartboard analogy is often used to help explain the difference between accuracy and precision. Imagine a person throwing darts, trying to hit the bullseye. The closer the dart hits to the bullseye, the more accurate the throws are. If the person misses the dartboard with every throw, but all of their shots land close together, they are still very precise.

(source: University of Adelaide, School of Chemistry)

In many cases, when precision is high and accuracy is low, the fault can lie with the instrument. If a balance or a thermometer is not working correctly, they might consistently give inaccurate answers, resulting in high precision and low accuracy.

Error

Error refers to the disagreement between a measurement and the true or accepted value. A source of error is a limitation of a procedure or an instrument that causes an inaccuracy in the quantitative results of an experiment. A ‘human error’ is not considered a source of error under this definition. Laboratories should strive to identify, understand and limit sources of error in their procedures whenever possible. Some examples include, reading accuracy on burettes and other volumetric dispensing apparatus, colour change interpretation on titrations end points, variation due to sampling, reagents out of date and deteriorated, and digital display output on, for example, balances.

Sampling errors can be a very common and large source of error. This can be because the material being sampled might not be completely homogeneous and, therefore, each sample being drawn might be very slightly different. In some cases, mislabelling or even drawing samples from the wrong batch can lead to serious misinterpretations. It is extremely important to remember that the test results can only be useful if the samples truly represent the material.

As with accuracy, you must know the true or correct value to discuss your error. However we do not know what the true value is ahead of time, and it is not possible to determine the error. However, if an error occurs we simply will not know it. The true value has not yet been established and there is no other guide.

It is worth remembering that ‘human error’ is rarely the sole source of measurement problems. Many problems producing bad laboratory results are due to analysis or calculation errors subsequent to the measurement! Look there first.

Uncertainty

It is important to remember that there is no such thing as a perfect measurement. Each measurement contains a degree of uncertainty due to the limitations of instruments and the people using them. Uncertainty, rather than error, is the important term in expressing results of chemical or any other type of measurement.

Uncertainty of a measured value is an interval of confidence around that value such that the measured value is certain not to lie outside this stated interval and any repetition of the measurement will produce a new result that lies within this interval. This uncertainty interval is assigned for a specific test procedure by following established principles of estimation of uncertainty. The estimation of uncertainty in itself can be a complex task and several publications exist to assist in the determination (EURACHEM/CITAC 2000, ISO 1993, NIST 1994).

Uncertainties should also be stated along with a probability. Usually, the measured value is stated to lie within a defined confidence interval with a corresponding probability. Commonly, a 95% confidence interval is used which means an interval of twice the standard deviation (i.e. the standard deviation [sd] for the results from a series of tests made on the same sample). This means that if the measurement is repeated, 95% of the time the new measurement will fall in this interval (i.e. the confidence interval).

Therefore, the format for expressing results is “value plus or minus uncertainty (95% confidence)”.

Examples:

  • the width of a standard piece of A4 paper is measured using a ruler and the result is stated as as 210 ± 0.5 mm. By stating the uncertainty to be 0.5 mm you are claiming with confidence that every reasonable measurement of this piece of paper by other experimenters will produce a value not less than 209.5 mm and not greater than 210.5 mm.
  • the concentration of free sulfur dioxide in a wine sample is measured and reported as 25 ± 3 mg/L (95% confidence), the interval of 3 mg/L having been determined as that for 95% confidence.

Therefore it is always possible to present results in a completely certain format. In the worst case, where uncertainty is large relative to the result value, the measurement might be nearly useless whilst still being completely certain! For example, if the distance to an arrow target board is estimated as 100 metres with an uncertainty of 50 metres, the statement is very certain but might be of little value to the archer. Don’t let this put you off laboratory testing – just keep in mind the concept of uncertainty and be aware of it when interpreting your test results. Always keep in mind the context in which the results are being used and ensure that the results are suitable for the intended use.

Therefore, good analysts will endeavour to develop procedures with the confidence intervals (the uncertainty) to be as small as possible. Every measurement should be considered along with a confidence interval which should then be assigned to the measurement as the uncertainty.

Quality control

The aim of quality control is to ensure that the results generated is of a quality that has been defined as acceptable. Quality control can be implemented in various ways and can include the following:

  • Testing of a known standard as a check on accuracy (made up or certified reference material)
  • Regular testing of a control sample (e.g. a cask wine previously analysed) to check on the equipment and the analytical techniques. It is an extremely useful and insightful practice to plot the results of such control samples on a chart as this can give a rapid indication if any problems are occurring.
  • Include replicates and compare with pre-determined acceptance/rejection criteria for repeatability (precision).
  • Interlaboratory proficiency testing – analyse the same samples at several laboratories (e.g. The Interwinery Analysis Group Inc.) and compare the group results using the same testing procedures. This can help to identify systematic errors and deficiencies in analytical procedures.
  • Internal auditing – present replicate samples ‘blind’ and compare the results against the precision criteria.

When using any of the quality control steps above, it is important that criteria are set in order to determine if the results are of acceptable quality. If the results of the quality control tests fail to meet the criteria, then there must be clear and unambiguous corrective action procedures in place to identify the cause of the failure and prevent recurrence. Such actions for identification can include trace back of records, or resubmitting test samples under ‘blind’ conditions to one or several operators. Corrective actions arising to address the causes can include calibration of the instrumentation, training of the operators and repairs and maintenance of instrumentation or equipment. By documenting these procedures to be undertaken you will be establishing the foundation of a quality management system for your laboratory. Once established, it will then serve well as the basis for a fully documented quality management system that could be certified, at some time in the future, to international standards such as ISO 17025 (Standards Australia 1999).

Web sites of interest

The World Wide Web is a very useful source of information and there are many sites that can provide relevant information. Using any one of the many available search engines is a good way to start.

Suggested key words to use in a search could include, amongst others: laboratory, accuracy, precision, chemistry, wine, analysis

References and further reading

  • EURACHEM/CITAC QUAM:2000.P1 Quantifying Uncertainty in Analytical Measurement. Editors Ellison, S.L.R.; Rosslein M.; Williams A. EURACHEM/CITAC Guide; 2000.
  • International Standards Organisation. Guide to the expression of uncertainty in measurement. ISO, Geneva; 1993 (ISBN 92-67-10188-9).
  • Standards Australia AS ISO/IEC 17025-1999 General requirements for the competence of testing and calibration laboratories. Standards Australia International Ltd PO Box 1055, Strathfield, NSW 2135, Australia.
  • NIST Technical Note 1297 Guidelines for evaluating and expressing the uncertainty of NIST measurement results. Taylor; B.N.; Kuyatt, C.E. National Institute of Standards and Technology, Gaithersburg, MD 20899-0001, 1994.