5 Uncertainty and proficiency testing

5.1 Commercial reality

Most calibration labs are commercial concerns and need to attract enough customers to cover costs and make a profit. If a lab can lower its `least uncertainty of measurement', it is in a position to increase the charge for its service. In addition, a lower uncertainty enables a lab to calibrate a wider range of instruments and hence it should be able to attract more customers.

5.2 ISO 17025 and NATA requirements for proficiency testing

All ISO 17025 accredited labs are required (section 5.9) to have quality control procedures for monitoring the validity of tests and calibrations undertaken. ... This monitoring ... may include ... participation in interlaboratory comparisons or proficiency-testing programs. Australian ISO 17025 accredited labs, however, are normally required to participate in all NATA proficiency testing programs[6] unless no suitable program is available.

5.3 Evaluation of proficiency tests

Calibration labs participating in a proficiency test are requested to calibrate an instrument or artefact that has been calibrated by a reference laboratory. NATA presently analyses each lab's results by calculating a normalised error E_n as follows[6]:

$${\frac{LAB - REF}{\sqrt{U^2_{\text{lab}}+ U^2_{\text{ref}}}}}$$ E_n = (1)

where:

LAB and U_ref are the participating laboratory's result and uncertainty respectively.

REF and U_ref are the reference laboratory's result and uncertainty respectively.

5.4 Discussion

If both reference and participating labs produce measurements with zero mean bias (no systematic errors) then we expect that on average E_n = 0. Typically U_lab and U_ref are based on a 95% level of confidence, hence the range - $\sqrt{U^2_{\text{lab}}+ U^2_{\text{ref}}}$ to + $\sqrt{U^2_{\text{lab}}+ U^2_{\text{ref}}}$ is an estimate of the 95% confidence interval for LAB - REF, taking both labs' uncertainties into account. If the observed deviation of E_n from zero is truly random then E_n should lie in the interval -1 to +1 in 95% of cases. In proficiency tests E_n values that lie outside this interval are considered unsatisfactory and labs are asked to investigate and explain their result.

The participating lab's uncertainty estimate appears in the denominator of equation 1. Therefore, all other things being equal, a lab with a large uncertainty (U_lab) is less likely to be asked to explain a large E_n than a lab with a small uncertainty.

Calibration laboratories are therefore faced with conflicting motives when they develop procedures for estimating uncertainties. On the one hand low uncertainty estimates may increase revenue. On the other hand high uncertainties decrease the probability of failing a proficiency test. During participation in proficiency tests calibration labs can expect to be asked to calibrate an instrument or artefact with a specification that is close to its best measurement capability, and hence contributes little to the overall uncertainty of the result. In other words, usually U_ref $\ll$ U_lab.

5.5 Conclusions

Proficiency testing can be viewed as an evaluation of both calibration ability and uncertainty estimates. As far as we are aware proficiency tests are the only occasions when uncertainty estimates are evaluated quantitatively.

Calibration labs should take care to develop procedures for estimating unbiased, realistic uncertainty values for their calibration processes.

©2002 Martin Turner B.Sc. (Eng) Ph.D.
Engineering and Measurement Consultant
12 Goodman Place, Cherrybrook, NSW 2126, Australia
Tel: 0403-007 305 (International: +61-403-007 305)
Email: mjturner at biccard.com
Disclaimer The views expressed and information provided in these documents are the opinions of the authors and do not represent specific advice on any topic.

First published: 8 Sept 02 Last modified: 2 Oct 02