Home
Question: How do we know when we can truly believe a measurement?
Short answer: We can never have 100% confidence in a measurement.
Longer answer:
BUT ...
Regular traceable calibration is a method for gaining quantifiable confidence in a measurement system. In this topic we discuss some fundamental aspects of calibration, why it is necessary at all, and why it is so important that a special standard is required to govern the calibration process.
Consider the measurement of the gauge pressure of air in a ventilation duct. In this hypothetical example we happen to know in advance that the pressure is typically a constant 4 kPa and should lie in the range 2 - 4.5 kPa. To prevent irreversible damage to the duct the manufacturer specifies pressure may not exceed 5 kPa. In this example we use two different instruments for the measurement - a water-filled U-tube and an electronic manometer with digital display.
gh where is the density of
water (
= 997.77 kg . m-2 at
22oC, Australian
Standard AS 2849), and g = 9.81 m . s-2 is the nominal
acceleration due to gravity. Our calculated gauge pressure is p = 4.92 kPa. Without doing any formal uncertainty analysis we
intuitively believe that the true pressure is likely to be within
about 1 mm of water or approximately 0.01 kPa of our measurement.
therefore we feel confident that the pressure does not exceed the
safe upper limit.
The U-tube manometer uses fundamental physical principles in its operation. Unless there is an unseen fault (such as a transparent blockage) the only mechanism that can hold the two menisci at different levels is a pressure difference. There are a limited number of modes of failure and an experienced technician who understands a little of the physics of fluids can easily verify the absence of faults with a high level of confidence and gain an intuitive, qualitative feel for the uncertainty in the measurement. Instruments such as the fluid manometer based on fundamental principles can be used by experienced technicians with confidence in many applications without reference to a second instrument or standard. The trend, however, is away from this type of instrument which is cumbersome and usually requires a skilled operator, towards instruments that are more compact, portable and simple to operate.
If we require greater confidence in the U-tube measurement and the associated uncertainty, or an uncertainty substantially lower than (0.01 kPa), then it is possible to analyse and in some cases correct quantitatively the potential systematic errors that might be caused by surface tension effects at the menisci, the angle of the U-tube, the millimetre markings on the glass of the tube, parallax errors, variations in the local value of g, etc.
The electronic manometer indicates a pressure that might be very close to the upper safe limit. This example encourages us to think about the confidence we have in our manometer and in its specified tolerance bands. At what level of confidence can we say that the pressure does not exceed the safe upper limit?
The number of modes in which the pressure transducer, the electronic circuitry and the digital display can fail or malfunction is large. Most of the faults and malfunctions would not be visible to an operator therefore it is impossible to verify the absence of faults and electronic drift by simple inspection. We cannot tell by inspection if the instrument has recently been dropped, subjected to an over-range pressure or otherwise mistreated. When we make a measurement in the field we are forced to trust the instrument. The only way we can gain confidence in the electronic manometer is by regularly comparing it's response with another similar or preferably superior instrument in which we have a high level of confidence. A quantitative comparison or verification of the performance of an instrument is called a calibration.
To calibrate our electronic manometer we could borrow, purchase or hire a similar or superior manometer and a pressure source, and perform a comparison of the two manometers over the pressure range of interest. Our recent experience in measuring a pressure that appears to be very close to a safety limit motivates us to attempt to estimate at each calibration point the range within which the true pressure is likely to lie. This estimate is called an uncertainty estimate. In this case quantitative analysis of the uncertainty associated with the comparison, however, is immediately frustrated by our lack of confidence in the output of the manometer we are using as a reference, and and the unknown uncertainty associated with that instrument. If we attempt a more complete uncertainty analysis we may come across other factors that are not controlled or monitored during the comparison, e.g. fluctuations in the source pressure, environmental temperature, humidity and barometric pressure. If we are honest with ourselves we soon appreciate that to truly gain confidence in our electronic manometer we require at least the following conditions:
VIM 3 defines calibration as:
a set of operations that establish, under specified conditions, the relationship between values of quantities indicated by a measuring instrument or measuring system, or values represented by a material measure or a reference material, and the corresponding values realised by standards.
A product of a formal calibration is usually a calibration report including a table containing a set of reference values in which the calibration lab has a high level of confidence, and the corresponding values indicated by the device under test (in this case our electronic manometer). To confirm that our manometer delivers measurements within its specified tolerance bands, at least at the time of the calibration, we need to very that the true pressure is unlikely to lie outside the tolerance bands at each calibration point. This verification process requires quantitative estimates of the uncertainty associated with the comparison and the reference instrument.
To calibrate our manometer in a manner that fulfills our requirements as enumerated above we have two options.
Options (1) and (2) above are feasible under limited circumstances. Maintaining our own dedicated calibration lab, however, is time-consuming and costly. We also soon discover that if we calibrate our instruments ourselves that our customers start auditing us to verify that we are competent, doing the job properly and keeping proper records etc. We are likely to find that managing audits of our labs and regularly auditing other laboratories we use is time consuming and costly.
After a little honest thought we come to the conclusion that we (and probably many other organisations who regularly make measurements in which a high level of confidence is required) would benefit from a national or international system which would give us confidence in calibration laboratory services. A system that provides confidence intervals around our critical measurements would be extremely valuable.
ISO 17025 is an international standard governing most of the important aspects of calibration processes. Laboratories who meet this standard should operate a quality control system, be technically competent and be capable of producing technically valid results. The intention of ISO 17025 is to provide a functional system or hierarchy of calibration laboratories in which we can have confidence. Any calibration performed by an ISO 17025 accredited lab should:
ISO 17025 maximises confidence in reference instrument and materials by requiring that they are traceable to SI (System Internationale) units defined by international agreements at the BIPM[8] in Paris.
Mutual recognition agreements (MRA's) between accrediting authorities in different countries extend the hierarchy of trusted laboratories to a world-wide pyramid-shaped structure which has BIPM[8] in Paris at the apex. Traceability to SI units also ensures that measurements that we make in Sydney, Australia can be compared with similar traceable measurements made in many other countries. For example, if the ventilation duct in the example in section 1.2 above is made in Holland then we can have confidence that the Dutch kPa is the same as an Australian kPa.
Once our manometer bas been calibrated how long can we trust its performance? The manometer is exposed to vibration, varying temperatures, humidity etc during storage and transport. After the initial calibration (which in some cases is performed by the manufacturer) have no information concerning its drift and response to normal handling. Only after the second and (preferably) subsequent calibrations do we have information from which we can deduce whether or not the performance of the instrument between calibrations is adequate.
Calibration interval is an aspect of calibration that can be critically important to the validity of measurements and confidence intervals, but is highly instrument-specific and hence is not covered by a general standard like ISO 17025. Many manufacturers recommend calibration intervals (often one year) for their instruments. In practice, however, the user should determine the calibration interval based on analyses of successive calibration reports, the costs of calibration, the manner in which the instrument is stored and treated during normal use, and the consequences of out-of specification measurements.
Making a measurement is simple. Anyone can do it. We have all done it. Making measurements in which we have a quantifiable level of confidence, however, is not a trivial task. Achievement of a measurement that can be compared with confidence with other measurements, possibly made in a different country, is even more difficult. Confidence and trust are critical in serious measurements. While it is feasible for small groups of individuals or organisations to audit each other, the development of mutual trust and confidence among larger groups of organisations and between nations is not feasible without some type of standard to which everyone agrees. To facilitate measurement comparisons between organisations in different countries this standard has to be international.
If a standard is to govern a world-wide activity successfully it should be unique, a genuine industry standard. Therefore there can be no alternative standards for calibration laboratories or users of calibrated measurement systems. A calibration is either performed by an ISO 17025 accredited laboratory and hence has documented confidence intervals and is traceable, or it is not. If the system is to work there can be no grey areas. If you don't like ISO 17025 your only recourse is to participate in the system and change it from within.
The fluid-filled manometer can be thought of as an instrument that
realises a pressure unit based on a fundamental physical law
(p = gh) and constants (, g). Instruments based on
fundamental physical principles, such as the fluid-filled
manometer, can deliver performance acceptable for many
applications in skilled hands under controlled conditions. With a
few exceptions these instruments tend to be bulky, costly and/or
difficult to operate and the modern trend is towards instruments
that are more compact and easier to use. To improve or verify
confidence in instruments such as the fluid-filled manometer, or
to fulfill contemporary ISO 17025 traceability requirements, these
instruments are often formally calibrated.
It is of interest to note that at present all the base SI units with the exception of mass are defined in terms of fundamental physical constants and hence can be reproduced in any laboratory by skilled technicians with the appropriate equipment. Many national measurement laboratories reproduce a number of the base units using these instruments and compare their realisations with the BIPM[8] or other national labs.
No standard can guarantee that a calibrated instrument performs within specified limits or according to the calibration certificate. Immediately after a calibration in an ISO 17025 accredited lab an instrument should perform within defined tolerances with a specified probability. To maintain that performance until the next calibration it is the user's responsibility to ensure that the instrument is not mishandled, subjected to environmental extremes, and to select an appropriate calibration interval.
The original question: How do we know when we can truly believe a measurement result?
Answer:
If we wish to make a measurement and estimate a range of values within which the true value is likely to lie with a quantifiable level of confidence, then our instrument has to be calibrated regularly by an ISO 17025 accredited laboratory. In our example in section 1.2 a traceable calibration would either confirm that our electronic manometer operates within specifications, or include uncertainty estimates at each calibration point. Uncertainty estimates would enable us to estimate a range of values within which the true pressure lay, and hence facilitate the determination at a specified confidence level, of whether or not the pressure exceeded the upper safety limit.
©2002 Martin Turner B.Sc. (Eng) Ph.D.
Engineering and Measurement Consultant
12 Goodman Place, Cherrybrook, NSW 2126, Australia
Tel: 0403-007 305 (International: +61-403-007 305)
Email: mjturner at biccard.com
Disclaimer The views expressed and information provided in
these documents are the opinions of the authors and do not
represent specific advice on any topic.
First published: 23 Feb 03 Last modified: 23 Feb 03
Home