Measurement Uncertainty Can be visualised

Internal quality control (IQC) charts plotted using data collected for a month or more illustrates measurement uncertainty (Figure 1). If the IQC material for a given measurand is stable, correctly stored and prepared for the measurement procedure, laboratories can assume that the concentration of the measurand will not change. However, even rapid repeat measurements of an IQC sample will produce different values for the measurand. Medical laboratories can also assume that multiple repeat measurements of the same patient sample will produce a similar pattern of different values.

This pattern of measurement results is termed Random Variation, which means the next measurement result cannot be predicted from the previous result (Figure 1).

Figure 1: Plasma Glucose Quality Control results over time

Causes of Measurement Uncertainty

Major potential contributors are:

• Instrument sampling
• Sample dilution
• Sample inhomogeneity
• Reconstitution procedures for lyophilised materials
• Uncertainty of calibrator values
• Reagent and calibrator instability
• Reagent and calibrator lot-to-lot variability
• Re-calibrations
• Reagent dispensing
• Differences between reagent batches
• Electro/mechanical fluctuations of measuring devices
• Performance changes in measuring devices following routine maintenance
• Differences between operators of measuring devices
• Fluctuations in laboratory environment
• The algorithms used by measuring devices for rounding raw results to reported results

For certain types of manual measurement methods, changes of operator can have a significant impact on random variation (Figure 2).

It is important to be aware that no matter how sophisticated the measuring device, results produced by all types of measurement has uncertainty. This means all measurement results are estimates of the true value of the measurand. Therefore there is a need to ensure measurement results are meaningful to the user. Making measurements are essential to everyday life in households, shops, industry, health services, science and research. It is important to ensure measurement results are sufficiently accurate for the purpose to which they will be applied. This is particularly critical for removing costly roadblocks to international trade. For example, a stated weight of wheat shipped to another country can be trusted by the receiver and not require re-weighing, or a patient’s stated serum glucose concentration is meaningful and trusted by global health services.

To assist this, the science of measurement (Metrology) was developed. Also essential has been the standardisation of measurement units, known as the SI system. An international organisation, the International Bureau of Weights and Measures (BIPM) is responsible for developing and maintaining the SI system, the world clock, the metre, mass and other constants of nature. It is also responsible for developing and publishing Guides in Uncertainty in Measurement (GUM) and for developing a standard International Vocabulary for Metrology (VIM). The GUM and its Supplements are the primary references for estimating uncertainty in measurements.

Terminology in Metrology

It is useful to know some of the terms used in Metrology because they can be difficult to understand:

Accuracy closeness of agreement between a measured value and a true quantity value of a measurand

Indication numerical result produced by a measuring instrument

Measurand quantity intended to be measured

Quantity the property of a substance that has a magnitude that can be expressed as a number and measurement unit

Property attribute of a substance, for example colour, nucleotide sequence, length, mass, light emission wavelength

Metrological traceability property of a measurement result whereby it can be related to a primary reference through a chain of calibrations, each step contributing to the MU of the patient’s result

Measurement method generic description such that it cannot be used to perform a measurement, for example, spectrophotometry

Measurement procedure detailed description of the measurement procedure that can be used by an experienced individual to perform a reliable measurement

Measurement Uncertainty Disappears

This occurs if the measurement procedure is very insensitive. For example, if a weighing machine reports weights to the nearest 10 kg, it is unlikely to show measurement uncertainty with repeated measurements of the same weight, whereas if the weighing machine is more sensitive and reports to the nearest gram (g), it will show measurement uncertainty for repeated measurements of the same object.

Concept of Measurement Uncertainty in Medical Laboratories

The GUM approach to estimating MU is not suitable for use by medical laboratories because it requires using very high-level statistics and mathematics. However, some basic GUM principles can be used to develop an approach to estimating MU in medical laboratories:

• Definition of the quantity being measured (measurand)
• Recognition that a measured value is an estimate because of the effects of imprecision and bias
• Expressing measurement uncertainties as a standard deviation or relative standard deviation (CV)
• Systematic and random uncertainties are statistically treated in the same way
• An estimate of MU allows definition of a set of possible values for the measurand that is believed by the laboratory to include, with a stated probability, the true value of the measurand
• The measured value accompanied by its stated MU is considered to be the best estimate of the true value

Estimating Measurement Uncertainty in Medical Laboratories

There are two approaches to estimating MU. The first method is termed Bottom Up which is not recommended for medical laboratories. Top Down is the much preferred approach whereby measurement data is used in calculating MU estimates.

For routine quantitative measurements of patient samples using automated instrumentation, a single measurement of each analyte is usually made. To generate sufficient data to calculate measurement uncertainty, internal quality control (IQC) measurement results are used because over time the effect of many changes in operating conditions are recorded. Data from external quality assurance programmes are not recommended because they do not provide such comprehensive coverage of changing measurement conditions.

Definition of a Measurand

Requires at least three pieces of information:

System containing the analyte For example, venous whole blood, urine, red blood cells, renal stone

Identity of the analyte

For example, rubella antibody, digoxin, subunit of hCG, HIV-1 RNA, CCG trinucleotide

Quantity

For example, amount of substance concentration, number, mass concentration, number concentration, number fraction, amount of substance rate concentration

An example would be the number concentration of white blood cells in whole venous blood.

Biological analytes can be complex (isoforms, fragments) and/or poorly defined, and therefore definition of a measurand may additionally depend on the specific measurement procedure used. For example, the catalytic activity concentration of a plasma enzyme can be affected by changes in the temperature, pH and co-factors used in performing the measurement.

Another example is the different epitope selectivity of antibodies used by different commercial measurement procedures to measure the ‘same’ glycoprotein hormone, for example, different antibodlies may recognise different isoforms, or bind them to different extents. In such cases, identification of the measurement procedure must be included in the measurand definition.

For example:

• Enzyme X: catalytic activity concentration by IFCC reference measurement procedure
• Protein hormone Z: reagent kit manufacturer Y
• Tumour marker B: reagent kit manufacturer A

Although not part of the formal definition of a measurand, it is usual practice to identify the measurement unit.

What do Medical Laboratories Measure?

Measurands are rarely directly measured. For example, the serum concentration of total calcium is not routinely directly measured by counting the number of calcium atoms per litre of serum. Serum total calcium is routinely measured using a surrogate marker. For example, the measurement of the colour intensity produced when the serum sample reacts with a chromogen. The measurement result is calculated using the value obtained for the calcium calibrator and an algorithm in the instrument software. Another example is the change in electrical resistance when red blood cells pass through an electronic gate in a blood cell counter.

Uncertainties can be introduced by the defined measurand:

• Incomplete definition of the measurand
• May not be fully measured because of inadequate analytical specificity
•  Analytical interferences
• Analyte not fully available to measurement system, for example caused by protein binding

Measurement Uncertainty Targets

Before estimating the MU of an analyte it is important to set a target value that is clinically acceptable for making good decisions for patient care. International expert panels may already have set MU targets for some analytes, for example plasma cholesterol. Setting other targets will require discussion with local clinical experts or professional organisations, for example international sports bodies setting upper limits for banned drug use.

Data Required for Estimating Measurement Uncertainty

Calibrator and reference material values are assigned by making measurements, and therefore the calibrator reference values themselves have an uncertainty, which is stated in the reference material certificate. It should be noted that WHO biological standards are not metrologically traceable back to an SI measurement Unit, for example the Mole. WHO reference materials are purified, bioactivity checked and allocated an International Unit (IU). International Units are therefore arbitrary and cannot be compared with other WHO reference materials.

Reporting Patient Results

It is recommended not to report measurement uncertainties to clinicians and other healthcare professionals unless specifically requested. They may be requested from medical laboratories that are providing test results to pharmaceutical companies undertaking clinical trials of new drugs.

How to Perform Estimates of Measurement Uncertainty

For detailed practical guidance on how to perform estimations of MU, readers are recommended to refer to Reference 1. This reference provides worked examples of calculating MU estimates for a wide range of routine analytes, for example parathyroid hormone, Anion Gap, urine calcium/creatinine ratio, number concentration of white blood cells, INR, human immunodeficiency virus type 1 viral load, BCR-ABL gene transcript measurements, Rubella IgG antibody measurements, hepatitis B surface antigen measurements. It also addresses problems such as medical laboratories using multiple analysers across an organisation.MU estimates expressed as SDs cannot be added together, they must be expressed as variances, where SD2 = variance. This is very useful for laboratories that have multiple measuring devices where an average MU is required because a patient specimen could be measured on any of the devices. For example, (SD2 + SD2)/X = average variance for X instruments. Square root of the variance provides the average SD across all the devices.

Initially 30 or so IQC values would be adequate for a reasonable estimate of MU for a single measuring device. One SD is the parameter of MU (standard measurement uncertainty, symbol u). Since ± 1 SD would cover approximately 68 % of the dispersion of obtained QC values. This is of limited practical application, so the uncertainty is widened by applying a coverage factor (k) to provide an expanded measurement uncertainty (symbol U). If 2 is chosen for k, then coverage is a more useful approximately 95.5 % of the dispersion of possible results.

Expressing Measurement Uncertainty Estimates

It is recommended to express MU estimates as Expanded MU (2 x MU) which provides approximately 95.5 % confidence that the true value is included in the expression: result value ± 2 x MU.

Measurement Bias

Bias cannot be eliminated, but significant bias should be minimised using recalibration or by applying an adjustment factor to the raw results. The residual bias will be small relative to the uncertainty of the imprecision. A rule of thumb is that if an SD is <25 % of the largest MU, it can be ignored when combining SDs (u).

Rule of Thumb: If two results on the same patient differ by >3 x MU they are measurably different.

Biological Variation

An advantage of using measurement uncertainty is that important uncertainties that arise from non-technical sources can easily be included in the calculation of the estimate. For example, within-individual and within-group biological variations. Such data is freely available from the website of the European Federation of Clinical and Laboratory Medicine (EFLM) Biological Variation Database. The relevant CV must be expressed as a variance then added into the estimate calculation. Including biological variation is not always physiologically appropriate, for example hCG in pregnancy, urine sodium.

Laboratory Quality Records

It is recommended that laboratories retain their MU estimates and the method used to obtain them is retained in the laboratory quality records, including the required frequency of re-estimation. MU estimates should also be regularly re-estimated if technical steps are changed.

Laboratory Value of Estimating Measurement Uncertainty

• Quantitative expression of the reliability of the test result
• Demonstrates the results meet clinical requirements
• Use of internal quality control data for estimating uncertainties
• Does not require additional work to gather data
• Estimates can assist interpretations if results are close to clinical decision values
• Estimates can be used to define grey zones for interpretation
• No need to separately determine bias and imprecision as used in the Total Error Concept
• Ability to include non-technical uncertainties, for example biological variation
• Is essential for meaningful comparison of results with reference values, with previous results, with results from other health systems and clinical research
• Can provide insights as to which technical steps might be open to improvement, thereby reducing overall MU
• Is an essential component for achieving standardised and harmonized measurement results for which there is increasing global demand

1. Technical Specification ISO/TS 20914 First Edition 2019 -07 Medical laboratories – Practical guidance for the estimation of measurement uncertainty (ISO have copyright on all their published Standards and Technical Specifications which means they have to be purchased. Note: the cost should be lower than the ISO cost if the Standards Australia version AS/TS 20914 is purchased).

2. International Vocabulary of Metrology (VIM) Basics and general concepts and associated terms JCGM 200: 2012 (free from www.bipm.org).

3. White GH. Basics of estimating measurement uncertainty. Clin Biochem Rev 2008; 29:S53-S60.

4. CLSI C51-A. Expression of Measurement Uncertainty in Laboratory Medicine. Clinical and Laboratory Standards Institute, Wayne, PA. 2012. Kallner A, Boyd J, Duer DL, Giroud C, Hatjimihail A, Klee G, Lo S, Pennello G, Sogin D, Tholen D, Toman B, White G. Now classified as EP29-A. (Covered by Copyright, so requires purchase).