Publications

Anemia and hypoxemia are common clinical conditions that are difficult to study and may impact pulse oximeter performance. Utilizing an in vitro circulation system, we studied performance of three pulse oximeters during hypoxemia and severe anemia. Three oximeters including one benchtop, one handheld, and one fingertip device were selected to reflect a range of cost and device types. Human blood was diluted to generate four hematocrit levels (40%, 30%, 20%, and 10%). Oxygen and nitrogen were bubbled through the blood to generate a range of oxygen saturations (OHb) and the blood was cycled through the in vitro circulation system. Pulse oximeter saturations (SpO) were paired with simultaneously-measured OHb readings from a reference CO-oximeter. Data for each hematocrit level and each device were least-squares fit to a 2nd-order equation with quality of each curve fit evaluated using standard error of the estimate. Bias and average root mean square error were calculated after correcting for the calibration difference between human and in vitro circulation system calibration. The benchtop oximeter maintained good accuracy at all but the most extreme level of anemia. The handheld device was not as accurate as the benchtop, and inaccuracies increased at lower hematocrit levels. The fingertip device was the least accurate of the three oximeters. Pulse oximeter performance is impacted by severe anemia in vitro. The use of in vitro calibration systems may play an important role in augmenting in vivo performance studies evaluating pulse oximeter performance in challenging conditions.

It has long been known that many pulse oximeters function less accurately in patients with darker skin. Reasons for this observation are incompletely characterized and potentially enabled by limitations in existing regulatory oversight. Based on decades of experience and unpublished data, we believe it is feasible to fully characterize, in the public domain, the factors that contribute to missing clinically important hypoxemia in patients with darkly pigmented skin. Here we propose 5 priority areas of inquiry for the research community and actionable changes to current regulations that will help improve oximeter accuracy. We propose that leading regulatory agencies should immediately modify standards for measuring accuracy and precision of oximeter performance, analyzing and reporting performance outliers, diversifying study subject pools, thoughtfully defining skin pigmentation, reporting data transparently, and accounting for performance during low-perfusion states. These changes will help reduce bias in pulse oximeter performance and improve access to safe oximeters.

Background

Universal access to pulse oximetry worldwide is often limited by cost and has substantial public health consequences. Low-cost pulse oximeters have become increasingly available with limited regulatory agency oversight. The accuracy of these devices often has not been validated, raising questions about performance.

Methods

The accuracy of 6 low-cost finger pulse oximeters during stable arterial oxygen saturations (SaO2) between 70% and 100% was evaluated in 22 healthy subjects. Oximeters tested were the Contec CMS50DL, Beijing Choice C20, Beijing Choice MD300C23, Starhealth SH-A3, Jumper FPD-500A, and Atlantean SB100 II. Inspired oxygen, nitrogen, and carbon dioxide partial pressures were monitored and adjusted via a partial rebreathing circuit to achieve 10 to 12 stable target SaO2 plateaus between 70% and 100% and PaCO2 values of 35 to 45 mm Hg. Comparisons of pulse oximeter readings (SpO2) with arterial SaO2 (by Radiometer ABL90 and OSM3) were used to calculate bias (SpO2 – SaO2) mean, precision (SD of the bias), and root mean square error (ARMS).

Results

Pulse oximeter readings corresponding to 536 blood samples were analyzed. Four of the 6 oximeters tested showed large errors (up to -6.30% mean bias, precision 4.30%, 7.53 ARMS) in estimating saturation when SaO2 was reduced <80%, and half of the oximeters demonstrated large errors when estimating saturations between 80% and 90%. Two of the pulse oximeters tested (Contec CMS50DL and Beijing Choice C20) demonstrated ARMS of <3% at SaO2 between 70% and 100%, thereby meeting International Organization for Standardization (ISO) criteria for accuracy.

Conclusions

Many low-cost pulse oximeters sold to consumers demonstrate highly inaccurate readings. Unexpectedly, the accuracy of some low-cost pulse oximeters tested here performed similarly to more expensive, ISO-cleared units when measuring hypoxia in healthy subjects. None of those tested here met World Federation of Societies of Anaesthesiologists standards, and the ideal testing conditions do not necessarily translate these findings to the clinical setting. Nonetheless, further development of accurate, low-cost oximeters for use in clinical practice is feasible and, if pursued, could improve access to safe care, especially in low-income countries.

Introduction

Pulse oximetry may overestimate arterial oxyhemoglobin saturation (Sao2) at low Sao2 levels in individuals with darkly pigmented skin, but other factors, such as gender and oximeter probe type, remain less studied.

Methods

We studied the relationship between skin pigment and oximeter accuracy in 36 subjects (19 males, 17 females) of a range of skin tones. Clip-on type sensors and adhesive/disposable finger probes for the Masimo Radical, Nellcor N-595, and Nonin 9700 were studied. Semisupine subjects breathed air-nitrogen-CO2 mixtures via a mouthpiece to rapidly achieve 2- to 3-min stable plateaus of Sao2. Comparisons of Sao2 measured by pulse oximetry (Spo2) with Sao2 (by Radiometer OSM-3) were used in a multivariate model to assess the source of errors.

Results

The mean bias (Spo2 – Sao2) for the 70%-80% saturation range was 2.61% for the Masimo Radical with clip-on sensor, -1.58% for the Radical with disposable sensor, 2.59% for the Nellcor clip, 3.6% for the Nellcor disposable, -0.60% for the Nonin clip, and 2.43% for the Nonin disposable. Dark skin increased bias at low Sao2; greater bias was seen with adhesive/disposable sensors than with the clip-on types. Up to 10% differences in saturation estimates were found among different instruments in dark-skinned subjects at low Sao2.

Conclusions

Multivariate analysis indicated that Sao2 level, sensor type, skin color, and gender were predictive of errors in Spo2 estimates at low Sao2 levels. The data suggest that clinically important bias should be considered when monitoring patients with saturations below 80%, especially those with darkly pigmented skin; but further study is needed to confirm these observations in the relevant populations.

Background

It is uncertain whether skin pigmentation affects pulse oximeter accuracy at low HbO2 saturation.

Methods

The accuracy of finger pulse oximeters during stable, plateau levels of arterial oxygen saturation (Sao2) between 60 and 100% were evaluated in 11 subjects with darkly pigmented skin and in 10 with light skin pigmentation. Oximeters tested were the Nellcor N-595 with the OxiMax-A probe (Nellcor Inc., Pleasanton, CA), the Novametrix 513 (Novametrix Inc., Wallingford, CT), and the Nonin Onyx (Nonin Inc., Plymouth, MN). Semisupine subjects breathed air-nitrogen-carbon dioxide mixtures through a mouthpiece. A computer used end-tidal oxygen and carbon dioxide concentrations determined by mass spectrometry to estimate breath-by-breath Sao2, from which an operator adjusted inspired gas to rapidly achieve 2- to 3-min stable plateaus of desaturation. Comparisons of oxygen saturation measured by pulse oximetry (Spo2) with Sao2 (by Radiometer OSM3) were used in a multivariate model to determine the interrelation between saturation, skin pigmentation, and oximeter bias (Spo2 – Sao2).

Results

At 60-70% Sao2, Spo2 (mean of three oximeters) overestimated Sao2 (bias +/- SD) by 3.56 +/- 2.45% (n = 29) in darkly pigmented subjects, compared with 0.37 +/- 3.20% (n = 58) in lightly pigmented subjects (P < 0.0001). The SD of bias was not greater with dark than light skin. The dark-light skin differences at 60-70% Sao2 were 2.35% (Nonin), 3.38% (Novametrix), and 4.30% (Nellcor). Skin pigment-related differences were significant with Nonin below 70% Sao2, with Novametrix below 90%, and with Nellcor at all ranges. Pigment-related bias increased approximately in proportion to desaturation.

Conclusions

The three tested pulse oximeters overestimated arterial oxygen saturation during hypoxia in dark-skinned individuals.