What is the root mean square error or deviation?

The Average Root Mean Square error or root mean square deviation (known as ARMS) is an established measure of SpO2 performance and is used by regulatory agencies to determine how accurately a pulse oximeter performs. Sometimes referred to as “Accuracy Root Mean Square error,  ARMS is the square root of the mean of the squared deviations between the pulse oximeter SpO2 measurement and the SaO2 measurement.  ARMS approximates the mean absolute deviation (MAD) between SpO2 and SaO2. It is always a positive value, so it cannot indicate whether SpO2 is negatively or positively biased for SaO2. 

ARMS  is calculated by combining both the bias and the standard deviation into one value. The FDA recommends an acceptable limit of ARMS ≤3% for transmittance devices and ARMS ≤3.5% for reflectance devices. If either component (bias or SD) is very large (i.e. inaccurate), the ARMS will be greater than the acceptable limit and will not be considered a well performing pulse oximeter.  ​​If a pulse oximeter device has an ARMS ≤3%, this means that it did not have a large scatter or large outliers in the data, and therefore can accurately measure SpO2. It is important to recognize that even a small difference in ARMS between devices can reflect a large difference in their performance. 

ARMS  is best referred to as a measure of performance (rather than accuracy).

References: FDA Guidelines; Clinical Application of ARMS in Pulse Oximetry – Clinimark June 2021; Brandmaier et al, Elife 2018

Keywords: bias, standard deviation, Arms, ARMS