Can pulse oximeter performance be validated by an in vitro device?

The short answer is no. 

There are several devices that exist to ‘simulate’ or ‘test’ pulse oximeter function including the Fluke ProSim SPOT, Fluke ProSim8 and Whaleteq AECG100. 

These devices work by detecting LED pulses from an oximeter, then fabricating their own output signal to the oximeter sensor. The Fluke preserves any noise in the LED light signal while the others simply trigger from it and in doing so eliminate all potential errors due to LED noise (a very common problem with implications for oximeter performance). Some of these devices also have a limited range of simulated conditions (ie perfusion) and require prior calibration to the specific oximeter being tested. Thus, for oximeters already calibrated into the in vitro simulator, the simulator can help confirm that a device is working (or at least is sensing signal) as it was designed to do in the factory. For devices not previously calibrated, conclusions about performance are uncertain. 

Multiple studies report to have used these devices to ‘validate’ oximeter performance, yet performance of an uncalibrated oximeter on an in vitro simulator does not ensure good oximeter performance in reality. 

Simulators of this type are useful for determining the operating range of an oximeter.  For example the limits in terms of dark skin and low perfusion can be found.  One can determine if the device reads an erroneous value when pushed beyond its effective range or if it reports nothing.

The OpenOximetry.org Project is working to develop novel in vitro testing devices and protocols that overcome the limitations of prior devices. The hope is that newer techniques can better augment human study subject testing and predict device performance. 

Read here for more on oximeter performance validation requirements