As US EPA stated in its Air Sensor Evaluation Toolbox:
“emerging air qualityAir quality refers to the state of the air we breathe and its composition in terms of pollutants present in the atmosphere. It is considered good when poll...
Read more sensors – with general traits of being more compact, directly reading pollutants, and lower in cost than traditional methods – have a wide appeal to professional researchers, community groups, students, and citizen scientists alike. […] Since this technology is still under development, little information exists on the quality of data that these sensors produce.”
Air Quality SensorMeasuring air quality is essential for improving human and environmental health. Changes in the natural composition of the air we breathe are common in ind...
Read more Performance Evaluation Center (AQ-SPEC) was established by the South Coast AQM as a programme to inform the general public about the actual performance of commercially available “low-cost” sensors. The objective of AQSPEC programme is to carry out a complete characterization of the currently available “low-cost” sensors.
In this context, the main goals of the AQSPEC programme are:
- Evaluate the performances of commercially available “low-cost” air quality sensors.
- Provide guidance and clarity for ever-evolving technology and data interpretation.
- Catalyse the successful evolution, development, and use of sensor technology.
Air Quality Innovation in Just 1 Click
Stay informed about the air you breathe!
Subscribe to our newsletter to receive the latest updates on environmental monitoring technology, air quality studies, and more.
“AQSPEC programme is carrying out an important task, testing and evaluating the performances of current “low-cost” sensors and systems which integrate those sensors, and giving actual information in an objective way.”
As it was already explained in the article The needs of standards and certification programmes on air quality monitoring, during the last decade, AQSPEC external evaluation centre has focused on determining the performance of different air quality sensors system, due to a lack of an accepted performance specification for “low-cost” sensors.
Hence, the AQSPEC programme is carrying out an important task, testing and evaluating the performances of current “low-cost” sensors and systems which integrate those sensors, and give actual information in an objective way.
The main objective of the report created is to assess the different “low-cost” sensors and air quality systems, regarding the results of the gas and particle sensors obtained in the AQSPEC evaluation. The original AQSPEC evaluation assessment can be found here.
It has to be noted that the evaluation processes shown in this report, and in the reports of the AQSPEC web, are carried out in different seasons and under different conditions. Thus, the results shown must be treated carefully, this report is only an indicative assessment.
KUNAK AIR A10 Field Evaluation
Three Kunak devices were evaluated by the AQSPEC programme from May to July 2019. They were run side-by-side with Federal Equivalent Method (FEM) and Federal Reference Method (FRM) instruments measuring the same pollutants.
Each device reported O3, CO, NO, NO2, and NOx, as well as PM1, PM2.5 and PM10, and meteorological parameters (temperature, relative humidity, pressure, wind speed and wind direction). The report of this field evaluation can be found on the AQSPEC web page.
Gas and Particle sensor assessment
The gas and particle sensors are compared against different reference (FRM) and equivalence (FEM) instruments. The tests performed by AQSPEC are:
- Carbon monoxide (CO)
- Nitrogen oxides (NO2 and NO)
- Ozone (O3)
- Particulate MatterAtmospheric particulate matter are microscopic elements suspended in the air, consisting of solid and liquid substances. They have a wide range of sizes an...
Read more (PM1, PM2.5 and PM10)
A correlation between the sensor data and the reference data is used for AQSPEC as an evaluation method. The parameter shown in the comparative assessment is the R2 and the equation line given by AQSPEC.
It has to notice that this metric has some limitations. The data needs to follow a normal distribution, which is not always the case in real studies. Besides, R2 is insensitive to bias between the air quality system or sensor and the reference data used for the evaluation (Karagulian et al. 2019). Thus, R2 should be accompanied by another metric to measure the error between the data and the references, i.e. MAE, RMSE, or the expanded uncertainty.
Besides, the intra-model variability of the sensor is given. This is an important parameter, especially when a network is going to be deployed and it is not always considered. It shows how different devices from the same manufacturer are going to behave. Low intra-model variability means that you can use one device of the deployment to assure the good operability of all the network since all the sensors and sensor systems will behave in the same way.
Table 1 shows the date, duration, and season in which the different sensors and sensor systems were evaluated by the AQSPEC programme.
Table 1. Air quality sensors and sensor systems evaluated by AQSPEC
| Air Quality System | Sensor | Study Date | Study Duration (days) | Season | |||||||||
| Kunak AIR A10 | Gas + PM | 28/04/2019 | 11/07/2019 | 74 | Spring/Summer | ||||||||
| Aeroqual | Gas | 22/12/2017 | 27/03/2018 | 95 | Winter | ||||||||
| APIS | PM | 13/03/2019 | 14/05/2019 | 62 | Spring | ||||||||
| AQMesh v4.0 | Gas | 26/06/2015 | 25/09/2015 | 91 | Summer | ||||||||
| AQMesh v5.1 | Gas | 11/04/2020 | 18/06/2020 | 68 | Spring | ||||||||
| AQMesh v3.0 | PM | 11/04/2020 | 18/06/2020 | 68 | Spring | ||||||||
| Vaisala | Gas | 06/03/2018 | 03/05/2018 | 58 | Spring | ||||||||
| Cairpol CO | Gas | 22/11/2018 | 17/01/2019 | 56 | Winter | ||||||||
| Cairpol NO2 | Gas | 02/05/2019 | 10/07/2019 | 69 | Summer | ||||||||
| UNITEC | Gas | 01/07/2015 | 31/07/2015 | 30 | Summer | ||||||||
| Perkin Elmer | Gas + PM | 22/07/2015 | 25/09/2015 | 65 | Summer | ||||||||
| urad | PM | 30/10/2018 | 08/01/2019 | 70 | Winter | ||||||||
| Alphasense OPC N3 | PM | 15/08/2018 | 11/10/2018 | 57 | Summer/Autumn | ||||||||
| Sensirion | PM | 07/03/2019 | 14/05/2019 | 68 | Spring | ||||||||
| Clarity | PM | 15/02/2018 | 25/04/2018 | 69 | Winter/Spring | ||||||||
| Purple Air | PM | 08/12/2016 | 26/01/2017 | 49 | Winter | ||||||||
| Met One | PM | 01/06/2015 | 15/07/2015 | 44 | Summer | ||||||||
| Air Beam | PM | 20/07/2018 | 19/09/2018 | 61 | Summer | ||||||||
| Air quality Egg | PM | 25/04/2018 | 26/06/2018 | 62 | Spring | ||||||||
| Sensirion Nubo | PM | 27/12/2019 | 27/02/2020 | 62 | Winter | ||||||||
| TSI AirAssure | PM | 18/12/2015 | 15/02/2016 | 59 | Winter | ||||||||
| TSI Bluesky | PM | 08/04/2020 | 15/06/2020 | 68 | Spring | ||||||||
Regarding Kunak AIR A10, it uses onboard sensors and Kunak embedded algorithm that calculates the concentrations in ppb or μg/m3 locally and in real time, not using or needing external reference data to calculate gas or particle concentrations. Kunak algorithm corrects the interferences and artifacts typically found in raw data compensating environmental effects like temperature, humidity and pressure as well as cross-sensitivities using onboard data.
For most of the electrochemical gas sensors, the humidity and temperature effect have a no-linear response (Lewis et al. 2018; Pang et al. 2018; Wei et al. 2018). The raw data is corrected by the Kunak algorithm converting the no-linear response into a linear response. However, none of the sensors were calibrated before deploying. The only calibration carried out was the factory calibration.
“The Kunak AIR A10 uses onboard sensors and embedded algorithm that calculates the concentrations in ppb or μg/m3 locally and in real time, not using or needing external reference data to calculate gas or particle concentrations.”
Figure 1 shows the R2 values obtained for all the sensors and sensors systems evaluated, and Figure 2 shows the intra-model variability. The blue star shows the value achieved by Kunak AIR A10, being possible to compare if Kunak AIR A10 system with the rest of the evaluations carried out by the AQSPEC programme.
For more information, you can read the full report, in which a detailed comparison with other sensors evaluated by AQSPEC is available.
Figure 1. R2 values obtained between the different sensors evaluated. The blue start shows the R2 obtained by Kunak AIR A10.
Figure 2. Intra-model variability values obtained between the different sensors evaluated. Blue start shows the intra-model variability obtained by Kunak AIR A10 (The lower intra-model variability the better)
In the specific case of Kunak AIR A10:
Figure 3. CO 24-hour means measures, Kunak AIR A10 vs FRM
- For O3, Kunak AIR A10 achieved an averaged R2 with a slope equal to 1, meaning that the concentration values collected by Kunak O3 sensor provide very accurate data.
- Kunak AIR A10 achieved a medium result for CO. In the case of the CO, most of the values were under the CO performance limit level un field conditions. Besides, due to a high-temperature event, up to 40ºC, the CO sensor started to overestimate the CO concentrations. At the beginning of the field evaluation, Kunak sensor systems followed the same trend and reach the same CO values as the reference, but after the high-temperature event, a sensor baseline recalibration is needed, as it is explained in Kunak Air A10 Manual, however it was not carried out during the field assessment.
Due to R2 metric have some limitations, as it was explained above, another metric should be considered. The future European Technical Specification, “Performance evaluation of air quality sensors”, will be use the expanded uncertainty for field evaluations, in which the maximum uncertainty allowed regarding the different pollutants will be set.
Thus, this expanded uncertainty was calculated in the case of CO sensor. The future specification said that the maximum expanded uncertainty allowed for CO is 25%, and the one achieved in the case of the CO sensor with the data obtained during the AQSPEC field evaluation was, 12.20% without needed any calibration, and using the regression line calculated, it could be achieved an expanded uncertainty of 4.25%, being even lower than the 15% expanded uncertainty required for reference stations according to the EU Directive/2008/50 CE.
- Kunak NO2 sensor obtains a low R2 value, like the rest of the sensors evaluated by the programme, except for the one that was evaluated during winter. The study was performed during summer where low concentrations of NO2 appeared, leading to a higher error. Besides, several humidity transients occurred during the study period provoking an effect on the sensors. Humidity transients’ effects in electrochemical sensors, especially in NO2 sensors, have not been studied yet. By the moment, the data obtained when a humidity transient occurs should be invalidated. However, it was not the case during the field evaluation where all the data were used for the evaluation study.
- Only four air quality systems have been evaluated for NO sensor. Only two NO sensors obtained a high R2 value (Kunak AIR A10 R2 = 0.87), while the other two sensor systems were not able to measure NO.
- Evaluating Kunak AIR A10 PM5 sensor against other sensors, it is on average with them, while for PM10, Kunak obtained the highest R2 when compared with the other sensors evaluated. It is noticed that some devices which obtained good results for PM2.5 were not evaluated for PM10 because the sensor used is not able to measure both PM2.5 and PM10. Kunak AIR A10 is the only sensor system which provides PM2.5 and PM10 accurate measurements.
- Regarding the intra-model variability, the Kunak AIR A10 device is the only sensor system which obtained the lowest intra-model variability values in every sensor.
Figure 4. Intra-model variability obtained regarding the different sensors integrated into the Kunak Air A10 system.
Conclusions
It must be highlighted that the evaluation processes were performed in different seasons and under different conditions by the AQSPEC programme.
To conclude:
- Kunak AIR A10 devices were not calibrated prior to the evaluation. It is shown how Kunak gas sensors do not need calibration, apart from the factory calibration, while Kunak PM sensors would need an adjustment of the mass (correction factor) which depends on the particle type specific in each location, as occurs with the particle FRM and FEM instruments.
- Kunak AIR A10 devices obtained very high correlation values for the O3 and NO sensors, medium correlation values for CO and poor for NO2 due to high temperatures, low pollutant concentrations, and in the specific case of the NO2 due to humidity transient effects. All these effects are a common problem when using electrochemical sensors and affect all manufacturers, who are working continuously to improve their algorithms to achieve a sensor system in which these temperature and humidity effects will not be a problem in the near future.
- Kunak AIR A10 obtained the lowest intra-model variability in all the gas and particle assessments, showing the good performance of the devices and trusting in the results when only one device could be compared with a reference station. This very low intra-model variability allows using a “Gold” device for calibration, relying on differential measurements and comparing sensor systems in different locations.
- The Kunak PM sensor is the only sensor that can measure PM2.5 and PM10 accurately. Even though some sensors obtain slightly higher PM2.5 R2 values, they are not able to measure PM10. Kunak AIR A10 provides precise PM2.5 and PM10 measures with only one PM sensor.
AQSPEC evaluation programme is doing a great effort to compare low-cost air monitoring sensors to establish performance standards, being the most complete external evaluation programme. Nevertheless, some improvements could be carried out for a more objective evaluation among the different sensors. Even though it is difficult to carry out, to get more objective results, all the sensors should be measured at least in the same season, such as winter, thus, high-temperature effects and humidity transients would be avoided. As it was mentioned before, due to the R2 limitations, another metric should be considered, i.e. RMSE, MAE, or the expanded uncertainty, as considered in the future European Technical Specification, “Performance evaluation of air quality sensors”.
To summarize, despite the environmental conditions provoking an effect on the sensors, such as high temperatures, and humidity transients, which not only occur in the Kunak AIR A10 sensor system, but they are common to all the low-cost sensors, Kunak AIR A10 is the best option as a multiparametric air quality monitoringControlling air quality is an essential task in order to enjoy optimal environmental conditions for healthy human development and to keep the environment i...
Read more system, being able to measure accurately different gases (O3, CO, NO2 and NO), particulate matter (PM1, PM2.5 and PM10) and environmental data (temperature and humidity).
Regarding Kunak Air A10, it uses onboard sensors and Kunak embedded algorithm that calculates the concentrations in ppb or μg/m3 locally and in real time, not using or needing external reference data to calculate gas or particle concentrations. Kunak algorithm corrects the interferences and artifacts typically found in raw data compensating environmental effects like temperature, humidity and pressure as well as cross-sensitivities using onboard data.
For most of the electrochemical gas sensors, the humidity and temperature effect have a no-linear response (Lewis et al. 2018; Pang et al. 2018; Wei et al. 2018). The raw data is corrected by the Kunak algorithm converting the no-linear response into a linear response. However, none of the sensors were calibrated before deploying. The only calibration carried out was the factory calibration.
Kunak Air A10 is the best option as a multiparametric air quality monitoring system, being able to measure accurately different gases (O3, CO, NO2 and NO), particulate matter (PM1, PM2.5 and PM10) and environmental data (temperature and humidity).
References
- Karagulian, Federico et al. 2019. “Review of the Performance of Low-Cost Sensors for Air Quality Monitoring.” Atmosphere 10(9).
- Lewis, Alastair C. et al. 2018. Wmo Low-Cost Sensors for the Measurement of Atmospheric Composition: Overview of Topic and Future Applications. http://www.wmo.int/pages/prog/arep/gaw/documents/Draft_low_cost_sensors.pdf.
- Pang, Xiaobing, Marvin D. Shaw, Stefan Gillot, and Alastair C. Lewis. 2018. “The Impacts of Water Vapour and Co-Pollutants on the Performance of Electrochemical Gas Sensors Used for Air Quality Monitoring.” Sensors and Actuators, B: Chemical 266: 674–84. https://doi.org/10.1016/j.snb.2018.03.144.
- Wei, Peng et al. 2018. “Impact Analysis of Temperature and Humidity Conditions on Electrochemical Sensor Response in Ambient Air Quality Monitoring.” Sensors (Switzerland) 18(2).