US National Science Foundation supports research and development of automated quality control algorithms

The US National Science Foundation (NSF) has awarded a 3-year grant to Oregon State University and City University of New York to develop new algorithms for automated data quality control (QC). The data collected by networks (such as TAHMO) must be checked for sensor failures. Traditionally, this has been done manually by having expert meteorologists and hydrologists examine the data from each station. While this works for small networks, it is completely impractical for a network the size of TAHMO. This project will develop new machine learning algorithms for anomaly detection and integrate them into a software framework, called SENSOR-DX, for analyzing sensor data in real-time to detect sensors that have drifted out of calibration or failed in some way.  Tom Dietterich (Oregon State) will lead the algorithm development and Michael Piasecki (City University of New York) will direct a subteam to implement the algorithms and workflows within the Kepler scientific workflow management system. The resulting system will be deployed for TAHMO Data QC and will be evaluated by several partner organizations (Oklahoma Mesonet, Earth Networks, and CUAHSI) for potential deployment as well.