National Weather Service Central Region
The
Hydrological and Surface Observation
Quality Control Program at NWSO
Central Illinois
Mark F. Britt, Thomas N. Frieders, Christopher W.
Geelhart,
and Billy R. Ousley - WFO
Central Illinois
Arno Perlow -
Introduction | Program
Formulation and Methodology
QC of
Individual Products | QC Manual | Concluding
Remarks
Acknowledgements | References
Return to Main
Page of Web Module
Introduction
Every day the National Weather Service (NWS) processes thousands of weather observations that become a part of the climatological database of the United States. These observations come from varying sources and are used for different purposes including forecasting hydrometeorological conditions. They are of little value if they contain errors, wasting NWS employees' time and taxpayer money. The NEXRAD Weather Service Office (NWSO) in Central Illinois (ILX) has developed a comprehensive program of quality controlling (QC) hydrological and surface observations taken within our county warning area (CWA).
To the authors' knowledge, there has been no meaningful exchange of information on how each office in the Central Region is performing quality control of these data. This became very evident when visiting offices looked at our program and took copies of it back to their respective offices. It is not known how many other offices have a formal program and what methods are being employed. This paper describes how such a program was created and implemented at ILX.
Program Formulation and Methodology
The goal of the QC program at NWSO ILX is to ensure the high quality of hydrometeorological observations for both internal and external use. It is designed to be a flexible program, changing as products are added and deleted, and as the needs of our customers change. The QC process will assist the hydrometeorological technician (HMT)/meteorologist intern (MIT) to perform his/her QC tasks and reduce meaningless tasks in the shift duties. Since its inception, the integrity of these data that ILX is responsible for has seen a noticeable improvement.
The program was originally assigned as a focal point duty for a MIT. Due to the program's growth and complexity, a team concept has been implemented, requiring input from the QC Focal Point (the team leader), the Data Acquisition Program Manager (DAPM), the Hydrology Focal Point (HFP), the Cooperative Program Focal Point (CPFC), and from an additional staff member experienced in computer programming.
The focus of the program is on the HMT/MIT who has to perform the conventional task of QC. The policies established by our program were not only designed to assist the QC focal point's duties, but were mainly designed to be the most effective methods the HMT/MIT could use to perform their job. With this in mind, the program began with a series of brainstorming sessions of the HMT/MIT staff. From these meetings, a plan was devised on how QC would be performed on each product. Since then, several additional meetings have been held to examine, update, and amend procedures, as well as educate the staff about the use of the SHEF code. These meetings also allowed the QC focal point to reiterate to the HMT/MIT staff the importance of QC and the need for uniformity when performing the tasks. A list of QC duties performed on every shift is available in Table 1.
The increased emphasis the NWS has placed on enhanced technology can also be used to improve the QC process. The team's computer programmer has written DOS and Visual BASIC programs, as well as several AFOS macros. Both the locally written programs along with those composed elsewhere (Table 2) have made the QC process easier and more time efficient for the duty HMT/MIT.
The QC program at ILX is not perfect. Motivation is still lacking at times, especially when active weather takes higher priority. This problem has been alleviated to some degree by constantly refining the procedures and adding easy-to-use helps that meet the needs of individual HMT/MIT's. Fully automating all tasks would be a key to overcoming this, but QC still requires human involvement for the foreseeable future. Ultimately, the success or failure of the office QC program falls onto the shoulders of the individual(s) performing the daily QC tasks. Therefore, open communication between the QC team and the HMT/MIT staff is necessary for a successful QC effort.
All changes to erroneous data are documented in the station's computerized shift log. All QC related problems and corrections are read by the HMT/MIT in the log when they start their shift. It is also read by members of the office staff (electronic technicians, HFP, DAPM, forecasters, etc.) who have to take specific actions to correct certain problems.
QC of Individual Products
ASOS Observations
The QC of Automated Surface Observing System (ASOS) observations has been a paramount concern of the NWS. The ASOS Operations and Monitoring Center (AOMC) is charged with continually monitoring the integrity of ASOS platforms around the country. Observations within the ILX CWA are also checked locally for availability, correct augmentation, and sensor reliability. The HMT/MIT can consult locally written help sheets that explain the format of an ASOS observation and the augmentation requirements at each station in the CWA. Additional information informs the duty HMT/MIT of what to do if an observation is not available.
METAR ASOS observations are received just before every hour. At approximately 2 minutes after the hour, the Automation of Field Operations and Services (AFOS) ADM alarms with the NOAA Weather Radio Hourly Weather Roundup script. The script allows the HMT/MIT the first chance to see whether an observation is missing, if it has missing elements, or if it is unrepresentative by comparing it to surrounding sites. Around 8 minutes after the hour, another product alarms at the AFOS ADM alerting the HMT/MIT of a missing observation or one with missing elements (Figure 1). At approximately 10 minutes past the hour, a local product alarms that displays the METAR format of all six ASOS sites within the ILX CWA (Figure 2). The HMT/MIT can check the observation for additional missing elements, for augmenter sign-on compliance (i.e. "AUTO" remark), for augmenter coding errors, and for the recognized system failure symbol ("$").
If the HMT/MIT notices an observation is missing, partially missing, or unrepresentative of surrounding stations, the outage is coordinated with AOMC. If the augmenter is not logged onto ASOS or the observation is improperly coded, the HMT/MIT can call the respective observer to correct the mistake.
River Stages
There are two types of river stage information received every 6 hours at NWSO ILX: Campbell Scientific Data Loggers (CR-10), and Geostationary Operational Environmental Satellite (GOES) Data Collection Platform (DCP).
DCP and CR-10 gages are part of the cooperative observation program, and the ILX HMTs are required to inspect gage sites at least once annually and record the visit on a locally written form (Figure 3) as part of their cooperative program duties. This adds an additional Quality Assurance (QA) step prior to receiving any data from the river gages. The HMT/CPM inspecting the river gage site must operate within the HFP guidelines in supplying details observed during visits. ILX HMTs have received training via the NWS Cooperative Network Operations course, ILX in-house training, and instruction provided by the Illinois District of the United States Geological Survey (USGS).
Stages are received at ILX in real time from the USGS-maintained CR-10 sites using a ProComm Plus (Datastorm, 1990) script program that automatically dials the sites and compiles the data into a coded SHEF ".B" format message using HABT (Callahan, 1994). Also received every 6 hours are stages from the Corps of Engineers (COE) and USGS GOES DCP sites via the Hydrometeorological Automated Data System (HADS). When the messages from the CR-10 and DCP sites are received in AFOS, a locally written river comparison program runs using observations decoded by PC-WXR (Edwards, 1994). This program (Figure 4) gives the name, station identifier, precipitation (since 12 UTC), current river stage, 6 hourly stages for the past 24 hours, and a 24 hour computed change in a tabular format for all locations within and immediately adjacent to the ILX Hydrologic Service Area (HSA). In addition, the program flags rapidly changing stages, stagnant stages, and suspicious precipitation amounts.
The HMT/MIT can use the Internet to aid them in diagnosing problems. Both the COE and USGS have home pages on the World Wide Web (WWW) that contain recent stages and updated hydrographs for many river gaging sites across the United States. The hydrometeorological staff at NWSO ILX is encouraged to make use of these data when conducting QC activities. If the HMT/MIT finds a stage from the Internet that seems to be accurate, then he/she can use that stage to replace the questionable one after consultation with the duty forecaster. The HMT/MIT also uses hydrographs of a questionable site to display graphically what may not be readily apparent by just looking at sequential numbers.
Every 6 hours, the HMT/MIT uses the river comparison printout and the Internet to recognize, diagnose, and remedy potential problems with the newly received stages. These problems include:
(1) Missing readings: If a stage is missing, it is likely due to an equipment or communication problem. Any CR-10 site having missing data can be redialed to find out if there are equipment or communication failures. When DCP information is not received through AFOS, but is accessible over the Internet, there could be problems within NWS communications. If a reading can be obtained from the Internet or a manual observer, then a coded SHEF ".BR" format message can be sent with the correct stage;
(2) Erroneous readings: The HMT/MIT must know the river flow characteristics of each river in the CWA as well as the hydrometeorological conditions occurring in a given situation. For instance, a heavy rainfall event in smaller river basins will usually be accompanied by a rapid increase and a proportional decrease in stage. Larger rivers have a more gradual change. If a heavy rainfall and/or rapid snow melting event has occurred, and data from a site is not changing accordingly or not changing at all, then a problem exists. Changes in river stages, even abrupt ones, tend to have some degree of smoothness on a hydrograph. Blocky changes (those with 90-degree angles) also indicate a problem with the river gage. Other changes in river stages are obviously wrong (e.g. a change of 400 feet in 12 hours). The QC process at ILX prevents bad data from the above problems from entering the river forecast models. If the HMT/MIT recognizes a problem, they can try to obtain a correct stage from other methods (Intranet, manual observer, etc.). The correct stage is then transmitted using a coded SHEF ".BR" format message. If backup information is not available, the correction will be sent as not available (i.e. "M").
(3) Low flow: Often during long periods with little precipitation, a river may be in low flow and the stage will not change. This is often represented by a flat line on the hydrograph. This is usually not a problem and the stage does not need to be corrected unless it remains stuck after a significant rain.
(4) Spring thaw/ice jams: During the spring thaw and/or whenever there is a heavy precipitation event, it may be necessary to request a manual observation of the river for ice jams. This would be evident if the river stages above/below a point(s) on the river are rising rapidly with little or no change after the point of the ice jam. If the problem poses an immediate threat to life and/or property, the HMT/MIT staff will notify the shift leader of the possible problem. Staff on duty may notify local emergency managers to investigate the existing conditions. If the possible problem poses less of a threat and/or is determined to be a gage failure, then such problems are reported to the HFP who will take appropriate actions as soon as possible.
If a stage is missing or unrepresentative for any of the above reasons for more than 2 or 3 days (less time during floods), it is reported to the HFP who will take the appropriate action to remedy the problem. In the absence of the HFP, appropriate actions to be taken by the shift leader are available in the ILX Hydrological Services Manual.
DCP Precipitation
DCP river stage locations are often collocated with tipping bucket rain gages. Precipitation measurements are available during warm season months along with the river stages. Two common problems found with DCP precipitation values are: Zero values due to the obstruction (most likely trees) of the gage when radar indicates rainfall has occurred, and precipitation amounts shown on a non-precipitation event because of snowmelt.
The resulting readings have great significance since they are used in river forecast models. The HMT/MIT accesses these stages on the river comparison program printout every 6 hours. The precipitation measurements are compared with radar estimates from the WSR-88D, ASOS tipping bucket amounts, and cooperative observer reports. If the sites report precipitation when the radar indicated nothing occurred, or differ greatly from other sources, then the duty forecaster and the HMT/MIT can deem them unreliable, and transmit a corrected report of missing (M) via a preformatted coded SHEF ".BR" format message. This is a subjective judgment and requires different approaches depending on whether the event was stratiform or convective in nature.
ROSA/COMPU-ROSA Observations
One of the most important tasks of the ILX HMT is assuring that all reports have three basic qualities: accuracy, dependability, and timeliness. All three qualities work together to provide not only quality data and a useful product, but a high confidence level in the observed data. If any one of the three qualities is missing, the confidence in the observed reports is reduced, and the product value is diminished or considered useless.
Therefore, as related to the ILX Cooperative Observation program, indirect QA measures in conjunction with direct QC methods are employed as follows:
(1) Before the Fact (Preventative) QA Measures: The most important QA measures are performed during ILX HMT station visitations. Proper initial training (along with refresher training) of the observer is priority number 1. Along with this, ILX HMTs try to establish objectives, expectations, and goals for individual observing stations. Setting deadlines in regards to received reports is yet another QA measure that is a necessity.
(2) Real Time QC: Every morning, numerous observations are received over AFOS from cooperative observers using Compu-ROSA in SHEF ".A" format code. These transmitted observations contain various information depending on the site's classification and instrumentation. Transmitted observations may include maximum, minimum, and current temperature; 24-hour precipitation; special rainfall and snowfall reports; river stage readings; soil temperatures; pan evaporation; and current weather. At around 1345 UTC, a product that compiles all the cooperative observations received is alarmed on AFOS (Figure 5). The duty HMT/MIT checks each observation for coding errors and data representativeness.
Examples of common errors include:
- Observer not placing the proper agricultural district code at the end of the observation;
- A precipitation type coded when no precipitation occurred;
- Including too few zeros in the precipitation group;
- Special observations improperly coded as 24 hour totals;
- Unrepresentative data when compared to surrounding stations (particularly water equivalent (SW)).
Erroneous observations are corrected and sent out over AFOS using a coded SHEF ".AR" format message.
(3) After the Fact QC: The ILX "Area Temperature and Precipitation Table" (both morning and evening reports), the North Central River Forecast Center (NCRFC) "Daily Precipitation Summary" (which flags possible erroneous precipitation data), the NCRFC "Weekly Precipitation Observations Summary", the Midwest Climate Center's (MCC) "Weekly ROSA Report", and the Central Region Headquarters' (CRH) "Monthly ROSA Quality Control Report" are additional generated product tools utilized in the ILX office QA/QC process. These products are generated after data has been transmitted and entered into appropriate databases for use.
This ILX "Area Temperature and Precipitation Table" (Figure 6) product is produced through AFOS and is a compilation of received cooperative observer reports for the day. It is alarmed on AFOS every morning at 1530 UTC and every evening at 0230 UTC. The on-duty HMT/MIT views the product for any immediate discrepancies, annotates these errors, and informs the HMT who is responsible for the cooperative observer making the error to take the appropriate action.
The NCRFC "Daily Precipitation Summaries" are scrutinized for extreme values and verified for accuracy. This allows for immediate feedback to observers to either correct an error or prevent future errors of the extreme variety. The NCRFC "Weekly Precipitation Summaries" are reviewed to identify observers that are not consistently reporting. It is also used to review for inconspicuous or repetitive mistakes that do not contain extreme values and thus may not be picked up on the Daily Precipitation Summaries.
The MCC "Weekly ROSA Report" contains a daily listing of ILX ROSA/COMPU-ROSA reporting stations. This product allows HMTs to examine which cooperative observers are filing reports consistently. Its value to the QC process is evident when considering that more available data allows for a better assurance of received products. The more information at hand for comparison allows for an improved decision making process when information is subject to question.
The CRH generated "ROSA Quality Control Report" is forwarded to local offices monthly. It contains information pertinent to each received ROSA report per station from the previous month. Information contained includes the date of the last report, the number of correct reports, the number of rejected reports, and the number of reports received in the 0700 (plus or minus) 2 hour window. The ability to see not only who is reporting via ROSA , but when, how often, and a record of errors for each particular station is of great value to the office QC effort and can be summed up in the statement "quality data in, quality data out"!
At times, after the fact QC may force continued after-action measures necessary to prevent persistent error or to deter future problem areas. Whether the "after the fact" process begins within a couple hours after receiving/transmitting products, or weeks after the products initiation, it is still a vital and necessary step in the total QC process.
QC Manual
When the QC program was first implemented at ILX, all procedures on how to perform QC could be found in the Station Duty Manual (SDM). A binder was developed shortly thereafter to show examples of each product and to provide comprehensive instructions on how to QC each product. Additionally, there are help sheets that contain information on the ASOS observation data flows between the observation site and our office, correct ASOS observation format and augmentation procedures for each station within the ILX CWA, and correct format/coding instructions for ROSA observations.
The contents of the binder have been converted to HTML code and now resides on the office Intranet. Any changes in QC policy is revised first on the Intranet. This ensures that the on-line manual is current and people do not have to depend on word of mouth or lost electronic mail messages.
Concluding Remarks
This paper has stated why and how the Hydrological and Surface Observation QC Program was created and implemented at ILX. The purpose of the paper is to share and exchange the information with other Central Region offices. Similar processes, formal or informal, are likely being used at offices elsewhere across the Central Region and sharing information like this would benefit the data acquisition program within the Region.
Acknowledgments
The authors wish to thank Ernest Goetsch (ILX MIC), Jeff Hedges (ILX SOO), Steve Schild (CRH SOD), Noreen Schwein (CRH RHO), and Rich Schwerdt (CRH MSD) for their review and helpful suggestions for improving this paper.
References
Callahan, Mike, 1994: Hydrological Auxiliary Backup Terminal. NWS CRCP No. 2MC, August 1994.
Datastorm Technologies, 1990. ProComm Plus Users Manual.
Edwards, Harvey , 1994: PC-WXR Users Manual. NWS SRCP No. 2MC, October 1994.