EDM and SOAF – Data Quality and Assessment
At RPS we understand that public perception of overflows is changing and the need for companies to have high confidence data to deliver performance reports is increasing.
With developing government pressure and public awareness, the number of Event Duration Monitors (EDM) has grown exponentially over the last few years with circa 17,500 overflows (in England and Wales) requiring monitors to be installed by the end of 2023. As a result, water companies are now able to gather more robust data on overflow performance than at any time in history, providing a consistent way of monitoring how often and for how long storm overflows spill, enabling improved targeting of investment.
Discharge permits now include monitoring conditions and reporting requirements, for each storm overflow identified for EDM. Once monitoring begins under a discharge permit, annual calendar year reports are submitted to environmental regulators (Environment Agency/Natural Resources Wales) at the end of February each year. These summarise the number of spills recorded in the preceding calendar year and the total duration of discharge, potentially triggering investigations in line with the Storm Overflow Assessment Framework (SOAF).
Water companies have an obligation to provide this as open-source data. This increasing visibility of storm overflow spill frequency information carries with it the risk of challenges from customers and non-governmental organisation (NGOs), so accuracy of data and calculation is critical in this highly politicised area of performance and RPS have been supporting our Water Company clients to enhance confidence in reporting statistics.
The challenges associated with EDM data that RPS are working to resolve can be broadly condensed into three main areas:
- improving the quality and confidence of data streams
- making better use of existing data and information related to EDM sites
- improving communications with third parties including the public
Increasing confidence in data assessment
As part of our work on SOAF studies for water companies, we undertake initial quality assessments of the data provided via EDM. Monitoring sites record water level/depth and are impacted by many data issues, including sudden spikes, deviation from established patterns and flat lining. In many cases, these issues can be highlighted using RPS data analytics processes developed to support these assessments. These activities may include:
- Flagging spills occurring in dry weather through assessment against historical rainfall. This determines operational investment needs to support overflow performance
- Developing a “normal” operational envelope and identify when the monitor signal deviates from this
- Identifying where the signal has jumped and returned to baseline over a single time step, allowing many spills to be discounted. Following this analysis many of these spills can then be discounted, but some may require investigation by operations staff to rule out possible operational issues first
- Reviewing monitor setup and calibration, confirming correct location and trigger levels. These challenges can be hard to identify by just using automated cleaning processes on the data
- Reviewing flatlining EDM under storm conditions to identify CSOs with low weir thresholds
Matching the Models – increasing confidence using existing tools
Assessing the performance of EDMs against hydraulic models simulating historical rainfall, provides benefits for assessing EDM data quality as well as providing a useful model verification exercise. RPS are using EDM monitors to historically verify models against the continuous level readings, increasing confidence in prediction and calibration of longer term seasonal variation in inflow. These higher confidence models will support an understanding of longer-term overflow performance, providing a better calibration against future overflow spill performance. These models also support an understanding of operation under ‘normal’ rainfall conditions as well as extreme rainfall, enabling pro-active intervention at overflows to minimise the impact of a spill in the future.
There is also a critical element here in making sure we use emerging technology and process to make EDM data review more efficient and consistent. RPS are building Artificial Intelligence and Machine Learning algorithms into our WaterNet Pro platform to deliver automated EDM assessment in real time, alongside the blockage-alerting functionality already present. This will mean the ability to proactively respond to performance issues.
Delivering messages to customers?
Water companies are required to have transparency in overflow performance data, and with this increased availability and visibility of spill data, coupled with the increasing public engagement in water quality, accuracy and confidence in data is critical to ensure confidence in messages around spill performance and water body safety. We have seen water companies being fined significantly in recent years for polluting water bodies, and the scale of fines mean this is front page news. So there is a significant reputational challenge, if overflows are not monitored effectively and should performance to be seen to improve. .
We do need to be mindful however that simply counting the number and duration of spills, provides no understanding of the environmental impact of spill flow entering the water body and ultimately the scale and significance on any pollution. Carrying this message into the public domain is important particularly for SOAF assessments, emphasising that if a spill threshold is breached, it is still the role of the assessment to identify the potential scale and severity of the pollution risk and mitigate it as appropriate. It may be that no/limited further action is undertaken.
RPS are continuing to work with our water company clients to drive improvements to the EDM process and understanding, increasing confidence through integrating wider information and modelling data into EDM performance analysis. This ensures that the public have access to the highest confidence data to better understand their local environment.
Download for later
Get in touch
If you would like to discuss this topic further and understand how RPS can support you to achieve your 'zero pollution' goals, please contact me anytime.
