How did ESB Networks, the country's distribution system operator (DSO) — for an island system that has a predominately overhead distribution network — plan for a major storm making landfall on the island? What lessons from the planning and restoration phases can be applied to other distribution system operators for future major weather events? Part I of this two-part series, 'Weathering the Storm - Part I' is republished here. Key insights • A Storm Emergency Management Team (SEMT) with clearly defined roles, responsibilities, and resources should be established in advance of the storm arrival when weather forecasting predicts a storm will impact the system. The SEMT should coordinate closely with the national agencies responsible for storm response, to ensure a safe and successful restoration process; • A multimedia approach to information provision to provide public awareness is recommended if possible; it should encompass traditional media, online and social media, and (if available) an interactive, map- based fault information system. Messaging to the public should be focused and consistent across all platforms, emphasising the safety risks to the public and updated estimated restoration time (ERT) for all faults; • Staff fatigue and exhaustion can become an issue if the restoration efforts extend beyond three to four days. These can be mitigated with compulsory rest periods and daily safety briefings highlighting the issue. The ability to integrate mutual aid crews from visiting utilities safely and quickly is a key force multiplier which allows for greater work volumes to be completed and is also a tool for mitigating staff fatigue.

About the Irish distribution system

In the Republic of Ireland, ESB Networks (ESBN) is responsible for the operation of the distribution system—the entire low- and medium-voltage network up to 38 kV, and the 110 kV in the Dublin area. The distribution network consists of 150,000km of overhead line providing electricity to 2.35 million electricity users. Due to the dispersed population in Ireland, the overhead network length per capita is six times greater than the European average figure, which results in increased exposure of overhead assets to major weather events. The National Distribution Control Center (NDCC) is located in Dublin and there are additional regional management offices throughout Ireland.

Hurricane Ophelia

Hurricane Ophelia originated in the North Atlantic near the Azores Islands some time on October 6, 2017. It was an unusually extreme weather event for Ireland; it was also the easternmost hurricane recorded since records began. Most hurricanes develop further south in the warmer waters of the Atlantic Ocean and, driven by the easterly trade winds, usually gain strength before making landfall over the mainland United States, Mexico, and the Caribbean Islands. Due to a combination of unusual weather factors, Ophelia tracked north towards Europe with the hurricane cone of uncertainty on track to make landfall somewhere on the southern coast of Ireland. While western Atlantic cities have experience dealing with the impact of hurricanes on transmission and distribution systems, Europe and the eastern Atlantic do not. Ophelia peaked as a Category 3 hurricane on October 14. By the time Ophelia made landfall in Ireland on the morning of October 16, it had weakened enough to be categorised as a tropical storm.

Ireland's weather warning system

The country has three weather warning categories: 1. Status Yellow – Weather Alert: Be Aware. These are for weather conditions that do not pose an immediate threat to the general population, only threaten those exposed to risk by the nature of their location and/or activity. 2. Status Orange – Weather Warning: Be Prepared. These weather warnings imply that all recipients in the affected areas should prepare themselves in an appropriate way for the anticipated conditions. 3. Status Red – Severe Weather Warning: Take Action. The issue of a RED severe weather warning is a comparatively rare event; it implies that recipients take action to protect themselves and/ or their properties. People in the affected areas may need to prepare by temporarily moving their families out of the danger zone, staying indoors, or by additional specific actions aimed at mitigating the effects of the weather conditions.

The pre-planning phase

Landfall Day - 3: Friday, October 13 A yellow weather warning was issued by the Irish Meteorological Service, Met Éireann, for Monday, October 16. At this stage, there was still uncertainty about the storm’s landfall, intensity, and potential impact. Based on the weather alert discussions with Met Éireann and reports of the anticipated potential effects from the hurricane, ESBN began escalating storm-response preparedness. The National Emergency Co-ordination Group (NECG), which co-ordinates and oversees the planning and response of state agencies to major emergencies in Ireland, was put on alert. ESBN alerted EirGrid (the national transmission systems operator) to the escalation in preparedness for the potential weather impacts. Within ESBN, all functions were escalated and an SEMT was set up. The emergency customer contact centre, in a contracted business contingency hot site, was invoked and a management structure was established and trained volunteers were alerted to the possibility that they may have to attend on Monday. With a view to post-storm restoration efforts, there was an initial briefing given to NEWSAC, the consortium of DSOs in the United Kingdom and Ireland that provides mutual aid (technicians and electricians, in particular) in the event of major weather events impacting a territory. Following these preliminary meetings of the NECG, SEMT, and NEWSAC, further meetings were scheduled for the following three days to prepare for the storm. Landfall Day - 2: Saturday, October 14 As the forecasted path of the hurricane became more certain, a red weather alert was issued by Met Éireann, for October 16 for the west coast of Ireland. An orange weather alert was issued for the remaining counties. A decision was made by the relevant government departments to call a meeting of the NECG to take place on Sunday morning at 10am. In ESBN, planning for the storm and its aftermath ramped up with the meetings of the SEMT and the establishment of divisional storm response teams in line with the national and divisional emergency plans. The emergency customer contact centre was further prepared with required IT and communications facilities. Additional staff were scheduled in the NDCC for Monday, October 16. ESBN Telecoms staff were escalated to be available to respond to damage to the communications infrastructure that may occur during the storm. Landfall Day - 1: Sunday, October 15 Met Éireann extended the red weather warning for October 16 to all counties in Ireland; warnings by officials through all media outlets meant that the population was on high alert. At the NECG meeting, a decision was made to close all educational institutions in Ireland on October 16. In ESBN, the SEMT met again to agree on priorities and actions. The SEMT agreed that the main focus of the team for the duration of the storm response would be on: safety; information; and restoration. At 5pm, the 'Emergency Management Office' was established close to the NDCC with hot desks, IT, and communications facilities. Among the key roles within the storm coordination group were: operations manager, divisional or regional managers, information co-ordination, system restoration, and resource manager. See Figure 2 for a high-level organisational chart. The medium-voltage (MV) network is normally under the control of operators in the NDCC. In advance of the storm making landfall, the control of the MV systems was transferred to the local MV system managers located around the country. There was close coordination with EirGrid and key transmission stations were staffed by ESBN staff on Monday morning to have approved staff on site if required. Co-ordination between government agencies and local authorities was also established ahead of the expected restoration effort. A data science team were invited to work with the operations manager during the storm, to provide data science input and to learn about how future weather events might be managed. All ESBN staff were requested to stay at home unless they were specifically required for the storm response. Staff responding to the storm in the south of the country (where the storm was due to make landfall) were requested to get to work safely-before storm landfall. Safety briefings were issued to all team members, but were particularly focused on field staff.

Day of the storm's landfall: Monday, October 16, 2017

The storm made landfall on the morning of October 16 and the first fault on the distribution network was reported at 7.45am. Throughout the day, NECG and the SEMT met at regular intervals to assess the situation, share information, communicate key messages to the public and key stakeholders, and to co-ordinate the ESBN response and plan the restoration. The storm passed along the western seaboard of Ireland and had cleared the south region by the early afternoon. For safety reasons, no work was carried out by ESBN operators during the storm and all switching was carried out remotely from the NDCC. In the early afternoon, when it was safe to do so, work on the restoration effort began and, initially, the main priority was assessing the damage to the network; most of the assessment was completed by foot patrol, but helicopter support was arranged from private helicopter contractors which commenced on Monday. Throughout the day ESBN provided regular media briefings through the NECG, media interviews in the NDCC and regular press releases and website updates. ESBN agreed a supply restoration prioritisation approach with the NECG ahead of the restoration activities on Day +1. The operations data science team (see below) developed a prioritisation algorithm for the approach to select the network faults to be scheduled for response in the coming days.

Impact of the storm on customers

At its peak on the October 16, the storm left 385,000 (17 per cent of total customers) customers in Ireland without supply equating to 11.7 million customer hours lost and almost 4,500 outages. Post event analysis indicates that approximately 90 per cent of these faults were caused by falling trees.

Post-event system restoration

Day + 1: Tuesday, October 17 After the storm passed, Tuesday, October 17, was the first full day of restoration work across the network in the affected areas. A detailed resource plan was developed by the SEMT. Efforts initially focused on patrolling the damaged network and damage-assessment plans were put in place on Tuesday to ensure that the materials and skills were correctly assigned for each repair job. Staff in unaffected areas were relocated to affected areas and regions. Over the course of the week, teams travelled in units from the northern part of the country to more damaged areas in the south of the country. The deployed units were self contained units that were equipped with skills and approvals to allow operation and repair of the network with local controller permission. Further meetings were held with the NEWSAC organisations and, once it was clear that the weather event would not impact other territories in the UK, the organisations committed to deploying staff to Ireland to aid the restoration effort. In total, 359 staff were deployed between October 17 and October 25, 2017. Schools remained closed on Tuesday and the public was requested to report lines that were down or other dangerous situations, which resulted in about 2,500 emergency calls. At the end of October 17, 207,000 customers remained without power. Day + 2: Wednesday, October 18 At the NECG press briefing on Tuesday, October 17, ESBN committed to providing Estimated Restoration Times for all outages on the morning of Day +2. These ERTs are communicated through several methods including its website and its Powercheck app. The app is interfaced with the Network Management System (NMS) in the NDCC and any logged calls and reports of outages are visible on the application map. The initial ERT lists were published at 9am on Wednesday, October 18. In addition, a town-by-town report for the whole country was published and updated regularly. The public were requested — via traditional and social media — to contact ESBN and report any outage that was not logged on the Powercheck app. This process aided the restoration effort and helped to narrow the focus of resource allocation. In the initial days, the response was planned to prioritise restoration for health service facilities, telecommunications sites, and water treatment facilities; a dedicated resource was assigned to manage the interaction and information exchange with these stakeholders. A dedicated email address was established prior to the storm to allow all public representatives to lodge queries for immediate response with ESBN. By the end of October 18, 142,000 customers remained without power. [caption id="attachment_41654" align="alignright" width="81"] Figure 6: crews work on faulted overhead lines[/caption] Day + 3: Thursday, October 19 While the restoration effort was proceeding apace and supply being restored, operator and worker safety continued to be of paramount importance. Staff fatigue was a matter of particular concern due to long hours in challenging conditions; this issue had to be monitored and managed by divisional managers and the SEMT. An influx of external contracted staff from other regions of the country and overseas, created a need for more accommodation. An army barracks in Co Cork was used for temporary accommodation. All visiting staff were inducted at the ESBN National Training Centre before being deployed to specific depots in the southern half of the country. This induction provided instruction on specific aspects of the ESBN system. On arrival at the assigned depot, a local network induction was then carried out. The army and air corps were escalated on Thursday to provide additional assistance in Co Cork. Air Corps helicopters assisted in the patrolling of overhead networks damaged during the storm. This support supplemented the private helicopter resources that had been deployed over the first two days of the storm response. The army also significantly assisted in the restoration effort by providing resources to deliver materials and clear fallen trees and debris that had impacted the network. On Thursday, a new weather event (Storm Brian) was strengthening and notification was given by Met Éireann about the potential for impact on Saturday, October 21. By the end of October 19, 113,000 customers remained without power. Day + 4: Friday, October 20 The restoration effort continued and the affected areas narrowed primarily to the south of the country. The communications effort for customers that were still without supply shifted from the national sources to local media sources. Storm Brian was now forecast to hit the south of the country on Saturday, October 21, and concerns were raised that the incoming storm might hamper the existing restoration effort. However, ESBN committed to continue the restoration efforts until all customers were restored. By the end of October 20, 65,000 customers had been without power for five days and a program was initiated to attempt to communicate with the most vulnerable and rural customers to ensure they were safe and aware of the continuing restoration efforts and ERT. Day + 5: Saturday, October 21 Met Éireann issued an orange weather warning in advance of Storm Brian making landfall on the south coast, but the storm had only a limited impact on the distribution system and on ESBN’s restoration efforts. An additional 7,000 customers lost power. At this point, all MV network faults had been restored by crews and only low-voltage (LV) faults remained. However, though each of the thousands of remaining LV faults affected a small number of customers, they required significant resources since the majority of LV faults were caused by trees on overhead lines. Staff fatigue and exhaustion continued to be an issue; the daily mandatory pre-work safety briefing concentrated on the dangers of working while fatigued. At various stages during the restoration effort, mandatory stand-down and work commencement times were enforced to manage the exposure of teams to long working days. By the end of October 21, 35,000 customers remained without power. [caption id="attachment_41655" align="alignright" width="300"] Figure 7: Some of the crew that worked on post-storm restoration[/caption] Day + 6: Sunday, October 22 Approximately five per cent of the remaining customers were still without supply. Forced rest periods were mandated for staff and work ceased at 7.30pm. At the end of October 22, 3,000 customers remained without power. [caption id="attachment_41656" align="alignright" width="300"] Figure 8: Trees collapsed on overhead line conductor[/caption] Day + 7: Monday, October 23 All reconnections were largely completed by Monday, October 23, and efforts continued to make permanent repairs to connections where supply had been restored on a temporary basis. Roles and functions were normalised across ESBN. A limited series of post-storm network patrols on portions of the network where no outages had been recorded commenced to assess the likelihood of any unreported network damage. The overseas mutual aid teams completed their tasks and began to return home. A rest day for staff was mandated following their return to their base. ESBN estimated that about €2 million worth of stock was used in the restoration effort including 2,000 poles, 350km of conductor, and 250 MV/LV transformers.

“The response to Ophelia, or any other storm, is dependent on the organization and commitment of all of our staff and partners. In this case, over 2,500 people came together and worked safely and efficiently to provide timely information and rapid restoration of electricity to all of our customers.” - Senan Colleran, head of distribution and customer services

Emergency operations data science team

A team of four data scientists were embedded for the first four days with the operations systems teams in NDCC. Over this period, they were tracking outage information and providing a series of bespoke reports for restoration teams around the country. They also developed a prioritisation algorithm that allowed the agreed restoration prioritisation approach to be built into the business intelligence systems, to allow the dynamic altering of priorities to produce lists of faults to be scheduled to restore electricity to the most critical customers. After four days in NDCC some of the data science team were deployed to the south of the country to assist local teams in the optimisation of resource deployment. They helped to ensure that the large numbers of deployed staff were effectively scheduled to the most effective fault locations. They produced time lapse animations of the progress of storm Ophelia as it caused faults across the country and the subsequent restoration of supply over the fol- lowing days.

Information and media communications

A detailed communication plan was developed before and during the storm by the information management section of the SEMT. During the storm, ESB communicated important public safety messages and updates on the restoration effort through a variety of media including: television, radio, online video, partnerships with online news resources, search advertising, the ESBN website and blog, Twitter, Facebook, and other social media. ESBN estimated a total reach of about 30 million impressions during the storm and restoration period. The news media were in NDCC for the whole day of the storm and they were continually briefed on the storm’s impact, restoration efforts, and public safety messages. As customers were restored and the affected areas were narrowed to the south of Ireland, local media were engaged for more localised messaging. It was essential to keep messaging focused and consistent across all media and the discussion topics were prioritised as shown in table below. The government were continually briefed via the NECG and continuous information was also exchanged with key stakeholders, such as the relevant local authorities, army, health service providers, and other utilities. On the day of the storm, there were 50,000 telephone contacts to ESBN by customers, the majority of which were by Interactive Voice Recording (IVR) but there were also 10,000 contacts by direct communication with call center staff and 10,000 faults logged or referenced online. This was a significant resource overhead and took considerable management to ensure correct and valid information was processed to allow resources to be planned and deployed as part of the restoration effort. Powercheck wesbsite and app ESB Powercheck is a Google Maps-based system of viewing outages on the Irish distribution system that includes precise information about the outage, customers affected, and the ERT, as well as a brief message on the outage. Faults can also be logged online directly by customers via the ESBN website. Logged faults can be viewed on the Powercheck website and a mobile application. On the day of the storm, Powercheck was viewed more than 900,000 times, as well as about two million times in the seven days after the storm. During the restoration process, Powercheck proved to be an effective means of communicating directly with affected customers about localised faults and the ERT.

Public and staff safety

Three deaths were directly attributable to storm Ophelia; two were a result of fallen trees and a third occurred when a man was clearing a tree. However, there were no deaths or injuries attributable to the bulk electricity system equipment. The potential risk to the public from grounded conductors was substantial for an extended period while restoration was in progress. The NDCC invoked special procedures during the first 24 hours of the storm to disconnect all sustained earth faults associated with these grounded conductors. ESBN was focused on ensuring that the risks were highlighted as often as possible since these type of weather events — and their impact on the electricity network — are relatively rare. The key element of ESBN’s public safety messages during and after the storm concerned public safety and the messages continually emphasised the importance of reporting issues via the emergency call phone number or the Powercheck system. As the restoration effort progressed into the fourth day and beyond, fatigue and exhaustion were identified by managers as a safety risk to staff. ESBN responded to the issue and risk to staff by mandating compulsory safety briefings at the start of each day, highlighting the risk of working while fatigued, compulsory rest periods, mandatory finish times, and rest days after the restoration was largely complete.

Putting the event into context

As resilience continues to be a major factor influencing the design and operation of transmission and distribution power systems, lessons learned from this experience provide insight into how system operators can plan for extreme events with the aim of restoring supply to affected customers as quickly and safely as possible. The experience demonstrated how convening and staffing a Storm Emergency Management Team — with clear roles and responsibilities — in advance of the event can be critical to effective post-event restoration efforts. Communicating with the public across as many media platforms as possible is also recommended in order to increase public awareness of the safety risks and the estimated restoration time for their supply. A clear and consistent messaging plan was enacted for storm Ophelia — both during and after the storm — and communications channels with key stakeholders in government and public utilities were established and maintained throughout the restoration effort. The ‘Engineers Journal’ gratefully acknowledges that this article has been republished with kind permission from the Electric Power Research Institute (EPRI) The EPRI would like to sincerely thank ESB Networks for providing excellent information and insights to prepare this technical brief.