This article is a collaborative effort by Adam Barth, David González, Jose Luis Gonzalez, Viktor Hanzlík, Gonçalo Pinheiro, Humayun Tai, and Alexander Weiss, representing views from McKinsey’s Electric Power & Natural Gas Practice.
Utilities confront two significant challenges when integrating RES into electric grids. First, they face network inadequacy, with a lack of physical capacity to accommodate supply and demand in locations with the best resources. Second, as the share of RES increases, the lack of real-time network management at low voltages could lead to network instability, which may affect high reliability standards and cause voltage instabilities, frequency inconsistency, and harmonic distortion of the power system.
Grids were not originally set up for such a fast-paced energy system; their tools and processes were developed in a slower, less volatile world. In this article, we examine the challenges that grid operators are facing across planning, connection, and operations, and explore coordinated solutions to benefit from the rapidly increasing need for RES.
To support RES integration into grids, grid operators may need to rethink their status quo and tools across their planning, connection, operation, and coordination approaches (Exhibit 1).
The energy transition is changing the landscape of electricity generation. As decarbonization drives RES demand, RESs are expected to account for 45 to 50 percent of the global power supply by 2030 and 60 to 70 percent by 2040. 2 Global Energy Perspective 2023, McKinsey, November 2023. With such booming demand, RES installed capacity could grow nine times from 2020 to 2050 (Exhibit 2). The transition to RES, coupled with economic growth, will likely cause electricity to grow rapidly. 3 Global Energy Perspective 2023, McKinsey, November 2023. Power grids will need to expand to meet the increasing demand for electricity and renewable energy: to achieve net-zero emissions by 2050, countries would need to double their investment in transmission lines and other infrastructure to €550 billion per year by 2030. 4 Electricity grids and secure energy transitions, IEA, November 2023.
Almost 1,000 gigawatts (GW) of solar projects are waiting for connection across Europe and the United States (which is close to four times the amount of new solar capacity installed globally in 2022). In addition, 500 GW of wind installed capacity is waiting to be put into the grid (five times the amount built in 2022). 1 “A power grid long enough to reach the sun is key to the climate fight,” BloombergNEF, March 8, 2023.
In Europe, this is causing delays in the phase-out of fossil fuel power plants, longer lead times for supply-and-demand connections, and increased development costs of RES projects. The consequences of capacity constraints are evident throughout the region. In the Netherlands, for example, grid congestion has led to connection rejections and caps on RES output, and several zones in Amsterdam have reached their maximum capacity. 2 “A case study in Amsterdam provides lessons to help resolve grid congestion,” Amsterdam Institute for Advanced Metropolitan Solutions, May 2022.
Similarly, in the United States, the average time spent waiting for connection has almost doubled since 2015, and now exceeds three years. 3 “Queued up 2022,” Lawrence Berkeley National Laboratory, April 2023.
To harness the RES opportunity, RES need to be integrated into the grid—this is no easy task. Grid operators face two major constraints in this regard.
First, we have noticed that network inadequacy is a challenge for many operators (see sidebar “Network inadequacy”). At present, there is not enough physical grid capacity to accommodate supply and demand connections. This is primarily due to difficulties in optimizing grid capacity (currently designed for centralized, mainly fossil fuel generation) and inefficient grid planning, leading to less new RES capacity being built than needed.
Network instabilityAn incident in the United Kingdom stands as an example of network instability—the country faced a major blackout in 2019, triggered by unexpected shutdowns of the Hornsea offshore wind farm and the Little Barford gas-fired power plant. This resulted in the collapse of much of the system, with one million electricity customers affected. The shutdown lasted 40 minutes, impacting several public services including Ipswich Hospital, Newcastle airport, and the rail system in London. 1 Jillian Ambrose and Kevin Rawlinson, “Transport chaos across England and Wales after major power cuts,” Guardian, August 9, 2019; Nina Chestney, “UK power grid, train systems need review after August blackout: Grid operator,” Reuters, September 10, 2019.
Another example can be found in Australia in 2018, where a high penetration of wind and solar plants (comprising mainly residential rooftop solar installations) contributed to considerable voltage fluctuations. The higher voltage caused appliances to reach saturation, with the excess being burned off as heat. Although the energy was not used by the customers, the excess resulted in inflated energy bills. In addition, the quality of the power supply dropped (causing lights to flicker and dim). An investigation of around 12,000 electricity meters that was conducted to understand the impact of voltage fluctuations on consumers showed that around 14 percent of meters had readings above the maximum voltage limit. 2 Liz Hobday, “Power bills up? Appliances burning out? You may have voltage problems,” ABC News, November 7, 2018..
Second, operators contend with network instability (see sidebar “Network instability”). Increased penetration of intermittent power sources, such as solar and wind, have caused a higher utility frequency and voltage volatility. 5 Nina Zalaznik and Anoop Gangadharan, “Integrating renewables with DER voltage control,” Landis+Gyr, January 12, 2023; Krzysztof Lowczowski, Voltage fluctuation and flicker in prosumer PV installation, ResearchGate, March 2022. Operators are facing the problem of having fewer “rotating masses,” losing the ability to stabilize the system due to the phase-out of current balancing assets, such as thermal generation. 6 “Electricity security matters more than ever,” IEA, 2020. They also dealing with the curtailment of distribution-connected plants and potential imbalances penalties. As a result, there’s a growing need for complex balancing services and restructuring of transmission and distribution system operator-coordination processes.
Would you like to learn more about our Electric Power & Natural Gas Practice?Grid operators face multiple challenges along the value chain that can potentially put them at risk of being underprepared for the energy transition. However, they have numerous avenues available to help them better plan, connect, and operate.
The tools and processes available at present for grid planning are not up to the task of optimizing current capacity and planning for the setup of efficient new capacity. Complex power flow models are not able to handle the long time frames, nor the uncertainties of today’s system regarding demand (absolute growth and profile effect) and supply (availability, location, and dispatchability) growth trends and could affect the future regulatory landscape.
This leaves operators unable to model uncertainty at a granular level across a wide variety of inputs, including:
These factors highlight the need for a more integrated grid planning approach (Exhibit 3).
Planners can draw on flexible and modular data architecture and AI-driven, stochastic optimization of investment plans, as these can assist in optimizing existing grid capacity and the buildout of new capacity using integrated grid planning.
Companies can redesign their data architectures to emphasize modularity (the ability to break down a complex system into smaller, more manageable components). This would allow for greater flexibility, scalability, and adaptability in the data architecture to help organizations achieve their data management goals more effectively and efficiently, including better cooperation with energy market platforms. The goal is to be able to operate with “data building blocks” as if the data were perfect so that, first, only one block needs to be updated when real information is available and, second, uncertainty on predicted data can be represented statistically based on the confidence of the prediction.
Such models combine macrogrid trends and microgrid dynamics to evaluate many options, helping to build confidence under uncertainty. This allows for better planning, enabling the modeling of a full year quantifying unpredictability of events such as asset failures and voltage fluctuations, which additionally can serve as a robust fact base for regulators and operators and provide valuable insights for informed decision making.
Stochastic grid modeling: A European exampleOne European distribution operator deployed a stochastic grid modeling approach to quantify grid impacts and determine the best potential grid upgrades in a specific region.
By deploying an advanced stochastic model, the grid operator was successful in a number of areas, including:
For example, these models can quantify the impact of several inputs—such as RES penetration, grid topology, and electric vehicle adoption—into the main Quality of Service (QoS) metrics (for instance, curtailments and energy not served), leveraging a stochastic approach (see sidebar “Stochastic grid modeling”). Models can include deep learning surrogate models to manage to bring down execution time from days to seconds when running full-year simulations across a medium and long time horizon.
Grid operators face various hurdles in their connection process, including outdated prioritization for connection analysis that does not account for future additions, insufficient digitization of the overall process, and a lack of standardization in equipment procurement and permitting processes.
To address these challenges, operators can optimize methodologies for hosting capacity estimation (for example, leveraging advanced grid planning models as described above); automate processes; leverage generative AI (gen AI) to improve customer interface and information flow; adapt connection requirements; standardize equipment; and ensure capital delivery excellence to accelerate the connection and permitting process. Executives can make the connection process more transparent for their customers in the following phases.
Operators could optimize their hosting capacity for renewable plant methodologies by adopting an integrated grid planning approach—supported by advanced analytics technologies—and by considering solutions currently not commonly used such as batteries. In parallel, operators could improve the visibility of grid constraints through publicly available information on grid congestions and hosting capacity in each point of the network. They could also define minimal requirements for RES developers to optimize the application process, while still staying within the applicable power quality requirements.
Operators could consider revising and optimizing connection requirements through conditional requirements. Such an example would be conditioning solar output to network congestions in specific times to increase the amount of renewable generation connected, without upgrading any grid equipment (such as transformers and power lines) and defining a predefined and standardized set of connection offerings to RES developers.
Operators could improve capital spending by better managing stretched supply chains (for instance, raw materials and components) and inflation risks. They could also apply a variety of design levers—such as equipment standardization—to optimize product design by applying design-to-cost and developing strategic partnerships with equipment suppliers. In addition, operators could focus on efficient cooperation with permitting authorities. New grid connections require many permissions during the application process and an efficient cooperation framework could increase the application success rate and minimize overall lead times.
Heavily centralized processes and limited data availability can lead to insufficient operational visibility that affects system stability. In general, distribution grid operators lack real-time information on power flows in the low-voltage grid and the operational practices that would allow them to maintain grid stability by managing grid assets connected to the network.
Operators could use system stability and network adequacy technologies—such as advanced inverters, prime exchangers, and virtual synchronous machines—and introduce innovative flexibility services (for example, inertia procurement, congestion services, and flexibility tenders) to address physical adequacy bottlenecks, improve stability, and streamline day-to-day operations.
Operators could also consider technologies that increase their ability to manage the network, or look at unconventional technologies as alternatives to upgrading power grid lines and substation capacity. Advanced grid technologies can be categorized into four groups, varying by impact and maturity: advanced components; system monitoring and optimization; distributed energy resources (DER) integration software; and energy storage technologies (Exhibit 4).
Advanced transformers, grid management, and energy storage are high-maturity, high-value-pool solutions. These could help grid operators integrate renewables into the system where grid monitoring presents itself as a key enabler to gain visibility into the power grid status and improve grid operations across their value chain (for instance, network planning and maintenance). Some regions, such as the United Kingdom, have already started to incentivize power operators to monitor low-voltage networks to support electric vehicle and renewable generation into the grid. They do so by installing smart devices with computing edge capabilities, coupling both the required field devices needed to capture the data on site with a fit-for-purpose telecommunications network that aims to gain visibility into the low-voltage grid.
Grid operators could use flexibility services to offer a more reliable, stable power supply to consumers by balancing supply and demand on the power grid in real time, reducing the need for higher-cost and higher-emitting peaking power plants. Flexibility providers could make temporary changes, when requested, in the way they consume, generate, or store electricity to support a more efficient use of the energy network (such as for congestion services and inertia capacity procurement). Effective coordination among key stakeholders—regulators, grid operators, and flexibility services providers, for example—would be required to set up the optimal regulatory framework and define the most efficient operational procedures among them, including data exchange protocols, remuneration, and penalties.
Operators could ensure faster and leaner internal coordination by enhancing their current operating models and cooperating with main stakeholders of the energy system, improving information flow throughout the end-to-end RES integration process.
To ensure proper coordination in the energy system, operators can focus on people, structure, and systems and processes.
In terms of people, operators can review their talent and capabilities management and organizational culture. This might involve hiring new talent and upskilling existing employees to ensure they have the skills and knowledge needed to succeed in the rapidly changing energy market.
Regarding structure, operators can set up a governance framework that provides guidelines on responsibilities and establishes a cross-functional team collaboration model. Operators can adopt different approaches (as below) to adapt the workforce and increase cooperation between different departments to enable the renewable integration strategy, depending on the degree of disruption compared with their current organization.
First, operators can implement dedicated periodic meetings between departments (monthly, for example). This would increase collaboration among business units on renewable integration (while keeping decision making within the departments), help address the company’s renewable integration priorities, and raise concerns when complications arise.
Second, operators can set up a renewable integration task force comprising department members. This team would be in charge of decision making, while departments would collaborate by raising concerns and potential pain points with the task force and provide support on decision making.
Third, operators can execute an agile, cross-functional operating model with autonomous “squads” (for example, a northern region). This would combine people from different departments into smaller, more dynamic teams for end-to-end accountability in the pursuit of the squad’s purpose.
Finally, in terms of systems and processes, operators can focus on improving and streamlining process design, as well as automating and digitizing systems and technology. This could help to eliminate bottlenecks and improve efficiency, allowing operators to better manage the energy system and respond to changes in demand and supply.
To ensure proper coordination in the energy system, operators can work collaboratively with five key stakeholders—transmission operators, regulators, RES developers, flexibility industries, and power consumers—by employing six key actions:
Decarbonizing the global economy will require vast amounts of electricity to be generated, of which one solution is renewable sources such as solar and wind. RES infrastructures are already available and rapidly increasing. However, taking advantage of renewables requires a power grid that can accommodate these intermittent energy sources. Operators have a way to go to make this happen, but they can start now by rethinking their planning, connections, and operations and coordinating across stakeholders to construct a global net-zero power grid for the future.
Adam Barth is a partner in McKinsey’s Houston office; David González is a senior partner in the Madrid office, where Jose Luis Gonzalez is an associate partner and Gonçalo Pinheiro is a solution manager; Viktor Hanzlík is partner in the Prague office; Humayun Tai is a senior partner in the New York office; and Alexander Weiss is a senior partner in the Berlin office.
The authors wish to thank Carolina Diniz, Andrés Garcerán, Mikolaj Jasinski, Benjamin Lefrere, Sam MacIntyre, Theodoros Margiolas, and Erik Rossbach for their contributions to this article.
