Talk "Forecasting and optimization to balance a renewables-powered electricity grid using batteries"

Forecasting and optimization to balance a renewables-powered electricity grid using batteries by Danil Vityazev & Wolfgang Ridinger was presented at Munich Datageeks - April Edition 2025

Abstract

Large-scale battery systems are becoming an increasingly important feature of the electricity grid as they balance intermittent production from renewables with demand for electricity. Batteries can serve the grid through different electricity markets, whose relative attractiveness differs each day. In this talk, we will outline how we forecast conditions in the electricity system each day and then use optimization on these forecasts to deploy the battery for maximum impact over the course of each day.

About the speaker

Wolfgang Ridinger is a Principal Data Scientist at Entrix who combines tools from Data Science, Optimization, and Economics to maximize the impact of batteries on the electricity system.

Transcript summary

Overview and Team Introduction

The Entrix data science team consists of five people with plans to grow to 10-12 members by year's end. The team brings diverse backgrounds including physics, mathematics, engineering, economics, and architecture. A distinguishing feature of Entrix and its sister company Flexa is that the data science team forms the core of their value proposition. The presentation showcased a project combining forecasting and optimization to balance renewable energy-powered electricity grids using batteries.

Grid-Scale Batteries: Infrastructure and Characteristics

Grid-scale batteries represent a relatively recent development in energy infrastructure. These systems consist of shipping containers filled with battery cells, placed on empty land parcels and connected directly to the electrical grid. Their primary function is energy storage and release based on grid requirements.

The scale of these installations is substantial, with planned facilities in Germany approaching the capacity of large conventional power plants. Two critical specifications define battery systems:

1. Power capacity: The amount of work a battery can deliver at any instant
2. Energy capacity: Power multiplied by time (power is the time derivative of energy)

A typical modern grid-scale battery has a duration of two hours, meaning it can provide full power output for two hours. This represents an energy-to-power ratio of 2:1, which is currently the standard configuration.

Electricity Market Structure

The electricity market ecosystem is complex, with multiple markets serving different purposes. The relevant markets for grid-scale batteries divide into two main categories:

Wholesale Markets: These handle energy shifting at longer time scales, typically measured in hours. A common use case involves shifting excess solar energy from midday (when production peaks) to evening hours (when residential demand increases as people return home and use appliances).

Frequency Response Markets: These address short-term grid stability by reacting to supply-demand fluctuations at the second level. Two specific frequency response markets are relevant:

• FCR (Frequency Containment Reserve)
• aFRR (automatic Frequency Restoration Reserve)

Market Timing and Dependencies

The temporal structure of these markets creates forecasting challenges. Frequency response markets require bids in the morning, while wholesale markets open later in the day. This timing gap means that when deciding on frequency response market participation, the wholesale market prices are still unknown. Therefore, forecasting wholesale market prices becomes essential for optimal decision-making, as these markets represent alternative revenue opportunities for the battery.

Market Interactions and Constraints

The different markets impose various constraints on battery operations. When participating in one market, specific rules limit actions in other markets. These constraints manifest as:

• Energy limits imposed by one market on others
• Power limits across different market combinations

These constraints form the foundation of the optimization problem that must be solved to maximize battery revenue across all available markets.

The Dual Challenge: Optimization and Forecasting

The project addresses two interconnected tasks:

Task 1: Optimization Problem The objective is to maximize total revenue from both frequency response markets and wholesale markets. The frequency response revenue component is straightforward - a linear relationship where revenue increases proportionally with committed capacity (X1 and X2 for FCR and aFRR respectively), with slopes determined by market prices. However, wholesale market revenue is more complex, represented by a function R that depends on frequency response commitments and is constrained by the previously mentioned market interaction rules.

Task 2: Price Forecasting Since market prices are unknown when optimization decisions must be made, forecasting models are required for four different price streams. These forecasting and optimization tasks, while separate, must work together seamlessly.

Price Forecasting Methodology

The forecasting approach recognizes that frequency response and wholesale market prices are highly correlated despite clearing at different times, as they share common fundamental drivers:

• Expected renewable energy production
• Expected consumption by large consumers
• Infrastructure availability and maintenance schedules

One particularly useful predictor is hydro-pump storage plant availability. Since these plants are major participants in the aFRR capacity market, their planned maintenance schedules (which are publicly available and announced well in advance) significantly impact prices. Dozens of such market indicators inform the forecasting models.

Model Architecture: The team primarily uses XGBoost models, incorporating features ranging from weather data to autocorrelative features derived from historical price patterns.

Training Data Strategy and Market Volatility

A crucial aspect of the forecasting approach is the limited training window. Models are trained on recent data starting from February of the previous year - typically no more than six months of historical data. This constraint exists because electricity markets exhibit substantial volatility, with fundamental driver relationships changing drastically over time.

The aFRR capacity market price evolution from 2023 to early 2025 illustrates this volatility. Prices were significantly higher in early 2023, reaching levels never seen again afterward. This shift resulted from changes in market rules - specifically, modifications to market clearing mechanisms and participant eligibility. When more players were allowed to participate after mid-2023, prices dropped permanently. A model trained on 2023 data would fail to perform accurately in 2025 because the underlying market dynamics fundamentally changed.

Similarly, the intraday continuous wholesale market has shown increasing volatility over time, reflecting greater integration of photovoltaic and wind energy into the grid. These renewable sources introduce less predictability into the system, requiring models to adapt to evolving market dynamics rather than relying on older historical patterns.

Forecasting Challenges: Data Availability Constraints

Beyond market volatility, the team faces practical data availability issues. FCR clearing prices serve as strong predictors for aFRR prices since these markets clear just one hour apart. However, when decisions must be made for aFRR market participation, FCR clearing results are not yet publicly available - the data exists because the auction has cleared, but dissemination to the general public is too slow for operational use.

Optimization Problem Formulation

The optimization problem centers on determining optimal bid allocations (X1 for FCR, X2 for aFRR) across frequency response markets, which must be decided before continuous wholesale markets open.

The fundamental constraint is that the total capacity committed across all markets cannot exceed the battery's power capacity. For example, with a 10-megawatt battery, a 5-megawatt commitment to FCR leaves only 5 megawatts available for other markets. If committed capacity proves unprofitable based on realized prices, the battery operator faces suboptimal returns.

Operational Trade-offs: State of Charge and Power Constraints

Real operational data from Entrix assets demonstrates the practical trade-offs between markets:

Unconstrained Operation: Without frequency response market commitments, batteries can freely charge and discharge at any power level and to any state of charge percentage, optimizing purely for wholesale market opportunities.

aFRR Market Constraints: Participation in aFRR restricts the battery's state of charge profile. The battery cannot charge or discharge to arbitrary percentages, pushing operations away from the optimal state of charge curve and limiting wholesale market flexibility. The compensation is revenue from aFRR capacity payments.

FCR Market Constraints: FCR participation constrains available power rather than state of charge. The battery cannot discharge or charge at full power capacity but maintains flexibility in state of charge management.

In practice, batteries participate in both markets simultaneously, and optimization decisions must determine the optimal balance of constraints on state of charge and power at different times throughout the day, based on forecasted prices across all markets.

Future Research Directions

The team is actively developing several enhancements to their current approach:

Probabilistic Forecasting: Current optimization relies on expected price values, but actual prices follow distributions around these expectations. Incorporating probabilistic forecasts would account for price uncertainty in optimization decisions.

Short-term Forecasting: Enhanced short-term predictions would improve performance in continuous wholesale markets where trading opportunities arise throughout the day.

MLOps Infrastructure: Development of systems for more frequent model updates and more accurate performance monitoring to maintain forecast quality as markets evolve.

Advanced Optimization Techniques: Exploration of reinforcement learning and stochastic optimization methods to discover alternative approaches and capture additional value from market participation.