Intelligently connected Using AI algorithms for data enrichment in energy systems

Topic was presented at Munich Datageeks by Linda Hong - April 2026

Abstract

This talk challenges the current AI hype by arguing that real business impact from AI can only be unlocked with a solid data foundation. Drawing on a project from E.ON's asset management division, the speaker shows how missing and inconsistent data plagues the company's electricity and gas grid assets, and how a machine learning approach was used to automatically fill these gaps rather than relying on infeasible manual data collection. The talk walks through the full project lifecycle: data cleansing, correlation and clustering analysis, a hybrid rule-based and machine learning imputation approach, validation against a held-out data set and business stakeholders, and the enterprise infrastructure (Azure, Databricks, GitLab, Power BI) used to operationalize the solution. The core message: without addressing data quality first, AI applied to "garbage" enterprise data will only produce garbage results.

About the speaker

Linda Hong is a senior data engineer at E.ON, working within a data and AI organization of over 170 people. Before joining E.ON, they worked in consulting at BCG. Their academic background combines data science with a bachelor's degree in international business studies, and they most recently worked in E.ON's asset management area on a data quality project applying machine learning to enhance and improve the quality of asset data.

Transcript summary

Motivation: AI hype vs. the data foundation

Linda opens by noting that while AI is widely discussed for solving impressive problems (from curing cancer to speeding up customer complaint resolution and code completion), the data foundation underlying these applications is rarely discussed. In their view, real impact from AI can only be unlocked with good data quality. Marketing narratives suggest a simple pipeline of data plus AI equals a solution to everything, but the reality inside enterprises is messier: garbage or incomplete data, disconnected legacy systems, and a lack of interconnection. Applying AI on top of this kind of data produces correspondingly poor ("garbage") output.

E.ON's business structure

To provide context, the speaker outlines E.ON's three business pillars:

1. Energy networks – operating the electricity and gas grid, which generates most of the company's revenue and requires substantial infrastructure investment, partly driven by the growing electricity demands of AI itself.
2. Energy infrastructure solutions – custom, proprietary solutions for large energy consumers with specific infrastructure needs.
3. Energy retail – the sale of energy to customers, which is the most publicly recognized part of the business.

The data quality problem in asset management

Working in asset management, the speaker's team is responsible for operating and managing a large volume of physical grid assets. A new asset management system revealed that asset data was highly incomplete, with a large share of missing values. Several structural reasons explain this:

• Long asset lifecycles: equipment such as transformers can be in use for several decades, and historically there was no requirement to record detailed data about them.
• Inconsistent business models: different parts of E.ON operate with different data practices, and the company is currently working toward a standardized, uniform data model.
• Increasing regulatory requirements: as operators of critical infrastructure, E.ON faces growing regulatory demands from both European and German authorities, which increases the need for complete, high-quality data.

Scale of the challenge

The speaker illustrates the scale of the problem using the mid-voltage grid level as an example: 76 distinct object types exist, described by 762 fields in total. For electricity stations alone, there are 150,000 physical stations described by 26 descriptive columns, many of which contain gaps. Manually reviewing and completing this volume of data field by field is not feasible. One alternative considered was having field technicians fill in missing data during routine maintenance visits, but this was judged impractical given the scale (150,000 stations) and the fact that data quality checks are not a task field staff are typically motivated to prioritize. This led the team to explore a machine learning approach to automatically fill in these data gaps.

Project methodology

The project followed a structured machine learning workflow:

1. Data cleansing
2. Outlier detection
3. Correlation and clustering analysis
4. Model training (multiple models tried)
5. Model application and business validation

The result of the process was a recommended value for each missing field, for each relevant object.

Data cleansing and correlation analysis

During data cleansing, the team performed standard data type transformations, examined outliers, corrected data quality issues where identified, and applied business logic to outlier detection. For correlation analysis, they used a classical correlation matrix to identify fields that were strongly (positively or negatively) correlated with each other, since strongly correlated fields are good candidates for inferring missing values from existing ones — analogous to inferring a person's weight from their height, given the typical positive correlation between the two. The correlation analysis confirmed relationships that also made sense from an electrical engineering perspective, which validated the approach on the data side.

Modeling approach: rules first, machine learning second

For imputing missing values, the team used a hybrid approach combining rule-based logic and machine learning, giving priority to rules whenever they were applicable. Given that the data supports critical infrastructure, the speaker emphasizes that hardcoded rules should always be preferred over machine learning when they reliably solve the problem. Two types of rules were used:

• Business rules, where the value of one field logically determines another.
• Hierarchy-based rules, where missing information at one level (e.g. a transformer) can sometimes be derived from related information at another level (e.g. the station it belongs to).

Where rules did not apply, two sets of input features were tested for the machine learning models: a set of clustered variables derived from the highly correlated fields identified earlier, and a fixed baseline set of variables considered generally important for prediction.

Validation approach

The team used a classical validation setup: for each variable to be imputed, 10% of the available data was set aside as a validation set. Performance was measured using the F1 score for categorical variables (higher is better) and mean absolute error for numerical variables (lower is better). The imputed results closely matched the original distribution of each variable, for both numerical and categorical fields, and the team noted that most of the performance improvement came from the data cleansing and correlation work done earlier, rather than from the choice of model itself.

Business validation and impact

Once satisfied with the model results, the team shared representative samples of the imputed data with business stakeholders for validation, who confirmed whether the results were acceptable. Given the sheer number of technical objects involved, this automated approach had a significant impact, since manually checking and completing this volume of data would not have been possible.

Technical infrastructure

To ensure the solution was operational rather than a one-off notebook exercise, the team built a production-oriented infrastructure: the SAP asset database served as the primary data source; the overall solution was built on Azure; computations and model training were conducted in Databricks; code was managed in GitLab; proprietary databases were maintained for each business area and object type within Databricks; and a Power BI dashboard was used to present and share the results.