Energy data management: what it is, and what a platform has to do

Energy data management is the discipline of getting every number that describes how an organization buys and uses energy into one place, in one structure, checked, and ready for whoever needs it next. The phrase sounds administrative. In practice it decides whether the energy manager, the AP clerk and the sustainability analyst are working from the same figures or from three spreadsheets that disagree.

This article is written for organizations that buy energy: property owners, campuses, retailers, manufacturers, municipalities and the consultants who serve them. Utilities use the same words to mean something different, and we cover that overlap below. For a buyer, the job is to turn bills, meter files, portal exports and a handful of reference tables into a dataset you can budget from, benchmark from and report from without re-keying anything.

What follows is a working definition, the data types, the lifecycle, the six things a platform must do, the ways programs fail, who should own the result, and how to evaluate before you commit.

What energy data management means when you are the customer

One vendor definition captures the shape of it: an energy data management system centralizes energy-related data and prepares it for analysis, reporting, operations, and downstream business processes. The important word is prepares. Collecting is the easy half. The value is in the preparation: every value in a typed field, every unit on a common basis, every figure checked against something independent, every record traceable to its source document. Without that, a central store is a pile, not a system.

For utilities, the same term overlaps heavily with meter data management: the validation, estimation and editing of interval reads coming off the meter network, which we cover in meter data management basics. A buyer rarely sees that layer. What a buyer manages is the customer-side record: what was billed, what was metered at its own points, and the context needed to interpret both.

The data types you are actually managing

Energy data for a portfolio is at least seven streams, from different sources, on different cadences, with different failure modes. The table lists them with what a managed version of each looks like.

The mix matters because commercial sites depend on more than electricity. Across US commercial buildings, electricity accounted for 60 percent and natural gas for 34 percent of energy use in 2018, so a complete record for one site means at least two bill formats, two unit systems and two billing cycles before water, steam or submeters are added. In a Canadian portfolio, where heating dominates, the gas record matters as much as the electric one.

The lifecycle: collect, structure, validate, normalize, store, serve

Whatever the platform, the data passes through six stages. Knowing them lets you ask a vendor where each happens and who is responsible when it fails.

1. Collect: pull bills, interval files and reference data from every source on a schedule, including the providers that only offer a PDF.
2. Structure: convert each document into typed fields (usage, demand, reads, dates, charges, rate class) rather than text on a page.
3. Validate: check every record against something independent: the meter read, the tariff, the prior period, the site master.
4. Normalize: put everything on a common basis: units, currencies, calendar periods, and weather adjustment where the use case needs it.
5. Store: keep the raw source, the structured record and every correction, with lineage between them.
6. Serve: deliver the same dataset to finance, facilities and sustainability tools through exports and APIs, not copies.

How to design each stage is covered in designing an energy data pipeline from ingestion to insight. The stages that consume the most effort are the second and third. Across data work generally, a 2021 Anaconda survey found respondents spending 39 percent of their time on data prep and data cleansing, more than model training, selection and deployment combined. Utility data is harder than average because of format sprawl: almost 3,000 electric distribution utilities were operating in the United States in 2017, each with its own bill, and every Canadian province adds its own providers and layouts.

Six things a platform has to do

Strip away the dashboards and a platform earns its keep by doing six things reliably. Whatever it cannot demonstrate on your own documents will end up being done by a person.

• Read every source format you have, including bills that arrive only as PDFs, and keep working when a provider changes its layout.
• Hold a site, meter and account hierarchy that survives renames, reorganizations and meter swaps.
• Validate each bill and interval file against reads, tariffs and history, and route exceptions to a person with the evidence attached.
• Normalize units, periods and rate classes so a gas bill in cubic meters and an electric bill in kilowatt-hours can sit in one report.
• Keep lineage: any figure in any report traces back to the document, page and field it came from, and every correction is logged.
• Deliver the dataset to other systems without manual export: benchmarking tools, BI, the ERP, the GHG inventory.

The last item separates energy data management from a reporting tool. A tool answers its own questions. A managed dataset answers everyone's, which is why we argue for one clean dataset with many destinations rather than an import into each application.

Where programs fail

The failures are rarely dramatic. Collection outpaces validation: bills are loaded on time, nobody checks them, and a rate change or a carried-forward balance is discovered at year end. The hierarchy lives in someone's head, so when that person leaves, the portfolio quietly stops reconciling. Copies multiply: finance exports for accruals, facilities for KPIs, sustainability for the inventory, and each copy is corrected independently until none agree.

Two more are specific to utility data. Estimated reads get treated as actuals, so a tidy monthly figure hides three months of estimates trued up in the fourth. And Green Button is assumed to cover everything. It is a real standard, offered by utilities in nearly all 50 US states, two Canadian provinces and the Republic of Korea, but most bills still arrive as documents, and the standard carries usage rather than charges.

Who owns it: finance, facilities or sustainability

Three groups consume the same data and each has a claim on it. Finance pays the bills and books the accruals, so it cares about totals, coding and timing. Facilities runs the buildings and cares about usage, demand and anomalies. Sustainability builds the inventory and cares about consumption by fuel, factors and boundaries. When each keeps its own copy, the numbers diverge, and the divergence surfaces at the worst moment: an audit, a board report, a budget review.

The arrangement that works is one owner and three consumers. Often the owner is the energy or facilities team, because they can judge whether a number is physically plausible; sometimes it is a data team. What matters is that the owner is accountable for the checks and the hierarchy, and the consumers stop keeping private versions. The roles, stewardship and quality rules are set out in energy data governance for multi-site organizations.

How to evaluate a platform

Evaluate on your own documents, not the vendor's samples. Provide a month of real bills from your hardest sites, including a provider that only issues PDFs, a site with submeters, and one account with a known billing error. Then check five things: how many fields were captured, whether the error was flagged and with what evidence, how a layout change is handled, whether a report figure traces back to its page, and how the data leaves the platform. The properties to test for are defined in what clean, structured utility data requires.

Whether to build this in-house or buy it is a separate question, and the answer depends on how many formats you face and how long you can wait; the trade-offs are in build vs buy for utility and energy data management. In Canada the serving side has one more requirement. Portfolio Manager has been adapted for Canada since 2013, with more than 150 Canadian weather stations, and several provincial and municipal reporting programs expect data in that form, so the platform should serve it there directly rather than through a spreadsheet.

This is where MartinAI sits in the lifecycle. It reads every field on every bill, meter file and portal export for electricity, gas, water and steam, validates each record against reads, tariffs and history, normalizes it, keeps the lineage, and delivers the result to the benchmarking, BI, ERP and emissions tools a team already uses. The point is one dataset that finance, facilities and sustainability can all defend.