Integrating a utility data API: what the payload should contain and how to build on it

Most guides to utility data APIs are written from the collection side: how to get bills and meter reads out of a utility. This one is written from the other side. You are a data engineer, a BI developer or an ERP integrator, a provider is going to hand you structured utility data, and you need to know what the payload should contain, how it is delivered, and how to land it in a warehouse, Power BI, an ERP or an ESG tool without rebuilding the provider's validation.

Treat this as the consumer's contract. If a field on this list is missing from the payload, you will end up reconstructing it from PDFs. For how the data is gathered, see automating utility data collection; for the internal stages after it lands, see designing an energy data pipeline.

What a good payload contains

A utility bill is a hierarchy, and the payload should preserve it rather than flatten it into totals. The public reference for the usage half is Green Button, which is based on the Energy Services Provider Interface (ESPI) standard released by NAESB in the fall of 2011 and covers electricity, gas and water. Its data model separates the usage point (service location and meter), the meter reading, the reading type (what is measured, in what unit, at what interval) and the interval blocks of values. A bill payload needs the same discipline plus the financial layer.

• Account and customer: utility account number, customer name as billed, service address, and your own site and organization identifiers
• Meter: meter number, commodity, multiplier, and the meter's active date range so meter swaps are explicit
• Bill period: start date, end date, days, bill date, due date, and whether each read was actual or estimated
• Reads and quantities: start and end reads, consumption, demand (billed and metered), power factor where billed, each with a unit
• Line items: every charge as its own record with description, quantity, unit, rate, amount, tax treatment and the tariff component it belongs to
• Tariff and rate class with effective dates; currency and totals (current charges, prior balance, payments, amount due)
• Source document reference: an identifier for the PDF, portal export or Green Button file the record was built from, with page references where possible
• Validation status and flags: whether the record passed the provider's checks, and which flags were raised (estimated read, period gap, unit change, tariff mismatch, duplicate)
• Lineage: when the source was received, extracted and validated, and the version of the record

Units, scaling and normalization

Units belong on every quantity, not in the documentation. Gas arrives as volume on some bills and energy on others; water as cubic meters, gallons or hundreds of cubic feet. Green Button is instructive here: the format applies power-of-ten multiplier scaling to readings and separate scaling to cost and currency, so a consumer that ignores the multiplier reads every value off by orders of magnitude. Your payload should deliver a normalized value alongside the original with the conversion factor stated: the bill's own figure for reconciliation, the normalized figure for analysis, and never one overwriting the other. The same goes for names: rate class labels, charge descriptions and meter identifiers drift between bills from the same utility, so carry both the canonical value and the source string. Warehouse-side mapping rules are in building a utility data warehouse.

Identifiers, idempotency and versioning

Utility bills get corrected, cancelled and rebilled, and an integration that cannot express that will double count. Three rules keep it honest. Every bill record carries a stable identifier that does not change when the content is corrected, plus a version number that does; Green Button's guidance on UUID generation exists for the same reason. Deliveries are idempotent: receiving the same bill version twice has no effect, so the consumer can safely retry. A correction arrives as a new version that references the one it supersedes, with a reason (rebill, cancel, estimated-to-actual true-up), and the consumer marks the prior version superseded rather than deleting it. Interval data follows the same pattern: a re-read of a day replaces that day's block by identifier.

Delivery patterns: webhook, polling, scheduled export, backfill

Delivery depends on how downstream systems consume: a finance system wants a complete batch to post, a dashboard wants the latest record as soon as it exists, and a warehouse wants both plus the ability to reload history.

Insist that the backfill and the incremental payload share one record shape; a provider that exports history in one format and streams updates in another has handed you two integrations.

Interval data specifics

Interval data is where integrations quietly break. Green Button providers deliver readings in 5-minute, hourly, daily or monthly intervals, depending on what the energy provider decides to make available, and Canadian smart meters commonly produce hourly or 15-minute data. The payload should state the interval length per reading type rather than leaving you to infer it from timestamp spacing. Every timestamp needs a UTC offset, and the contract should say whether it marks the start or the end of the interval. Daylight saving transitions produce a 23-hour day and a 25-hour day each year; local time without offsets will lose or duplicate an hour, and that day's demand peak will be wrong.

Keep energy and power distinct: a 15-minute value in kilowatt-hours is not a 15-minute average in kilowatts. Carry a quality code per reading (actual, estimated, missing, edited) so a gap is visible rather than filled with zero. And since intervals and bills describe the same meter over the same period, expose the interval-to-bill reconciliation as a field rather than leaving each consumer to compute it.

Error handling and reconciliation

The validation status on each record is the consumer's routing key: passed records load straight through, flagged records go to a review queue or quarantine table with the flags attached, and structural failures are rejected with a reason the provider can act on. Reconciliation then runs at three levels. Record level: line items sum to the bill total, and reads and multiplier produce the consumption. Period level: consecutive bills on a meter abut with no gaps or overlaps, and the interval sum matches the bill. Ledger level: the amounts your ERP posted for a period match the bills the API says exist for it, so a missing invoice or duplicate payment is caught by comparison. Each level needs the identifiers and versions above, which is why they are in the contract.

Security basics

Green Button's Connect My Data flow is a reasonable model: the customer gives a one-time authorization and the third party receives data through a transfer built on the IETF OAuth Authorization Framework; the Alliance's developer page references OAuth 2.0 RFCs 6749 and 6750. NAESB describes ESPI as a data exchange protocol for the transfer of energy usage information between a utility and a third party with customer authorization. For your own integration the rules are ordinary: OAuth 2.0 client credentials or scoped API keys rather than shared passwords, tokens scoped to the accounts and sites the consumer needs (least privilege), secrets in a vault, scheduled rotation, TLS everywhere, signed webhooks, and an access log, because utility data includes customer names and addresses.

Reference mapping: payload fields to destinations

The warehouse takes everything; the other destinations take a projection.

The ERP and BI columns are expanded in integrating utility data with ERP and Power BI for utility data. Benchmarking tools add their own rules: ENERGY STAR Portfolio Manager wants 12 full calendar months per meter, and a mid-month billing cycle needs 13 bills to cover them, so deliver the full period, not just the calendar year. And where a utility already pushes to Portfolio Manager through its web services API, ENERGY STAR notes it typically delivers aggregate whole-building consumption data as opposed to meter-by-meter data, so your meter-level payload complements that feed.

Where MartinAI fits

MartinAI is the provider side of this contract. It reads every field on every bill and meter file for electricity, gas, water and steam, normalizes units and names while keeping the originals, validates each record, versions corrections and rebills, and delivers structured records with validation status, flags and a link to the source document.

A utility data API is only as useful as the fields it carries and the guarantees behind them. Ask for the full hierarchy, explicit units, versioned identifiers, validation flags and lineage, and the destination mappings become configuration rather than engineering.