The FERC EQR (Electric Quarterly Report) compiles the details of electricity contracts and transactions between utilities, merchant generators, and grid operators. The EQR is one of the most detailed publicly accessible electricity market data sets available. It includes high time resolution information about the economic value of renewable energy resources under a wide variety of market and environmental circumstances.
EQR is reported quarterly by electricity market participants, within 1 month of the end of each quarter. It is available for bulk download as comma separated value (CSV) files in current format from 2013Q3 to the present. Data for 2013Q3-2018Q3 is ~200GB, and currently growing by ~20GB per quarter.
Why is it useful/interesting?
In combination with bottom-up marginal cost of electricity estimates derived from other data sources, the market prices recorded in EQR should allow for the estimation of profits or losses accrued by individual generation resources, and indicate how they have changed in response to market conditions (renewable energy deployment, low natural gas prices, extreme weather events, stagnating load growth, etc.).
EQR contains information about renewable energy purchases which are not widely published. EQR includes information about electricity products other than energy and capacity, which may become more important as complex differentiated markets develop. The dataset is large enough to both require and benefit from recent developments in machine learning algorithms and Big Data processing techniques.
Who would use it?
- Users trying to understand electricity market dynamics, including what factors affect real time pricing of various power attributes.
- Users interested in tracking the market price of renewable energy over time, and in different jurisdictions subject to different environmental circumstances, regulatory regimes, and policy interventions.
What information is included in this dataset?
- Terms of contracts between merchant generators, utilities and ISOs. Includes contract start and end dates, date of execution, pricing mechanism, product being transacted, terms of contract renewal, point of delivery and receipt, etc.
- Details of approximately 1 billion individual electricity market transactions, including the names of the buyers and sellers, what power attributes are being bought and sold, their prices, quantities, the time and locations of product delivery and receipt, and the ID of the contract under which the transaction took place.
Risks & Challenges
The primary challenge with the FERC EQR dataset is its size. At ~200GB, it cannot be easily manipulated or queried on a personal computer, and network bandwidth constraints make redistribution of the data challenging. Integration and utilization of this dataset as a unified whole can most easily be performed on a cloud based platform, or a dedicated high performance computing resource.
As with other FERC datasets, transacting entities are identified by name rather than a unique ID, so there may be some work required to create unique IDs. While there are a billion recorded transactions, there are only thousands of market participants to be identified. Delivery locations identified by name (e.g. “Nobles County Substation”), and are not explicitly associated with a physical location. Open data services such as the Wikimapia API exist, but it may not be possible to find complete and accurate locations based on place names and other available information.
Each transaction lists a buyer and seller, but does not explicitly indicate the source of electricity — power plant, fuel type, etc. Additional attributes of the power being transacted would need to be inferred from the names of transacting entities (e.g. “Zephyr Wind LLC” is likely a wind energy provider) and the nature of the contracts (e.g. a 20 year fixed price contract for energy only with no associated capacity is likely a renewable energy PPA).