Version 2 Summary

[dt-woods]

The following summarizes the updates and improvements made to the ElectricityLCI Python package completed as a part of the version 2, which included over 20 new issues on GitHub over the past six months¹. The updates since March 2024 resolve over 16 tracked issues with the goals of preserving the 2016 electricity baseline and refining the 2022 baseline. Several quality improvements, fixes, documentation updates, and miscellaneous changes to the code were made over this period. The key changes include:

1. Improve module and method documentation throughout the package (address Complete missing documentation #204).
2. Adopt a new temporal balancing authority list with FERC mapping using EIA's 930 reference table (resolve How to map balancing authorities within EIA and FERC regions? #215 and NCR ID column name mismatch #194).
3. Remove industrial co-generating facilities from inventory list (resolve Addressing the Industrial Cogeneration Problem and Implementing the filter in model_config #232).
4. Correct output exchange flows labeled as resources (resolve Fix output exchange flows mislabeled as resources #233).
5. Correct the electricity flow for at-grid consumption mixes from 120 V to 2300--7659 V (resolve Incorrect output flow for "at grid; consumption mix" #238).
6. Incorporate missing metadata from the Federal Elementary Flow List (e.g., CAS number, formula, synonyms, and FEDEFL version) (resolve FLCAC Metadata Requirements #239 and error accessing FLCAC api in _read_fedefl() #257).
7. Remove problematic electricity from NETL's 2016 plant water inventory (resolve Issues with the electricity column in generate_plant_water_use()  #242).
8. Implement 2023 coal model transportation and extraction and processing LCIs (address Update inventories | Coal #243).
9. Create facility-level coal transportation LCI from 2023 coal model for coal facilities since 2016, gap-filled with U.S. average transportation modes and distances (address Update inventories | Coal #243).
10. Correct the temporal representativeness of inventory processes to match the reporting agencies' inventory vintages and not the configured LCA modeling year (resolve Set temporal representativeness attribute for processes to inventory vintage, not target year #244).
11. Correct inventory sourcing, emissions aggregation, and facility matching across variable inventory years (resolve Who's putting NaNs in my exchange tables? #246).
12. Correct the unit conversion, aggregation method, and omit nuclear, wind, solar, and solar thermal from plant efficiency filter (resolve Error in calculate_plant_efficiency #247).
13. Introduce the Hawkins-Young uncertainty calculation for emissions inventories and make its calculation configurable (resolve Add configurable uncertainty calc #248).
14. Remove legacy code and outdated datasets (resolve What is the intended use of technosphereflows.py? #249).
15. Improve EIA bulk data filtering to support backwards compatibility---see EIA bulk data problem in Appendix (resolve Empty dataframe in eia_io_trading.py #251).
16. Implement EIA API call for U.S. Electric System Operating Data (resolve Introduce EIA API for managing U.S. Electric System Operating Data #253) and fix EPA API calls for CEMS data, including the reduction of data handling (i.e., daily data in place of hourly) and improvement of temporal representation (i.e., use U.S. time zone, not GMT).
17. Fix EIA923 fuel receipt CSV creation (resolve Error in reduced schedule CSV file #259).
18. Incorporate 2023 solar PV construction LCI fixes (resolve Update inventories | Solar PV construction #265).
19. Fix product flows labeled as elementary flows (resolve Product flows labeled as elementary flows in coal and natural gas extraction and processing #267).
20. Remove geothermal power plants from the efficiency filter, returning geothermal to the U.S. energy mix (resolve Add missing sources from renewables #278).
21. Remove electricity input flows from renewable generation processes (resolve Remove Electricity input from renewable O&M processes #281).
22. Purify power-plant construction processes found in fuel-specific generation processes, allowing multiple construction processes in mixed-fuel generation processes (resolve Clean up power plant construction at fuel-generation process level #282).
23. Scale renewable upstream operations and maintenace emissions on their data source year (resolve Renewable electricity O&M emissions scaling based on EIA year rather than source year #283).
24. Remove unmapped flows from the upstream coal inventories (address Remove non-FEDEFL elementary flows from coal mining and processing LCI #285).
25. Implement and test new regionalized power plant construction processes, with new renewable power plant construction metadata (address Split renewable upstream into separate system processes #150); see memory management challenge.
26. Add API sections and coal and renewable model vintages to model configuration YAML files and improve YAML file documentation.
27. Provide facility-level inventory return option in `get_generation_process_df` method (i.e., skips aggregation for facility-level analyses).
28. Correct the RCRAInfo inventory vintage from 2019 to 2021 for the 2021 and 2022 baselines in the YAML model configuration files.
29. Fix coal basin process proxy process gap-filling.
30. Correct zero-flow filtering to skip elementary flows.
31. Improve file and folder handling during startup (e.g., for saving new log files) and processing (e.g., missing FRS bridge file).
32. Add post-processing as a model configuration parameter.
33. Make main in main.py a runnable method.

Unresolved new issues include failing to read the XML format of EIA's public coal workbooks (#230), adding energy storage to generation or storage mix process (#245), identifying unlinked coal and natural gas upstream processes (#252), addressing high tritium emissions in coal and renewable inventories (#255), and aggregating U.S. average inventory by fuel type (#256). In addition to these unresolved issues, a current list of challenges is discussed in the Appendix.

Additional key changes and improvements to the code base include the following:

1. Significant code formatting, cleaning and organizing (e.g., removal of unused code, commented code, duplicate methods, unused imports, and print statements and addition of log statements).
2. Abstract methods were introduced to avoid spaghetti code and improve modularity. Smaller, reusable functions were created, and the utils.py and globals.py modules were expanded for better usability.
3. Syntax updates were added for latest packages, including the new olca package with all new syntax and the latest pandas version, which required rewrites of defunct methods.
4. Error handling was improved (e.g., during math operations).
5. Documentation was added or updated across all methods, adding documentation to methods where it was lacking. Module documentation was copied from the wiki to the module documentation for accessibility and readability.
6. The StEWI package dependency was incorporated to support additional parts of the code, such as eGRID subregion generation by fuel category.
7. Legacy data transparency was improved by updating outdated data files, tracking down their sources, and reproducing methods for digesting them into the model. URLs that have changed since 2018 were also updated.
8. Missing data was addressed by finding new avenues to gap fill, such as using the EPA's API for CEMS data.
9. Bad data was documented, such as the latest versions of EIA's public coal data being published to 1995 Excel workbooks but appearing to be mistakenly saved as XML spreadsheets, which crash the model.
10. Logging was updated to allow both streaming and a rotating file handler at two different levels (i.e., info and debug, respectively).
11. New postprocess fixes were applied, including removing zero-value quantitative reference flows, fixing flows that are untracked, and correcting exchange internal IDs so they are consecutively numbered.
12. A product system generator was added to the code to improve transparency and match the JSON-LD output with the Fed Commons website.
13. Hotfixes were applied throughout the code to improve efficiency, make corrections, and amend mis-matched data types.
14. Impact assessment methods were removed from the JSON-LD as they were determined to be superfluous to the baseline dataset.
15. The Python setup script was updated for the latest list of dependencies and a contributor's guide was developed for the repository.

Appendix

The Single Citation Problem

The electricity baseline, as it is found on the Federal LCA Commons, points to a single source, "Ingwersen et al. electricitylci: A Python package for U.S. regionalized electricity life cycle inventory data creation," which is labeled as "in development." This work is abandoned. In the interim, the following citation was recommended: "EPA, 'Electricity Life Cycle Inventory,' March 2022 [Online]. Available: 43ab081"

In 2024, a meeting was held between NETL and EPA to discuss the ElectricityLCI manuscript. A historical manuscript draft was shared between parties. NETL (if resources are available) intends to complete the manuscript, which will focus on describing the tool.

The EIA Bulk Data Problem

EIA's U.S. Electric System Operating Data for 2016–2018 are no longer supported in either the bulk EBA (http://api.eia.gov/bulk/EBA.zip) or via their Opendata API portal (https://www.eia.gov/opendata/), which are needed for legacy versions of the electricity baseline. A version of the EBA bulk data file was archived (from March 12, 2024) that has historical data dating back to 2015, which is the only known surviving copy of EIA bulk data available to support the 2016 electricity baseline.

The API Requirement Problem

API key registration is required from two services to access all the required data for running the electricity baselines. The first API key required is from the EPA², which provides data from the Continuous Emission Monitoring Systems (CEMS) through the Clean Air Markets Program Data (CAMPD)---one of the reporting agencies for developing power plant emissions inventories. The second API key required is from EIA³, which provides bulk U.S. Electric Systems Operating Data necessary for net trading calculations.

The Tritium Problem

In Issue #255, high levels of tritium were identified across renewable and coal inventories, with values ranging from 1×10^9^ to 1×10^22^ kBq, which are orders of magnitude higher than the tritium emissions from the Fukushima disaster of 2011 (i.e., 3×10^12^ kBq), and were tracked to the aluminum production (e.g., from primary and secondary ingot) from 2009--2014 Gabi databases. These high emission values impact renewable inventories (e.g., wind, solar, solar thermal, and geothermal), and coal and gas inventories (via coal mining and petroleum extraction and processing).

The Memory Management Problem

There are a few computations in ElectricityLCI that spike a user's computer's memory, namely: upstream construction regionalization (new), uncertainty calculation, and electricity trading. Memory requirements can be in excess of 20 GB and crash a user's model execution if their computer does not have enough memory.

The NETL Water Use Problem

A significant amount of energy is unaccounted for in the plant water use methods in ElectricityLCI. This error is rooted in the aggregation method, which uses the water type category for grouping. The presence of "not a number" values in this category cause a significant portion of water entries in the dataset to get filtered out of the calculation. A proposed method is presented on GitHub that replaces this data file and calculates power-plant water consumption from EIA's water data (https://www.eia.gov/electricity/data/water/).

The EPA Trading Method Problem

While version 2 of the ElectricityLCI was updated to support the reproduction of the 2016 baseline (i.e., ELCI_1_config), the EPA trading method used in other model configurations needs to be updated, as it is not known to be working.

Footnotes

1. https://github.com/USEPA/ElectricityLCI/issues ↩

2. https://www.epa.gov/power-sector/cam-api-portal#/api-key-signup ↩

3. https://www.eia.gov/opendata/ ↩