Skip to content

Commit

Permalink
Ports existing functionality to v4 API (#3)
Browse files Browse the repository at this point in the history
Closes #2
  • Loading branch information
danielpatenaude authored Dec 6, 2022
1 parent 260a94f commit e52d62d
Show file tree
Hide file tree
Showing 8 changed files with 443 additions and 190 deletions.
209 changes: 209 additions & 0 deletions Enpahse Enlighten v4.postman_collection.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,209 @@
{
"info": {
"_postman_id": "274a5c9b-df54-48b1-a24e-25405890f016",
"name": "Enpahse Enlighten v4",
"schema": "https://schema.getpostman.com/json/collection/v2.1.0/collection.json"
},
"item": [
{
"name": "Fetch systems",
"request": {
"auth": {
"type": "bearer",
"bearer": [
{
"key": "token",
"value": "{{access_token}}",
"type": "string"
}
]
},
"method": "GET",
"header": [
{
"key": "Host",
"value": "api.enphaseenergy.com",
"type": "text"
}
],
"url": {
"raw": "https://api.enphaseenergy.com/api/v4/systems?key={{app_api_key}}",
"protocol": "https",
"host": [
"api",
"enphaseenergy",
"com"
],
"path": [
"api",
"v4",
"systems"
],
"query": [
{
"key": "key",
"value": "{{app_api_key}}"
}
]
}
},
"response": []
},
{
"name": "Generate OAuth2 access_token",
"request": {
"auth": {
"type": "basic",
"basic": [
{
"key": "password",
"value": "{{client_secret}}",
"type": "string"
},
{
"key": "username",
"value": "{{client_id}}",
"type": "string"
}
]
},
"method": "POST",
"header": [],
"url": {
"raw": "https://api.enphaseenergy.com/oauth/token?grant_type=authorization_code&redirect_uri=https://api.enphaseenergy.com/oauth/redirect_uri&code={{auth_code}}",
"protocol": "https",
"host": [
"api",
"enphaseenergy",
"com"
],
"path": [
"oauth",
"token"
],
"query": [
{
"key": "grant_type",
"value": "authorization_code"
},
{
"key": "redirect_uri",
"value": "https://api.enphaseenergy.com/oauth/redirect_uri"
},
{
"key": "code",
"value": "{{auth_code}}"
}
]
}
},
"response": []
},
{
"name": "Refresh access_token",
"event": [
{
"listen": "test",
"script": {
"exec": [
""
],
"type": "text/javascript"
}
}
],
"request": {
"auth": {
"type": "basic",
"basic": [
{
"key": "password",
"value": "{{client_secret}}",
"type": "string"
},
{
"key": "username",
"value": "{{client_id}}",
"type": "string"
}
]
},
"method": "POST",
"header": [],
"url": {
"raw": "https://api.enphaseenergy.com/oauth/token?grant_type=refresh_token&refresh_token={{refresh_token}}",
"protocol": "https",
"host": [
"api",
"enphaseenergy",
"com"
],
"path": [
"oauth",
"token"
],
"query": [
{
"key": "grant_type",
"value": "refresh_token"
},
{
"key": "refresh_token",
"value": "{{refresh_token}}"
}
]
}
},
"response": []
},
{
"name": "inverters_summary_by_envoy_or_site",
"request": {
"auth": {
"type": "bearer",
"bearer": [
{
"key": "token",
"value": "{{access_token}}",
"type": "string"
}
]
},
"method": "GET",
"header": [
{
"key": "Host",
"value": "api.enphaseenergy.com",
"type": "text"
}
],
"url": {
"raw": "https://api.enphaseenergy.com/api/v4/systems/inverters_summary_by_envoy_or_site?key={{app_api_key}}&site_id={{system_id}}",
"protocol": "https",
"host": [
"api",
"enphaseenergy",
"com"
],
"path": [
"api",
"v4",
"systems",
"inverters_summary_by_envoy_or_site"
],
"query": [
{
"key": "key",
"value": "{{app_api_key}}"
},
{
"key": "site_id",
"value": "{{system_id}}"
}
]
}
},
"response": []
}
]
}
82 changes: 58 additions & 24 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
# python_enlighten_api
# python_enlighten_api (v4)

_Note: this repository has been updated to include using the new Enphase Enlighten v4 API (https://developer-v4.enphase.com/aboutproduct.html)._

Enphase Enlighten API & Google Sheets application to pull data and monitor panel performance and populate a Google Sheet with historical and visual data. This allows tracking individual panel performance over the lifetime of the system. Most of the functionlaity provided here is also provided by the Enphase Enlighten website and app, but this allows for granular panel tracking and performance over time.

For example: the panel underperforming (red) is partially blocked by a neighbor's tree during the morning hours. So it's reporting performance outside a few standard deviations.
Expand All @@ -7,42 +10,68 @@ For example: the panel underperforming (red) is partially blocked by a neighbor'
<img src="solar_performance_example.png" width="500">
</p>


## Known Limitations
1. Token Expiration:
* The Enlighten v4 API does not provide any documentation on what happens whe `refresh_token` expires (1 week validity) and how a new one is generated. The step to generate the original `access_token` and `refresh_token` in [Generate OAuth2 access_token and refresh_token](https://developer-v4.enphase.com/docs/quickstart.html#step_8) seems to only allow the route to be used a single time for an authorization code (auth_code).
* As such, _the enlightenAPI_v4_ constructor provided in this repository will refresh the tokens using the [Generate new access_token and refresh_token using refresh_token](https://developer-v4.enphase.com/docs/quickstart.html#step_10) and update the _enlighten_v4_config.json_ to contain the new tokens. This ensures that each day when the script is run, the tokens are refreshed and kept well within the refresh_token's 1 week validity.
2. Inverter Telemetry Reporting:
* It appears there's a new v4 route (/api/v4/systems/{System_id}/devices/micros/{serial_no}/telemetry) that can get microinverter data based on a date range. This seems the more ideal way to get daily inverter data than my current way of having to get the current lifetime data minus the stored value. But unfortunately I have not been able to successfully call this route using the Enlighten API v4 docs. So maybe this route doesn't do what I think it does, and their docs don't make it very clear what it does. I always get a 401 - Not Authorized. If someone is able to get that figured out, please feel free to let me know.

## Requirements
Requires Python 3.6.8 or later installed. Much effort has been taken to ensure this application does not require additional modules besides what is included standard with Python.

pip install requests
pip install --upgrade google-api-python-client google-auth-httplib2 google-auth-oauthlib
pip install -r requirements.txt

This will install the following:
1. requests
2. google-api-python-client
3. google-auth-httplib2
4. google-auth-oauthlib

## Getting Started

### Postman Scripts
The _Enpahse Enlighten v4.postman_collection.json_ is provided to assist setting up the `access_token` and `refresh_token`. In addition, you can use the GET routes to test your API connectivity and configuration.

### Script/API Setup
1. Allow app access to your Enlighten Account:
* Follow the instructions here to add a new application to your developer account: https://developer.enphase.com/docs/quickstart.html
2. Open run_inverter_daily_stats.py and set the following in the User Settings section:
* user_id: within MyEnlighten go to your account Settings and find the API Settings user ID
* site_id: within MyEnlighten the Site ID should be displayed (or if using the web browser should be in the URL [https://enlighten.enphaseenergy.com/pv/systems/:SITE_ID/overview]
* api_url: The Enphase API, most likely: https://api.enphaseenergy.com/api/v2/systems
* api_key: within the https://developer.enphase.com/admin/applications page copy your app's API Key
* spreadsheet_id: The SpreadSheet ID from Google Sheets. See step 5.
4. Setup a Google API key for your python script and put the following files in your main working script directory: credentials.json, token.pickle.
* https://developers.google.com/sheets/api/quickstart/python
1. [Enlighten API] Allow app access to your Enlighten Account:
* Follow the Enlighten [Quickstart instructions Steps 1-4](https://developer-v4.enphase.com/docs/quickstart.html#step_1) to add a new application to your developer account
2. [Enlighten API] Open the _enlighten_v4_config.json_ and set the following:
* `name`: a generic name to identify this system.
* `system_id`: (previously known as `Site ID`) Within MyEnlighten the System ID should be displayed (or if using the web browser should be in the URL [https://enlighten.enphaseenergy.com/web/:SYSTEM_ID/today/graph/hours].
* `app_api_key`: Within the [Enlighten API App Page](https://developer-v4.enphase.com/admin/applications) page, copy your app's API key.
* `app_client_id`: Within the [Enlighten API App Page](https://developer-v4.enphase.com/admin/applications) page, copy your app's Client ID.
* `app_client_secret`: Within the [Enlighten API App Page](https://developer-v4.enphase.com/admin/applications) page, copy your app's Client Secret.
* `access/refresh_token`: See step 3 below.
* `spreadsheet_id`: The SpreadSheet ID from Google Sheets. See step 5.
3. [Enlighten API] Generate access_token and refresh_token (this condenses some of the Enlighten API [Quickstart Guide](https://developer-v4.enphase.com/docs/quickstart.html)
* Follow the Enlighten [Quickstart instructions Steps 6-7](https://developer-v4.enphase.com/docs/quickstart.html#step_6), generate the auth_code for your application
* Using the _Enpahse Enlighten v4.postman_collection.json_ open the _Generate OAuth2 access_token_ request:
* Update the Postman environment variables to set the: auth_code, client_id, and client_secret
* Run the request. The resulting access_token and refresh_token should be added to your enlighten_v4_config.json
* _Note: it appears that you can only run this request ONCE per app authorization code (auth_code). Once you have the tokens generated for this auth_code, it appears you will be unable to run this route again. So make sure to save them. The Enlighten v4 API does not provide any details on this, but this appears to be the case._
* You can call the Postman _Fetech Systems_ request using your new tokens to ensure proper configuration of your application and API tokens.
4. [Google Sheets] Setup a Google API key for your python script and put the following files in your main working script directory: credentials.json, token.pickle.
* Follow the [Google Python Quickstart Guide](https://developers.google.com/sheets/api/quickstart/python)
* Allow your API token to access your Google Sheets account: https://console.developers.google.com/apis/api/sheets.googleapis.com/overview?
5. Google Sheet Setup
* The google sheet can be accessed from: https://docs.google.com/spreadsheets/d/1JPnT5T4xvDIKaefL8Z7AoxRNFv6HnVBF7SH-J9Yqfdk. It is a working copy of an example system setup. You may need to manually clear and remove data to use.
How to Setup:
5. [Google Sheets] Google Sheet Setup
* The _'Solar Performance'_ Google sheet can be accessed from the [Solar Performance (Template)](https://docs.google.com/spreadsheets/d/1JPnT5T4xvDIKaefL8Z7AoxRNFv6HnVBF7SH-J9Yqfdk). It is a working copy of an example system setup. You will need to manually clear and remove the demo data to use.
* How to Setup the _'Solar Perforance'_ Sheet:
1. You'll need to make a copy of this sheet to your personal Google Drive.
2. Populate each of your inverter serial numbers into the 'Panel Data-Template' Sheet.
3. Copy and Paste, using values and transpose, your inverter serial numbers into the 'Last 7 Days' sheet.
4. Update the 'Dashboard' Sheet panel layout to match your panels.
5. Update the 'Dashboard' Sheet panel numbers and serial numbers to match your panel data
* Note: Enphase Enlighten did not provide a good way to do this. So I manually had to match up each inverter serial number with the panel number in the layout by tracking panel energy produced over a few days on the 'Panel Data-<Year>' sheet vs the Enphase Enlighten website/app. After a few days each panel's historical data output allowed me to match up each panel on the Dashboard/Panel Data Sheet with the layout of the Enphase app.
2. Populate each of your inverter serial numbers into the _'Panel Data-Template'_ Sheet.
* This template sheet will automatically be created into a _'Panel Data-<YEAR>'_ sheet where the historical data for each panel per day will be stored.
3. Copy and Paste, using values and transpose, your inverter serial numbers into the 'Last 7 Days' sheet from the _'Panel Data-Template'_ sheet.
4. Update the _'Dashboard'_ Sheet panel layout to match your panels.
5. Update the _'Dashboard'_ Sheet panel numbers and serial numbers to match your panel data
* Note: Enphase Enlighten did not provide a good way to do this. So I manually had to match up each inverter serial number with the panel number in the layout by tracking panel energy produced over a few days on the _'Panel Data-<Year>'_ sheet vs the Enphase Enlighten website/app. After a few days each panel's historical data output allowed me to match up each panel on the Dashboard/Panel Data Sheet with the layout of the Enphase app.

### Running
The Enlighten API has a long lag time between when data is updated on their end. If you run these scripts once a day after the Enlighten data updates AND before your solar is producing power (e.g.: 4am) you get the total lifetime power produced by each inverter, including the previous day.

Run with run_inverter_daily_stats.sh or copy the logic this script is using.

#### Setting Automated Cron Jobs
#### Linux: Setting Automated Cron Jobs
If you're using Linux, you can add these scripts to crontab jobs to run automatically at night by:

Run:
Expand All @@ -53,17 +82,22 @@ Add a crontab job:
# At 4am local time run the python script via shell script to ensure we're in the right directory
0 4 * * * /home/<user>/python_enlighten_api/run_inverter_daily_stats.sh >> /home/<user>/python_enlighten_api/cron.log 2>&1

#### Windows: Scheduled Task
If you're using Windows, you can automate running this by doing similar to what the Linux job is doing:
1. Add a new Windows scheduled task to run at 4am
2. Create and have the scheduled task run a .bat file that performs the directory change and python call from `run_inverter_daily_stats.sh`

## Enphase Enlighten API Documentation

* https://developer.enphase.com/docs
* https://developer-v4.enphase.com/docs.html

## Scripts Explanation

This repository contains a few scripts used to hit the Enphase Enlighten API and collect data. The scripts inclide:

### run_inverter_daily_stats.py

Runs the Enlighten API route 'inverters_summary_by_envoy_or_site' to collect the lifetime energy produced by each inverter. The Enphase API lacks the granulatiry of seeing per inveter daily states (documented here: https://developer.enphase.com/forum/topics/per-inverter-stats). So this script provides a means to do that. If you call this route once a day before your solar is producing power (e.g.: 4am) you get the total lifetime power produced by each inverter, including the previous day. If you track this total lifetime energy value every day, you can then subtract the current day's total from the previous day lifetime total. That gives you the daily production value for that inverter. Note: if your Envoy is connected via low bandwidth Cellular, data only refreshes to Enlighten every 6 hours. So perform this route the next day in the early morning to ensure you get complete data.
Runs the Enlighten API route 'inverters_summary_by_envoy_or_site' to collect the lifetime energy produced by each inverter. The Enphase API lacks the granulatiry of seeing per inveter daily stats. So this script provides a means to do that. If you call this route once a day before your solar is producing power (e.g.: 4am) you get the total lifetime power produced by each inverter, including the previous day. If you track this total lifetime energy value every day, you can then subtract the current day's total from the previous day lifetime total. That gives you the daily production value for that inverter. Note: if your Envoy is connected via low bandwidth Cellular, data only refreshes to Enlighten every 6 hours. So perform this route the next day in the early morning to ensure you get complete data.

The resulting data is stashed in a .json file. The file organizes the data by microinverter (by ID), then by day. So you can easily parse this historical data for daily production values.
For example:
Expand Down
11 changes: 11 additions & 0 deletions enlighten_v4_config.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{
"name": "<some name>",
"system_id": "<system/site_id",
"api_url": "https://api.enphaseenergy.com/",
"app_api_key": "<your app's api_key>",
"app_client_id": "<your app's client ID>",
"app_client_secret": "<your app's client secret>",
"access_token": "<your access_token from running 'Enpahse Enlighten v4.postman_collection' -> 'Generate OAuth2 access_token'>",
"refresh_token": "<your refresh_token from running 'Enpahse Enlighten v4.postman_collection' -> 'Generate OAuth2 access_token'>",
"spreadsheet_id": "<google sheet_id>"
}
6 changes: 5 additions & 1 deletion populate_google_sheet.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
# Author: Daniel Patenaude
# Date: 12/02/2022
# Desc: Populate the enphase data into the google sheet template

import datetime
import json
import os
Expand All @@ -20,7 +24,7 @@ def run(config):

# Grab the historical data from the data store
inverter_historical_data = {}
datafile = f'data/inverter_daily_data-{config["site_id"]}.json'
datafile = f'data/inverter_daily_data-{config["system_id"]}.json'
if os.path.isfile(datafile) and os.access(datafile, os.R_OK):
with open(datafile) as json_file:
inverter_historical_data = json.load(json_file)
Expand Down
4 changes: 4 additions & 0 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
requests==2.24.0
google-api-python-client==1.12.3
google-auth-httplib2==0.0.4
google-auth-oauthlib==0.4.1
Loading

0 comments on commit e52d62d

Please sign in to comment.