Version 20.49 broke this

[jackkorber]

Powerwall version 20.49 broke this

[nswfugitive]

Yep, not working at all now.

[lordyavin]

Yes, access to the used API endpoints requires to be logged in. 😢 Anyone an idea how to add authentication?

[nswfugitive]

This link has some info on it - I'm afraid I don't understand it because it is python related. https://github.com/rhodesman/teslaPowerDash

[mihailescu2m]

Yes, access to the used API endpoints requires to be logged in. 😢 Anyone an idea how to add authentication?

There is no way to add authentication in telegraf as of yet. The only workaround found is to login on the webpage and copy the cookie to the telegraf config file - but this will get it going only as long the cookie is valid, then it needs to be replaced.
See here: vloschiavo/powerwall2#41 (comment)
I will wait and see if a solution can be found, otherwise I am afraid the fix is implementing a custom telegraf module.

[lordyavin]

I'll think about extending the telegraf inputs.http plugin. Unfortunately the login mechanism to the powerwall is a custom one (credentials via URL parameters or via JSON body). So maybe you are right that a special plugin would be better than extending the existing one.

[schumi]

you can use [[inputs.exec]] instead of [[inputs.http]] in the telegraf.conf file. in exec you can add any command line entry, including bash scripts or direct curl statements that can include authentication cookies. I did that, adapted some Grafana dashboards, works so far, but have not yet gotten the 20.49 update yet for final testing.

[lordyavin]

@schumi could you please provide your changes via a pull request? I will test them then.

[schumi]

Using this in the [[inputs.exec]] section of telegraf.conf see attached file (did not do pull request, as I removed actual ip address and PW below)

Then a cron job that renews the authentication cookie (currently at 1 minute). not sure what timing will be actually needed, will test & adapt once Tesla has upgraded me to 20.49:

*/1 * * * * curl -s -k -i -c /tmp/cookies/pw.txt -X POST -H 'Content-Type: application/json' -d {"username":"customer","password":"password as configured for customer","email":"e-mail as configured for customer","force_sm_off":false} https://[ip addr]/api/login/Basic

hope this helps!

exec.telegraf.conf.txt

[nswfugitive]

So how's the testing going?

[schumi]

still on 20.40.3, so no news from my side

[lordyavin]

I tried to use input.exec but I'm a novice in administration all of the components. I'm not sure but I think the telegraf config works but I don't see any data in grafana.

[schumi]

you need to adjust the Grafana Dashboards, see screenshot below for a Dashboard query where I added both http tags and exec tags

Spelled out as SELECT statement this is as follows:

SELECT mean("load_instant_power") FROM "exec" WHERE $timeFilter GROUP BY time($__interval) fill(none)

SELECT mean("load_instant_power") FROM "http" WHERE ("url" = 'https://192.168.7.14/api/meters/aggregates') AND $timeFilter GROUP BY time($__interval) fill(none)

the dashboard then populates nicely.

[lordyavin]

Sure that this is everything I need to adapt based on your input.exec config? I'm sorry. Currently I don't have enough time to RTFM all the stuff that is necessary to get it working. Please provide an export of your dashboard and check if the influxdb needs adjustment. How to check if there is data in the DB?

[schumi]

Below is the dashboard definition to check, whether the database has been populated by Telegraf using the [[inputs.exec]] section as described above. It should plot the http values on top of the exec values for house consumption as provided by the Powerwall Gateway (adapt IP address of gateway to your actual IP address). In my case it looks as on screenshot, in cases where http access does not work anymore, only the blue curve would be visible:

{
"aliasColors": {
"Consumption via exec": "semi-dark-blue",
"exec.mean": "blue"
},
"bars": false,
"dashLength": 10,
"dashes": false,
"fieldConfig": {
"defaults": {
"custom": {}
},
"overrides": []
},
"fill": 1,
"fillGradient": 0,
"gridPos": {
"h": 9,
"w": 12,
"x": 0,
"y": 0
},
"hiddenSeries": false,
"id": 2,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
},
"lines": true,
"linewidth": 3,
"nullPointMode": "null",
"options": {
"alertThreshold": true
},
"percentage": false,
"pluginVersion": "7.4.0",
"pointradius": 2,
"points": false,
"renderer": "flot",
"seriesOverrides": [
{
"$$hashKey": "object:355",
"alias": "Consumption via exec",
"linewidth": 7
}
],
"spaceLength": 10,
"stack": false,
"steppedLine": false,
"targets": [
{
"alias": "Consumption via exec",
"groupBy": [
{
"params": [
"$__interval"
],
"type": "time"
},
{
"params": [
"none"
],
"type": "fill"
}
],
"measurement": "exec",
"orderByTime": "ASC",
"policy": "default",
"refId": "A",
"resultFormat": "time_series",
"select": [
[
{
"params": [
"load_instant_power"
],
"type": "field"
},
{
"params": [],
"type": "mean"
}
]
],
"tags": []
},
{
"alias": "Consumption via http",
"groupBy": [
{
"params": [
"$__interval"
],
"type": "time"
},
{
"params": [
"none"
],
"type": "fill"
}
],
"hide": false,
"measurement": "http",
"orderByTime": "ASC",
"policy": "default",
"refId": "B",
"resultFormat": "time_series",
"select": [
[
{
"params": [
"load_instant_power"
],
"type": "field"
},
{
"params": [],
"type": "mean"
}
]
],
"tags": [
{
"key": "url",
"operator": "=",
"value": "https://192.168.7.14/api/meters/aggregates"
}
]
}
],
"thresholds": [],
"timeRegions": [],
"title": "Telegraf fetching test",
"tooltip": {
"shared": true,
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"$$hashKey": "object:272",
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
},
{
"$$hashKey": "object:273",
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
}
],
"yaxis": {
"align": false,
"alignLevel": null
},
"timeFrom": null,
"timeShift": null,
"datasource": null
}

[lordyavin]

Thanks, will use it tomorrow - hopefully.

[Gerynant24]

I've just been updated today so will give this a try.

I'm a bit of a novice at all this, but thank you all for the details, I hope this works!

[Gerynant24]

Using this in the [[inputs.exec]] section of telegraf.conf see attached file (did not do pull request, as I removed actual ip address and PW below)

Then a cron job that renews the authentication cookie (currently at 1 minute). not sure what timing will be actually needed, will test & adapt once Tesla has upgraded me to 20.49:

*/1 * * * * curl -s -k -i -c /tmp/cookies/pw.txt -X POST -H 'Content-Type: application/json' -d {"username":"customer","password":"password as configured for customer","email":"e-mail as configured for customer","force_sm_off":false} https://[ip addr]/api/login/Basic

hope this helps!

exec.telegraf.conf.txt

I've tried this with my credentials, but the pw.txt file created only seems to have 4 lines in it. I'm not sure what should be in the file, but all I see is:

Netscape HTTP Cookie File
https://curl.haxx.se/docs/http-cookies.html
This file was generated by libcurl! Edit at your own risk.

If I run the curl command at the pi command prompt, it returns an error 401 bad credentials. (I've tested my credentials on the new login screen so I know they are correct.
curl output.txt

Any suggestions most welcome!

Update:
Further experimentation with the above on my Pi confirms that I need to send:
curl "https://192.168.x.xx/api/login/Basic" -s -k -i -H "content-type: application/json" --data-raw "{"username":"customer","password":"xxxxx","force_sm_off":false}" -c ~/powerwall_monitor/pw.txt

You need to escape the quotes on the command and you need --data-raw otherwise the Powerwall replies with the 401 error.
Now getting the UserRecord and AuthCookie in the pw.txt file:

Netscape HTTP Cookie File
https://curl.haxx.se/docs/http-cookies.html
This file was generated by libcurl! Edit at your own risk.

192.168.x.xx FALSE / FALSE 0 UserRecord eyJl...
192.168.x.xx FALSE / FALSE 0 AuthCookie ZwYJ...

Hope this helps other newbies out there!

[Gerynant24]

I've followed your steps schumi, but when I try to run your test dashboard, I just get a blank screen.

I've tried replacing http with exec in my dashboard.json file, but no data is being displayed.
{
"alias": "Home Usage",
"datasource": "InfluxDB",
"groupBy": [
{
"params": [
"5m"
],
"type": "time"
}
],
"measurement": "exec",
"orderByTime": "ASC",
"policy": "autogen",
"query": "SELECT "mean_load_instant_power" FROM "exec" WHERE $timeFilter tz('Europe/London')",
"rawQuery": false,
"refId": "A",
"resultFormat": "time_series",
"select": [
[
{
"params": [
"home"
],
"type": "field"
},
{
"params": [],
"type": "mean"
}
]
],
"tags": [],
"tz": "Europe/London"

Does this suggest that my telegraf modification is wrong?
[[inputs.exec]]
commands = [
"curl -k -b ~/powerwall_monitor/pw.txt 'https://192.168.x.xx/api/system_status/soe'",
"curl -k -b ~/powerwall_monitor/pw.txt 'https://192.168.x.xx/api/meters/aggregates'"
]
timeout = "5s"
data_format = "json"

I've tried the curl command at the command prompt and it all seems to work with data retrieved from the powerwall, I just can't seem to get the connection to the graphs sorted.

Any help would be gratefully received.

[mihailescu2m]

I'm exploring changing this to a cloud-based solution (based off the same stack here), where the code (still open sourced) will be pushed automatically to devices when there are updates. Would this kind of solution be of interest to the people using this now? You would still need your own device (I'm targeting raspberry pi initially, but x86 platform should work too), but you would not have to worry about updates, either on the firmware side or the client side.

[Gerynant24]

You have my interest. I love the view your solution gives of my system. If you are adding anything I’d love to know how to export all the data to something like excel, but to have a stable graphic display would be excellent! I’ve figured out the cookie paste so I’m running again for now. Thanks!

…

On 20 Feb 2021, at 22:17, mihailescu2m ***@***.***> wrote:  I'm exploring changing this to a cloud-based solution (based off the same stack here), where the code (still open sourced) will be pushed automatically to devices when there are updates. Would this kind of solution be of interest to the people using this now? You would still need your own device (I'm targeting raspberry pi initially, but x86 platform should work too), but you would not have to worry about updates, either on the firmware side or the client side. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

[Gerynant24]

As a bit of a novice to the low down details of telegraf & grafana I failed to get the above solutions to work, so I've resorted to a bit of a cumbersome workaround - but it seems to work for my setup which is based on the original configuration from milhailescu2m instructions - but will only work on a Pi (as far as I know...)
Basically I'm prodding the Powerwall every minute to see if the current cookie is working and getting a new one (hoping that the SD card is happy with so many writes...)
Then if the request for the soe fails with an error, I recreate the telegraf.conf file using the fresh cookie and restart the Pi.

I've attached my little python program. Please feel free to either laugh at it or use the bits that work for you!

I hope anyone else out there with a Raspberry Pi for monitoring their Powerwall finds it useful until there is a simpler solution!

assemble v1.2_anon.txt

[pridkett]

@mihailescu2m: With PR #17 being accepted - can we close this issue? It's been running great for me for the past couple of weeks.