Multiple Zappi support, Optimiser occasionally runs continuously #269

pbcjh10 · 2024-09-29T16:39:20Z

Running pv_opt on a HA Green.
Set up eventually!

Solis inverter, Pyton tech batteries.
Using Solax modbus where I have 169 entities.

However, I have a dashboard error and my dashboard never displays a PV Opt Result.
pv_opt.log
main.log
error.log

This is the dashboard:

stevebuk1 · 2024-09-29T21:04:40Z

Hmmmm, Pv_opt is occasionally failing to load consumption data from sensor.solis_house_load. It strictly doesn't need it, it loads both power and energy and will use power in preference of energy.

It might be a generic problem with Beta-9 you are running as I only download the energy sensors, however I think your energy sensor is not correct in config.yaml (so it has nothing to fall back on if power isn't working right) can you attach that file please?

stevebuk1 · 2024-09-29T21:35:30Z

Note: from tomorrow I'm on hols for the week, will pick this up again on my return.

pbcjh10 · 2024-09-29T22:45:41Z

@stevebuk1 voila!
config.yaml.txt

stevebuk1 · 2024-10-01T18:56:15Z

Try commenting out all of these lines in config.yaml:

id_consumption_today: sensor.{device_name}_consumption_today
id_consumption:
- sensor.{device_name}_house_load
- sensor.{device_name}_bypass_load

The first entry is incorrect for the Solax modbus integration - it should be sensor.solis_house_load_today. I don't think that is the cause of the problems tho as Pv_opt seems to recognise that its not found and instead revert to the system default.

I think the cause of the problem is the 2nd and 3rd entries when a call is made through Appdeamon to load the consumption history in W . Its clear from #270 that some systems struggle to get it done with 10seconds and I suspect loading consumption as W is a significant size. Here, the system defaults appear deliberately set to entities that don't exist, and if this happens the system reverts to using kWh, which it then converts to W. I suspect loading kWh history is significantly smaller but perhaps slightly less accurate. Try it and post a log please.

pbcjh10 · 2024-10-02T06:10:15Z

@stevebuk1 voila!
pv_opt(1).log
main(1).log
error(1).log

It looks like its working, although my read only 'off' keeps flicking back over to on, similarly choosing Zappi from the dropdown box, but not sure how much of a difference they make.
Are you able to explain why its dumping my battery to grid at 16:30 ? Solar will be negligable but I'll still have the evenings electricity use?

Also not sure I understand the "current setting forced discharge" is that right?

stevebuk1 · 2024-10-02T12:34:16Z

re the readonly setting, if you want Pv_opt to start up in readonly mode, find this line in config.yaml:

read_only: false # If true the inverter will not be controlled

and change false to true.

The EV charger control is the beginnings of support for EVs. If you have a Zappi and its seen as part of the house load, then you'll need to set this to Zappi otherwise the consumption history that Pv_opt uses to predict your use will include the EV charging data. If your Zappi is outside of the house load (i.e on its own Henley block) you can leave it set to None.

If you do have a Zappi that is seen as part of the house load and/or are on IOG let me know as I have a mature Beta that supports IOG i.e will pick up any extra cheap slots and prevent the house battery discharging when the EV is charging. It also includes all the code for setting the default for the EV selector correctly.

stevebuk1 · 2024-10-02T12:58:44Z

Apologies I've got myself confused. You were running the Beta release of Pv_opt before (v3.16.0-Beta-9) but your latest logs show its changed back to the main production release (v3.15.5). Was this deliberate?

pbcjh10 · 2024-10-02T13:04:02Z

Yep we have a zappi and are on IOG ! From: stevebuk1 ***@***.***> Sent: 02 October 2024 13:35 To: fboundy/pv_opt ***@***.***> Cc: pbcjh10 ***@***.***>; Author ***@***.***> Subject: Re: [fboundy/pv_opt] Entity not avaliable: pvopt_opt_cost, pvopt_base_cost (Issue #269) re the readonly setting, if you want Pv_opt to start up in readonly mode, find this line in config.yaml: read_only: false # If true the inverter will not be controlled and change false to true. The EV charger control is the beginnings of support for EVs. If you have a Zappi and its seen as part of the house load, then you'll need to set this to Zappi otherwise the consumption history that Pv_opt uses to predict your use will include the EV charging data. If your Zappi is outside of the house load (i.e on its own Henley block) you can leave it set to None. If you do have a Zappi that is seen as part of the house load and/or are on IOG let me know as I have a mature Beta that supports IOG i.e will pick up any extra cheap slots and prevent the house battery discharging when the EV is charging. — Reply to this email directly, view it on GitHub <#269 (comment)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE3A5ETLAGUGQGCM7XMXJALZZPR55AVCNFSM6AAAAABPB3MNY2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOBYGUZTCNJRGA> . You are receiving this because you authored the thread. <https://github.com/notifications/beacon/AE3A5EW7YYAWP2I2XHZFPKLZZPR55A5CNFSM6AAAAABPB3MNY2WGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTUOLYMTM.gif> Message ID: ***@***.*** ***@***.***> >

pbcjh10 · 2024-10-02T13:04:10Z

No! From: stevebuk1 ***@***.***> Sent: 02 October 2024 13:59 To: fboundy/pv_opt ***@***.***> Cc: pbcjh10 ***@***.***>; Author ***@***.***> Subject: Re: [fboundy/pv_opt] Entity not avaliable: pvopt_opt_cost, pvopt_base_cost (Issue #269) Apologies I've got myself confused. You were running the Beta release of Pv_opt before (v3.16.0-Beta-9) but your latest logs show its changed back to the main production release (v3.15.5). Was this deliberate? — Reply to this email directly, view it on GitHub <#269 (comment)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE3A5EXK4RQ5RAE2N2CNOY3ZZPUZXAVCNFSM6AAAAABPB3MNY2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOBYGU4DIMRSGQ> . You are receiving this because you authored the thread. <https://github.com/notifications/beacon/AE3A5EULVIFGMXXXVEGXPA3ZZPUZXA5CNFSM6AAAAABPB3MNY2WGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTUOL3TSA.gif> Message ID: ***@***.*** ***@***.***> >

stevebuk1 · 2024-10-02T13:10:24Z

Apologies I got myself confused. You are already running v3.16.0-Beta-9 and are on IOG.

In which case, take a look at the config.yaml here:
https://github.com/stevebuk1/pv_opt/blob/dev/apps/pv_opt/config/config.yaml

Add the section called "EV parameters" to your own config.yaml - this will ensure Zappi is selected on each startup.

I'll post later on with some additional lines to add to config.yaml which will enable addtional logging, then I can have a look at your forced discharge question.

stevebuk1 · 2024-10-02T19:57:21Z

I'll post later on with some additional lines to add to config.yaml which will enable additional logging, then I can have a look at your forced discharge question.

Near the top of your config.yaml file there is the following line:

debug: false

Change false to true.

In the user area, add the following line:

debug_categories: W

Once you see a charge/discharge plan that doesnt make sense, attach Pv_opt.log.

pbcjh10 · 2024-10-07T09:31:48Z

So I re-downloaded files after the beta 9 confusion above and now its failing again. Although i've fixed the Zappi default as you suggested.
I'm back to having errors as above in my first post. However the three lines you suggested to comment out, are already commented out!

I've just tried to tweak it to correct the sensor to be this one; id_consumption_today: sensor.solis_house_load_today but having it correct or commented out didn't seem to make a difference.

I attach logs and my yaml!
Same behaviour as before, its comparing tarrifs, the "PV Opt Results" box in the dashboard momentorily appears and then disappears with no results.

![image](https://github.com/user-attachments/assets/040f9d9c-63ab-
config.yaml.log
4726-b324-de19724f432b)

pv_optc.log
[pv_optb.log](https://github.com/user-attachments/fi
[error(2).log](https://github.com/user-attachments/file
[mainb.log](https://github.com/user-attach
mainc.log
ments/files/17276304/mainb.log)
s/17276299/error.2.log)
les/17276297/pv_optb.log)
pv_opta.log

stevebuk1 · 2024-10-07T10:34:12Z

Hi, I think its just the case that now that debug is set true, there is just so much logging the system takes forever.

Beta-9 contains a logging filter so we can see the little bits we need to.
To enable this, you still need to do the following bit from my previous post:

In the user area, add the following line:

debug_categories: W

into to your config.yaml.

Unless you really need it, I'd also comment out the last 13 lines in config.yaml, i.e. all this text:

 # alt_tariffs:
 #   - name: Agile_Fix
 #     octopus_import_tariff_code: E-1R-AGILE-23-12-06-G
 #     octopus_export_tariff_code: E-1R-OUTGOING-FIX-12M-19-05-13-G
 #
 #   - name: Eco7_Fix
 #     octopus_import_tariff_code: E-2R-VAR-22-11-01-G
 #     octopus_export_tariff_code: E-1R-OUTGOING-FIX-12M-19-05-13-G
 #     
 #   - name: Flux
 #     octopus_import_tariff_code: E-1R-FLUX-IMPORT-23-02-14-G
 #     octopus_export_tariff_code: E-1R-FLUX-EXPORT-23-02-14-G

pbcjh10 · 2024-10-20T09:47:27Z

So I had a bit of time to try and look at this again.
I have the debug w as requested and have removed the alt tariffs bit as you suggested.

I still have the entity not avaliable sensor.pvopt_status error. I played with things a bit and did a few reboots and now it seems to have randomly sprung to live.

I attach the latest bits including config.yaml

error(5).log
main(4).log
pv_opt(4).log
config(5).yaml.log

I have this error remaining:

The charging plan is like this;

I want to check your thoughts on it.
I can see that its doing some forced discharge this am.
A forced charge this pm in the cheap electric bit.
And then a further forced discharge at 17:00
But then overnight its saying its going to not charge at all? The bit with the IOG marker?

Surely what I want (in the cloudy weather today) is accumulate solar today, perhaps with a discharge just prior to 13:00. But then after that, its nearly evening, I dont want it to dump the battery to grid just after. It should surely hold onto the this (or atleast provide it to house load) rather than discharge to grid?
Then overnight, why is it not charging to 100% on ti IOG cheap section? Same question again to charging on expensive rate at 6-7am And then almost immediately discharging battery down to 23% from 8:30-9:30?

Is it trying to be too clever? Or is this a settings problem in terms of charge and discharge thresholds?
Halp please:

stevebuk1 · 2024-10-20T19:35:30Z

A dump at some point early evening is usual (Pv_opt doesnt tend to delay it to the very last slots unless those slots save you more money than earlier slots), but I'd agree a discharge to 23% doesnt look right, especially when it thinks its still going to be at 22% at 23:00.

The next day, your car charging plan lasts till 7am, so the 6:30-7am charge slot is at cheap rate, but I would have expected Pv_opt have a continuous charging slot from 23:30 to 07:00 rather than a car hold for 23:00 to 6:30, then a charge.

I think the problem is the setting of ev_part_of_house_load in config.yaml.

At the moment, when you charge your car, does the car charging drain the house battery?

If it doesn't, then you need to change this line:

# ev_part_of_house_load = False # Default = True

to this:

ev_part_of_house_load = False # Default = True

i.e. uncomment the line.

As it is, EV charge consumption is being substracted from house consumption which is going to give you almost zero house consumption at the points the car was historically charging, which would explain the forecast of a 23% battery at 17:30 but still at 22% at 23:00. This would explain most of the strange behaviour.

Could you also change:

debug: false

to

debug: true

as without that the line debug_categories: W has no effect.

*** edits - corrected a few times and explanations.

stevebuk1 · 2024-10-20T20:10:41Z

As it is, EV charge consumption is being substracted from house consumption

Upon further inspection I don't think this is the problem as Pv_opt isnt finding any Zappi consumption sensors (or any Zappi entities at all for that matter). This is the cause of the entries in error.log, but setting ev_part_of_house_load = False will correct that as consumption data isnt needed.

I'll take a longer look at what is going on from the logs you've provided, but a Pv_opt.log with debug= true will help immensely as it includes a printout of the expected energy flows, pricing, SOC and car charging expectations for each half hour slot making diagnosis of most problems easy.

pbcjh10 · 2024-10-20T21:23:30Z

Thanks - sorry I'd re-copied bits in from your sample above as somewhere I'd introduced a formatting error.
My car does pull from battery (although obviously the ideal would be for battery to charge in cheap rates to full, and let car charge from grid). So I haven't un-commented the line you suggested. As the default is correct.

I have the zappi integration set up with 4 devices and 75 entities. Where can I see which ones pvopt is looking for?
pv_opt(5).log

I attach a new pv_opt where you can see its failing to find sensors.

pbcjh10 · 2024-10-20T21:28:25Z

And based on this PVopt log;

It looks like its holding 84% SOC over the IOG period - why not 100%? and then an odd discharge at 10am?

stevebuk1 · 2024-10-20T21:43:30Z

I have the zappi integration set up with 4 devices and 75 entities. Where can I see which ones pvopt is looking for?

For Zappi consumption, PV_opt looks for all entities of type "sensor" that have "zappi" and also "charge_added_session" in the name.
For Zappi plug status, PV_opt looks for all entities of type "sensor" that have "zappi" and also "plug_status" in the name.

If it finds these it will report them as follows:

21:02:19     INFO: Checking for Zappi Sensors
21:02:19     INFO: --------------------------
21:02:19     INFO: 
21:02:19     INFO:   sensor.myenergi_zappi_********_charge_added_session
21:02:19     INFO:   sensor.myenergi_zappi_********_plug_status

(the ***** bit is the redacted personal details that is the Zappi S/N etc)

stevebuk1 · 2024-10-20T21:51:10Z

Thanks for the new logs. Your house consumption data has some extremely weird figures in it, its around 4W(!) all night and then literally zero after 5pm. We need to fix this first before doing anything else.

You are logging from the right sensor (sensor.solis_house_load_today) so it must be the data that it is reading from it.

In config.yaml, can you change:

debug_categories: W

to

debug_categories: WP

which will add power consumption logging.

Just run one interation (10 mins or so) and post the new pv_opt.log. That will allow me to see what the 7 day consumption history is.

pbcjh10 · 2024-10-21T05:35:17Z

Voila!
Should I say we have an ASHP? The solis is set on the device itself (or was before pvopt) to charge between 00:30-05:30 and then I guess the battery would take over and supply house, giving a 0 house demand?
pv_opt(6).log

Another odd charging plan with only charging between 05:30 and 7? Is it getting confused on history IOG charging sessions (and ignoring the obvious 23:30-05:30 session?

stevebuk1 · 2024-10-21T08:13:25Z

Voila!

Thanks! So, Pv_opt expects sensor.solis_house_load_today to be reporting kWh, so it increments all day and resets to 0 at midnight. It is what the house consumes, irrespective of whether the grid is supplying it or the battery is supplying it.

Your sensor.solis_house_load_today is reporting very low values overnight with no increments, its resetting itself regularly, and consistently reports no increments from 4.30pm to 11.30pm. I don't think its the right sensor.

Below is an image from my own system that shows two days history from sensor.solis_house_load_today and also sensor.solis_grid_import_today.

The first day shows an EV charging day and the 2nd shows a non-EV charging day.

If we take the 2nd day, you can see that the grid sensor is showing the battery being charged overnight and then it stops incrementing as the battery takes over,. sensor.solis_house_load_today reports the house consumption for the entire 24 hour period.

You'll need to locate the two sensors that show this behaviour otherwise Pv_opt isnt going to work. Post a screenshot when you've found them and we can then change the names over in config.yaml so Pv_opt can find them.

stevebuk1 · 2024-10-21T08:45:11Z

To assist with the above and also the Zappi issue, I'll do a Beta-10 this evening that will do some additional power consumption logging and also log any/all Zappi sensors.

pbcjh10 · 2024-10-21T19:33:13Z

So interesting times;

Grid seems 'easier'

I found this, which looks right I guess, but mislabled?

This certainly doesn't look right;

You're right house load is weird;

This is whats happend today;

pbcjh10 · 2024-10-21T19:40:39Z

So I had a thought (because we had an ASHP fitted 2 weeks ago) and wondered if they have fitted the house CT clamp back on the wire backwards? Triggering solis to think its an export when its an import? could this be a thing?
Certainly my solis app is a mess now too.

stevebuk1 · 2024-10-21T19:43:14Z

Was about to say the same. I think they've fitted the Solis CT clamp on backwards, as your grid export looks more like your grid import. That will also probably sort out house_load_today as well.

Its easily done, as the Solis CT clamp arrow points the opposite direction to almost everyone elses CT clamp arrow!!

stevebuk1 · 2024-10-21T19:55:24Z

As promised here is a Beta-10 release which adds logging of the kWh sensor on each optimiser run and also will log all entities with "zappi" in the name on startup to see if we can find the Zappi consumption sensor.
It at:
https://github.com/stevebuk1/pv_opt/blob/dev/apps/pv_opt/pv_opt.py

Just copy over your existing pv_opt.py, Appdeamon will automatically restart PV_opt.

stevebuk1 · 2024-11-02T21:08:20Z

I went back through the thread and re-read this.
I've corrected the first setting to solis_ ...
I then have commented out the 2nd two as you suggested. Which seems to have fixed the problem.
Will delete logs and see what happens.

Yes I think you've deduced the issue correctly. You probably copied config.yaml from the repo at some point and lost your changes that we made to remove the consumption sensors in W, which on some systems are so big they don't get read in the 10 seconds that AppDaemon allows.

Your kWh consumption sensors seem to be making good sense now. The only concern is the bar chart has a bar called "Loadrate" that is showing in excess of 35,000kWh. It should be set to a sensor that reads in W (power), not kWh (energy), mine is set to sensor.solis_house_load. The same problem is in your "Solar Forecasts versus Actual" chart.

I had turned your 'auto' tarrif for octopus on which seems to be working. Do I then need to comment out the other bit?

No, if octopus_auto: True is set and works, then the rest of the section is ignored. This part is working fine.

pbcjh10 · 2024-11-02T21:55:06Z

Yep thats fair.

I looked at loadrate and this is what I have in W

mine for some reason is pointing at house_load_total thus the huge numbers;

Where can i change this?

stevebuk1 · 2024-11-02T22:04:42Z

On your dashboard, go into edit mode, click on the 3 vertical dots on the right hand side and then click "raw configuration editor":

Then change the entry at what should be at line 24 to what your entity name is in your screengrab above.

Be patient with this editor, its very slow, don't hold down any delete keys!

stevebuk1 · 2024-11-02T22:08:34Z

Apologies, thats the standard production dashboard. I've editted my post above to say change it to whatever yours is called in "Solis Inverter House load". To find the actual name, click on the 'gear' icon.

stevebuk1 · 2024-11-02T22:10:33Z

For you, Line 24 should read sensor.solis_house_load

Everyones entity names are different so the dashboards supplied tend to need editting.

pbcjh10 · 2024-11-02T22:11:37Z

Sorry - too fast for your edit!

I've adjusted to sensor.solis_house_load which has fixed that;

Do I need to make any adjustments to the tuning bits? I turned off the cycling discharge bit because for now I didn't want to force dump the battery for a few pence profit. The documentation is a bit scarce about how to adjust or tune, or if its best to leave alone.

stevebuk1 · 2024-11-02T22:20:04Z

Do I need to make any adjustments to the tuning bits? I turned off the cycling discharge bit because for now I didn't want to force dump the battery for a few pence profit.

I havent needed to adjust much. I have no export tariff so discharging is off for me. I've changed "Solcast Confidence Level" from 50% to 40%, as I find mine mostly underestimates my solar, and have upped "Load Margin" from 10% to 20% as I have a 12kWh battery and a 10kWH average daily consumption, so I'd like a fuller battery to deal with the randomness we switch the oven on at. Other than that I left everything else unchanged.

I think until we subtract your 2nd Zappi from the house load your plans won't make much sense, but you can always turn "Use Consumption History" to Off and set an average kWh per day until PV_opt supports multiple chargers. I too started with 5kWh and have since added another US2000C and then a US5000C to increase capacity to my current level. If I get an ASHP I'll be adding more!

pbcjh10 · 2024-11-03T12:41:56Z

Is there a specific setting that will force charge the battery, or atleast make the system think about charging battery whenever we get a cheap IOG block?

For example - battery currently empty (well 20% which is limit), just plugged in car and its 7p until 4pm. I'd like the battery to go "oh hey, crap solar today lets top up".
Is this a generic problem or a setting I can tweak? or is it because I have cyclic discharge off? Like whenever its got a <= car sign here, why's it not also topping up the battery?

stevebuk1 · 2024-11-03T13:33:43Z

That didn't look right. It should pick up the cheap slots as you say. Can you post a log?

…

On Sun, 3 Nov 2024, 12:42 pbcjh10, ***@***.***> wrote: Is there a specific setting that will force charge the battery, or atleast make the system think about charging battery whenever we get a cheap IOG block? For example - battery currently empty (well 20% which is limit), just plugged in car and its 7p until 4pm. I'd like the battery to go "oh hey, crap solar today lets top up". Is this a generic problem or a setting I can tweak? or is it because I have cyclic discharge off? Like whenever its got a <= car sign here, why's it not also topping up the battery? image.png (view on web) <https://github.com/user-attachments/assets/82ff4632-692d-4814-a2be-b34ef038c1c8> — Reply to this email directly, view it on GitHub <#269 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ASVRJMCP6AH6BNKR4MMSZH3Z6YK2XAVCNFSM6AAAAABPB3MNY2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINJTGQYTINJUHE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

pbcjh10 · 2024-11-03T19:52:57Z

pv_opt.log
main.log
error.log
pv_opt2.log
pv_opt1.log
pv_opt3.log

voila

stevebuk1 · 2024-11-03T21:53:17Z

With the power consumption logging we had enabled, unfortunately the log files only stretch back an hour or so.

However, most of the testing on Pv-opt has been done with car plugins after 5pm and I suspect the problem is that reloads of the slot prices on a morning / early afternoon plugin don't go far enough in advance to create a decent charge plan (Pv_opt was originally written around Agile where prices don't become available until 4pm).

Its either that or the recent algorithm changes that Octopus have made in the last week - charge plans now update regularly. I've recently changed the code to load car slots every 10 minutes rather than just on plugin (and thats working as these are the "<= Car" entries you are seeing) but the prices fed to house battery charge planning are only reloaded on car plugin and will take 20 minutes to become active (delays added to ensure Octopus recalculations have time to complete).

I've created 17.0-Beta-13 with additional logging to try and get to the bottom of root cause.

Overwrite your local files with these:

https://github.com/stevebuk1/pv_opt/blob/dev/apps/pv_opt/pv_opt.py
https://github.com/stevebuk1/pv_opt/blob/dev/apps/pv_opt/pvpy.py
https://github.com/stevebuk1/pv_opt/blob/dev/apps/pv_opt/solis.py

and then please change the "debug_categories" entry in config.yaml to the following:

debug_categories: TBOC # Tariff and Pricing Debugging

pbcjh10 · 2024-11-06T17:59:42Z

Thanks Steve,

I've done as you suggested and attach logs, I gave it 36hrs or so to see what happened.
pv_opt.log
pv_opt2.log
pv_opt1.log

main.log
error.log
pv_opt3.log

Still weird and not charging up despite a plug in IOG charge from 18:30 today.

stevebuk1 · 2024-11-06T21:35:34Z

Unfortunately despite the reduction in logging text, the 3 logs provided just stretch back to 1pm today rather than covering the 36 hours.

In those logs, you've plugged the car in around 16:53 and it scheduled charging until 17:00pm, then again at 18:30pm. Pv_opt hasnt picked up the really short 7 minutes due to the way it searches for car slots - I've now fixed that and that is under test. The image you've provided shows Pv_opt has correctly picked up the 18:30 slot and intends to charge. The log runs out at 17:11, did it not charge at 18:30?

I note you've still got "Use Consumption History" turned on - turn it off and set a daily consumption with the slider that will appear. This should then mean Pv_opt has a more accurate idea of what your house is going to consume rather than being corrupted due to it not subtracting your 2nd Zappi load.

I think its worth increasing your log size and also generating 10 historic logs rather than the default 3. To do this, find your Appdaemon.yaml file and edit to include the two extra lines "log_generations" and "log_size" as below. Don't change any other lines - the below is from my system and may be different on yours. Make sure both entries are perfectly aligned with the entry
"filename: /homeassistant/appdaemon/pv_opt.log" otherwise errors are likely to occur:

  pv_opt_log:
     name: PV_Opt
     filename: /homeassistant/appdaemon/pv_opt.log
     log_generations: 9
     log_size: 10000000
     date_format: '%H:%M:%S'      
     format: '{asctime} {levelname:>8s}: {message}'

pbcjh10 · 2024-11-11T23:16:03Z

Well here you go! Left it running for a good while!
Despite having set things to override HA on start - I'm loosing the 'false' on use previous 7 days consumption.
main.log
error.log
pv_opt.log
main1.log

pv_opt7.log
pv_opt4.log
pv_opt3.log
pv_opt2.log
pv_opt1.log
pv_opt9.log
pv_opt6.log
pv_opt8.log
pv_opt5.log

Question remains why its ending charge at 05:30 ?
Surely it should atleast hold charge until 06:30 when the cheap rate ends?

I edited my dashboard Solis House Load Total - I presume thats supposed to be today? Rather than all time?
So I've edited in this sensor.solis_house_load graph;

To give this;

Any thoughts?

pbcjh10 · 2024-11-12T12:11:30Z

And another weird one; Battery is at baseline empty, on a cheap rate, but no plans to charge, despite naff all solar predicted.

stevebuk1 · 2024-11-12T12:15:12Z

Can you grab the pv_opt.log for your 12:10 scrrengrab please?

…

On Tue, 12 Nov 2024, 12:11 pbcjh10, ***@***.***> wrote: And another weird one; Battery is at baseline empty, on a cheap rate, but no plans to charge, despite naff all solar predicted. image.png (view on web) <https://github.com/user-attachments/assets/3fbe1795-f385-4696-a611-96d4bf6c2d3a> — Reply to this email directly, view it on GitHub <#269 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ASVRJMBAI4WICPF7L724WYD2AHWARAVCNFSM6AAAAABPB3MNY2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINZQGM3TAMRYHA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

pbcjh10 · 2024-11-12T12:18:33Z

pv_opt(1).log

These should cover it I think?
pv_opt999.log

stevebuk1 · 2024-11-12T15:23:22Z

These should cover it I think?

Yes that covers it. For completeness can you attach config.yaml as well.

I'll take a longer look through that and all the logs you've provided, but I can see three immediate issues:

The optimiser is supposed to run every 10 minutes but up until 5.30am when it sorts itself out its running continuously, every couple of minutes. This will mean that all defaults are loaded from config.yaml every time rather than remembering anything you previously setup on the HA dashboard. Not sure of the reason for this.
The Zappi car plugin detector isnt working properly, and car plugin detection is needed to load the new prices into Pv_opt, so Pv_opt is only seeing 23:30 to 05:30 at cheap rate. I think this is because I'd changed the search string to "myenergi" at the same time you renamed the entity to "Zappi", and its now picking up the carport plugin status which is not the one that triggers Octopus to give you a charge plan. I think the solution to this is to do the searches as currently done (I'll change back to Zappi) but also allow an entity remapping as part of config.yaml for your case / those who have renamed the Zappi entities and / or have multiple chargers.
At 22:53, Beta-14 was generally running ok and generating plans as expected (other than issues 1 and 2 above), but then something happened and Pv_opt lost connection with HA. I think at that point you'd rebooted HA and also Appdaemon, but from that point on whats been running is Pv_opt v3.1.7.1 which is the main production release of Pv_opt. If your HA automatically updates integrations via HACS this will be the cause, as Francis released a new production version a few nights ago (that doesnt contain the latest EV stuff).

I'll look at 1 and 2 and any others I find, in the interim please fix 3) by downloading the following .py files:
https://github.com/stevebuk1/pv_opt/blob/dev/apps/pv_opt/pv_opt.py
https://github.com/stevebuk1/pv_opt/blob/dev/apps/pv_opt/pvpy.py
https://github.com/stevebuk1/pv_opt/blob/dev/apps/pv_opt/solis.py

Be careful not to overwrite your config.yaml!

stevebuk1 · 2024-11-12T15:29:26Z

To give this;

Any thoughts?

This is now looking right.

stevebuk1 · 2024-11-12T15:32:29Z

edited my dashboard Solis House Load Total - I presume thats supposed to be today? Rather than all time?
So I've edited in this sensor.solis_house_load graph;

Also correct. HA stores the last 7 days by default. Your entity history image is showing around 2 days worth. The dashboard however will just extract the bit it needs to display the last day only.

pbcjh10 · 2024-11-12T15:33:09Z

Ahha yes thats probably it.

Here's config.yaml
config(2).yaml.log

fboundy · 2024-11-12T16:48:44Z

My guess that the continuous optimisation is because one of the entities that appDaemon is monitoring for its state change is changing - triggering a new optimisation. This is all the entities listed in DEFAULT_CONFIG. The list is saved as self.change.items in _expose_config. If an entity has got into this list which isn't manually controlled but is changed by PV Opt it will trigger an infitite loop.

Been there done that.

stevebuk1 · 2024-11-12T20:38:47Z

My guess that the continuous optimisation is because one of the entities that appDaemon is monitoring for its state change is changing - triggering a new optimisation. This is all the entities listed in DEFAULT_CONFIG. The list is saved as self.change.items in _expose_config. If an entity has got into this list which isn't manually controlled but is changed by PV Opt it will trigger an infitite loop.

Been there done that.

I'd realised this was a thing when adding a fix for #265 in this commit. stevebuk1@f9981da

There must be an errant entity being updated thats lying around somewhere......
The bit that I'm flumoxxed over is that I'm not getting a "State change detected for....." log entry that should be generated when optimise_state_change is called?

stevebuk1 · 2024-11-18T21:55:49Z

I have now updated Pv_opt to deal with multiple Zappis.

The entity names have to have the word "zappi" in the name somewhere so please rename the four zappi entities ( "plug_status" and "charge_added_session", for each Zappi) to include this.

All consumption sensors are autodetected and summed together, which should mean the consumption history reported by Pv_opt will just be of the house load, which is what you need for house battery charge plans based on historic consumption.

If found the logfile will report something like:

22:02:54     INFO: Attempting to autodetect Zappi consumption sensor(s)
22:02:54     INFO: Zappi consumption sensor(s) found:
22:02:54     INFO:   sensor.myenergi_zappi_********_charge_added_session
22:02:54     INFO:   sensor.myenergi_zappi_********_charge_added_session

(the ***** are redacted Zappi S/Ns that Pv_opt will automatically do for logging, so the two sensors above are actually different)

All plug status sensors are searched for, if there is more than one it will simply assume the first. (This means Pv_opt continues to work as is for users with single Zappis).
If this first entry happens to be the correct one, no further work is needed. You'll see something in your log file that looks like this:

21:38:36     INFO: Attempting to Autodetect Zappi Plugin status sensor
21:38:36     INFO:   myenergi_zappi********_plug_status
21:38:36     INFO:   myenergi_zappi_********_plug_status
21:38:36     INFO: 
21:38:36     INFO: Multiple Zappis found, setting myenergi_zappi_zzzzzzzzz_plug_status to trigger car charge planning updates.
21:38:36     INFO: Checking config.yaml for presence of id_zappi_plug_status...

(The ******* are serial numbers that PV_opt redacts automatically, but the log will print the zzzzzz S/N as you'll need to know which one its chosen. If you upload the log here please manually delete it).

If its picked up the wrong sensor, you'll need to tell it the right one in config.yaml by setting id_zappi_plug_status to the Zappi thats connected to IOG, something like the following:

  ev_charger: Zappi   # Default = None
  id_zappi_plug_status: sensor.myenergi_zappi_12345678_plug_status     # Default = autodetect

On startup the log will then confirm the entity has been mapped correctly:

21:38:36     INFO: 
21:38:36     INFO: id_zappi_plug_status value in config.yaml = sensor.myenergi_zappi_xxxxxxxxx_plug_status
21:38:36     INFO: Overriding auto detected plug_status sensor (myenergi_zappi_xxxxxxxx_plug_status) to mapping provided in config.yaml (sensor.myenergi_zappi_yyyyyyyy_plug_status)

New files here:
https://github.com/fboundy/pv_opt/blob/dev/apps/pv_opt/pv_opt.py
https://github.com/fboundy/pv_opt/blob/dev/apps/pv_opt/pvpy.py
https://github.com/fboundy/pv_opt/blob/dev/apps/pv_opt/solis.py

Probably best to just update the files, wait until Pv_opt cycles to idle and post the log, I can check things have been detected correctly.

I havent yet tracked down the bug that causes Pv_opt to cycle continuously.

*** edit: version is 3.19.0-Beta-16.

stevebuk1 · 2024-11-21T22:11:29Z

Version on the file links in the last post is now 3.19-0-Beta-17, but otherwise the rest applies.

stevebuk1 self-assigned this Sep 29, 2024

stevebuk1 mentioned this issue Sep 30, 2024

Coroutine took too long. ERROR: No data returned from HASS entity sensor.solmod_house_load #270

Closed

stevebuk1 added the query Query on function or performance label Oct 5, 2024

stevebuk1 changed the title ~~Entity not avaliable: pvopt_opt_cost, pvopt_base_cost~~ Multiple Zappi support, Optimiser runs continuously Nov 12, 2024

stevebuk1 added bug Something isn't working enhancement New feature or request and removed query Query on function or performance labels Nov 12, 2024

stevebuk1 changed the title ~~Multiple Zappi support, Optimiser runs continuously~~ Multiple Zappi support, Optimiser occasionally runs continuously Nov 12, 2024

Multiple Zappi support, Optimiser occasionally runs continuously #269

Multiple Zappi support, Optimiser occasionally runs continuously #269

Comments

pbcjh10 commented Sep 29, 2024

stevebuk1 commented Sep 29, 2024 • edited Loading

stevebuk1 commented Sep 29, 2024 • edited Loading

pbcjh10 commented Sep 29, 2024

stevebuk1 commented Oct 1, 2024

pbcjh10 commented Oct 2, 2024 • edited Loading

stevebuk1 commented Oct 2, 2024 • edited Loading

stevebuk1 commented Oct 2, 2024 • edited Loading

pbcjh10 commented Oct 2, 2024 via email

pbcjh10 commented Oct 2, 2024 via email

stevebuk1 commented Oct 2, 2024

stevebuk1 commented Oct 2, 2024

pbcjh10 commented Oct 7, 2024

stevebuk1 commented Oct 7, 2024

pbcjh10 commented Oct 20, 2024

stevebuk1 commented Oct 20, 2024 • edited Loading

stevebuk1 commented Oct 20, 2024 • edited Loading

pbcjh10 commented Oct 20, 2024

pbcjh10 commented Oct 20, 2024

stevebuk1 commented Oct 20, 2024 • edited Loading

stevebuk1 commented Oct 20, 2024 • edited Loading

pbcjh10 commented Oct 21, 2024

stevebuk1 commented Oct 21, 2024 • edited Loading

stevebuk1 commented Oct 21, 2024

pbcjh10 commented Oct 21, 2024

pbcjh10 commented Oct 21, 2024

stevebuk1 commented Oct 21, 2024

stevebuk1 commented Oct 21, 2024 • edited Loading

stevebuk1 commented Nov 2, 2024

pbcjh10 commented Nov 2, 2024

stevebuk1 commented Nov 2, 2024 • edited Loading

stevebuk1 commented Nov 2, 2024

stevebuk1 commented Nov 2, 2024

pbcjh10 commented Nov 2, 2024

stevebuk1 commented Nov 2, 2024 • edited Loading

pbcjh10 commented Nov 3, 2024

stevebuk1 commented Nov 3, 2024 via email

pbcjh10 commented Nov 3, 2024

stevebuk1 commented Nov 3, 2024 • edited Loading

pbcjh10 commented Nov 6, 2024

stevebuk1 commented Nov 6, 2024 • edited Loading

pbcjh10 commented Nov 11, 2024 • edited Loading

pbcjh10 commented Nov 12, 2024

stevebuk1 commented Nov 12, 2024 via email

pbcjh10 commented Nov 12, 2024

stevebuk1 commented Nov 12, 2024 • edited Loading

stevebuk1 commented Nov 12, 2024

stevebuk1 commented Nov 12, 2024

pbcjh10 commented Nov 12, 2024

fboundy commented Nov 12, 2024

stevebuk1 commented Nov 12, 2024 • edited Loading

stevebuk1 commented Nov 18, 2024 • edited Loading

stevebuk1 commented Nov 21, 2024

stevebuk1 commented Sep 29, 2024 •

edited

Loading

stevebuk1 commented Sep 29, 2024 •

edited

Loading

pbcjh10 commented Oct 2, 2024 •

edited

Loading

stevebuk1 commented Oct 2, 2024 •

edited

Loading

stevebuk1 commented Oct 2, 2024 •

edited

Loading

stevebuk1 commented Oct 20, 2024 •

edited

Loading

stevebuk1 commented Oct 20, 2024 •

edited

Loading

stevebuk1 commented Oct 20, 2024 •

edited

Loading

stevebuk1 commented Oct 20, 2024 •

edited

Loading

stevebuk1 commented Oct 21, 2024 •

edited

Loading

stevebuk1 commented Oct 21, 2024 •

edited

Loading

stevebuk1 commented Nov 2, 2024 •

edited

Loading

stevebuk1 commented Nov 2, 2024 •

edited

Loading

stevebuk1 commented Nov 3, 2024 •

edited

Loading

stevebuk1 commented Nov 6, 2024 •

edited

Loading

pbcjh10 commented Nov 11, 2024 •

edited

Loading

stevebuk1 commented Nov 12, 2024 •

edited

Loading

stevebuk1 commented Nov 12, 2024 •

edited

Loading

stevebuk1 commented Nov 18, 2024 •

edited

Loading