Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue with March on energy dashboard #16058

Closed
3 tasks done
Kratax opened this issue Apr 4, 2023 · 24 comments
Closed
3 tasks done

Issue with March on energy dashboard #16058

Kratax opened this issue Apr 4, 2023 · 24 comments
Labels

Comments

@Kratax
Copy link

Kratax commented Apr 4, 2023

Checklist

  • I have updated to the latest available Home Assistant version.
  • I have cleared the cache of my browser.
  • I have tried a different browser to see if it is related to my browser.

Describe the issue you are experiencing

I have an issue with March on energy dashboard. Although I can see in detail normal on monthly detail, in yearly detail, there is no March and the values are included in Apr. Second issue I have (not sure if related) is one missing day in March corresponding to date when time has moved to summer time by 1 hour. I have checked on Facebook group and it seems that I am not the only one. Energy stat delivered via Goodwe integration, i.e. with solar production as visible from the charts.

1
2
3

Describe the behavior you expected

Sum up of Mar values included in Mar on yearly chart
No issue with DST within March

Steps to reproduce the issue

Go to energy dashboard - select Month - March; then Month - April; then Year

What version of Home Assistant Core has the issue?

2023.3.6

What was the last working version of Home Assistant Core?

No response

In which browser are you experiencing the issue with?

Firefox 111.0.1 (64 bit); Chrome Version 111.0.5563.146 (Official Build) (x86_64)

Which operating system are you using to run this browser?

macOs Monterey 12.6.4

State of relevant entities

No response

Problem-relevant frontend configuration

No response

Javascript errors shown in your browser console/inspector

No response

Additional information

1
2
3

@Kratax
Copy link
Author

Kratax commented Apr 14, 2023

Anybody able to look at this? Thanks a lot!

@Kratax
Copy link
Author

Kratax commented Apr 14, 2023

One more point that I have found, apparently, what is visible on the chart (yearly) is March number in April column and nothing for April... again, I checked the database and on an examaple of sensor feeding PV production, there is no inconsistency in data, using this script, I can see hourly values, daily ok as well, ... is it the one (table and column) used for creating the chart?

SELECT datetime(start_ts, 'unixepoch'), datetime(created_ts, 'unixepoch'), state, sum from statistics where metadata_id == 237 and date(start_ts, 'unixepoch') between '2023-02-01' and '2023-04-22'

@Kratax
Copy link
Author

Kratax commented Apr 14, 2023

Adding info to database - I ran
sqlite3 home-assistant_v2.db "PRAGMA quick_check"
no issue identified (result OK)

@Kratax
Copy link
Author

Kratax commented Apr 14, 2023

Adding one more info, in statistics chart, I have the following lines... aparently there is some issue with Mar data, reported the same number as for Feb. But in statistics database, this is not the case... could you advice me from which table is this value calculated?
Screenshot 2023-04-14 at 23 30 42
Screenshot 2023-04-14 at 23 29 41 1

@Kratax
Copy link
Author

Kratax commented Apr 24, 2023

Anybody has any advice / view on this? Thanks a lot.

@Kratax
Copy link
Author

Kratax commented May 1, 2023

After Apr moth end looks like this...

image

@Kratax
Copy link
Author

Kratax commented Jun 2, 2023

Still nobody can help? June works and continues well, ie really seems to be connect DST and the way how data are interpreted by frontend...

@karwosts
Copy link
Contributor

karwosts commented Jun 2, 2023

It's going to be pretty hard to remotely debug what issues you're having on your specific system with your specific dataset.

If you really want to dig into this further, I would suggest attaching a development frontend onto your HA and start digging into the debugging to see where things are going off.

I'm happy to work with you to offer advice if you want to go down that road, but without having access to your system and seeing the actual data in the frontend debugger it's difficult for me to really say what's going on. My month looks fine.

Here's some info if you feel like going down that path.
https://developers.home-assistant.io/docs/frontend/development/

@Kratax
Copy link
Author

Kratax commented Jun 3, 2023

I can try that ... I a bit hoped that sharing data / database backup would be possible and sufficient as well to replicate on developer side but if not, I will go through the link you provided.

@karwosts
Copy link
Contributor

karwosts commented Jun 3, 2023

Well I guess if you did want to make a backup and upload all your data somewhere I might take a look at it out of curiousity, but I'm not sure where you would host that, or how to deal with any privacy implications, and I can't promise I would actually spend any significant time on it or claim I can provide a fix (I just do minor bugfixes from time to time as a volunteer).

You'll be your own best advocate if you have the ability to look into it yourself, you've noticed by now that there's not a huge developer attention paid to bug reports that don't affect a large number of people.

@Kratax
Copy link
Author

Kratax commented Jun 17, 2023

Hi @karwosts ,
thanks a lot for the offer, I managed to duplicate my production system on testing HW, i.e. install on raspberry HAOS, still I can see the same issue as described above, i.e. specifically on energy tab, for year, there is missing March. On monthly view, I can see all days in March...
I.e. how to debug? Any help with that? The reference you mentioned goes to forking development version of frontend...
Thanks a lot for any help!

@Kratax
Copy link
Author

Kratax commented Jun 30, 2023

Hi @karwosts , any further advice how to proceed? Thanks a lot!!

@karwosts
Copy link
Contributor

Hmm, sorry its difficult for me to say from here.

I guess the initial idea was you could setup a development environment, which would allow you to use the debugger, though I know that's a big task.

Maybe one thing that could be done is just export all your data from statistics table for these entities in question to a csv and upload it somewhere to browse, see if anything looks off?

@Kratax
Copy link
Author

Kratax commented Jun 30, 2023

Well, as I noted above, I have already testing environment on separate HW... i.e. I have there separate HA with restored dtb... and see the same behaviour. Not sure how can I use debugger or share more info ...

Regarding export of data - I did that already - you can see it below... the point is that there does not seem to be anything strange...

home-assistant/core#91033

@karwosts
Copy link
Contributor

Sorry don't know how to help you further.

@karwosts
Copy link
Contributor

Maybe check out the community forums to see if anyone else has hit this issue.

@Kratax
Copy link
Author

Kratax commented Jun 30, 2023

I've tried that as well but the same - no response / explanation. I have only bunch of people on facebook forum seeing the same behaviour. Unfortunatelly no one being developer...

@GiZMoSK1221
Copy link

hi, I have same problem.
I tried to look at the data, found only one suspicious thing:
obrazek

when you decode unixtimes, there is a hole in create and start time, but in different rows

SELECT from_unixtime(s.created_ts) as cr, from_unixtime(s.start_ts) as st, s.* FROM `statistics` s where metadata_id=35 and start is not null and EXTRACT(YEAR FROM start)=2022 and EXTRACT(month FROM start)=3 and EXTRACT(day FROM start) in (26,27)
order by id asc;

maybe someone, who doesn't have this problem, could post his data. Just change metdadata_id to one, you are using in energy dashboard

@Kratax
Copy link
Author

Kratax commented Sep 22, 2023

@GiZMoSK1221 I have opened another issue in core as I was not sure where the problem is, there is as well dump of my database...

home-assistant/core#91033

@karwosts
Copy link
Contributor

maybe someone, who doesn't have this problem, could post his data.

I tried your SQL query in SQLIte Web but it's not liking it, can you double check it? (I'm not as much of a SQL expert).

image

@GiZMoSK1221
Copy link

@karwosts i'm running mariadb
this is the query for sqlite default db:
SELECT datetime(s.created_ts, 'unixepoch', 'localtime') as cr, datetime(s.start_ts, 'unixepoch', 'localtime') as st, s.* FROM statistics s where metadata_id=35 and start is not null and strftime('%Y', start)='2022' and strftime('%m',start)='03' and strftime('%d',start) in ('26','27') order by id asc;

@karwosts
Copy link
Contributor

That didn't quite work for me, but I think I figured it out (replaced start with st in the strftimes.

SELECT datetime(s.created_ts, 'unixepoch', 'localtime') as cr, datetime(s.start_ts, 'unixepoch', 'localtime') as st, s.* FROM statistics s where metadata_id=56 and strftime('%Y', st)='2023' and strftime('%m',st)='03' and strftime('%d',st) in ('11','12') order by id asc;

Also I realize my DST is on March 12, not March 27, but here's my data from my DST window:

<style> </style>
cr st id created metadata_id start mean min max last_reset state sum created_ts start_ts last_reset_ts
3/11/2023 21:00 3/11/2023 20:00 1153942   56           2965.345 2965.345 1678597210 1678593600  
3/11/2023 22:00 3/11/2023 21:00 1154076   56           2965.865 2965.865 1678600810 1678597200  
3/11/2023 23:00 3/11/2023 22:00 1154208   56           2965.865 2965.865 1678604410 1678600800  
3/12/2023 0:00 3/11/2023 23:00 1154340   56           2965.865 2965.865 1678608010 1678604400  
3/12/2023 1:00 3/12/2023 0:00 1154472   56           2965.865 2965.865 1678611610 1678608000  
3/12/2023 3:00 3/12/2023 1:00 1154604   56           2965.865 2965.865 1678615210 1678611600  
3/12/2023 4:00 3/12/2023 3:00 1154736   56           2967.062 2967.062 1678618810 1678615200  
3/12/2023 5:00 3/12/2023 4:00 1154868   56           2967.062 2967.062 1678622410 1678618800  
3/12/2023 6:00 3/12/2023 5:00 1155000   56           2967.418 2967.418 1678626010 1678622400  

Chart for March 12:
image

Chart for March:
image

Copy link

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates.
Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍
This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

@github-actions github-actions bot added the stale label Dec 24, 2023
@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Dec 31, 2023
@Kratax
Copy link
Author

Kratax commented Apr 2, 2024

Just an update from this DST change - the issue is persisting... btw not related to one integration but seems to be general issue how database is solving DST... see couple of charts showing that... ie CO2 sensor, temperature sensor, and energy sensor

obrazek
obrazek
obrazek

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants