[Bug]: config.yaml set up and GUI setup not working for EKM Mittelsachsen (ics) #2907

riegera2412 · 2024-10-23T10:43:25Z

I Have A Problem With:

A specific source, The integration in general

What's Your Problem

After updating wcs to 2.30 my manual configuration in the config.yaml doesn't work. I tried downgrading to 2.2.0 and 2.1.0 but neither does the trick. I used a downloaded ics from my waste collection company that worked flawlessly for years. The config is shown in the config section.

I get three log messages:

ics source not available. I tried different paths to the ics file, but neither of them worked. Before it was in config/www/file.ics

Logger: custom_components.waste_collection_schedule.waste_collection_schedule.source_shell
Quelle: custom_components/waste_collection_schedule/waste_collection_schedule/source_shell.py:221
Integration: Waste Collection Schedule ([Dokumentation](https://github.com/mampfes/hacs_waste_collection_schedule#readme))
Erstmals aufgetreten: 12:25:50 (1 Vorkommnisse)
Zuletzt protokolliert: 12:25:50

source not found: ics

setup of wcs is taking more than 10s warning

Logger: homeassistant.setup
Quelle: runner.py:189
Erstmals aufgetreten: 12:25:59 (1 Vorkommnisse)
Zuletzt protokolliert: 12:25:59

Setup of waste_collection_schedule is taking over 10 seconds.

Error while setting up waste_collection_schedule platform for sensor

Logger: homeassistant.components.sensor
Quelle: helpers/entity_platform.py:361
Integration: Sensor (Dokumentation, Probleme)
Erstmals aufgetreten: 12:26:17 (5 Vorkommnisse)
Zuletzt protokolliert: 12:26:17

Error while setting up waste_collection_schedule platform for sensor
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 361, in _async_setup_platform
    await asyncio.shield(awaitable)
  File "/config/custom_components/waste_collection_schedule/sensor.py", line 147, in async_setup_platform
    raise ValueError(
ValueError: source_index 0 out of range (0--1) please check your sensor configuration

Next, I commented out the configuration and tried the setup via the GUI, but when choosing Germany and then my source (EKM Mittelsachsen (ics)), I get an unknown error message.

In the logs the following warning appears:

Logger: homeassistant.util.loop
Quelle: util/loop.py:136
Erstmals aufgetreten: 12:29:13 (1 Vorkommnisse)
Zuletzt protokolliert: 12:29:13

Detected blocking call to open with args (PosixPath('/config/custom_components/waste_collection_schedule/sources.json'),) inside the event loop by custom integration 'waste_collection_schedule' at custom_components/waste_collection_schedule/config_flow.py, line 331: with p.open(encoding="utf-8") as json_file: (offender: /config/custom_components/waste_collection_schedule/config_flow.py, line 331: with p.open(encoding="utf-8") as json_file:), please report it to the author of the 'waste_collection_schedule' custom integration For developers, please see https://developers.home-assistant.io/docs/asyncio_blocking_operations/#open Traceback (most recent call last): File "<frozen runpy>", line 198, in _run_module_as_main File "<frozen runpy>", line 88, in _run_code File "/usr/src/homeassistant/homeassistant/__main__.py", line 223, in <module> sys.exit(main()) File "/usr/src/homeassistant/homeassistant/__main__.py", line 209, in main exit_code = runner.run(runtime_conf) File "/usr/src/homeassistant/homeassistant/runner.py", line 189, in run return loop.run_until_complete(setup_and_run_hass(runtime_config)) File "/usr/local/lib/python3.12/asyncio/base_events.py", line 674, in run_until_complete self.run_forever() File "/usr/local/lib/python3.12/asyncio/base_events.py", line 641, in run_forever self._run_once() File "/usr/local/lib/python3.12/asyncio/base_events.py", line 1990, in _run_once handle._run() File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, *self._args) File "/usr/local/lib/python3.12/site-packages/aiohttp/web_protocol.py", line 556, in start task = asyncio.Task(coro, loop=loop, eager_start=True) File "/usr/local/lib/python3.12/site-packages/aiohttp/web_protocol.py", line 477, in _handle_request resp = await request_handler(request) File "/usr/local/lib/python3.12/site-packages/aiohttp/web_app.py", line 559, in _handle return await handler(request) File "/usr/local/lib/python3.12/site-packages/aiohttp/web_middlewares.py", line 117, in impl return await handler(request) File "/usr/src/homeassistant/homeassistant/components/http/security_filter.py", line 92, in security_filter_middleware return await handler(request) File "/usr/src/homeassistant/homeassistant/components/http/forwarded.py", line 210, in forwarded_middleware return await handler(request) File "/usr/src/homeassistant/homeassistant/components/http/request_context.py", line 26, in request_context_middleware return await handler(request) File "/usr/src/homeassistant/homeassistant/components/http/ban.py", line 85, in ban_middleware return await handler(request) File "/usr/src/homeassistant/homeassistant/components/http/auth.py", line 242, in auth_middleware return await handler(request) File "/usr/src/homeassistant/homeassistant/components/http/headers.py", line 32, in headers_middleware response = await handler(request) File "/usr/src/homeassistant/homeassistant/helpers/http.py", line 73, in handle result = await handler(request, **request.match_info) File "/usr/src/homeassistant/homeassistant/components/http/decorators.py", line 81, in with_admin return await func(self, request, *args, **kwargs) File "/usr/src/homeassistant/homeassistant/components/http/data_validator.py", line 74, in wrapper return await method(view, request, data, *args, **kwargs) File "/usr/src/homeassistant/homeassistant/components/config/config_entries.py", line 172, in post return await self._post_impl(request, data) File "/usr/src/homeassistant/homeassistant/components/config/config_entries.py", line 179, in _post_impl return await super()._post_impl(request, data) File "/usr/src/homeassistant/homeassistant/helpers/data_entry_flow.py", line 84, in _post_impl result = await self._flow_mgr.async_init( File "/usr/src/homeassistant/homeassistant/config_entries.py", line 1287, in async_init flow, result = await self._async_init(flow_id, handler, context, data) File "/usr/src/homeassistant/homeassistant/config_entries.py", line 1312, in _async_init flow = await self.async_create_flow(handler, context=context, data=data) File "/usr/src/homeassistant/homeassistant/config_entries.py", line 1505, in async_create_flow flow = handler() File "/config/custom_components/waste_collection_schedule/config_flow.py", line 306, in __init__ self._sources = self._get_source_list() File "/config/custom_components/waste_collection_schedule/config_flow.py", line 331, in _get_source_list with p.open(encoding="utf-8") as json_file:

There is also an HA error message at the moment of setting up wcs.

Logger: aiohttp.server
Quelle: /usr/local/lib/python3.12/site-packages/aiohttp/web_protocol.py:448
Erstmals aufgetreten: 12:51:24 (1 Vorkommnisse)
Zuletzt protokolliert: 12:51:24

Error handling request
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/aiohttp/web_protocol.py", line 477, in _handle_request
    resp = await request_handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aiohttp/web_app.py", line 559, in _handle
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aiohttp/web_middlewares.py", line 117, in impl
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/security_filter.py", line 92, in security_filter_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/forwarded.py", line 210, in forwarded_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/request_context.py", line 26, in request_context_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/ban.py", line 85, in ban_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/auth.py", line 242, in auth_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/headers.py", line 32, in headers_middleware
    response = await handler(request)
               ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/http.py", line 73, in handle
    result = await handler(request, **request.match_info)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/decorators.py", line 81, in with_admin
    return await func(self, request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/config/config_entries.py", line 222, in post
    return await super().post(request, flow_id)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/data_validator.py", line 74, in wrapper
    return await method(view, request, data, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/data_entry_flow.py", line 122, in post
    result = await self._flow_mgr.async_configure(flow_id, data)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 370, in async_configure
    result = await self._async_configure(flow_id, user_input)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 417, in _async_configure
    result = await self._async_handle_step(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 520, in _async_handle_step
    result: _FlowResultT = await getattr(flow, method)(user_input)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/waste_collection_schedule/config_flow.py", line 366, in async_step_source
    return await self.async_step_args()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/waste_collection_schedule/config_flow.py", line 548, in async_step_args
    schema, module = await self.__get_arg_schema(
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/waste_collection_schedule/config_flow.py", line 389, in __get_arg_schema
    module = await self.hass.async_add_executor_job(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/util/loop.py", line 200, in protected_loop_func
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 995, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/source/ics.py", line 9, in <module>
    from waste_collection_schedule.service.ICS import ICS
  File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/service/ICS.py", line 7, in <module>
    from icalevents import icalevents
  File "/usr/local/lib/python3.12/site-packages/icalevents/icalevents.py", line 3, in <module>
    from .icalparser import parse_events, Event
  File "/usr/local/lib/python3.12/site-packages/icalevents/icalparser.py", line 18, in <module>
    from icalendar.windows_to_olson import WINDOWS_TO_OLSON
ModuleNotFoundError: No module named 'icalendar.windows_to_olson'

Any help and/or advice is greatly appreciated.

A.

Source (if relevant)

No response

Logs

see above

Relevant Configuration

configuration.yaml:
...
waste_collection_schedule: !include waste.yaml
...


waste.yaml:

sources:
  - name: ics
    args:
      url: https://www.ekm-mittelsachsen.de/service-dienstleistungen/entsorgungstermine-abfallkalender?tx_ekmabfallkalender_abfallkalender%5Baction%5D=getIcal&tx_ekmabfallkalender_abfallkalender%5Bcity_id%5D=11400&tx_ekmabfallkalender_abfallkalender%5Bstreet_id%5D=52187&tx_ekmabfallkalender_abfallkalender%5Byear%5D={%Y}}&cHash=83925ca887a5af7bdb638a5df97c696e&tx_ekmabfallkalender_abfallkalender[types][]=r&tx_ekmabfallkalender_abfallkalender[types][]=p&tx_ekmabfallkalender_abfallkalender[types][]=l&tx_ekmabfallkalender_abfallkalender[types][]=b&tx_ekmabfallkalender_abfallkalender[trigger_days]=0
      #file: "homeassistant/www/abfallkalenderFreiberg2022_2023_2024.ics"
    customize:
      - type: Bio
        alias: biomuell
        icon: mdi:flower-outline
      - type: Leichtstoffe
        alias: leichtstoffe
        icon: mdi:recycle
      - type: Papier
        alias: papier
        icon: mdi:trash-can-outline
      - type: Restmuell
        alias: restmuell
        icon: mdi:trash-can          
fetch_time: "04:00"
day_switch_time: "10:00"

Checklist Source Error

Use the example parameters for your source (often available in the documentation) (don't forget to restart Home Assistant after changing the configuration)
Checked that the website of your service provider is still working
Tested my attributes on the service provider website (if possible)
I have tested with the latest version of the integration (master) (for HACS in the 3 dot menu of the integration click on "Redownload" and choose master as version)

Checklist Sensor Error

Checked in the Home Assistant Calendar tab if the event names match the types names (if types argument is used)

Required

I have searched past (closed AND opened) issues to see if this bug has already been reported, and it hasn't been.
I understand that people give their precious time for free, and thus I've done my very best to make this problem as easy as possible to investigate.

The text was updated successfully, but these errors were encountered:

5ila5 · 2024-10-23T12:04:10Z

ModuleNotFoundError: No module named 'icalendar.windows_to_olson'

Sounds like icalendar >= 6.0.0 is installed, but icalevents (Libraary we use to parse ICS files) currently only supports icalender < 6.0.0 (see jazzband/icalevents#145).

I'm not quite sure how this happens, as icalevents dependency should be respected when HA installs components dependencies

ReneNulschDE · 2024-10-23T12:14:41Z

Just guessing: the manifest.json has no version identifier for icalendar. Could be that the user has a fresh install and the highest version was installed... Looks like the ics_v1 service is using icalendar.

5ila5 · 2024-10-23T12:35:03Z

Not exactly sure how HA resolves dependencies but as icalevents requires icalendar==5.0.13 so HA (I think the se uv now) should not install icalendar 6....

I doubt ics_v1 is the problem here, as it does not use the changed functions/modules of icalendar 6 (the stack trace also shows File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/service/ICS.py", line 7, in <module> from icalevents import icalevents)

It seems to be issue with multiple instances as there a few source not found issues reported here.

The error message is pretty bad as every import error inside a source will trigger this error message. I fixed this in #2908 (will test and merge it later)

ReneNulschDE · 2024-10-23T13:14:49Z

... When I install in a clean venv environment with pip:
pip install icalendar icalevents!=0.1.28
icalendar gets installed with version
icalendar-5.0.13 icalevents-0.1.29

when I use uv with:
uv pip install icalendar icalevents!=0.1.28
the result is:

+ icalendar==6.0.1
+ icalevents==0.1.25

see that icalevents get installed with 0.1.25

I'll check it in a fresh HA environement now.

Update:
No this is not the reason:

after a fresh install ha install and adding WCS via UI the result is:

icalendar                          5.0.13
icalevents                         0.1.29

ReneNulschDE · 2024-10-23T13:28:54Z

@riegera2412 : Could you share the list of other custom_components that you have installed? And please share some context about your environment. This would help us to create a solution description as this is not a code more an environment problem.

ReneNulschDE · 2024-10-23T13:59:58Z

@5ila5 : Besides my comments above...

It could happen that icalendar >5 gets installed when you try to install/use a custom_component with a hard dependency to icalendar>=6.0.0.

Flow:

Install HA
copy WCS and for example the cutom component "ics_calendar" into the custom_components folder
restart and check that no icalendar file is installed (pip list)
add integration "wcs" in the ha frontend --> icalendar version 5.0.13 is installed
add integration "ics_calendar" in the ha frontend --> icalendar version 6.0.1is installed and WCS is failing

ReneNulschDE · 2024-10-23T19:31:41Z

Potential solution

riegera2412 · 2024-10-23T19:53:29Z

@riegera2412 : Could you share the list of other custom_components that you have installed? And please share some context about your environment. This would help us to create a solution description as this is not a code more an environment problem.

I am running HassOS 2024.10.3 on a HP T620 Thin Client.

Also using the RaspberryMatic Addon to integrate Hometic IP hardware.
For Ikea hardware I use a igbee 3 USB Dongle.

Not sure what other information you might need. If you require any specific information please ask.

EDIT: I have ics_calendar installed. Just checked the manifest.json and it specifies the following requirements:

"requirements": ["icalendar>=6.0.0","python-dateutil>=2.9.0.post0","pytz>=2024.1","recurring_ical_events>=3.3.2","ics>=0.7.2","arrow"],

Guess that could be the problem?

ReneNulschDE · 2024-10-23T19:54:54Z

thx, please check the solution that I added above.

riegera2412 · 2024-10-23T20:35:07Z

Potential solution

Worked like a charm! Thanks a lot!

5ila5 · 2024-10-24T11:07:36Z

icalevents 0.1.25 did not define a specific version of icalendar as dependency, this is why it installs it like this

We could set icalevents>=0.1.26,!=0.1.28 as dependency this would also force icalendar<6 as long as jazzband/icalevents#145 is not resolved and a new version of icalevents is released.

But this approach would lead to dependency conflicts with integrations like ics_calendar.

But this is probably still the way to go, so users need to decide and check what they want.

I opened a PR for this (#2911)

@ReneNulschDE what do you think?

ReneNulschDE · 2024-10-24T13:43:40Z

This will not have an impact based on my tests. Whenever the user are upgrading to ics_calendar 5.0.4, the icalendar >=6 will be installed. I checked all HACS-published integrations and only ics_calendar has this dependency.

My opinion: Let it as it is, say do not upgrade ics_calendar higher then 5.0.3, in case you have it done, use the process that I published - and wait for a new version of icalevents. And, you should pin the icalendar version in the manifest.json.

Maybe you could ask the maintainer of ics_calendar to revert to 5.0.3 or go back to the old icalendar lib? your component has 10x more users than ics_calendar. (could be offending... :-()

btw the command that I used to download the hacs-repos (found on discord some month before, can't remember who gets the credit) :
for repo in $(curl -sSL https://data-v2.hacs.xyz/integration/repositories.json | jq -r '.[]'); do if [ ! -d "${repo}" ]; then echo mkdir "${repo%%/*}" && git clone "[email protected]:${repo}" "${repo}"; else git -C "${repo}" pull origin; fi; done

5ila5 · 2024-10-25T16:51:41Z

Pinning icalevents>=0.1.26 should have the same effect as pinning icalendar, as icalevents>=0.1.26 defines a fixed version of icalevents as dependency. But if icalevents publishes a new version with support for icalendar=6 the dependency conflict should vanish without the need for a release of WCS

riegera2412 changed the title ~~[Bug]: config-yaml set up and GUI setup not working for EKM Mittelsachsen (ics)~~ [Bug]: config.yaml set up and GUI setup not working for EKM Mittelsachsen (ics) Oct 23, 2024

5ila5 mentioned this issue Oct 23, 2024

source shell improve error message when a import inside the source script fails #2908

Merged

This was referenced Oct 23, 2024

[Bug]: ICS source fail: 'NoneType' object has no attribute 'calendar_title' #2896

Open

[Bug]: kwu_de source not found #2893

Open

[Bug]: ICS Sensor Not longer working #2180

Closed

5ila5 mentioned this issue Oct 24, 2024

fix dependency issue with icalendar and icalevents #2911

Merged

5ila5 mentioned this issue Oct 30, 2024

[Bug]: Waste Collection Schedule suddenly stopped working #2948

Closed

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug]: config.yaml set up and GUI setup not working for EKM Mittelsachsen (ics) #2907

[Bug]: config.yaml set up and GUI setup not working for EKM Mittelsachsen (ics) #2907

riegera2412 commented Oct 23, 2024 •

edited

Loading

5ila5 commented Oct 23, 2024

ReneNulschDE commented Oct 23, 2024 •

edited

Loading

5ila5 commented Oct 23, 2024

ReneNulschDE commented Oct 23, 2024 •

edited

Loading

ReneNulschDE commented Oct 23, 2024 •

edited

Loading

ReneNulschDE commented Oct 23, 2024 •

edited

Loading

ReneNulschDE commented Oct 23, 2024

riegera2412 commented Oct 23, 2024 •

edited

Loading

ReneNulschDE commented Oct 23, 2024

riegera2412 commented Oct 23, 2024

5ila5 commented Oct 24, 2024

ReneNulschDE commented Oct 24, 2024 •

edited

Loading

5ila5 commented Oct 25, 2024

[Bug]: config.yaml set up and GUI setup not working for EKM Mittelsachsen (ics) #2907

[Bug]: config.yaml set up and GUI setup not working for EKM Mittelsachsen (ics) #2907

Comments

riegera2412 commented Oct 23, 2024 • edited Loading

I Have A Problem With:

What's Your Problem

Source (if relevant)

Logs

Relevant Configuration

Checklist Source Error

Checklist Sensor Error

Required

5ila5 commented Oct 23, 2024

ReneNulschDE commented Oct 23, 2024 • edited Loading

5ila5 commented Oct 23, 2024

ReneNulschDE commented Oct 23, 2024 • edited Loading

ReneNulschDE commented Oct 23, 2024 • edited Loading

ReneNulschDE commented Oct 23, 2024 • edited Loading

ReneNulschDE commented Oct 23, 2024

riegera2412 commented Oct 23, 2024 • edited Loading

ReneNulschDE commented Oct 23, 2024

riegera2412 commented Oct 23, 2024

5ila5 commented Oct 24, 2024

ReneNulschDE commented Oct 24, 2024 • edited Loading

5ila5 commented Oct 25, 2024

riegera2412 commented Oct 23, 2024 •

edited

Loading

ReneNulschDE commented Oct 23, 2024 •

edited

Loading

ReneNulschDE commented Oct 23, 2024 •

edited

Loading

ReneNulschDE commented Oct 23, 2024 •

edited

Loading

ReneNulschDE commented Oct 23, 2024 •

edited

Loading

riegera2412 commented Oct 23, 2024 •

edited

Loading

ReneNulschDE commented Oct 24, 2024 •

edited

Loading