Simplifying User Experience for NSPanel Firmware

[edwardtfn]

Description:

This request aims to transform our solution into a more user-friendly, plug-and-play system. The goal is to minimize technical barriers for entry-level users, particularly those who face challenges in the compiling process due to limited system resources or lack of experience.

Current Challenges:

1. Compiling Difficulties: Users with limited system resources often struggle to compile the firmware, especially for complex projects like ours.
2. Technical Intimidation: The process of compiling, flashing, and setting up can be daunting for entry-level users. The technical terminology and steps involved may discourage them from engaging with the solution.
3. Initial Setup Hurdles: The first-time flashing process can be challenging and off-putting for non-technical users.
4. YAML Settings: The need to configure YAML settings, including downloading TFT to a local HTTP server and manually typing the URL, often leads to failures in the setup process. This not only causes user frustration but also overloads our support channels.

Proposed Enhancements:

1. User-Friendly Interface for Settings: All settings should be accessible and modifiable through an intuitive graphical user interface (GUI), eliminating the need for technical know-how post-initial setup.
2. Simplified Firmware Update: Implement a one-click firmware update system within the UI, similar to commercial systems. This feature would allow users to keep their devices up-to-date without technical hurdles.
3. Streamlined Onboarding Process: Develop a simplified onboarding process for the network, making it straightforward and user-friendly. This could include step-by-step guides, clear instructions, and automated procedures wherever possible.
4. Enhanced Support and Documentation: Provide comprehensive, easy-to-understand documentation and user support. This can include FAQs, video tutorials, and a glossary of terms for those unfamiliar with technical jargon.
5. Optimization for Low-Resource Systems: Optimize the firmware to be more accommodating of systems with limited resources, ensuring a wider range of users can compile and use the firmware successfully.

Challenges:

1. ESPHome and Home Assistant Limitations: Our solution relies on ESPHome and Home Assistant, which come with their own set of limitations. We need to work within these constraints while trying to enhance user experience.
2. Balancing Functionality and Performance: As we continue to add new functionalities, maintaining a balance between feature richness and the lightness of the firmware for easy compiling becomes challenging.
3. Customizability vs. User-Friendliness: Our desire to support diverse user needs and maintain high customizability can sometimes conflict with our goal of creating a user-friendly, plug-and-play experience.
4. Resource Constraints: There may be limitations in terms of development resources and expertise available, which could affect the pace and scope of enhancements.

Benefits:

• Makes the NSPanel firmware more accessible to a broader audience, including those with little to no technical background.
• Enhances user satisfaction by simplifying the initial setup and ongoing maintenance.
• Reduces support requests related to technical complications of setup and updating.

Conclusion:

Transforming the NSPanel firmware into a more commercial-like, user-friendly solution will not only enhance the user experience for entry-level users but also expand our community by making our solution more accessible to a wider audience.

[rudgej]

I suspect the main way to make this " mass market " is for sell pre flashed ns panels so from that point on all updates are over WiFi.

That said as a user with 15 panels I now see that I need to physically re flash them on the next release which in itself will be quite a challenge and time consuming.

The challenge of pre flashed panels of course is that they'll this takes someone time , they have to hold stock and pay for postage which likely means a price point of £100 per panel. You then of course need to find someone to do it ! An arme of teenagers maybe ?

The project is excellent. I would just add one thing that some form of central management console would make managing large implementations simpler.

[edwardtfn]

Well, I don't wanna go that far to sell pre-flashed panels, but maybe a pre-compiled firmware with an update button to download the latest version instead of compiling locally could be feasible.
Still users will have to flash it for the first time, but that will be the only one, hopefully.

[edwardtfn]

That said as a user with 15 panels I now see that I need to physically re flash them on the next release which in itself will be quite a challenge and time consuming.

You can still with arduino and migrate on your pace, but I'm sure you will miss the advantages of using esp-idf. In my opinion, the additional memory available (which I use for bluetooth_proxy) is by itself quite convincing, but the support to https and the improvements this brings when uploading TFT is unbeatable.
I don't have to copy tft files to my local server anymore. Now with the new file selection and downloading directly from GitHub the historically source of pain of this project became a quite simple and reliable task:

[edwardtfn]

By the way, I'm saying I don't wanna go to the pre-flash business, but I will fully support is someone wanna take that way. 😉

[kondakos]

Excellent idea. The MQTT can be the protocol for controlling the device. The firmware update can be independent of the home assistant.

[rudgej]

So if I move to the new frnawork I then see a drop down like the above and can use this to update the panel? Looks good.

I have also revised some questions on another discussion thread to clarify the update process.

If there is now a dropdown then I assume the ESP yams changes as that currently points to the file locally?

[edwardtfn]

So if I move to the new frnawork I then see a drop down like the above and can use this to update the panel? Looks good.

That's right. This is already available.

If there is now a dropdown then I assume the ESP yams changes as that currently points to the file locally?

The yaml configuration is still possible, but it is optional. If you wanna use that, you select the first option on the dropdown: "Use nextion_update_url".

[illuzn]

My suggested improvements:

1. Pre-compiled firmware: This is probably the only thing you can do to ease the challenges. This probably needs to come with the stated caveat that it is only for esp-idf framework. That said I'm not sure why you would want to retain an arduino framework (the only reason why you would do this before was because esp-idf did not have the requisite support for ARM architectures i.e. raspberry pi). Not sure how much it simplifies the backend but you might even deprecate the arduino framework since I suspect it brings more trouble than its worth - also I'm not sure how many arduino people are pulling their devices off the wall to update it every time (whereas OTA is pretty painless). Doing this also removes challenge 4, in fact I had issues with flashing the TFT locally hosted (probably because of NGINX proxy or somesuch) but because I had ESP-IDF flashing TFT from github was very easy.

2. Better documentation: A dedicated page for documentation is probably the biggest thing that is possible to alleviate the difficulty of using this excellent tool. I came from NSPanel-lovelace-UI (which was fantastic because the documentation is extensive and shows precisely what you are changing) because the new weather.forecasts broke it, it had significant responsiveness delays due to the layers of abstraction and I couldn't get the new add-on implementation to work (lack of documentation - understandable since it's still in alpha). It takes a lot to understand something as basic as "what does each swipe action do?", "what entity domains are supported where?", "I've configured the climate entity, now how do I access it?" or "what is a chip, button page, entity page, etc.?" I've used mkdocs with success myself (my example here) and it automatically builds based upon your repository (the repository which builds that example is here).

As for 2, I'm happy to spearhead that effort. It does require additional configuration of workflows and repository pages settings (to deploy from the gh-pages branch) and also some consideration of: what theme to use (matching branding), what extensions are required (probably pymdownx and some others).

Edit: I should probably flesh out why I make point 1. In order to initially install this, you need to: (1) take apart your NSPanel and (2) flash the device with a serial flash device (taking care not to short circuit anything etc).

This is arguably the highest barrier to entry to doing this. I posit that anyone willing to go to these lengths is well and truly tech savvy (and probably has some level of coding experience - even if not specific experience in YAML or HA scripting). Unless someone is willing to sell pre-flashed devices, that high barrier to entry is probably the determining factor.

~~Looking at the issues generated a lot of the legwork could be fixed with proper issues templates (requesting YAML and logs) and redirecting people to a troubleshooting page (which includes the new issue you've created for build errors).~~Looks like a lot of people are putting their issues into a blank template (because it's enabled) - I would disable that. I would also direct questions to the discussion pages. Issues are for bug reports and feature requests; discussions are for general questions and such.

[edwardtfn]

That said I'm not sure why you would want to retain an arduino framework (the only reason why you would do this before was because esp-idf did not have the requisite support for ARM architectures i.e. raspberry pi). Not sure how much it simplifies the backend but you might even deprecate the arduino framework since I suspect it brings more trouble than its worth - also I'm not sure how many arduino people are pulling their devices off the wall to update it every time (whereas OTA is pretty painless)

The main reason we took too long to add support to esp-idf was the fact that the Nextion component in ESPHome wasn't supporting TFT upload with that framework. I've then added the code for esp-idf in ESPHome.
Right now, my main concern about using esp-idf now is the lack of OTA on the web_server, which would make's hard to update the firmware, so right now I'm starting to play with esphome/esphome#5586 as a firmware update with download directly from GitHub would be even better, but still pending lots of tests.
By the way, it is interesting to note that they are using https over arduino also, which means it should be possible to rebuild the Upload TFT for arduino in a way it could support https also.
But anyways, I don't have plans to invest in arduino and I believe we will soon face some situation where we will have to develop funcionality for esp-idf only, to the point the number of users with arduino will reduce drastically and maybe won't make sense to support that anymore. User needing arduino can always stay with an older version until ESPHome breaks something else.

[edwardtfn]

By the way, my sandbox version of the pre-compiled is this: https://github.com/Blackymas/NSPanel_HA_Blueprint/blob/dev/advanced/esphome/nspanel_esphome_prebuilt.yaml

Still a long way to go to support add-ons, some of the customizations, etc.

[illuzn]

Also, just thought of this, is there any reason why "captive_portal" is not enabled by default in the recommended settings?

Makes life a heck of a lot easier if you need to change wifi passwords (or if something else goes wrong you can upload new firmware directly).

[edwardtfn]

In reality when we started having users complaining about compiling issues (which becomes more critical on each version of our project or from ESPHome) I've moved everything that wasn't essential, including captive_portal, to the advanced package, so it still there, it's just not installed by default.

[illuzn]

Does ota via captive portal work? I assume it does (not ideal but might be another way to do OTA updates while esphome sorts out its web_server OTA).

[illuzn]

Actually scratch that, would recommend ota_http which does exactly what is required and seemingly works via https as well which would allow you to direct it to a prebuilt image hosted here.

Working in the dev branch for now. Expect it to filter through to beta and production branches before the March 2024 deadlines.

[edwardtfn]

Working in the dev branch for now. Expect it to filter through to beta and production branches before the March 2024 deadlines.

I don't think that will be possible by March. 😆

[illuzn]

Here's what the docs look like with mkdocs and my theme: https://illuzn.github.io/NSPanel_HA_Blueprint

Code highlighting and such may be off because I mostly copied and pasted without looking into it too much.

[illuzn]

@edwardtfn Did you have a look at this? Thoughts, as I said happy to spearhead the effort to get this up.

[edwardtfn]

I loved that and will love to use it.

[illuzn]

Just noticed that you aren't the code owner here and the code owner has been AWOL for over 6 months. Has there been some consideration to moving this project to your local fork so that you have full control over the repo?

[edwardtfn]

Sorry for the late reply here.
I think we can ask @Blackymas for whatever is needed from his side.

I believe after setting up the website access we can do everything with repo files, right?

[rudgej]

Right now, my main concern about using esp-idf now is the lack of OTA on the web_server, which would make's hard to update the firmware, so right now I'm starting to play with

So the esp-idf does not support ota updates. That is a big big thing as that means I have to rip 15 panels off the wall on each update .... is this fixable ?

[edwardtfn]

It does supports OTA, but it's version of web_server doesn't shows the OTA button.
You can use it normally when flashing using ESPHome. OTA works just fine.

But for a version where user's shouldn't be required to compile, we shouldn't require them to have ESPHome installed, so they need a way to get a new firmware OTA, and this is what I'm looking for. In arduino, that could be web_server, although that is far from user friendly, but on esp-idf that is not even available.

[illuzn]

Ota is enabled (from the esphome core yaml)

ota:
  id: ota_std
  password: ${ota_password}
  safe_mode: true
  reboot_timeout: 3min
  num_attempts: 3

Currently you have to update by building your own firmware and updating like so (just updated mine now) :

[twasilczyk]

Honestly, this project is already way easier to set up than other NSPanel firmware options I tried. The first one required me to dive super deep into configuration of MQTT broker, HASS, AppDaemon, outdated instructions and incompatible dependency versions with breaking changes.

You folks did a great job. Thanks!

[arikastarvo]

If the aim is to stay tightly related to HA, then rest of my comment is quite useless. But if there is a even slight chance that you will be aiming for a larger audience for as the backend - then migrating to MQTT (or smth similar) as a connection medium would be essential. There is a comment that hints it would not be a big change - #1273 (comment) .

Full disclosure - I'd really like that feature because I would like to be able to cut off HA (or any other setup that needs half a rack of servers) if I needed to. And by doing that I'd like my awesome looking wall panels to stay working (with as little effort as possible). Actually there is a longer story behind that, but thats.. well a longer story.

[edwardtfn]

MQTT is definitely feasible, however a big part of the actions are currently executed from the Blueprint side, including constructing pages, etc., so it would either require HA anyways or require lots of development or reduce flexibility quite a lot.
But I will keep it in the radar...

[twasilczyk]

I've seen another project take an exact opposite turn. If you decide to pull the trigger on MQTT, please make sure:

• performance stays the same as in HA API variant
• dependencies number won't skyrocket (that mentioned project requires manual installation and configuration of a handful addons, including MQTT broker)

[arikastarvo]

If it gives any more ideas then my end game is to do must-have day-to-day automations within ESP (mostly simple room-based things like climate schedule, light based on movement etc). Basically simple rules posting MQTT messages. In ideal world - this would be achieved using ble, esp-now (or smth similar) to skip the persistent wifi connection.

HA (or any other similar solution) could be used to tap in to the system - aggregate and store the data, see the big picture or add some nifty smartness based on history and other rooms. But if the central controller goes sideways .. for whatever reason - then the basics still would work. Hopefully with a great looking UI that blueprint solution has created.

[arikastarvo]

The pull that @twasilczyk mentioned is one of the reasons that I came here. I do see the reasoning behind the (potential) decision but I did want to describe my point of view. I know it doesn't fit well with the scene who always wants to tinker more using web UI and crash their automations every other Tuesday. But in a way it does hook with idea of making the solution more commercial like. By that I mean central control system vendor-neutrality, skipping flaky updates for control systems etc.

[edwardtfn]

I have no plans to abandon the native API when communicating with Home Assistant.
If I find some time to work with an mqtt implementation in the future, and esp memory allows, that will be as an alternative protocol, but the API will remain.0 as the primary one.

[quenthal]

Regarding tft files and releases - is there overview of changes to tft per version available somewhere?

If one could check if any changes or not are made to tft, it wouldn't be always "necessary" to update it. This would be extremely helpful if using bluetooth proxy or other additional capabilities with upload tft disabled.

Maybe managing compatible versions within blueprint and/or esphome yaml?

[edwardtfn]

Usually.when releasing patches we inform for each fix which are the components changed, but there's no log per component (TFT files), although I it isn't hard to get this from GitHub list of commits:

https://github.com/Blackymas/NSPanel_HA_Blueprint/commits/main/nspanel_eu.tft

[illuzn]

Wait what? Isn't that what the tag "TFT" in the "Affected Components" of the release notes is? Obviously it doesn't mention all changes but again that would be where the commit list comes in.

[edwardtfn]

Yes, that's right, however you need to read every single release note to keep track of that. And not everything is registered there (specially when we have major releases).

[MichaelHeimann]

I'd like to share some progress here about coming features in dev as @edwardtfn suggested:
I'm selling nspanels with this firmware and thus share the desire for simple user experience.

That made me improve the already existing prebuilt code further. With the latest update in dev, we can integrate a preflashed panel nicely:

• improv support to get WiFi credentials to the panel. No app required! A browser and bluetooth is enough. Smartphone and laptop both work fine for me.
• No esphome addon needed!
• No firmware compiling needed!
• Still, the panels can be adopted into esphome addon and then the yaml can be modified as wanted, but now this is optional.

That further removes the effort around opening the device, flashing nspanel and overall makes nspanels with this firmware even easier to get running 🥇

I'll continue testing and working on improvements. One idea is update notifications with an update entity to get notified about new firmware and maybe even flash esphome, tft and update blueprint with a click on update in the GUI. We'll see.

[edwardtfn]

One idea is update notifications with an update entity to get notified about new firmware and maybe even flash esphome, tft and update blueprint with a click on update in the GUI.

Update the Bluetooth can be a bit tricky, as HA don't have a native engine for this, but an auto-update for ESPHome firmware and TFT is feasible. 😃

[edwardtfn]

esphome/esphome#5586 (ota_http) got merged to ESPHome dev, so it should be released with v2024.6.0.
As soon I release v4.3.8 (already on beta, planned to be release next Sunday) I will update to the latest ota_http and prepare a new patch. 😉

• [ota] http_request update platform esphome/esphome#5586
• https://deploy-preview-3291--esphome.netlify.app/components/ota_http_request