Another way of editing default config, docker deployment

[g4xx]

Hey again!

Currently hosting start-page via Docker/DockerHub has one fundamental issue - changes made through web UI will not persist if container is restarted/updated.

Building image locally with modified source kind of defeats the purpose of having images on DockerHub and adds a lot of overhead to building and modifying source.

Using docker volumes would be obvious answer but I am not sure how that would work with ability to edit from UI - this would require some research.

Best Regards!

[excalith]

Hey there! Actually all configs made through config edit stored in the local storage within your browser. Even if you stop or update the docker container, it will persist within your browser as long as you have the same IP and port. But I understand it can be frustrating depending on your network setup due to changes, or your browser clears all data after you quit.

However, if you are still having problems, you can use config import <url> command to import a remote json config to import your settings back. For more information regarding this, please check out How To Customize wiki page for different ways to customize.

For instance, to use my own config on my mobile devices, I use the web build and import my own configuration from my dotfiles repository with config import https://raw.githubusercontent.com/excalith/.dotfiles/main/config/startpage/excalith-start-page.json command.

I would suggest creating your own images if you forked this repository and made some source code changes besides the configuration. Otherwise, if the source code is the same, managing the configuration should persist on browsers. For this reason, I backed up my configuration on my dotfiles repository.

I am not sure if volumes are the best way for this. In order to do it, I think I need to serve it as an API to return configuration file from another container since it will be more of a server thing.

Let me know what you think!

Cheers!

[g4xx]

Setting config with via URL is one way of solving that issue but in my opinion is not ideal. Accessing start-page from multiple devices (including mobile) becomes a hassle to setup. Using private browsing or cleaning cache is an issue too. Not to mention each new device will face same issue or that you'd have to have some reliable way of remembering that link. I'd rather have global configuration that simply works across devices.

Usually when setting up similar projects you'd go and use DNS server and reverse proxy to be able to access it through some domain like https://dashboard.home. This way you can easily access it through multiple devices and have exactly same setup without all the hassle of configuration each time you're accessing it from new device/OS/session.

I must say that I really like the project and in my eyes it would be a great Homarr, Homepage or Dashy alternative. I would like to make it work for me and I guess a lot of folks on /r/homelab would want that too (highly encourage you to advert project there). All those projects however offer some easy way of changing config and not worry too much about updates or container restarts.

[excalith]

Hey @g4xx, thanks for clarifying those in detail, I have a very basic home setup so I lacked the knowledge of good improvements. However, checking out those projects gave me a great vision! Also, I now that I see, I misinterpreted the idea behind volumes before. I will definetely look into this but as you mentioned earlier, I really need some time to research before starting refactoring the source code.

I suppose the first thing I should do is to get rid of the browser local storage. It will definetely break hearts of the users who prefer using online preview version, but with a simplified docker setup phase (like compose you mentioned as well) it will be easier for everyone to get into the Docker world. This will definetely further improve the stability and usability of the app.

I will also try building for armv7 for my old rusty RPi2 to try things locally. This will take some time, but I will try to do my best!

Thanks so much again for lots of great suggestions!

[g4xx]

I suppose the first thing I should do is to get rid of the browser local storage. It will definetely break hearts of the users who prefer using online preview version

Not sure if alienating your current audience would serve the project but that depends on how many users are actually using it.

I guess you could do both. If someone wants to use URL config feature / browser cache it is fine. I don't think that it has to be one or the other but it would definitely add code complexity.

Glad I could help!

[excalith]

Hey again @g4xx!

I think I managed to seperate Docker and web-based configurations using a volume for settings to persist.

Now, while using Docker either config edit and config import <url> overrides the json file within that volume, completely omitting local browser storage, so you should be able to share the same configuration between devices. On the other hand, web-based version still utilizes local storage for storing settings. So, no hearts broken :D

I would be grateful if you could try out give me another feedback regarding this!

version: '3.8'

services:
  start-page:
    container_name: start-page
    image: excalith/start-page:docker-volume-test
    restart: unless-stopped
    ports:
      - 8080:3000
    volumes:
      - data:/app/public/data

volumes:
  data:
    name: excalith-start-page

Cheers!

[excalith]

I guess I have found a better solution alternative to the chmod approach. This creates the container with root access. Still better than 777 it seems, and while trying to replicate your issue, I managed to run it without any issue this time. Still looking for a more elegant solution by avoiding both root and chmod and will let you know.

Here is the compose file with root just in case you wonder:

version: '3.8'

services:
  start-page:
    container_name: excalith-start-page
    image: excalith/start-page:docker-volume-test
    restart: unless-stopped
    user: 0:0
    ports:
      - 6699:3000
    volumes:
      - /home/gaxx/projects/start-page/:/app/public/data

volumes:
  data:
    name: excalith-start-page

To simply put the situation:

• The default compose file uses a named mount, which docker has all the access the app needs.
• Your compose file is a bind mount which requires additional permissions to be able to rwx. Adding root access seems to solve this issue.

If there is no workaround, I believe the best solution without enforcing anyone to have root access by default is to document both named and bind composes seperately like so:

If you want to use bind mounts, you should have root access to container to be able to read and write the files.
[Compose file example with root here]

Regarding the wallpaper, I am willing to do this. I have already exposed the assets folder, but having trouble showing the assets within while using named mounts. Will get back to it after solving this.

[excalith]

Hey again @g4xx!

I managed to seperate the data folder completely from the nextjs static folder hell. Now you should be able to use data folder like your own directory. You can add wallpapers, fetch image and favicon using either config edit and changing settings.json.

I have asked docker wizards around and they all agreed about one thing:
If a user does bind a mount to docker system, it should be his responsibility to give permissions. So, giving root access to compose file or setting your user uid and gid should work fine, better than chmod 777.

As a best practice, I only create the necessary settings.json within that data folder. Leaving rest to user to add files as they require.

With this, you should be able to add and use

• Custom themes within themes folder
• Custom images within assets folder

Here is your latest compose file updated with user root access using user: 0:0.

version: '3.8'

services:
  start-page:
    container_name: excalith-start-page
    image: excalith/start-page:docker-volume-test
    restart: unless-stopped
    user: 0:0
    ports:
      - 6699:3000
    volumes:
      - /home/gaxx/projects/start-page/:/app/public/data

volumes:
  data:
    name: excalith-start-page

Please let me know what you think! If everything is in order, I will merge this huge update as v3.1.0. Hopefully, I will only need to update the documentation for all those changes :D

[g4xx]

I deleted start-page dir and re-pulled image again. The directory is not being created at all now, so no way of saving settings.json. Is there a more direct way to contact you, it seems this kind of thing would be easier to discuss and live test instead of discussing here.

[excalith]

Agreed. This turned out to be more complex than it sounds, but I believe we can find a solution.

Could you drop me a message from this link https://cancellek.com/contact We can discuss how we can do a live session from there

[rubinghv]

I just want to get back to this issue since I absolutely love the concept of this project and am very excited about customizing it. I know I could fork the project etc. but I think respecting a custom settings.json file correctly and importing that on startup when self hosting (through reverse proxy like @g4xx ) is definitely preferable.

I tried binding the docker volume through /app/public/data but this is still not loaded correctly on startup. Before when binding to app/data it would just overwrite the settings.json file on startup so at least this isn't happening anymore.

Typically the way containers work is that on first docker container startup, it would auto create all the local specific files (including wallpapers, themes which I've seen discussed in other issues), in the bound docker volume. This way users have their own settings/assets that they can easily modify seperate from the app logic (which is containerized and can be upgraded easily). This makes it very easy to upgrade the underlying software without losing the customization (and is one of the advantages of docker). The docker container can then read the contents of the bound volume and would not need to recreate any asset/themes/settings.json as they would be present on the bound docker volume (after the initial creation or after the user has created their own custom versions).

So having said all that (and I'm just explaining in case you're not familiar), I am not sure why currently only the settings.json actually ends up mounted back in the docker volume, but not assets folder that is created inside the data folder. I'm guessing it has to do with permissions (as 50% of all issues are related to permissions 😃 ).

I think I'll play around with creating my own docker file and seeing if I can address this.

[Watenk]

Hi, When i bind the docker volume using this docker compose file:

start-page:
      container_name: exalith-start-page
      restart: unless-stopped
      user: 1000:1000
      ports:
          - 3001:3000
      image: excalith/start-page
      volumes:
          - /MyOwnFolder:/app/data

it only generates the settings.json in MyOwnFolder and not the themes and assets folder. Is this intended?

[excalith]

Hey there! Welcome to the discussions mate

I believe a better explanation would be nice for it in the Wiki pages. It is indeed intended, following best practices to keep things tidy and respect your space. The goal is to minimize any interference with your own mounted directory, putting only place the essential configuration file needed for the app to run.

However, you can download default assets using a script from your terminal. Here is an example script that downloads your choice of assets through a bash script.

For more information, please check out the new Q&A Post regarding this topic.

[Watenk]

Hi, thx for the reply. I added the assets and themes folder so everything works now.

[excalith]

Just in case you might have missed, I provided some bash and powershell scripts to do this automatically. You can check out the scripts and their usage examples in that Q&A Discussion. Also added this information throughout the wiki pages. Thanks for the feedback!

[Watenk]

Nice. One other small thing i ran into was that the settings.json only generates after you load the webpage once. So i initially tough the bind didn't work because no files generated in the host dir.

[excalith]

Hmm, might improve with another clarification regarding that. When the app launches, it tries to read from that directory instead of copying anything. This eases the pain if you use these configurations symlinked directly from your dotfiles repository. If nothing exists, it will create a default configuration file instead. But thanks for bringing that up!

[excalith]

@g4xx @rubinghv @Watenk I got a PR that makes it available for offline use: #89
Can you please try it? Hopefully it will provide that support. I will also try it whenever I find time :/