Introduce Quickstart deployment mode

[Cccccrystall]

Purpose

This Pull Request introduces a new docker-compose.quickstart.yml file and related configurations, aiming to streamline and enhance the user experience for quick project setups. Here is a comprehensive summary of the changes and additions:
 
New Docker Compose File
 
Introduced docker-compose.quickstart.yml, derived from the existing docker-compose.full.yml.
Added additional configuration for the script-runner service, detailing build context, volume mapping (including the mapping of zip_folder), network dependencies, and service dependencies.
 
Dockerfile for Script-Runner
 
Created a Dockerfile specifying the base image as ubuntu:latest.
Installed necessary utilities: curl and jq.
Added and made executable two Bash scripts: upload.sh and wait-for-it.sh.
 
Upload Script
 
The script checks environment variables for user credentials, logs in to obtain an access token, and processes and uploads ZIP configuration files located in /zip_folder to the backend service.
 
ZIP Configuration Files and Folder Mapping
 
Included a new folder zip_folder containing ZIP configuration files to be used by the upload.sh script.
The zip_folder is mapped to the script-runner container, allowing direct access and processing of the ZIP files within the container.
The introduction of zip_folder mapping facilitates easy additions of default ZIP configurations. Whenever there is a need to introduce new default configurations, we can simply add the corresponding ZIP files to the zip_folder, making the process more efficient and user-friendly.
 
Wait-for-it Script
 
Incorporated the wait-for-it.sh script, enhancing the container’s capability to wait for the availability of the dependent backend service before executing upload.sh.
Usage Instructions:
 
To utilize the new configurations, the user should build the script-runner service with the command: docker-compose -f docker-compose.quickstart.yml build script-runner.
Subsequently, to trigger the setup, the user should run: docker-compose -f docker-compose.quickstart.yml up -d.
 
Future Plan
As part of our ongoing efforts to enhance user experience, we plan to package the script-runner service as a separate Docker image and upload it to Docker Hub. This step will allow users to skip the build step and trigger the service directly, thereby further simplifying the setup process.
 
These enhancements are designed to facilitate easier and more efficient project setups, especially beneficial for users seeking a quickstart option. By introducing the script-runner service with automated script executions and additional configurations, we are enabling seamless interactions and operations for different project use cases.

Quick Demo

Remarks

PR introduces (a) breaking change(s): no

PR introduces (a) deprecation(s): no

[Cccccrystall]

Hi @bossenti @RobertIndie @tenthe @dominikriemer, I kindly request your assistance in reviewing this Pull Request. Feel free to add your comments or suggestions; your input will be highly appreciated. Thank you in advance.

[dominikriemer]

@Cccccrystall I think it's better to put the Dockerfile, zip files and scripts into a separate folder so that users don't confuse the Dockerfile with the project's other Dockerfiles.

Also, I notice there are three zip files..these seem to be more placeholders with very basic examples - do you plan to provide a full example that helps users doing their first steps?

[dominikriemer]

The checks are currently faliing due to missing license headers.

[Cccccrystall]

@Cccccrystall I think it's better to put the Dockerfile, zip files and scripts into a separate folder so that users don't confuse the Dockerfile with the project's other Dockerfiles.

Also, I notice there are three zip files..these seem to be more placeholders with very basic examples - do you plan to provide a full example that helps users doing their first steps?

Hello, @dominikriemer! I greatly appreciate your valuable suggestions! They indeed have been quite beneficial. In an effort to enhance the user experience, I'm keen on understanding which type of asset would be most suitable for users trying Streampipes for the first time. However, in my own search efforts across platforms like Youtube, I seem to only encounter basic examples for tutorials. I'm eager to provide a more comprehensive example for our users, given your expertise, @dominikriemer, would you be able to provide some guidance on finding the most suitable asset? Your insights would be greatly beneficial in helping to enhance the initial user experience with Streampipes Quickstart mode.

[Cccccrystall]

Hi @dominikriemer, @bossenti! Two weeks ago I made some changes, including providing new example assets, merging quickstart scripts and resources into a separate folder, and changing quickstart.yml based on compose.yml. Could you review these changes for me and give me some advice? I would appreciate it!

[bossenti]

I've had a look at your provided StreamPipes assets in zip folder as well. Can you please give both the StreamPipes assets definition and their contained assets themselves more speaking names than Default-*. I think this can cause a lot of confusion for first-time users.
In addition, can you please clean up the assets? In one asset group, there is for example an empty dashboard.

[bossenti]

@Cccccrystall thanks a lot for addressing the review comments.
Your assets really look good and I think they are greatly suited for the quickstart mode.
However, I have two small remarks with respect to the assets.

• Can we give the file MOCK_DATA.json an appropriate name?
• Can we remove the 02 from Flowrate02 e.g. in Flowrate02 sensor mass flow condition

Sorry for being so nit-picking here but I think for the purpose of this improvement such details are important

[bossenti]

Thanks for your updates!

Just recognized that there are some charts in the data explorer where the caption is wrong. E.g. in the image below this is a histogram and not a time series

Can you please go through your data views and adjust them accordingly?

[Cccccrystall]

Thanks for your updates!

Just recognized that there are some charts in the data explorer where the caption is wrong. E.g. in the image below this is a histogram and not a tim

Hi, @bossenti ! Thanks for the suggestion! I have addressed this problem in the latest commit.

[bossenti]

Great, everything looks fine now 🙂

[Cccccrystall]

Hi, @bossenti ! In the run-cypress-e2e-tests, I find that UI test for Test live data download dialog didn't pass, The error shows: CypressError: cy.click()can only be called on a single element. Your subject contained 2 elements. Pass{ multiple: true } if you want to serially click each element. But actually I didn't change the UI file, so I am confused about this error message. Could you give me some suggestions about this?

[bossenti]

@Cccccrystall great that you recognized it!
This is not related by any change you have made.
We have some issues with flaky E2E tests in the last time, so no need to worry 🙂

[Cccccrystall]

Hi, @bossenti! The recent commit appears to have successfully passed the validation checks, and I am keen to know if there are any parts of improvement you might suggest. Should there be any, I am more than willing to make any necessary revisions promptly to expedite the approval process.
Furthermore, I want to extend my heartfelt gratitude for taking the time to review my commits and providing suggestions, despite your busy schedule. Your feedback helps in enhancing both my code and documentation. A sincere thank you for your help! ✨

[bossenti]

Thanks @Cccccrystall
I will take care of merging this PR this weekend