[FEAT]: Improve/Create setup guide

[heubeck]

Request Description

New adopters have a hard time, we should have a getting started guide with minimal viable configuration example.

Additional Information

No response

[heubeck]

Since I haven't been able to run technolinator successfully yet

@emil-wire are you willing to share your obstacles with us, and the information you missed for getting started?

Don't hesitate to ask anything, we really would love to see adoption grow.

[beiertu-mms]

Since I haven't been able to run technolinator successfully yet

@emil-wire are you willing to share your obstacles with us, and the information you missed for getting started?

Don't hesitate to ask anything, we really would love to see adoption grow.

maybe we can also create a draft pull request with what we know and @emil-wire can provide feedback via comments?

[heubeck]

@donaldedwin long time no see ⏳
did you made it? 🎸
did you surrender? 🗑️
have you found/built a (even) better solution? 💭

[emil-wire]

You guys are awesome <3

In no particular order:

• on https://github.com/MediaMarktSaturn/technolinator/blob/main/docs/Runtime_Config.md the relative links to the docker file don't work, a ../ in front of the path should do it :)
• same thing for the link to the grafana dashboard
• I'm not a big fan of kubernetes and whipped up a docker-compose file that describes the entire stack: traefik, dependency track, postgres and technolinator. Makes it easier to try it out. I'll clean it up and will make a PR
• It would be cool if one could provide the private key as a path/file instead of its contents - but it's workable
• Only a me problem because I'm a klutz: I spent way to much time with smee.io and the issues around it until I realized that I don't need it.
• The getting started guide should include the steps outlined on the quarkus site on how to configure a github app
• Java error messages... are hard to read, parse and understand :)
• Github app requirements should be more prominent
• I'm still not sure, what roles/permissions I need in my github organization to request an app install from my admin
• on https://github.com/MediaMarktSaturn/technolinator/blob/main/docs/Runtime_Config.md one example per var, to see how the option should be formatted (api.example.com or http://api.example.com?) unless there is already an example in the default column

[heubeck]

thx @emil-wire

I'm not a big fan of kubernetes and whipped up a docker-compose file that describes the entire stack: traefik, dependency track, postgres and technolinator. Makes it easier to try it out. I'll clean it up and will make a PR

💟

It would be cool if one could provide the private key as a path/file instead of its contents - but it's workable

need to check with the quarkus-gh-app ext, what you can do, is putting the properties in an .env file - don't know if that helps

Only a me problem because I'm a klutz: I spent way to much time with smee.io and the issues around it until I realized that I don't need it.

can you elaborate on that, especially the that I don't need it - how do you try it out?

Java error messages... are hard to read, parse and understand :)

because of the json format, or in general? you can set env QUARKUS_LOG_CONSOLE_JSON=false to get text only logged.

Github app requirements should be more prominent

did you figure it out, or are still struggling? (but yes, you're right)

I'm still not sure, what roles/permissions I need in my github organization to request an app install from my admin

the app can only be installed by an org admin as far as I know, but having the app in place, you can become app admin, being able to manage the apps config.

on https://github.com/MediaMarktSaturn/technolinator/blob/main/docs/Runtime_Config.md one example per var, to see how the option should be formatted (api.example.com or http://api.example.com/?) unless there is already an example in the default column

got it. any properties still problematic to you?

[emil-wire]

thx @emil-wire

of course <3

need to check with the quarkus-gh-app ext, what you can do, is putting the properties in an .env file - don't know if that helps

that's what I ended up doing, has the nice side benefit of making the docker compose file cleaner

can you elaborate on that, especially the that I don't need it - how do you try it out?

sure. I thought that webhook forwarding was a necessary functionality in general to run technolinator. however, this only applies to local setups or setups which have no inbound connections from the outside world. I tried it out on a dedicated server with a reverse proxy for TLS termination fronting technolinator and it took me way longer to realize than I'd like to admit that smee isn't necessary in such a setup 😅 🙈
So this is only a me problem, not a you problem :)

because of the json format, or in general? you can set env QUARKUS_LOG_CONSOLE_JSON=false to get text only logged.

mainly because I haven't set up centralized logging yet and parsing log messages via ssh has downsides ;) I appreciate the json though, it allows pretty printing which I ended up doing to make my life easier.
Could you make sure your docs outline, that there are more ways to configure quarkus and maybe link to the pertinent docs?

did you figure it out, or are still struggling? (but yes, you're right)

Figured it out, found the necessary permissions at the bottom of the doc. What I haven't found yet, is which scopes the GITHUB_TOKEN should have. could you give some guidance @heubeck ?

the app can only be installed by an org admin as far as I know, but having the app in place, you can become app admin, being able to manage the apps config.

could be. The struggle I had was specific to our setup, pretty sure about that. Because of weird reasons, Wire has several github orgs and while I was able to request the app installation in two of them, it didn't work in the third. But that's not an issue with technolinator, just a general problem I had.

got it. any properties still problematic to you?

Nope figured it out and technolinator seems to work well now :)

Thank you guys, this is a really valuable tool and it's made my life a lot easier!

[heubeck]

• elaborate on the different configuration options (.env, env vars, check quarkus docs for further options)
• elaborate on the general setup, locally (using smee) as well as regular using internet exposed endpoint
• elaborate on log configuration options (check/reference quarkus docs therefore)
• detail out the necessary and optional github (app/token) settings
• add github app installation guide/hints

[heubeck]

@emil-wire

What I haven't found yet, is which scopes the GITHUB_TOKEN should have.

A GITHUB_TOKEN indeed isn't necessary for Technolinator to work, because all it's mandatory GH communication is done using the authentication provided by the GH app installation.

But It's not possible to re-use the GH app auth for non-app related things, like for go get on private repos or cdxgen fetching licence information from GH.

So the GITHUB_TOKEN is used by the tools, Technolinator uses for accessing repos and higher GH api quotas on public repos.

The required scope is just "read private repos" (so "repo" scope on classic tokens).

Could you make sure your docs outline, that there are more ways to configure quarkus and maybe link to the pertinent docs?

Definite! For now, if you like to use log files, please check https://quarkus.io/guides/logging#quarkus-log-logging-log-config_quarkus.log.file-handlers-file-handlers - indeed haven't done that yet, because my Quarkus apps so far all run on container environments with log collectors from stdout 😞

while I was able to request the app installation in two of them, it didn't work in the third

as soon as you figured it out, would be interested in the issue and solution 🙏

and technolinator seems to work well now

awesome.

Thank you guys, this is a really valuable tool and it's made my life a lot easier!

really happy to hear, we hope to improve it further, so it fulfills a general demand and is not tailored only for our own need.

[emil-wire]

A GITHUB_TOKEN indeed isn't necessary for Technolinator to work, because all it's mandatory GH communication is done using the authentication provided by the GH app installation.
[...]
So the GITHUB_TOKEN is used by the tools, Technolinator uses for accessing repos and higher GH api quotas on public repos.

Yeah, the quotas haven been the main reason why I wanted technolinator to have a token :)

The required scope is just "read private repos" (so "repo" scope on classic tokens).

Awesome, thanks!

Definite! For now, if you like to use log files, please check https://quarkus.io/guides/logging#quarkus-log-logging-log-config_quarkus.log.file-handlers-file-handlers - indeed haven't done that yet, because my Quarkus apps so far all run on container environments with log collectors from stdout 😞

Makes total sense! Could you recommend a lightweight log collector and browser? I've used elasticsearch + kibana in the past, but that seems overkill for a small scale deployment of 10 applications...

as soon as you figured it out, would be interested in the issue and solution 🙏

I'll be sure to provide feedback!

really happy to hear, we hope to improve it further, so it fulfills a general demand and is not tailored only for our own need.

You're doing the lords work!
@heubeck are you following DependencyTrack/dependency-track#3251 ?

[heubeck]

are you following DependencyTrack/dependency-track#3251 ?

now I do, thx @emil-wire ;)

Makes total sense! Could you recommend a lightweight log collector and browser? I've used elasticsearch + kibana in the past, but that seems overkill for a small scale deployment of 10 applications...

non setup by my own so far, but would give Grafana Loki a try, if I would need to - no idea it fits your setup, but the tool itself is quite nice.

[emil-wire]

@heubeck @beiertu-mms I've created #377 which addresses some of the things mentioned here.