Simplify Thoth part of the toolchain

[fridex]

Is your feature request related to a problem? Please describe.

As suggested by @frenzymadness, our tooling could be simplified. Instead of creating a patch, we could maintain our own assemble script and propagate it to containers.

Describe the solution you'd like

• substitute the assemble script and replace it with Thoth specific assemble script that is copied to the container (instead of having a patch which might be harder to maintain across changes in S2I tooling)
• verify that Thoth s2i containers work properly with the new Thoth specific assemble script
• check what are the differences across Thoth based S2I container images and what should be maintained on our end to have a proper compatibility layer available

[codificat]

/triage accepted
/priority important-longterm

worth checking the impact on ODH

[sesheta]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[fridex]

/remove-lifecycle stale

This might be easy to achieve and can reduce maintenance cost.

[goern]

/help

[sesheta]

@goern:
This request has been marked as needing help from a contributor.

Please ensure the request meets the requirements listed here.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-help command.

In response to this:

/help

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[codificat]

/sig devsecops

[VannTen]

After working on thoth-station/adviser#2411 (comment) I'm starting to wonder whether we should implicate Thoth directly as part of container build, at all ; and rather using it as a "lockfile updater" rather than a installer.

Some reasons:

• Python lockfiles are all about environment reproducibility. If we modify that during the build based on advise from Thoth, it's not necessarily reproducible.
• increased maintenance on our side (this repo)
• We can always compose thoth-advise then s2i-build if we truly want, with tekton pipelines for example.

I might be missing some context though on why integrating Thoth advise inside the s2i process is more valuable. I see Thoth more in the role of a pipenv update than a python -m build, to resume this.

@goern @codificat @harshad16 @KPostOffice @Gkrumbach07 wdyt ?

[harshad16]

The initial thoughts were to provide most of the possible integration points (given that a user trusts the system), and the following was derived:

• For developers: who want to interact with thoth advise while developing tools
  • kebechet
  • API
  • cli
  • jupyter extension
  • etc
• For DevOps: want to have a trusted image to deploy the tools in the cloud
  • s2i image.

with the s2i image, the idea was that trusting thoth advice would fix the package issue during the build time. Though it wasn't used the most.
also, it was to cover some cases, like GPU enable images
as the developer build might not have required them. However, they are required for cluster images, so thoth advice could be helpful in those scenarios, where the environment dictates the package installation.

[VannTen]

with the s2i image, the idea was that trusting thoth advice would fix the package issue during the build time. Though it wasn't used the most. also, it was to cover some cases, like GPU enable images as the developer build might not have required them. However, they are required for cluster images, so thoth advice could be helpful in those scenarios, where the environment dictates the package installation.

In cases of GPU (or any accelerator / custom resources) in a cluster, the image would not be built on the same node that it's used on, would it ? I think the match would more occur at the Scheduling stage (taking a k8s cluster as model) using limits on custom resources (checkout https://kubernetes.io/docs/tasks/manage-gpus/scheduling-gpus/). Do some python AI/ML python packages **require** a GPU/accelerator during build time ?

[harshad16]

Suggestion:

• Move all component using s2i-thoth directly use s2i-python .
• No longer maintain s2i-thoth and add disclaimer.
• Analyse the impact on downstream images.