Add flat repository support for debian images

[ericlchen1]

Fixes #56 and followup to #55

Problem:

When working with "flat repository format" external repositories, the assumption made by rules_distroless is to use the known repository hierarchy within dists directory. See: https://wiki.debian.org/DebianRepository/Format#Flat_Repository_Format

In a flat repository, everything is "flat" within the top-level of the repository and as such - the {}/dists/{}/{}/binary-{}/Packages.{}".format(url, dist, comp, arch, file_type) logic will always fail to find the Package indices.

Example: https://cloud.r-project.org/bin/linux/debian/bullseye-cran40/ or https://developer.download.nvidia.com/compute/cuda/repos/debian11/x86_64/ which are both flat repositories with arch specific packages (amd64/i386)

Solution:

Manifest:
sources: add flat_repository and arch keys

• flat_repository: default false, boolean that represents if source is a flat_repository
• arch: lock repository to a specific architecture. Other architectures will not be considered for lockfile if this is set.

Add arch_specific_packages section with format

arch_specific_packages:
  architecture:
    - "package1"
    - "package2"

[jjmaestro]

@ericlchen1 I think this PR conflates / mixes two issues: support for flat repositories (which #55 addresses) plus then having single-architecture sources mixed with multi-architecture sources.

The "single-architecture sources" forces the hacks in the YAML manifest to mark one of the sources as single-architecture and the arch_specific_packages section.

IMHO this is unnecessary because you can have a separate manifest for those repos. For example, in this specific case, just split the bullseye bullseye-cran40 channel into its own manifest, with arch: amd64 as the only architecture in it and just one package: r-doc-html.

I think that's cleaner, easier to understand and should be much better overall since you won't be trying to resolve packages that (I think) won't exist in the majority of the sources given.

[jjmaestro]

As I mentioned in #67 (comment), this is not what the spec says, check https://wiki.debian.org/DebianRepository/Format#Filename:

Filename

The mandatory Filename field shall list the path of the package archive relative to the base directory of the repository. The path should be in canonical form, that is, without any components denoting the current or parent directory ("." or ".."). It also should not make use of any protocol-specific components, such as URL-encoded parameters.

I've checked the R-Project flat repos and this doesn't happen / is not needed. However, this is something that happens in the NVIDIA CUDA flat repos . Thus, I think that this it's a quirk of some specific flat repos that don't conform with the Debian repo spec.

[jjmaestro]

OK, so I finally did try to write a PR that adds support for multi-arch and single-arch flat repos and addresses the issue with the NVIDIA CUDA flat repos. Hopefully that PR superseeds the others and can be merged and 1 + 3 PRs closed 😄

@ericlchen1 please have a look and let me know what you think and if #86 works for you. Happy to address any issues or concerns if there's something that breaks in your use-case!