Skip to content

Commit

Permalink
Support registries in gzipped tarballs
Browse files Browse the repository at this point in the history
Start with Julia 1.7, a registry can also be stored in a depot as a
gzipped tarball. This commit unpacks the tarball into a temporary
directory. This was the simplest way to support this new feature.

This implementation will unpack a fresh copy of the registry on every
call to `reachable_registries`. We could instead cache the temporary
directory for future calls.

There's an open PR for the Tar.jl package that will be of future
interest:

JuliaIO/Tar.jl#95

After that is merged, we could instead stream the individual files
straight out of the tarball, rather than having to unpack it into a
temporary location.
  • Loading branch information
danielmatz committed May 26, 2022
1 parent a568954 commit 77895d7
Show file tree
Hide file tree
Showing 2 changed files with 25 additions and 8 deletions.
2 changes: 2 additions & 0 deletions Project.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,12 @@ license = "MIT"
version = "0.6.2"

[deps]
CodecZlib = "944b1d66-785c-5afd-91f1-9de20f533193"
Compat = "34da2185-b29b-5c13-b0c7-acf172513d20"
Pkg = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f"
REPL = "3fa0cd96-eef1-5676-8a61-b3b8758bbffb"
TOML = "fa267f1f-6049-4f14-aa54-33bafae1ed76"
Tar = "a4e569a6-e804-4fa4-b0f3-eef7a1d5b13e"
UUIDs = "cf7118a7-6976-5b1a-9a39-7adc72f591a4"

[compat]
Expand Down
31 changes: 23 additions & 8 deletions src/PkgDeps.jl
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ using REPL
using TOML: parsefile
using UUIDs
using Compat
import Tar
using CodecZlib: GzipDecompressorStream

export PkgEntry, RegistryInstance
export NoUUIDMatch, PackageNotInRegistry
Expand Down Expand Up @@ -53,14 +55,27 @@ function reachable_registries(

for d in depots
isdir(d) || continue
reg_dir = joinpath(d, "registries")
isdir(reg_dir) || continue

for name in readdir(reg_dir)
if isempty(registry_names) || name in registry_names
file = joinpath(reg_dir, name, "Registry.toml")
isfile(file) || continue
push!(registries, RegistryInstance(joinpath(reg_dir, name)))
registries_dir = joinpath(d, "registries")
isdir(registries_dir) || continue

for file_name in readdir(registries_dir)
if isdir(registries_dir, file_name)
registry_name = file_name
registry_path = joinpath(registries_dir, registry_name)
elseif endswith(file_name, ".tar.gz")
registry_name = chop(file_name, tail = length(".tar.gz"))

# We need to unpack the registry into a temporary directory.
registry_path = open(joinpath(registries_dir, file_name)) do io
Tar.extract(GzipDecompressorStream(io))
end
else
continue
end

if isempty(registry_names) || registry_name in registry_names
isfile(registry_path, "Registry.toml") || continue
push!(registries, RegistryInstance(registry_path))
end
end
end
Expand Down

0 comments on commit 77895d7

Please sign in to comment.