Skip to content

Commit

Permalink
Automatically include all dependencies when precompiling
Browse files Browse the repository at this point in the history
In precompile.jl, precompile-no-run.jl and
precompile-makie-post-processing.jl, we can just use the default of
create_sysimage(), which is to include all packages associated with the
project.

For precompile_dependencies.jl where we explicitly want to exclude
moment_kinetics from the system image, we do need to pass a list of
packages, but we can read them from the Project.toml instead of
maintaing a separate, hard-coded list in the script.
  • Loading branch information
johnomotani committed Sep 10, 2023
1 parent 6a2aade commit 592322d
Show file tree
Hide file tree
Showing 4 changed files with 7 additions and 20 deletions.
5 changes: 1 addition & 4 deletions precompile-makie-post-processing.jl
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,11 @@ Pkg.activate(".")

using PackageCompiler

packages = [:moment_kinetics, :PackageCompiler, :ArgParse, :CairoMakie, :Combinatorics, :DelimitedFiles, :FFTW, :Glob, :HDF5, :IJulia, :LinearAlgebra, :LsqFit, :MPI, :NaturalSort, :NCDatasets, :OrderedCollections, :Primes, :Roots, :SHA, :SpecialFunctions, :Statistics, :TOML, :TimerOutputs]

# Create the sysimage 'makie_postproc.so' in the base moment_kinetics source directory
# with both moment_kinetics and the dependencies listed above precompiled.
# Warning: editing the code will not affect what runs when using this .so, you
# need to re-precompile if you change anything.
create_sysimage(packages;
sysimage_path="makie_postproc.so",
create_sysimage(; sysimage_path="makie_postproc.so",
precompile_execution_file="util/precompile_makie_plots.jl",
include_transitive_dependencies=false, # This is needed to make MPI work, see https://github.com/JuliaParallel/MPI.jl/issues/518
sysimage_build_args=`-O3`, # Assume if we are precompiling we want an optimized, production build. No `--check-bounds=no` because Makie doesn' like it https://github.com/MakieOrg/Makie.jl/issues/3132
Expand Down
10 changes: 1 addition & 9 deletions precompile-no-run.jl
Original file line number Diff line number Diff line change
Expand Up @@ -5,19 +5,11 @@ Pkg.activate(".")

using PackageCompiler

using TOML
project_file = TOML.parsefile("Project.toml")
deps = (Symbol(d) for d keys(project_file["deps"]))

packages = [:moment_kinetics, :PackageCompiler, deps...]
println("precompling $packages")

# Create the sysimage 'moment_kinetics.so' in the base moment_kinetics source directory
# with both moment_kinetics and the dependencies listed above precompiled.
# Warning: editing the code will not affect what runs when using this .so, you
# need to re-precompile if you change anything.
create_sysimage(packages;
sysimage_path="moment_kinetics.so",
create_sysimage(; sysimage_path="moment_kinetics.so",
include_transitive_dependencies=false, # This is needed to make MPI work, see https://github.com/JuliaParallel/MPI.jl/issues/518
sysimage_build_args=`-O3 --check-bounds=no`, # Assume if we are precompiling we want an optimized, production build
)
5 changes: 1 addition & 4 deletions precompile.jl
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,11 @@ Pkg.activate(".")

using PackageCompiler

packages = [:moment_kinetics, :PackageCompiler, :ArgParse, :Combinatorics, :DelimitedFiles, :FFTW, :Glob, :IJulia, :LinearAlgebra, :LsqFit, :MPI, :NaturalSort, :NCDatasets, :OrderedCollections, :Plots, :Primes, :Roots, :SHA, :SpecialFunctions, :Statistics, :TOML, :TimerOutputs]

# Create the sysimage 'moment_kinetics.so' in the base moment_kinetics source directory
# with both moment_kinetics and the dependencies listed above precompiled.
# Warning: editing the code will not affect what runs when using this .so, you
# need to re-precompile if you change anything.
create_sysimage(packages;
sysimage_path="moment_kinetics.so",
create_sysimage(; sysimage_path="moment_kinetics.so",
precompile_execution_file="util/precompile_run.jl",
include_transitive_dependencies=false, # This is needed to make MPI work, see https://github.com/JuliaParallel/MPI.jl/issues/518
sysimage_build_args=`-O3 --check-bounds=no`, # Assume if we are precompiling we want an optimized, production build
Expand Down
7 changes: 4 additions & 3 deletions precompile_dependencies.jl
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,13 @@ Pkg.activate(".")

using PackageCompiler

packages = [:PackageCompiler, :ArgParse, :Combinatorics, :DelimitedFiles, :FFTW, :Glob, :IJulia, :LinearAlgebra, :LsqFit, :MPI, :NaturalSort, :NCDatasets, :OrderedCollections, :Plots, :Primes, :Roots, :SHA, :SpecialFunctions, :Statistics, :TOML, :TimerOutputs]
using TOML
project_file = TOML.parsefile("Project.toml")
packages = collect(Symbol(d) for d keys(project_file["deps"]))

# create the sysimage 'dependencies.so' in the base moment_kinetics source directory
# with the above pre-compiled packages
create_sysimage(packages;
sysimage_path="dependencies.so",
create_sysimage(packages; sysimage_path="dependencies.so",
precompile_execution_file="util/precompile_run_short.jl",
include_transitive_dependencies=false, # This is needed to make MPI work, see https://github.com/JuliaParallel/MPI.jl/issues/518
sysimage_build_args=`-O3 --check-bounds=no`, # Assume if we are precompiling we want an optimized, production build
Expand Down

0 comments on commit 592322d

Please sign in to comment.