diff --git a/README.md b/README.md index 86951a29c..8d887bf59 100644 --- a/README.md +++ b/README.md @@ -2,25 +2,28 @@ The full documentation is online at [https://mabarnes.github.io/moment_kinetics](https://mabarnes.github.io/moment_kinetics). -## Basics -0) Ensure that the Julia version is >= 1.7.0 by doing +## Setup + +If you are working on a supported machine, use the `machines/machine_setup.sh` +script, see [Setup for `moment_kinetics` on known clusters](@ref). Otherwise: + +1) Ensure that the Julia version is >= 1.7.0 by doing ``` $ julia --version ``` at command line. -1) If you are working on a supported machine, use the - `machines/machine_setup.sh` script, see [Setup for `moment_kinetics` on - known clusters](@ref). + 2) Dependencies need to be installed to the project environment. Start Julia with ``` $ julia --project ``` - (which activates the 'project' in the current directory) (or after starting with `julia`, in the REPL type `]` to enter `pkg>` mode, enter `activate .` and then backspace to leave `pkg>` mode). Once in the `moment_kinetics` project, enter `pkg>` mode by typing `]` and then run the command + (which activates the 'project' in the current directory, or after starting with `julia`, in the REPL type `]` to enter `pkg>` mode, enter `activate .` and then backspace to leave `pkg>` mode). Once in the `moment_kinetics` project, enter `pkg>` mode by typing `]` and then run the command ``` (moment_kinetics) pkg> instantiate ``` this should download and install all the dependencies. + 3) For julia>=1.6, pre-compiling dependencies manually is not necessary any more due to improvements to the native pre-compilation, so this step can be skipped (although precompiling the whole `moment_kinetics` code may still be useful sometimes). To pre-compile a static image (`dependencies.so`) that includes most of the external packages required for running and post-processing, run ``` $ julia -O3 precompile_dependencies.jl @@ -31,83 +34,51 @@ The full documentation is online at [https://mabarnes.github.io/moment_kinetics] $ julia -O3 precompile.jl ``` this significantly decreases the load time but prevents code changes from taking effect when `moment_kinetics.so` is used without repeating the precompilation (to use this option, add an option `-Jmoment_kinetics.so` when starting julia). -4) To run julia with optimization, type - ``` - $ julia -O3 --project run_moment_kinetics.jl input.toml - ``` - Options are specified in a TOML file, e.g. `input.toml` here. The defaults are specified in `moment_kinetics_input.jl`. - * To run in parallel, just put `mpirun -np ` in front of the call you would normally use, with `` the number of processes to use. - * It may be more convenient when running `moment_kinetics` more than once to work from the Julia REPL, e.g. - ``` - $ julia -O3 --project - julia> using moment_kinetics - julia> run_moment_kinetics(input) - ``` - where `input` is a `Dict()` containing any non-default options desired. Input can also be loaded from a TOML file passing the filaname as a String to the second argument, e.g. - ``` - julia> run_moment_kinetics("input.toml") - ``` -5) To restart a simulation using `input.toml` from the last time point in the existing run directory, + +4) In the course of development, it is sometimes helpful to upgrade the Julia version. Upgrading the version of Julia or upgrading packages may require a fresh installation of `moment_kinetics`. To make a fresh install with the latest package versions it is necessary to remove (or rename) the `Manifest.jl` file in the main directory, and generate a new `Manifest.jl` with step 1) above. It can sometimes be necessary to remove or rename the `.julia/` folder in your root directory for this step to be successful. + +5) One may have to set an environment variable to avoid error messages from the Qt library. If you execute the command ``` - $ julia -O3 --project run_moment_kinetics --restart input.toml + $ julia --project run_post_processing.jl runs/your_run_dir/ ``` - or to restart from a specific output file - either from the same run or (if the settings are compatible) a different one - here `runs/example/example.dfns.h5` + and see the error message ``` - $ julia -O3 --project run_moment_kinetics input.toml runs/example/example.dfns.h5 + qt.qpa.xcb: could not connect to display + qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "" even though it was found. + This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem. ``` - The output file must include distribution functions. When not using parallel I/O there will be multiple output files from different MPI ranks - any one of these can be passed. - * To do the same from the Julia REPL - ``` - $ julia -O3 --project - julia> run_moment_kinetics("input.toml", restart=true) - ``` - or - ``` - julia> run_moment_kinetics("input.toml", restart="runs/example/example.dfns.h5") - ``` - * When calling the `run_moment_kinetics()` function you can also choose a particular time index to restart from, e.g. - ``` - julia> run_moment_kinetics("input.toml", restart="runs/example/example.dfns.h5", restart_time_index=42) - ``` -6) To make plots and calculate frequencies/growth rates, run + this can be suppressed by setting ``` - $ julia --project run_post_processing.jl runs/ + export QT_QPA_PLATFORM=offscreen ``` - passing the directory to process as a command line argument. Input options for post-processing can be specified in post_processing_input.jl. + in your `.bashrc` or `.bash_profile` files. -7) Parameter scans (see [Running parameter scans](#running-parameter-scans)) or performance tests can be performed by running - ``` - $ julia -O3 --project driver.jl - ``` - If running a scan, it can be parallelised by passing the number of processors as an argument. Scan options are set in `scan_inputs.jl`. +## Run a simulation -8) Post processing can be done for several directories at once using +To run julia with optimization, type +``` +$ julia -O3 --project run_moment_kinetics.jl input.toml +``` +Options are specified in a TOML file, e.g. `input.toml` here. The defaults are +specified in `moment_kinetics_input.jl`. +* To run in parallel, just put `mpirun -np ` in front of the call you would + normally use, with `` the number of processes to use. +* It may be more convenient when running `moment_kinetics` more than once to + work from the Julia REPL, e.g. ``` - $ julia --project post_processing_driver.jl runs/ runs/ ... + $ julia -O3 --project + julia> using moment_kinetics + julia> run_moment_kinetics(input) ``` - passing the directories to process as command line arguments. Optionally pass a number as the first argument to parallelise post processing of different directories. Input options for post-processing can be specified in `post_processing_input.jl`. - -9) In the course of development, it is sometimes helpful to upgrade the Julia veriosn. Upgrading the version of Julia or upgrading packages may require a fresh installation of `moment_kinetics`. To make a fresh install with the latest package versions it is necessary to remove (or rename) the `Manifest.jl` file in the main directory, and generate a new `Manifest.jl` with step 1) above. It can sometimes be necessary to remove or rename the `.julia/` folder in your root directory for this step to be successful. - -10) One may have to set an environment variable to avoid error messages from the Qt library. If you execute the command - + where `input` is a `Dict()` containing any non-default options desired. + Input can also be loaded from a TOML file passing the filaname as a String + to the second argument, e.g. ``` - $ julia --project run_post_processing.jl runs/your_run_dir/ + julia> run_moment_kinetics("input.toml") ``` - -and see the error message - - - qt.qpa.xcb: could not connect to display - qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "" even though it was found. - This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem. - - -this can be suppressed by setting -``` -export QT_QPA_PLATFORM=offscreen -``` -in your `.bashrc` or `.bash_profile` files. + Especially when developing the code, a lot of compilation time can be saved + by using [Revise.jl](https://timholy.github.io/Revise.jl/stable/), and + re-running a test case in the REPL (without restarting `julia`). ### Stopping a run @@ -127,6 +98,64 @@ it is present writes all output and then returns cleanly. The 'stop file' is deleted when a run is (re-)started, if present, so you do not have to manually delete it before (re-)starting the run again. +## Restarting + +To restart a simulation using `input.toml` from the last time point in the +existing run directory, +``` +$ julia -O3 --project run_moment_kinetics --restart input.toml +``` +or to restart from a specific output file - either from the same run or (if the +settings are compatible, see below) a different one - here +`runs/example/example.dfns.h5` +``` +$ julia -O3 --project run_moment_kinetics input.toml runs/example/example.dfns.h5 +``` +The output file must include distribution functions. When not using parallel +I/O there will be multiple output files from different MPI ranks - any one of +these can be passed. + +To do the same from the Julia REPL +``` +$ julia -O3 --project +julia> run_moment_kinetics("input.toml", restart=true) +``` +or +``` +julia> run_moment_kinetics("input.toml", restart="runs/example/example.dfns.h5") +``` + +When calling the `run_moment_kinetics()` function you can also choose a +particular time index to restart from, e.g. +``` +julia> run_moment_kinetics("input.toml", restart="runs/example/example.dfns.h5", restart_time_index=42) +``` + +## Post processing quickstart + +To make plots and calculate frequencies/growth rates, run +``` +$ julia --project run_post_processing.jl runs/ +``` +passing the directory to process as a command line argument. Input options +for post-processing can be specified in `post_processing_input.jl`. Note that +even when running interactively, it is necessary to restart Julia after +modifying `post_processing_input.jl`. + +Post processing can be done for several directories at once using +``` +$ julia --project post_processing_driver.jl runs/ runs/ ... +``` +passing the directories to process as command line arguments. Optionally pass a +number as the first argument to parallelise post processing of different +directories. + +### Alternative post-processing + +An alternative post-processing module, written to be a bit more generic and +flexible, and able to be used interactively, is provided in +`makie_post_processing`, see [Post processing](@ref). + ## Parallel I/O Note that to enable parallel I/O, you need to get HDF5.jl to use the system @@ -147,20 +176,49 @@ links to an incompatible libcurl, causing an error. When compiled from source dependency), avoiding the problem. ## Running parameter scans -Parameter scans can be run, and can (optionally) use multiple processors. Short summary of implementation and usage: -1) `mk_input()` takes a Dict argument, which can modify values. So `mk_input()` sets the 'defaults' (for a scan), which are overridden by any key/value pairs in the Dict. -2) `mk_scan_inputs()` (in `scan_input.jl`) creates an Array of Dicts that can be passed to `mk_input()`. It first creates a Dict of parameters to scan over (keys are the names of the variable, values are an Array to scan over), then assembles an Array of Dicts (where each entry in the Array is a Dict with a single value for each variable being scanned). Most variables are combined as an 'inner product', e.g. `{:ni=>[0.5, 1.], :nn=>[0.5, 0.]}` gives `[{:ni=>0.5, :nn=>0.5}, {ni=>1., nn=>0.}]`. Any special variables specified in the `combine_outer` array are instead combined with the rest as an 'outer product', i.e. an entry is created for every value of those variables for each entry in the 'inner-producted' list. [This was just complicated enough to run the scans I've done so far without wasted simulations.] -3) The code in `driver.jl` picks between a single run (normal case), a performance_test, or creating a scan by calling `mk_scan_input()` and then looping over the returned array, calling `mk_input()` and running a simulation for each entry. This loop is parallelised (with the set of simulations dispatched over several processes - each simulation is still running serially). Running a scan (on 12 processes - actually 13 but the 'master' process doesn't run any of the loop bodies, so there are 12 'workers'): +Parameter scans can be run, and can (optionally) use multiple processors. Short +summary of implementation and usage: +1) `mk_input()` takes a Dict argument, which can modify values. So `mk_input()` + sets the 'defaults' (for a scan), which are overridden by any key/value + pairs in the Dict. +2) `mk_scan_inputs()` (in `scan_input.jl`) creates an Array of Dicts that can + be passed to `mk_input()`. It first creates a Dict of parameters to scan + over (keys are the names of the variable, values are an Array to scan + over), then assembles an Array of Dicts (where each entry in the Array is a + Dict with a single value for each variable being scanned). Most variables + are combined as an 'inner product', e.g. `{:ni=>[0.5, 1.], :nn=>[0.5, 0.]}` + gives `[{:ni=>0.5, :nn=>0.5}, {ni=>1., nn=>0.}]`. Any special variables + specified in the `combine_outer` array are instead combined with the rest + as an 'outer product', i.e. an entry is created for every value of those + variables for each entry in the 'inner-producted' list. [This was just + complicated enough to run the scans I've done so far without wasted + simulations.] +3) The code in `driver.jl` picks between a single run (normal case), a + performance_test, or creating a scan by calling `mk_scan_input()` and then + looping over the returned array, calling `mk_input()` and running a + simulation for each entry. This loop is parallelised (with the set of + simulations dispatched over several processes - each simulation is still + running serially). Running a scan (on 12 processes - actually 13 but the + 'master' process doesn't run any of the loop bodies, so there are 12 + 'workers'): ``` $ julia -O3 --project driver.jl 12 ``` (runs in serial if no argument is given) -4) The scan puts each run in a separate directory, named with a prefix specified by `base_name` in `scan_input.jl` and the rest the names and values of the scanned-over parameters (the names are created in `mk_scan_input()` too, and passed as the `:run_name` entry of the returned Dicts). -5) To run `post_processing.analyze_and_plot_data()` over a bunch of directories (again parallelized trivially, and the number of processes to use is an optional argument, serial if omitted): +4) The scan puts each run in a separate directory, named with a prefix + specified by `base_name` in `scan_input.jl` and the rest the names and + values of the scanned-over parameters (the names are created in + `mk_scan_input()` too, and passed as the `:run_name` entry of the returned + Dicts). +5) To run `post_processing.analyze_and_plot_data()` over a bunch of directories + (again parallelized trivially, and the number of processes to use is an + optional argument, serial if omitted): ``` $ julia -O3 --project post_processing_driver.jl 12 runs/scan_name_* ``` -6) Plotting the scan is not so general, `plot_comparison.jl` does it, but is only set up for the particular scans I ran - everything except the charge exchange frequencies is hard-coded in. +6) Plotting the scan is not so general, `plot_comparison.jl` does it, but is + only set up for the particular scans I ran - everything except the charge + exchange frequencies is hard-coded in. ``` $ julia -O3 --project plot_comparison.jl ``` @@ -193,7 +251,9 @@ There is a test suite in the `test/` subdirectory. It can be run in a few ways: ``` Individual test files can also be used instead of `runtests.jl`, which runs all the tests. -By default the test suite should run fairly quickly (in a few minutes). To do so, it skips many cases. To run more comprehensive tests, you can activate the `--long` option: +By default the test suite should run fairly quickly (in a few minutes). To do +so, it skips many cases. To run more comprehensive tests, you can activate the +`--long` option: * Using `test_args` argument ``` julia> Pkg.test(; test_args=["--long"]) @@ -209,4 +269,6 @@ By default the test suite should run fairly quickly (in a few minutes). To do so $ julia -O3 --project --long test/runtests.jl ``` -To get more output on what tests were successful, an option `--verbose` (or `-v`) can be passed in a similar way to `--long` (if any tests fail, the output is printed by default). +To get more output on what tests were successful, an option `--verbose` (or +`-v`) can be passed in a similar way to `--long` (if any tests fail, the output +is printed by default).