Skip to content

Commit

Permalink
Merge pull request #268 from ExCALIBUR-NEPTUNE/fix/helper-script-updates
Browse files Browse the repository at this point in the history 
Helper script updates
  • Loading branch information
@oparry-ukaea
oparry-ukaea authored Dec 9, 2024
2 parents 3acf307 + 3030887 commit 0a13b29
Show file tree
Hide file tree
Showing 5 changed files with 180 additions and 135 deletions.
13 changes: 6 additions & 7 deletions .gitignore
View file
Original file line number Diff line number Diff line change
@@ -1,8 +1,3 @@
build*/
bin
lib
thirdparty/
include/revision.hpp
.venv
*.bak
*.gcda
Expand All @@ -14,21 +9,25 @@ include/revision.hpp
**/*.fld
**/*.opt
*~

# Covers spack build symlinks and other common manual build locations
build*

# spack output
.spack-env
spack-build-*.txt
spack-configure-args.txt
spack-build-*/
install-time-test-log.txt
spack.lock
views/
builds/

# CMake
cmake.check_cache
CMakeCache.txt

# Directory generated by scripts/run_eg.sh
example-runs
runs

# IDE settings
.vscode
Expand Down
37 changes: 23 additions & 14 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -69,23 +69,19 @@ packages and their dependencies will be installed in the usual Spack
locations. They will also be linked into ["filesystem
views"](https://spack.readthedocs.io/en/latest/environments.html#filesystem-views)
`view/gcc-hipsycl` and `view/oneapi-dpcpp`. The NESO builds will be
done in directories called something like `spack-build-abc1234` (the
done in directories called something like `build-arch-abc1234` (the
hashes at the end will differ). If you change your spack installation
in some way (e.g., upgrading the version of a dependency) then the
hash will change and NESO and/or Nektar++ will be rebuilt. The
activation provides the convenience command `cleanup` to delete these
old builds.

In order to tell which build is which, symlinks `builds/gcc-hipsycl`
and `builds/oneapi-dpcpp` are provided. As Nektar++ is being built
from the submodule, its build trees are located at
`nektar/spack-build-abc1234` (the hashes at the end will differ) and
can be accessed with symlinks `nektar/builds/gcc` and
`nektar/builds/oneapi`. Test binaries will be contained within
these build directories. The activation script launches a background
task which regularly checks whether the hashes of your NESO and
Nektar++ builds has changed. If they have, it will update the
symlinks. They will also be checked whenever the environment is
In order to tell which NESO build is which, symlinks are generated at
`builds/gcc-<hash>` and `builds/oneapi-<hash>`. Similar links are generated for
Nektar++ and neso-particles in their own `builds` subdirectories. The activation
script also launches a background task which regularly checks whether the hashes
of your NESO, Nektar++ and neso-particles builds have changed. If they have, it
will update the symlinks. They will also be checked whenever the environment is
activated or deactivated.

#### Developing
Expand All @@ -97,7 +93,7 @@ configuration for NESO (as specificed in the NESO [package
repository](https://github.com/ExCALIBUR-NEPTUNE/NESO-Spack) and is
the same as if you were doing a traditional Spack installation of a
named version of NESO. This has the particular advantage of building
with all toolchaings (i.e., GCC, OneAPI) at one time. It also works
with all toolchains (i.e., GCC, OneAPI) at one time. It also works
well if you are developing NESO and Nektar++ simultaneously, as it
will rebuild both. The main disadvantage of this approach is that
Spack hides the output of CMake during the build process and will only
Expand Down Expand Up @@ -302,7 +298,9 @@ To run a solver example:
```
./scripts/run_eg.sh [solver_name] [example_name] <-n num_MPI> <-b build_dir>
```
which will look for the solver executable in the most recently modified spack-build-* directory, unless one is supplied with `-b`. Output is generated in `example-runs/<solver_name>/<example_name>`.
which will look for the solver executable in the most recently modified build
directory (or symlink) in `builds`, unless an alternative location is supplied
with `-b`. Output is generated in `runs/<solver_name>/<example_name>`.

## Address Sanitizers

Expand All @@ -319,6 +317,16 @@ One pipeline for creating simple Nektar++ meshes is through Gmsh and NekMesh.
.msh -> .xml with NekMesh

A .geo file can be created using the Gmsh UI (each command adds a new line to the .geo file). For simple meshes it may be easier to produce the .geo file in a text editor. .geo files can also be loaded into the UI to edit.

The script at `scripts/geo_to_xml.sh` can be used to convert .geo files to
Nektar++ XML meshes. Execute the script with no arguments to see the available
options. Note that no `<EXPANSIONS>` node is generated; users need to add one
themselves, either to the mesh file itself, or to another xml file that gets
passed to Nektar++. Alternatively, an xml mesh can be generated by running Gmsh
and NekMesh separately, using the instructions below.


An example .geo file is shown below:
<details>
<summary>Expand</summary>
mesh.geo
Expand Down Expand Up @@ -364,8 +372,9 @@ Recombine Surface "*";
```
</details>
Loading the above file in the Gmsh GUI, then selecting `2D mesh` and saving will produce a .msh file.
Alternatively for simple meshes one can jump straight to the next step by writing the .msh file directly in a text editor.
Selecting 2D mesh in Gmsh and saving will produce a .msh file. The mesh should be visible in the UI to check before saving. Alternatively for simple meshes one can jump straight to this step by writing the .msh file in a text editor.
<details>
<summary>Expand</summary>
mesh.msh
Expand Down
152 changes: 70 additions & 82 deletions activate
View file
Original file line number Diff line number Diff line change
Expand Up @@ -27,59 +27,57 @@ deactivate() {
unset NESO_DEV_DIR

# Undeclare all the functions in this file
unset -f deactivate
unset -f list-neso-builds
unset -f list-builds
unset -f cleanup
unset -f cleanup-deprecated-builds
unset -f create-link
unset -f deactivate
unset -f gen-build-link-maps
unset -f link-builds
unset -f link-nektar-builds
unset -f link-neso-builds
unset -f link-neso-particles-builds
unset -f setup
unset -f monitor-build-dirs
unset -f setup

spack env deactivate
}

# Get a list of the NESO builds, including both compiler and SYCL
# implementation
list-neso-builds() {
spack find --format "{compiler.name}-{^sycl.name}/{hash:7}" neso
return $?
}

# Get a list of the builds of a package (specified as the first
# argument), with the name of the compiler.
list-builds() {
spack find --format "{compiler.name}/{hash:7}" $1
return $?
}

# Deletes spack build directories for hashes which are no longer
# Deletes build directory links for hashes which are no longer
# installed in the environment. First argument is the package name and
# second is the directory in which to cleanup installs.
# second is the directory in which to cleanup links.
cleanup-deprecated-builds() {
local package="$1"
local basedir="$2"
local pkg_name="$1"
local pkg_root_dir="$2"
local links_dir="$pkg_root_dir/builds"

# Use the build-link-map to compile a list of valid link paths
local valid_hashes=() valid_link_paths=()
for entry in $(gen-build-link-maps "$pkg_name" "{compiler.name}-{hash:7}"); do
IFS="|" read -ra _split <<<"$entry"
valid_link_paths+=("${links_dir}/${_split[1]}")
IFS="-" read -ra _split2 <<<"${_split[1]}"
valid_hashes+=("${_split2[1]}")
done

local good_hashes=()
local h _vals
for h in $(list-builds "${package}"); do
IFS="/" read -ra _vals <<<"$h"
good_hashes+=("${_vals[1]}")
# Remove links from links_dir if they don't appear in valid_link_paths
link_paths=$(find "$links_dir" -maxdepth 1 -type l -regextype posix-egrep -regex '.*-[a-z0-9]{7}$')
for link_path in $link_paths; do
if ! [[ " ${valid_link_paths[*]} " =~ " ${link_path} " ]]
then
echo "Cleaning up stale ${pkg_name} build link at ${link_path}"
rm -Rf "$link_path" > /dev/null
fi
done

local f
for f in "$basedir"/spack-build-???????; do
if [[ -d "$f" ]]
# Also cleanup top-level build links created by spack install
spack_link_paths=$(find "$pkg_root_dir" -type l -regextype posix-egrep -regex "${pkg_root_dir}/build-.*-[a-z0-9]{7}$")
for link_path in $spack_link_paths; do
local hash="${link_path:(-7)}"
if ! [[ " ${valid_hashes[*]} " =~ " ${hash} " ]]
then
local hash="${f:(-7)}"
if ! [[ " ${good_hashes[*]} " =~ " ${hash} " ]]
then
echo "Removing build directory $f"
rm -Rf "$f" > /dev/null
fi
fi
echo "Cleaning up stale ${pkg_name} SPACK build link at ${link_path}"
rm -Rf "$link_path" > /dev/null
fi
done
}

Expand All @@ -92,68 +90,58 @@ create-link() {
link="$2"
if ! [[ -L "$link" && $(readlink "$link") == "$target" ]]
then
echo " Linking $(realpath -s --relative-to="$(pwd)" "$link") => $target"
echo " Linking $(realpath -s --relative-to="$NESO_DEV_DIR" "$link") => $target"
rm -f "$link" > /dev/null
ln -s "$target" "$link"
fi
}

# Creates symlinks to the nektar build directories which have
# meaningful names
link-nektar-builds() {
mkdir -p "${NESO_DEV_DIR}/nektar/builds"
local build _vals builds
builds=$(list-builds nektar) # Need to declare builds already or else declaration would overwrite $?
local list_err=$?
if [[ $list_err != 0 ]]
then
return $list_err
fi
# Find all instances of a package installed in the environment and return a
# string for each instance of the form "package_name/hash|package_link"
gen-build-link-maps() {
local package="$1"
local link_fmt="$2"

for build in $builds; do
IFS="/" read -ra _vals <<<"$build"
create-link "../spack-build-${_vals[1]}" "${NESO_DEV_DIR}/nektar/builds/${_vals[0]}"
local identifier_fmt="{package.name}/{hash:7}"
spack find --format "$identifier_fmt|$link_fmt" "$package"
return $?
}


# Generate soft links to the spack build directories for all instances of a
# package installed in the environment
link-builds() {
local package="$1"
local link_fmt="$2"
local link_dir="$3"

local build_dir
for entry in $(gen-build-link-maps "$package" "$link_fmt"); do
IFS="|" read -ra _split <<<"$entry"
build_dir=$(spack location -b "${_split[0]}")
create-link "${build_dir}" "${link_dir}/${_split[1]}"
done

}

# Creates symlinks to the nektar build directories which have meaningful names
link-nektar-builds() {
link-builds "nektar" "{compiler.name}-{hash:7}" "${NESO_DEV_DIR}/nektar/builds"
}

# Creates symlinks to the neso-particles build directories which have
# meaningful names
link-neso-particles-builds() {
mkdir -p "${NESO_DEV_DIR}/neso-particles/builds"
local build _vals builds
builds=$(list-builds neso-particles) # Need to declare builds already or else declaration would overwrite $?
local list_err=$?
if [[ $list_err != 0 ]]
then
return $list_err
fi

for build in $builds; do
IFS="/" read -ra _vals <<<"$build"
create-link "../spack-build-${_vals[1]}" "${NESO_DEV_DIR}/neso-particles/builds/${_vals[0]}"
done
link-builds "neso-particles" "{compiler.name}-{hash:7}" "${NESO_DEV_DIR}/neso-particles/builds"
}

# Creates symlinks to the NESO build directories which have
# meaningful names
# Creates symlinks to the NESO build directories which have meaningful names
link-neso-builds() {
mkdir -p "${NESO_DEV_DIR}/builds"
local build _vals builds
builds=$(list-neso-builds) # Need to declare builds already or else declaration would overwrite $?
list_err=$?
if [[ $list_err != 0 ]]
then
return $list_err
fi

for build in ${builds}; do
IFS="/" read -ra _vals <<<"$build"
create-link "../spack-build-${_vals[1]}" "${NESO_DEV_DIR}/builds/${_vals[0]}"
done
link-builds "neso" "{compiler.name}-{hash:7}" "${NESO_DEV_DIR}/builds"
}


# Create symlink to the build directories
# Create symlinks to the build directories
setup() {
echo "Checking links to NESO build directories"
link-neso-builds
Expand Down Expand Up @@ -183,7 +171,7 @@ cleanup() {
cleanup-deprecated-builds neso-particles "${NESO_DEV_DIR}/neso-particles"
}

spack env activate -p -d ${NESO_DEV_DIR}
spack env activate -p -d "${NESO_DEV_DIR}"
setup
( monitor-build-dirs ) & __MONITOR_PROCESS=$!

Expand Down
Loading

0 comments on commit 0a13b29

Please sign in to comment.