Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add alternative command to run npm ci with cache #96

Draft
wants to merge 5 commits into
base: trunk
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions bin/install_npm_packages
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,13 @@

# Ensure package.json is present
if [ ! -f package.json ]; then
echo "No valid package.json file found"
echo "No package.json found!"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Me being pedantic:

  • No validation made on package.json
  • We know that package.json is a file

exit 1
fi

# Ensure package-lock.json is present
# Ensure package-lock.json is present
if [ ! -f package-lock.json ]; then
echo "No valid package-lock.json file found"
echo "No package-lock.json found!"
exit 1
fi

Expand Down
70 changes: 70 additions & 0 deletions bin/install_npm_packages_clean
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
#!/bin/bash -eu

# Ensure package.json is present
if [ ! -f package.json ]; then
echo "No package.json found!"
exit 1
fi

# Ensure package-lock.json is present
if [ ! -f package-lock.json ]; then
echo "No package-lock.json found!"
exit 1
fi

PLATFORM=$(uname -s)
ARCHITECTURE=$(uname -m)
NODE_VERSION=$(node --version)
PACKAGE_HASH=$(hash_file package-lock.json)

# See https://www.npmjs.com/package/patch-package#circleci
if [ -d patches ]; then
PATCHES_HASH=$(hash_directory patches/)
else
PATCHES_HASH=nopatch
fi

CACHEKEY="$BUILDKITE_PIPELINE_SLUG-npm-$PLATFORM-$ARCHITECTURE-node-$NODE_VERSION-$PACKAGE_HASH-$PATCHES_HASH"

LOCAL_NPM_CACHE=./vendor/npm
mkdir -p $LOCAL_NPM_CACHE
echo "--- :npm: Set npm to use $LOCAL_NPM_CACHE for cache"
npm set cache $LOCAL_NPM_CACHE
echo "npm cache set to $(npm get cache)"
Comment on lines +29 to +33
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be worth making this configurable?


echo "--- :npm: Restore npm cache if present"
restore_cache "$CACHEKEY"

echo "--- :npm: Install Node dependencies"

# To avoid constant ECONNRESET errors a limit is set for Linux,
# as this is not happening with the Mac jobs.
# This issue is being tracked here:
# https://github.com/npm/cli/issues/4652
if [ "$PLATFORM" = "Linux" ]; then
MAX_SOCKETS=1
else
# Default value from npm.
# TODO: Link to source
MAX_SOCKETS=15
Comment on lines +47 to +49
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code this script was extracted from had the comment about 15 being the default. I had a quick look and couldn't find a reference. Need to look more and document.

fi

npm ci \
--unsafe-perm \
--prefer-offline \
--no-audit \
--no-progress \
--maxsockets "$MAX_SOCKETS" \
"$@" # TODO: Use approach from other commands where only a subset of flags are allowed

echo "--- :npm: Save cache if necessary"
# Notice that we don't cache the local node_modules.
# Those get regenerated by npm ci as per Node reccomendations.
# What we are caching is the root npm folder, which stores pacakge downloads so they are available if the package.json resolution demands them.
#
# npm stores temporary files in its cache that we don't want to extract because they might run into naming conflicts.
# So, before archiving it, we remove those tmp files.
#
# Example: https://buildkite.com/automattic/gutenberg-mobile/builds/8857#018e37eb-7afc-4280-b736-cba76f02f1a3/524
rm -rf "$LOCAL_NPM_CACHE/_cacache/tmp"
save_cache "$LOCAL_NPM_CACHE" "$CACHEKEY"