Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Packaging enhancements #32

Merged
merged 17 commits into from
May 22, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 0 additions & 3 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,9 +29,6 @@ jobs:
with:
submodules: true

- name: check bash
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this check removed? It was checking something pretty subtle and weird but I see there are no comments as to what exactly :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Not against removing, just checking why)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I looked at the code and couldn't understand why this check needs to be present. It was doing no useful work (basically if true; then echo ok; fi then repeat for another time), and there was no comment to explain its significance. I thought it was legacy leftovers not cleaned up. I double checked my CI and packaging runs and this step never failed (I would be surprised if it did). My guess is it might be checking whether the Windows association step worked (script is run with bash rather than powershell), and was likely added for debugging purposes, so I removed it. If anything breaks in the future I can always come back to it.
Please do share any extra context you have on what the problem was.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it was likely something like that, figuring out the association, and that a good version of bash was running. Doing bash-like stuff in windows was delicate, and I could easily imagine it breaking in future again. Still, since these checks have no comments, fine to remove them. Thanks for explaining.

run: yarn run check:bash:ci

- uses: actions/setup-node@v3
with:
node-version: ${{ matrix.node }}
Expand Down
48 changes: 37 additions & 11 deletions .github/workflows/package.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,8 @@ jobs:
matrix:
os:
- macos-14
- ubuntu-20.04
- windows-latest
- ubuntu-22.04
- windows-2022
host:
- x64
target:
Expand All @@ -35,10 +35,14 @@ jobs:
node: 18
host: arm64
target: arm64
- os: windows-2022
node: 18
host: x64
target: x86
name: ${{ matrix.os }} (node=${{ matrix.node }}, host=${{ matrix.host }}, target=${{ matrix.target }})
steps:
- name: association
if: contains(matrix.os, 'windows')
- name: Set up bash association
if: startsWith(matrix.os, 'windows')
run: |
ls
ls "c:\\"
Expand All @@ -58,20 +62,35 @@ jobs:
git submodule foreach --recursive git clean -ffdx
git submodule foreach --recursive git reset --hard

- name: check bash
paulfitz marked this conversation as resolved.
Show resolved Hide resolved
run: yarn run check:bash:ci

- uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node }}
architecture: ${{ matrix.host }}
paulfitz marked this conversation as resolved.
Show resolved Hide resolved

- name: Add yarn
if: matrix.os == 'macos-13' || matrix.os == 'macos-14'
run: npm install -g yarn
- uses: actions/cache@v4
id: yarn-cache
with:
path: |
node_modules
core/node_modules
key: ${{ matrix.os }}-${{ matrix.target }}-yarn-${{ hashFiles('**/yarn.lock') }}
restore-keys: |
${{ matrix.os }}-${{ matrix.target }}-yarn-

- name: Install dependencies
if: steps.yarn-cache.outputs.cache-hit != 'true'
run: yarn install --network-timeout 300000

- name: Fix Windows x86 sqlite3 binding
if: steps.yarn-cache.outputs.cache-hit != 'true' && startsWith(matrix.os, 'windows') && matrix.target == 'x86'
run: yarn upgrade sqlite3
SleepyLeslie marked this conversation as resolved.
Show resolved Hide resolved
# Prebuilt binding is for x64. We must build from source for x86 target.
# See:
# https://stackoverflow.com/questions/72553650/how-to-get-node-sqlite3-working-on-mac-m1
# https://yarnpkg.com/lang/en/docs/envvars/#toc-npm-config
env:
npm_config_build_from_source: true
npm_config_target_arch: ia32
npm_config_fallback_to_build: true

- name: Hooks and crooks
run: yarn run setup
Expand All @@ -89,10 +108,17 @@ jobs:
# github endpoints hang up quite a bit, add one retry.
run: yarn run electron:ci || yarn run electron:ci
env:
TARGET_ARCH: ${{ matrix.target }}
GITHUB_TOKEN: ${{ github.token }}
DEBUG: electron-builder
APPLE_ID: ${{ secrets.APPLE_ID }}
APPLE_ID_PASSWORD: ${{ secrets.APPLE_ID_PASSWORD }}
APPLE_TEAM_ID: ${{ secrets.APPLE_TEAM_ID }}
CSC_KEY_PASSWORD: ${{ secrets.CSC_KEY_PASSWORD }}
CSC_LINK: ${{ secrets.CSC_LINK }}

- uses: actions/upload-artifact@v4
with:
name: ${{ matrix.os }}-${{ matrix.host }}-${{ matrix.target }}
path: dist/grist-electron-*
if-no-files-found: "error"
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,7 @@ and from an early standalone version of Grist developed at Grist Labs.

## Roadmap

* [ ] Set up a Windows x86 build (need to resolve OOM issue)
* [x] Set up a Windows x86 build
* [x] Set up a Windows x64 build
* [x] Set up a Linux x64 build
* [x] Set up a Mac x64 build
Expand Down
26 changes: 18 additions & 8 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,9 @@
"electron:preview": "electron core/_build/ext/app/electron/main.js",
"electron:dir": "electron-builder build --linux --dir",
"electron:linux": "electron-builder build --linux",
"electron:ci": "electron-builder build --publish always",
"electron:ci": "./scripts/ci.sh",
"electron": "electron-builder build --publish never",
"test": "cd core && rm -f junk.db && node _build/app/server/companion.js sqlite gristify junk.db && ls junk.db",
"check:bash:ci": "./scripts/check.sh"
"test": "cd core && rm -f junk.db && node _build/app/server/companion.js sqlite gristify junk.db && ls junk.db"
},
"resolutions": {
"jquery": "3.5.0",
Expand All @@ -40,9 +39,23 @@
"includeSubNodeModules": true,
"icon": "core/static/icons/grist.png",
"win": {
"artifactName": "${productName}-windows-${version}-${arch}.${ext}",
"target": [
{
"target": "nsis"
},
{
"target": "portable"
SleepyLeslie marked this conversation as resolved.
Show resolved Hide resolved
}
],
"icon": "core/static/icons/grist.ico"
},
"portable": {
"artifactName": "${productName}-windows-portable-${version}-${arch}.${ext}"
},
"nsis": {
"artifactName": "${productName}-windows-installer-${version}-${arch}.${ext}",
"perMachine": true
},
"mac": {
"artifactName": "${productName}-mac-${version}-${arch}.${ext}",
"icon": "core/static/icons/grist.icns",
Expand Down Expand Up @@ -108,9 +121,6 @@
"role": "Editor",
"icon": "core/static/icons/gristdoc"
}
],
"nsis": {
"perMachine": true
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you know what this was?

Copy link
Collaborator Author

@SleepyLeslie SleepyLeslie May 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to this documentation, this option does nothing when oneClick is true (default, we didn't change it), so I removed it.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI otherwise this option tells NSIS to skip the installation mode page and always installs Grist Desktop system-wide. This might not be a good idea in restricted environments where user does not have admin privileges, but we now have the portable build so this shouldn't matter much?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pretty sure this was the hard-to-find solution to some gnarly problem, but we shouldn't keep it out of superstition. Someone will shout if it turns out to have some effect that was important, and then we can document the reason better next time :-). Cheers.

}
]
}
}
7 changes: 0 additions & 7 deletions scripts/check.sh

This file was deleted.

6 changes: 0 additions & 6 deletions scripts/check2.sh

This file was deleted.

21 changes: 21 additions & 0 deletions scripts/ci.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
#!/bin/bash

set -e

case $TARGET_ARCH in
"x86")
ARCHFLAG="--ia32"
;;
"x64")
ARCHFLAG="--x64"
;;
"arm64")
ARCHFLAG="--arm64"
;;
*)
echo "Target architecture $TARGET_ARCH not supported"
exit 1
;;
esac

electron-builder build $ARCHFLAG --publish always
30 changes: 15 additions & 15 deletions yarn.lock
Original file line number Diff line number Diff line change
Expand Up @@ -2578,9 +2578,9 @@ camelcase@^6.0.0:
integrity sha512-Gmy6FhYlCY7uOElZUSbxo2UCDH8owEk996gkbrpsgGtrJLM3J7jGxl9Ic7Qwwj4ivOE5AWZWRMecDdF7hqGjFA==

caniuse-lite@^1.0.30001587:
version "1.0.30001618"
resolved "https://registry.yarnpkg.com/caniuse-lite/-/caniuse-lite-1.0.30001618.tgz#fad74fa006aef0f01e8e5c0a5540c74d8d36ec6f"
integrity sha512-p407+D1tIkDvsEAPS22lJxLQQaG8OTBEqo0KhzfABGk0TU4juBNDSfH0hyAp/HRyx+M8L17z/ltyhxh27FTfQg==
version "1.0.30001620"
resolved "https://registry.yarnpkg.com/caniuse-lite/-/caniuse-lite-1.0.30001620.tgz#78bb6f35b8fe315b96b8590597094145d0b146b4"
integrity sha512-WJvYsOjd1/BYUY6SNGUosK9DUidBPDTnOARHp3fSmFO1ekdxaY6nKRttEVrfMmYi80ctS0kz1wiWmm14fVc3ew==

[email protected]:
version "7.1.1"
Expand Down Expand Up @@ -3590,9 +3590,9 @@ [email protected]:
mime "^2.5.2"

electron-to-chromium@^1.4.668:
version "1.4.770"
resolved "https://registry.yarnpkg.com/electron-to-chromium/-/electron-to-chromium-1.4.770.tgz#a26df8541a7fd92d938a2b42c70ba2502d0e9c62"
integrity sha512-ONwOsDiVvV07CMsyH4+dEaZ9L79HMH/ODHnDS3GkIhgNqdDHJN2C18kFb0fBj0RXpQywsPJl6k2Pqg1IY4r1ig==
version "1.4.774"
resolved "https://registry.yarnpkg.com/electron-to-chromium/-/electron-to-chromium-1.4.774.tgz#1017d1758aaeeefe5423aa9d67b4b1e5d1d0a856"
integrity sha512-132O1XCd7zcTkzS3FgkAzKmnBuNJjK8WjcTtNuoylj7MYbqw5eXehjQ5OK91g0zm7OTKIPeaAG4CPoRfD9M1Mg==

electron-updater@^5.3.0:
version "5.3.0"
Expand Down Expand Up @@ -3753,9 +3753,9 @@ es-errors@^1.3.0:
integrity sha512-Zf5H2Kxt2xjTvbJvP2ZWLEICxA6j+hAmMzIlypy4xcBg1vKVnx89Wy0GbS+kf5cwCVFFzdCFh2XSCFNULS6csw==

es-module-lexer@^1.2.1:
version "1.5.2"
resolved "https://registry.yarnpkg.com/es-module-lexer/-/es-module-lexer-1.5.2.tgz#00b423304f2500ac59359cc9b6844951f372d497"
integrity sha512-l60ETUTmLqbVbVHv1J4/qj+M8nq7AwMzEcg3kmJDt9dCNrTk+yHcYFf/Kw75pMDwd9mPcIGCG5LcS20SxYRzFA==
version "1.5.3"
resolved "https://registry.yarnpkg.com/es-module-lexer/-/es-module-lexer-1.5.3.tgz#25969419de9c0b1fbe54279789023e8a9a788412"
integrity sha512-i1gCgmR9dCl6Vil6UKPI/trA69s08g/syhiDK9TG0Nf1RJjjFI+AzoWW7sPufzkgYAn861skuCwJa0pIIHYxvg==

es6-error@^4.1.1:
version "4.1.1"
Expand Down Expand Up @@ -4219,9 +4219,9 @@ fast-text-encoding@^1.0.0:
integrity sha512-VhXlQgj9ioXCqGstD37E/HBeqEGV/qOD/kmbVG8h5xKBYvM1L3lR1Zn4555cQ8GkYbJa8aJSipLPndE1k6zK2w==

fast-xml-parser@^4.2.2:
version "4.3.6"
resolved "https://registry.yarnpkg.com/fast-xml-parser/-/fast-xml-parser-4.3.6.tgz#190f9d99097f0c8f2d3a0e681a10404afca052ff"
integrity sha512-M2SovcRxD4+vC493Uc2GZVcZaj66CCJhWurC4viynVSTvrpErCShNcDz1lAho6n9REQKvL/ll4A4/fw6Y9z8nw==
version "4.4.0"
resolved "https://registry.yarnpkg.com/fast-xml-parser/-/fast-xml-parser-4.4.0.tgz#341cc98de71e9ba9e651a67f41f1752d1441a501"
integrity sha512-kLY3jFlwIYwBNDojclKsNAC12sfD6NwW74QB2CoNGPvtVxjliYehVunB3HYyNi+n4Tt1dAcgwYvmKF/Z18flqg==
dependencies:
strnum "^1.0.5"

Expand Down Expand Up @@ -7506,9 +7506,9 @@ schema-utils@^3.1.1, schema-utils@^3.2.0:
ajv-keywords "^3.5.2"

selenium-webdriver@^4.20.0:
version "4.20.0"
resolved "https://registry.yarnpkg.com/selenium-webdriver/-/selenium-webdriver-4.20.0.tgz#14941ab4a59e8956a5e4b4491a8ba2bd6619d1ac"
integrity sha512-s/G44lGQ1xB3tmtX6NNPomlkpL6CxLdmAvp/AGWWwi4qv5Te1+qji7tPSyr6gyuoPpdYiof1rKnWe3luy0MrYA==
version "4.21.0"
resolved "https://registry.yarnpkg.com/selenium-webdriver/-/selenium-webdriver-4.21.0.tgz#d38aebfc34770421a880afcfdb7bd8fe85ce9174"
integrity sha512-WaEJHZjOWNth1QG5FEpxpREER0qptZBMonFU6GtAqdCNLJVxbtC3E7oS/I/+Q1sf1W032Wg0Ebk+m46lANOXyQ==
dependencies:
jszip "^3.10.1"
tmp "^0.2.3"
Expand Down
Loading