Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add command to build standalone binary #260

Merged
merged 1 commit into from
Feb 14, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions .github/actions/phar/action.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,19 +18,19 @@ runs:
shell: bash
working-directory: tools/phar

- name: Compile phar Linux
- name: Build Castor phar for Linux
run: bin/castor castor:phar:linux
shell: bash

- name: Compile phar Darwin
- name: Build Castor phar for Darwin
run: bin/castor castor:phar:darwin
shell: bash

- name: Compile phar Windows
- name: Build Castor phar for Windows
run: bin/castor castor:phar:windows
shell: bash

- name: Ensure PHAR is OK
- name: Ensure phar is OK
run: build/castor.linux-amd64.phar --version
shell: bash
working-directory: tools/phar
61 changes: 53 additions & 8 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,16 +35,33 @@ jobs:
REQUIRE_DEV: true

phpunit:
name: "PHPUnit on ${{ matrix.php }} ${{ matrix.phar && '(with phar)' || '' }}"
name: "PHPUnit on ${{ matrix.php }} | Castor from ${{ matrix.castor.method }}"
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
php: [ "8.1", "8.2", "8.3" ]
phar: [ false ]
include:
- php: "8.1"
phar: true
castor:
bin: 'bin/castor'
method: 'bin/castor'
- php: "8.1"
castor:
bin: 'tools/phar/build/castor.linux-amd64.phar'
method: 'phar'
- php: "8.1"
castor:
bin: 'castor'
method: 'binary'
- php: "8.2"
castor:
bin: 'bin/castor'
method: 'bin/castor'
- php: "8.3"
castor:
bin: 'bin/castor'
method: 'bin/castor'

steps:
- name: Checkout
uses: actions/checkout@v4
Expand All @@ -62,10 +79,25 @@ jobs:
run: composer install --prefer-dist --no-progress --optimize-autoloader --classmap-authoritative
working-directory: tools/phar

- name: Compile phar Linux
- name: Build Castor phar for Linux
run: bin/castor castor:phar:linux
shell: bash
if: matrix.phar
if: matrix.castor.method == 'phar' || matrix.castor.method == 'binary'

- name: Restore PHP static building artifacts cache
uses: actions/cache/restore@v4
with:
path: |
/home/runner/.cache/castor/castor-php-static-compiler
# https://github.com/actions/cache/blob/main/tips-and-workarounds.md#update-a-cache
key: dynamic-key-${{ github.run_id }}
restore-keys: |
php-static-building-artifacts-cache-
Comment on lines +92 to +95
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand this part, It could be written:

Suggested change
# https://github.com/actions/cache/blob/main/tips-and-workarounds.md#update-a-cache
key: dynamic-key-${{ github.run_id }}
restore-keys: |
php-static-building-artifacts-cache-
key: php-static-building-artifacts-cache

More over in the Save PHP static building artifacts cache the key could be only php-static-building-artifacts-cache`

see https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows#matching-a-cache-key

Copy link
Contributor Author

@tigitz tigitz Feb 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was my initial idea, unfortunately it doesn't work as expected.

When the restore action with key: X is executed, it looks for a cache[X] and use it (hit) if it exists.
When the save action with key: X is executed, it tries to save cache[X] but if fails if it already exists, it doesn't update the cache key.

And that's not what we want because when a new arg is added to the build command, a new directory is created: /home/runner/.cache/castor/castor-php-static-compiler/{new_compile_command_build_hash}/* and we want to update the cache with this new build artifacts.

If the current config what's happening is:

Run 1

  • Restore Action:
    • Does dynamic-key-${{ github.run_id }} cache hit ? No
    • Does php-static-building-artifacts-cache-* cache hit ? No
  • PHP Build
    • Generate artifacts in /home/runner/.cache/castor/castor-php-static-compiler/{compiler_test_command}/*
    • Generate artifacts in /home/runner/.cache/castor/castor-php-static-compiler/{build_compiler_for_CASTOR_BIN}/*
  • Save Action:
    • Save /home/runner/.cache/castor/castor-php-static-compiler/* as key: php-static-building-artifacts-cache-XXX where XXX is hash(castor-php-static-compiler/*)

Run 2

  • Restore Action:
    • Does dynamic-key-${{ github.run_id }} cache hit ? No
    • Does php-static-building-artifacts-cache-* cache hit ? Yes it founds php-static-building-artifacts-cache-XXX
  • PHP Build
    • Use artifacts cached in /home/runner/.cache/castor/castor-php-static-compiler/**/*, no build
  • Save Action:
    • Tries to save /home/runner/.cache/castor/castor-php-static-compiler/* as key: php-static-building-artifacts-cache-XXX but fails. It's ok no new artifacts were generated anyway.

Run 3, new compilation is added or options of existing build are changed in CompileCommandTest for example:

  • Restore Action:
    • Does dynamic-key-${{ github.run_id }} cache hit ? No
    • Does php-static-building-artifacts-cache-* cache hit ? Yes, restore from Run 1
  • PHP Build
    • Build hash is different than Run 1, generate artifacts in /home/runner/.cache/castor/castor-php-static-compiler/{compiler_test_command}/*
    • Build hash is same as Run 1, reuse artifacts in /home/runner/.cache/castor/castor-php-static-compiler/{build_compiler_for_CASTOR_BIN}/*
  • Save Action:
    • Save /home/runner/.cache/castor/castor-php-static-compiler/* as key: php-static-building-artifacts-cache-YYY where YYY is a new hash of castor-php-static-compiler/* because new artifacts where generated since XXX

Copy link
Member

@lyrixx lyrixx Feb 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are two folders

  • Generate artifacts in /home/runner/.cache/castor/castor-php-static-compiler/{compiler_test_command}/*
  • Generate artifacts in /home/runner/.cache/castor/castor-php-static-compiler/{build_compiler_for_CASTOR_BIN}/*

because

  • there is one for castor itself?
  • there is another one for tests/CompileCommandTest.php

If yes, I almost got it, but not perfectly :)
If not I'm totally lost 😅

Anyway, I'll approve the PR, play a bit with it, with the cache, etc :)

Thanks

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes exactly! Both build are not using the same extensions, the one in the test has less extension, it just makes sure it compiles and can run a hello world.

The compiled castor has more extensions because it needs to rune the whole suit of examples.


- name: Compile Custom Built PHP along Castor phar for Linux
run: bin/castor compile tools/phar/build/castor.linux-amd64.phar --php-extensions=mbstring,phar,posix,tokenizer,pcntl
shell: bash
if: matrix.castor.method == 'binary'

- name: Link box
run: sudo ln -s $GITHUB_WORKSPACE/tools/phar/vendor/bin/box /usr/local/bin/box
Expand All @@ -76,10 +108,23 @@ jobs:
- name: Run tests
run: vendor/bin/simple-phpunit
env:
CASTOR_BIN: ${{ github.workspace }}/${{ matrix.phar && 'tools/phar/build/castor.linux-amd64.phar' || 'bin/castor'}}
CASTOR_BIN: ${{ github.workspace }}/${{ matrix.castor.bin }}

# Workaround because hashFiles() can only include files inside the GITHUB_WORKSPACE
- name: Generate PHP static building artifacts cache key
run: echo "php_build_artifact_cache_key=$(find /home/runner/.cache/castor/castor-php-static-compiler -type f -exec md5sum {} + | sort -k 2 | md5sum | awk '{print $1}')" >> $GITHUB_ENV

- name: Save PHP static building artifacts cache
uses: actions/cache/save@v4
with:
path: |
/home/runner/.cache/castor/castor-php-static-compiler
key: php-static-building-artifacts-cache-${{ env.php_build_artifact_cache_key }}
# Cache outputs from the "binary" job specifically, as it generates an additional PHP build not produced by other jobs
if: matrix.castor.method == 'binary'

phar:
name: Ensure PHAR is OK
name: Ensure phar is OK
runs-on: ubuntu-latest

steps:
Expand Down
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
* Set the process title according to the current application name and task name
* Ignore some low level env vars in runnable command showed in logs
* Fix section output to work on Windows
* Add a `compile` command that puts together a customizable PHP binary with a repacked castor app into one executable file

## 0.12.1 (2024-02-06)

Expand Down
1 change: 1 addition & 0 deletions bin/generate-tests.php
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@
'log:with-context',
'parallel:sleep',
'repack',
'compile',
'run:ls',
'run:run-parallel',
];
Expand Down
57 changes: 57 additions & 0 deletions doc/going-further/compile.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# Compiling your application into a standalone binary

[Packing your Castor application as a phar](repack.md) can be a good way to easily
share and use it in various environments.

However, you need to ensure that PHP is installed and configured correctly in all
the environments where you want to use your Castor app.
This can be a hassle, especially if you don't have control over the environments.

To make things simpler, Castor's `compile` command can help by creating a
customizable PHP binary with a phar, making one executable file that can be used in any setting.

Just pass your repacked Castor app phar as an argument of this command.

## Pre-requisites

Follow the [`repack` documentation](repack.md) to produce a phar of your
Castor app.

## Running the Compile Command

To compile your Castor application, navigate to your project directory and run:

```bash
vendor/bin/castor compile my-custom-castor-app.phar
```

> [!WARNING]
> Compiling is not supported yet on Windows.

### Options

Make sure to take a look at the command description to see all the available options:
```bash
vendor/bin/castor compile --help
```

### Behavior

The `compile` command performs several steps:

1. Downloads or uses an existing [Static PHP CLI tool](https://github.com/crazywhalecc/static-php-cli) to compile PHP and the phar into a binary.
2. If required, it automatically installs dependencies and compiles PHP with the specified extensions.
3. Combines the compiled PHP and your phar file into a single executable.

## Post-Compilation

Once the compilation is finished, your Castor application is transformed into a standalone binary named `castor` by default (you can use the `--output` option to change it).

This binary is now ready to be distributed and run in environments that do not have PHP installed.

You can simply run it like any other executable:

```bash
./castor
tigitz marked this conversation as resolved.
Show resolved Hide resolved
```

1 change: 1 addition & 0 deletions doc/going-further/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@

* [Listening to events](events.md)
* [Repacking your application in a new phar](repack.md)
* [Compiling your application in a standalone binary](compile.md)

## Examples

Expand Down
10 changes: 10 additions & 0 deletions doc/going-further/repack.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
You have created a Castor application, with many tasks, and you want to
distribute it as a single phar file? Castor can help you with that.

## Pre-requisites

In your project, install Castor as a dependency:

```bash
Expand All @@ -18,6 +20,8 @@ configuration. See the [PHP
documentation](https://www.php.net/manual/en/phar.configuration.php#ini.phar.readonly) to disabled
`phar.readonly`.

## Running the Repack Command

Then, run the repack command to create the new phar:

```
Expand All @@ -34,3 +38,9 @@ vendor/bin/castor repack --help
> Castor will automatically import all files in the current directly.
> So ensure to have the less files possible in the directory where you run the
> repack task to avoid including useless files in the phar.

## Going further

Packaging your Castor app as a phar simplifies distribution but requires PHP setup on target systems.

[Castor's `compile` command](compile.md) streamlines this by embedding the phar in a PHP binary, creating a standalone executable for diverse environments.
2 changes: 1 addition & 1 deletion examples/failure.php
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
#[AsTask(description: 'A failing task not authorized to fail')]
function failure(): void
{
run('i_do_not_exist', path: '/tmp');
run('i_do_not_exist', path: '/tmp', pty: false);
}

#[AsTask(description: 'A failing task authorized to fail')]
Expand Down
5 changes: 4 additions & 1 deletion src/Console/ApplicationFactory.php
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

namespace Castor\Console;

use Castor\Console\Command\CompileCommand;
use Castor\Console\Command\DebugCommand;
use Castor\Console\Command\RepackCommand;
use Castor\ContextRegistry;
Expand Down Expand Up @@ -45,6 +46,7 @@ public static function create(): SymfonyApplication
$cacheDir = PlatformUtil::getCacheDirectory();
$cache = new FilesystemAdapter(directory: $cacheDir);
$logger = new Logger('castor', [], [new ProcessProcessor()]);
$fs = new Filesystem();

/** @var SymfonyApplication */
// @phpstan-ignore-next-line
Expand All @@ -56,7 +58,7 @@ public static function create(): SymfonyApplication
new ExpressionLanguage($contextRegistry),
new StubsGenerator($logger),
$logger,
new Filesystem(),
$fs,
$httpClient,
$cache,
new WaitForHelper($httpClient, $logger),
Expand All @@ -67,6 +69,7 @@ public static function create(): SymfonyApplication

if (!class_exists(\RepackedApplication::class)) {
$application->add(new RepackCommand());
$application->add(new CompileCommand($httpClient, $fs));
}

return $application;
Expand Down
Loading
Loading