Extensions cleanup

[Frzk]

Following the comments made by @Soulou in #286, I suggest that we get rid of some useless code related to extensions.

Here is a summary that I completed with what I found in both support/ext-internal and lib (!?)

Extension	Source	Where	Decision	Comment
apcu	PECL	support/ext	✔️ KEEP (bundled)
oci8	PECL	lib	✔️ KEEP (prebuilt)	Req. external stuff. Hence keeping.
blackfire	External	support/ext	✔️ KEEP (prebuilt)
memcached	External	support/ext	✔️ KEEP (prebuilt)
mongodb	External	support/ext	✔️ KEEP (prebuilt)
redis	External	support/ext	✔️ KEEP (prebuilt)	Also available on PECL.
calendar	PHP	support/ext-internal	✔️ KEEP (prebuilt)
ftp	PHP	support/ext-internal	✔️ KEEP (prebuilt)
gettext	PHP	support/ext-internal	✔️ KEEP (prebuilt)
gmp	PHP	support/ext-internal	✔️ KEEP (prebuilt)
imap	PHP	support/ext-internal	✔️ KEEP (prebuilt)
sodium	PHP	support/ext-internal	✔️ KEEP (prebuilt)
tidy	PHP	support/ext-internal	✔️ KEEP (prebuilt)
datadog	External	lib	✔️ KEEP (prebuilt)
newrelic	External	lib	✔️ KEEP (prebuilt)
apfd	PECL	support/ext	❌ DROP
ds	PECL	support/ext	❌ DROP
imagick	PECL	support/ext	❌ DROP
mcrypt	PECL	support/ext	❌ DROP
amqp	PECL	support/ext	❓ TBD	Req. librabbitmq-dev
event	PECL	support/ext	❓ TBD
igbinary	PECL	support/ext	❓ TBD
lua	PECL	support/ext	❓ TBD	Req. lua
scoutapm	PECL/Zend	lib	❓ TBD

• ✔️ KEEP (bundled): extensions that we want to build, ship and enable by default.
• ✔️ KEEP (prebuilt): extension that we want to build and ship, but that we don't want to enable by default.
• ❌ DROP: extensions that we don't want to care about anymore. They can be built via the php-buildpack (or multi-buildpack + apt-buildpack, as suggested in our public documentation).
• ❓ TBD: Is there any good reason to mark them as KEEP (prebuilt) instead of DROP?
  • amqp: Seems to work well with a multibuildpack (using the APT buildpack to install librabbitmq-dev). I think we can DROP.
  • event: Seems to work well out of the box. I think we can DROP.
  • igbinary: Seems to work well out of the box. I think we can DROP.
  • scoutapm: Does not work out of the box because it's a Zend extension, which means it must be loaded via zend_extension=scoutapm.so. Supporting Zend extensions means we must have specific code to handle them. It shouldn't be hard to maintain, so I think we can mark it as KEEP (prebuilt).
  • lua: Can't make it work for now (configure: error: Please reinstall the lua distribution - lua.h should be in <lua-dir>/include/). It seems we stopped supporting it a while ago (4396536).

---

• External extensions have dependencies or requirements that are hard to factorize, hence marking them all as KEEP (prebuilt).
• Some External extensions (namely datadog and newrelic are enabled and configured by using a .extra.paas.<extension_name> key in composer.json. Should this be expanded to other extensions as well? Should we keep this?
• PHP extensions (Source is set to PHP in the table above) are included in PHP. They need to be enabled with a specific configure flag. This shouldn't be hard to support, hence marking them all as KEEP (prebuilt).
• Some PHP extensions (gmp, sodium, tidy,...) still have dependencies. It's unclear to me whether we should keep downloading + compiling + packaging these dependencies or if we could apt install them.
• Some extensions marked as DROP (or that we don't support at all) have known dependencies and configure options (for example, the yaml extension requires libyaml-dev and needs to be compiled with the --with-libyaml-dir= flag):
  • Do we want to publicly document this somewhere?
  • Do we want to add code to handle this somewhere?

---

I consequently suggest the following plan:

Plan

Preview Give feedback

1. 0. Decide what we do with the extensions marked as TBD.
2. 1. Remove the support/ext-deprecated directory and its content.
3. 2. Remove all code related to the extensions marked as DROP (except if we want to keep minimal support for things that we know - see question above).
4. 3. Put code related to building and packaging the extensions marked as KEEP (bundled) in a directory called support/bundled. Each file should be named package_<extension_name>.sh.
5. 4. Put code related to building and packaging the extensions marked as KEEP (prebuilt) in a directory called support/prebuilt. Each file should be named package_<extension_name>.sh.
6. 5. Move some code used to package PHP extensions dependencies into support/prebuilt/package_<dependency_name>.sh scripts. Document why we need this.
7. 6. Remove code related to enabling an extension in favor of a generic, dedicated function (to be written, the process is always the same: download the package (if necessary), copy the .so file, enable extension by creating /app/vendor/php/etc/conf.d/${extension_name}.ini).
8. 7. Remove the support/ext and support/ext-internal directories after their content has been dispatched between the support/{bundled,prebuilt} directories or replaced by the function.
9. 8. Update our public documentation to list the bundled and prebuilt extensions.
10. 9. Update out internal documentation with procedures describing how to update everything (PHP, bundled and prebuilt extensions, dependencies, ...).
11. 10. Figure out what to do with code related to Scout APM, Datadog, New Relic and oci8... that crawls here and there.
12. 11. Figure out how to organize the support/get_* scripts.

@EtienneM, @Soulou, (others are welcome too) could you please let me know:

• If we agree on the table above ;
• What you think about the extensions marked as TBD ;
• What you think about the PHP extensions that have dependencies: should we keep it as they are, should we try to use apt, should we include the deps in our stacks? ;
• What you think about keeping some sort of minimal support for extensions that we don't want to build/package/ship ;
• What you think about the Plan.

Thanks a billion.
🙇‍♂️ 🙏

[EtienneM]

Thanks for this great work! 👏

What you think about the PHP extensions that have dependencies: should we keep it as they are, should we try to use apt, should we include the deps in our stacks? ;

Yes I would keep them as-is. The problem of the workflow with multibuildpacks and the apt buildpack is that it's a bit complicated for newcomers. Hence it's a bit of an entrance barrier for newcomers.

What you think about keeping some sort of minimal support for extensions that we don't want to build/package/ship ;

Even though I agree with the idea, I am kind of reluctant to drop some extensions as it would break an existing workflow for some customers. Would it be possible to gather the usage of some of these extensions to know if they are actually used or if we can drop the support with no impact?

What you think about the extensions marked as TBD ;

For those working well out of the box, I agree to simply drop them.

For those requiring the multic-buildpacks, cf. my points raised above: multibuildpacks is an entrance barrier + it's breaking currently working workflow.

Yes lua extension is not supported in PHP 8.1 but PHP 8.0 is still supported for a couple of months (https://www.php.net/supported-versions.php) so I think we should keep it until we drop support for PHP 8.0.