Drush 10.3.1 bootstrap exception when using absolute paths in composer.json "extra"

[varenius]

[Moved here from https://github.com/drush-ops/drush/issues/4488]

Describe the bug
Installing drupal 8.9.2 using composer (1.10.9 2020-07-16 12:57:00) produces a composer.json file with the relative path "web/" in composer.json "extra" (web-root and installer-paths). If I change this to an absolute path (pointing to the same web-folder) before running "composer install", drush commands (e.g. drush si) fail with "Bootstrap error" as it cannot find the drupal-core files.

To Reproduce

I ran these commands (note that I use an absolute path for composer as I have installed the latest composer version locally on a shared webhost):

mkdir -p /home/goforbun/drupal
cd /home/goforbun/drupal
php -d memory_limit=-1 /home/goforbun/composer/composer.phar create-project drupal/recommended-project:^8.9.2 --no-install newsite
cd /home/goforbun/drupal/newsite
sed -i "s/web\//\/home\/goforbun\/drupal\/newsite\/web\//g" composer.json
php -d memory_limit=-1 /home/goforbun/composer/composer.phar install
php -d memory_limit=-1 /home/goforbun/composer/composer.phar require drush/drush:^10.3.1
vendor/bin/drush si -vvv

The last step "drush si" fails with "Bootstrap Error", see "Actual behaviour" below. The relevant parts of my composer.json are

    "extra": {
        "drupal-scaffold": {
            "locations": {
                "web-root": "/home/goforbun/drupal/newsite/web/"
            }
        },
        "installer-paths": {
            "/home/goforbun/drupal/newsite/web/core": [
                "type:drupal-core"
            ],
            "/home/goforbun/drupal/newsite/web/libraries/{$name}": [
                "type:drupal-library"
            ],
            "/home/goforbun/drupal/newsite/web/modules/contrib/{$name}": [
                "type:drupal-module"
            ],
            "/home/goforbun/drupal/newsite/web/profiles/contrib/{$name}": [
                "type:drupal-profile"
            ],
            "/home/goforbun/drupal/newsite/web/themes/contrib/{$name}": [
                "type:drupal-theme"
            ],
            "drush/Commands/contrib/{$name}": [
                "type:drupal-drush"
            ],
            "/home/goforbun/drupal/newsite/web/modules/custom/{$name}": [
                "type:drupal-custom-module"
            ],
            "/home/goforbun/drupal/newsite/web/themes/custom/{$name}": [
                "type:drupal-custom-theme"
            ]
        },

and "ls /home/goforbun/drupal/newsite/web/core" does show the drupal files, so there are no obvious typos etc. The files are there, but drush doesn't find them.

For comparison, the following works (i.e. "drush si" runs) which modifies the default path to add an extra ../ relative level:

mkdir -p /home/goforbun/drupal
cd /home/goforbun/drupal
php -d memory_limit=-1 /home/goforbun/composer/composer.phar create-project drupal/recommended-project:^8.9.2 --no-install newsite
cd /home/goforbun/drupal/newsite
sed -i "s/web\//..\/newsite\/web\//g" composer.json
php -d memory_limit=-1 /home/goforbun/composer/composer.phar install
php -d memory_limit=-1 /home/goforbun/composer/composer.phar require drush/drush:^10.3.1
vendor/bin/drush si -vvv

Expected behavior
I expect "drush si" to run without errors, as long as the paths given in composer.json point to the correct drupal folder.

Actual behavior
"drush si" fails with Bootstrap error:

[goforbun@premium27 newsite]$ vendor/bin/drush si -vvv
 [preflight] Config paths: /home/goforbun/drupal/newsite/vendor/drush/drush/drush.yml
 [preflight] Alias paths: /home/goforbun/drupal/newsite/drush/sites
 [preflight] Commandfile search paths: /home/goforbun/drupal/newsite/vendor/drush/drush/src
 [debug] Starting bootstrap to root [0.09 sec, 8.08 MB]

In BootstrapHook.php line 32:
                                                                      
  [Exception]                                                         
  Bootstrap failed. Run your command with -vvv for more information.  
                                                                      

Exception trace:
  at /home/goforbun/drupal/newsite/vendor/drush/drush/src/Boot/BootstrapHook.php:32
 Drush\Boot\BootstrapHook->initialize() at /home/goforbun/drupal/newsite/vendor/consolidation/annotated-command/src/Hooks/Dispatchers/InitializeHookDispatcher.php:34
 Consolidation\AnnotatedCommand\Hooks\Dispatchers\InitializeHookDispatcher->callInitializeHook() at /home/goforbun/drupal/newsite/vendor/consolidation/annotated-command/src/Hooks/Dispatchers/InitializeHookDispatcher.php:27
 Consolidation\AnnotatedCommand\Hooks\Dispatchers\InitializeHookDispatcher->initialize() at /home/goforbun/drupal/newsite/vendor/consolidation/annotated-command/src/CommandProcessor.php:145
 Consolidation\AnnotatedCommand\CommandProcessor->initializeHook() at /home/goforbun/drupal/newsite/vendor/consolidation/annotated-command/src/AnnotatedCommand.php:289
 Consolidation\AnnotatedCommand\AnnotatedCommand->initialize() at /home/goforbun/drupal/newsite/vendor/symfony/console/Command/Command.php:221
 Symfony\Component\Console\Command\Command->run() at /home/goforbun/drupal/newsite/vendor/symfony/console/Application.php:1005
 Symfony\Component\Console\Application->doRunCommand() at /home/goforbun/drupal/newsite/vendor/symfony/console/Application.php:255
 Symfony\Component\Console\Application->doRun() at /home/goforbun/drupal/newsite/vendor/symfony/console/Application.php:148
 Symfony\Component\Console\Application->run() at /home/goforbun/drupal/newsite/vendor/drush/drush/src/Runtime/Runtime.php:118
 Drush\Runtime\Runtime->doRun() at /home/goforbun/drupal/newsite/vendor/drush/drush/src/Runtime/Runtime.php:49
 Drush\Runtime\Runtime->run() at /home/goforbun/drupal/newsite/vendor/drush/drush/drush.php:72
 require() at /home/goforbun/drupal/newsite/vendor/drush/drush/drush:4

Workaround
No workaround known to me. I need to specify a non-default path for my "web" folder.

System Configuration

Q	A
Drush version?	10.3.1
Drupal version?	8.9.2
PHP version	7.3.20
OS?	Linux (on shared host)

Additional information
My ultimate goal here is to have the folder with compose.json and vendor outside my webroot:
/home/account/drupal/newsite - outside webroot, containing composer.json, composer.lock and vendor folder. Not "web".
/home/account/public_html/newsite - for the drupal files, i.e. the "web/" created by composer install with default json.

I also tried using the default drupal-recommended setup, placing the files in the subfolder "web", then moving that folder and replacing it with a symbolic link. This also doesn't work, it seems drupal-finder refuses to follow the symbolic link. That would be an OK behaviour to me, but I think it should accept an absolute path, no matter where it is on the system.