Drush 10.3.1 bootstrap exception when using absolute paths in composer.json "extra"

[varenius]

Describe the bug
Installing drupal 8.9.2 using composer (1.10.9 2020-07-16 12:57:00) produces a composer.json file with the relative path "web/" in composer.json "extra" (web-root and installer-paths). If I change this to an absolute path (pointing to the same web-folder) before running "composer install", drush commands (e.g. drush si) fail with "Bootstrap error" as it cannot find the drupal-core files.

To Reproduce

I ran these commands (note that I use an absolute path for composer as I have installed the latest composer version locally on a shared webhost):

mkdir -p /home/goforbun/drupal
cd /home/goforbun/drupal
php -d memory_limit=-1 /home/goforbun/composer/composer.phar create-project drupal/recommended-project:^8.9.2 --no-install newsite
cd /home/goforbun/drupal/newsite
sed -i "s/web\//\/home\/goforbun\/drupal\/newsite\/web\//g" composer.json
php -d memory_limit=-1 /home/goforbun/composer/composer.phar install
php -d memory_limit=-1 /home/goforbun/composer/composer.phar require drush/drush:^10.3.1
vendor/bin/drush si -vvv

The last step "drush si" fails with "Bootstrap Error", see "Actual behaviour" below. The relevant parts of my composer.json are

    "extra": {
        "drupal-scaffold": {
            "locations": {
                "web-root": "/home/goforbun/drupal/newsite/web/"
            }
        },
        "installer-paths": {
            "/home/goforbun/drupal/newsite/web/core": [
                "type:drupal-core"
            ],
            "/home/goforbun/drupal/newsite/web/libraries/{$name}": [
                "type:drupal-library"
            ],
            "/home/goforbun/drupal/newsite/web/modules/contrib/{$name}": [
                "type:drupal-module"
            ],
            "/home/goforbun/drupal/newsite/web/profiles/contrib/{$name}": [
                "type:drupal-profile"
            ],
            "/home/goforbun/drupal/newsite/web/themes/contrib/{$name}": [
                "type:drupal-theme"
            ],
            "drush/Commands/contrib/{$name}": [
                "type:drupal-drush"
            ],
            "/home/goforbun/drupal/newsite/web/modules/custom/{$name}": [
                "type:drupal-custom-module"
            ],
            "/home/goforbun/drupal/newsite/web/themes/custom/{$name}": [
                "type:drupal-custom-theme"
            ]
        },

and "ls /home/goforbun/drupal/newsite/web/core" does show the drupal files, so there are no obvious typos etc. The files are there, but drush doesn't find them.

For comparison, the following works (i.e. "drush si" runs) which modifies the default path to add an extra ../ relative level:

mkdir -p /home/goforbun/drupal
cd /home/goforbun/drupal
php -d memory_limit=-1 /home/goforbun/composer/composer.phar create-project drupal/recommended-project:^8.9.2 --no-install newsite
cd /home/goforbun/drupal/newsite
sed -i "s/web\//..\/newsite\/web\//g" composer.json
php -d memory_limit=-1 /home/goforbun/composer/composer.phar install
php -d memory_limit=-1 /home/goforbun/composer/composer.phar require drush/drush:^10.3.1
vendor/bin/drush si -vvv

Expected behavior
I expect "drush si" to run without errors, as long as the paths given in composer.json point to the correct drupal folder.

Actual behavior
"drush si" fails with Bootstrap error:

[goforbun@premium27 newsite]$ vendor/bin/drush si -vvv
 [preflight] Config paths: /home/goforbun/drupal/newsite/vendor/drush/drush/drush.yml
 [preflight] Alias paths: /home/goforbun/drupal/newsite/drush/sites
 [preflight] Commandfile search paths: /home/goforbun/drupal/newsite/vendor/drush/drush/src
 [debug] Starting bootstrap to root [0.09 sec, 8.08 MB]

In BootstrapHook.php line 32:
                                                                      
  [Exception]                                                         
  Bootstrap failed. Run your command with -vvv for more information.  
                                                                      

Exception trace:
  at /home/goforbun/drupal/newsite/vendor/drush/drush/src/Boot/BootstrapHook.php:32
 Drush\Boot\BootstrapHook->initialize() at /home/goforbun/drupal/newsite/vendor/consolidation/annotated-command/src/Hooks/Dispatchers/InitializeHookDispatcher.php:34
 Consolidation\AnnotatedCommand\Hooks\Dispatchers\InitializeHookDispatcher->callInitializeHook() at /home/goforbun/drupal/newsite/vendor/consolidation/annotated-command/src/Hooks/Dispatchers/InitializeHookDispatcher.php:27
 Consolidation\AnnotatedCommand\Hooks\Dispatchers\InitializeHookDispatcher->initialize() at /home/goforbun/drupal/newsite/vendor/consolidation/annotated-command/src/CommandProcessor.php:145
 Consolidation\AnnotatedCommand\CommandProcessor->initializeHook() at /home/goforbun/drupal/newsite/vendor/consolidation/annotated-command/src/AnnotatedCommand.php:289
 Consolidation\AnnotatedCommand\AnnotatedCommand->initialize() at /home/goforbun/drupal/newsite/vendor/symfony/console/Command/Command.php:221
 Symfony\Component\Console\Command\Command->run() at /home/goforbun/drupal/newsite/vendor/symfony/console/Application.php:1005
 Symfony\Component\Console\Application->doRunCommand() at /home/goforbun/drupal/newsite/vendor/symfony/console/Application.php:255
 Symfony\Component\Console\Application->doRun() at /home/goforbun/drupal/newsite/vendor/symfony/console/Application.php:148
 Symfony\Component\Console\Application->run() at /home/goforbun/drupal/newsite/vendor/drush/drush/src/Runtime/Runtime.php:118
 Drush\Runtime\Runtime->doRun() at /home/goforbun/drupal/newsite/vendor/drush/drush/src/Runtime/Runtime.php:49
 Drush\Runtime\Runtime->run() at /home/goforbun/drupal/newsite/vendor/drush/drush/drush.php:72
 require() at /home/goforbun/drupal/newsite/vendor/drush/drush/drush:4

Workaround
No workaround known to me. I need to specify a non-default path for my "web" folder.

System Configuration

Q	A
Drush version?	10.3.1
Drupal version?	8.9.2
PHP version	7.3.20
OS?	Linux (on shared host)

Additional information
My ultimate goal here is to have the folder with compose.json and vendor outside my webroot:
/home/account/drupal/newsite - outside webroot, containing composer.json, composer.lock and vendor folder. Not "web".
/home/account/public_html/newsite - for the drupal files, i.e. the "web/" created by composer install with default json.

[weitzman]

The project that finds drupal root is https://github.com/webflo/drupal-finder so this report actually belongs there, not in drush.

My ultimate goal here is to have the folder with compose.json and vendor outside my webroot:

Thats how drupal/recommended-project works by default. Seems to me that you could use it as normal and then symlink web to /home/account/public_html/newsite as an additional step.

[varenius]

Thats how drupal/recommended-project works by default. Seems to me that you could use it as normal and then symlink web to /home/account/public_html/newsite as an additional step.

Nope, I tried that - symbolic links do not work. It doesn't find my drupal installation.

The project that finds drupal root is https://github.com/webflo/drupal-finder so this report actually belongs there, not in drush.

Aha, thank you for this information. I hereby close this ticket and I have instead opened one at the drupal-finder issue-queue at webflo/drupal-finder#51.