Dispatch to remote host per Drush site aliases

[weitzman]

I would love to see this program handle remote dispatching of requests like drush @foo status. The goal is to simplify Drush such that once it receives a request, it knows that it will be operating locally (at least initially - the command itself can remote dispatch if needed). We will need to extract the alias parts of Drush into a new project so this shim can depend on it. This was barely started at https://github.com/drush-ops/drush-aliases. Would be good to spec this out some more.

ping @webflo @greg-1-anderson

[greg-1-anderson]

I agree that this would be a good thing to do.

[weitzman]

I think the equivalent code for Drupal Console is https://github.com/hechoendrupal/drupal-console-launcher/blob/master/src/Utils/Remote.php. They are doing SSH with PHPSecLib instead of ssh in the shell. Dunno what the benefits and risks of that might be.

[thomscode]

Benefits of using an SSH library built into PHP instead of relying on the host to have SSH, is that you aren't relying on SSH to be installed (like in a Windows environment). For those running macOS or Linux it would have less benefit. As I haven't done this before I'm not sure how they would implement no password authentication. Currently this can be done with Public/Private keys or GSSAPI, via an SSH Config file. Hopefully this either provides a similar method of doing the same thing or continues to work with an SSH config file.

So long as you require a sufficient version of PHP and write secure code, I don't see there being that many (if any) risks. The only one I can think of off the top of my head would be if you provide a way to store an SSH password for authentication in cleartext (or some other easily decrypted format like Base64)

[thomscode]

@weitzman, this would theoretically allow a user to have globally installed aliases and execute drush commands globally using those aliases as if there was a globally installed drush (with the exception that it's actually using the locally installed drush after it determines the root from the alias)?

If that's too long to follow I can reword the question.

[greg-1-anderson]

@thomscode Yes, that's right. The site alias implementation in Drush 9 is fairly well factored-out now, and could easily become an external library. In order to really support what you're asking here, we'd also need to factor out backend invoke, so that remote aliases could also be used. That would be more work.

[thomscode]

If backend invoke is going to be refactored in the "near-ish" future per Drush-Ops/Drush/Pull/2932, making it an external library could become part of that re-write.

Also the aforementioned Drush-Ops/Drush-Aliases would be a good place to put the Drush 9 site alias implementation as an external library, yes?

[greg-1-anderson]

There's a whole pile of things related to Drupal 8.4.0 + Drush 8 / Drush 9 that are higher priority than the backend invoke rewrite, but yes, we'd like to get to that soon.

I just moved drush-ops/drush-aliases to consolidation/site-aliases. If you'd like to put a PR in that project updating the documentation and inserting the implementation from Drush 9 site-aliases, that would be appreciated. The tests from the isolation folder would probably also move over as well.

[thomscode]

Thanks, I'll start looking at that.

[weitzman]

Some folks will have their needs met by using this project's new Fallback feature.

[weitzman]

Current status:

• Site aliases are now factored out.
• Backend.inc factoring has not started.

[greg-1-anderson]

I did make a placeholder project for backend.inc, which I hope to start factoring out soon:

https://github.com/consolidation/backend

[weitzman]

Backend invoke got factored out a while into into https://github.com/consolidation/site-process. So this is more doable now.

[Ambient-Impact]

The lack of global aliases is the one last thing holding me back from using this on my remote servers. I still have Drush 8.1.x installed globally on those just so I can interact with the sites without having to change directories. It hands off to the locally installed Drush (10.x) and Drupal (9.3.x) just fine, but I'd be happy to get rid of it.

Is there anything I can do to help out with this?

[greg-1-anderson]

We were recently discussing this. The motivation is to remove the ability to run Drush 12 as a global install. @weitzman was concerned that we would be increasing the complexity of the launcher without decreasing the complexity of Drush by a similar amount. It is on my todo list to make a PR in Drush demonstrating what the simplified preflight in Drush would look like.

Recently, though, I got to thinking, why not continue to use Drush as the global launcher? Is it size on disk? That usually doesn't matter too much, but if it did, we could factor out the code that bootstraps Drupal, and all of the site-specific commands into a separate package. Or, alternatively, build the launcher with a subtree split of Drush. I don't know if it would be worth the complexity, but it's something we could do.

Does the Drush launcher exist because using Drush as the global launcher is a lot slower? The Drush launcher includes the autoloader for Drush and redispatches to it without using exec. In general, I am -1 on this technique. (See "The trouble with two autoloaders" and "Fixing the Composer global commands.) The Drush launcher makes this technique tenable by extremely limiting the number of dependencies that it uses; however, there is some small overlap between the Drush launcher dependencies and the Drush / Drupal dependencies. This could cause a problem someday. Is the reduced latency worth it?

Adding the ability to parse aliases and redispatch with them would definitely add a lot more dependency overlap, making the quick redispatch without exec impossible. This is the other reason that I wondered if it would be better to just use Drush itself as its own global launcher. This is what I used to do, although now all of my remote sites are on Pantheon, so I use Terminus as my global launcher.

@Ambient-Impact: Is there any reason that you can't or don't want to use Drush 11 as your global launcher? Or, if there is a reason you wish to stick with the Drush 8 alias file format, is there a reason you have not upgraded to the most recent version of Drush 8?

[webflo]

The basic idea for Drush Launcher was to build a tool that can handle any Drush version, i.e. Drush 8 for Drupal 7 and Drush 10 for Drupal 9 (and all versions in between). It should ease migration from global Drush to site-local Drush.

I don't know if Drush 10 still has the ability to bootstrap Drush 8. I think it failed in the past because global options in different Drush versions have different names and the global Drush failed to bootstrap even though it was supposed to just pass all the parameters.

I think the latency didn't matter to me. Direct implementation with php was easier. Because then the same php binary is used, with the same configuration, and also the xdebug integration (composer/xdebug-handler) was very easy to implement.

---

The two autoloaders problem can be further improved with PHAR code isolation. See https://github.com/box-project/box/blob/master/doc/code-isolation.md#phar-code-isolation this is build with PHP-Scoper.

[Ambient-Impact]

@greg-1-anderson I was not aware that Drush 9+ could be used as a global install at all - I just assumed that had already been removed. 😅

I could arguably simplify things on my end by removing the global Drush I'm using and just set up bash aliases for the Drush executable in each install. I tend to have staging and live/production copies of sites running under the same user so it's easy to deploy to staging, make sure everything is working fine, and then sync it to live if so. Might be a good time to revisit that I suppose.

[greg-1-anderson]

Because then the same php binary is used, with the same configuration, and also the xdebug integration (composer/xdebug-handler) was very easy to implement.

Yeah, I understand. In Drush 8 and earlier, we took pains to maintain these settings whenever Drush called through to Drush. Some of that code might still exist in Drush core; however, the base expectation in Drush 9+ is that the user should make sure that the PHP that they want to use is first in the $PATH, and has all of its config.