What args are supported by extensionHost launch type?

[vaclavHala]

I'm trying to launch VSCode for running tests in from my development VSCode with these args:

{
  "name": "Extension Tests",
  "type": "extensionHost",
  "request": "launch",
  "args": [
    "${workspaceFolder}/.vscode-test/ws/test.code-workspace",
    "--verbose",
    "--extensions-dir=${workspaceFolder}/.vscode-test/extensions",
    "--user-data-dir=${workspaceFolder}/.vscode-test/user-data",
    "--extensionDevelopmentPath=${workspaceFolder}/out-src",
    "--extensionTestsPath=${workspaceFolder}/out-test/test-vscode-entrypoint.js"
  ],
  ...

but --verbose, --extensions-dir and --user-data-dir seem to be ignored (the exact same setup launched via @vscode/test-electron works without problems).

Here is more context for why I want to pass --extensions-dir and --user-data-dir.

The --verbose I need because in some of our tests we need to listen for File Watcher events like this

[File Watcher (node.js)] Request to start watching: ...
[File Watcher (node.js)] Started watching: ...
[File Watcher (node.js)] >> normalized [CHANGED] ...

to have reliable tests which use the watcher API without using sleeps or similar
(there may be better way to go about that use case, please let me know if there is, but that is beside the point of this question).

So the question is - what is the supported subset of options (I'd also like to pass in runtimeExecutable but that is also ignored, see the linked issue) and args that can be used when launching the extensionHost type debug session and is this documented somewhere?

If there is no documentation for this, please point me to source where I can find this out for myself.
I tried looking a bit but could not find anything yet (I think the place to start is extensionHostStarter.ts
but could not find how/where the options from launch.json are actually processed). Thanks

[abrilurena]

Hi, @vaclavHala! i hope you're doing great. I'm adding @isidorn he might be able to help here :D

[sandy081]

I doubt following arguments work because we cannot launch VS Code from a different user data directory or extensions directory

"--extensions-dir=${workspaceFolder}/.vscode-test/extensions",
"--user-data-dir=${workspaceFolder}/.vscode-test/user-data",

@bpasero please confrim.

I would recommend to use --profile arg if you want to launch tests in a clean environment - https://code.visualstudio.com/updates/v1_72#_extension-debugging-in-a-clean-environment

[vaclavHala]

@sandy081 yes I'm aware of profiles, and it is what we are going with (in tests launched from dev vscode) for the time being since the --*-dir args don't work, but that approach has several downsides - hence we tried to use the dir approach instead:

• The profile has to be registered manually into the vscode where you want to use it upfront.
  • This make it impossible (or very cumbersome) to use in CI where we download latest stable version of vscode every time (using @vscode/test-electron).
    • which in turn forces me to maintain two "clean environment" mechanisms - one for CI (using the --*dir approach) and another for tests launched during dev (using --profile)
  • It also means new developers checking out the repo for the first time have to perform the extra step of setting up profile for all the extensions they want to test
  • same thing every time anyone checks out git branch where the profile changed...
• when the profile is missing once the tests start there is no reliable way to determine if the profile existed and the environment is set up correctly, because new empty profile is created silently when reference to unknown profile is given - again this makes for bad dev experience.
  • This causes difficult to diagnose problems, especially for devs who are not aware of how exactly profiles work and just expect stuff to work out of the box (i.e. without additional manual steps after git checkout)
  • this point could be fixed if it were possible to give profile by path to the profile file rather than symbolic name, but I found no way to do that
• the profiles have to be maintained manually by clicking in GUI and exporting, as opposed to just scanning package.json of the tested extension for dependencies / having simple list of ids in test bootstrap (used as pre task when running from development vscode) that gets installed automatically from code

Don't get me wrong, it is possible to use --profile to run the tests, it just makes for a not so great experience.

If some of the problems I outlined can actually be worked around somehow and I'm just not aware please let me know how to improve

[sandy081]

If it is a throw away profile, have you used --profile-temp option?

[vaclavHala]

We used --profile-temp as long as we had no extensionDependencies and that worked fine, but now because we have dependencies that need to be installed upfront that is no longer an option. I tried installing the extensions by command in the test entrypoint, then reloading window to activate our extension which depends on them, but the window reload broke the debug session.

Are you saying this approach should work?

• use --profile-temp,
• once launched install the dependencies from inside the debug session using API (I think command workbench.extensions.installExtension is what I want)
• reload window to activate our extension (command workbench.action.reloadWindow)
• await until the reload command finishes, then run tests in the session

If yes I will try it again, maybe there was something else which caused the problems and I abandoned this approach to soon.

[vaclavHala]

Oh I see what is going on, I expected the execution to continue in the calling vscode after the reload command

// in my entrypoint/test code
await vscode.commands.executeCommand("workbench.action.reloadWindow")
// I expected execution to continue here

So when it did not I thought it completely kicked out the debug session and it can't attach back.
But it actually starts the entrypoint from scratch in the new window, so I can check at the start of entrypoint if I want to install and reload, or run in current session.

I did a quick experiment now trying to put basically this code at the very start of the test entrypoint

export async function run(): Promise<void> {
    const myDependencies = extractMyDependencies()
    if(haveRequiredDependencies(vscode.extensions.all, myDependencies)){
        console.log("run")
    } else {
        console.log("install")
        for(const dep of myDependencies){
            await vscode.commands.executeCommand("workbench.extensions.installExtension", dep)
        }        
        await vscode.commands.executeCommand("workbench.action.reloadWindow")
        // never gets here
    }

    // actual entrypoint start

and it seems to work.

Let me do some testing with this and get back to you, this looks promising though

[theReynald]

Hi @vaclavHala.

Sounds like you're good for now so I'm closing this issue but don't hesitate to reopen if more questions come up.

/r

[nagilson]

It would be pretty nice to have a document that answers the question of this issue: What args are supported by extensionHost launch type?

When looking into it, this post is now the first result that comes up in search engines. After more digging the only doc I could find is https://code.visualstudio.com/api/working-with-extensions/testing-extension#debugging-the-tests. Maybe there's more out there already. But if @vaclavHala hadn't read the code, I would have no idea there was even such a thing as a --verbose option ;)

[mforkel]

+1 for a documentation of the args supported by the extensionHost launch type

[James4Ever0]

I was wondering how to add --enable-source-maps to extension host.

microsoft/vscode#145473 (comment)