Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Completion breaks with multibyte characters #427

Open
arp242 opened this issue Jun 20, 2021 · 3 comments · May be fixed by #428
Open

Completion breaks with multibyte characters #427

arp242 opened this issue Jun 20, 2021 · 3 comments · May be fixed by #428

Comments

@arp242
Copy link
Contributor

arp242 commented Jun 20, 2021 •
edited
Loading

The following file:

package x

import (
    "fmt"
    "net/mail"
)

func t() {
    var m mail.Address
    fmt.Printf("%s → %s\n", m.)
}

Completing the m doesn't work in lsc. I tried VSCode, and it works there, so it seems a problem in vim-lsc rather than gopls.

What it sends to gopls is:

{
  "id": 2,
  "jsonrpc": "2.0",
  "method": "textDocument/completion",
  "params": {
    "position": {
      "character": 29,
      "line": 9
    },
    "textDocument": {
      "uri": "file:///tmp/tgo_20210620_w6ByRcao/main.go"
    }
  }
}

Putting the cursor right on top of the m and starting completion does work. Replacing that → (U+2192) with e.g. > also fixes it. So it seems it should send "character": 27 instead of "character": 29.

Looking at the LSC specification, character offsets are kinda weird:

The offsets are based on a UTF-16 string representation. So a string of the form a𐐀b the character offset of the character a is 0, the character offset of 𐐀 is 1 and the character offset of b is 3 since 𐐀 is represented using two code units in UTF-16. To ensure that both client and server split the string into the same line representation the protocol specifies the following end-of-line sequences: ‘\n’, ‘\r\n’ and ‘\r’.

Microsoft's love for UTF-16 strikes again 🙃

Right now it looks like it just sends the byte offset(?)
@arp242
Copy link
Contributor Author

arp242 commented Jun 20, 2021

Sorry, I forgot to mention I'm using the latest master (4b0fc48)

I don't know what you comparability requirements are for vim-lsc, but Vim somewhat recently added some multibyte-aware functions to make all of this easier, e.g.:

:echo charidx(getline('.'), 29)
27

This requires Vim 8.2.2233 though (Dec 2020), and it'll be quite some time before e.g. Debian people will get this, but you can use e.g. has() to check for it. Older Vims can get the broken behaviour, new Vims the new behaviour.

Also see charcol().

I couldn't really figure out the logic in completion.vim right now or I'd send a patch 😅 But with some luck it should be a comparatively simple fix(?)

natebosch added a commit that referenced this issue Jun 24, 2021
@natebosch
Fix cursor position following multibyte characters
8851e05 
Closes #427

Use `charcol` over `col` where it exists.
Gate the function definition behind the feature check to avoid repeating
it for every call.
@natebosch natebosch linked a pull request Jun 24, 2021 that will close this issue
Open
@natebosch
Copy link
Owner

@arp242 - Thanks for the pointer to charcol!

Could you try the multibyte-position branch and confirm that it solves your issue? From my debugging it seems like it should, but I think the server I'm running against might have a complimentary bug.

@arp242
Copy link
Contributor Author

arp242 commented Jun 24, 2021 •
edited
Loading

Thanks! It seems improved, but it doesn't display all completions:

screenshot_2021-06-24-10-25-08_border

screenshot_2021-06-24-10-25-25_border

I checked the LSP logs, and the request and responses are completely identical for the two examples (x and →), so it seems this part is correct, but somehow something else is still causing problems? 🤔

I tested with a clean vimrc as well.

Logs for completeness sake (but I don't think this is where the problem is):

{
  "id": 2,
  "jsonrpc": "2.0",
  "method": "textDocument/completion",
  "params": {
    "position": {"character": 26, "line": 11},
    "textDocument": {"uri": "file:///tmp/tgo_20210624_hs3UF4yn/main.go"}
  }
}

{
  "id": 2,
  "jsonrpc": "2.0",
  "result": {
    "isIncomplete": true,
    "items": [
      {
        "detail": "string",
        "documentation": "user@domain\n",
        "filterText": "Address",
        "insertTextFormat": 1,
        "kind": 5,
        "label": "Address",
        "labelDetails": {},
        "preselect": true,
        "sortText": "00000",
        "textEdit": {
          "newText": "Address",
          "range": {"end": {"character": 26, "line": 11}, "start": {"character": 26, "line": 11}}
        }
      },
      {
        "detail": "string",
        "documentation": "Proper name; may be empty.\n",
        "filterText": "Name",
        "insertTextFormat": 1,
        "kind": 5,
        "label": "Name",
        "labelDetails": {},
        "sortText": "00001",
        "textEdit": {
          "newText": "Name",
          "range": {"end": {"character": 26, "line": 11}, "start": {"character": 26, "line": 11}}
        }
      },
      {
        "detail": "func() string",
        "documentation": "String formats [..].\n",
        "filterText": "String",
        "insertTextFormat": 1,
        "kind": 2,
        "label": "String",
        "labelDetails": {},
        "sortText": "00002",
        "textEdit": {
          "newText": "String",
          "range": {"end": {"character": 26, "line": 11}, "start": {"character": 26, "line": 11}}
        }
      }
    ]
  }
}

AlxKrispy added a commit to AlxKrispy/vim-lsc that referenced this issue Mar 21, 2022
@AlxKrispy
Squashed commit of the following:
ff58da9 
commit 8851e05
Author: Nate Bosch <[email protected]>
Date:   Wed Jun 23 18:20:09 2021 -0700

    Fix cursor position following multibyte characters

    Closes natebosch#427

    Use `charcol` over `col` where it exists.
    Gate the function definition behind the feature check to avoid repeating
    it for every call.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix cursor position following multibyte characters natebosch/vim-lsc
2 participants
@arp242 @natebosch