spec: Text Selection and Key Processing

[rc-swag]

Text Selection, Key processing and Keyman Core

Note: ⁞ here is used to delineate text selection throughout this spec:

• One ⁞ indicates the text caret without a selection; or
• Text between two ⁞ is selected.

For 'legacy' apps, where we do not have data about text selection or context prior to the selected text, none of this information applies. This information is only about 'modern' apps which supply context (e.g. on Windows, TSF-aware apps).

Related issues

• #8827 bug(linux): Backspace doesn't work properly with text selection
• ✔ #5853 bug(web): Highlight and replace/delete is not working
• ✔ #6177 bug: iOS and Android Text Select weird behavior
• ✔ #6261 bug(android): selection is not deleted when typing, and existing text becomes undeletable
• #7161 meta(android): team review of Android UX
• #7168 feat(android): when selecting text, the keyboard has no context and thus gives Initial case, usually wrongly
• #7865 bug(iOS): selecting text and then backspacing also deletes the character before the selection
• #7866 bug(web): text selection + typing interaction issues
  • Comes to a similar conclusion re: selected text -> backspace
  • (#7866 [comment])
• #7870 bug(windows): backspace with selected text deletes an extra character in TSF only

Related pull requests

• ✖ PR#5854 fix(web): Allow touch-alias elements to clear text selection
• ✖ PR#5922 fix(android/engine): Don't pass selected text to KeymanWeb
• ✔ PR#6272 fix(web): properly handle selection
• ✔ PR#6274 fix(android): properly handle selection
• ✔ PR#6280 fix(ios): properly handle selection
• ✔ PR#8462 feat(common/web): mocks may now model selected text
• ✔ PR#8611 fix(android/engine): Fix resetContext calls when selection changes

Introduction

When a user selects text then presses a key; either a Backspace or a Character Key we need to define the expect result. Looking at a variety of scenarios and the current related issues makes it clear that we could:

 a) over complicate the expected behaviour unnecessarily. 

 b) have many edge cases, touch, start of sentence etc

This specification will describe the way Keyman handles processing when a section of text is selected and then a key is pressed. Followed by some example scenarios.

Specification

1. Backspace with a selection shall not trigger keystroke rules
   1. It shall trigger newcontext because a text-changing event has occurred which is outside the engines control. (as the backspace will be passed to the app so it can delete text as required, e.g. a selection across multiple fields or a multi-selection)
2. For character-generating keystrokes with selection. The selection shall be masked from the context, and apply rules to the resulting context (see example 2).
   1. Note: implementation in Engine means we may need to do output a backspace to delete the selection before applying actions. (For legacy apps)
   2. For 'context-aware' apps, we'll still need to delete the selection before applying actions.

Applied Examples

• How do we treat context when we have a selection?

  • Example: ⁞^⁞ in doc, selection as indicated.

    Typing A result: A⁞.

  • Example: ^⁞X⁞ in doc.

    Typing A result: Â⁞

  • Example: ^⁞X the quick brown fox⁞ in doc.

    Typing A give us Â⁞

  • Example: ^⁞X the quick brown fox⁞ in doc.

    Typing Backspace, then A, result: Â⁞

  • Example: ^⁞X the quick brown fox⁞A in doc.

    Typing Backspace give us ^⁞A

Additional questions and answers

• What shift state should we give the keyboard on touch keyboards when we have a selection?

  • Hello. ⁞world⁞, note that 'world' is lower-case, but a sentence-aware keyboard would expect to start a new sentence due to the full stop prior. So, with a selection, should the touch keyboard go to shift layer to match 'start of sentence'?
  • This would trigger start-of-sentence. Doing the text selection would trigger begin newcontext, and be given the context Hello. Thus, a sentence-aware keyboard would set the touch keyboard to the Shift layer.

• What should Backspace do when there is a selection?

  • It should always delete the selection and not run the keyboard rule processing; in effect it has the same behavior as pressing Del (forward-delete).

Future solution not covered by this specification -- &hasSelection

• Suggestion for the future: add a &hasSelection system store. And &intent (text,number,etc) would be nice to throw in.

• Until we implement &hasSelection, the answer is to apply keyboard rules (thus, the Example 2 above), except for Backspace, which behaves the same as Delete.

• Adding the &hasSelection flag allows keyboard authors to override the default, and will allow them to process Backspace or Delete when there is a selection, and provide a sensible result. This will require updating the keyboard &version to vX.Y (the version where we introduce this feature).

• We will need to work through the edge cases carefully here when we implement this.

Reference

• https://docs.google.com/document/d/1qDmJHInvyYm-7apAzzvkwC2BOHf8DUOaPI_sPSxmeg4/edit
• Test matrix

[ermshiperete]

Text Selection Tests

See the Google doc for a test matrix for the different platforms.

(Selection and insertion pointer marked with ⁞ throughout this document)

General test scenarios

Run the following tests with a keyboard that has the rule '^' + [K_A] > 'Â', e.g. text_selection_tests_keyboard or sil_euro_latin:

1. (T01) : ^⁞X the quick brown fox⁞A in doc. Typing BKSP.
   Expected result: ^⁞A
2. (T02) ^⁞X the quick brown fox⁞ in doc. Typing BKSP, then Shift+A.
   Expected result: Â⁞
3. (T03) ⁞^⁞ in doc, selection as indicated, typing Shift+A.
   Expected result: A⁞
4. (T04) ^⁞X⁞ in doc. Typing Shift+A.
   Expected result: Â⁞
5. (T05) ^⁞X the quick brown fox⁞ in doc. Typing Shift+A.
   Expected result: Â⁞

Run the following tests with a keyboard that has the rule '^' + [K_BKSP] > 'foo', e.g. text_selection_tests_keyboard:

1. (T06) ^⁞X⁞ in doc. Typing BKSP.
   Expected result: ^⁞
2. (T07) ^⁞X⁞ in doc. Typing BKSP, BKSP.
   Expected result: foo⁞

The following tests can be run with any keyboard, e.g. text_selection_tests_keyboard:

1. (T08) ^⁞X⁞ in doc. Pressing and releasing SHIFT key.
   Expected result: ^⁞X⁞ (no change). On Touch keyboard switch to Shift
   layer. (cf bug(web): pressing layer switch key with selected text would delete the selection #7866)
2. (T09) a⁞bc de⁞ in doc with selection starting at b. Typing BKSP.
   Expected result: a⁞. (cf bug(linux): Backspace doesn't work properly with text selection #8827)
3. (T10) a⁞bc de⁞ in doc with selection starting at e. Typing BKSP.
   Expected result: a⁞. (cf bug(linux): Backspace doesn't work properly with text selection #8827)
4. (T11) abc ⁞de⁞ fg in doc. Typing BKSP.
   Expected result: abc ⁞ fg. (cf bug(iOS): selecting text and then backspacing also deletes the character before the selection #7865)
5. (T12) a→⁞ in doc (i.e. A followed by TAB. With text_selection_tests_keyboard you can press T to get the TAB character.).
   Typing BKSP.
   Expected result: a⁞. (cf bug(linux): Wrong handling of backspace after tab #9970)
6. (T13) Use Firefox with https://onlinenotepad.org/notepad. abc⁞ in doc. Typing BKSP.
   Expected result: ab⁞ (cf bug(linux): Wrong backspace handling in Firefox #9971)

Run the following tests with the text_selection_tests_keyboard:

1. (T14) Type A followed by [K_BKQUOTE] and B and C.
   Press BKSP twice.
   Expected result: ok1
2. (T15) Type X, A followed by [K_BKQUOTE] and B. Select B (Xa⁞b⁞).
   Press BKSP twice.
   Expected result: X⁞
3. (T16) Type X, A followed by [K_BKQUOTE] and B. Select B (Xa⁞b⁞).
   Press BKSP then O.
   Expected result: Xao⁞

(Desktop only) Run the following test with the EMUFI keyboard and Ge'ez Manuscript Zemen font (You might have to save the result as UTF-16 and then inspect in a hex editor, if the current editor doesn't show the correct font):

1. (T17) Type 1 followed by TAB key, then type 1.
   Expected result: (U+1369 U+FE01) (cf bug(mac): EMUFI Ge'ez Manuscript Incorrect Symbol Replacement #9559)

Start-of-sentence tests

Run these tests with a keyboard with start-of-sentence rules, e.g. English - EuroLatin (SIL).

1. (T51) Hello ⁞world⁞ in doc.
   Expected result: Shift layer should NOT be active (cf feat(android): when selecting text, the keyboard has no context and thus gives Initial case, usually wrongly #7168)
2. (T52) Hello. ⁞world⁞ in doc.
   Expected result: Shift layer should be active (cf feat(android): when selecting text, the keyboard has no context and thus gives Initial case, usually wrongly #7168)
3. (T53) Hello. ⁞world⁞ in doc. Typing BKSP.
   Expected result: Hello. ⁞, shift layer should be active
4. (T54) Hello. ⁞world⁞ in doc. Pressing A key.
   Expected result: Hello. A⁞, default layer active

Predictive text tests

With English - EuroLatin (SIL) keyboard:

1. (T61) Hello ⁞world⁞ in doc.
   Expected results: lowercase predictions (cf feat(android): when selecting text, the keyboard has no context and thus gives Initial case, usually wrongly #7168)
2. (T62) Hello. ⁞world⁞ in doc.
   Expected results: uppercase predictions (cf feat(android): when selecting text, the keyboard has no context and thus gives Initial case, usually wrongly #7168)
3. (T63) Hello ⁞world⁞ in doc. Pick one prediction, e.g. and.
   Expected results: Hello and⁞
4. (T64) Hello. ⁞world⁞ in doc. Pick one prediction, e.g. The.
   Expected results: Hello. The⁞
5. (T65) ⁞Hello⁞. world in doc. Pick one prediction, e.g. The.
   Expected results: The⁞. world (cf feat(android): accepting a suggestion then adding punctuation should delete automatically added space #7163)

Suitable apps for testing

	Windows	Linux
TSF/surrouning-text	Wordpad, Word, Firefox	gedit, LibreOffice, Firefox
legacy	LibreOffice, Chrome	Chrome, gnome-terminal

A suitable page for testing in a browser is https://onlinenotepad.org/notepad.

text_selection_tests_keyboard test keyboard

The text_selection_tests_keyboard test keyboard is available in the artifacts of the last successful Keyman Developer test build, as keyboards/text_selection_tests_keyboard_9073/build/text_selection_tests_keyboard_9073.kmp (or here:
text_selection_tests_keyboard.zip).

A test page for Web/Android/iOS is in the static folder on downloads.keyman.com