Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(web): add recent-history log to gesture engine #11277

Merged
merged 1 commit into from
Apr 22, 2024

Conversation

jahorton
Copy link
Contributor

To facilitate debugging of the gesture engine when needed, this PR adds configurable history-tracking to the gesture engine.

Within KMW:

// Provides an array of the live-JS form of the most recently-seen gestures
keyman.osk.vkbd.gestureEngine.history

// Provides a prettified JSON version of the latter - great for preserving as text logs
console.log(keyman.osk.vkbd.gestureEngine.historyJSON)

@keymanapp-test-bot skip

@keymanapp-test-bot keymanapp-test-bot bot added this to the B17S6 milestone Apr 22, 2024
@jahorton
Copy link
Contributor Author

Example JSON-log for a quick sil_euro_latin down-flick:

[
  [
    {
      "gestureSetId": "default",
      "matchedId": "flick-start",
      "linkType": "chain",
      "sources": [
        {
          "identifier": "touch:1",
          "isFromTouch": true,
          "path": {
            "coords": [
              {
                "targetX": 105,
                "targetY": 105.4521484375,
                "t": 4361,
                "item": "shift-K_D"
              },
              {
                "targetX": 105,
                "targetY": 106.32171630859375,
                "t": 4393,
                "item": "shift-K_D"
              },
              {
                "targetX": 105,
                "targetY": 107.19134521484375,
                "t": 4398.099999964237,
                "item": "shift-K_D"
              },
              {
                "targetX": 105,
                "targetY": 111.53915405273438,
                "t": 4404.5,
                "item": "shift-K_D"
              },
              {
                "targetX": 105,
                "targetY": 115.88699340820312,
                "t": 4413,
                "item": "shift-K_D"
              },
              {
                "targetX": 105.86958312988281,
                "targetY": 121.97393798828125,
                "t": 4420.5,
                "item": "shift-K_D"
              },
              {
                "targetX": 107.60869598388672,
                "targetY": 131.53912353515625,
                "t": 4427.399999976158,
                "item": "shift-K_D"
              }
            ],
            "stats": {
              "angle": 3.0419240590114853,
              "cardinal": "s",
              "netDistance": 26.21708535444175,
              "duration": 66.39999997615814,
              "sampleCount": 7,
              "rawDistance": 26.305590510382302
            }
          },
          "stateToken": "shift"
        }
      ],
      "allSourceIds": [
        "touch:1"
      ]
    },
    {
      "gestureSetId": "default",
      "matchedId": "flick-mid",
      "linkType": "chain",
      "sources": [
        {
          "identifier": "touch:1",
          "isFromTouch": true,
          "path": {
            "coords": [
              {
                "targetX": 105,
                "targetY": 105.4521484375,
                "t": 4361,
                "item": "shift-K_D"
              },
              {
                "targetX": 105,
                "targetY": 106.32171630859375,
                "t": 4393,
                "item": "shift-K_D"
              },
              {
                "targetX": 105,
                "targetY": 107.19134521484375,
                "t": 4398.099999964237,
                "item": "shift-K_D"
              },
              {
                "targetX": 105,
                "targetY": 111.53915405273438,
                "t": 4404.5,
                "item": "shift-K_D"
              },
              {
                "targetX": 105,
                "targetY": 115.88699340820312,
                "t": 4413,
                "item": "shift-K_D"
              },
              {
                "targetX": 105.86958312988281,
                "targetY": 121.97393798828125,
                "t": 4420.5,
                "item": "shift-K_D"
              },
              {
                "targetX": 107.60869598388672,
                "targetY": 131.53912353515625,
                "t": 4427.399999976158,
                "item": "shift-K_D"
              }
            ],
            "stats": {
              "angle": 3.0419240590114853,
              "cardinal": "s",
              "netDistance": 26.21708535444175,
              "duration": 66.39999997615814,
              "sampleCount": 7,
              "rawDistance": 26.305590510382302
            }
          },
          "stateToken": "shift"
        }
      ],
      "allSourceIds": [
        "touch:1"
      ]
    },
    {
      "gestureSetId": "default",
      "matchedId": "flick-end",
      "linkType": "complete",
      "item": "shift-K_SPACE",
      "sources": [
        {
          "identifier": "touch:1",
          "isFromTouch": true,
          "path": {
            "coords": [
              {
                "targetX": 105,
                "targetY": 105.4521484375,
                "t": 4361,
                "item": "shift-K_D"
              },
              {
                "targetX": 105,
                "targetY": 106.32171630859375,
                "t": 4393,
                "item": "shift-K_D"
              },
              {
                "targetX": 105,
                "targetY": 107.19134521484375,
                "t": 4398.099999964237,
                "item": "shift-K_D"
              },
              {
                "targetX": 105,
                "targetY": 111.53915405273438,
                "t": 4404.5,
                "item": "shift-K_D"
              },
              {
                "targetX": 105,
                "targetY": 115.88699340820312,
                "t": 4413,
                "item": "shift-K_D"
              },
              {
                "targetX": 105.86958312988281,
                "targetY": 121.97393798828125,
                "t": 4420.5,
                "item": "shift-K_D"
              },
              {
                "targetX": 107.60869598388672,
                "targetY": 131.53912353515625,
                "t": 4427.399999976158,
                "item": "shift-K_D"
              },
              {
                "targetX": 108.47826385498047,
                "targetY": 141.1043701171875,
                "t": 4433.799999952316,
                "item": "shift-K_X"
              },
              {
                "targetX": 109.34783172607422,
                "targetY": 151.53912353515625,
                "t": 4441.5,
                "item": "shift-K_X"
              },
              {
                "targetX": 109.34783172607422,
                "targetY": 161.97393798828125,
                "t": 4448.599999964237,
                "item": "shift-K_X"
              },
              {
                "targetX": 109.34783172607422,
                "targetY": 171.53912353515625,
                "t": 4456.299999952316,
                "item": "shift-K_X"
              },
              {
                "targetX": 111.08695220947266,
                "targetY": 181.97393798828125,
                "t": 4464.5,
                "item": "shift-K_X"
              },
              {
                "targetX": 111.08695220947266,
                "targetY": 192.40869140625,
                "t": 4481.199999988079,
                "item": "shift-K_X"
              },
              {
                "targetX": 111.08695220947266,
                "targetY": 202.843505859375,
                "t": 4486.799999952316,
                "item": "shift-K_X"
              },
              {
                "targetX": 111.08695220947266,
                "targetY": 214.1478271484375,
                "t": 4494.399999976158,
                "item": "shift-K_SPACE"
              },
              {
                "targetX": 111.08695220947266,
                "targetY": 217.6260986328125,
                "t": 4503.099999964237,
                "item": "shift-K_SPACE"
              },
              {
                "targetX": 111.9565200805664,
                "targetY": 223.7130126953125,
                "t": 4511.399999976158,
                "item": "shift-K_SPACE"
              },
              {
                "targetX": 111.9565200805664,
                "targetY": 223.7130126953125,
                "t": 4512.299999952316,
                "item": "shift-K_SPACE"
              },
              {
                "targetX": 111.9565200805664,
                "targetY": 227.1912841796875,
                "t": 4517.399999976158,
                "item": "shift-K_SPACE"
              },
              {
                "targetX": 111.9565200805664,
                "targetY": 230.6695556640625,
                "t": 4526.799999952316,
                "item": "shift-K_SPACE"
              },
              {
                "targetX": 111.9565200805664,
                "targetY": 234.1478271484375,
                "t": 4531.699999988079,
                "item": "shift-K_SPACE"
              },
              {
                "targetX": 112.82608795166016,
                "targetY": 235.886962890625,
                "t": 4538.899999976158,
                "item": "shift-K_SPACE"
              },
              {
                "targetX": 112.82608795166016,
                "targetY": 237.6260986328125,
                "t": 4545.799999952316,
                "item": "shift-K_SPACE"
              },
              {
                "targetX": 112.82608795166016,
                "targetY": 238.49566650390625,
                "t": 4555.199999988079,
                "item": "shift-K_SPACE"
              },
              {
                "targetX": 112.82608795166016,
                "targetY": 239.36517333984375,
                "t": 4570.799999952316,
                "item": "shift-K_SPACE"
              }
            ],
            "wasCancelled": false,
            "stats": {
              "angle": 3.083217477723039,
              "cardinal": "s",
              "netDistance": 134.14151442086398,
              "duration": 209.79999995231628,
              "sampleCount": 25,
              "rawDistance": 134.61826311234893
            }
          },
          "stateToken": "shift"
        }
      ],
      "allSourceIds": [
        "touch:1"
      ]
    }
  ]
]

@mcdurdin mcdurdin changed the title feat(web): adds recent-history log to gesture engine feat(web): add recent-history log to gesture engine Apr 22, 2024
Copy link
Member

@mcdurdin mcdurdin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM except the name toJSON which returns a Javascript object, not a JSON blob.

@@ -138,4 +138,13 @@ export class GesturePath<Type, StateToken = any> extends EventEmitter<EventMap<T

this.removeAllListeners();
}

public toJSON(): any {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not quite the right name -- because it's not returning JSON, it's returning a Javascript object.

Copy link
Contributor Author

@jahorton jahorton Apr 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a special name and is correct.

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/stringify

For other objects:

  • If the value has a toJSON() method, it's responsible to define what data will be serialized. Instead of the object being serialized, the value returned by the toJSON() method when called will be serialized. JSON.stringify() calls toJSON with one parameter, the key, which has the same semantic as the key parameter of the replacer function:

    • if this object is a property value, the property name
    • if it is in an array, the index in the array, as a string
    • if JSON.stringify() was directly called on this object, an empty string

I'd have to test to be sure, but I believe returning an object allows other, higher-level filters to be applied to the returned object as well - and that can be useful.

@@ -92,8 +94,7 @@ describe("GesturePath", function() {
"wasCancelled": true
}
`.trim();

assert.equal(JSON.stringify(serializationObj, null, 2), SERIALIZATION_TO_MATCH);
assert.equal(JSON.stringify(serializationObj, (key, value) => key == 'stats' ? undefined : value, 2), SERIALIZATION_TO_MATCH);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The object returned by .toJSON can be further processed; here, we filter out the stats field via JSON.stringify replacer. (It wasn't there when the original test was written, so we get rid of it to match the original expectation.)

@jahorton jahorton merged commit 122351b into beta Apr 22, 2024
20 checks passed
@jahorton jahorton deleted the feat/web/gesture-engine-history branch April 22, 2024 08:29
@keyman-server
Copy link
Collaborator

Changes in this pull request will be available for download in Keyman version 17.0.312-beta

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants