Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug(web): automated test instability for prediction test #11844

Closed
jahorton opened this issue Jun 21, 2024 · 2 comments · Fixed by #11933
Closed

bug(web): automated test instability for prediction test #11844

jahorton opened this issue Jun 21, 2024 · 2 comments · Fixed by #11933
Assignees
Milestone

Comments

@jahorton
Copy link
Contributor

jahorton commented Jun 21, 2024

Build failure seen for "Test: Language Modeling Layer (Common)":


15:56:57 
  LMLayer using the trie model

15:56:57 
    Prediction

15:56:57 
    Regression tests

15:56:57 
      should use the default searchTermToKey()

15:56:58 
        expected +0 to be at least 1

15:56:58 
        AssertionError: expected +0 to be at least 1
            at file:///C:/BuildAgent/work/99b311828f4ee7c/keyman/common/predictive-text/unit_tests/headless/worker-trie-integration.js:85:16

The worker's "test mode" is on, so we shouldn't be exiting early... but perhaps recent changes from #11784 and its predecessors have had an unintended consequence here.

Originally posted by @jahorton in #11264 (comment)

@jahorton jahorton added this to the A18S5 milestone Jun 21, 2024
@jahorton jahorton self-assigned this Jun 21, 2024
@jahorton
Copy link
Contributor Author

jahorton commented Jul 5, 2024

Just saw another one:

04:03:41           expected undefined to be truthy
04:03:41           AssertionError: expected undefined to be truthy
              at Context.<anonymous> (file:///C:/BuildAgent/work/99b311828f4ee7c/keyman/common/web/lm-worker/src/test/mocha/cases/worker-model-compositor.js:194:16

From https://build.palaso.org/buildConfiguration/Keyman_Common_LMLayer_TestPullRequests/476399?buildTab=log&linesState=400.982.983.984.995&logView=flowAware&focusLine=983

This is for a relatively newer unit test, so the instability cause might be a little different, but it still falls under the same overall category.

@darcywong00 darcywong00 modified the milestones: A18S5, A18S6 Jul 5, 2024
@jahorton
Copy link
Contributor Author

jahorton commented Jul 8, 2024

Ah, so for the original issue, the worker's "test mode" is not on - the affected test is an integration test, ensuring that code outside the worker interfaces properly with code inside the worker. That codepath actually does have a way to set the "test mode" flag, but it never got enabled for... whatever reason.

The error seen just above this comment does have test-mode setting access, though, so its failure is for a different reason.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants