Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[tests] JavaScript: extract searchindex.js-format test fixtures. #12102

Conversation

jayaddison
Copy link
Contributor

@jayaddison jayaddison commented Mar 15, 2024

Feature or Bugfix

  • Refactoring

Purpose

  • Reformats the test index data in the JavaScript HTML search tests to match the format emitted by the Sphinx (Python) HTML builder.

Detail

  • The content of searchindex.js is a JavaScript code file that runs Search.setIndex(...) with the contents of the index as an argument.
  • Instead of loading the index per-test from a variable, retrieve each fixture from the built-in local karma HTTP server and eval it.
  • Includes fixtures generated from genuine, buildable test Sphinx project sources included within this repository.
  • Fails the Python test suite when regenerated searchindex.js fixture content does not match the existing fixture files.

Relates

@jayaddison

This comment was marked as resolved.

@jayaddison
Copy link
Contributor Author

In follow-up pull requests, the plan is to generate fixtures from genuine, buildable test Sphinx project sources included within this repository.

This turned out to be less code than I expected, so I've bundled it in here too, but could separate it again if that's easier for review/understanding.

@jayaddison jayaddison marked this pull request as ready for review March 15, 2024 22:33
tests/js/searchtools.js Outdated Show resolved Hide resolved
@jayaddison jayaddison marked this pull request as draft March 16, 2024 10:43
@jayaddison

This comment was marked as outdated.

@picnixz
Copy link
Member

picnixz commented Mar 16, 2024

I know there are a lot of these small changes / PRs, so it's probably tricky to develop the context for what's changing.

Which is the reverse of what I'm doing on the test suite where the PRs are huge. So don't worry

tests/js/fixtures/multiterm/searchindex.js Outdated Show resolved Hide resolved
tests/js/fixtures/cpp/searchindex.js Outdated Show resolved Hide resolved
tests/js/searchtools.js Show resolved Hide resolved
tests/js/searchtools.js Outdated Show resolved Hide resolved
Copy link
Contributor

@wlach wlach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work! I left a few comments which are mostly just nits.

CHANGES.rst Outdated Show resolved Hide resolved
* ~~~~~~~~~~~~~~~~
*
* This script contains the language-specific data used by searchtools.js,
* namely the list of stopwords, stemmer, scorer and splitter.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comments in this file seem a little misleading? This is really just test scaffolding AFAICT

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mm, yep - this is an interesting case - it's an evaluated copy of a template file from the basic theme. That's me attempting to make the JavaScript test environment realistic, but it is duplicative (and a bit confusing).

Main Page
=========

Welcome to the... main page!
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would personally use this opportunity to add a bit of context about what this file tests.

sphinx_utils module
===================

Useful utilities.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same, I would personally use this opportunity to add a bit of context about what this file tests.

function loadFixture(name) {
req = new XMLHttpRequest();
req.open("GET", `base/tests/js/fixtures/${name}`, false);
req.send(null);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm guessing you used XMLHttpRequest() because you want to make a synchronous? Should probably add a comment about that.

Apparently using XMLHttpRequest this way is discouraged and might be deprecated at some point, but they've been saying that since 2014. 😉 It's probably fine.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, that's good to know :) Yep; we don't want the test to proceed until the fixture has definitely loaded -- I'm inclined to use a straightforward (even if seemingly dated) approach here.

tests/js/searchtools.js Outdated Show resolved Hide resolved
tests/test_search.py Outdated Show resolved Hide resolved
@jayaddison
Copy link
Contributor Author

Thank you @wlach for the code review! I've left a couple of not-entirely-resolved items open, and let me know what you think of the adjustments.

Copy link
Contributor

@wlach wlach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good from my perspective! Obviously you'll need a committer to approve/land.

@wlach wlach requested a review from picnixz May 22, 2024 01:27
@wlach
Copy link
Contributor

wlach commented May 22, 2024

@picnixz Could you take a look at this? I think it makes sense to merge this before proceeding with any more work on the search system. I reviewed it a couple weeks ago and all looks good to me (not to say I'm infallible but that's gotta count for something)

utils/generate_js_fixtures.py Outdated Show resolved Hide resolved
tests/test_search.py Outdated Show resolved Hide resolved
tests/js/searchtools.js Outdated Show resolved Hide resolved
tests/js/fixtures/partial/searchindex.js Show resolved Hide resolved
@picnixz
Copy link
Member

picnixz commented May 22, 2024

Could you take a look at this? I think it makes sense to merge this before proceeding with any more work on the search system

Lucky you that I've submitted my thesis and have more free time!

Copy link
Member

@picnixz picnixz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For me it's fine now (modulo this small suggestion but we can just decide later).

tests/test_search.py Outdated Show resolved Hide resolved
@picnixz
Copy link
Member

picnixz commented Jun 13, 2024

Once you're done, just ping me and I'll merge it !

@jayaddison
Copy link
Contributor Author

Thanks @picnixz - all done with those changes, so I think this is ready for merge.

@picnixz picnixz merged commit 568e26c into sphinx-doc:master Jun 13, 2024
24 checks passed
@picnixz
Copy link
Member

picnixz commented Jun 13, 2024

Thank you!

@jayaddison jayaddison deleted the issue-12099-prep/extract-js-search-index-fixtures branch June 13, 2024 11:10
@AA-Turner AA-Turner added this to the 7.4.0 milestone Jul 13, 2024
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 13, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Tests: generate the indexes in the HTML search JavaScript tests using the application code.
4 participants