Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

New db to speed up full text queries and library updates #114

Open
wants to merge 13 commits into
base: master
Choose a base branch
from
Open

New db to speed up full text queries and library updates #114

wants to merge 13 commits into from

Commits on Feb 19, 2024

  1. Allow fetch requests to read from a single downloaded tar.gz file

    @dougy83
    dougy83 committed Feb 19, 2024
    Configuration menu
    Copy the full SHA
    1e2fc32 View commit details
    Browse the repository at this point in the history

  2. Add all-data.tar.gz creation to the buildtables function

    @dougy83
    dougy83 committed Feb 19, 2024
    Configuration menu
    Copy the full SHA
    415fae8 View commit details
    Browse the repository at this point in the history

  3. Split combined data file into two parts

    @dougy83
    dougy83 committed Feb 19, 2024
    Configuration menu
    Copy the full SHA
    1fd9ada View commit details
    Browse the repository at this point in the history

  4. Allow using downloaded chunks of the combined, even if some chunks ar… 

    …e missing
    @dougy83
    dougy83 committed Feb 19, 2024
    Configuration menu
    Copy the full SHA
    16c4251 View commit details
    Browse the repository at this point in the history

  5. Squash database into 3 gz files. Single file download. Use gz files f… 

    …or queries.
    @dougy83
    dougy83 committed Feb 19, 2024
    Configuration menu
    Copy the full SHA
    0b2a27e View commit details
    Browse the repository at this point in the history

  6. Generate new db combined data file in github action

    @dougy83
    dougy83 committed Feb 19, 2024
    Configuration menu
    Copy the full SHA
    21df052 View commit details
    Browse the repository at this point in the history

  7. Update first-time and update-available code for new db. Remove old up… 

    …date db code.
    @dougy83
    dougy83 committed Feb 19, 2024
    Configuration menu
    Copy the full SHA
    d4dacd6 View commit details
    Browse the repository at this point in the history

  8. Reverting inconsequential changes to datatables.py

    @dougy83
    dougy83 committed Feb 19, 2024
    Configuration menu
    Copy the full SHA
    5f1ce99 View commit details
    Browse the repository at this point in the history

  9. Fix update download finished status

    @dougy83
    dougy83 committed Feb 19, 2024
    Configuration menu
    Copy the full SHA
    56f4d38 View commit details
    Browse the repository at this point in the history

  10. Update readme.md to include the generateJsonlinesDatabaseFiles step

    @dougy83
    dougy83 committed Feb 19, 2024
    Configuration menu
    Copy the full SHA
    e312765 View commit details
    Browse the repository at this point in the history

Commits on Feb 20, 2024

  1. Fix slow generateJsonlinesDatabaseFiles.js processing (now takes <15s… 

    …econds)
    @dougy83
    dougy83 committed Feb 20, 2024
    Configuration menu
    Copy the full SHA
    26fbcbd View commit details
    Browse the repository at this point in the history

Commits on Feb 23, 2024

  1. Move generateJsonlinesDatabaseFiles.js to correct location

    @dougy83
    dougy83 committed Feb 23, 2024
    Configuration menu
    Copy the full SHA
    1a7899f View commit details
    Browse the repository at this point in the history

Commits on Feb 24, 2024

  1. Move all datatable processing into the python script. Don't create js… 

    …on datatables
    @dougy83
    dougy83 committed Feb 24, 2024
    Configuration menu
    Copy the full SHA
    6e2bf8b View commit details
    Browse the repository at this point in the history