Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lib tool upgrades 44x #1934

Draft
wants to merge 4 commits into
base: 4.4.x
Choose a base branch
from
Draft

Lib tool upgrades 44x #1934

wants to merge 4 commits into from

Commits on Oct 29, 2024

  1. LibraryTool add read_index (#1533)

    #### Reference Issues/PRs
    Implements #1391 and fixes #1532.
    - Added read_index in LibraryTool (as required in #1391)
    - Allowed Library to get LibraryTool and call read_index (to fix #1532).
    Also updated the docs accordingly.
    
    #### What does this implement or fix?
    
    #### Any other comments?
    
    #### Checklist
    
    <details>
      <summary>
       Checklist for code changes...
      </summary>
     
    - [ ] Have you updated the relevant docstrings, documentation and
    copyright notice?
    - [ ] Is this contribution tested against [all ArcticDB's
    features](../docs/mkdocs/docs/technical/contributing.md)?
    - [ ] Do all exceptions introduced raise appropriate [error
    messages](https://docs.arcticdb.io/error_messages/)?
     - [ ] Are API changes highlighted in the PR description?
    - [ ] Is the PR labelled as enhancement or bug so it appears in
    autogenerated release notes?
    </details>
    
    <!--
    Thanks for contributing a Pull Request to ArcticDB! Please ensure you
    have taken a look at:
    - ArcticDB's Code of Conduct:
    https://github.com/man-group/ArcticDB/blob/master/CODE_OF_CONDUCT.md
    - ArcticDB's Contribution Licensing:
    https://github.com/man-group/ArcticDB/blob/master/docs/mkdocs/docs/technical/contributing.md#contribution-licensing
    -->
    muhammadhamzasajjad authored and IvoDD committed Oct 29, 2024
    Configuration menu
    Copy the full SHA
    e277469 View commit details
    Browse the repository at this point in the history
  2. Fix lib_tool.read_to_keys

    - Fixes reading correct index value for timestamps
    - Fixes reading VERSION_REF keys
    - Adds elaborate tests for iterating the version chain
    IvoDD committed Oct 29, 2024
    Configuration menu
    Copy the full SHA
    755160a View commit details
    Browse the repository at this point in the history
  3. Adds a type change utility for append data keys in lib_tool

    Does this by adding a cpp layer functionality to overwrite append data
    keys. And using a read + type change + overwrite in the python layer.
    
    More specifically this change involves:
    - Using a `LocalVersionEngine` instead of an `AsyncStore` as library
      tool state
    - Exposing some more python bindings to allow iteration over `APPEND_DATA`
      keys with the library tool
    - Allow using normalization in library tool to allow overwriting append
      data keys with a custom dataframe
    - Provide the type change functionality by reading a dataframe, changing
      its type with pandas and overwriting it
    - Adds an elaborate test to verify iterating and reading append data
      linked list is fine with various overwrites.
    IvoDD committed Oct 29, 2024
    Configuration menu
    Copy the full SHA
    86181f2 View commit details
    Browse the repository at this point in the history
  4. Backport fixes to make newer lib tool code work correctly

    - Replaces a few function calls with their old names
    - Removes some timeseries descriptor bindings exposed
    IvoDD committed Oct 29, 2024
    Configuration menu
    Copy the full SHA
    cc6fb57 View commit details
    Browse the repository at this point in the history