Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

maint: Support protobuf Python library v5 #1422

Open
poodlewars opened this issue Mar 14, 2024 · 9 comments · May be fixed by #1857
Open

maint: Support protobuf Python library v5 #1422

poodlewars opened this issue Mar 14, 2024 · 9 comments · May be fixed by #1857
Assignees
@IvoDD
Labels
enhancement New feature or request

Comments

@poodlewars
Copy link
Collaborator

poodlewars commented Mar 14, 2024 •
edited
Loading

Is your feature request related to a problem? Please describe.
Protobuf has released version 5 but we only support version 3 and 4. PR #1421 has constrained protobuf version for now.

We should relax this constraint that we are imposing on user's envs.

Our proto handling is quite complex - we compile Python proto stubs suitable for both proto 3 and proto 4 and choose which a user will "see" at runtime based on their installed proto version.

The solution should consider both wheels and Conda builds.
@poodlewars poodlewars added the enhancement New feature or request label Mar 14, 2024
@jjerphan
Copy link
Collaborator

Currently ArcticDB's conda packages are built for:

protobuf 5 has not yet been published on conda-forge. conda-forge/libprotobuf-feedstock#213 is the first PR to start this work.

We might need to define a build variant for protobuf 3, 4, and 5 once this last version is supported.

@jjerphan
Copy link
Collaborator

jjerphan commented Apr 3, 2024

Related grpc/grpc#36142.

@poodlewars poodlewars mentioned this issue May 28, 2024
Closed
@rhoens
Copy link

rhoens commented May 31, 2024

Hi, any movement on this? Just curious if there's a timeline

@jjerphan
Copy link
Collaborator

jjerphan commented Jun 3, 2024 •
edited
Loading

Likely addressable for conda-forge's builds once conda-forge/protobuf-feedstock#216 is completed.

@muhammadhamzasajjad muhammadhamzasajjad self-assigned this Jun 20, 2024
@muhammadhamzasajjad muhammadhamzasajjad mentioned this issue Jun 20, 2024
Merged
5 tasks
muhammadhamzasajjad added a commit that referenced this issue Jul 3, 2024
@muhammadhamzasajjad
Upgrade to Protobuf 5 (#1635)
66a5d19 
Implements #1422 for pip wheels.

#### Checklist

<details>
  <summary>
   Checklist for code changes...
  </summary>
 
- [ ] Have you updated the relevant docstrings, documentation and
copyright notice?
- [ ] Is this contribution tested against [all ArcticDB's
features](../docs/mkdocs/docs/technical/contributing.md)?
- [ ] Do all exceptions introduced raise appropriate [error
messages](https://docs.arcticdb.io/error_messages/)?
 - [ ] Are API changes highlighted in the PR description?
- [ ] Is the PR labelled as enhancement or bug so it appears in
autogenerated release notes?
</details>

<!--
Thanks for contributing a Pull Request to ArcticDB! Please ensure you
have taken a look at:
- ArcticDB's Code of Conduct:
https://github.com/man-group/ArcticDB/blob/master/CODE_OF_CONDUCT.md
- ArcticDB's Contribution Licensing:
https://github.com/man-group/ArcticDB/blob/master/docs/mkdocs/docs/technical/contributing.md#contribution-licensing
-->
@muhammadhamzasajjad
Copy link
Collaborator

Upgrade to protobuf 5 for pip wheel has been done in #1635. Waiting for conda-forge/protobuf-feedstock#216 for protobuf 5 upgrade of conda

@jjerphan
Copy link
Collaborator

jjerphan commented Jul 8, 2024

FYI conda-forge/protobuf-feedstock#215 can also be watched (it was started before the bot opened conda-forge/protobuf-feedstock#216).

@jjerphan
Copy link
Collaborator

Protobuf 5 is being released on conda-forge thanks to conda-forge/protobuf-feedstock#215.

@IvoDD IvoDD linked a pull request Sep 26, 2024 that will close this issue
Draft
5 tasks
@jjerphan
Copy link
Collaborator

jjerphan commented Oct 1, 2024

ArcticDB can be build for libprotobuf 5 on the feedstock: conda-forge/arcticdb-feedstock#298

@jjerphan
Copy link
Collaborator

jjerphan commented Oct 7, 2024

See: conda-forge/arcticdb-feedstock#307

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@IvoDD IvoDD

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Remove protobuf pin for conda man-group/ArcticDB
6 participants
@IvoDD @rhoens @jjerphan @poodlewars @maxim-morozov @muhammadhamzasajjad