Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[enhancement]: Support tikv as a redis replacement #634

Open
1 task done
williamdes opened this issue Jul 17, 2024 · 8 comments · May be fixed by #668
Open
1 task done

[enhancement]: Support tikv as a redis replacement #634

williamdes opened this issue Jul 17, 2024 · 8 comments · May be fixed by #668
Labels
enhancement New feature or request

Comments

@williamdes
Copy link
Contributor

Which feature or improvement would you like to request?

See #23 as a thread reference

https://github.com/tikv/tikv

https://github.com/tikv/client-rust

Please let me know if this is a viable option to be implemented, so I can do the server setup and let you know if it works like it promises.

To be noted that in fact it runs a RocksDB on each node.
See https://tikv.org/docs/7.1/reference/architecture/storage/

Also, why can tikv not also be a data store?

Is your feature request related to a problem?

I do not want to deploy Redis, and will probably deploy something that I feel is better designed for multi site/datacenter

Code of Conduct

  • I agree to follow this project's Code of Conduct
@williamdes williamdes added the enhancement New feature or request label Jul 17, 2024
@alvinpeters
Copy link

alvinpeters commented Aug 2, 2024

Looks similar to FoundationDB. I will work on it.

Working branch: https://github.com/alvinpeters/stalwart-mail-server/tree/add-tikv

@alvinpeters
Copy link

alvinpeters commented Aug 4, 2024

@williamdes LOL I got it working somehow (albeit in a shithouse condition). Wanna help me dogfood/test this?

You can clone and compile the branch now:

git clone --branch add-tikv https://github.com/alvinpeters/stalwart-mail-server.git
cd stalwart-mail-server
cargo build --bin stalwart-mail  --features tikv

Since web-admin doesn't show it yet, you'd have to manually add it to the config.toml:

[store."tikv"]
type = "tikv"
pd-endpoints = ["127.0.0.1:2379"]

Then, you have to manually set the stores used since you can't just select it in the web admin, like:

[storage]
data = "tikv"
fts = "tikv"
blob = "tikv"
lookup = "tikv"
directory = "internal"

[directory."internal"]
type = "internal"
store = "tikv"

Currently setting up tests. I reckon I can get this in a ready shape within next week.

@williamdes
Copy link
Contributor Author

Thank you so much, I will help you testing
That said I keep the data in Garage garagehq.deuxfleurs.fr/
And directory in LDAP

@mdecimus
Copy link
Member

mdecimus commented Aug 4, 2024

@alvinpeters have you tried running the store tests?

Add the tikv store to tests/src/store/mod.rs and then run:

$ cd tests
$ STORE=tikv cargo test store_tests -- --nocapture

If that works try with the IMAP and JMAP tests:

$ STORE=tikv cargo test imap_tests -- --nocapture
$ STORE=tikv cargo test jmap_tests -- --nocapture

@alvinpeters
Copy link

Thanks! I didn't know how to properly run the test, so was only testing by just running the entire thing and was working kinda smoothly lol.

Either I messed up with the blobbing or I set the limit too high. I'll fix it when I wake up.

running 1 test
Testing store tikv...
Creating blobs...
Creating account data...
Creating queue, config and lookup data...
Creating directory data...
Calculating store hash...
thread 'store::store_tests' panicked at tests/src/store/import_export.rs:330:14:
called `Result::unwrap()` on an `Err` value: Event { inner: Store(TikvError), keys: [(Reason, String("gRPC api error: status: OutOfRange, message: \"Error, message length too large: found 21149150 bytes, the limit is: 4194304 bytes\", details: [], metadata: MetadataMap { headers: {\"content-type\": \"application/grpc\", \"grpc-accept-encoding\": \"identity, deflate, gzip\"} }")), (CausedBy, Static("crates/store/src/dispatch/store.rs:117"))] }

Should I submit a PR now so other people can easily contribute?

@williamdes
Copy link
Contributor Author

I definitely think a PR would be great to discuss the implementation

@alvinpeters
Copy link

Good thing I saw this before I sleep. Submitted #668 and stalwartlabs/webadmin#15 as draft PRs so other people can see/help.

@alvinpeters alvinpeters linked a pull request Aug 4, 2024 that will close this issue
5 tasks
@williamdes
Copy link
Contributor Author

williamdes commented Sep 8, 2024

Thank you for the implementation, it will help me if I try to implement etcd
Here is a pure Rust implementation compatible with etcd (https://etcd.io): https://github.com/xline-kv/Xline

etcd is used in the PD of TiKV

I am still searching for the best lightweight solution to setup.
For now most of them seem to use RocksDB: TiKV, Xline

For now TiKV launches quite a lot of processes and requires a PD process.
TiKV is not in Debian, but etcd-server is.

Will keep you all updated

Update 1, see: #742 for etcd

PS (personal opinion): I think that having to use software like tiup to run a setup is bullshit, it is nice to have but hides config details. Xline and etcd are much simpler, just run them after install. No downloading of components..

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants