Add TiKV as a data store backend

[alvinpeters]

This will add TiKV as a backend to all store types. All help/reviews/roasting my shitty coding welcome!

Resolves #634

To do's:

• Add TiKV code and get it compiling
• Add to the webadmin: Add TiKV as a data store backend webadmin#15
• Pass the tests
• Optimise the code
• Clean up

Can I trust this branch with my personal emails?

No.

Current issues (2338f08):

Fail on Store query tests (first errors are 'failed to resolve lock' then the last errors show 'epoch not match')

running 1 test
using backoff Backoff { kind: DecorrelatedJitter, current_attempts: 0, max_attempts: 30, base_delay_ms: 500, current_delay_ms: 500, max_delay_ms: 2000 }
Testing store tikv...
Creating blobs...
Creating account data...
Creating queue, config and lookup data...
Creating directory data...
Calculating store hash...
Exporting store...
Exporting database to /var/folders/b2/1ntvvtl16h16hr0gj39nms2c0000gn/T/art_vandelay_tests/property.
Exporting database to /var/folders/b2/1ntvvtl16h16hr0gj39nms2c0000gn/T/art_vandelay_tests/fts_index.
Exporting database to /var/folders/b2/1ntvvtl16h16hr0gj39nms2c0000gn/T/art_vandelay_tests/acl.
Exporting database to /var/folders/b2/1ntvvtl16h16hr0gj39nms2c0000gn/T/art_vandelay_tests/lookup.
Exporting database to /var/folders/b2/1ntvvtl16h16hr0gj39nms2c0000gn/T/art_vandelay_tests/directory.
Exporting database to /var/folders/b2/1ntvvtl16h16hr0gj39nms2c0000gn/T/art_vandelay_tests/blob.
Exporting database to /var/folders/b2/1ntvvtl16h16hr0gj39nms2c0000gn/T/art_vandelay_tests/config.
Exporting database to /var/folders/b2/1ntvvtl16h16hr0gj39nms2c0000gn/T/art_vandelay_tests/bitmap.
Exporting database to /var/folders/b2/1ntvvtl16h16hr0gj39nms2c0000gn/T/art_vandelay_tests/queue.
Exporting database to /var/folders/b2/1ntvvtl16h16hr0gj39nms2c0000gn/T/art_vandelay_tests/index.
Exporting database to /var/folders/b2/1ntvvtl16h16hr0gj39nms2c0000gn/T/art_vandelay_tests/log.
Destroying store...
Importing store...
Importing database dump from /var/folders/b2/1ntvvtl16h16hr0gj39nms2c0000gn/T/art_vandelay_tests/directory.
Importing database dump from /var/folders/b2/1ntvvtl16h16hr0gj39nms2c0000gn/T/art_vandelay_tests/index.
Importing database dump from /var/folders/b2/1ntvvtl16h16hr0gj39nms2c0000gn/T/art_vandelay_tests/lookup.
Importing database dump from /var/folders/b2/1ntvvtl16h16hr0gj39nms2c0000gn/T/art_vandelay_tests/fts_index.
Importing database dump from /var/folders/b2/1ntvvtl16h16hr0gj39nms2c0000gn/T/art_vandelay_tests/log.
Importing database dump from /var/folders/b2/1ntvvtl16h16hr0gj39nms2c0000gn/T/art_vandelay_tests/bitmap.
Importing database dump from /var/folders/b2/1ntvvtl16h16hr0gj39nms2c0000gn/T/art_vandelay_tests/config.
Importing database dump from /var/folders/b2/1ntvvtl16h16hr0gj39nms2c0000gn/T/art_vandelay_tests/blob.
Importing database dump from /var/folders/b2/1ntvvtl16h16hr0gj39nms2c0000gn/T/art_vandelay_tests/property.
Importing database dump from /var/folders/b2/1ntvvtl16h16hr0gj39nms2c0000gn/T/art_vandelay_tests/queue.
Importing database dump from /var/folders/b2/1ntvvtl16h16hr0gj39nms2c0000gn/T/art_vandelay_tests/acl.
Verifying store hash... GREAT SUCCESS!
Running Store ID assignment tests...
Creating 1000 documentIds concurrently...
Deleting 1000 documentIds concurrently...
Reusing 1000 freed documentIds concurrently...
Running dynamic ID assignment tests...
Incrementing counter 1000 times concurrently...
Running chunking tests...
test store::store_tests has been running for over 60 seconds
Running Store query tests...
Parsed 69201 entries in 547 ms.
Inserting...  [2785 ms] [3298 ms] [4576 ms] [4237 ms] [4267 ms] [4453 ms] [4662 ms] [4326 ms] [2925 ms] [4019 ms] [6429 ms] [4925 ms] [2952 ms] [3909 ms] [4216 ms] [5048 ms] [4405 ms]thread 'store::store_tests' panicked at tests/src/store/query.rs:218:43:
called `Result::unwrap()` on an `Err` value: Event { inner: Store(TikvError), keys: [(Reason, String("Region error: Error { message: \"EpochNotMatch current epoch of region 8 is conf_ver: 1 version: 6, but you sent conf_ver: 1 version: 5\", not_leader: None, region_not_found: None, key_not_in_region: None, epoch_not_match: Some(EpochNotMatch { current_regions: [Region { id: 8, start_key: [118, 0, 0, 0, 0, 111, 114, 103, 255, 0, 18, 0, 0, 9, 17, 0, 0, 253], end_key: [120, 0, 0, 0, 0, 0, 0, 0, 251], region_epoch: Some(RegionEpoch { conf_ver: 1, version: 6 }), peers: [Peer { id: 9, store_id: 1, role: Voter }], encryption_meta: None }, Region { id: 12, start_key: [114, 0, 0, 1, 0, 0, 0, 0, 251], end_key: [118, 0, 0, 0, 0, 111, 114, 103, 255, 0, 18, 0, 0, 9, 17, 0, 0, 253], region_epoch: Some(RegionEpoch { conf_ver: 1, version: 6 }), peers: [Peer { id: 13, store_id: 1, role: Voter }], encryption_meta: None }] }), server_is_busy: None, stale_command: None, store_not_match: None, raft_entry_too_large: None, max_timestamp_not_synced: None, read_index_not_ready: None, proposal_in_merging_mode: None, data_is_not_ready: None, region_not_initialized: None, disk_full: None }"))] }
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
test store::store_tests ... FAILED

failures:

failures:
    store::store_tests

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 71 filtered out; finished in 192.63s

Development/testing environment:

• Machine: Macbook Pro 2021 (M1 Pro 500GB SSD 16GB RAM)
• OS: macOS Sonoma 14.6.1
• Database: TiKV and PD v8.3.0, single-node instance with tiup playground

Running and testing yourself

Expand me

Config entry:

[store."tikv"]
type = "tikv"
pd-endpoints = ["127.0.0.1:2379"]

Building:

git clone --branch add-tikv https://github.com/alvinpeters/stalwart-mail-server.git
cd stalwart-mail-server
cargo build --bin stalwart-mail  --features tikv

I'm too tired to add the running and setting up webadmin. Will do tomorrow lol

Testing:

cd tests && STORE=tikv cargo test store_tests -- --nocapture

[williamdes]

Can you enable GitHub actions on your fork repository so it is easier to access tests reports please?

[alvinpeters]

Can you enable GitHub actions on your fork repository so it is easier to access tests reports please?

Done. Although, I am kinda clueless on how to test a specific backend using a GA workflow.