Rework heed to be lightweight and simpler to maintain

[Kerollmops]

This PR closes #127, closes #94, closes #56, closes #88, closes #40, closes #94, closes #132, closes #138, closes #95.

• Publish the lmdb-master-sys crate to crates.io.
• Use the published version of lmdb-master-sys in heed.
• Bump the version of heed, heed-types, and heed-traits to crates.io.
• Retrieve the latest commit of the mdb.master LMDB branch.
• Import Add POSIX semaphores Cargo feature #142
• Import add a method to compute the size of an env #131 and provide multiple way to access the size of the environment #133
• Expose a method to reserve a buffer for writing directly into it i.e. MDB_RESERVE.
• Change the macro rule to be more explicit
• Move to edition 2021
• Replace the BEU32 x OwnedType combo by new simpler codecs.
• Read the LMDB documentation about aborting or committing read transactions, remove those methods from RoTxn is necessary.
• The RwTxn::abort function, must not return a Result as LMDB can't fail when doing so.
• The BadOpenOptions error must be more helpful and should maybe return the database with the expected options.
• Implement Debug on most of the structs.
• Fix the generated test files issue. Running the tests changes the test LMDB env files #138
• Cherry-pick fix of Add feature to force vendoring of the provided lmdb lmdb-rs#11
• Cherry-pick fix of Force vendoring of LMDB #136.
• Remove the Fuchsia LICENSE.
• Remove the double lifetime constraints from the RwTxn, the parent txn replaces the env one.
• Bring the documentation of LMDB into the bindings Feature request: Generate Rust docs from Doxygen comments rust-lang/rust-bindgen#955 (comment)

[arthurprs]

I was trying this branch yesterday and it was returning MDB_PROBLEM sometimes when committing whereas the main branch was working fine. I researched some more and it seems there's something off with this branch, but I can't tell what exactly.

For example, if you run cargo run --example clear-database and then try to stat the database with mdb_stat target/clear-database.mdb you get

mdb_env_open failed, error -30793 MDB_INVALID: File is not an LMDB file

whereas it works in the main branch it works as expected

Status of Main DB
  Tree depth: 1
  Branch pages: 0
  Leaf pages: 1
  Overflow pages: 0
  Entries: 1

Looking at the hexdump of the datafile you can see

00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000010  00 00 08 00 00 00 00 00  de c0 ef be 02 00 00 00  |................|

but the main branch gives

00000000  00 00 00 00 00 00 00 00  00 00 08 00 00 00 00 00  |................|
00000010  de c0 ef be 01 00 00 00  00 00 00 00 00 00 00 00  |................|

Maybe I'm doing something stupid... but I wanted to mention in case I'm not.

Edit: testing was done using x64 linux, gcc 12.1, rustc 1.60.0 (7737e0b5c 2022-04-04

[Kerollmops]

Hey @arthurprs,

Thank you very much for trying out this work-in-progress branch! I indeed had the same kind of issues and it looked like it was related to the fact that the folder to store the LMDB environment was already there, I fixed the whole list of tests by using a temporary directory. But it is very strange, I moved from the master to the master3 branch of LMDB, maybe there is an issue when we use nested write transactions to create databases or environments.

[Kerollmops]

Hey @arthurprs,

Thank you again for testing this new branch, and for finding this potential issue.

Unfortunately, I wasn't able to trigger the same issue when using mdb_stat on a database created with the clear-database example. I don't know if you notice but this branch is using the mdb.master3 branch of LMDB which is incompatible with the previous version I was using i.e. mdb.master. If you want to use the right mdb_stat binary you can go to the lmdb_master3_sys/lmdb/libraries/liblmdb folder and call make, the mdb_stat binary will be available in the folder.

I would be glad if you try out this branch and tell me whether your program is correctly working or is completely broken. Don't expect to be able to read a previous version of an LMDB database. Note that the API is changing a lot and I didn't document that for now, I will do it once this PR is merged.

[Techie-Pi]

Hi! I've seen that this PR uses doxygen-rs, and fyi I've just uploaded the crate to crates.io: https://crates.io/crates/doxygen-rs
Just in case you'd like to use that instead.

If you find any problem with the crate, feel free to open an issue or sending me a DM, I'll be more than happy to help! :)

[Kerollmops]

Thank you very much @Techie-Pi. I just updated the dependency to use a safer source which is crates.io 🙏

[Kerollmops]

For information, I will merge this PR with the mdb.master LMDB branch and not the mdb.master3, for now. I will bring mdb.master3 and the encryption/decryption features in #134 later. It will be easier for Meilisearch/milli to first try the new version of the library without bringing the new LMDB version for now, I prefer moving step by step.

[Kerollmops]

And here we are, heed v0.20.0-alpha.0 just released on crates.io 🎉