v0.4.0

What's Changed

• feat: improve late chunking and optimize pgvector settings by @lsorber in #51
  • Add a workaround for #24 to increase the embedder's context size from 512 to a user-definable size.
  • Increase the default embedder context size to 1024 tokens (more degrades bge-m3's performance).
  • Upgrade llama-cpp-python to the latest version.
  • More robust testing of rerankers with Kendall's rank correlation coefficient.
  • Optimise pgvector's settings.
  • Offer better control of oversampling in hybrid and vector search.
  • Upgrade to the PostgreSQL 17.

Full Changelog: v0.3.0...v0.4.0