Feat(embeddings): Embeddable derive macro #59

marieaurore123 · 2024-10-15T18:28:51Z

TODO: RENAME Embeddable trait to ToEmbedabble, ToEmbedTargets, Extract, ExtractEmbeddable, ...

This PR accomplishes:

Defines Embeddable trait which selects the portions of a struct that need ot be embedded. The embeddable() method returns a OneOrMany object which returns either one or many items to embed.
Write custom macro that derives the Embeddable trait for any user-defined object

…at(embeddings)/derive-macro

…macro

rig-core/src/embeddings/builder.rs

rig-core/src/embeddings/tool.rs

cvauclair · 2024-10-16T17:38:39Z

@garance-buricatu can you split the PR into two: 1) a PR that removes DocumentEmbeddings, adds the Embeddable trait.. anything not related to the macro; and 2) a PR specifically for the derive macro implementation?

That would make it much easier to review. Thank you!

rig-core/src/embeddings/embeddable.rs

marieaurore123 · 2024-10-17T16:40:08Z

rig-core/src/embeddings/embeddable.rs

+    pub fn rest(&self) -> Vec<T> {
+        self.rest.clone()
+    }
+


Note: I created the methods one and many to create objects of type OneOrMany instead of doing
impl From<T> for OneOrMany<T> and impl From<Vec<T>> for OneOrMany<T> because these are conflicting. See my comment below for why I would use impl From<Vec<T>> for OneOrMany<T> instead of impl TryFrom<Vec<T>> for OneOrMany<T>.

cvauclair

Not done reviewing, but there's a lot of stuff to change so here is the first part

rig-core/Cargo.toml

rig-core/src/embeddings/mod.rs

rig-core/examples/rag_dynamic_tools.rs

rig-core/examples/rag.rs

rig-core/examples/calculator_chatbot.rs

rig-core/src/embeddings/embeddable.rs

rig-core/tests/embeddable_macro.rs

rig-core/rig-core-derive/src/basic.rs

rig-core/rig-core-derive/src/custom.rs

devops: Add cargo check for all features + doc check

cvauclair

Excellent! Just a tiny change to the OneOrMany error definition and then you can merge :)

rig-core/src/one_or_many.rs

marieaurore123 added 26 commits October 2, 2024 18:07

feat: setup derive macro

b7d146f

test: test out writing embeddable macro

5904734

test: continue testing custom macro implementation

ee9b5c3

feat: macro generate trait bounds

c8c1e9c

refactor: split up macro into multiple files

dff0aeb

refactor: move macro derive crate inside rig-core

0d10011

Merge branch 'main' into feat(embeddings)/derive-macro

982a17b

feat: replace embedding logic with new embeddable trait and macro

79754aa

refactor: refactor rag examples, delete document embedding struct

3438fab

feat: remove document embedding from in memory store

597e6c3

revert: revert vector store to main

6c7ab8d

Merge branch 'refactor(vector-store)/in-memeory-vector-store' into fe…

d7d2c19

…at(embeddings)/derive-macro

docs: update emebddings builder docstrings

bb712e3

refactor: derive macro

efa2b65

feat: add error type on embeddable trait

8325164

refactor: move embeddings to its own module and seperate embeddable

de022c4

refactor: split up macro into more files, fix all imports

220d9fc

Merge branch 'feat/embeddings-overhaul' into feat(embeddings)/derive-…

f5e60f5

…macro

feat: handle tools with embeddingsbuilder

5a8c361

bug(macro): fix error when embed tags missing

dc89e54

style: cargo fmt

ae66d08

fix(tests): clippy

4305952

docs&revert: revert embeddable trait error type, add docstrings

24e3b98

style: cargo clippy

a7dbf6c

clippy(lancedb): fix unused function error

886ebcb

fix(test): remove useless assert false statement

79dea45

marieaurore123 commented Oct 15, 2024

View reviewed changes

rig-core/src/embeddings/builder.rs Outdated Show resolved Hide resolved

marieaurore123 commented Oct 15, 2024

View reviewed changes

rig-core/src/embeddings/tool.rs Outdated Show resolved Hide resolved

marieaurore123 changed the title ~~Feat(embeddings)/derive macro~~ Feat(embeddings): Embeddable derive macro Oct 15, 2024

marieaurore123 added 3 commits October 17, 2024 09:09

clippy: cargo clippy

cca6059

docs: add docstring to oneormany

f785b8c

fix(macro): update error handling

0e2ade9

marieaurore123 commented Oct 17, 2024

View reviewed changes

rig-core/src/embeddings/embeddable.rs Outdated Show resolved Hide resolved

marieaurore123 requested a review from cvauclair October 17, 2024 14:33

refactor: reexport EmbeddingsBuilder in rig and update imports

a98769c

marieaurore123 closed this Oct 17, 2024

marieaurore123 reopened this Oct 17, 2024

marieaurore123 added 2 commits October 17, 2024 12:31

feat: implement IntoIterator and Iterator for OneOrMany

067894c

refactor: rename from methods

32bcc61

marieaurore123 commented Oct 17, 2024

View reviewed changes

tests: fix failing tests

564bef4

cvauclair requested changes Oct 17, 2024

View reviewed changes

marieaurore123 added 4 commits October 17, 2024 14:39

refactor&fix: make PR review changes

04f1f3e

fix: fix tests failing

c8f6646

test: add test on OneOrMany

40f3c18

style: cargo fmt

68d88b6

cvauclair requested changes Oct 17, 2024

View reviewed changes

cvauclair and others added 9 commits October 17, 2024 17:52

devops: Add cargo check for all features + doc check

6ddc3c7

devops: Fix missing dep

f799f94

devops: Make cargo doc strict

3558085

docs: Fix docstring links

dc248ed

docs&fix: fix doc strings, implement iter_mut for OneOrMany

4bc7d07

Merge pull request #63 from 0xPlaygrounds/devops/ci-fix

093c434

devops: Add cargo check for all features + doc check

fix: update borrow and owning of macro

4d2ffdb

clippy: add back print statements

6f04225

Merge branch 'main' into feat(embeddings)/derive-macro

bdd98e5

cvauclair requested changes Oct 18, 2024

View reviewed changes

rig-core/src/one_or_many.rs Outdated Show resolved Hide resolved

refactor: use thiserror for OneOtMany::EmptyListError

485ad3b

marieaurore123 merged commit 4039cf6 into feat/embeddings-overhaul Oct 18, 2024
4 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feat(embeddings): Embeddable derive macro #59

Feat(embeddings): Embeddable derive macro #59

marieaurore123 commented Oct 15, 2024 •

edited

Loading

cvauclair commented Oct 16, 2024 •

edited

Loading

marieaurore123 Oct 17, 2024

cvauclair left a comment

cvauclair left a comment

Feat(embeddings): Embeddable derive macro #59

Feat(embeddings): Embeddable derive macro #59

Conversation

marieaurore123 commented Oct 15, 2024 • edited Loading

cvauclair commented Oct 16, 2024 • edited Loading

marieaurore123 Oct 17, 2024

Choose a reason for hiding this comment

cvauclair left a comment

Choose a reason for hiding this comment

cvauclair left a comment

Choose a reason for hiding this comment

marieaurore123 commented Oct 15, 2024 •

edited

Loading

cvauclair commented Oct 16, 2024 •

edited

Loading