Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Add chunk_index to the unique_id in the rag__unified_document end model #11

Open
1 of 4 tasks
fivetran-joemarkiewicz opened this issue Nov 18, 2024 · 0 comments · May be fixed by #13
Open
1 of 4 tasks

[Bug] Add chunk_index to the unique_id in the rag__unified_document end model #11

fivetran-joemarkiewicz opened this issue Nov 18, 2024 · 0 comments · May be fixed by #13
Assignees
Labels
good first issue Good for newcomers

Comments

@fivetran-joemarkiewicz
Copy link
Collaborator

fivetran-joemarkiewicz commented Nov 18, 2024

Is there an existing issue for this?

  • I have searched the existing issues

Describe the issue

At the moment the unique_id field has the possibility of not truly being unique if there are multiple chunks associated with a document. Therefore, there is a chance where the uniqueness fails and the key is no longer unique.

Relevant error log or model output

Uniqueness test fails and the unique key is not truly unique to the individual row.

Expected behavior

The field is truly unique to the chunk.

Possible solution

Add chunk_index to this code update here in order to properly have a true unique field to the individual row.

In addition to the above code change we will also need to update the documentation here to account for the new field factored into the surrogate key.

dbt Project configurations

Default.

Package versions

Latest

What database are you using dbt with?

postgres, redshift, snowflake, bigquery, databricks

How are you running this dbt package?

Fivetran Quickstart Data Model

dbt Version

Latest

Additional Context

No response

Are you willing to open a PR to help address this issue?

  • Yes.
  • Yes, but I will need assistance.
  • No.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants