add citation to rag example

[mpyen3]

Important

Adds citation functionality to RAG example by extending BAML model and updating Python code to handle citations.

• Behavior:
  • Adds citation functionality to RAG example in rag.mdx and rag.py.
  • Extends BAML model with ResponseWithCitations class and RAGWithCitations function.
  • Updates Python loop to use RAGWithCitations for generating responses with citations.
• Tests:
  • Adds TestCitations in rag.baml to verify citation functionality.
• Documentation:
  • Updates rag.mdx with advanced section on creating citations with LLM.

^{This description was created by for f10a51f. It will automatically update as commits are pushed.}

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
baml	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Mar 10, 2025 4:49am

[ellipsis-dev]

❌ Changes requested. Reviewed everything up to b7c9ca2 in 2 minutes and 15 seconds

More details

• Looked at 115 lines of code in 1 files
• Skipped 0 files when reviewing.
• Skipped posting 6 drafted comments based on config settings.

1. fern/01-guide/06-prompt-engineering/rag.mdx:179

• Draft comment:
  Consider ensuring heading level consistency. The new 'Advanced: Creating Citations with LLM' section uses a '###' heading; verify it aligns with the document’s overall hierarchy.
• Reason this comment was not posted:
  Confidence changes required: 50% <= threshold 50%
  None

2. fern/01-guide/06-prompt-engineering/rag.mdx:194

• Draft comment:
  Clarify the prompt instructions in RAGWithCitations. Consider expanding on the expected citation format to ensure consistent output.
• Reason this comment was not posted:
  Confidence changes required: 50% <= threshold 50%
  None

3. fern/01-guide/06-prompt-engineering/rag.mdx:230

• Draft comment:
  In the updated Python snippet using RAGWithCitations, consider adding error handling or sanity checks on the response structure, ensuring the citations array is correctly returned by the model.
• Reason this comment was not posted:
  Confidence changes required: 50% <= threshold 50%
  None

4. fern/01-guide/06-prompt-engineering/rag.mdx:77

• Draft comment:
  Typo in the heading: consider renaming 'Creating a VectoreStore' to 'Creating a VectorStore' for consistency with the class name in the Python code.
• Reason this comment was not posted:
  Comment was not on a location in the diff, so it can't be submitted as a review comment.

5. fern/01-guide/06-prompt-engineering/rag.mdx:192

• Draft comment:
  Consider clarifying how citations are extracted in the prompt for RAGWithCitations. A brief note on the criteria for selecting cited quotes could aid users.
• Reason this comment was not posted:
  Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 10% vs. threshold = 50%
  The existing prompt is already clear and working well based on the example outputs shown. The suggested addition "ensuring they are directly relevant to the question" is implicit in the current behavior and doesn't add meaningful clarity. The examples demonstrate that the citations are already relevant to the questions without this additional instruction.
  Perhaps adding explicit criteria for citation relevance could help prevent edge cases where irrelevant citations are included. The current prompt relies on the LLM to infer what makes a citation relevant.
  The example outputs show the current prompt already produces relevant, well-formatted citations. Adding more words to the prompt without clear evidence of a problem would just make it more verbose.
  Delete the comment. The suggested change would make the prompt more verbose without demonstrably improving its effectiveness, and the current implementation is already working well as shown in the examples.

6. fern/01-guide/06-prompt-engineering/rag.mdx:230

• Draft comment:
  Ensure that the updated Python snippet using RAGWithCitations is accompanied by corresponding client updates so that the new function is available. This will help avoid runtime errors.
• Reason this comment was not posted:
  Comment did not seem useful. Confidence is useful = 0% <= threshold 50%
  The comment is asking the PR author to ensure that client updates are made to avoid runtime errors. This falls under the category of asking the author to ensure something is tested or verified, which is against the rules.

Workflow ID: wflow_S6cygXHmPSWVCA4p

---

Want Ellipsis to fix these issues? Tag @ellipsis-dev in a comment. You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

[ellipsis-dev]

Typo: In the heading ### Creating a VectoreStore, VectoreStore appears to be misspelled. It should likely be VectorStore for consistency with the code below.

Suggested change

	### Creating a VectoreStore
	### Creating a VectorStore

[ellipsis-dev]

👍 Looks good to me! Incremental review on f10a51f in 43 seconds

More details

• Looked at 13 lines of code in 1 files
• Skipped 0 files when reviewing.
• Skipped posting 3 drafted comments based on config settings.

1. fern/01-guide/06-prompt-engineering/rag.mdx:81

• Draft comment:
  Typo fixed: 'VectoreStore' -> 'VectorStore'.
• Reason this comment was not posted:
  Comment did not seem useful. Confidence is useful = 0% <= threshold 50%
  This comment is purely informative as it only points out a typo correction. It does not provide any actionable feedback or suggestions for improvement.

2. fern/01-guide/06-prompt-engineering/rag.mdx:81

• Draft comment:
  Typo fixed: updated 'VectoreStore' to 'VectorStore' for consistency.
• Reason this comment was not posted:
  Comment did not seem useful. Confidence is useful = 0% <= threshold 50%
  This comment is purely informative, as it only mentions a typo correction without suggesting any action or asking for confirmation. According to the rules, purely informative comments should be removed.

3. fern/01-guide/06-prompt-engineering/rag.mdx:72

• Draft comment:
  Typographical error: The context in the TestMissingContext test reads "...the best way to get structured with LLMs." It seems it is missing a word (perhaps 'outputs') after 'structured'. Please correct it for clarity.
• Reason this comment was not posted:
  Comment was not on a location in the diff, so it can't be submitted as a review comment.

Workflow ID: wflow_pOG1zUm3TgBlkHEA

---

You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.