Skip to content

Commit

Permalink
update rationale to reasoning for docs for 2.5+ default CoT behavior (#…
Browse files Browse the repository at this point in the history
  • Loading branch information
arnavsinghvi11 authored Nov 3, 2024
1 parent 7e78199 commit 3b688c6
Show file tree
Hide file tree
Showing 5 changed files with 37 additions and 21 deletions.
6 changes: 3 additions & 3 deletions docs/docs/building-blocks/2-signatures.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,10 +78,10 @@ The 21-year-old Lee made seven appearances and scored one goal for West Ham last

Many DSPy modules (except `dspy.Predict`) return auxiliary information by expanding your signature under the hood.

For example, `dspy.ChainOfThought` also adds a `rationale` field that includes the LM's reasoning before it generates the output `summary`.
For example, `dspy.ChainOfThought` also adds a `reasoning` field that includes the LM's reasoning before it generates the output `summary`.

```python
print("Rationale:", response.rationale)
print("Rationale:", response.reasoning)
```
**Output:**
```text
Expand Down Expand Up @@ -147,7 +147,7 @@ faithfulness(context=context, text=text)
**Output:**
```text
Prediction(
rationale="produce the faithfulness. We know that Lee had two loan spells in League One last term, with Blackpool and then Colchester United. He scored twice for the U's but was unable to save them from relegation. However, there is no mention of him scoring three goals for Colchester United.",
reasoning="produce the faithfulness. We know that Lee had two loan spells in League One last term, with Blackpool and then Colchester United. He scored twice for the U's but was unable to save them from relegation. However, there is no mention of him scoring three goals for Colchester United.",
faithfulness='False'
)
```
Expand Down
8 changes: 4 additions & 4 deletions docs/docs/building-blocks/3-modules.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,12 +67,12 @@ response.completions.answer

Let's discuss the output object here.

The `dspy.ChainOfThought` module will generally inject a `rationale` before the output field(s) of your signature.
The `dspy.ChainOfThought` module will generally inject a `reasoning` before the output field(s) of your signature.

Let's inspect the (first) rationale and answer!
Let's inspect the (first) reasoning and answer!

```python
print(f"Rationale: {response.rationale}")
print(f"Reasoning: {response.reasoning}")
print(f"Answer: {response.answer}")
```
**Output:**
Expand All @@ -86,7 +86,7 @@ This is accessible whether we request one or many completions.
We can also access the different completions as a list of `Prediction`s or as several lists, one for each field.

```python
response.completions[3].rationale == response.completions.rationale[3]
response.completions[3].reasoning == response.completions.reasoning[3]
```
**Output:**
```text
Expand Down
29 changes: 22 additions & 7 deletions docs/docs/deep-dive/modules/chain-of-thought.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,15 +13,30 @@ class ChainOfThought(Predict):

self.activated = activated

signature = ensure_signature(self.signature)
self.signature = signature = ensure_signature(signature)
*_keys, last_key = signature.output_fields.keys()

rationale_type = rationale_type or dspy.OutputField(
prefix="Reasoning: Let's think step by step in order to",
desc="${produce the " + last_key + "}. We ...",
)

self.extended_signature = signature.prepend("rationale", rationale_type, type_=str)
prefix = "Reasoning: Let's think step by step in order to"

if isinstance(dspy.settings.lm, dspy.LM):
desc = "${reasoning}"
elif dspy.settings.experimental:
desc = "${produce the output fields}. We ..."
else:
# For dspy <2.5
desc = f"${{produce the {last_key}}}. We ..."

rationale_type = rationale_type or dspy.OutputField(prefix=prefix, desc=desc)

# Add "rationale" field to the output signature.
if isinstance(dspy.settings.lm, dspy.LM) or dspy.settings.experimental:
extended_signature = signature.prepend("reasoning", rationale_type, type_=str)
else:
# For dspy <2.5
extended_signature = signature.prepend("rationale", rationale_type, type_=str)

self._predict = dspy.Predict(extended_signature, **config)
self._predict.extended_signature = extended_signature
```

**Parameters:**
Expand Down
13 changes: 7 additions & 6 deletions docs/docs/deep-dive/modules/guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,18 +87,19 @@ response.completions.answer

Let's discuss the output object here.

The `dspy.ChainOfThought` module will generally inject a `rationale` before the output field(s) of your signature.
The `dspy.ChainOfThought` module will generally inject a `reasoning` before the output field(s) of your signature.

Let's inspect the (first) rationale and answer!
Let's inspect the (first) reasoning and answer!


```python
print(f"Rationale: {response.rationale}")
print(f"Reasoning: {response.reasoning}")
print(f"Answer: {response.answer}")
```

Rationale: produce the answer. We can consider the fact that ColBERT has shown to outperform other state-of-the-art retrieval models in terms of efficiency and effectiveness. It uses contextualized embeddings and performs document retrieval in a way that is both accurate and scalable.
Answer: One great thing about the ColBERT retrieval model is its superior efficiency and effectiveness compared to other models.
Reasoning: ColBERT (Contextualized Late Interaction over BERT) is a retrieval model that effectively combines the strengths of dense retrieval and traditional BM25 methods. One of its great features is that it allows for efficient and scalable retrieval by using late interaction techniques, which enables the model to leverage the contextual embeddings generated by BERT while still maintaining a fast retrieval speed. This means that it can handle large document collections more effectively than many other models, providing both high relevance in search results and efficiency in processing time.
Answer: A great feature of the ColBERT retrieval model is its ability to efficiently combine contextualized embeddings from BERT with a late interaction mechanism, allowing for scalable and high-performance document retrieval.



This is accessible whether we request one or many completions.
Expand All @@ -107,7 +108,7 @@ We can also access the different completions as a list of `Prediction`s or as se


```python
response.completions[3].rationale == response.completions.rationale[3]
response.completions[3].reasoning == response.completions.reasoning[3]
```

```text
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/deep-dive/optimizers/copro.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ class CoTPipeline(dspy.Module):
result = self.predictor(question=question)
return dspy.Prediction(
answer=result.answer,
reasoning=result.rationale,
reasoning=result.reasoning,
)
```

Expand Down

0 comments on commit 3b688c6

Please sign in to comment.