Skip to content

Commit

Permalink
Update code injection docs to use cookiecutter
Browse files Browse the repository at this point in the history
  • Loading branch information
rjambrecic committed Nov 28, 2024
1 parent 93df27c commit 925c857
Show file tree
Hide file tree
Showing 4 changed files with 103 additions and 94 deletions.
4 changes: 2 additions & 2 deletions docs/docs/en/user-guide/api/code_injection/images/result.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
125 changes: 74 additions & 51 deletions docs/docs/en/user-guide/api/code_injection/index.md
Original file line number Diff line number Diff line change
@@ -1,122 +1,145 @@
# Code Injection

This guide explains how to integrate external functions into `AutoGen` agents using **code injection** with `FastAgency`. We'll create a banking agent that securely accesses user savings through injected parameters. The example includes setting up agents, registering functions, and facilitating conversations between agents.
Code injection is a way to securely connect external functions to agents in `AutoGen` without exposing sensitive data like passwords or tokens. This guide will show you how to use `FastAgency` to build workflows where sensitive information is handled safely, even when working with LLMs.

We’ll create a simple example: a banking agent that retrieves a user's balance. The cool part? Sensitive info like tokens is kept secure and is never shared with the language model. Instead, it’s injected directly into the function when needed.

## Why Use Code Injection?
When working with LLMs, security is a big deal. You don’t want things like passwords or tokens ending up in a conversation or being logged somewhere. Code injection solves this problem by keeping sensitive information out of reach while still letting your agents do their job.

Here’s what makes it useful:

- **It’s Secure**: Your private data stays private.
- **It’s Easy**: Functions can use secure data without needing a lot of extra setup.
- **It’s Flexible**: You can integrate all kinds of workflows safely.

In this guide, we’ll walk through setting everything up, creating a workflow, and running a secure application step-by-step. Let’s get started!

---

## Install

We **strongly recommend** using [**Cookiecutter**](../../../user-guide/cookiecutter/index.md) for setting up the project. Cookiecutter creates the project folder structure, default workflow, automatically installs all the necessary requirements, and creates a [devcontainer](https://code.visualstudio.com/docs/devcontainers/containers){target="_blank"} that can be used with [Visual Studio Code](https://code.visualstudio.com/){target="_blank"}.
We will use [**Cookiecutter**](../../../user-guide/cookiecutter/index.md) for setting up the project. Cookiecutter creates the project folder structure, default workflow, automatically installs all the necessary requirements, and creates a [devcontainer](https://code.visualstudio.com/docs/devcontainers/containers){target="_blank"} that can be used with [Visual Studio Code](https://code.visualstudio.com/){target="_blank"}.

You can setup the project using Cookiecutter by following the [**project setup guide**](../../../user-guide/cookiecutter/index.md).

Alternatively, you can use **pip + venv**. Before getting started, you need to install FastAgency with OpenAPI submodule. You can do this using `pip`, Python's package installer.
In this example, we will create Mesop application without authentication. To do so, select the following settings after running the `cookiecutter` command:

```console
pip install "fastagency[autogen,mesop]"
[1/5] project_name (My FastAgency App): My Bank App
[2/5] project_slug (my_bank_app):
[3/5] Select app_type
1 - fastapi+mesop
2 - mesop
3 - nats+fastapi+mesop
4 - fastapi
Choose from [1/2/3/4] (1): 2
[4/5] Select python_version
1 - 3.12
2 - 3.11
3 - 3.10
Choose from [1/2/3] (1): 2
[5/5] Select authentication
1 - basic
2 - google
3 - none
Choose from [1/2/3] (1): 3
```

## Complete Workflow Code
The only file you need to modify to run the application is `my_bank_app/workflow.py`. Simply copy and paste the following content into the file:

<details>
<summary>workflow.py</summary>
```python
{! docs_src/user_guide/code_injection/workflow.py !}
```
</details>

## Step-by-Step Guide

## Imports
### Imports
These imports are similar to the imports section we have already covered, with the only difference being the additional imports of the [**`inject_params`**](../../../api/fastagency/api/code_injection/inject_params.md) function:

```python hl_lines="8"
{! docs_src/user_guide/code_injection/mesop_main.py [ln:1-11] !}
```python hl_lines="7"
{! docs_src/user_guide/code_injection/workflow.py [ln:1-8] !}
```

## Define the Bank Savings Function
### Define the Bank Savings Function

The `get_savings` function is central to this workflow. It retrieves the user's savings based on the provided **bank** name and **token**.
The `get_balance` function is central to this workflow. It retrieves the user's balance based on the provided **username** name and **password**.

The key consideration here is that the **token** should NEVER be exposed to the LLM. Instead, the **token** will be securely injected into the `get_savings` function later in the workflow using the [**`inject_params`**](../../../api/fastagency/api/code_injection/inject_params.md) mechanism, ensuring that sensitive information remains confidential while still allowing the function to access the required data.
The key consideration here is that both username and password should **NEVER** be exposed to the LLM. Instead, they will be securely injected into the `get_balance` function later in the workflow using the [**`inject_params`**](../../../api/fastagency/api/code_injection/inject_params.md) mechanism, ensuring that sensitive information remains confidential while still allowing the function to access the required data.

```python
{! docs_src/user_guide/code_injection/mesop_main.py [ln:12-35] !}
{! docs_src/user_guide/code_injection/workflow.py [ln:10-23] !}
```


## Configure the Language Model (LLM)
### Configure the Language Model (LLM)
Here, the large language model is configured to use the `gpt-4o-mini` model, and the API key is retrieved from the environment. This setup ensures that both the user and weather agents can interact effectively.

```python
{! docs_src/user_guide/code_injection/mesop_main.py [ln:38-46] !}
{! docs_src/user_guide/code_injection/workflow.py [ln:26-34] !}
```

## Define the Workflow and Agents
### Define the Workflow and Agents

The `banking_workflow` handles user interaction and integrates agents to retrieve savings securely.
The `bank_workflow` handles user interaction and integrates agents to retrieve balance securely.


1. **User Input Collection**:
- At the beginning of the workflow, the user is prompted to provide:
- **Bank Name**: The workflow asks, *"Enter your bank"*.
- **Token**: The workflow then asks, *"Enter your token"*.
- **Username**: The workflow asks, *"Enter your username:"*.
- **Password**: The workflow then asks, *"Enter your password:"*.

2. **Agent Setup**:
- Two agents are created to handle the workflow:
- **UserProxyAgent**: Simulates the user's perspective, facilitating secure communication.
- **ConversableAgent**: Acts as the banker agent, retrieving savings information based on the user's input.
- **ConversableAgent**: Acts as the banker agent, retrieving the user's balance.

```python
{! docs_src/user_guide/code_injection/mesop_main.py [ln:49-76] !}
{! docs_src/user_guide/code_injection/workflow.py [ln:36-63] !}
```

## Code Injection
### Code Injection
The token provided by the user is stored securely in a **context dictionary (`ctx`)**.
This token is **never shared with the LLM** and is only used internally within the workflow.

Using [**`inject_params`**](../../../api/fastagency/api/code_injection/inject_params.md), the sensitive `token` from the `ctx` dictionary is injected into the `get_savings` function.
Using [**`inject_params`**](../../../api/fastagency/api/code_injection/inject_params.md), the sensitive `token` from the `ctx` dictionary is injected into the `get_balance` function.

```python
{! docs_src/user_guide/code_injection/mesop_main.py [ln:78-79] !}
{! docs_src/user_guide/code_injection/workflow.py [ln:65-69] !}
```

## Register Function with the Agents
In this step, we register the `get_savings_with_params`
### Register Function with the Agents
In this step, we register the `get_balance_with_params`
```python
{! docs_src/user_guide/code_injection/mesop_main.py [ln:80-85] !}
{! docs_src/user_guide/code_injection/workflow.py [ln:70-75] !}
```

## Enable Agent Interaction and Chat
Here, the user agent initiates a chat with the banker agent, which retrieves information about the savings. The conversation is summarized using a method provided by the LLM.
### Enable Agent Interaction and Chat
Here, the user agent initiates a chat with the banker agent, which retrieves the user's balance. The conversation is summarized using a method provided by the LLM.

```python
{! docs_src/user_guide/code_injection/mesop_main.py [ln:87-95] !}
{! docs_src/user_guide/code_injection/workflow.py [ln:77-84] !}
```

## Define FastAgency Application

Next, define your FastAgency application.

```python
{! docs_src/user_guide/code_injection/mesop_main.py [ln:98] !}
```

## Complete Application Code

<details>
<summary>main.py</summary>
```python
{! docs_src/user_guide/code_injection/mesop_main.py !}
```
</details>


## Run Application

You can run this chapter's FastAgency application using the following command:

```console
fastagency run
gunicorn my_bank_app.deployment.main:app
```

## Output
At the beginning, the user is asked to provide the **bank name** and **token**.
At the beginning, the user is asked to provide the **username** and **password**.

![User Input](./images/user_input.png)

Once the user provides the information, the agent executes the `get_savings` function with the **bank name** as a parameter.
The **token** is securely injected into the function using the [**`inject_params`**](../../../api/fastagency/api/code_injection/inject_params.md) mechanism, ensuring the token is not exposed to the LLM.
Once the user provides the information, the agent executes the `get_balance` function with both parameters securely injected into the function using the [**`inject_params`**](../../../api/fastagency/api/code_injection/inject_params.md) mechanism, ensuring the token is not exposed to the LLM.

The agent processes the request, retrieves the user's savings information, and provides a summary of the results without compromising sensitive data.
The agent processes the request, retrieves the user's balance, and provides a summary of the results without compromising sensitive data.

![Result](./images/result.png)
Original file line number Diff line number Diff line change
@@ -1,38 +1,26 @@
import os
from typing import Annotated, Any, Literal
from typing import Annotated, Any

from autogen import UserProxyAgent, register_function
from autogen.agentchat import ConversableAgent

from fastagency import UI, FastAgency
from fastagency import UI
from fastagency.api.code_injection import inject_params
from fastagency.runtimes.autogen import AutoGenWorkflows
from fastagency.ui.mesop import MesopUI

erste_tokens_amount_dict = {
"token-1-c": 100,
"token-1-f": 200,
}

rba_tokens_amount_dict = {
"token-1-a": 1_000,
"token-1-b": 20_000,
"token-2-a": -200,
}

bank_tokens_amount_dict = {
"erste": erste_tokens_amount_dict,
"rba": rba_tokens_amount_dict,
account_ballace_dict = {
("alice", "password123"): 100,
("bob", "password456"): 200,
("charlie", "password789"): 300,
}


def get_savings(
bank: Annotated[Literal["erste", "rba"], "Bank name: 'erste' or 'rba'"],
token: Annotated[str, "Token"],
def get_balance(
username: Annotated[str, "Username"],
password: Annotated[str, "Password"],
) -> str:
if token not in bank_tokens_amount_dict[bank]:
raise ValueError("Token not found")
return f"Your savings: {bank_tokens_amount_dict[bank][token]}$"
if (username.lower(), password) not in account_ballace_dict:
return "Invalid username or password"
return f"Your balance is {account_ballace_dict[(username, password)]}$"


llm_config = {
Expand All @@ -45,21 +33,20 @@ def get_savings(
"temperature": 0.8,
}


wf = AutoGenWorkflows()


@wf.register(name="banking_chat", description="Banking chat")
def banking_workflow(ui: UI, params: dict[str, str]) -> str:
bank = ui.text_input(
@wf.register(name="bank_chat", description="Bank chat") # type: ignore[misc]
def bank_workflow(ui: UI, params: dict[str, str]) -> str:
username = ui.text_input(

Check warning on line 41 in docs/docs_src/user_guide/code_injection/workflow.py

View check run for this annotation

Codecov / codecov/patch

docs/docs_src/user_guide/code_injection/workflow.py#L41

Added line #L41 was not covered by tests
sender="Workflow",
recipient="User",
prompt="Enter your bank:",
prompt="Enter your username:",
)
token = ui.text_input(
password = ui.text_input(
sender="Workflow",
recipient="User",
prompt="Enter your token:",
prompt="Enter your password:",
)

user_agent = UserProxyAgent(

Check warning on line 52 in docs/docs_src/user_guide/code_injection/workflow.py

View check run for this annotation

Codecov / codecov/patch

docs/docs_src/user_guide/code_injection/workflow.py#L52

Added line #L52 was not covered by tests
Expand All @@ -75,24 +62,23 @@ def banking_workflow(ui: UI, params: dict[str, str]) -> str:
human_input_mode="NEVER",
)

ctx: dict[str, Any] = {"token": token}
get_savings_with_params = inject_params(get_savings, ctx)
ctx: dict[str, Any] = {
"username": username,
"password": password,
}
get_balance_with_params = inject_params(get_balance, ctx)
register_function(

Check warning on line 70 in docs/docs_src/user_guide/code_injection/workflow.py

View check run for this annotation

Codecov / codecov/patch

docs/docs_src/user_guide/code_injection/workflow.py#L69-L70

Added lines #L69 - L70 were not covered by tests
f=get_savings_with_params,
f=get_balance_with_params,
caller=banker_agent,
executor=user_agent,
description="Get savings",
)

initial_message = f"We need to get user's savings for {bank}"
chat_result = user_agent.initiate_chat(

Check warning on line 77 in docs/docs_src/user_guide/code_injection/workflow.py

View check run for this annotation

Codecov / codecov/patch

docs/docs_src/user_guide/code_injection/workflow.py#L77

Added line #L77 was not covered by tests
banker_agent,
message=initial_message,
message="We need to get user's balance.",
summary_method="reflection_with_llm",
max_turns=3,
)

return chat_result.summary # type: ignore[no-any-return]

Check warning on line 84 in docs/docs_src/user_guide/code_injection/workflow.py

View check run for this annotation

Codecov / codecov/patch

docs/docs_src/user_guide/code_injection/workflow.py#L84

Added line #L84 was not covered by tests


app = FastAgency(provider=wf, ui=MesopUI())

0 comments on commit 925c857

Please sign in to comment.