From 0abe269b00ccbd74bbeed09c05925a0518a525a3 Mon Sep 17 00:00:00 2001 From: Diwank Singh Tomer Date: Wed, 2 Oct 2024 17:21:27 -0400 Subject: [PATCH 1/2] doc: Update README.md to add a callout for hacktoberfest people Signed-off-by: Diwank Singh Tomer --- README-CN.md | 29 +++++++++++- README.md | 124 +++++++++++---------------------------------------- 2 files changed, 54 insertions(+), 99 deletions(-) diff --git a/README-CN.md b/README-CN.md index 22bd8cb81..8dca6dcbb 100644 --- a/README-CN.md +++ b/README-CN.md @@ -26,6 +26,30 @@ GitHub 许可证

+***** + +## 🌟 诚邀贡献者！ + +我们很高兴欢迎新的贡献者加入 Julep 项目！我们创建了几个"适合新手的问题"来帮助您入门。以下是您可以贡献的方式： + +1. 查看我们的 [CONTRIBUTING.md](CONTRIBUTING.md) 文件，了解如何贡献的指南。 +2. 浏览我们的[适合新手的问题](https://github.com/julep-ai/julep/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22)，找到一个您感兴趣的任务。 +3. 如果您有任何问题或需要帮助，请随时在我们的 [Discord](https://discord.com/invite/JTSBGRZrzj) 频道上联系我们。 + +您的贡献，无论大小，对我们都很宝贵。让我们一起创造令人惊叹的东西吧！🚀 + +### 🎉 DevFest.AI 2024年10月 + +激动人心的消息！我们将在整个2024年10月参与 DevFest.AI 活动！🗓️ + +- 在此活动期间为 Julep 做出贡献，有机会赢得超棒的 Julep 周边和礼品！🎁 +- 加入来自世界各地的开发者，为 AI 仓库做出贡献并参与精彩的活动。 +- 非常感谢 DevFest.AI 组织这个fantastic的活动！ + +> [!TIP] +> 准备好加入这场盛会了吗？**[发推文开始参与](https://twitter.com/intent/tweet?text=Pumped%20to%20be%20participating%20in%20%40devfestai%20with%20%40julep_ai%20building%20%23ai%20%23agents%20%23workflows%20Let's%20gooo!%20https%3A%2F%2Fgit.new%2Fjulep)**，让我们开始编码吧！🖥️ + +![Julep DevFest.AI](https://media.giphy.com/media/YjyUeyotft6epaMHtU/giphy.gif) ***** @@ -50,6 +74,9 @@ 请继续关注我们即将发布的稳定版本的更多更新！📢 + +***** + ## 简介 Julep 是一个开源平台，用于创建具有可定制工作流的持久 AI 代理。它提供了开发、管理和部署 AI 驱动应用程序的工具，注重灵活性和易用性。 @@ -634,4 +661,4 @@ results = client.documents.search( ## 致谢 -我们要感谢所有贡献者和开源社区为他们宝贵的资源和贡献。 +我们要感谢所有贡献者和开源社区为他们宝贵的资源和贡献。 \ No newline at end of file diff --git a/README.md b/README.md index 37a3651be..af87426b1 100644 --- a/README.md +++ b/README.md @@ -7,104 +7,6 @@ The **Quick Start Guide Focused README** is the most promising for optimizing the time to first workflow. It allows developers to get hands-on experience quickly, which is essential for engagement and understanding. -* * * - -**Outline for the README:** - -1. **Title and Badges** - * Julep Logo or Title - * Build status, npm version, license badges -2. **Introduction** - * _Briefly explain what Julep is and its purpose._ - * Emphasize how it simplifies building persistent AI agents with workflows. -3. **Features** - * _Highlight key features with a focus on "tasks" (AI workflows)._ - * Mention support for persistent sessions, tool integration, and document management. -4. **Installation** - * _Provide npm installation command:_ - - bash - - Copy code - - `npm install julep` - -5. **Quick Start Guide** - * **Step 1: Import Julep** - * _Show how to import Julep into a project._ - - javascript - - Copy code - - `const Julep = require('julep');` - - * **Step 2: Initialize the Agent** - * _Guide on creating a new agent with basic settings._ - - javascript - - Copy code - - `const agent = new Julep.Agent({  name: 'MyAgent',  model: 'gpt-4-turbo', });` - - * **Step 3: Chat with the Agent** - * _Provide a simple example of a chat with the agent._ - - javascript - - Copy code - - `const response = await client.sessions.chat({  session_id: session.id,  message: 'Hello, how are you?' });` - - * **Step 4: Define a Task (Workflow)** - * _Provide a simple example of a task definition._ - - javascript - - Copy code - - `const task = {  name: 'GreetingTask',  main: [    {      prompt: 'Say hello to the user.',    },  ], }; agent.addTask(task);` - - * **Step 5: Execute the Task** - * _Show how to run the task and handle the output._ - - javascript - - Copy code - - `agent.executeTask('GreetingTask').then((output) => {  console.log(output); });` - -6. **Understanding Tasks** - * _Explain what tasks are and how they function within Julep._ - * Describe different types of workflow steps. - * Prompt, Tool Call, Evaluate, etc. - * _Note:_ Link to detailed documentation for each step type. -7. **Advanced Features** - * _Briefly mention other capabilities:_ - * Adding tools to agents. - * Managing sessions and users. - * Document integration and search. -8. **API Reference** - * _Link to full API documentation._ - * Mention key endpoints for agents, tasks, and executions. -9. **Examples and Tutorials** - * _Provide links to example projects or further tutorials._ - * Encourage users to explore and build upon provided examples. -10. **Contributing** - * _Instructions for contributing to the project._ - * Link to contribution guidelines and code of conduct. -11. **Support and Community** - * _Information on how to get help._ - * Links to community forums, chat groups, or issue trackers. -12. **License** - * _State the project's license._ - * Provide a link to the LICENSE file. -13. **Acknowledgements** - * _Credit to contributors and used resources._ - -* * * - **Notes:** * **Code Examples:** Ensure all code snippets are easy to understand and copy-paste ready. @@ -144,6 +46,31 @@ The **Quick Start Guide Focused README** is the most promising for optimizing th ***** +## 🌟 Call for Contributors! + +We're excited to welcome new contributors to the Julep project! We've created several "good first issues" to help you get started. Here's how you can contribute: + +1. Check out our [CONTRIBUTING.md](CONTRIBUTING.md) file for guidelines on how to contribute. +2. Browse our [good first issues](https://github.com/julep-ai/julep/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) to find a task that interests you. +3. If you have any questions or need help, don't hesitate to reach out on our [Discord](https://discord.com/invite/JTSBGRZrzj) channel. + +Your contributions, big or small, are valuable to us. Let's build something amazing together! 🚀 + +### 🎉 DevFest.AI October 2024 + +Exciting news! We're participating in DevFest.AI throughout October 2024! 🗓️ + +- Contribute to Julep during this event and get a chance to win awesome Julep merch and swag! 🎁 +- Join developers from around the world in contributing to AI repositories and participating in amazing events. +- A big thank you to DevFest.AI for organizing this fantastic initiative! + +> [!TIP] +> Ready to join the fun? **[Tweet to start participating](https://twitter.com/intent/tweet?text=Pumped%20to%20be%20participating%20in%20%40devfestai%20with%20%40julep_ai%20building%20%23ai%20%23agents%20%23workflows%20Let's%20gooo!%20https%3A%2F%2Fgit.new%2Fjulep)** and let's get coding! 🖥️ + +![Julep DevFest.AI](https://media.giphy.com/media/YjyUeyotft6epaMHtU/giphy.gif) + +***** + ## 🎉🚀 **Exciting News: Julep 1.0 Alpha Release!** 🚀🎉 We're thrilled to announce the **alpha** release of Julep 1.0! 🥳 @@ -165,6 +92,7 @@ We're thrilled to announce the **alpha** release of Julep 1.0! 🥳 Stay tuned for more updates as we approach our stable release! 📢 +***** ## Introduction From 46ed8facb35c1b17978bbb21bdf20246c19d3bf0 Mon Sep 17 00:00:00 2001 From: Hamada Salhab Date: Thu, 3 Oct 2024 00:47:50 +0300 Subject: [PATCH 2/2] feat(agents-api): Add static checking for Jinja templates & Python expressions in task creation | Add validation for subworkflows (#570) Closes #535 > [!IMPORTANT] > Add static validation for Python expressions and Jinja templates in task creation and validate subworkflows in `openapi_model.py` and `steps.tsp`. > > - **Validation**: > - Add `validate_python_expression()` and `validate_jinja_template()` in `openapi_model.py`. > - Integrate validation into `EvaluateStep`, `ToolCallStep`, `PromptStep`, `SetStep`, `LogStep`, `ReturnStep`, `YieldStep`, `IfElseWorkflowStep`, and `MapReduceStep` in `openapi_model.py`. > - **Models**: > - Update `CreateTaskRequest` in `openapi_model.py` to validate subworkflows using `WorkflowType`. > - Add `YieldStep` to `MappableWorkflowStep` and `NonConditionalWorkflowStep` in `steps.tsp`. > - **Misc**: > - Reorder `YieldStep` in `Tasks.py` to maintain consistency. > > This description was created by [Ellipsis](https://www.ellipsis.dev?ref=julep-ai%2Fjulep&utm_source=github&utm_medium=referral) for 9952ad5700812126c0aa7f1bfa26467e88b60aab. It will automatically update as commits are pushed. --------- Co-authored-by: Diwank Singh Tomer --- agents-api/agents_api/autogen/Tasks.py | 26 ++- .../agents_api/autogen/openapi_model.py | 216 +++++++++++++++++- typespec/tasks/steps.tsp | 3 +- 3 files changed, 233 insertions(+), 12 deletions(-) diff --git a/agents-api/agents_api/autogen/Tasks.py b/agents-api/agents_api/autogen/Tasks.py index 48dba4ad7..9dd531c47 100644 --- a/agents-api/agents_api/autogen/Tasks.py +++ b/agents-api/agents_api/autogen/Tasks.py @@ -35,10 +35,10 @@ class CaseThen(BaseModel): | LogStep | EmbedStep | SearchStep + | YieldStep | ReturnStep | SleepStep | ErrorWorkflowStep - | YieldStep | WaitForInputStep ) """ @@ -63,10 +63,10 @@ class CaseThenUpdateItem(BaseModel): | LogStep | EmbedStep | SearchStep + | YieldStep | ReturnStep | SleepStep | ErrorWorkflowStep - | YieldStep | WaitForInputStep ) """ @@ -130,10 +130,10 @@ class CreateTaskRequest(BaseModel): | LogStep | EmbedStep | SearchStep + | YieldStep | ReturnStep | SleepStep | ErrorWorkflowStep - | YieldStep | WaitForInputStep | IfElseWorkflowStep | SwitchStep @@ -227,6 +227,7 @@ class ForeachDo(BaseModel): | LogStep | EmbedStep | SearchStep + | YieldStep ) """ The steps to run for each iteration @@ -251,6 +252,7 @@ class ForeachDoUpdateItem(BaseModel): | LogStep | EmbedStep | SearchStep + | YieldStep ) """ The steps to run for each iteration @@ -324,10 +326,10 @@ class IfElseWorkflowStep(BaseModel): | LogStep | EmbedStep | SearchStep + | YieldStep | ReturnStep | SleepStep | ErrorWorkflowStep - | YieldStep | WaitForInputStep ) """ @@ -342,10 +344,10 @@ class IfElseWorkflowStep(BaseModel): | LogStep | EmbedStep | SearchStep + | YieldStep | ReturnStep | SleepStep | ErrorWorkflowStep - | YieldStep | WaitForInputStep | None, Field(None, alias="else"), @@ -376,10 +378,10 @@ class IfElseWorkflowStepUpdateItem(BaseModel): | LogStep | EmbedStep | SearchStep + | YieldStep | ReturnStep | SleepStep | ErrorWorkflowStep - | YieldStep | WaitForInputStep ) """ @@ -394,10 +396,10 @@ class IfElseWorkflowStepUpdateItem(BaseModel): | LogStep | EmbedStep | SearchStep + | YieldStep | ReturnStep | SleepStep | ErrorWorkflowStep - | YieldStep | WaitForInputStep | None, Field(None, alias="else"), @@ -462,6 +464,7 @@ class Main(BaseModel): | LogStep | EmbedStep | SearchStep + | YieldStep ) """ The steps to run for each iteration @@ -503,6 +506,7 @@ class MainModel(BaseModel): | LogStep | EmbedStep | SearchStep + | YieldStep ) """ The steps to run for each iteration @@ -543,6 +547,7 @@ class ParallelStep(BaseModel): | LogStep | EmbedStep | SearchStep + | YieldStep ], Field(max_length=100), ] @@ -569,6 +574,7 @@ class ParallelStepUpdateItem(BaseModel): | LogStep | EmbedStep | SearchStep + | YieldStep ], Field(max_length=100), ] @@ -596,10 +602,10 @@ class PatchTaskRequest(BaseModel): | LogStep | EmbedStep | SearchStep + | YieldStep | ReturnStep | SleepStep | ErrorWorkflowStep - | YieldStep | WaitForInputStep | IfElseWorkflowStepUpdateItem | SwitchStepUpdateItem @@ -874,10 +880,10 @@ class Task(BaseModel): | LogStep | EmbedStep | SearchStep + | YieldStep | ReturnStep | SleepStep | ErrorWorkflowStep - | YieldStep | WaitForInputStep | IfElseWorkflowStep | SwitchStep @@ -1009,10 +1015,10 @@ class UpdateTaskRequest(BaseModel): | LogStep | EmbedStep | SearchStep + | YieldStep | ReturnStep | SleepStep | ErrorWorkflowStep - | YieldStep | WaitForInputStep | IfElseWorkflowStep | SwitchStep diff --git a/agents-api/agents_api/autogen/openapi_model.py b/agents-api/agents_api/autogen/openapi_model.py index 5c5a8c86f..48811eb20 100644 --- a/agents-api/agents_api/autogen/openapi_model.py +++ b/agents-api/agents_api/autogen/openapi_model.py @@ -1,10 +1,12 @@ # ruff: noqa: F401, F403, F405 +import ast from typing import Annotated, Any, Generic, Literal, Self, Type, TypeVar, get_args from uuid import UUID +import jinja2 from litellm.utils import _select_tokenizer as select_tokenizer from litellm.utils import token_counter -from pydantic import AwareDatetime, Field +from pydantic import AwareDatetime, Field, field_validator, model_validator, validator from ..common.utils.datetime import utcnow from .Agents import * @@ -152,6 +154,179 @@ def from_model_input( ) +# Patch Task Workflow Steps +# -------------------------------------- + + +def validate_python_expression(expr: str) -> tuple[bool, str]: + try: + ast.parse(expr) + return True, "" + except SyntaxError as e: + return False, f"SyntaxError in '{expr}': {str(e)}" + + +def validate_jinja_template(template: str) -> tuple[bool, str]: + env = jinja2.Environment() + try: + parsed_template = env.parse(template) + for node in parsed_template.body: + if isinstance(node, jinja2.nodes.Output): + for child in node.nodes: + if isinstance(child, jinja2.nodes.Name): + # Check if the variable is a valid Python expression + is_valid, error = validate_python_expression(child.name) + if not is_valid: + return ( + False, + f"Invalid Python expression in Jinja template '{template}': {error}", + ) + return True, "" + except jinja2.exceptions.TemplateSyntaxError as e: + return False, f"TemplateSyntaxError in '{template}': {str(e)}" + + +_EvaluateStep = EvaluateStep + + +class EvaluateStep(_EvaluateStep): + @field_validator("evaluate") + def validate_evaluate_expressions(cls, v): + for key, expr in v.items(): + is_valid, error = validate_python_expression(expr) + if not is_valid: + raise ValueError(f"Invalid Python expression in key '{key}': {error}") + return v + + +_ToolCallStep = ToolCallStep + + +class ToolCallStep(_ToolCallStep): + @field_validator("arguments") + def validate_arguments(cls, v): + if isinstance(v, dict): + for key, expr in v.items(): + if isinstance(expr, str): + is_valid, error = validate_python_expression(expr) + if not is_valid: + raise ValueError( + f"Invalid Python expression in arguments key '{key}': {error}" + ) + return v + + +_PromptStep = PromptStep + + +class PromptStep(_PromptStep): + @field_validator("prompt") + def validate_prompt(cls, v): + if isinstance(v, str): + is_valid, error = validate_jinja_template(v) + if not is_valid: + raise ValueError(f"Invalid Jinja template in prompt: {error}") + elif isinstance(v, list): + for item in v: + if "content" in item: + is_valid, error = validate_jinja_template(item["content"]) + if not is_valid: + raise ValueError( + f"Invalid Jinja template in prompt content: {error}" + ) + return v + + +_SetStep = SetStep + + +class SetStep(_SetStep): + @field_validator("set") + def validate_set_expressions(cls, v): + for key, expr in v.items(): + is_valid, error = validate_python_expression(expr) + if not is_valid: + raise ValueError( + f"Invalid Python expression in set key '{key}': {error}" + ) + return v + + +_LogStep = LogStep + + +class LogStep(_LogStep): + @field_validator("log") + def validate_log_template(cls, v): + is_valid, error = validate_jinja_template(v) + if not is_valid: + raise ValueError(f"Invalid Jinja template in log: {error}") + return v + + +_ReturnStep = ReturnStep + + +class ReturnStep(_ReturnStep): + @field_validator("return_") + def validate_return_expressions(cls, v): + for key, expr in v.items(): + is_valid, error = validate_python_expression(expr) + if not is_valid: + raise ValueError( + f"Invalid Python expression in return key '{key}': {error}" + ) + return v + + +_YieldStep = YieldStep + + +class YieldStep(_YieldStep): + @field_validator("arguments") + def validate_yield_arguments(cls, v): + if isinstance(v, dict): + for key, expr in v.items(): + is_valid, error = validate_python_expression(expr) + if not is_valid: + raise ValueError( + f"Invalid Python expression in yield arguments key '{key}': {error}" + ) + return v + + +_IfElseWorkflowStep = IfElseWorkflowStep + + +class IfElseWorkflowStep(_IfElseWorkflowStep): + @field_validator("if_") + def validate_if_expression(cls, v): + is_valid, error = validate_python_expression(v) + if not is_valid: + raise ValueError(f"Invalid Python expression in if condition: {error}") + return v + + +_MapReduceStep = MapReduceStep + + +class MapReduceStep(_MapReduceStep): + @field_validator("over") + def validate_over_expression(cls, v): + is_valid, error = validate_python_expression(v) + if not is_valid: + raise ValueError(f"Invalid Python expression in over: {error}") + return v + + @field_validator("reduce") + def validate_reduce_expression(cls, v): + if v is not None: + is_valid, error = validate_python_expression(v) + if not is_valid: + raise ValueError(f"Invalid Python expression in reduce: {error}") + return v + + # Workflow related models # ----------------------- @@ -228,6 +403,29 @@ class Task(_Task): # Patch some models to allow extra fields # -------------------------------------- +WorkflowType = RootModel[ + list[ + EvaluateStep + | ToolCallStep + | PromptStep + | GetStep + | SetStep + | LogStep + | EmbedStep + | SearchStep + | ReturnStep + | SleepStep + | ErrorWorkflowStep + | YieldStep + | WaitForInputStep + | IfElseWorkflowStep + | SwitchStep + | ForeachStep + | ParallelStep + | MapReduceStep + ] +] + _CreateTaskRequest = CreateTaskRequest @@ -240,6 +438,22 @@ class CreateTaskRequest(_CreateTaskRequest): } ) + @model_validator(mode="after") + def validate_subworkflows(self) -> Self: + subworkflows = { + k: v + for k, v in self.model_dump().items() + if k not in _CreateTaskRequest.model_fields + } + + for workflow_name, workflow_definition in subworkflows.items(): + try: + WorkflowType.model_validate(workflow_definition) + setattr(self, workflow_name, WorkflowType(workflow_definition)) + except Exception as e: + raise ValueError(f"Invalid subworkflow '{workflow_name}': {str(e)}") + return self + CreateOrUpdateTaskRequest = CreateTaskRequest diff --git a/typespec/tasks/steps.tsp b/typespec/tasks/steps.tsp index 3495def1b..2267ae320 100644 --- a/typespec/tasks/steps.tsp +++ b/typespec/tasks/steps.tsp @@ -49,7 +49,8 @@ alias MappableWorkflowStep = | SetStep | LogStep | EmbedStep - | SearchStep; + | SearchStep + | YieldStep; alias NonConditionalWorkflowStep = | MappableWorkflowStep