Merge branch 'julep-ai:dev' into dev

CONTRIBUTING.md

@@ -22,6 +22,115 @@ To contribute code changes:
 
 Please ensure your code follows the existing style and passes all tests.
 
+## Project Overview and Architecture
+
+### Key Components
+
+1. **agents-api**: The core API service for Julep.
+2. **typespec**: API specifications and contracts.
+3. **integrations-service**: Handles external integrations.
+4. **embedding-service**: Manages text embeddings.
+5. **memory-store**: Handles persistent storage.
+6. **llm-proxy**: Proxy for language model interactions.
+7. **scheduler**: Manages task scheduling.
+8. **gateway**: API gateway and routing.
+9. **monitoring**: System monitoring and metrics.
+
+### Technology Stack
+
+- **FastAPI**: Web framework for building APIs
+- **TypeSpec**: API specification language
+- **Cozo**: Database system
+- **Temporal**: Workflow engine
+- **Docker**: Containerization
+
+### Relationships Between Components
+
+The `agents-api` serves as the central component, interacting with most other services:
+- It uses `typespec` definitions for API contracts.
+- Communicates with `integrations-service` for external tool interactions.
+- Utilizes `embedding-service` for text processing.
+- Stores data in `memory-store`.
+- Interacts with language models through `llm-proxy`.
+- Uses `scheduler` for task management.
+- All API requests pass through the `gateway`.
+- `monitoring` observes the entire system.
+
+## Understanding the Codebase
+
+To get a comprehensive understanding of Julep, we recommend exploring the codebase in the following order:
+
+1. **Project Overview**
+   - Read `README.md` in the root directory
+   - Explore `docs/` for detailed documentation
+
+2. **System Architecture**
+   - Examine `docker-compose.yml` in the root directory
+   - Review `deploy/` directory for different deployment configurations
+
+3. **API Specifications**
+   - Learn about TypeSpec: https://typespec.io/docs/
+   - Explore `typespec/` directory:
+     - Start with `common/` folder
+     - Review `main.tsp`
+     - Examine each module sequentially
+
+4. **Core API Implementation**
+   - Learn about FastAPI: https://fastapi.tiangolo.com/
+   - Explore `agents-api/` directory:
+     - Review `README.md` for an overview
+     - Examine `routers/` for API endpoints
+     - Look into `models/` for data models
+
+5. **Database and Storage**
+   - Learn about Cozo: https://docs.cozodb.org/en/latest/tutorial.html
+   - Review `agents-api/README.md` for database schema
+   - Explore `agents-api/models/` for database queries
+
+6. **Workflow Management**
+   - Learn about Temporal: https://docs.temporal.io/develop/python
+   - Explore `agents-api/activities/` for individual workflow steps
+   - Review `agents-api/workflows/task_execution/` for main workflow logic
+
+7. **Testing**
+   - Examine `agents-api/tests/` for test cases
+
+8. **Additional Services**
+   - Explore other service directories (`integrations-service/`, `embedding-service/`, etc.) to understand their specific roles and implementations
+
+## Contributing Guidelines
+
+1. **Set Up Development Environment**
+   - Clone the repository
+   - Install Docker and Docker Compose
+   - Set up necessary API keys and environment variables
+
+2. **Choose an Area to Contribute**
+   - Check the issue tracker for open issues
+   - Look for "good first issue" labels for newcomers
+
+3. **Make Changes**
+   - Create a new branch for your changes
+   - Write clean, well-documented code
+   - Ensure your changes adhere to the project's coding standards
+
+4. **Test Your Changes**
+   - Run existing tests
+   - Add new tests for new functionality
+   - Ensure all tests pass before submitting your changes
+
+5. **Submit a Pull Request**
+   - Provide a clear description of your changes
+   - Reference any related issues
+   - Be prepared to respond to feedback and make adjustments
+
+6. **Code Review**
+   - Address any comments or suggestions from reviewers
+   - Make necessary changes and push updates to your branch
+
+7. **Merge**
+   - Once approved, your changes will be merged into the main branch
+
 ### Documentation Improvements 
 
 Improvements to documentation are always appreciated! If you see areas that could be clarified or expanded, feel free to make the changes and submit a pull request.
@@ -30,44 +139,75 @@ Improvements to documentation are always appreciated! If you see areas that coul
 
 We'd love to hear your feedback and ideas for the project! Feel free to submit an issue or contact the maintainers directly to share your thoughts. Your input is very valuable in shaping the future direction of the project.
 
-## Building Docker Images with Buildx Bake
+### Setup Instructions
+
+##### 1. Clone the Repository
+Clone the repository from your preferred source:
+
+```bash
+git clone <repository_url>
+```
+
+##### 2. Navigate to the Root Directory
+Change to the root directory of the project:
+
+```bash
+cd <repository_root>
+```
+
+##### 3. Set Up Environment Variables
+- Create a `.env` file in the root directory.
+- Refer to the `.env.example` file for a list of required variables.
+- Ensure that all necessary variables are set in the `.env` file.
+
+##### 4. Create a Docker Volume for Backup
+Create a Docker volume named `cozo_backup`:
+
+```bash
+docker volume create cozo_backup
+```
+
+##### 5. Run the Project using Docker Compose
+You can run the project in two different modes: **Single Tenant** or **Multi-Tenant**. Choose one of the following commands based on your requirement:
+
+###### Single-Tenant Mode
+Run the project in single-tenant mode:
+
+```bash
+docker compose --env-file .env --profile temporal-ui --profile single-tenant --profile embedding-cpu --profile self-hosted-db up --force-recreate --build --watch
+```
+
+> **Note:** In single-tenant mode, you can interact with the SDK directly without the need for the API KEY.
+
+###### Multi-Tenant Mode
+Run the project in multi-tenant mode:
+
+```bash
+docker compose --env-file .env --profile temporal-ui --profile multi-tenant --profile embedding-cpu --profile self-hosted-db up --force-recreate --build --watch
+```
+
+> **Note:** In multi-tenant mode, you need to generate a JWT token locally that act as an API KEY to interact with the SDK.
 
-We use Docker Buildx Bake to build our Docker images. This allows us to build multiple images concurrently and efficiently. Follow these steps to build the Docker images:
+##### 6. Generate a JWT Token (Only for Multi-Tenant Mode)
 
-1. Ensure you have Docker and Docker Buildx installed on your system.
+To generate a JWT token, `jwt-cli` is required. Kindly install the same before proceeding with the next steps.
 
-2. Navigate to the root directory of the project where the `docker-bake.hcl` file is located.
+Use the following command and replace `JWT_SHARED_KEY` with the corresponding key from your `.env` file to generate a JWT token:
 
-3. To build all services, run:
-   ```
-   docker buildx bake --file docker-bake.hcl
-   ```
+```bash
+jwt encode --secret JWT_SHARED_KEY --alg HS512 --exp=$(date -j -v +10d +%s) --sub '00000000-0000-0000-0000-000000000000' '{}'
+```
 
-4. To build a specific service, use:
-   ```
-   docker buildx bake --file docker-bake.hcl <service-name>
-   ```
-   Replace `<service-name>` with one of the following:
-   - agents-api
-   - agents-api-worker
-   - cozo-migrate
-   - memory-store
-   - integrations
-   - gateway
-   - embedding-service-cpu
-   - embedding-service-gpu
+This command generates a JWT token that will be valid for 10 days.
 
-5. To set a custom tag for the images, use:
-   ```
-   docker buildx bake --file docker-bake.hcl --set *.tags=myorg/myimage:v1.0
-   ```
-   Replace `myorg/myimage:v1.0` with your desired image name and tag.
+##### 7. Access and Interact
+- **Temporal UI**: You can access the Temporal UI through the specified port in your `.env` file.
+- **API Interactions**: Depending on the chosen mode, interact with the setup using the provided endpoints.
 
-6. By default, the images are built with the "latest" tag. To specify a different tag, you can set the TAG variable:
-   ```
-   docker buildx bake --file docker-bake.hcl --set TAG=v1.2.3
-   ```
+##### Troubleshooting
+- Ensure that all required Docker images are available.
+- Check for missing environment variables in the `.env` file.
+- Use the `docker compose logs` command to view detailed logs for debugging.
 
-Note: The `docker-bake.hcl` file defines the build contexts, Dockerfiles, and tags for each service. If you need to modify the build process for a specific service, update the corresponding target in the `docker-bake.hcl` file.
 
-Thank you for your interest in contributing to this project!
+Remember, contributions aren't limited to code. Documentation improvements, bug reports, and feature suggestions are also valuable contributions to the project.

README-CN.md

@@ -26,6 +26,30 @@
     <a href="https://choosealicense.com/licenses/apache/"><img src="https://img.shields.io/github/license/julep-ai/julep" alt="GitHub 许可证"></a>
 </p>
 
+*****
+
+## 🌟 诚邀贡献者！
+
+我们很高兴欢迎新的贡献者加入 Julep 项目！我们创建了几个"适合新手的问题"来帮助您入门。以下是您可以贡献的方式：
+
+1. 查看我们的 [CONTRIBUTING.md](CONTRIBUTING.md) 文件，了解如何贡献的指南。
+2. 浏览我们的[适合新手的问题](https://github.com/julep-ai/julep/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22)，找到一个您感兴趣的任务。
+3. 如果您有任何问题或需要帮助，请随时在我们的 [Discord](https://discord.com/invite/JTSBGRZrzj) 频道上联系我们。
+
+您的贡献，无论大小，对我们都很宝贵。让我们一起创造令人惊叹的东西吧！🚀
+
+### 🎉 DevFest.AI 2024年10月
+
+激动人心的消息！我们将在整个2024年10月参与 DevFest.AI 活动！🗓️
+
+- 在此活动期间为 Julep 做出贡献，有机会赢得超棒的 Julep 周边和礼品！🎁
+- 加入来自世界各地的开发者，为 AI 仓库做出贡献并参与精彩的活动。
+- 非常感谢 DevFest.AI 组织这个fantastic的活动！
+
+> [!TIP]
+> 准备好加入这场盛会了吗？**[发推文开始参与](https://twitter.com/intent/tweet?text=Pumped%20to%20be%20participating%20in%20%40devfestai%20with%20%40julep_ai%20building%20%23ai%20%23agents%20%23workflows%20Let's%20gooo!%20https%3A%2F%2Fgit.new%2Fjulep)**，让我们开始编码吧！🖥️
+
+![Julep DevFest.AI](https://media.giphy.com/media/YjyUeyotft6epaMHtU/giphy.gif)
 
 *****
 
@@ -50,6 +74,9 @@
 
 请继续关注我们即将发布的稳定版本的更多更新！📢
 
+
+*****
+
 ## 简介
 
 Julep 是一个开源平台，用于创建具有可定制工作流的持久 AI 代理。它提供了开发、管理和部署 AI 驱动应用程序的工具，注重灵活性和易用性。
@@ -634,4 +661,4 @@ results = client.documents.search(
 
 ## 致谢
 
-我们要感谢所有贡献者和开源社区为他们宝贵的资源和贡献。
+我们要感谢所有贡献者和开源社区为他们宝贵的资源和贡献。