[0.0.6] - 3-21-2024 Bug Fixes (#47)

* 🧪 asyncify tests * ✨ asyncify client * Basic Test for Page based pagination * add sync buildstep and client * add vscode DX * Added Testing for generators and updated examples * feat: example updates * readme exists now * Stylistic changes and generic message * Metamessages with other refactoring - untested * Work with unit tests * Fix Examples * MEME-78 Update Changelogs * Docstrings to client * 🧪 autogenerate sync tests * test one * add db type * sync client * add status badge * add coverage * add file * give perms * properly output coverage * split test and coverage * rename action * 🧪 autogenerate sync tests (#16) * Vector Support (#18) * Scaffold for PGVector support * Buggy crud with logic skeleton on api * Crud logic and schema definition for pgvector * Populate all routes and refactor to name Collection * vince's progress * AsyncCollection progress * Local PGVector Docker Container * client methods for sdk except document delete and update * Vector Support Passing All Test Cases * Docs Updates --------- Co-authored-by: vintro <[email protected]> * Add reverse parameters for paginated routes * Address dependabot * Formatting * initial commit on honcho dspy personas * working, hit token limit and can't test dspy optimization * initial version working, need to test optimization * optimizers working, but appending any example * ready for user object (tbomk) * Revert "add test actions and coverage" * Refactor to add User and App Tables * User Object passing test cases * Update examples * DSPy Todo and documentation updates * Add is_active filtering * Add is_active filtering to the generator * Fix update user metadata * working, but weird compiler error * fixed str error in optimizer * ship * sentry * Open Telemetry * optional logging with environment variables * add actions again? (#29) * add postgres * add openai key * readd coverage * desyncify and add detailed coverage * ⚙️ chore: update start script in VS Code to include poetry install --no-root before running uvicorn (#33) * Refactored code but need to tweak asyncpg * Working Async API using Psycopg3 * Update Workflow Connection URI * Update Workflow Connection URI in coverage test as well * Skeleton for Dialectic API * Fixes DEV-217 URL Encoding * Add Built-in Langchain Utility function * Sphinx Docs MVP * Metadata filtering for all fixes dev-261 * Basic Dialectic Endpoint fixes dev-253 * Working Fact Deriver * 0.0.5 Docs and README updates * Cloudflare Sphinx * update example to use right function (#36) * 🚀 feat: add support for running API using docker-compose with configurable environment variables and update docker-compose.yml for API and database services. (#34) Co-authored-by: Vineeth Voruganti <[email protected]> * add interrogate * routerify everything * full docstring coverage * remove unused imports and fix env issue * Update docker-compose connection uri and remove auto-stop to deriver process * Docstrings and langchain message converter in reverse * Sentry, OTEL, langchain both directions, fly.toml for deriver * Rename to deriver * Fix favicon and remove metadata from schema * 0.0.6 Notes * Changelog edit * Route bug fix * Route bug fix again --------- Co-authored-by: hyusap <[email protected]> Co-authored-by: vintro <[email protected]> Co-authored-by: vintro <[email protected]>
plastic-labs · Mar 21, 2024 · 29cd37b · 29cd37b
1 parent 663d065
commit 29cd37b
Show file tree

Hide file tree

Showing 39 changed files with 2,352 additions and 1,595 deletions.
diff --git a/.github/workflows/run_coverage.yml b/.github/workflows/run_coverage.yml
@@ -36,6 +36,9 @@ jobs:
           poetry install
           poetry run coverage run -m pytest
           poetry run coverage report --format=markdown > coverage.md
+          echo -e "\n---\n# Docstring Coverage\n\`\`\`" >> coverage.md
+          poetry run interrogate -v honcho >> coverage.md
+          echo -e "\`\`\`" >> coverage.md
           cd ..
       - name: Add Coverage PR Comment
         uses: marocchino/sticky-pull-request-comment@v2

diff --git a/.gitignore b/.gitignore
@@ -1,5 +1,6 @@
 api/**/*.db
-
+api/data
+api/docker-compose.yml
 
 
 # Byte-compiled / optimized / DLL files

diff --git a/README.md b/README.md
@@ -1,17 +1,30 @@
-# Honcho
-![Static Badge](https://img.shields.io/badge/Version-0.0.5-blue)
+# 🫡 Honcho
+![Static Badge](https://img.shields.io/badge/Version-0.0.6-blue)
 [![Discord](https://img.shields.io/discord/1016845111637839922?style=flat&logo=discord&logoColor=23ffffff&label=Plastic%20Labs&labelColor=235865F2)](https://discord.gg/plasticlabs)
 ![GitHub License](https://img.shields.io/github/license/plastic-labs/honcho)
 ![GitHub Repo stars](https://img.shields.io/github/stars/plastic-labs/honcho)
 [![X (formerly Twitter) URL](https://img.shields.io/twitter/url?url=https%3A%2F%2Ftwitter.com%2Fplastic_labs)](https://twitter.com/plastic_labs)
 
-A User context management solution for building AI Agents and LLM powered
-applications.
+Honcho is a platform for making AI agents and LLM powered applications that are personalized
+to their end users.
 
 Read about the motivation of this project [here](https://blog.plasticlabs.ai).
 
 Read the user documenation [here](https://docs.honcho.dev)
 
+## Table of Contents
+
+- [Project Structure](#project-structure)
+- [Usage](#usage)
+    - [API](#api)
+        - [Docker](#docker)
+        - [Manually](#manually)
+        - [Deploying on Fly.io](#deploy-on-fly)
+    - [Client SDK](#client-sdk)
+        - [Use Locally](#use-locally) 
+- [Contributing](#contributing)
+- [License](#license)
+
 ## Project Structure
 
 The Honcho repo is a monorepo containing the server/API that manages database
@@ -35,6 +48,19 @@ directories.
 
 ### API
 
+#### Docker
+
+The API can be run using docker-compose. The `docker-compose.yml.example` file can be copied to `docker-compose.yml` and the environment variables can be set in the `.env` file.
+
+```bash
+cd honcho/api
+cp docker-compose.yml.example docker-compose.yml
+[ update the file with openai key and other wanted environment variables ]
+docker compose up -d
+```
+
+#### Manually
+
 The API can be run either by installing the necessary dependencies and then
 specifying the appropriate environment variables.
 
@@ -48,7 +74,7 @@ poetry install # install dependencies
 
 2. Copy the `.env.template` file and specify the type of database and
    connection_uri. For testing sqlite is fine. The below example uses an
-   in-memory sqlite database. 
+   in-memory sqlite database.
 
 > Honcho has been tested with Postgresql and PGVector
 
@@ -72,26 +98,11 @@ poetry shell # Activate virtual environment if not already enabled
 python -m uvicorn src.main:app --reload
 ```
 
-#### Docker
-
-Alternatively there is also a `Dockerfile` included to run the API server from a
-docker container.
-
-The `.env` file is not loaded into the docker container and should still be
-configured from outside.
-
-```bash
-cd honcho/api
-docker build -t honcho-api .
-docker run --env-file .env -p 8000:8000 honcho-api:latest
-```
-
 #### Deploy on Fly
 
 The API can also be deployed on fly.io. Follow the [Fly.io
 Docs](https://fly.io/docs/getting-started/) to setup your environment and the
-`flyctl`. 
-
+`flyctl`.
 
 Once `flyctl` is set up use the following commands to launch the application:
 
@@ -116,7 +127,10 @@ alternatively if you are using poetry run:
 poetry add honcho-ai
 ```
 
-checkout the [example folder](./example/) for examples of how to use the sdk
+checkout the [SDK Reference](https://api.python.honcho.dev) for a detailed
+look at the different methods and how to use them. 
+
+Also, check out the[example folder](./example/) for examples of how to use the sdk 
 
 #### Use Locally
 
@@ -134,12 +148,12 @@ See more information [here](https://python-poetry.org/docs/cli/#add)
 This project is completely open source and welcomes any and all open source
 contributions. The workflow for contributing is to make a fork of the
 repository. You can claim an issue in the issues tab or start a new thread to
-indicate a feature or bug fix you are working on. 
+indicate a feature or bug fix you are working on.
 
 Once you have finished your contribution make a PR pointed at the `staging`
 branch, and it will be reviewed by a project manager. Feel free to join us in
 our [discord](http://discord.gg/plasticlabs) to discuss your changes or get
-help. 
+help.
 
 Once your changes are accepted and merged into staging they will undergo a
 period of live testing before entering the upstream into `main`

diff --git a/api/CHANGELOG.md b/api/CHANGELOG.md
@@ -4,6 +4,23 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](http://keepachangelog.com/)
 and this project adheres to [Semantic Versioning](http://semver.org/).
 
+## [0.0.6] — 2024-03-21
+
+### Added
+
+* Full docker-compose for API and Database 
+
+### Fixed
+
+* API Response schema removed unnecessary fields
+* OTEL logging to properly work with async database engine
+* `fly.toml` default settings for deriver set `auto_stop=false`
+
+### Changed
+
+* Refactored API server into multiple route files
+
+
 ## [0.0.5] — 2024-03-14
 
 ### Added

diff --git a/api/docker-compose.yml.example b/api/docker-compose.yml.example
@@ -0,0 +1,41 @@
+version: "3.8"
+services:
+  api:
+    build:
+      context: .
+      dockerfile: Dockerfile
+    ports:
+      - 8000:8000
+    volumes:
+      - .:/app
+    environment:
+      - DATABASE_TYPE=postgres
+      - CONNECTION_URI=postgresql+psycopg://testuser:testpwd@database:5432/honcho
+      - OPENAI_API_KEY=[YOUR_OPENAI_API_KEY]
+      - OPENTELEMETRY_ENABLED=false
+      - SENTRY_ENABLED=false
+      - SENTRY_DSN=
+      - OTEL_SERVICE_NAME=honcho
+      - OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED=true
+      - OTEL_PYTHON_LOG_CORRELATION=true
+      - OTEL_PYTHON_LOG_LEVEL=
+      - OTEL_EXPORTER_OTLP_PROTOCOL=
+      - OTEL_EXPORTER_OTLP_ENDPOINT=
+      - OTEL_EXPORTER_OTLP_HEADERS=
+      - OTEL_RESOURCE_ATTRIBUTES=
+      - DEBUG_LOG_OTEL_TO_PROVIDER=false
+      - DEBUG_LOG_OTEL_TO_CONSOLE=true
+  database:
+    image: ankane/pgvector
+    restart: always
+    ports:
+      - 5432:5432
+    environment:
+      - POSTGRES_DB=honcho
+      - POSTGRES_USER=testuser
+      - POSTGRES_PASSWORD=testpwd
+      - POSTGRES_HOST_AUTH_METHOD=trust
+      - PGDATA=/var/lib/postgresql/data/pgdata
+    volumes:
+      - ./local/init.sql:/docker-entrypoint-initdb.d/init.sql
+      - ./data:/var/lib/postgresql/data/
diff --git a/api/fly.toml b/api/fly.toml
@@ -14,7 +14,17 @@ kill_timeout = "5s"
 
 [processes]
   api = "python -m uvicorn src.main:app --host 0.0.0.0 --port 8000"
-  deriver = "python -m src.harvester"
+  deriver = "python -m src.deriver"
+
+[[services]]
+  auto_stop_machines = false
+  auto_start_machines = true
+  min_machines_running = 1
+  processes = ["deriver"]
+  protocol = "tcp"
+  [services.concurrency]
+    hard_limit = 250
+    soft_limit = 200
 
 [http_service]
   internal_port = 8000
@@ -31,4 +41,4 @@ kill_timeout = "5s"
   cpu_kind = "shared"
   cpus = 1
   memory_mb = 512
-  processes = ["api"]
+  processes = ["api", "deriver"]
diff --git a/api/pyproject.toml b/api/pyproject.toml
@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "honcho"
-version = "0.0.5"
+version = "0.0.6"
 description = "Honcho Server"
 authors = ["Plastic Labs <[email protected]>"]
 readme = "README.md"

diff --git a/api/src/db.py b/api/src/db.py
@@ -5,7 +5,7 @@
 from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
 from sqlalchemy.ext.declarative import declarative_base
 
-load_dotenv()
+load_dotenv(override=True)
 
 connect_args = {}
 

diff --git a/api/src/dependencies.py b/api/src/dependencies.py
@@ -0,0 +1,15 @@
+from fastapi import Depends
+from .db import SessionLocal
+from sqlalchemy.ext.asyncio import AsyncSession
+
+
+async def get_db():
+    """FastAPI Dependency Generator for Database"""
+    db: AsyncSession = SessionLocal()
+    try:
+        yield db
+    finally:
+        await db.close()
+
+
+db: AsyncSession = Depends(get_db)