Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(cli): Package cli for execution in dependent projects #164

Merged
merged 2 commits into from
Feb 5, 2025

Conversation

planetf1
Copy link
Collaborator

@planetf1 planetf1 commented Jan 31, 2025

  • Enables projects to specify a dependency on the bee project and run 'beeAI' directly

Detail:

  • Correct scope of some imports to work portably when picked up as a dependency (ie cli.XX)
  • added script definition to poetry config (poetry will automatically create shell script)
  • minor refactor to beeAI.py to move code to callable function (didn't work directly in main)

Example

In a new project which uses bee-hive, we can simply add bee-hive as a dependency:

[tool.poetry.dependencies]
python = ">= 3.11, < 3.13"
bee_hive = { git = "ssh://[email protected]/planetf1/bee-hive", subdirectory="bee-hive", branch="dev" }

Then in this new project, the following command works after a poetry install

(.venv) ➜  bee-hive git:(dev) beeAI validate file1 file2
validate file1 with schema file1

Use Case

  • Building demos in bee-hive-demos where a project can just express dependency & all the right things happen

Caveats

  • Example currently imports as git repo, and seems to pull in bee_hive & cli together - this may need to be separated - and seems to be causing an issue for dependent libraries currently. (
  • not yet checked original script, removed original script, or verified other use cases
  • current demo uses my fork/dev branch. When merged can use main repo/branch.
  • if using a private repo, authentication is needed (works with ssh-agent)

@planetf1
Copy link
Collaborator Author

Using pip the hope would be something like this works:

 pip install -e "git+https://github.com/planetf1/bee-hive.git@dev#egg=bee_hive&subdirectory=bee-hive"

@planetf1
Copy link
Collaborator Author

planetf1 commented Jan 31, 2025

Example in a clean environment:

➜  tmp3 python3.12 -m venv .venv                                                                                                        <<<
➜  tmp3 source .venv/bin/activate                                                                                                       <<<
(.venv) ➜  tmp3 pip install -e "git+https://github.com/planetf1/bee-hive.git@dev#egg=bee_hive&subdirectory=bee-hive"
Obtaining bee_hive from git+https://github.com/planetf1/bee-hive.git@dev#egg=bee_hive&subdirectory=bee-hive
  Cloning https://github.com/planetf1/bee-hive.git (to revision dev) to ./.venv/src/bee-hive
  Running command git clone --filter=blob:none --quiet https://github.com/planetf1/bee-hive.git /Users/jonesn/AI/tmp3/.venv/src/bee-hive
  Running command git checkout -b dev --track origin/dev
  Switched to a new branch 'dev'
  branch 'dev' set up to track 'origin/dev'.
  Resolved https://github.com/planetf1/bee-hive.git to commit 9d3f0ba99fcbd1d4d8096caebc2a4da270858b4c
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Preparing editable metadata (pyproject.toml) ... done
Collecting docopt-ng<0.10.0,>=0.9.0 (from bee_hive)
  Using cached docopt_ng-0.9.0-py3-none-any.whl.metadata (13 kB)
Collecting jsonschema<5.0.0,>=4.23.0 (from bee_hive)
  Using cached jsonschema-4.23.0-py3-none-any.whl.metadata (7.9 kB)
Collecting openai<2.0.0,>=1.56.2 (from bee_hive)
  Using cached openai-1.60.2-py3-none-any.whl.metadata (27 kB)
Collecting python-dotenv<2.0.0,>=1.0.1 (from bee_hive)
  Using cached python_dotenv-1.0.1-py3-none-any.whl.metadata (23 kB)
Collecting pyyaml<7.0.0,>=6.0.2 (from bee_hive)
  Using cached PyYAML-6.0.2-cp312-cp312-macosx_11_0_arm64.whl.metadata (2.1 kB)
Collecting attrs>=22.2.0 (from jsonschema<5.0.0,>=4.23.0->bee_hive)
  Using cached attrs-25.1.0-py3-none-any.whl.metadata (10 kB)
Collecting jsonschema-specifications>=2023.03.6 (from jsonschema<5.0.0,>=4.23.0->bee_hive)
  Using cached jsonschema_specifications-2024.10.1-py3-none-any.whl.metadata (3.0 kB)
Collecting referencing>=0.28.4 (from jsonschema<5.0.0,>=4.23.0->bee_hive)
  Using cached referencing-0.36.2-py3-none-any.whl.metadata (2.8 kB)
Collecting rpds-py>=0.7.1 (from jsonschema<5.0.0,>=4.23.0->bee_hive)
  Using cached rpds_py-0.22.3-cp312-cp312-macosx_11_0_arm64.whl.metadata (4.2 kB)
Collecting anyio<5,>=3.5.0 (from openai<2.0.0,>=1.56.2->bee_hive)
  Using cached anyio-4.8.0-py3-none-any.whl.metadata (4.6 kB)
Collecting distro<2,>=1.7.0 (from openai<2.0.0,>=1.56.2->bee_hive)
  Using cached distro-1.9.0-py3-none-any.whl.metadata (6.8 kB)
Collecting httpx<1,>=0.23.0 (from openai<2.0.0,>=1.56.2->bee_hive)
  Using cached httpx-0.28.1-py3-none-any.whl.metadata (7.1 kB)
Collecting jiter<1,>=0.4.0 (from openai<2.0.0,>=1.56.2->bee_hive)
  Using cached jiter-0.8.2-cp312-cp312-macosx_11_0_arm64.whl.metadata (5.2 kB)
Collecting pydantic<3,>=1.9.0 (from openai<2.0.0,>=1.56.2->bee_hive)
  Using cached pydantic-2.10.6-py3-none-any.whl.metadata (30 kB)
Collecting sniffio (from openai<2.0.0,>=1.56.2->bee_hive)
  Using cached sniffio-1.3.1-py3-none-any.whl.metadata (3.9 kB)
Collecting tqdm>4 (from openai<2.0.0,>=1.56.2->bee_hive)
  Using cached tqdm-4.67.1-py3-none-any.whl.metadata (57 kB)
Collecting typing-extensions<5,>=4.11 (from openai<2.0.0,>=1.56.2->bee_hive)
  Using cached typing_extensions-4.12.2-py3-none-any.whl.metadata (3.0 kB)
Collecting idna>=2.8 (from anyio<5,>=3.5.0->openai<2.0.0,>=1.56.2->bee_hive)
  Using cached idna-3.10-py3-none-any.whl.metadata (10 kB)
Collecting certifi (from httpx<1,>=0.23.0->openai<2.0.0,>=1.56.2->bee_hive)
  Using cached certifi-2025.1.31-py3-none-any.whl.metadata (2.5 kB)
Collecting httpcore==1.* (from httpx<1,>=0.23.0->openai<2.0.0,>=1.56.2->bee_hive)
  Using cached httpcore-1.0.7-py3-none-any.whl.metadata (21 kB)
Collecting h11<0.15,>=0.13 (from httpcore==1.*->httpx<1,>=0.23.0->openai<2.0.0,>=1.56.2->bee_hive)
  Using cached h11-0.14.0-py3-none-any.whl.metadata (8.2 kB)
Collecting annotated-types>=0.6.0 (from pydantic<3,>=1.9.0->openai<2.0.0,>=1.56.2->bee_hive)
  Using cached annotated_types-0.7.0-py3-none-any.whl.metadata (15 kB)
Collecting pydantic-core==2.27.2 (from pydantic<3,>=1.9.0->openai<2.0.0,>=1.56.2->bee_hive)
  Using cached pydantic_core-2.27.2-cp312-cp312-macosx_11_0_arm64.whl.metadata (6.6 kB)
Using cached docopt_ng-0.9.0-py3-none-any.whl (16 kB)
Using cached jsonschema-4.23.0-py3-none-any.whl (88 kB)
Using cached openai-1.60.2-py3-none-any.whl (456 kB)
Using cached python_dotenv-1.0.1-py3-none-any.whl (19 kB)
Using cached PyYAML-6.0.2-cp312-cp312-macosx_11_0_arm64.whl (173 kB)
Using cached anyio-4.8.0-py3-none-any.whl (96 kB)
Using cached attrs-25.1.0-py3-none-any.whl (63 kB)
Using cached distro-1.9.0-py3-none-any.whl (20 kB)
Using cached httpx-0.28.1-py3-none-any.whl (73 kB)
Using cached httpcore-1.0.7-py3-none-any.whl (78 kB)
Using cached jiter-0.8.2-cp312-cp312-macosx_11_0_arm64.whl (310 kB)
Using cached jsonschema_specifications-2024.10.1-py3-none-any.whl (18 kB)
Using cached pydantic-2.10.6-py3-none-any.whl (431 kB)
Using cached pydantic_core-2.27.2-cp312-cp312-macosx_11_0_arm64.whl (1.8 MB)
Using cached referencing-0.36.2-py3-none-any.whl (26 kB)
Using cached rpds_py-0.22.3-cp312-cp312-macosx_11_0_arm64.whl (342 kB)
Using cached sniffio-1.3.1-py3-none-any.whl (10 kB)
Using cached tqdm-4.67.1-py3-none-any.whl (78 kB)
Using cached typing_extensions-4.12.2-py3-none-any.whl (37 kB)
Using cached annotated_types-0.7.0-py3-none-any.whl (13 kB)
Using cached idna-3.10-py3-none-any.whl (70 kB)
Using cached certifi-2025.1.31-py3-none-any.whl (166 kB)
Using cached h11-0.14.0-py3-none-any.whl (58 kB)
Building wheels for collected packages: bee_hive
  Building editable for bee_hive (pyproject.toml) ... done
  Created wheel for bee_hive: filename=bee_hive-0.1.0-py3-none-any.whl size=2768 sha256=62943a4a702c58bfcaca3bd6aeee8ac5143b1a0e5b55c05594d91c2069782918
  Stored in directory: /private/var/folders/y0/dtxnsmfs0sz8_01kd0hhfjrc0000gn/T/pip-ephem-wheel-cache-bznqrn18/wheels/70/6f/78/f4eb56e5858d757847667fd1d834df00ccf95d10de49c27ea7
Successfully built bee_hive
Installing collected packages: typing-extensions, tqdm, sniffio, rpds-py, pyyaml, python-dotenv, jiter, idna, h11, docopt-ng, distro, certifi, attrs, annotated-types, referencing, pydantic-core, httpcore, anyio, pydantic, jsonschema-specifications, httpx, openai, jsonschema, bee_hive
Successfully installed annotated-types-0.7.0 anyio-4.8.0 attrs-25.1.0 bee_hive-0.1.0 certifi-2025.1.31 distro-1.9.0 docopt-ng-0.9.0 h11-0.14.0 httpcore-1.0.7 httpx-0.28.1 idna-3.10 jiter-0.8.2 jsonschema-4.23.0 jsonschema-specifications-2024.10.1 openai-1.60.2 pydantic-2.10.6 pydantic-core-2.27.2 python-dotenv-1.0.1 pyyaml-6.0.2 referencing-0.36.2 rpds-py-0.22.3 sniffio-1.3.1 tqdm-4.67.1 typing-extensions-4.12.2

[notice] A new release of pip is available: 24.3.1 -> 25.0
[notice] To update, run: pip install --upgrade pip

Then this just works:

(.venv) ➜  tmp3 beeai
Usage:
  beeAI validate SCHEMA_FILE YAML_FILE [options]

Copy link
Collaborator

@maximilien maximilien left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK with this if you address my recommendations... However, get a second opinion since I am not running demos yet and unclear how that works with the standard @AngeloDanducci is setting.

bee-hive/cli/beeAI.py Outdated Show resolved Hide resolved
bee-hive/pyproject.toml Outdated Show resolved Hide resolved
@planetf1
Copy link
Collaborator Author

planetf1 commented Feb 3, 2025

OK with this if you address my recommendations... However, get a second opinion since I am not running demos yet and unclear how that works with the standard @AngeloDanducci is setting.

Any feedback @AngeloDanducci

I think it's still work in progress. This feature does consistently allow a new python environment to be set up (tested on macOS, multiple linux), a pip install done, and the beeAI cli will work. So it's a general 'build a package that can be reused' capability

Using that in demo is tbd (here) which needs to go further and ensure the other bee hive modules work correctly, in particular as we are installing 2 packages from a single git source

@planetf1 planetf1 marked this pull request as ready for review February 3, 2025 10:21
@planetf1 planetf1 marked this pull request as draft February 3, 2025 12:26
@planetf1
Copy link
Collaborator Author

planetf1 commented Feb 3, 2025

Converting back to draft - some issues came up when testing within the bee-hive-demos project.

@planetf1 planetf1 force-pushed the dev branch 3 times, most recently from 072c0ed to a4ef7eb Compare February 4, 2025 14:23
@planetf1
Copy link
Collaborator Author

planetf1 commented Feb 4, 2025

Updated PR. This mostly covers two areas

  • Creating a poetry script that will run the beeAI cli once the python package is installed - within the project, or in a dependent project.
  • Making the imports consistent so that they are always based off the top of the project ie under bee_hive (not relative to current directory for example)

@maximilien Please check this doesn't break your usage pattern.

Example:

(bee-hive-py3.12) ➜  bee-hive git:(dev) pwd
/Users/jonesn/AI/bee/bee-hive/bee-hive
(bee-hive-py3.12) ➜  bee-hive git:(dev) beeAI
Usage:
  beeAI validate SCHEMA_FILE YAML_FILE [options]
  beeAI create AGENTS_FILE [options]
  beeAI run AGENTS_FILE WORKFLOW_FILE [options]
  beeAI deploy AGENTS_FILE WORKFLOW_FILE [options]
(bee-hive-py3.12) ➜  bee-hive git:(dev) beeAI validate
Warning: found unmatched (duplicate?) arguments [Argument(None, 'validate')]
Usage:
  beeAI validate SCHEMA_FILE YAML_FILE [options]
  beeAI create AGENTS_FILE [options]
  beeAI run AGENTS_FILE WORKFLOW_FILE [options]
  beeAI deploy AGENTS_FILE WORKFLOW_FILE [options]
(bee-hive-py3.12) ➜  bee-hive git:(dev) cd tests
(bee-hive-py3.12) ➜  tests git:(dev) beeAI validate
Warning: found unmatched (duplicate?) arguments [Argument(None, 'validate')]
Usage:
  beeAI validate SCHEMA_FILE YAML_FILE [options]
  beeAI create AGENTS_FILE [options]
  beeAI run AGENTS_FILE WORKFLOW_FILE [options]
  beeAI deploy AGENTS_FILE WORKFLOW_FILE [options]
(bee-hive-py3.12) ➜  tests git:(dev) 

In another separate project, using the dependency as follows within the project.dependencies[] array

    "bee_hive @ git+ssh://[email protected]/planetf1/bee-hive.git@dev#subdirectory=bee-hive",

I can similarly use the beeAI

Note that over there the repo/branch can be i-am-bee & main once this PR is merged. The subdir may change when we merge the related directory move here. That doesn't affect this repo, just usage

(bee-hive-demos-py3.12) ➜  activity-planner-demo git:(dev/jonesn/issue12) ✗ pwd
/Users/jonesn/AI/bee/bee-hive-demos/demos/activity-planner-demo
(bee-hive-demos-py3.12) ➜  activity-planner-demo git:(dev/jonesn/issue12) ✗ beeAI 
Usage:
  beeAI validate SCHEMA_FILE YAML_FILE [options]
  beeAI create AGENTS_FILE [options]
  beeAI run AGENTS_FILE WORKFLOW_FILE [options]
  beeAI deploy AGENTS_FILE WORKFLOW_FILE [options]

@planetf1 planetf1 marked this pull request as ready for review February 4, 2025 14:38
bee-hive/cli/beeAI.py Outdated Show resolved Hide resolved
@planetf1 planetf1 changed the title [DRAFT] fix(cli): Package cli for execution in dependent projects fix(cli): Package cli for execution in dependent projects Feb 4, 2025
@maximilien
Copy link
Collaborator

LGTM

@planetf1 planetf1 merged commit ebf3620 into i-am-bee:main Feb 5, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants