Version 0.3 (#4)

* Trying to generalize tool agent

* bumping reqs

* Raising error to trigger next retry

* Better error handling here

* Bump

* Adding runtime oai_kwargs

* Some docs

* Version bump

* Better handling of temperature default (if there should even be one at all)

* Handling case where multiple tool calls are needed
- In this case, we might try to access completion message as a dict but this will fail

* Appending chat message to cache before we parse the arguments

* Maybe copying?

* We must go deeper

* Does this work?

* .

* Revert "."

This reverts commit 8b4e50c.

* Revert "Does this work?"

This reverts commit 0bd8f40.

* bump

* another bump

* Function is deprecated

* Don't init mutables

* Actually extend, don't pass method

* Adding a structured prediction agent
- This also expands the dependencies quite a bit, so I'm not sure about it

* WIP async

* Fixing to appropriate api usage

* dbug

* Does ths work better?

* even more async

* redoing exports

* Whoops

* retying

* Thinking

* No more async for now

* Missed tiktoken and backoff, somehow

* bump README

* Version bump

* Trying to add in justification too

* whoops

* Adding batch processor to API

* A few more tweaks

* Generalizing Prediction agent
- Making it easier to subclass and not being project-specific with formatting
- df no longer copied

* Fixing parallel processing logic

* Avoiding async directives

* more classes

* Handle agents which return >1 str per sample

* Version bump

* Adding general batch processor for lists and such

* Allowing additional kwargs in agent call method

* Trying async

* Forgot an await

* Avoid getting hung up on a single agent

* Making sure the task_done signal gets called

* Forgot this piece too

* Moving these to where I think they should go

* A pretty subtantial re-write
- ToolAwareAgent is now just wrapped into Agent
- Agent is no longer abstract, can be inited directly (though it doesn't make a lot of sense to do so)
- Agent gets Tool and Callback handling natively
- Wrote some more doc string

* Substantial re-write (#3)

* WIP re-write

* Huge re-write to API
- Moving a few things around
- Will probably move more

* Updating Reqs accordingly

* Adding agent callback example

* Some addn tweaks

* Updating setup

* Trying to figure out stopping conditions

* Forgot to add tools, actually

* Updating reqs

* Trying to work in a few things

* Adding structured output agent
- Will maybe replace Prediction agent? Not sure

* Structured Prediction example

* Don't force a tool call because it's API dependent

* Changing symbol

* A few bits of housecleaning

* Storing this here for now

* Removing MultiStepToolAgent, which wasn't used

* Slimming down prediction agent and making it a subclass

* Updating reqs to include tqdm which I forgot

* Cleaning up processors

* Fixing types and a few other things wrt mypy

* Fixing issue where structured responses would only append key instead of whole object

* Adding batch processing example

* Moving to the correct place

* Updating readme

* Updating typing in callback

* Bumping Version

* Fixing issue where messages appened to tool res list even though no tool calls were made

* Updating stopping condition to check for length too

* Fixing issue where tools was modified by reference leading to errors in subsequent calls

* Actually fixing this

* Baking in some additional logic to correct when agent tries to call an unknown function

* Sending the actual name of the tool instead of the error

* Trying to handle this at a different level
- Also updating class with a property

* It would help to also have the assertion in there

* missp

README.md

@@ -3,18 +3,24 @@ Sean Browning (NCEZID/OD Data Science Team)
 
 ## Background
 
-I wanted to learn how to create and use language agents to solve complex problems. Langchain wasn't cutting it for me, so I made my own library from first principles.
-Right now, everything is quite experimental and I'm still figuring out the API.
+I wanted to learn how to create and use language agents to solve complex problems. LangChain wasn't cutting it for me, so I made my own library from first principles to suit a few projects we're working on.
+
+This package contains a few classes that can be used as building blocks for language agents and agentic systems. I plan to expand it with additional functionality as I need it, but keep a minimal footprint (ie. if you're already using openai and pydantic, this should bring no additional dependencies).
+
+All code uses asyncio by design, and though I've tried to generalize as I can, I mostly built around OpenAI and specifically Azure OpenAI since that's what we are allowed to work with internally.
 
 ## Installation
 
-Should be pip installable from this repo.
+Not currently on pypi, so just use pip to install via GitHub:
 
 ```sh
-pip install git+https://github.com/cdcai/agents.git
+pip install git+https://github.com/cdcai/multiagent.git
 ```
 
 ## Examples
 
-#TODO
-
+| Example | Link |
+| ---- | ---- |
+| Taking output from one agent as input to another in a callback | [agent_with_callback.py](examples/agent_with_callback.py) |
+| Getting structured output from agent / Text Prediction | [structured_prediction.py](examples/structured_prediction.py) |
+| Batch processing large inputs over the same agent in parallel | [batch_processing.py](examples/batch_processing.py) |

agents/__init__.py

@@ -1,3 +1,5 @@
 from .agent import *
-from .env import WikiQAEnv
 from .generic import *
+from .processors import BatchProcessor, DFBatchProcessor
+from .callbacks import *
+from .stopping_conditions import *

agents/abstract.py

@@ -0,0 +1,113 @@
+"""
+All abstract classes
+"""
+import abc
+import logging
+import os
+from typing import Callable, List, Optional, Union, Any
+
+import openai
+from openai.types.chat.chat_completion import Choice, ChatCompletionMessage
+
+logger = logging.getLogger(__name__)
+
+class _StoppingCondition(metaclass=abc.ABCMeta):
+    """
+    A callable that contains some logic to determine whether a language agent
+    has finished it's run.
+
+    The main call should always return the final answer, if we've finished the run, or None otherwise
+    """
+    @abc.abstractmethod
+    def __call__(self, cls: '_Agent', response: str) -> Optional[Any]:
+        raise NotImplementedError()
+
+class _Agent(metaclass=abc.ABCMeta):
+    terminated: bool = False
+    truncated: bool = False
+    curr_step: int = 1
+    output_len: int = 1
+    scratchpad: str = ""
+    answer: Any = ""
+    BASE_PROMPT: str = ""
+    SYSTEM_PROMPT: str = ""
+    oai_kwargs : dict
+    TOOLS : list
+    CALLBACKS : list
+    callback_output: list
+    tool_res_payload: list[dict]
+
+    def __init__(
+        self,
+        model_name: str,
+        stopping_condition: _StoppingCondition,
+        llm: Optional[openai.AsyncOpenAI] = None,
+        tools: Optional[List[dict]] = None,
+        callbacks: Optional[List[Callable]] = None,
+        oai_kwargs: Optional[dict[str, Any]] = None,
+        **fmt_kwargs
+    ):
+        pass
+
+    @abc.abstractmethod
+    async def prompt_agent(self, prompt: Union[dict[str, str], list[dict[str, str]]], n_tok: Optional[int] = None, **addn_oai_kwargs) -> Choice:
+        raise NotImplementedError()
+
+    @abc.abstractmethod
+    async def step(self):
+        """
+        Run a single "step" of the agent logic.
+        Handles prompting OpenAI, optionally handling tool calls, and determining whether we've
+        finished or run out of tokens.
+        """
+        raise NotImplementedError()
+
+    @abc.abstractmethod
+    def _check_stop_condition(self, response: ChatCompletionMessage) -> None:
+        """
+        Called from within :func:`step()`.
+        Checks whether our stop condition has been met and handles assignment of answer, if so.
+
+        It's broken out this way because we may not always want to use message.content as the answer (tool call output, for instance)
+        """
+
+    @abc.abstractmethod
+    def format_prompt(self) -> str:
+        """
+        Method which formats the BASE_PROMPT string, possibly inserting additional content.
+        This is usually called within get_next_message() to populate the first user message.
+        """
+        raise NotImplementedError()
+
+    @abc.abstractmethod
+    def get_next_messages(self) -> list[dict[str, str]]:
+        raise NotImplementedError()
+
+    @abc.abstractmethod
+    def _handle_tool_calls(self, response: Choice) -> None:
+        raise NotImplementedError()
+
+    @property
+    def is_terminated(self) -> bool:
+        return self.terminated
+
+    @property
+    def _known_tools(self) -> list[str]:
+        return [tool["function"]["name"] for tool in self.TOOLS]
+
+    @property
+    def is_truncated(self) -> bool:
+        return self.truncated
+
+    @abc.abstractmethod
+    def dump(self, outfile: Union[str, os.PathLike]) -> None:
+        raise NotImplementedError()
+
+    @staticmethod
+    @abc.abstractmethod
+    def authenticate() -> None:
+        raise NotImplementedError()
+
+    @abc.abstractmethod
+    def reset(self):
+        raise NotImplementedError()

agents/agent/__init__.py

@@ -1,6 +1,7 @@
 from .base import *
-from .sas import *
-from .reflexion import *
 from .prediction import *
 
-__all__ = ["ReactAgent", "ReactandReflectAgent", "SASConvertAgent", "PythonRefineAgent", "CodeOutlinerAgent", "OutlineSummarizeAgent", "PythonSummarizeAgent", "PseudocodeSummarizeAgent", "PredictionAgent"]
+__all__ = [
+    "Agent", "StructuredOutputAgent",
+    "PredictionAgent", "PredictionAgentWithJustification"
+]