Skip to content

Latest commit

 

History

History
109 lines (74 loc) · 4.96 KB

README.md

File metadata and controls

109 lines (74 loc) · 4.96 KB
Raw

AI Powered

codecov

Motivation

Currently, AI has reached a stage of considerable practicality, yet many traditional software applications have not yet benefited from it. This project aims to provide convenient tools for integrating AI capabilities (primarily referring to Large Language Models) into various software.

With these tools, you don't need to redesign the entire software to leverage AI capabilities, nor do you need to add any user-visible functional modules. As long as you find that a function in your project could potentially be better answered by AI, you can replace its implementation with AI.

Even so, you don't need to learn anything about the OpenAI SDK to achieve these tasks, all within just a few minutes!

Features

  1. The @ai_powered decorator which can auto provide an AI implementation for your python function
  2. Support LLM Models with or without tool_calls support
  3. Full async support and sync compatible (supported by easy-sync), use either style you want

Code Quality Promise

  1. Strict type annotation (validated by pylance the strict mode)
  2. With full unit test cases, and test coverage ratio is monitored

Usage

Installation is done by pip install ai_powered or poetry add ai_powered. (pypi)

And provide your API keys via environment variables (more details):

export OPENAI_API_KEY=---YOUR-REAL-API-KEY---

It provides the following tools:

@ai_powered Decorator

This decorator imbues your function signature with an AI implementation, which reads the docstring and invokes LLM to obtain the function's return value

from ai_powered import ai_powered

@ai_powered
def get_python_expression(expr: str) -> str:
    """ Convert the user-input mathematical expression into a valid Python expression """
    ...

You can also use more complex data structures in parameters and return values, but make sure they have complete type annotations

@dataclass
class UserInfo:
    name: str
    country: Optional[str]
    age: Optional[int]

@ai_powered
def extract_user_info(raw_text: str) -> UserInfo:
    ''' Extract user information from this self-introduction '''
    ...

And async function is also supported:

@ai_powered
async def add(a: int, b: int) -> int:
    ...

async def main():
    print(await add(1, 1))

You can even call it in the sync context

(this feature is supported by easy-sync)

print(add(1, 1).wait())

More examples can be found here

Current Limitations and Future Plans

  • Currently, only a Python implementation is provided, but it is possible to be replicated in any other language that supports runtime type annotations.
  • The data generated by the current LLM does not strictly adhere to the provided JSON Schema 100% of the time. The error rate may decrease as LLM providers continue to train, but in the end, we may need to introduce a retry mechanism to get it to approach 100% correctness (a retry mechanism is currently being planned). (Things changed now, but there is still models which does not support this)
  • At present, recursive structures is not supported, since some LLM function calling capability cannot recognize references in the schema, so we deref all the references in the schema, but deref doesn't work with recursive types, so this issue may ultimately require LLM providers to add such data in their training datasets to truly resolve the problem.

Regarding Code Contribution

  1. Currently, all code is under Pyright strict mode type checking, and any type errors will be blocked by GitHub Actions. We do not recommend using Any or #type: ignore unless absolutely necessary.
  2. Test coverage will be continuously monitored. It is recommended to always provide tests for your code, and even prepare the tests before coding. tips: you can mark tests for WIP features use @pytest.mark.xfail.
  3. Regarding the development environment, it is recommended to install nix and direnv so that you automatically get a usable development environment. Of course, poetry shell is also a good choice (if you are already using poetry).

Acknowledgments

  • instructor: It has made significant contributions to advancing LLM in supporting structured output.
  • msgspec: Rapid serialization/deserialization of data and providing JSON schema, which is one of the most crucial libraries relied upon by this project.