Skip to content
This repository has been archived by the owner on Sep 12, 2024. It is now read-only.

Mtllm Move back into Jaclang Repo #609

Open
wants to merge 123 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
123 commits
Select commit Hold shift + click to select a range
e98fb7c
Initial commit
chandralegend May 29, 2024
06e00ce
init
chandralegend May 30, 2024
6a2f91b
chore: Add pre-commit and release workflows
chandralegend May 30, 2024
233587f
chore: added github workflow for release to pypi
chandralegend May 30, 2024
9d6279e
chore: Update release workflow to use PyPI token from secrets
chandralegend May 30, 2024
e4aef19
chore: Update MTLLM API library version to 0.1.1 and provide description
chandralegend May 30, 2024
90ff195
chore: Update MTLLM API library version to 0.1.2 and provide description
chandralegend May 30, 2024
efe12ce
chore: Update dependencies and workflows for release to PyPI
chandralegend Jun 4, 2024
1ff6bb6
chore: Bump MTLLM API library version to 0.1.3
chandralegend Jun 4, 2024
46d1e30
chore: Install @semantic-release/github and @semantic-release/git dep…
chandralegend Jun 4, 2024
a50f2f7
chore: Bump MTLLM API library version to 0.1.4
chandralegend Jun 4, 2024
0ed5f7e
chore: Update release workflow to use pip install for githubrelease a…
chandralegend Jun 4, 2024
188fe31
chore: Bump MTLLM API library version to 0.1.5
chandralegend Jun 4, 2024
c62a3e9
chore: Update repository reference in gh_release.py
chandralegend Jun 4, 2024
73c7723
chore: Bump MTLLM API library version to 0.1.6
chandralegend Jun 4, 2024
cc775de
chore: Update MTLLM API library version to 0.1.7 and fix GitHub token…
chandralegend Jun 4, 2024
4ad3e74
linting fix
chandralegend Jun 4, 2024
cdb7d40
chore: Update MTLLM API library version to 0.1.7 and fix GitHub token…
chandralegend Jun 9, 2024
80d9077
chore: Bump MTLLM API library version to 0.1.8
chandralegend Jun 9, 2024
33c455b
chore: Update MTLLM API library version to 0.1.8
chandralegend Jun 9, 2024
6d3d239
chore: Bump MTLLM API library version to 0.1.9
chandralegend Jun 9, 2024
0217a84
added docs
chandralegend Jun 10, 2024
f7501ab
chore: Update MTLLM API library version to 0.1.10
chandralegend Jun 10, 2024
47f3e2e
chore: Update MTLLM API library version to 0.1.11 and fix multimodal …
chandralegend Jun 12, 2024
4b3e57e
chore: Update MTLLM API library version to 0.1.11 and fix multimodal …
chandralegend Jun 12, 2024
51c5f75
feat: Add Video and Image Classes for MultiModals
chandralegend Jun 21, 2024
9f4531c
chore: Update .flake8 and mtllm/llms/utils.py
chandralegend Jun 21, 2024
e54951f
feat: Add Video and Image Classes for MultiModals
chandralegend Jun 21, 2024
c8d1e37
Refactor process_image method to process in Image class
chandralegend Jun 21, 2024
a0bbd19
feat: Add seconds_per_frame parameter to Video class constructor
chandralegend Jun 21, 2024
07180ef
chore: Update video dependency to remove moviepy
chandralegend Jun 21, 2024
8a5c539
Refactor Video class process method to adhere to PEP8 style guide
chandralegend Jun 21, 2024
3207b0a
Refactor regex pattern to include multiline matching in BaseLLM class
chandralegend Jun 21, 2024
c60a4ae
added the plugin support of jaclang
chandralegend Jun 26, 2024
4756af4
liniting fix
chandralegend Jun 26, 2024
e91cfc3
Refactor wikipedia tool to import wikipedia_utils locally
chandralegend Jun 26, 2024
b22abcc
added tests
chandralegend Jun 26, 2024
c7c2bdc
chore: Update pytest dependency and install opencv-python-headless an…
chandralegend Jun 26, 2024
693f740
chore: Update pytest dependency and install opencv-python-headless an…
chandralegend Jun 26, 2024
44419f5
Update README.md
chandralegend Jun 26, 2024
c1fcf00
added examples
chandralegend Jun 26, 2024
a520a2f
Refactor get_input_information function to use input_type variable in…
chandralegend Jun 26, 2024
42b48ed
chore: Bump version to 0.1.14
chandralegend Jun 26, 2024
e48b453
chore: Update landing page banners
chandralegend Jun 26, 2024
76b55cd
updated docs structure
chandralegend Jun 27, 2024
a194e82
fix pre-commit
chandralegend Jun 27, 2024
99fc0d5
docs: added installation and minimal example pages
chandralegend Jul 1, 2024
77e8a9d
chore: Update dependencies and add optional packages
chandralegend Jul 1, 2024
d3dc075
feat: Add support for OpenAI's GPT-4o language model client
chandralegend Jul 1, 2024
310f2ab
doc: added language models section
chandralegend Jul 1, 2024
b386545
chore: Update font smoothing styles in HTML files
chandralegend Jul 1, 2024
ab52642
docs: added functions and methods
chandralegend Jul 1, 2024
89e6a35
doc: added object initialisation doc
chandralegend Jul 1, 2024
64bf121
added multimodal dos
chandralegend Jul 1, 2024
cee1ca1
chore: Update font smoothing styles in HTML files
chandralegend Jul 1, 2024
a383aca
chore: Remove unused code and update dependencies
chandralegend Jul 1, 2024
5fb8626
feat: Update video link in Multimodality.md
chandralegend Jul 1, 2024
f278d2d
Update font smoothing styles in HTML files
chandralegend Jul 1, 2024
d02db76
codedoc guide
Jayanaka-98 Jul 3, 2024
d92f628
docs: added how to integrate mtllm to existing applications
chandralegend Jul 4, 2024
85a2904
Merge branch 'Jaseci-Labs:main' into main
Jayanaka-98 Jul 4, 2024
437e2bf
doc: on how to bring your own llm llm to MTLLM
chandralegend Jul 4, 2024
70e7613
RPG Tutorial
Jayanaka-98 Jul 5, 2024
9a7fff6
RPG docs update
Jayanaka-98 Jul 5, 2024
6a5582f
Semstrings doc update
Jayanaka-98 Jul 5, 2024
bce9f93
Merge pull request #1 from Jayanaka-98:docs_jayanaka
Jayanaka-98 Jul 5, 2024
e06cdfb
Update semstring doc
Jayanaka-98 Jul 5, 2024
ec5b547
Merge branch 'Jaseci-Labs:main' into main
Jayanaka-98 Jul 5, 2024
25c5efa
Merge pull request #2 from Jayanaka-98/main
chandralegend Jul 5, 2024
81ea0b1
docs: added semstring and level gen tutorial
chandralegend Jul 5, 2024
fd43315
Update README.md
chandralegend Jul 5, 2024
ce02da6
doc: made the toc to be opened default
chandralegend Jul 5, 2024
437d27f
doc: moved instructions on your mtllm on python to Tips and Tricks
chandralegend Jul 5, 2024
8f3b8b9
Remove Optimal way of integrating MTLLM documentation
chandralegend Jul 5, 2024
1cfb0bf
chore: pre-commit fix
chandralegend Jul 5, 2024
0a68a79
added the mtllm finetuning scripts
chandralegend Jul 8, 2024
e100df7
chore: Refactor dataset preparation function
chandralegend Jul 8, 2024
12d3d10
chore: Add utility functions for finetuning LLMs
chandralegend Jul 8, 2024
ebbfa17
detupled type explanations
chandralegend Jul 9, 2024
ddb84a0
refactored Input Information and OutputInformation
chandralegend Jul 9, 2024
334dbcb
refactored Include Information and detupled
chandralegend Jul 9, 2024
ed6ab17
chore: Fix assertion error in JacFeature class
chandralegend Jul 9, 2024
c04923c
refactor: Remove unnecessary code for importing cv2 and PILImage
chandralegend Jul 9, 2024
cbe7593
refactor: Update incl_info type in JacFeature class
chandralegend Jul 9, 2024
6fee8d3
Merge pull request #5 from Jaseci-Labs/detupling
chandralegend Jul 10, 2024
9a81ece
refactor: Update OpenAI model to gpt-4o in essay_review.jac
chandralegend Jul 23, 2024
e355170
Updated the release procedure
chandralegend Jul 23, 2024
f9c0c16
chore: Update dependencies for GitHub release workflow
chandralegend Jul 23, 2024
38d8feb
changed the release workflow
chandralegend Jul 23, 2024
ce74d16
chore: Update release notes for MTLLM
chandralegend Jul 23, 2024
81a8fc9
feat: Add support for task-specific finetuning
chandralegend Jul 24, 2024
a8f25c2
feat: Update program_args.json and llm_synthetic_gen.py
chandralegend Jul 25, 2024
4a8168a
added data generation scripts
chandralegend Jul 25, 2024
719b974
Merge pull request #6 from Jaseci-Labs/finetune_smollm_for_mapgen
chandralegend Jul 25, 2024
447da3f
adde dev container config
chandralegend Jul 26, 2024
81e159b
chore: added the training config
chandralegend Jul 26, 2024
4f984fd
Merge branch 'main' of https://github.com/Jaseci-Labs/mtllm into main
chandralegend Jul 26, 2024
ded9b1e
These changes aim to improve the training process and fine-tuning of …
chandralegend Jul 26, 2024
b83c119
feat: Improve finetuning process and configuration
chandralegend Jul 26, 2024
aab8933
feat: Update finetuning process and configuration for MTLLM model
chandralegend Jul 27, 2024
48f6e3b
feat: Update finetuning process and configuration for MTLLM model
chandralegend Jul 29, 2024
993bb7d
feat: Update finetuning process and configuration for MTLLM model
chandralegend Jul 29, 2024
e62e7ad
feat: Update test notebook link in README
chandralegend Jul 29, 2024
7f92a4d
moved the py ast gen pass function related to mtllm to here
chandralegend Aug 1, 2024
e8cb80c
feat: added eval fixers
chandralegend Aug 1, 2024
977d570
added the eval fixing ability
chandralegend Aug 1, 2024
3d63fd0
Update BaseLLM initialization in test fixtures
chandralegend Aug 1, 2024
6cb6891
working react implementation. currently cannot pass abilities.
chandralegend Aug 9, 2024
7aadf62
reduced the length of the type explanations by removing unccessary to…
chandralegend Aug 10, 2024
44bcfa1
Reduce length of type explanations by removing unnecessary tokens
chandralegend Aug 13, 2024
a0a288e
Merge pull request #7 from Jaseci-Labs/feat-react-tool-usage
chandralegend Aug 14, 2024
0fcb496
Update jaclang dependency to version 0.7.16
chandralegend Aug 14, 2024
793d7cc
Merge branch 'feat-react-tool-usage'
chandralegend Aug 14, 2024
1fbc3d1
feat: Added support for the new `Jaclang` version, moved mtllm relate…
chandralegend Aug 14, 2024
3a9d2a6
updated the jac programs with new syntax
chandralegend Aug 20, 2024
c617d44
feat: added self healing for the react
chandralegend Aug 20, 2024
f5e7398
feat: Add marketing agency example with LLM capabilities
chandralegend Aug 20, 2024
a6e1ca9
feat: Add Serper Tools for Search Integration
chandralegend Aug 20, 2024
64ca679
feat: Add collapsible sections for design and implementation in table…
chandralegend Aug 28, 2024
6716033
feat: Add codedoc-mermaid-plugin to improve documentation with mermai…
chandralegend Aug 28, 2024
d5f3cc7
chore: Remove unused reloadOnChange function
chandralegend Aug 28, 2024
20a87c5
refactor: Update files involved in the LLM Interface and Output Proce…
chandralegend Aug 29, 2024
b46bebe
Add 'support/plugins/mtllm/' from commit '20a87c5a3f60dd5cfa397e5c8e9…
Jayanaka-98 Sep 6, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions support/plugins/mtllm/.devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
{
"name": "Python 3",
"image": "mcr.microsoft.com/devcontainers/python:1-3.12-bullseye",
"features": {
"ghcr.io/meaningful-ooo/devcontainer-features/fish:1": {
"fisher": true
},
"ghcr.io/devcontainers/features/conda:1": {
"addCondaForge": true,
"version": "latest"
}
},
"postCreateCommand": "pip3 install --user -r dev.requirements.txt"
}
5 changes: 5 additions & 0 deletions support/plugins/mtllm/.flake8
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
[flake8]
exclude = __jac_gen__, build, dist
plugins = flake8_import_order, flake8_docstrings, flake8_comprehensions, flake8_bugbear, flake8_annotations, pep8_naming, flake8_simplify
max-line-length = 120
ignore = E203, W503, ANN101, ANN102, D401
25 changes: 25 additions & 0 deletions support/plugins/mtllm/.github/workflows/precommit.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
name: Linting and Pre-commit checks

on:
pull_request:
push:
branches:
- main

jobs:
pre-commit:
runs-on: ubuntu-latest
steps:
- name: Check out code
uses: actions/checkout@v2

- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: 3.12

- name: Install pre-commit
run: pip install pre-commit

- name: Run pre-commit hooks
run: pre-commit run --all-files
29 changes: 29 additions & 0 deletions support/plugins/mtllm/.github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
name: PyPI Release

on:
workflow_dispatch:

jobs:
release:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v2
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: '3.12'
- name: Install Poetry
run: |
pip install poetry
- name: Build package
run: poetry build
- name: Publish package
run: |
poetry config pypi-token.pypi ${{ secrets.PYPI_TOKEN }}
poetry publish
continue-on-error: true
- run: pip install githubrelease markdown-to-json
- run: python scripts/gh_release.py
env:
GITHUB_TOKEN: ${{ secrets.GH_TOKEN }}
28 changes: 28 additions & 0 deletions support/plugins/mtllm/.github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
name: tests

on:
pull_request:
push:
branches:
- main

jobs:
test:
runs-on: ubuntu-latest
steps:
- name: Check out code
uses: actions/checkout@v2

- name: Set up Python 3.12
uses: actions/setup-python@v2
with:
python-version: 3.12

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -e .
pip install pytest opencv-python-headless pillow

- name: Run tests
run: pytest -x
167 changes: 167 additions & 0 deletions support/plugins/mtllm/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,167 @@
# Jaclang
__jac_gen__/
.jac_mypy_cache/
temp*

# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class

# C extensions
*.so

# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/

# Translations
*.mo
*.pot

# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal

# Flask stuff:
instance/
.webassets-cache

# Scrapy stuff:
.scrapy

# Sphinx documentation
docs/_build/

# PyBuilder
.pybuilder/
target/

# Jupyter Notebook
.ipynb_checkpoints

# IPython
profile_default/
ipython_config.py

# pyenv
# For a library or package, you might want to ignore these files since the code is
# intended to run in multiple environments; otherwise, check them in:
# .python-version

# pipenv
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
# However, in case of collaboration, if having platform-specific dependencies or dependencies
# having no cross-platform support, pipenv may install dependencies that don't work, or not
# install all needed dependencies.
#Pipfile.lock

# poetry
# Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
# This is especially recommended for binary packages to ensure reproducibility, and is more
# commonly ignored for libraries.
# https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
#poetry.lock

# pdm
# Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
#pdm.lock
# pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
# in version control.
# https://pdm.fming.dev/latest/usage/project/#working-with-version-control
.pdm.toml
.pdm-python
.pdm-build/

# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
__pypackages__/

# Celery stuff
celerybeat-schedule
celerybeat.pid

# SageMath parsed files
*.sage.py

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# Spyder project settings
.spyderproject
.spyproject

# Rope project settings
.ropeproject

# mkdocs documentation
/site

# mypy
.mypy_cache/
.dmypy.json
dmypy.json

# Pyre type checker
.pyre/

# pytype static type analyzer
.pytype/

# Cython debug symbols
cython_debug/

# PyCharm
# JetBrains specific template is maintained in a separate JetBrains.gitignore that can
# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
# and can be added to the global gitignore or merged into this file. For a more nuclear
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
#.idea/
25 changes: 25 additions & 0 deletions support/plugins/mtllm/.pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v2.3.0
hooks:
- id: check-yaml
args: [--allow-multiple-documents]
- id: check-json
- id: trailing-whitespace
- repo: https://github.com/psf/black
rev: 24.1.1
hooks:
- id: black
- repo: https://github.com/PyCQA/flake8
rev: 6.1.0
hooks:
- id: flake8
additional_dependencies: [pep8-naming, flake8_import_order, flake8_docstrings, flake8_comprehensions, flake8_bugbear, flake8_annotations, flake8_simplify]
- repo: https://github.com/pre-commit/mirrors-mypy
rev: v1.8.0
hooks:
- id: mypy
additional_dependencies: [types-PyYAML, types-requests]
args:
- --follow-imports=silent
- --ignore-missing-imports
19 changes: 19 additions & 0 deletions support/plugins/mtllm/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# RELEASES

## `0.3.1` - 2024-08-20
- [IMPROVEMENT] Added Self Healing for the ReAct capabilities
- [FEATURE] Added Serper Tools for Search Integration

## `0.3.0` - 2024-08-14
- [IMPROVEMENT] Added support for the new `Jaclang` version
- [IMPROVEMENT] Moved all the mtllm related AST capabilities to mtllm repo from jaclang
- [FEATURE] Added ReAct and Tool Calling capabilities

## `0.2.0` - 2024-07-23
- [IMPROVEMENT] Seperated the different llms and vision capabilities into extras for the main package

## `0.1.13` - 2024-06-26
- [IMPROVEMENT] Added Plugin support for Jaclang

## `0.1.12` - 2024-06-21
- [FEATURE] Multi Modal capabilities for the Vision UseCases
21 changes: 21 additions & 0 deletions support/plugins/mtllm/LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
MIT License

Copyright (c) 2024 Chandra Irugalbandara

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
Loading
Loading