diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml new file mode 100644 index 00000000..62dab408 --- /dev/null +++ b/.github/workflows/docs.yml @@ -0,0 +1,28 @@ +name: documentation + +on: [push] + +permissions: + contents: write + +jobs: + docs: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v3 + - uses: actions/setup-python@v3 + - name: Install dependencies + run: | + pip install -e ".[documentation]" + - name: Sphinx build + run: | + sphinx-build docs _build + - name: Deploy to GitHub Pages + uses: peaceiris/actions-gh-pages@v3 +# if: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' }} + if: ${{ github.event_name == 'push' && github.ref == 'refs/heads/initial-prototypes' }} + with: + publish_branch: gh-pages + github_token: ${{ secrets.GITHUB_TOKEN }} + publish_dir: _build/ + force_orphan: true diff --git a/.gitignore b/.gitignore index 68bc17f9..6c016ebb 100644 --- a/.gitignore +++ b/.gitignore @@ -158,3 +158,5 @@ cython_debug/ # and can be added to the global gitignore or merged into this file. For a more nuclear # option (not recommended) you can uncomment the following to ignore the entire idea folder. #.idea/ +# +_build diff --git a/README.md b/README.md index ac6ef190..cbbaed50 100644 --- a/README.md +++ b/README.md @@ -19,14 +19,6 @@ Let us take a look at of how one might design a gpt-like chatbot. It: 6. If this succeeds, we present the response to the user 7. Await a new prompt, GOTO (1) -Simple, right? Until you start tracking how state flows through: -- The prompt is referenced by multiple future steps -- The chat history is referred to at multiple points, and appended to (at (1) and (5)) -- The decision on mode is opaque, and referred to both by (4), to know what model to query and by (6) to know how to render the response -- You will likely want to add more capabilities, more retries, etc... - -Chatbots, while simple at first glance, turn into something of a beast when you want to bring them to production and understand exactly *why* -they make the decisions they do. We can model this as a _State Machine_, using the following two concepts: 1. `Actions` -- a function that has two jobs. These form Nodes. @@ -36,6 +28,17 @@ We can model this as a _State Machine_, using the following two concepts: The set of these together form what we will call a `Application` (effectively) a graph. +Why do we need all this abstraction? + +This is all simple until you start tracking how state flows through: +- The prompt is referenced by multiple future steps +- The chat history is referred to at multiple points, and appended to (at (1) and (5)) +- The decision on mode is opaque, and referred to both by (4), to know what model to query and by (6) to know how to render the response +- You will likely want to add more capabilities, more retries, etc... + +Chatbots, while simple at first glance, turn into something of a beast when you want to bring them to production and understand exactly *why* +they make the decisions they do. + For those into the CS details, this is reverse to how a state machine is usually represented (edges are normally actions and nodes are normally state). We've found this the easiest way to express computation as simple python functions. diff --git a/burr/core/action.py b/burr/core/action.py index 36b31eb0..6e556781 100644 --- a/burr/core/action.py +++ b/burr/core/action.py @@ -148,6 +148,12 @@ def name(self) -> str: class Result(Action): def __init__(self, fields: list[str]): + """Represents a result action. This is purely a convenience class to + pull data from state and give it out to the result. It does nothing to + the state itself. + + :param fields: Fields to pull from the state + """ super(Result, self).__init__() self._fields = fields diff --git a/docs/Makefile b/docs/Makefile new file mode 100644 index 00000000..d4bb2cbb --- /dev/null +++ b/docs/Makefile @@ -0,0 +1,20 @@ +# Minimal makefile for Sphinx documentation +# + +# You can set these variables from the command line, and also +# from the environment for the first two. +SPHINXOPTS ?= +SPHINXBUILD ?= sphinx-build +SOURCEDIR = . +BUILDDIR = _build + +# Put it first so that "make" without argument is like "make help". +help: + @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) + +.PHONY: help Makefile + +# Catch-all target: route all unknown targets to Sphinx using the new +# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). +%: Makefile + @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) diff --git a/docs/README-internal.md b/docs/README-internal.md new file mode 100644 index 00000000..8f4334fc --- /dev/null +++ b/docs/README-internal.md @@ -0,0 +1,11 @@ +# Internal README for docs + +This documentation aims to follow the [diataxis](diataxis.fr) approach to documentation. This outlines: +1. Tutorials [getting_started](./getting_started) +2. How-to guides (examples in the repo) +3. References [reference](./reference) +4. Explanation [concepts](./concepts) + +TODO: +- [ ] fill in all docs todos +- [ ] Add examples for Hamilton integration, streamlit integration diff --git a/docs/concepts/actions.rst b/docs/concepts/actions.rst new file mode 100644 index 00000000..1df43565 --- /dev/null +++ b/docs/concepts/actions.rst @@ -0,0 +1,3 @@ +==================== +Actions +==================== diff --git a/docs/concepts/index.rst b/docs/concepts/index.rst new file mode 100644 index 00000000..8bebd8a8 --- /dev/null +++ b/docs/concepts/index.rst @@ -0,0 +1,12 @@ +==================== +Concepts +==================== + +Overview of the concepts -- read these to get a mental model for how Burr works. + +.. toctree:: + state-machine + state + transitions + actions + more diff --git a/docs/concepts/more.rst b/docs/concepts/more.rst new file mode 100644 index 00000000..ae55d068 --- /dev/null +++ b/docs/concepts/more.rst @@ -0,0 +1,8 @@ +==================== +More (TODO) +==================== + +- Execution +- Error handling +- Lifecycle methods +- Async diff --git a/docs/concepts/state-machine.rst b/docs/concepts/state-machine.rst new file mode 100644 index 00000000..abc43c50 --- /dev/null +++ b/docs/concepts/state-machine.rst @@ -0,0 +1,3 @@ +==================== +State Machine +==================== diff --git a/docs/concepts/state.rst b/docs/concepts/state.rst new file mode 100644 index 00000000..3b73f911 --- /dev/null +++ b/docs/concepts/state.rst @@ -0,0 +1,3 @@ +==================== +State +==================== diff --git a/docs/concepts/transitions.rst b/docs/concepts/transitions.rst new file mode 100644 index 00000000..3888e6bb --- /dev/null +++ b/docs/concepts/transitions.rst @@ -0,0 +1,3 @@ +==================== +Transitions +==================== diff --git a/docs/conf.py b/docs/conf.py new file mode 100644 index 00000000..b91a837b --- /dev/null +++ b/docs/conf.py @@ -0,0 +1,41 @@ +# Configuration file for the Sphinx documentation builder. +# +# For the full list of built-in configuration values, see the documentation: +# https://www.sphinx-doc.org/en/master/usage/configuration.html + +# -- Project information ----------------------------------------------------- +# https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information + +project = "Burr" +copyright = "2024, Elijah ben Izzy, Stefan Krawczyk" +author = "Elijah ben Izzy, Stefan Krawczyk" + +# -- General configuration --------------------------------------------------- +# https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration + +extensions = ["sphinx.ext.autodoc", "sphinx.ext.autosummary", "myst_parser", "sphinx_sitemap"] + +templates_path = ["_templates"] +exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"] + +html_theme = "furo" +html_static_path = ["_static"] + +html_title = "Burr" +html_theme_options = { + "source_repository": "https://github.com/dagworks-inc/burr", + "source_branch": "main", + "source_directory": "docs/", + "light_css_variables": { + "color-announcement-background": "#ffba00", + "color-announcement-text": "#091E42", + }, + "dark_css_variables": { + "color-announcement-background": "#ffba00", + "color-announcement-text": "#091E42", + }, +} + +html_baseurl = "https://burr.dagworks.io/en/latest/" # TODO -- update this + +exclude_patterns = ["README-internal.md"] diff --git a/docs/examples/chatbot.rst b/docs/examples/chatbot.rst new file mode 100644 index 00000000..97b6c0ad --- /dev/null +++ b/docs/examples/chatbot.rst @@ -0,0 +1,5 @@ +==================== +GPT-like chatbot +==================== + +TODO -- link to github diff --git a/docs/examples/index.rst b/docs/examples/index.rst new file mode 100644 index 00000000..ab42a65c --- /dev/null +++ b/docs/examples/index.rst @@ -0,0 +1,13 @@ +==================== +Getting Started +==================== + +Welcome to Burr's documentation! + +The following section of the docs will walk you through Burr and how to integrate into your project: + +.. toctree:: + simple + chatbot + ml_training + simulation diff --git a/docs/examples/ml_training.rst b/docs/examples/ml_training.rst new file mode 100644 index 00000000..f2d7395d --- /dev/null +++ b/docs/examples/ml_training.rst @@ -0,0 +1,5 @@ +==================== +ML Model Training +==================== + +TODO -- link to github diff --git a/docs/examples/simple.rst b/docs/examples/simple.rst new file mode 100644 index 00000000..853fb5f0 --- /dev/null +++ b/docs/examples/simple.rst @@ -0,0 +1,7 @@ +==================== +Simple examples +==================== + +TODO -- link to github +- cowsay +- counter diff --git a/docs/examples/simulation.rst b/docs/examples/simulation.rst new file mode 100644 index 00000000..aac33c75 --- /dev/null +++ b/docs/examples/simulation.rst @@ -0,0 +1,5 @@ +==================== +Simulation +==================== + +TODO -- link to github diff --git a/docs/getting_started/index.rst b/docs/getting_started/index.rst new file mode 100644 index 00000000..93a390b4 --- /dev/null +++ b/docs/getting_started/index.rst @@ -0,0 +1,13 @@ +==================== +Getting Started +==================== + +Welcome to Burr's documentation! + +The following section of the docs will walk you through Burr and how to integrate into your project: + +.. toctree:: + why-burr + install + simple-example + up-next diff --git a/docs/getting_started/install.rst b/docs/getting_started/install.rst new file mode 100644 index 00000000..15f64ec7 --- /dev/null +++ b/docs/getting_started/install.rst @@ -0,0 +1,24 @@ +==================== +Getting Started +==================== + +Burr requires almost no dependencies. Every "extra"/"plugin" is an additional install target. Note, if you're using zsh, +you'll need to add quotes around the install target, like `pip install "burr[visualization]"`. + +.. code-block:: bash + + pip install burr + +To get visualization capabilities, you can install the `burr[visualization]` extra: + +.. code-block:: bash + + pip install burr[visualization] + +And to visualize your state machines on streamlit, you can install the `burr[streamlit]` extra: + +.. code-block:: bash + + pip install burr[streamlit] + +Don't worry, you can always install these extras later. diff --git a/docs/getting_started/simple-example.rst b/docs/getting_started/simple-example.rst new file mode 100644 index 00000000..a7cd26c5 --- /dev/null +++ b/docs/getting_started/simple-example.rst @@ -0,0 +1,4 @@ +================= +Simple Example +================= +TODO -- cowsay diff --git a/docs/getting_started/up-next.rst b/docs/getting_started/up-next.rst new file mode 100644 index 00000000..1a11885e --- /dev/null +++ b/docs/getting_started/up-next.rst @@ -0,0 +1,4 @@ +================= +Next Steps +================= +TODO -- cowsay diff --git a/docs/getting_started/why-burr.rst b/docs/getting_started/why-burr.rst new file mode 100644 index 00000000..34a82aa8 --- /dev/null +++ b/docs/getting_started/why-burr.rst @@ -0,0 +1,5 @@ +================= +Why Burr +================= + +TODO -- cowsay diff --git a/docs/index.rst b/docs/index.rst new file mode 100644 index 00000000..02b31522 --- /dev/null +++ b/docs/index.rst @@ -0,0 +1,27 @@ +.. Burr documentation master file, created by + sphinx-quickstart on Thu Feb 8 10:04:25 2024. + You can adapt this file completely to your liking, but it should at least + contain the root `toctree` directive. + +.. include:: main.md + +Welcome to Burr's documentation! +================================ + +.. toctree:: + :maxdepth: 2 + :caption: User guide: + + getting_started/index + concepts/index + +.. toctree:: + :maxdepth: 2 + :caption: Examples: + + examples/index +.. toctree:: + :maxdepth: 2 + :caption: Reference: + + reference/index diff --git a/docs/main.md b/docs/main.md new file mode 100644 index 00000000..87f4572f --- /dev/null +++ b/docs/main.md @@ -0,0 +1 @@ +# Welcome to Burr! diff --git a/docs/make.bat b/docs/make.bat new file mode 100644 index 00000000..32bb2452 --- /dev/null +++ b/docs/make.bat @@ -0,0 +1,35 @@ +@ECHO OFF + +pushd %~dp0 + +REM Command file for Sphinx documentation + +if "%SPHINXBUILD%" == "" ( + set SPHINXBUILD=sphinx-build +) +set SOURCEDIR=. +set BUILDDIR=_build + +%SPHINXBUILD% >NUL 2>NUL +if errorlevel 9009 ( + echo. + echo.The 'sphinx-build' command was not found. Make sure you have Sphinx + echo.installed, then set the SPHINXBUILD environment variable to point + echo.to the full path of the 'sphinx-build' executable. Alternatively you + echo.may add the Sphinx directory to PATH. + echo. + echo.If you don't have Sphinx installed, grab it from + echo.https://www.sphinx-doc.org/ + exit /b 1 +) + +if "%1" == "" goto help + +%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O% +goto end + +:help +%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O% + +:end +popd diff --git a/docs/reference/actions.rst b/docs/reference/actions.rst new file mode 100644 index 00000000..1b620acc --- /dev/null +++ b/docs/reference/actions.rst @@ -0,0 +1,8 @@ +================= +Action API +================= + +Use the action API to specify actions on your workflow. + +.. autoclass:: burr.core.action.Action + :special-members: __init__ diff --git a/docs/reference/application.rst b/docs/reference/application.rst new file mode 100644 index 00000000..133e5837 --- /dev/null +++ b/docs/reference/application.rst @@ -0,0 +1,3 @@ +================= +Application +================= diff --git a/docs/reference/conditions.rst b/docs/reference/conditions.rst new file mode 100644 index 00000000..0b92d3d8 --- /dev/null +++ b/docs/reference/conditions.rst @@ -0,0 +1,3 @@ +================= +Conditions +================= diff --git a/docs/reference/index.rst b/docs/reference/index.rst new file mode 100644 index 00000000..0852d46c --- /dev/null +++ b/docs/reference/index.rst @@ -0,0 +1,11 @@ +======================== +Reference Documentation +======================== + +.. toctree:: + application + actions + state + conditions + lifecycle + integrations/index diff --git a/docs/reference/integrations/hamilton.rst b/docs/reference/integrations/hamilton.rst new file mode 100644 index 00000000..9198b2d2 --- /dev/null +++ b/docs/reference/integrations/hamilton.rst @@ -0,0 +1,7 @@ +======================= +Hamilton +======================= + +Full Hamilton integration. Touch-points are custom Hamilton actions. This is installed by default. + +.. automodule:: burr.integrations.hamilton diff --git a/docs/reference/integrations/index.rst b/docs/reference/integrations/index.rst new file mode 100644 index 00000000..d84e9b8f --- /dev/null +++ b/docs/reference/integrations/index.rst @@ -0,0 +1,11 @@ +================ +Integrations +================ + +Integrations -- we will be adding more + +.. toctree:: + :maxdepth: 2 + + hamilton + streamlit diff --git a/docs/reference/integrations/streamlit.rst b/docs/reference/integrations/streamlit.rst new file mode 100644 index 00000000..176af4fb --- /dev/null +++ b/docs/reference/integrations/streamlit.rst @@ -0,0 +1,14 @@ +======================= +Streamlit +======================= + +Full Streamlit integration. Touch-points are custom Hamilton actions. + +Install with pip: + +.. code-block:: bash + + pip install burr[streamlit] + +.. autoclass:: burr.integrations.hamilton.Hamilton + :special-members: __init__ diff --git a/docs/reference/lifecycle.rst b/docs/reference/lifecycle.rst new file mode 100644 index 00000000..7c66265d --- /dev/null +++ b/docs/reference/lifecycle.rst @@ -0,0 +1,3 @@ +================= +Lifecycle API +================= diff --git a/docs/reference/state.rst b/docs/reference/state.rst new file mode 100644 index 00000000..e6b95795 --- /dev/null +++ b/docs/reference/state.rst @@ -0,0 +1,3 @@ +================= +State API +================= diff --git a/pyproject.toml b/pyproject.toml index e80f3ff1..6a6924ef 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -40,6 +40,14 @@ tests = [ "pytest-asyncio", ] +documentation = [ + "sphinx", + "sphinx-autobuild", + "myst-parser", + "furo", + "sphinx-sitemap" +] + [tool.poetry.packages] include = [ { include = "burr", from = "." },