feat: implement baml-cli serve (#908)

Implements the following: - new output type `rest/openapi` generates an openapi.yaml - generators now may specify an `on_generate` hooks that run after baml-cli codegen - `baml-cli init` now supports `--client-type rest/openapi` and `--openapi-client-type` - when `--openapi-client-type` is specified, if it's one of go, java, php, ruby, or rust, it will bootstrap a generator that runs `openapi-generator-cli` with recommended args for that generator - introduce `baml-cli dev` which generates the openapi.yaml spec, calls `openapi-generator-cli` on it, starts a BAML-over-HTTP server, and hot reloads all of this if any of the baml files change - `MyBamlFunction` is exposed as `POST /call/MyBamlFunction`, with args as a json object mapping arg name to arg value - streaming responses are exposed via `POST /stream/MyBamlFunction` as well, but are unstable - 4xx and 5xx errors are deliberately distinguished for the user - `/_debug/ping` and `/_debug/status` are present for debugging connectivity and auth state - `baml-cli serve` exposes the BAML-over-HTTP server as well, but without hot reload - provide docs explaining how to use this in `docker-compose` - supports `BAML_PASSWORD` for a basic authz check - if users pass in mismatched args or the wrong args for a given BAML function, or call the wrong BAML function, we return a distinct `UserFailure` error variant of `LLMResponse` To implement or discuss before initial release: - default log level is currently `info`, including `baml_events`, even though the python/ts FFI libraries default log level to `warn` - I think this is reasonable behavior (there's a lot of different things we use `log::info` for) - `rest/openapi` does not support generating code for fields with type of `null` and causes codegen to fail (unions and optionals are handled specially) - we could in theory generate a `{ nullable: true, type: PLACEHOLDER }` type for these, but as a user of this i think i would be very surprised and confused if i ran into this Fast follow work: - streaming is unstable right now - dynamic types, dynamic clients are unsupported - maybe: provide installer scripts for `baml-cli` without going through `npx`
BoundaryML · Sep 8, 2024 · 484fa93 · 484fa93
1 parent 6f58c9e
commit 484fa93
Show file tree

Hide file tree

Showing 98 changed files with 16,216 additions and 7,761 deletions.
diff --git a/.envrc b/.envrc
@@ -3,3 +3,5 @@ PATH_add "$(brew --prefix llvm)/bin"
 
 # Use mise for tool versioning, e.g. Ruby
 eval "$(mise activate bash)" || echo "Please run 'brew install mise'"
+
+export CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER=aarch64-linux-gnu-gcc
diff --git a/.github/workflows/primary.yml b/.github/workflows/primary.yml
@@ -86,5 +86,21 @@ jobs:
         with:
           workspaces: engine
       - name: Test Rust
-        run: cargo test
+        # skip doctests & integration tests (the latter require OPENAI_API_KEY and ANTHROPIC_API_KEY to be set)
+        run: cargo test --lib
         working-directory: engine
+  build-wasm:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: jdx/mise-action@v2
+      - uses: dtolnay/rust-toolchain@stable
+        with:
+          toolchain: stable
+      - run: rustup target add wasm32-unknown-unknown
+      - uses: Swatinem/rust-cache@v2
+        with:
+          workspaces: engine
+      - name: Build rust for wasm32
+        run: cargo build --target=wasm32-unknown-unknown
+        working-directory: engine/baml-schema-wasm
diff --git a/.mise.toml b/.mise.toml
@@ -1,5 +1,4 @@
 [tools]
 node = "latest"
 ruby = "3.1"
-pnpm = "latest"
-poetry = "latest"
+pnpm = "9.9"
diff --git a/docs/assets/images/docs_latest/openapi-diagram.drawio b/docs/assets/images/docs_latest/openapi-diagram.drawio
@@ -0,0 +1,34 @@
+<mxfile host="app.diagrams.net" agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36" version="24.7.7">
+  <diagram name="Page-1" id="IBP3ZyO2FRTOo5MFRLAC">
+    <mxGraphModel dx="1434" dy="761" grid="1" gridSize="5" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="850" pageHeight="1100" math="0" shadow="0">
+      <root>
+        <mxCell id="0" />
+        <mxCell id="1" parent="0" />
+        <mxCell id="iyppU8A8im01KiWM7F2Y-2" value="POST /call/MyBamlFunction" style="text;html=1;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;fontFamily=Helveticaui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, &quot;Liberation Mono&quot;, &quot;Courier New&quot;, monospace;fontColor=#7E55F8;fontStyle=0;fontSize=14;" parent="1" vertex="1">
+          <mxGeometry x="420" y="290" width="130" height="30" as="geometry" />
+        </mxCell>
+        <mxCell id="iyppU8A8im01KiWM7F2Y-5" value="&lt;pre&gt;&lt;br&gt;&lt;/pre&gt;" style="rounded=1;whiteSpace=wrap;html=1;" parent="1" vertex="1">
+          <mxGeometry x="585" y="285" width="160" height="90" as="geometry" />
+        </mxCell>
+        <mxCell id="iyppU8A8im01KiWM7F2Y-8" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;entryX=0;entryY=0.5;entryDx=0;entryDy=0;strokeColor=#7E55F8;strokeWidth=1.5;" parent="1" source="iyppU8A8im01KiWM7F2Y-6" target="iyppU8A8im01KiWM7F2Y-5" edge="1">
+          <mxGeometry relative="1" as="geometry" />
+        </mxCell>
+        <mxCell id="iyppU8A8im01KiWM7F2Y-6" value="&lt;pre style=&quot;font-size: 14px;&quot;&gt;&lt;font style=&quot;font-size: 14px;&quot; face=&quot;Helvetica&quot;&gt;OpenAPI&lt;br style=&quot;font-size: 14px;&quot;&gt;client&lt;/font&gt;&lt;/pre&gt;" style="rounded=1;whiteSpace=wrap;html=1;fontFamily=OpenSans, -apple-system, &quot;system-ui&quot;, &quot;Segoe UI&quot;, Roboto, Oxygen, Ubuntu, Cantarell, &quot;Open Sans&quot;, &quot;Helvetica Neue&quot;, sans-serif;fontSize=14;" parent="1" vertex="1">
+          <mxGeometry x="290" y="290" width="100" height="80" as="geometry" />
+        </mxCell>
+        <mxCell id="Q0mZcIDexRsLYvT9Odtg-4" value="MyApp" style="text;html=1;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;fontSize=16;fontFamily=OpenSans, -apple-system, &quot;system-ui&quot;, &quot;Segoe UI&quot;, Roboto, Oxygen, Ubuntu, Cantarell, &quot;Open Sans&quot;, &quot;Helvetica Neue&quot;, sans-serif;" vertex="1" parent="1">
+          <mxGeometry x="290" y="260" width="100" height="30" as="geometry" />
+        </mxCell>
+        <mxCell id="Q0mZcIDexRsLYvT9Odtg-5" value="baml-cli serve" style="text;html=1;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;fontFamily=Helveticaui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, &quot;Liberation Mono&quot;, &quot;Courier New&quot;, monospace;fontSize=16;" vertex="1" parent="1">
+          <mxGeometry x="585" y="255" width="160" height="30" as="geometry" />
+        </mxCell>
+        <mxCell id="Q0mZcIDexRsLYvT9Odtg-6" value="MyBamlFunction.baml" style="text;html=1;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;fontFamily=Helveticaui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, &quot;Liberation Mono&quot;, &quot;Courier New&quot;, monospace;fontColor=#000000;" vertex="1" parent="1">
+          <mxGeometry x="585" y="345" width="165" height="30" as="geometry" />
+        </mxCell>
+        <mxCell id="Q0mZcIDexRsLYvT9Odtg-8" value="" style="shape=image;imageAspect=0;aspect=fixed;verticalLabelPosition=bottom;verticalAlign=top;image=data:image/svg+xml,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIGNsYXNzPSJsdWNpZGUgbHVjaWRlLWZpbGUtY29kZS0yIiBzdHJva2UtbGluZWpvaW49InJvdW5kIiBzdHJva2UtbGluZWNhcD0icm91bmQiIHN0cm9rZS13aWR0aD0iMSIgc3Ryb2tlPSJjdXJyZW50Q29sb3IiIGZpbGw9Im5vbmUiIHZpZXdCb3g9IjAgMCAyNCAyNCIgaGVpZ2h0PSIyNCIgd2lkdGg9IjI0Ij48cGF0aCBkPSJNNCAyMmgxNGEyIDIgMCAwIDAgMi0yVjdsLTUtNUg2YTIgMiAwIDAgMC0yIDJ2NCIvPjxwYXRoIGQ9Ik0xNCAydjRhMiAyIDAgMCAwIDIgMmg0Ii8+PHBhdGggZD0ibTUgMTItMyAzIDMgMyIvPjxwYXRoIGQ9Im05IDE4IDMtMy0zLTMiLz48L3N2Zz4=;" vertex="1" parent="1">
+          <mxGeometry x="643" y="296" width="49" height="49" as="geometry" />
+        </mxCell>
+      </root>
+    </mxGraphModel>
+  </diagram>
+</mxfile>
diff --git a/docs/assets/images/docs_latest/openapi-diagram.drawio.svg b/docs/assets/images/docs_latest/openapi-diagram.drawio.svg
diff --git a/docs/docs.yml b/docs/docs.yml
@@ -8,6 +8,7 @@ instances:
         repo: baml
         branch: canary
 
+
 title: BAML Documentation
 
 navigation:
@@ -26,6 +27,8 @@ navigation:
             path: docs/get-started/quickstart/typescript.mdx
           - page: Ruby
             path: docs/get-started/quickstart/ruby.mdx
+          - page: Any Language (OpenAPI)
+            path: docs/get-started/quickstart/openapi.mdx
           - page: VSCode
             path: docs/get-started/quickstart/editors-vscode.mdx
           - page: Other Editors
@@ -47,6 +50,8 @@ navigation:
             path: docs/get-started/deploying/nextjs.mdx
           - page: AWS
             path: docs/get-started/deploying/aws.mdx
+          - page: Docker (OpenAPI)
+            path: docs/get-started/deploying/openapi.mdx
   - section: BAML Snippets
     contents:
       - section: General BAML Syntax

diff --git a/docs/docs/calling-baml/calling-functions.mdx b/docs/docs/calling-baml/calling-functions.mdx
@@ -104,5 +104,15 @@ end
 
 ```
 
+</Tab>
+<Tab title="OpenAPI">
+
+If you're using an OpenAPI-generated client, every BAML function will generate
+a corresponding OpenAPI function.
+
+See our [OpenAPI
+quickstart](/docs/get-started/quickstart/openapi#use-your-openapi-client) for
+more details.
+
 </Tab>
 </Tabs>
diff --git a/docs/docs/calling-baml/client-registry.mdx b/docs/docs/calling-baml/client-registry.mdx
@@ -6,9 +6,11 @@ slug: docs/calling-baml/client-registry
 
 If you need to modify the model / parameters for an LLM client at runtime, you can modify the `ClientRegistry` for any specified function.
 
-<CodeBlocks>
+<Tabs>
 
-```python Python
+<Tab title="Python">
+
+```python
 import os
 from baml_py import ClientRegistry
 
@@ -27,7 +29,10 @@ async def run():
     res = await b.ExtractResume("...", { "client_registry": cr })
 ```
 
-```typescript TypeScript
+</Tab>
+
+<Tab title="TypeScript">
+```typescript
 import { ClientRegistry } from '@boundaryml/baml'
 
 async function run() {
@@ -45,8 +50,11 @@ async function run() {
     const res = await b.ExtractResume("...", { clientRegistry: cr })
 }
 ```
+</Tab>
+
+<Tab title="Ruby">
 
-```ruby Ruby
+```ruby
 require_relative "baml_client/client"
 
 def run
@@ -73,8 +81,18 @@ end
 # Call the asynchronous function
 run
 ```
+</Tab>
+
+<Tab title="OpenAPI">
+Dynamic types are not yet supported when used via OpenAPI.
+
+Please let us know if you want this feature, either via [Discord] or [GitHub][openapi-feedback-github-issue].
+
+[Discord]: https://discord.gg/BTNBeXGuaS
+[openapi-feedback-github-issue]: https://github.com/BoundaryML/baml/issues/892
+</Tab>
 
-</CodeBlocks>
+</Tabs>
 
 ## ClientRegistry Interface
 

diff --git a/docs/docs/calling-baml/dynamic-types.mdx b/docs/docs/calling-baml/dynamic-types.mdx
@@ -36,9 +36,10 @@ define in BAML will be available as properties of `TypeBuilder`. Think of the
 typebuilder as a registry of modified runtime types that the baml function will
 read from when building the output schema in the prompt.
 
-<CodeBlocks>
+<Tabs>
 
-```python Python
+<Tab title="Python">
+```python
 from baml_client.type_builder import TypeBuilder
 from baml_client import b
 
@@ -52,8 +53,10 @@ async def run():
   print(res)
 
 ```
+</Tab>
 
-```typescript TypeScript
+<Tab title="TypeScript">
+```typescript
 import TypeBuilder from '../baml_client/type_builder'
 import {
   b
@@ -68,8 +71,10 @@ async function run() {
   console.log(res)
 }
 ```
+</Tab>
 
-```ruby Ruby
+<Tab title="Ruby">
+```ruby
 require_relative '../baml_client'
 
 def run
@@ -81,7 +86,18 @@ def run
   puts res
 end
 ```
-</CodeBlocks>
+</Tab>
+
+<Tab title="OpenAPI">
+Dynamic types are not yet supported when used via OpenAPI.
+
+Please let us know if you want this feature, either via [Discord] or [GitHub][openapi-feedback-github-issue].
+
+[Discord]: https://discord.gg/BTNBeXGuaS
+[openapi-feedback-github-issue]: https://github.com/BoundaryML/baml/issues/892
+</Tab>
+
+</Tabs>
 
 ### Dynamic BAML Classes
 Existing BAML classes marked with @@dynamic will be available as properties of `TypeBuilder`.

diff --git a/docs/docs/calling-baml/generate-baml-client.mdx b/docs/docs/calling-baml/generate-baml-client.mdx
@@ -3,7 +3,6 @@ title: Generate the BAML Client
 slug: docs/calling-baml/generate-baml-client
 ---
 
-
 <Info>This page assumes you've already defined a function in BAML. If you
 haven't done that yet, check out [how to define a function].</Info>
 
@@ -56,6 +55,10 @@ bundle add baml
 bundle exec baml-cli generate --from path/to/baml_src
 ```
 
+```bash OpenAPI
+npx @boundaryml/baml-cli generate --from path/to/baml_src
+```
+
 </CodeBlocks>
 
 ## Best Practices
@@ -85,7 +88,7 @@ generator target {
     default_client_mode "sync"
     
     // Version of runtime to generate code for (should match installed baml-py version)
-    version "0.50.0"
+    version "0.54.0"
 }
 ```
 
@@ -103,23 +106,37 @@ generator target {
     default_client_mode "async"
     
     // Version of runtime to generate code for (should match the package @boundaryml/baml version)
-    version "0.50.0"
+    version "0.54.0"
 }
 ```
 
-```baml Ruby
+```baml Ruby (beta)
 generator target {
     // Valid values: "python/pydantic", "typescript", "ruby/sorbet"
     output_type "ruby/sorbet"
 
     // Where the generated code will be saved (relative to baml_src/)
     output_dir "../"
     
-    // This is a no-op for Ruby and can be ignored as futures are colorless in Ruby
-    // default_client_mode "sync"
+    // Version of runtime to generate code for (should match installed `baml` package version)
+    version "0.54.0"
+}
+```
 
+```baml OpenAPI
+generator target {
+    // Valid values: "python/pydantic", "typescript", "ruby/sorbet", "rest/openapi"
+    output_type "rest/openapi"
+
+    // Where the generated code will be saved (relative to baml_src/)
+    output_dir "../"
+    
     // Version of runtime to generate code for (should match installed `baml` package version)
-    version "0.50.0"
+    version "0.54.0"
+
+    // 'baml-cli generate' will run this after generating openapi.yaml, to generate your OpenAPI client
+    // This command will be run from within $output_dir
+    on_generate "npx @openapitools/openapi-generator-cli generate -i openapi.yaml -g OPENAPI_CLIENT_TYPE -o ."
 }
 ```
 

diff --git a/docs/docs/calling-baml/streaming.mdx b/docs/docs/calling-baml/streaming.mdx
@@ -227,5 +227,10 @@ if __FILE__ == $0
 end
 ```
 
+</Tab>
+<Tab title="OpenAPI">
+
+Streaming is not yet supported via OpenAPI, but it will be coming soon!
+
 </Tab>
 </Tabs>
diff --git a/docs/docs/comparisons/pydantic.mdx b/docs/docs/comparisons/pydantic.mdx
@@ -337,7 +337,7 @@ The Boundary toolkit helps you iterate seamlessly compared to Pydantic.
 
 Here's all the BAML code you need to solve the Extract Resume problem from earlier (VSCode prompt preview is shown on the right):
 
-<img src="../../assets/images/v3/resume_playground1.png" />
+<img src="/assets/images/v3/resume_playground1.png" />
 
 <Note>
 Here we use a "GPT4" client, but you can use any model. See [client docs](/docs/syntax/client/client)
@@ -395,16 +395,16 @@ Converting the `Resume` into an array of resumes requires a single line change i
 
 In this image we removed `education` and made `Resume` an array:
 
-<img src="../../assets/images/v3/resume_change1.png" />
+<img src="/assets/images/v3/resume_change1.png" />
 
 After changing the BAML file, the Python-BAML function instantly changes as well. VSCode shows you the output is a list now. Everything is fully typed.
-<img src="../../assets/images/v3/resume_error.png" />
+<img src="/assets/images/v3/resume_error.png" />
 
 
 Adding retries or resilience requires just [a couple of modifications](/docs/syntax/client/retry). And best of all, **you can test things instantly, without leaving your VSCode**:
 
 {/* <video src="../../assets/images/v3/testing_20s.mp4" autoplay loop playsInline muted controls className="w-full aspect-video"></video> */}
-<img src="../../assets/images/v3/testing_2.gif" />
+<img src="/assets/images/v3/testing_2.gif" />
 
 
 ### Conclusion

diff --git a/docs/docs/contact.mdx b/docs/docs/contact.mdx
@@ -9,10 +9,12 @@ it's a bug, feature request, or just general comment.
 
 You can reach us using:
 
-    - [Discord](https://discord.gg/BTNBeXGuaS) (fastest for a "how do i... ?")
+    - [Discord] (fastest for a "how do i... ?")
     - [GitHub](https://github.com/BoundaryML/baml/issues) (for bugs and feature requests)
     - LinkedIn: [https://www.linkedin.com/company/boundaryml](https://www.linkedin.com/company/boundaryml)
     - Email: [[email protected]](mailto:[email protected])
     - Twitter: [@boundaryml](https://twitter.com/boundaryml)
 
 We try our best to respond as quickly as possible, so don't hesitate to reach out!
+
+[Discord]: https://discord.gg/BTNBeXGuaS
diff --git a/docs/docs/doc-snippets/vscode-settings.mdx b/docs/docs/doc-snippets/vscode-settings.mdx
@@ -9,6 +9,8 @@ If set, will make the extension use the baml-cli you have installed on your syst
 If you use unix, you can run `where baml-cli` in your project to figure out what the path is.
 
 If you have multiple BAML projects, ensure that this path is set to baml package installed for that project.
+
+If you're using it via `npx`, you can set it as `npx @boundaryml/baml@VERSION` where `VERSION` is the version you want to pin your version to.
 </ParamField>
 
 <ParamField

diff --git a/docs/docs/get-started/debugging/vscode-playground.mdx b/docs/docs/get-started/debugging/vscode-playground.mdx
@@ -7,7 +7,7 @@ slug: docs/get-started/debugging/vscode-playground
 - Close the playground and reopen it
 - Try reloading the entire window by pressing `Ctrl + Shift + P` or `Cmd + Shift + P` and typing `Developer: Reload Window`
 - Ensure your VSCode Extension for BAML is up-to-date (It should should it its up-to-date in the Extensions tab in VSCode)
-<img src="../../../assets/images/docs_latest/vscode/extension-status.png" />
+<img src="/assets/images/docs_latest/vscode/extension-status.png" />
 
 - If nothing works, please file an issue on [Github](https://github.com/BoundaryML/baml/issues), ideally with a screenshot of the error and the steps to reproduce it.
 
@@ -29,7 +29,11 @@ We've seen sparse repros of this, but closing the playground and reopening it sh
 
 You can debug the actual network request being made by BAML by opening developer tools:
 
-<img src="../../../assets/images/docs_latest/vscode/dev-tools.png" />
+<img src="/assets/images/docs_latest/vscode/dev-tools.png" />
 
 ### BAML Client Generation Disabled
-This means your `generator` clause in your BAML file is out of sync with the version of the BAML extension you have installed. See [Generate](../../calling-baml/generate-baml-client.mdx) for more details.
+This means that either:
+
+- your `generator` clause in your BAML file is out of sync with the version of the BAML extension you have installed. See [Generate](../../calling-baml/generate-baml-client.mdx) for more details, or
+
+- one of your `generator` clauses sets `output_type "rest/openapi"`, which means you need to use `baml-cli dev` to run the generator
Original file line number	Diff line number	Diff line change
Expand Up		@@ -3,3 +3,5 @@ PATH_add "$(brew --prefix llvm)/bin"

		# Use mise for tool versioning, e.g. Ruby
		eval "$(mise activate bash)" \|\| echo "Please run 'brew install mise'"

		export CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER=aarch64-linux-gnu-gcc
-Original file line number
+Diff line change
@@ Expand Up / @@ -227,5 +227,10 @@ if __FILE__ == $0 @@
     end
     ```
+    </Tab>
+    <Tab title="OpenAPI">
+    Streaming is not yet supported via OpenAPI, but it will be coming soon!
     </Tab>
     </Tabs>