Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

/v1/models - format output to match OpenAI styling #34

Merged
merged 4 commits into from
Oct 29, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -288,7 +288,7 @@

Please note that while streaming is available for regular text generation, the streaming implementation for function calling is still in development and does not yet fully comply with the OpenAI specification.

5. **Listing Available Models**
5. **List Supported Models**

To see all vision and language models supported by MLX:

Expand All @@ -300,7 +300,7 @@
print(response.json())
```

6. **List Available Models**
6. **Add Available Model**

You can add new models to the API:

Expand All @@ -316,9 +316,9 @@
print(response.json())
```

7. **Listing Available Models**
7. **List Available Models**

To see all available models:
Provides the list of available models that have been added in a OpenAI compliant format:

```python
import requests
Expand Down
19 changes: 14 additions & 5 deletions fastmlx/fastmlx.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
import argparse
import asyncio
import os
import time
from typing import Any, Dict, List
from urllib.parse import unquote

Expand Down Expand Up @@ -266,12 +267,20 @@ async def get_supported_models():
@app.get("/v1/models")
async def list_models():
"""
List all available (loaded) models.

Returns:
dict (dict): A dictionary containing the list of available models.
Get list of models - provided in OpenAI API compliant format.
"""
return {"models": await model_provider.get_available_models()}
models = await model_provider.get_available_models()
models_data = []
for model in models:
models_data.append(
{
"id": model,
"object": "model",
"created": int(time.time()),
"owned_by": "system",
}
)
return {"object": "list", "data": models_data}


@app.post("/v1/models")
Expand Down
Loading