Skip to content

Commit

Permalink
chore: Add skip_metadata parameter to RelikServer
Browse files Browse the repository at this point in the history
  • Loading branch information
Riccorl committed Aug 6, 2024
1 parent 6e1f3f7 commit ee6f03f
Show file tree
Hide file tree
Showing 2 changed files with 14 additions and 1 deletion.
14 changes: 13 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -407,7 +407,19 @@ docker pull sapienzanlp/relik:latest
and run the image with:

```bash
docker run -p 12345:8000 sapienzanlp/relik:latest -c relik-ie/relik-cie-small
docker run -p 12345:8000 sapienzanlp/relik:latest --config relik-ie/relik-cie-small
```

`sapienzanlp/relik:latest` points to the latest CUDA version of the model. To run with GPU support:

```bash
docker run -p 12345:8000 --gpus all sapienzanlp/relik:latest --config relik-ie/relik-cie-small --device cuda
```

Tip: you can mount your `.cache/huggingface` folder to the docker container to avoid downloading the model weights every time you run the container.

```bash
docker run -p 12345:8000 -v ~/.cache/huggingface:/home/relik-user/.cache/huggingface sapienzanlp/relik:latest --config relik-ie/relik-cie-small
```

The API will be available at `http://localhost:12345`. It exposes a single endpoint `/relik` with several parameters that can be passed to the model.
Expand Down
1 change: 1 addition & 0 deletions relik/inference/serve/backend/fastapi_be.py
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,7 @@ def __init__(
retriever_precision=self.retriever_precision,
document_index_precision=self.document_index_precision,
reader_precision=self.reader_precision,
skip_metadata=True,
)

self.router = APIRouter()
Expand Down

0 comments on commit ee6f03f

Please sign in to comment.