added Text embedding pipeline

[nanbo-liu]

Tested it with the following model_setting.json in a directory st_embed
/home/nliu/all-mpnet-base-v2 is a sentence_transformer model
sentence-transformers/all-MiniLM-L6-v2 that I downloaded to my local directory

{
    "name": "st_embed",
    "implementation": "mlserver_huggingface.HuggingFaceRuntime",
    "parameters": {
        "extra": {
            "task": "sentence-embedding",
            "pretrained_model": "/home/nliu/all-mpnet-base-v2"
        }
    }
}

and run mlserver start st_embed

Python script to run inference

import requests
import json
import numpy as np

payload = {
    "inputs": [
        {
            "name": "sentences",
            "shape": [1],
            "datatype": "BYTES",
            "data": [
                "My kitten's name is JoJo,",
                "Today is the best day of my life",
                "The quick brown fox jumps over the lazy dog.",
            ],
        }
    ],
}
response = requests.post("http://localhost:8080/v2/models/st_embed/infer", json=payload)

data = response.json()

pred = json.loads(data["outputs"][0]["data"][0])
pred = np.array(pred["embeddings"])

[geodavic]

nit: could we make it sentence-embedding, as that is the format other pipelines use?

[nanbo-liu]

resolved in the latest commit