v5.5.0

This release adds workflow streams and DuckDB as a database backend

↪️️ Workflow streams enable server-side processing of large datasets. Streams iteratively pass content to workflows, no need to pass bulk data through the API.

🦆 DuckDB is a new database backend. Certain larger non-vector driven queries and aggregations will now run significantly faster than with SQLite.

See below for full details on the new features, improvements and bug fixes.

New Features

• Add workflow streams (#461)
• Add DuckDB support (#462)

Improvements

• Modify translation pipeline langdetect parameter to accept language detection function good first issue - Thank you @saucam! (#423, #444)
• Pass generation keyword arguments to underlying text generation pipeline (#457)
• Replace original prompt in text generation pipeline (#459)

Bug Fixes

• Issue with upsert and graph (#421)
• Upsert API fails with graph config while performing after /delete (#435)
• Build errors with latest onnxmltools package (#449)
• Fix issue with embeddings reindex and stale function references (#453)
• Problem with the workflow builder (#454)
• Check for empty queue before attempting to convert inputs to dictionaries (#456)
• Fix issue with latest version of Transformers and TokenDetection.save_pretrained (#458)