Code

ML inference fixes & features#251

Code

Merged

percevalw merged 10 commits intomasteraphp/edsnlp:masterfrom ml-inferenceaphp/edsnlp:ml-inferenceCopy head branch name to clipboardJan 18, 2024

+285-37

Commits

Commits on Jan 12, 2024

• fix: don't turn parquet fragments gen into list to avoid oom errors

  percevalw

  committed
  1b41057

  Copy full SHA for 1b41057

• fix: support doc->list converter with parquet files

  percevalw

  committed
  dd6f09c

  Copy full SHA for dd6f09c

Commits on Jan 15, 2024

• feat: support loading model from package

  percevalw

  committed
  ef6c48e

  Copy full SHA for ef6c48e

• fix: list both edsnlp & spacy factories when lookup fail

  percevalw

  committed
  b2d61e3

  Copy full SHA for b2d61e3

• feat: slice & batch over transformer windows to avoid GPU OOM errors

  percevalw

  committed
  9e3c61b

  Copy full SHA for 9e3c61b

• fix: avoid putting all first batches to the same gpu

  percevalw

  committed
  2b050cb

  Copy full SHA for 2b050cb

• feat: support filesystem parameter in parquet functions

  percevalw

  committed
  30e3050

  Copy full SHA for 30e3050

Commits on Jan 17, 2024

• perf: decrease the max number of enqueued batches per process

  percevalw

  committed
  8e001eb

  Copy full SHA for 8e001eb

Commits on Jan 18, 2024

• docs: fix typos and add a docstring to the deprecated pipe helper

  percevalw

  committed
  afa3cb6

  Copy full SHA for afa3cb6

• feat: auto cast doc._.note_datetime from int/float/str/date/datetime to pendulum

  percevalw

  committed
  999fcdc

  Copy full SHA for 999fcdc