Skip to content

v0.15.2

Compare
Choose a tag to compare
@github-actions github-actions released this 25 Oct 04:07
· 109 commits to main since this release

Added

  • pw.io.deltalake.read now supports custom S3 Delta Lakes with HTTP endpoints.
  • pw.io.deltalake.read now supports specifying both a custom endpoint and a custom region for Delta Lakes via pw.io.s3.AwsS3Settings.

Changed

  • Indices in pathway.stdlib.indexing.nearest_neighbors can now work also on numpy arrays. Previously they only accepted list[float]. Working with numpy arrays improves memory efficiency.
  • pw.io.s3.read has been optimized to minimize new object requests whenever possible.
  • It is now possible to set the size limit of cache in pw.udfs.DiskCache.
  • State persistence now uses a single backend for both metadata and stream storage. The pw.persistence.Config.simple_config method is therefore deprecated. Now you can use the pw.persistence.Config constructor with the same parameters that were previously used in simple_config.

Fixed

  • pw.io.bigquery.write connector now correctly handles pw.Json columns.