v0.5.2

Added

• interval_join now supports forgetting old entries. The configuration can be passed using behavior parameter of interval_join method.
• Decorator @table_transformer for marking that functions take Tables as arguments.
• Namespace for all columns Table.C.*.
• Output connectors now provide logs about the number of entries written and time taken.
• Filesystem connectors now support reading whole files as rows.

v0.5.1

Fixed

• select operates only on consistent states.

v0.5.0

Added

• Schema method typehints that returns dict of mypy-compatible typehints.
• Support for JSON parsing from CSV sources.
• restrict method in Table to restrict table universe to the universe of the other table.
• Better support for postgresql types in the output connector.

Changed

• BREAKING: tuple reducer used after intervals_over window now sorts values by time.
• BREAKING: expressions used in select, filter, flatten, with_columns, with_id, with_id_from have to have the same universe as the table. Earlier it was possible to use an expression from a superset of a table universe. To use expressions from wider universes, one can use restrict on the expression source table.
• BREAKING: pw.universes.promise_are_equal(t1, t2) no longer allows to use references from t1 and t2 in a single expression. To change the universe of a table, use with_universe_of.
• BREAKING: ix and ix_ref are temporarily broken inside joins (both temporal and ordinary).
• select, filter, concat keep columns as a single stream. The work for other operators is ongoing.
• BREAKING: renamed Table method dtypes to typehints. It now returns a dict of mypy-compatible typehints.
• BREAKING: Schema.__getitem__ returns a data class ColumnSchema containing all related information on particular column.

Fixed

• Optional types other than string correctly output to PostgreSQL.

v0.4.1

Added

• Support for messages compressed with zstd in the Kafka connector.

v0.4.0

Added

• Support for JSON data format, including pw.Json type.
• Methods as_int(), as_float(), as_str(), as_bool() to convert values from Json.

Changed

• Method get() and [] to support accessing elements in Jsons.
• Function pw.assert_table_has_schema for writing asserts checking, whether given table has the same schema as the one that is given as an argument.
• BREAKING: ix and ix_ref operations are now standalone transformations of pw.Table into pw.Table. Most of the usages remain the same, but sometimes user needs to provide a context (when e.g. using them inside join or groupby operations). ix and ix_ref are temporarily broken inside temporal joins.

Fixed

• Fixed a bug where new-style optional types (e.g. int | None) were translated to Any dtype.

v0.3.4

Fixed

• Incompatible beartype version is now excluded from dependencies.

v0.3.3

Added

• Module pathway.dt to construct and manipulate DTypes.
• New argument keep_queries in pw.io.http.rest_connector.

Changed

• Internal representation of DTypes. Inputting types is compatible backwards.
• Temporal functions now accept arguments of mixed types (ints and floats). For example, pw.temporal.interval can use ints while columns it interacts with are floats.
• Single-element arrays are now treated as arrays, not as scalars.

Fixed

• to_string() method on datetimes always prints 9 fractional digits.
• %f format code in strptime() parses fractional part of a second correctly regardless of the number of digits.

v0.3.2

Added

• Table.cast_to_types() function that can perform pathway.cast on multiple columns.
• intervals_over window, which allows to get temporally close data to given times.
• demo.replay_csv_with_time function that can replay a CSV file following the timestamps of a given column.

Fixed

• Static data is now copied to ensure immutability.
• Improved error tracing mechanism to work with any type of error.

v0.3.1

Added

• tuple reducer, that returns a tuple with values.
• ndarray reducer, that returns an array with values.

Changed

• numpy arrays of int32, uint32 and float32 are now converted to their 64-bit variants instead of tuples.
• KNNIndex interface to take columns as inputs.
• Reducers now check types of their arguments.

Fixed

• Fixed delayed reporting of output connector errors.
• Python objects are now freed more often, reducing peak memory usage.

v0.3.0

Added

• @ (matrix multiplication) operator.

Changed

• Python version 3.10 or later is now required.
• Type checking is now more strict.