Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update keras to 3.7.0 #309

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Conversation

pyup-bot
Copy link
Collaborator

This PR updates keras from 3.4.1 to 3.7.0.

Changelog

3.7.0

API changes

- Add `flash_attention` argument to `keras.ops.dot_product_attention` and to `keras.layers.MultiHeadAttention`.
- Add `keras.layers.STFTSpectrogram` layer (to extract STFT spectrograms from inputs as a preprocessing step) as well as its initializer `keras.initializers.STFTInitializer`.
- Add `celu`, `glu`, `log_sigmoid`, `hard_tanh`, `hard_shrink`, `squareplus` activations.
- Add `keras.losses.Circle` loss.
- Add image visualization utilities `keras.visualization.draw_bounding_boxes`, `keras.visualization.draw_segmentation_masks`, `keras.visualization.plot_image_gallery`, `keras.visualization.plot_segmentation_mask_gallery`.
- Add `double_checkpoint` argument to `BackupAndRestore` to save a fallback checkpoint in case the first checkpoint gets corrupted.
- Add bounding box preprocessing support to image augmentation layers `CenterCrop`, `RandomFlip`, `RandomZoom`, `RandomTranslation`, `RandomCrop`.
- Add `keras.ops.exp2`, `keras.ops.inner` operations.

Performance improvements

- JAX backend: add native Flash Attention support for GPU (via cuDNN) and TPU (via a Pallas kernel). Flash Attention is now used automatically when the hardware supports it.
- PyTorch backend: add native Flash Attention support for GPU (via cuDNN). It is currently opt-in.
- TensorFlow backend: enable more kernel fusion via `bias_add`.
- PyTorch backend: add support for Intel XPU devices.

New Contributors

* mostafa-mahmoud made their first contribution in https://github.com/keras-team/keras/pull/20313
* TrAyZeN made their first contribution in https://github.com/keras-team/keras/pull/20321
* dryglicki made their first contribution in https://github.com/keras-team/keras/pull/20353
* jm-willy made their first contribution in https://github.com/keras-team/keras/pull/20352
* Gopi-Uppari made their first contribution in https://github.com/keras-team/keras/pull/20377
* nicolaspi made their first contribution in https://github.com/keras-team/keras/pull/20383
* sineeli made their first contribution in https://github.com/keras-team/keras/pull/20368
* LakshmiKalaKadali made their first contribution in https://github.com/keras-team/keras/pull/20403
* mwtoews made their first contribution in https://github.com/keras-team/keras/pull/20427
* mrry made their first contribution in https://github.com/keras-team/keras/pull/20438
* rohithpudari made their first contribution in https://github.com/keras-team/keras/pull/20447
* ma7555 made their first contribution in https://github.com/keras-team/keras/pull/20452
* jakevdp made their first contribution in https://github.com/keras-team/keras/pull/20469
* lcs-crr made their first contribution in https://github.com/keras-team/keras/pull/20503
* rameshdange5191 made their first contribution in https://github.com/keras-team/keras/pull/20525

**Full Changelog**: https://github.com/keras-team/keras/compare/v3.6.0...v3.7.0

3.6.0

Highlights

* New file editor utility: `keras.saving.KerasFileEditor`. Use it to inspect, diff, modify and resave Keras weights files. [See basic workflow here](https://colab.research.google.com/drive/1b1Rxf8xbOkMyvjpdJDrGzSnisyXatJsW?usp=sharing).
* New `keras.utils.Config` class for managing experiment config parameters.

BREAKING changes

* When using `keras.utils.get_file`, with `extract=True` or `untar=True`, the return value will be the path of the extracted directory, rather than the path of the archive.

Other changes and additions

* Logging is now asynchronous in `fit()`, `evaluate()`, `predict()`. This enables 100% compact stacking of `train_step` calls on accelerators (e.g. when running small models on TPU).
 - If you are using custom callbacks that rely on `on_batch_end`, this will disable async logging. You can force it back by adding `self.async_safe = True` to your callbacks. Note that the `TensorBoard` callback isn't considered async safe by default. Default callbacks like the progress bar are async safe.
* Added `keras.saving.KerasFileEditor` utility to inspect, diff, modify and resave Keras weights file.
* Added `keras.utils.Config` class. It behaves like a dictionary, with a few nice features:
 - All entries are accessible and settable as attributes, in addition to dict-style (e.g. `config.foo = 2` or `config["foo"]` are both valid)
 - You can easily serialize it to JSON via `config.to_json()`.
 - You can easily freeze it, preventing future changes, via `config.freeze()`. 
* Added bitwise numpy ops:
 * `bitwise_and`
 * `bitwise_invert`
 * `bitwise_left_shift`
 * `bitwise_not`
 * `bitwise_or`
 * `bitwise_right_shift`
 * `bitwise_xor`
* Added math op `keras.ops.logdet`.
* Added numpy op `keras.ops.trunc`.
* Added `keras.ops.dot_product_attention`.
* Added `keras.ops.histogram`.
* Allow infinite `PyDataset` instances to use multithreading.
* Added argument `verbose` in `keras.saving.ExportArchive.write_out()` method for exporting TF SavedModel.
* Added `epsilon` argument in `keras.ops.normalize`.
* Added `Model.get_state_tree()` method for retrieving a nested dict mapping variable paths to variable values (either as numpy arrays or backend tensors (default)). This is useful for rolling out custom JAX training loops.
* Added image augmentation/preprocessing layers `keras.layers.AutoContrast`, `keras.layers.Solarization`.
* Added `keras.layers.Pipeline` class, to apply a sequence of layers to an input. This class is useful to build a preprocessing pipeline. Compared to a `Sequential` model, `Pipeline` features a few important differences:
 - It's not a `Model`, just a plain layer.
 - When the layers in the pipeline are compatible with `tf.data`, the pipeline will also remain `tf.data` compatible, independently of the backend you use.


New Contributors
* alexhartl made their first contribution in https://github.com/keras-team/keras/pull/20125
* Doch88 made their first contribution in https://github.com/keras-team/keras/pull/20156
* edbosne made their first contribution in https://github.com/keras-team/keras/pull/20151
* ghsanti made their first contribution in https://github.com/keras-team/keras/pull/20185
* joehiggi1758 made their first contribution in https://github.com/keras-team/keras/pull/20223
* AryazE made their first contribution in https://github.com/keras-team/keras/pull/20228
* sanskarmodi8 made their first contribution in https://github.com/keras-team/keras/pull/20237
* himalayo made their first contribution in https://github.com/keras-team/keras/pull/20262
* nate2s made their first contribution in https://github.com/keras-team/keras/pull/20305
* DavidLandup0 made their first contribution in https://github.com/keras-team/keras/pull/20316

**Full Changelog**: https://github.com/keras-team/keras/compare/v3.5.0...v3.6.0

3.5.0

What's Changed

* Add integration with the Hugging Face Hub. You can now save models to Hugging Face Hub directly from `keras.Model.save()` and load `.keras` models directly from Hugging Face Hub with `keras.saving.load_model()`.
* Ensure compatibility with NumPy 2.0.
* Add `keras.optimizers.Lamb` optimizer.
* Improve `keras.distribution` API support for very large models.
* Add `keras.ops.associative_scan` op.
* Add `keras.ops.searchsorted` op.
* Add `keras.utils.PyDataset.on_epoch_begin()` method.
* Add `data_format` argument to `keras.layers.ZeroPadding1D` layer. 
* Bug fixes and performance improvements.


**Full Changelog**: https://github.com/keras-team/keras/compare/v3.4.1...v3.5.0
Links

@pyup-bot pyup-bot mentioned this pull request Nov 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant