Category: Developer Guide

  • Migrating Keras 2 code to multi-backend Keras 3

    Setup First, lets install keras-nightly. This example uses the TensorFlow backend (os.environ[“KERAS_BACKEND”] = “tensorflow”). After you’ve migrated your code, you can change the “tensorflow” string to “jax” or “torch” and click “Restart runtime” in Colab, and your code will run on the JAX or PyTorch backend. Going from Keras 2 to Keras 3 with the TensorFlow backend First, replace your imports: Next,…

  • Distributed training with Keras 3

    Introduction The Keras distribution API is a new interface designed to facilitate distributed deep learning across a variety of backends like JAX, TensorFlow and PyTorch. This powerful API introduces a suite of tools enabling data and model parallelism, allowing for efficient scaling of deep learning models on multiple accelerators and hosts. Whether leveraging the power…

  • Multi-GPU distributed training with PyTorch

    Introduction There are generally two ways to distribute computation across multiple devices: Data parallelism, where a single model gets replicated on multiple devices or multiple machines. Each of them processes different batches of data, then they merge their results. There exist many variants of this setup, that differ in how the different model replicas merge…

  • Multi-GPU distributed training with TensorFlow

    Introduction There are generally two ways to distribute computation across multiple devices: Data parallelism, where a single model gets replicated on multiple devices or multiple machines. Each of them processes different batches of data, then they merge their results. There exist many variants of this setup, that differ in how the different model replicas merge…

  • Multi-GPU distributed training with JAX

    Introduction There are generally two ways to distribute computation across multiple devices: Data parallelism, where a single model gets replicated on multiple devices or multiple machines. Each of them processes different batches of data, then they merge their results. There exist many variants of this setup, that differ in how the different model replicas merge…

  • Transfer learning & fine-tuning

    Setup Introduction Transfer learning consists of taking features learned on one problem, and leveraging them on a new, similar problem. For instance, features from a model that has learned to identify racoons may be useful to kick-start a model meant to identify tanukis. Transfer learning is usually done for tasks where your dataset has too little…

  • Writing your own callbacks

    Introduction A callback is a powerful tool to customize the behavior of a Keras model during training, evaluation, or inference. Examples include keras.callbacks.TensorBoard to visualize training progress and results with TensorBoard, or keras.callbacks.ModelCheckpoint to periodically save your model during training. In this guide, you will learn what a Keras callback is, what it can do, and how you can…

  • Customizing Saving and Serialization

    Introduction This guide covers advanced methods that can be customized in Keras saving. For most users, the methods outlined in the primary Serialize, save, and export guide are sufficient. APIs We will cover the following APIs: When restoring a model, these get executed in the following order: Setup State saving customization These methods determine how the state…

  • Save, serialize, and export models

    Introduction A Keras model consists of multiple components: The Keras API saves all of these pieces together in a unified format, marked by the .keras extension. This is a zip archive consisting of the following: Let’s take a look at how this works. How to save and load a model If you only have 10 seconds to…

  • Writing a training loop from scratch in PyTorch

    Setup Introduction Keras provides default training and evaluation loops, fit() and evaluate(). Their usage is covered in the guide Training & evaluation with the built-in methods. If you want to customize the learning algorithm of your model while still leveraging the convenience of fit() (for instance, to train a GAN using fit()), you can subclass the Model class and implement your own train_step() method, which is…