Author: Awais Farooq
-
Python & NumPy utilities
set_random_seed function Sets all random seeds (Python, NumPy, and backend framework, e.g. TF). You can use this utility to make almost any Keras program fully deterministic. Some limitations apply in cases where network communications are involved (e.g. parameter server distribution), which creates additional sources of randomness, or when certain non-deterministic cuDNN ops are involved. Calling this…
-
Tensor utilities
get_source_inputs function Returns the list of input tensors necessary to compute tensor. Output will always be a list of tensors (potentially with 1 element). Arguments Returns List of input tensors. is_keras_tensor function Returns whether x is a Keras tensor. A “Keras tensor” is a symbolic tensor, such as a tensor that was created via Input(). A “symbolic tensor” can be understood as…
-
Structured data preprocessing utilities
FeatureSpace class One-stop utility for preprocessing and encoding structured data. Arguments Available feature types: Note that all features can be referred to by their string name, e.g. “integer_categorical”. When using the string name, the default argument values are used. Examples Basic usage with a dict of input data: Basic usage with tf.data: Basic usage with the Keras Functional…
-
Model plotting utilities
plot_model function Converts a Keras model to dot format and save to a file. Example Arguments Returns A Jupyter notebook Image object if Jupyter is installed. This enables in-line display of the model plots in notebooks. model_to_dot function Convert a Keras model to dot format. Arguments
-
Random operations
categorical function Draws samples from a categorical distribution. This function takes as input logits, a 2-D input tensor with shape (batch_size, num_classes). Each row of the input represents a categorical distribution, with each column index containing the log-probability for a given class. The function will output a 2-D tensor with shape (batch_size, num_samples), where each row contains…
-
SeedGenerator class
SeedGenerator class Generates variable seeds upon each call to a RNG-using function. In Keras, all RNG-using methods (such as keras.random.normal()) are stateless, meaning that if you pass an integer seed to them (such as seed=42), they will return the same values at each call. In order to get different values at each call, you must use a SeedGenerator instead as…
-
Distribution utilities
set_distribution function Set the distribution as the global distribution setting. Arguments distribution function Retrieve the current distribution from global context. list_devices function Return all the available devices based on the device type. Note: in a distributed setting, global devices are returned. Arguments Return: List of devices that are available for distribute computation. initialize function Initialize the distribution system for…
-
ModelParallel API
ModelParallel class Distribution that shards model variables. Compare to DataParallel which replicates the variables across all devices, ModelParallel allows you to shard variables in addition to the input data. To construct a ModelParallel distribution, you need to provide a DeviceMesh and a LayoutMap. Example You can quickly update the device mesh shape to change the sharding factor of the variables. E.g. To figure out a…
-
ModelParallel API
ModelParallel class Distribution that shards model variables. Compare to DataParallel which replicates the variables across all devices, ModelParallel allows you to shard variables in addition to the input data. To construct a ModelParallel distribution, you need to provide a DeviceMesh and a LayoutMap. Example You can quickly update the device mesh shape to change the sharding factor of the variables. E.g. To figure out a…
-
DataParallel API
DataParallel class Distribution for data parallelism. You can choose to create this instance by either specifying the device_mesh or devices arguments (but not both). The device_mesh argument is expected to be a DeviceMesh instance, and is expected to be 1D only. In case that the mesh has multiple axes, then the first axis will be treated as the data parallel dimension (and a warning…