Category: 17. KerasNLP

Tokenizers convert raw string input into integer input suitable for a Keras Embedding layer. They can also convert back from predicted integer sequences to raw string output. All tokenizers subclass keras_nlp.tokenizers.Tokenizer, which in turn subclasses keras.layers.Layer. Tokenizers should generally be applied inside a tf.data.Dataset.map for training, and can be included inside a keras.Model for inference.

KerasNLP contains end-to-end implementations of popular model architectures. These models can be created in two ways: Below, we list all presets available in the library. For more detailed usage, browse the docstring for a particular class. For an in depth introduction to our API, see the getting started guide. Backbone presets The following preset names correspond…