beyondml.tflow

TensorFlow compatibility for building MANN models.

The beyondml.tflow package contains two subpackages, beyondml.tflow.layers and beyondml.tflow.utils, which contain the functionality to create and train MANN layers within TensorFlow. For individuals who are familiar with the former name of this package, mann, backwards compatibility can be achieved (assuming only TensorFlow support is needed), by replacing the following line of code:

>>> import mann

with the following line:

>>> import beyondml.tflow as mann

in all existing scripts.

Within the layers package, there is current functionality for the the following layers:

Note that with any of the sparse layers (such as the SparseDense layer), any model which utilizes these layers will not be loadable using the traditional load_model functions available in TensorFlow. Instead, the model should be saved using either joblib or pickle.

Within the utils package, there are the current functions and classes:

  • ActiveSparsification
  • build_transformer_block
  • build_token_position_embedding_block
  • get_custom_objects
  • mask_model
  • remove_layer_masks
  • add_layer_masks
  • quantize_model
  • get_task_masking_gradients
  • mask_task_weights
  • train_model_iteratively
 1"""
 2## TensorFlow compatibility for building MANN models.
 3
 4The `beyondml.tflow` package contains two subpackages, `beyondml.tflow.layers` and `beyondml.tflow.utils`, which contain
 5the functionality to create and train MANN layers within TensorFlow. For individuals who are
 6familiar with the former name of this package, `mann`, backwards compatibility can be achieved
 7(assuming only TensorFlow support is needed), by replacing the following line of code:
 8
 9>>> import mann
10
11with the following line:
12
13>>> import beyondml.tflow as mann
14
15in all existing scripts.
16
17Within the `layers` package, there is current functionality for the the following layers:
18- `beyondml.tflow.layers.FilterLayer`
19- `beyondml.tflow.layers.MaskedConv2D`
20- `beyondml.tflow.layers.MaskedDense`
21- `beyondml.tflow.layers.MultiConv2D`
22- `beyondml.tflow.layers.MultiDense`
23- `beyondml.tflow.layers.MultiMaskedConv2D`
24- `beyondml.tflow.layers.MultiMaskedDense`
25- `beyondml.tflow.layers.MultiMaxPool2D`
26- `beyondml.tflow.layers.SelectorLayer`
27- `beyondml.tflow.layers.SumLayer`
28- `beyondml.tflow.layers.SparseDense`
29- `beyondml.tflow.layers.SparseConv`
30- `beyondml.tflow.layers.SparseMultiDense`
31- `beyondml.tflow.layers.SparseMultiConv`
32
33**Note that with any of the sparse layers (such as the `SparseDense` layer), any model which
34utilizes these layers will not be loadable using the traditional `load_model` functions available
35in TensorFlow. Instead, the model should be saved using either joblib or pickle.**
36
37Within the `utils` package, there are the current functions and classes:
38- `ActiveSparsification`
39- `build_transformer_block`
40- `build_token_position_embedding_block`
41- `get_custom_objects`
42- `mask_model`
43- `remove_layer_masks`
44- `add_layer_masks`
45- `quantize_model`
46- `get_task_masking_gradients`
47- `mask_task_weights`
48- `train_model_iteratively`
49"""
50
51import beyondml.tflow.layers
52import beyondml.tflow.utils