constraints.md

Layer weight constraints

Usage of constraints

Classes from the keras.constraints module allow setting constraints (eg. non-negativity) on model parameters during training. They are per-variable projection functions applied to the target variable after each gradient update (when using fit()).

The exact API will depend on the layer, but the layers Dense, Conv1D, Conv2D and Conv3D have a unified API.

These layers expose two keyword arguments:

• kernel_constraint for the main weights matrix

• bias_constraint for the bias.

from keras.constraints import max_norm
model.add(Dense(64, kernel_constraint=max_norm(2.)))

---

Available weight constraints

{{autogenerated}}

Creating custom weight constraints

A weight constraint can be any callable that takes a tensor and returns a tensor with the same shape and dtype. You would typically implement your constraints as subclasses of keras.constraints.Constraint.

Here's a simple example: a constraint that forces weight tensors to be centered around a specific value on average.

from keras import ops

class CenterAround(keras.constraints.Constraint):
  """Constrains weight tensors to be centered around `ref_value`."""

  def __init__(self, ref_value):
    self.ref_value = ref_value

  def __call__(self, w):
    mean = ops.mean(w)
    return w - mean + self.ref_value

  def get_config(self):
    return {'ref_value': self.ref_value}

Optionally, you an also implement the method get_config and the class method from_config in order to support serialization -- just like with any Keras object. Note that we don't have to implement from_config in the example above since the constructor arguments of the class the keys in the config returned by get_config are the same. In this case, the default from_config works fine.