Skip to content

Normalization

SO(3) normalization rescales complete, l-major spherical features without changing their rotation rules. SO3RMSNorm combines a shared RMS scale for each sample with optional centering and learnable affine parameters that preserve equivariance.

SO(2) normalization acts on local, m-major features with optional order truncation. SO2RMSNorm(l_max, channels, center=False) averages the squares of all retained real components and channels, then applies a learned gain per channel shared across angular components. The first l_max + 1 components must contain the complete zero-mode block. No statistics are shared across edges or other leading dimensions.

With center=True, SO2RMSNorm subtracts one mean over all zero-mode (l, c) entries, leaving nonzero modes unchanged before calculating the common RMS. SO3RMSNorm instead centers only the l=0 channels and defaults to center=True. These are the respective rotation-invariant subspaces; centering every angular component as in ordinary LayerNorm would generally break equivariance. SO2RMSNorm has no additive bias, including when centering is enabled; SO3RMSNorm includes a learned scalar bias when centering and learnable parameters are enabled.

SO2RMSNorm

SO2RMSNorm(
    l_max: int,
    channels: int,
    *,
    eps: float = 1e-05,
    center: bool = False,
    device: device | str | None = None,
    dtype: dtype | None = None,
)

Bases: Module

Normalize an m-major SO(2) tensor with one RMS and channel gains.

Inputs have shape (..., K, channels), where the first l_max + 1 components are the complete zero-mode block. All retained real components and channels contribute equally to one RMS per input tensor. Real and imaginary partners share the gain weight[c], preserving SO(2) equivariance.

With center=True, subtract one common mean over all (l, c) entries of the zero-mode block before computing the RMS. Nonzero modes are not centered. This differs from ordinary LayerNorm, which centers every entry in its normalization axes. No additive bias is used in either mode.

Parameters:

  • l_max (int) –

    Highest degree; determines the zero-mode block size.

  • channels (int) –

    Channels per retained real component.

  • eps (float, default: 1e-05 ) –

    Positive value added to the mean square.

  • center (bool, default: False ) –

    Center all zero-mode entries jointly; defaults to false.

  • device (device | str | None, default: None ) –

    Initial device of the channel gains.

  • dtype (dtype | None, default: None ) –

    Initial floating dtype of the channel gains.

forward

forward(m_tensor: Tensor) -> Tensor

Normalize [..., K, C] features without mixing leading dimensions.

SO3RMSNorm

SO3RMSNorm(
    l_max: int,
    channels: int,
    *,
    eps: float = 1e-05,
    learnable: bool = True,
    center: bool = True,
    weighting: Literal["l_balanced", "component"] = "l_balanced",
    device: device | str | None = None,
    dtype: dtype | None = None,
)

Bases: Module

Normalize complete SO(3) tensors with one RMS per sample.

Inputs contain one complete real SO(3) irrep for every l = 0, ..., l_max and use the l-major shape (..., (l_max + 1) ** 2, channels). When center is true, the scalar channels are first centered as

\[ \mu=\frac{1}{C}\sum_c x_{00c}, \qquad \widetilde{x}_{lmc}=x_{lmc}-\delta_{l0}\mu. \]

The default l_balanced weighting gives every l block equal weight:

\[ q=\frac{1}{C}\sum_c\frac{1}{l_{\max}+1} \sum_{l=0}^{l_{\max}}\frac{1}{2l+1} \sum_{m=-l}^{l}\widetilde{x}_{lmc}^2. \]

component weighting instead averages all (l, m, c) entries uniformly. The normalized tensor shares the sample-dependent scale \((q+\varepsilon)^{-1/2}\). When learnable is true, the result is multiplied by a learned weight[l, c] shared across m; centered scalar channels also receive a learned bias[c]:

\[ y_{lmc}=\gamma_{lc}\frac{\widetilde{x}_{lmc}} {\sqrt{q+\varepsilon}}+\delta_{l0}\beta_c. \]

Sharing parameters across m and restricting the bias to l = 0 makes the operation commute with rotations. Contiguous CUDA float16, bfloat16, and float32 inputs use fused Triton forward and first-order backward kernels; higher-order gradients and other inputs use differentiable PyTorch expressions.

Parameters:

  • l_max (int) –

    Highest included non-negative l.

  • channels (int) –

    Number of channels shared by every spherical component.

  • eps (float, default: 1e-05 ) –

    Positive value added before the reciprocal square root.

  • learnable (bool, default: True ) –

    Learn the equivariant scale and, when centering, scalar bias.

  • center (bool, default: True ) –

    Subtract the mean of the l = 0 channels for every sample.

  • weighting (Literal['l_balanced', 'component'], default: 'l_balanced' ) –

    Give each l block or each spherical component equal weight.

  • device (device | str | None, default: None ) –

    Initial device of parameters and buffers.

  • dtype (dtype | None, default: None ) –

    Initial floating dtype of parameters and weights.

Attributes:

  • weight (Tensor | None) –

    Learned scale with shape (l_max + 1, channels), or None.

  • bias (Tensor | None) –

    Learned scalar bias with shape (channels,), or None.

forward

forward(l_tensor: Tensor) -> Tensor

Parameters:

  • l_tensor (Tensor) –

    [..., (l_max+1)**2, C].

Returns:

  • normalized_l_tensor ( Tensor ) –

    [..., (l_max+1)**2, C].