Normalization
SO(3) normalization rescales complete, l-major spherical features without changing their rotation rules. SO3RMSNorm combines a shared RMS scale for each sample with optional centering and learnable affine parameters that preserve equivariance.
SO(2) normalization acts on local, m-major features with optional order truncation. SO2RMSNorm(l_max, channels, center=False) averages the squares of all retained real components and channels, then applies a learned gain per channel shared across angular components. The first l_max + 1 components must contain the complete zero-mode block. No statistics are shared across edges or other leading dimensions.
With center=True, SO2RMSNorm subtracts one mean over all zero-mode (l, c) entries, leaving nonzero modes unchanged before calculating the common RMS. SO3RMSNorm instead centers only the l=0 channels and defaults to center=True. These are the respective rotation-invariant subspaces; centering every angular component as in ordinary LayerNorm would generally break equivariance. SO2RMSNorm has no additive bias, including when centering is enabled; SO3RMSNorm includes a learned scalar bias when centering and learnable parameters are enabled.
SO2RMSNorm
SO2RMSNorm(
l_max: int,
channels: int,
*,
eps: float = 1e-05,
center: bool = False,
device: device | str | None = None,
dtype: dtype | None = None,
)
Bases: Module
Normalize an m-major SO(2) tensor with one RMS and channel gains.
Inputs have shape (..., K, channels), where the first l_max + 1
components are the complete zero-mode block. All retained real components
and channels contribute equally to one RMS per input tensor. Real and
imaginary partners share the gain weight[c], preserving SO(2) equivariance.
With center=True, subtract one common mean over all (l, c) entries of
the zero-mode block before computing the RMS. Nonzero modes are not
centered. This differs from ordinary LayerNorm, which centers every entry
in its normalization axes. No additive bias is used in either mode.
Parameters:
-
l_max(int) –Highest degree; determines the zero-mode block size.
-
channels(int) –Channels per retained real component.
-
eps(float, default:1e-05) –Positive value added to the mean square.
-
center(bool, default:False) –Center all zero-mode entries jointly; defaults to false.
-
device(device | str | None, default:None) –Initial device of the channel gains.
-
dtype(dtype | None, default:None) –Initial floating dtype of the channel gains.
forward
forward(m_tensor: Tensor) -> Tensor
Normalize [..., K, C] features without mixing leading dimensions.
SO3RMSNorm
SO3RMSNorm(
l_max: int,
channels: int,
*,
eps: float = 1e-05,
learnable: bool = True,
center: bool = True,
weighting: Literal["l_balanced", "component"] = "l_balanced",
device: device | str | None = None,
dtype: dtype | None = None,
)
Bases: Module
Normalize complete SO(3) tensors with one RMS per sample.
Inputs contain one complete real SO(3) irrep for every l = 0, ..., l_max
and use the l-major shape (..., (l_max + 1) ** 2, channels). When
center is true, the scalar channels are first centered as
The default l_balanced weighting gives every l block equal weight:
component weighting instead averages all (l, m, c) entries uniformly.
The normalized tensor shares the sample-dependent scale
\((q+\varepsilon)^{-1/2}\). When learnable is true, the result is multiplied
by a learned weight[l, c] shared across m; centered scalar channels also
receive a learned bias[c]:
Sharing parameters across m and restricting the bias to l = 0 makes
the operation commute with rotations. Contiguous CUDA float16, bfloat16,
and float32 inputs use fused Triton forward and first-order backward
kernels; higher-order gradients and other inputs use differentiable
PyTorch expressions.
Parameters:
-
l_max(int) –Highest included non-negative
l. -
channels(int) –Number of channels shared by every spherical component.
-
eps(float, default:1e-05) –Positive value added before the reciprocal square root.
-
learnable(bool, default:True) –Learn the equivariant scale and, when centering, scalar bias.
-
center(bool, default:True) –Subtract the mean of the
l = 0channels for every sample. -
weighting(Literal['l_balanced', 'component'], default:'l_balanced') –Give each
lblock or each spherical component equal weight. -
device(device | str | None, default:None) –Initial device of parameters and buffers.
-
dtype(dtype | None, default:None) –Initial floating dtype of parameters and weights.
Attributes:
-
weight(Tensor | None) –Learned scale with shape
(l_max + 1, channels), orNone. -
bias(Tensor | None) –Learned scalar bias with shape
(channels,), orNone.
forward
forward(l_tensor: Tensor) -> Tensor
Parameters:
-
l_tensor(Tensor) –[..., (l_max+1)**2, C].
Returns:
-
normalized_l_tensor(Tensor) –[..., (l_max+1)**2, C].