Skip to content

Dataset statistics

Dataset statistics describe a selected collection of samples: how many neighbors its atoms have, or the means and scales of a target quantity. elfes.data.stats lets several accumulators share one pass over the same batches, while each produces an independent result.

Statistics are computed separately from loading and normalization. Connectivity statistics require connected batches; orbital statistics require the chosen physical-data entry and its basis definitions. Results provide tensor-only state dictionaries for saving and can be used to configure orbital affine transforms.

Accumulation workflow

Each accumulator receives batches through update() and produces its result through finalize(). accumulate_stats() calls update() on every supplied accumulator for each batch; finalization remains explicit. For example, given a loader that returns connected batches:

from elfes.data.stats import ConnectivityStatsAccumulator, accumulate_stats

neighbors = ConnectivityStatsAccumulator()
accumulate_stats(loader, neighbors)
result = neighbors.finalize()
avg_num_neighbors = result.avg_num_neighbors

Additional accumulators can be passed to the same accumulate_stats() call to share the scan. Use the intended training subset when computing statistics for training normalization.

StatsAccumulator

Bases: Protocol

Streaming computation that finalizes into one statistics result.

update

update(batch: Batch) -> None

Accumulate sufficient state from one batch.

finalize

finalize() -> ResultT

Construct the result from all accumulated batches.

accumulate_stats

accumulate_stats(
    batches: Iterable[Batch], *accumulators: StatsAccumulator[object]
) -> None

Update multiple statistics accumulators in one pass over batches.

Connectivity statistics

The mean directed neighbor count is the total edge count divided by the total atom count, giving each atom equal weight.

ConnectivityStatsAccumulator

ConnectivityStatsAccumulator()

Accumulate atom-weighted statistics of directed model connectivity.

update

update(batch: Batch) -> None

Accumulate connectivity counts from one connected batch.

finalize

finalize() -> ConnectivityStats

Return the accumulated connectivity statistics.

ConnectivityStats dataclass

ConnectivityStats(total_edges: int, total_atoms: int)

Atom-weighted statistics of directed model connectivity.

avg_num_neighbors property

avg_num_neighbors: float

Return the atom-weighted mean directed neighbor count.

state_dict

state_dict() -> dict[str, Tensor]

Return a tensor-only state suitable for torch.save.

from_state_dict classmethod

from_state_dict(state: Mapping[str, Tensor]) -> ConnectivityStats

Reconstruct statistics from state_dict().

Orbital-vector statistics

These statistics operate on real orbital-vector values. Each atom-shell occurrence has equal weight. Scalar shells use their mean and centered population standard deviation; higher-degree shells keep zero mean and use component RMS so all components of a spherical shell share the same scale.

OrbVectorStatsAccumulator

OrbVectorStatsAccumulator(basis_set: BasisSet, *, target_name: str, basis_role: str)

Accumulate equivariant normalization moments for an orbital vector.

Scalar shells use their centered population standard deviation. Higher angular momenta keep zero mean and use component RMS. Every atom-shell occurrence has equal weight, and a path with zero variance receives unit scale.

update

update(batch: Batch) -> None

Accumulate moments from one batch containing the configured target.

finalize

finalize() -> OrbVectorStats

Return the accumulated orbital-vector statistics.

OrbVectorStats dataclass

OrbVectorStats(shell_means: tuple[Tensor, ...], shell_scales: tuple[Tensor, ...])

Equivariant shell moments of an orbital vector.

state_dict

state_dict() -> dict[str, Tensor]

Return a tensor-only state suitable for torch.save.

from_state_dict classmethod

from_state_dict(state: Mapping[str, Tensor]) -> OrbVectorStats

Reconstruct statistics as views into the stored tensors.

Block-sparse matrix statistics

These statistics operate on real Hermitian block-sparse values after conversion to coupled coordinates. Onsite and offsite moments are accumulated separately, with Hermitian reverse species and shell pairs sharing one path. Every explicitly stored shell-pair occurrence has equal weight; scalar paths use centered moments, while higher-degree paths use zero mean and component RMS.

BlockSparseStatsAccumulator

BlockSparseStatsAccumulator(
    basis_set: BasisSet,
    cg_transform: _BlockSparseCGTransform,
    *,
    target_name: str,
    basis_role: str,
)

Accumulate canonical coupled-path moments of block-sparse data.

Physical blocks are first changed to coupled coordinates by the configured CG transform. Onsite means and scales are independent from offsite means and scales. A block is onsite exactly when its two atom indices are equal and its cell shift is zero. Hermitian reverse species and shell pairs share one path. Scalar paths use their centered population standard deviation; all higher-\(L\) paths keep zero mean and use component RMS. Every explicitly stored shell-pair occurrence has equal weight, and a path with zero variance receives unit scale.

update

update(batch: Batch) -> None

Accumulate moments from one batch containing the configured target.

finalize

finalize() -> BlockSparseStats

Return the accumulated block-sparse statistics.

BlockSparseStats dataclass

BlockSparseStats(
    onsite_means: Tensor,
    onsite_scales: Tensor,
    offsite_means: Tensor,
    offsite_scales: Tensor,
)

Canonical coupled-path moments of Hermitian block-sparse data.

state_dict

state_dict() -> dict[str, Tensor]

Return a tensor-only state suitable for torch.save.

from_state_dict classmethod

from_state_dict(state: Mapping[str, Tensor]) -> BlockSparseStats

Reconstruct statistics using the stored tensors.