Physical data structures

Named physical quantities are accessed through data.physical_data for a sample or batch.physical_data for a batch. Atomistic quantities use tensors directly. Electronic quantities use dictionaries that retain orbital, block, or grid boundaries alongside their values.

The layouts below describe one sample; Samples and batches explains how the variable-size axes are concatenated. extra_shape denotes fixed additional value axes. Quantity descriptions belong to the dataset and identify the physical type and conventions independently of the runtime dictionary structure.

Atomistic values

For selected atomistic data, data.physical_data[name] is its values tensor. The owning Dataset exposes physical_data_descriptions[name] with its data_type, precision, and extra_shape.

data_type="scalar"                   values float32 [1, *extra_shape]
data_type="atom_scalar"              values float32 [n_atoms, *extra_shape]
data_type="cartesian_vector"         values float32 [1, *extra_shape, 3]
data_type="atom_cartesian_vector"    values float32 [n_atoms, *extra_shape, 3]
data_type="cartesian_tensor_2"       values float32 [1, *extra_shape, 3, 3]
data_type="atom_cartesian_tensor_2"  values float32 [n_atoms, *extra_shape, 3, 3]

The length-one leading axes mark structure-level values so PyG concatenates them along the sample axis. AtomScalar, AtomCartesianVector, and AtomCartesianTensor2 instead use their atom axis as the leading runtime axis and are concatenated in the same order as pos. Their physical arrays place n_atoms immediately before any Cartesian axes; conversion moves that atom axis forward for batching.

Electronic values

For selected electronic data, data.physical_data[name] is an ordinary Python dictionary. Its four possible dictionary structures are published as OrbData, BlockSparseOrbData, UniformVolumetricData, and QuadratureVolumetricData; PhysicalDataDict is the named outer mapping shared with atomistic tensors. The dense OrbMatrix hierarchy and OrbVector share OrbData because their PyG fields are identical. For example, select the Hamiltonian data with:

hamiltonian_data = data.physical_data["hamiltonian"]

The data dictionary always contains:

hamiltonian_data["num_values"]          int64   [1]
hamiltonian_data["values_real"]         float32 [n_values, *extra_shape]
hamiltonian_data["values_imag"]         float32 [n_values, *extra_shape]  # complex data only

Thus, for example, the Hamiltonian's real values are read as data.physical_data["hamiltonian"]["values_real"]; fields inside an entry use dictionary keys rather than attribute access.

The entry's data_type, basis_role, pauli, complexity, and fixed extra_shape describe the whole Dataset and are available from dataset.physical_data_descriptions[name] rather than repeated in every sample.

A typical access path therefore looks like:

data = dataset[0]
positions = data.pos

hamiltonian = data.physical_data["hamiltonian"]
values_real = hamiltonian["values_real"]
atom_pair_index = hamiltonian["atom_pair_index"]
description = dataset.physical_data_descriptions["hamiltonian"]

Consumers such as losses can accept one typed electronic-data entry without depending on the complete Data:

from torch import Tensor
from elfes.data import OrbData

def orbital_loss(prediction: OrbData, target: OrbData) -> Tensor:
    ...

Because physical-data names are dataset-defined, callers use the corresponding physical_data_descriptions[name].data_type when choosing or statically narrowing one entry to a specific dictionary type. PhysicalDataDescription is re-exported from elfes.io.hdf5; its definition is documented in the data-type reference below.

The leading values axis is the variable-size axis concatenated by PyG. A complex entry uses parallel real tensors; ELFES does not place complex tensors in Data. When Pauli components are present, their component axis is the final axis of extra_shape.

BlockSparseOrbData similarly names a runtime structure rather than a coordinate system. Dataset entries contain physical orbital-matrix components, while model-side operations may reuse the same pair blocks and boundaries for coupled components.

Block-sparse orbital matrices

data_type="block_sparse_orb_matrix" and data_type="herm_block_sparse_orb_matrix" add:

block_data["num_blocks"]               int64 [1]
block_data["atom_pair_index"]          int64 [2, n_blocks]
block_data["pair_shifts"]              int64 [n_blocks, 3]
block_data["block_lengths"]            int64 [n_blocks]

block_data["atom_pair_index"][:, b] contains sample-local row and column atom indices, while block_data["pair_shifts"][b] identifies the cell image of the column atom. General matrices contain actual directed blocks; Hermitian matrices contain one independent block from each partner pair. The C-order-flattened spatial values of consecutive blocks are concatenated in block_data["values_real"] and, when present, block_data["values_imag"]; block_data["block_lengths"] retains their boundaries. Both use the same PyG dictionary fields, while dataset.physical_data_descriptions[name].data_type retains the mathematical distinction.

Dense orbital matrices

All dense orbital-matrix data types use only the common value fields. For an orbital matrix with n_orbitals orbitals:

• data_type="orb_matrix" uses n_orbitals**2 values in full-matrix C order.
• data_type="herm_orb_matrix" uses n_orbitals * (n_orbitals + 1) // 2 values from the upper triangle, including the diagonal, in NumPy triu_indices row-major order; the omitted lower triangle is its conjugate transpose.
• data_type="triu_orb_matrix" uses the same packed upper order and value count; the omitted strict lower triangle is zero.

Orbital vectors

data_type="orb_vector" also uses only the common value fields. Its orbital axis becomes the leading values axis, so n_values is the total orbital count for the entry's basis role.

Uniform volumetric data

data_type="uniform_volumetric" is real and adds:

uniform_data["origin"]                 float32 [1, 3]
uniform_data["step_vectors"]           float32 [1, 3, 3]
uniform_data["shape"]                  int64   [1, 3]

The three grid axes are flattened in C order into n_values = grid_shape[0] * grid_shape[1] * grid_shape[2] value rows. The stored shape reconstructs those axes; grid periodicity is the sample's pbc.

Quadrature volumetric data

data_type="quadrature_volumetric" is real and adds:

quadrature_data["coordinates"]         float32 [n_values, 3]
quadrature_data["weights"]             float32 [n_values]

Each value row is aligned with one explicit Cartesian quadrature point and weight.

Data types

PhysicalDataDescription describes the dataset-level physical type and storage precision. The remaining types describe ordinary runtime dictionaries and their fields. PhysicalDataDict is the named outer mapping shared by atomistic tensors and electronic dictionaries.

PhysicalDataDescription = AtomisticDataDescription | ElectronicDataDescription

PhysicalDataDict = dict[
    str,
    Tensor
    | OrbData
    | BlockSparseOrbData
    | UniformVolumetricData
    | QuadratureVolumetricData,
]