PyTorch#

RadioMapDataset in openworld_radio_twin.torch_dataset reads the radio maps of a generated dataset, or of a package whose scene archives are extracted into the package directory (Packaging for release), as a PyTorch dataset. It needs the torch extra:

python -m pip install -e '.[torch]'

One sample is one whole radio map with all its transmitters or, with per_transmitter=True, one transmitter of a radio map, as a dictionary of tensors on the grid of its radio arrays. Rows run south to north and columns west to east, as in the stored arrays (Dataset layout). Values keep their physical units; nothing is normalised.

Samples#

With T transmitters and H by W cells:

Key

Type and shape

Contents

geometry

float32 (3, H, W)

Roof height above ground in metres (0 where there is no building), building footprint mask, and ground elevation in the scene frame in metres

surface_classes

int64 (H, W)

The surface class ID (Surfaces and materials) of the terrain cell that holds the cell centre; -1 when the scene has no surface class grid

indoor

bool (H, W)

The receiver point lies inside a building

tx_map

float32 (T, H, W)

1 in the cell of each transmitter; all 0 for a transmitter off the grid

tx_cell

int64 (T, 2)

Row and column of that cell, which can lie outside the grid

tx_position_m

float32 (T, 3)

East, north and up in the scene frame

tx_height_agl_m, tx_power_dbm, tx_azimuth_deg, tx_downtilt_deg

float32 (T,)

Height above ground, transmit power, azimuth and downtilt

frequency_hz

float64 scalar

Carrier frequency

path_gain_db, rss_dbm, sinr_db

float32 (T, H, W)

Path gain, received power and SINR per transmitter; 0 where coverage is false

coverage

bool (T, H, W)

The transmitter’s path gain, RSS and SINR are all positive

serving_tx

int64 (H, W)

The serving transmitter, -1 where none serves

radio_map_id

str

<scene>/<name>, as in the index

The radio values are those of load_radio_map (Dataset layout), so they carry its quantisation. The geometry comes from the building rasters and the measurement surface of the radio map: the ground elevation is the height of the measurement cell centre minus the receiver height. The surface classes come from the scene’s terrain/ arrays, whose cells can be larger than the radio cells. Each dataset instance keeps the scene inputs of the last two measurement surfaces it read. The index is in scene order, so reading it in order loads each surface once when a scene’s radio maps use at most two surfaces, for example one resolution; a shuffled loader over many scenes loads them for most samples.

One transmitter per sample#

The path gain and RSS of a transmitter do not depend on the other transmitters of its radio map; SINR and the serving transmitter do. With per_transmitter=True, one sample is one transmitter of a radio map: the radio maps in index order, and the transmitters of each in record order. The dataset length is the sum of n_tx over the selected radio maps.

A sample keeps the transmitter axis with size 1. tx_map, path_gain_db, rss_dbm and coverage have shape (1, H, W), tx_cell (1, 2), tx_position_m (1, 3), and the transmitter parameters (1,). The sample has no sinr_db and no serving_tx, and its coverage is where path gain and RSS are positive. It adds transmitter (int64 scalar), the index of the transmitter in its radio map. A sample reads the stored path gain of every transmitter of its radio map and unpacks only that transmitter’s. The scene inputs are shared and cached as for whole radio maps.

Use one transmitter per sample for path gain or RSS models: every sample has one transmitter, so samples of one grid size batch without padding (Batching). Use whole radio maps for SINR or serving-transmitter targets, which need all transmitters of a radio map.

Selecting radio maps#

The dataset reads index.parquet with read_index, so it refuses a stale index with the same instruction (Dataset layout). radio_map_ids keeps the listed radio maps, and an id that is not in the index raises KeyError. split then keeps those of one split of a package. A split that the package’s splits.json does not declare raises ValueError with the declared splits; a declared split with no radio maps gives an empty dataset. A generated dataset has no splits, so any split raises ValueError there. Both keep the index order. A package whose scene archives are not extracted raises FileNotFoundError with the tar command that extracts them. index is the pyarrow.Table of the index rows of the selected radio maps, one per radio map in index order in both modes, so it can be queried to select radio maps:

import pyarrow.compute as pc
from openworld_radio_twin.torch_dataset import RadioMapDataset

train = RadioMapDataset("releases/my-radio-dataset", split="train")
two_tx = train.index.filter(pc.field("n_tx") == 2).column("radio_map_id").to_pylist()

Batching#

Radio maps of different radii or resolutions can differ in size. RandomCrop(size) crops every grid entry of a sample to one random window of size cells (an int for a square, or (rows, columns)), drawn from PyTorch’s global random number generator, which the DataLoader seeds for each worker. It adds crop_origin, the row and column of the window’s first cell in the whole radio map, and shifts tx_cell to the window. The cropped entries are copies of the window, so a crop does not keep the whole radio map in memory. Every transmitter is kept, and one outside the window has an all-zero tx_map. A size that is not one or two positive integers, and a radio map smaller than the window, raise ValueError.

With one transmitter per sample, samples of one grid size stack into batches with the default collate function, and so do crops of one size. A minimal training loop over crops of the training split:

import torch
from torch.utils.data import DataLoader
from openworld_radio_twin.torch_dataset import RadioMapDataset, RandomCrop

crops = RadioMapDataset(
    "releases/my-radio-dataset", split="train", per_transmitter=True, transform=RandomCrop(256)
)
loader = DataLoader(crops, batch_size=8, shuffle=True, num_workers=4)

model = torch.nn.Conv2d(3 + 1, 1, kernel_size=3, padding=1)   # geometry and tx_map in
optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)
for batch in loader:
    inputs = torch.cat([batch["geometry"], batch["tx_map"]], dim=1)
    prediction = model(inputs)                                  # path gain of the transmitter
    mask = batch["coverage"]
    if not mask.any():                                          # no covered cell to fit
        continue
    loss = torch.nn.functional.mse_loss(prediction[mask], batch["path_gain_db"][mask])
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

Whole radio maps stack only when their sizes and transmitter counts agree. Load them with batch_size=1, crop radio maps with one transmitter count, such as two_tx above, or write a collate_fn of your own that pads the transmitter axis and returns a transmitter mask.

The Radio maps as a PyTorch dataset tutorial runs these steps on a sample package.