PyTorch#
RadioMapDataset in openworld_radio_twin.torch_dataset reads the radio maps of a
generated dataset, or of a package whose scene archives are extracted into the package
directory (Packaging for release), as a PyTorch dataset. It needs the torch extra:
python -m pip install -e '.[torch]'
One sample is one whole radio map with all its transmitters or, with
per_transmitter=True, one transmitter of a radio map, as a dictionary of tensors on the grid
of its radio arrays. Rows run south to north and columns west to east, as in the stored
arrays (Dataset layout). Values keep their physical units; nothing is normalised.
Samples#
With T transmitters and H by W cells:
Key |
Type and shape |
Contents |
|---|---|---|
|
|
Roof height above ground in metres (0 where there is no building), building footprint mask, and ground elevation in the scene frame in metres |
|
|
The surface class ID (Surfaces and materials) of the terrain cell that holds the cell centre; -1 when the scene has no surface class grid |
|
|
The receiver point lies inside a building |
|
|
1 in the cell of each transmitter; all 0 for a transmitter off the grid |
|
|
Row and column of that cell, which can lie outside the grid |
|
|
East, north and up in the scene frame |
|
|
Height above ground, transmit power, azimuth and downtilt |
|
|
Carrier frequency |
|
|
Path gain, received power and SINR per transmitter; 0 where |
|
|
The transmitter’s path gain, RSS and SINR are all positive |
|
|
The serving transmitter, -1 where none serves |
|
|
|
The radio values are those of load_radio_map (Dataset layout), so they carry its
quantisation. The geometry comes from the building rasters and the measurement surface of the
radio map: the ground elevation is the height of the measurement cell centre minus the
receiver height. The surface classes come from the scene’s terrain/ arrays, whose cells
can be larger than the radio cells. Each dataset instance keeps the scene inputs
of the last two measurement surfaces it read. The index is in scene order, so reading it in
order loads each surface once when a scene’s radio maps use at most two surfaces, for
example one resolution; a shuffled loader over many scenes loads them for most samples.
One transmitter per sample#
The path gain and RSS of a transmitter do not depend on the other transmitters of its radio
map; SINR and the serving transmitter do. With per_transmitter=True, one sample is one
transmitter of a radio map: the radio maps in index order, and the transmitters of each in
record order. The dataset length is the sum of n_tx over the selected radio maps.
A sample keeps the transmitter axis with size 1. tx_map, path_gain_db, rss_dbm and
coverage have shape (1, H, W), tx_cell (1, 2), tx_position_m (1, 3), and the
transmitter parameters (1,). The sample has no sinr_db and no serving_tx, and its
coverage is where path gain and RSS are positive. It adds transmitter (int64 scalar),
the index of the transmitter in its radio map. A sample reads the stored path gain of every
transmitter of its radio map and unpacks only that transmitter’s. The scene inputs are shared
and cached as for whole radio maps.
Use one transmitter per sample for path gain or RSS models: every sample has one transmitter, so samples of one grid size batch without padding (Batching). Use whole radio maps for SINR or serving-transmitter targets, which need all transmitters of a radio map.
Selecting radio maps#
The dataset reads index.parquet with read_index, so it refuses a stale index with the same
instruction (Dataset layout). radio_map_ids keeps the listed radio maps, and an id that is
not in the index raises KeyError. split then keeps those of one split of a package. A
split that the package’s splits.json does not declare raises ValueError with the declared
splits; a declared split with no radio maps gives an empty dataset. A generated dataset has no
splits, so any split raises ValueError there. Both keep the index order. A package whose
scene archives are not extracted raises FileNotFoundError with the tar command that
extracts them. index is the pyarrow.Table of the index rows of the selected radio maps,
one per radio map in index order in both modes, so it can be queried to select radio maps:
import pyarrow.compute as pc
from openworld_radio_twin.torch_dataset import RadioMapDataset
train = RadioMapDataset("releases/my-radio-dataset", split="train")
two_tx = train.index.filter(pc.field("n_tx") == 2).column("radio_map_id").to_pylist()
Batching#
Radio maps of different radii or resolutions can differ in size. RandomCrop(size) crops every
grid entry of a sample to one random window of size cells (an int for a square, or
(rows, columns)), drawn from PyTorch’s global random number generator, which the
DataLoader seeds for each worker. It adds crop_origin, the row and column of the window’s
first cell in the whole radio map, and shifts tx_cell to the window. The cropped entries are
copies of the window, so a crop does not keep the whole radio map in memory. Every
transmitter is kept, and one outside the window has an all-zero tx_map. A size that is not
one or two positive integers, and a radio map smaller than the window, raise ValueError.
With one transmitter per sample, samples of one grid size stack into batches with the default collate function, and so do crops of one size. A minimal training loop over crops of the training split:
import torch
from torch.utils.data import DataLoader
from openworld_radio_twin.torch_dataset import RadioMapDataset, RandomCrop
crops = RadioMapDataset(
"releases/my-radio-dataset", split="train", per_transmitter=True, transform=RandomCrop(256)
)
loader = DataLoader(crops, batch_size=8, shuffle=True, num_workers=4)
model = torch.nn.Conv2d(3 + 1, 1, kernel_size=3, padding=1) # geometry and tx_map in
optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)
for batch in loader:
inputs = torch.cat([batch["geometry"], batch["tx_map"]], dim=1)
prediction = model(inputs) # path gain of the transmitter
mask = batch["coverage"]
if not mask.any(): # no covered cell to fit
continue
loss = torch.nn.functional.mse_loss(prediction[mask], batch["path_gain_db"][mask])
optimizer.zero_grad()
loss.backward()
optimizer.step()
Whole radio maps stack only when their sizes and transmitter counts agree. Load them with
batch_size=1, crop radio maps with one transmitter count, such as two_tx above, or write a
collate_fn of your own that pads the transmitter axis and returns a transmitter mask.
The Radio maps as a PyTorch dataset tutorial runs these steps on a sample package.