Resume and extend#
Dataset generation supports interruption recovery and deterministic extension. Generate 100
radio maps, raise the same scene’s radio_maps.count to 200 or add scenes after the
existing ones later, and resume the existing dataset:
result = generate_dataset_sync(request, output_root="data/generated", resume=True)
The batch page exposes the same behaviour through Resume / extend, and the HTTP API
through POST /api/batches?resume=true.
How resume decides what to run#
Resume reads each radio map’s record and arrays; the counts in the manifest can lag behind
the radio map directories (Dataset layout). A radio map is complete when its
radio_map_info.json has status completed and its path-gain array exists, with the PNG
previews when output.previews is set; failed, interrupted, corrupt or incomplete radio maps
are generated again. Because each radio map has an independent seed derived from the dataset
seed and its scene and radio map indices, existing radio maps keep their parameters when the
target count grows.
Compatibility checks#
Resume accepts only a dataset in the current dataset format and a request whose following parts are unchanged:
the dataset name and seed,
the definition of every existing scene, including its
site, its solver settings and parameter ranges, and the resolved building and antenna sources with their versions,the
output.previewsoption,the software version and, for Sionna radio maps, the radio-map contract version and the RF material profile version.
Previews are fixed for the lifetime of a dataset: a request that changes output.previews
is refused with a message that says so. Per-scene radio map counts may stay the same or
increase, and new scenes may follow the existing ones (below). A refused scene change names
the scene and the fields that changed. A refused request leaves the dataset as it was.
Persisted building, terrain and antenna inputs rebuild the runtime scene, so a new provider
release is never mixed into an existing dataset.
A dataset in another dataset format, and Sionna data with another radio-map contract version or RF material profile version, are refused with a message that names the recorded version. Such a dataset is continued with the generator that created it, or generated again in a new output directory.
Each run appends a record to generation_runs in the manifest with its software version,
status, and initial, generated and failed radio map counts, so the history of a dataset stays
visible. Each radio map record names the run that wrote it in generation_run; the run
records no times.
Growing a dataset#
A dataset grows by resuming it with a larger request. The request may:
raise a scene’s
radio_maps.count: the existing radio maps keep their seeds, records and arrays, and the new ones are generated;append scenes after the existing ones: an appended scene takes the next scene number, as in a new dataset generated from the whole request. Its measurement surfaces and building rasters are written with it, for any receiver height and grid.
Everything else about an existing scene is fixed, its site included: the site decides which
scenes share a split when the dataset is packaged (Packaging for release), so a new site could
move a scene into another split. Removing or reordering scenes, lowering a count and
changing output.previews are refused as well. An appended scene is fixed from the first
run that includes it, also when that run stops before reaching it, so try a new scene in a
small separate dataset first.
When the run ends, also when it stops on an error, the manifest records the run’s status, and
its data_licences, LICENSE.md and index.parquet are rebuilt from the records of the
grown dataset. The error that stopped the run reaches the caller unchanged. Stopping a run
with Ctrl+C lets the running solver step finish, records the run as failed with the error
interrupted, and leaves the radio map in progress to the next run. A killed run cannot do
this.
Package the grown dataset into a new directory, with the earlier package as --previous, so
that the earlier scenes keep their split and only new scene groups are assigned
(Packaging for release):
owrt dataset package data/generated/my-radio-dataset releases/my-radio-dataset-v2 \
--previous releases/my-radio-dataset
The index after a stopped run#
Each run advances the manifest’s index_revision before it writes a radio map and rebuilds
index.parquet when it ends, also when it stops on an error (Dataset layout). A run that is
killed leaves the index of the previous revision, which read_index refuses, as it does while
a run is in progress. Resume does not read the index; the next run rebuilds it, and so does
owrt dataset index data/generated/my-radio-dataset
This command rebuilds only the index. LICENSE.md can miss the sources of scenes the killed
run reached first; the next run rebuilds it when it ends.
A resume whose request is refused, such as one with a changed seed, leaves the index and its revision as they were. A run stopped later by a check of a scene’s saved inputs ends as a failed run and rebuilds the index.