DEoptim driver for Dynamic Fire calibration

Sets up a warm Docker pool (for the landis simulator on Docker) and a FORK cluster of n_cores workers, then invokes DEoptim::DEoptim() with the multi-component loss as the objective. Pool + cluster are torn down via on.exit() regardless of success / error / interrupt.

Usage

calibrate_dynamic_fire(observed_targets_path, scenario_template, cfg, out_dir)

Arguments

observed_targets_path

Character. Path to the .rds from save_observed_fire_targets().

scenario_template

Character. Path to the calibration scenario's scenario.txt (the return of build_calibration_scenario_template()).

cfg

List. Calibration config. Expected keys:

lower, upper: Named numeric vectors keyed by calibration_par_names().
NP, itermax, strategy: DEoptim control args.
reltol, steptol: Optional DEoptim early-stopping controls. When set, DEoptim halts before itermax if the best-of-population objective fails to improve by more than reltol for steptol consecutive generations. Defaults: reltol = 1e-3 (0.1% relative improvement) and steptol = 25 generations. Pass steptol = itermax (or any value >= itermax) to disable early stopping and always run the full schedule; a run configured that way says so in a startup message. Omitting steptol (or setting it to NULL) gives the 25-generation default, not DEoptim's own steptol = itermax.
n_reps, sim_years, weights, base_seed: Per-trial settings.
n_cores, parallel: Parallelism settings.
simulator: "landis" (default), "r_reimpl", or "mock".
method: "docker" (default) or "local".
image, cpu_limit, mem_limit, pull: Pool settings (Docker only).
nodes: Optional named vector of workers per host, e.g. c(host1 = 30, host2 = 30), spreading the search across machines via a PSOCK cluster. Each worker runs its own single container on its own host. Per-host counts are capped against that host's available RAM and the total is trimmed to NP (workers beyond NP never receive a task but still hold a container). Requires the parallelly package, the Docker image on every host, and – under renv – the project at the same path everywhere. Unset (default) uses a local FORK cluster and one shared pool, which is unchanged. Note the workload is memory-bandwidth-bound, so one host saturates well before its cores are busy; spreading a fixed NP over more hosts therefore helps more than the host count suggests, because each host also returns to its unsaturated regime.
rscript: Path to Rscript on the worker hosts. Defaults to the coordinator's own (file.path(R.home("bin"), "Rscript")), which keeps the workers on a matching R version.
checkpoint_every: Optional integer >= 1. When set, run the search in blocks of this many generations and persist a resumable checkpoint to out_dir between blocks (see Details). NULL (default) = single monolithic DEoptim() call.
resume: "auto" (default), "never", or "force". Only used when checkpoint_every is set. "auto" resumes from out_dir/checkpoint.rds iff BOTH its population fingerprint (par names + bounds + NP) AND its loss-config fingerprint (weights + sim settings + observed + image) match; "force" resumes regardless of either; "never" ignores any checkpoint and starts from an empty memoization cache (a clean slate).
trial_timeout_sec: Optional numeric. Wall-clock ceiling on ONE simulator execution (see sim_landis()). retries only rescues a simulator that exits; one that wedges never returns, and the whole generation waits behind it. Recommended for any unattended search. Deliberately excluded from both fingerprints, so it can be added to or changed on an in-flight search without invalidating its checkpoint.

out_dir

Character. Where to write the DEoptim trace + scratch sub-directory. Created if missing.

Value

List with best_params (named numeric), objective (scalar), deoptim (full DEoptim return), trace_path (per-iter best-value CSV path), trial_trace_path (per-trial loss-decomposition CSV path, with one row per objfn evaluation; columns: wall_clock_iso, pid, par_<name>..., total, comp_<name>..., w_<name>..., weighted_<name>.... Useful for plotting how DEoptim trades off the four loss components over iterations.), cfg (echo), pool_image / pool_digest (provenance; NA when no pool was started).

Details

Designed to be called from a tar_target with deployment = "main" so the outer targets crew doesn't try to dispatch this as a single worker while it manages its own internal cluster.

Per-worker container assignment: each FORK worker sets its LANDIS_POOL_CONTAINER_IDX env var to its 1-based pool index. sim_landis() reads this when running inside the worker.

DEoptim is gated on requireNamespace("DEoptim"); install via renv::install("DEoptim") before calling.

Checkpoint / resume (opt-in)

Set cfg$checkpoint_every = K to make the search resumable. The DEoptim run is then executed in blocks of K generations; after each block the full population, best-so-far parameters, and best-value history are written atomically to out_dir (checkpoint.rds and best_params_so_far.rds) and the next block reseeds DEoptim.control(initialpop=) from the saved population. If the run is interrupted (crash, node reboot, kill), the next call resumes from the last checkpoint instead of restarting from generation 1. Resume is a warm restart: it restores the population and best-so-far, not DEoptim's internal RNG stream or generation counter, so it is not bit-for-bit identical to an uninterrupted run (acceptable for calibration). Previously evaluated parameter vectors are memoized via the trial-trace CSVs, so resumed points and per-block re-evaluations skip their (expensive) simulator runs. Point out_dir at storage that survives a reboot (the pipeline passes the NFS outputs/calibration/). When checkpoint_every is NULL (the default), behaviour is unchanged: a single monolithic DEoptim() call, no checkpoint files.

Resume and the memoization cache are both scoped to a loss-config fingerprint – a hash of the weights, per-trial sim settings (n_reps, sim_years, base_seed, simulator, method, Docker image), and the observed targets – in addition to the population fingerprint (par names, bounds, NP). Change any loss-affecting input and a checkpoint/cache left in the same out_dir is silently ignored rather than resumed or folded in, so reusing a single out_dir (e.g. the persistent outputs/calibration/) across successive calibrations with different weights or observations does NOT poison the new run's objective. You therefore do not need to clear out_dir by hand after a config change; only the population geometry (par count / bounds / NP) and the loss config must be stable for a resume to take effect.

What the fingerprint does NOT cover is the loss COMPUTATION. It digests the calibration's inputs, not the code that turns them into a number, and not the package version. So a change to how a component is calculated leaves the fingerprint byte-identical, the cache is accepted rather than rejected, and a post-change run is served pre-change losses for every parameter vector it has seen before – silently, and mixed in with correctly computed ones.

Deleting checkpoint.rds is NOT sufficient to get a clean slate: the memoized losses live in the trial-trace and worker_*.csv files, which .augment_eval_cache() folds in separately and RECURSIVELY from out_dir. After any release that changes a loss component, start the next calibration with resume = "never", which skips that step entirely.

The honest framing is that the fingerprint is a cheap guard against obviously mismatched reuse, not a correctness guarantee in either direction. It has been reported as too SENSITIVE (rejecting a valid resume after a cosmetic template rebuild) and, as above, as not sensitive ENOUGH. Both follow from digesting inputs rather than the computation.

Usage

Arguments

Value

Details

Checkpoint / resume (opt-in)

See also