Skip to contents

Move a completed LANDIS-II replicate from scratch to its final location

landis_archive_rep.Rd

Moves a finished replicate directory run_dir (typically on fast, local, Docker-bind-mountable scratch) to final_dir (typically a slower, networked final/archive location such as an NFS project share) in a way that is fault-tolerant and all-or-nothing at the destination:

Usage

landis_archive_rep(run_dir, final_dir, max_tries = 5L, backoff_sec = 5)

Arguments

run_dir

Character. The completed replicate directory to move (source).

final_dir

Character. The destination directory (created if needed).

max_tries

Integer. Maximum rsync attempts before giving up (default 5).

backoff_sec

Numeric. Base seconds for linear backoff between attempts (attempt i waits backoff_sec * i; default 5).

Value

Character. final_dir, on success.

Details

  1. rsync -a --partial copies run_dir into a sibling staging directory paste0(final_dir, ".partial") on the destination filesystem, with retry + linear backoff so a transient network blip does not abort the run (--partial keeps partially transferred files so a retry resumes rather than restarts). Crucially this is a COPY – the scratch source stays the complete, authoritative replicate until the destination is verified, so a total transfer failure loses nothing.

  2. After a verified rsync exit status of 0, the staging directory is published with an atomic rename into final_dir. Because the rename is atomic, final_dir only ever appears complete – a partial transfer is never visible to a downstream skip-check that reads final_dir. (This is why --remove-source-files is deliberately NOT used: it would delete scratch incrementally and could leave a half-emptied source on total failure, and a partial destination could be mistaken for a complete run.)

  3. Only once final_dir is in place is the scratch source deleted.

If run_dir and final_dir already resolve to the same path (no scratch in use), this is a no-op that returns final_dir without copying or deleting.

On Linux and macOS this requires the rsync executable on PATH. A missing rsync, or repeated failures, raises an error after max_tries attempts so the run target fails loudly and the scratch copy is retained for inspection rather than silently lost.

On Windows the staging copy is made with fs::dir_copy() instead. rsync parses host:path, so a drive-qualified path like C:/Users/... reads as the remote host C; with both ends drive-qualified it refuses to run at all ("The source and destination cannot both be remote"). What rsync buys here – resumable, fault-tolerant transfer over a network share – is a property of the Linux/macOS scratch-to-NFS deployment this function was written for, so a plain recursive copy is the right substitute rather than a degraded one. Steps 2 and 3 (atomic publish, then delete scratch) are identical on every platform.

See also

tar_landis(), landis_run_docker(), landis_replicate()

Other LANDIS-II execution helpers: host_cpu_info(), landis_assert_version(), landis_find(), landis_find_docker(), landis_pool_exec(), landis_pool_restart_one(), landis_pool_start(), landis_pool_stop(), landis_replicate(), landis_run_docker(), landis_run_local(), landis_target_version(), landis_version(), read_landis_resource_logs(), tar_landis(), validate_landis_scenario(), write_landis_scenario_file()