Running Experiments

This page covers common invocation patterns and workflows. For the full description of every option, see Core Command Line Reference. For a conceptual overview of what each pipeline stage does, see Pipeline.

Basic Invocation

A minimal sierra invocation looks like this:

sierra                                        \
  --project        myproject                      \
  --engine         engine.argos                   \
  --expdef-template exp/template.argos            \
  --batch-criteria  population_size.1.16.C4       \
  --scenario        myScenario.10x10x1            \
  --controller      myCategory.myController

The bootstrap options (--engine, --execenv, --expdef, --storage, --proc, --prod) all have defaults and only need to be specified when overriding them. The four options that change most often between runs are --batch-criteria, --scenario, --controller, and --expdef-template — together they determine both the shape of the batch and where outputs are stored under --sierra-root. See Runtime Directory Tree for the resulting directory layout.

Specifying Batch Criteria

--batch-criteria accepts N space-separated strings, each of the form <category>.<definition>:

# Univariate: one parameter swept across 5 experiments
sierra ... --batch-criteria population_size.Linear16.C5

# Bivariate: two parameters producing a 5x4 grid of 20 experiments
sierra ... --batch-criteria population_size.Linear16.C5 max_speed.1.4.C4

# Trivariate: 3 parameters producing a 2x4x6 grid of 48 experiments
sierra ... --batch-criteria comm_distance.1.2.C2 population_size.Linear8.C4 max_speed.1.6.C6

# Built-in: 8 Monte Carlo replicates with no parameter variation
sierra ... --batch-criteria builtin.MonteCarlo.C8

<category> is a Python module name from your project's variables/ directory, or sierra.core.variables for built-ins. <definition> is a string whose format is defined by that module and parsed by its factory() function.

N criteria produce an N-dimensional Cartesian product of experiments. Stage 4 generates line graphs for univariate batches and heatmaps for bivariate; graph generation beyond 2 dimensions is not yet supported. See Batch Criteria for how experiments are named and structured on disk, and Create A New Batch Criteria to create a new criteria.

Running a Subset of Stages

A handy flowchart to help you determine which pipeline stages to run for some common workflows:

By default SIERRA runs stages 1–4. The most common reason to run a subset is to regenerate graphs after adjusting graph configuration, without re-running experiments:

sierra ... --pipeline 3 4

Or to re-run stage 2 only, after fixing a crashed experiment:

sierra ... --pipeline 2

Stage N generally requires stage N-1 to have completed successfully — running stage 4 against missing stage 3 outputs will crash or produce empty graphs. Stages 3 and 4 are idempotent and freely overwrite their own outputs; stage 2 is not. See --pipeline for the full option reference.

Re-Running Part of a Batch

To process only a contiguous slice of experiments from a batch, use --exp-range. This is useful after a partial HPC failure, or when iterating on a single experiment without waiting for the full batch:

# Re-run only experiments 2 and 3 (0-based) from a batch of 5
sierra ... --pipeline 2 --exp-range 2:3

The same range applies to stages 3 and 4 if passed there — only the specified experiments will be processed or graphed.

Resuming After a Crash

If stage 2 is killed partway through (e.g. by an HPC time limit), pass --exec-resume with the same arguments on the next invocation:

sierra ... --pipeline 2 --exec-resume

SIERRA skips any experimental run whose output directory already exists and runs only those that did not complete. Without --exec-resume, re-running stage 2 against an existing batch results in redundant work.

Starting a Batch Over

To discard and regenerate an existing batch experiment entirely, pass --exp-overwrite:

sierra ... --exp-overwrite

Warning

This deletes stage 1 inputs and regenerates them from scratch. Any existing stage 2 outputs will no longer correspond to the inputs that produced them. Use only when you intend to re-run the full batch. See Design Philosophy for why SIERRA requires explicit permission for this.

Factoring Out Common Options

Invocations grow long quickly when options are shared across many runs. Put stable options in an rcfile rather than repeating them:

# ~/.sierrarc
--project=myproject
--engine=engine.argos
--execenv=hpc.local
--expdef=expdef.xml
--sierra-root=/data/experiments

# Invocation now only needs the per-run arguments
sierra                                        \
  --batch-criteria population_size.1.16.C4        \
  --scenario       myScenario.10x10x1             \
  --controller     myCategory.myController        \
  --expdef-template exp/template.argos

Command line arguments override rcfile arguments. See --rcfile and SIERRA_RCFILE for the full priority order and non-default rcfile path options.

Note

Shortform aliases (-p, -e, -x, -s) cannot be used inside an rcfile. Only longform --option=value> syntax is supported.

Tuning Resource Usage

On machines with many cores or limited memory, two options control how aggressively SIERRA uses resources during stages 3 and 4:

• --processing-parallelism — number of worker processes for results processing and graph generation. On I/O-bound systems, increasing this above the CPU count can improve throughput.

• --processing-mem-limit — caps memory usage as a percentage of total available memory. Useful on shared machines.

For stage 2, parallelism is determined by the --execenv plugin. See Execution Environment (--execenv) for per-environment details.