Command Line Interface

Unless an option says otherwise, it is applicable to all batch criteria. That is, option batch criteria applicability is only documented for options which apply to a subset of the available Batch Criteria.

If an option is given more than once, the last such occurrence is used.

SIERRA Core

These options are for all Platforms.

Bootstrap+Multi-stage Options

usage: __main__.py [-h] [--project PROJECT] [--version]
                   [--log-level {ERROR,INFO,WARNING,DEBUG,TRACE}]
                   [--platform PLATFORM] [--skip-pkg-checks]
                   [--exec-env EXEC_ENV] [--template-input-file filepath]
                   [--exp-overwrite] [--sierra-root dirpath]
                   [--batch-criteria [<category>.<definition>,...]
                   [[<category>.<definition>,...] ...]]
                   [--pipeline [PIPELINE ...]] [--exp-range EXP_RANGE]
                   [--platform-vc] [--processing-serial] [--n-runs N_RUNS]
                   [--skip-collate] [--plot-log-xscale]
                   [--plot-enumerated-xscale] [--plot-log-yscale]
                   [--plot-regression-lines PLOT_REGRESSION_LINES]
                   [--plot-primary-axis PLOT_PRIMARY_AXIS] [--plot-large-text]
                   [--plot-transpose-graphs]

Bootstrap options

Bare-bones options for bootstrapping SIERRA

--project

Specify which Project to load.

Tip

Used by stage {1,2,3,4,5}; can be omitted otherwise. If omitted: N/A.

--version, -v

Display SIERRA version information on stdout and then exit.

Default: False

--log-level

Possible choices: ERROR, INFO, WARNING, DEBUG, TRACE

The level of logging to use when running SIERRA.

Tip

Used by stage {1,2,3,4,5}; can be omitted otherwise. If omitted: N/A.

Default: “INFO”

--platform

This argument defines the Platform you want to run experiments on.

The value of this argument determines the execution environment for experiments; different platforms (e.g., simulator, real robots) will have different configuration options.

Valid values can be any folder name inside a folder on the SIERRA_PLUGIN_PATH (with / replaced with . as you would expect for using path names to address python packages). The platforms which come with SIERRA are:

  • platform.argos - This directs SIERRA to run experiments using the ARGoS simulator. Selecting this platform assumes your code has been developed and configured for ARGoS.

  • platform.ros1gazebo - This directs SIERRA to run experiments using the Gazebo simulator and ROS1. Selecting this platform assumes your code has been developed and configured for Gazebo and ROS1.

Default: “platform.argos”

--skip-pkg-checks

Skip the usual startup package checks. Only do this if you are SURE you will never use the SIERRA functionality which requires packages you don’t have installed/can’t install.

Default: False

--exec-env

This argument defines how experiments are going to be run, using the --platform you have selected.

Valid values can be any folder name inside a folder on the SIERRA_PLUGIN_PATH (with / replaced with . as you would expect for using path names to address python packages). The execution environments which come with SIERRA are:

  • hpc.local - This directs SIERRA to run experiments on the local machine. See Local HPC Plugin for a detailed description.

  • hpc.pbs - The directs SIERRA to run experiments spread across multiple allocated nodes in an HPC computing environment managed by TORQUE-PBS. See PBS HPC Plugin for a detailed description.

  • hpc.slurm - The directs SIERRA to run experiments spread across multiple allocated nodes in an HPC computing environment managed by SLURM. See SLURM HPC Plugin for a detailed description.

  • hpc.adhoc - This will direct SIERRA to run experiments on an ad-hoc network of computers. See Adhoc HPC Plugin for a detailed description.

  • robot.turtlebot3 - This will direct SIERRA to run experiments on real Turtlebots.

Not all platforms support all execution environments.

Tip

Used by stage {1,2}; can be omitted otherwise. If omitted: N/A.

Default: “hpc.local”

Multi-stage options

Options which are used in multiple pipeline stages

--template-input-file

The template .xml input file for the batch experiment. Beyond the requirements for the specific --platform, the content of the file can be any valid XML, with the exception of the SIERRA requirements detailed in Template Input Files.

Tip

Used by stage {1,2,3,4}; can be omitted otherwise. If omitted: N/A.

--exp-overwrite

When SIERRA calculates the batch experiment root (or any child path in the batch experiment root) during stage {1, 2}, if the calculated path already exists it is treated as a fatal error and no modifications to the filesystem are performed. This flag overwrides the default behavior. Provided to avoid accidentally overwrite input/output files for an experiment, forcing the user to be explicit with potentially dangerous actions.

Tip

Used by stage {1,2}; can be omitted otherwise. If omitted: N/A.

Default: False

--sierra-root

Root directory for all SIERRA generated and created files.

Subdirectories for controllers, scenarios, experiment/experimental run inputs/outputs will be created in this directory as needed. Can persist between invocations of SIERRA.

Tip

Used by stage {1,2,3,4,5}; can be omitted otherwise. If omitted: N/A.

Default: “<home directory>/exp”

--batch-criteria

Definition of criteria(s) to use to define the experiment.

Specified as a list of 0 or 1 space separated strings, each with the following general structure:

<category>.<definition>

<category> must be a filename from the core/variables/ or from a --project <project>/variables/ directory, and <definition> must be a parsable namne (according to the requirements of the criteria defined by the parser for <category>).

Tip

Used by stage {1,2,3,4,5}; can be omitted otherwise. If omitted: N/A.

Default: []

--pipeline

Define which stages of the experimental pipeline to run:

  • Stage1 - Generate the experiment definition from the template input file, batch criteria, and other command line options. Part of default pipeline.

  • Stage2 - Run a previously generated experiment. Part of default pipeline.

  • Stage3 - Post-process experimental results after running the batch experiment; some parts of this can be done in parallel. Part of default pipeline.

  • Stage4 - Perform deliverable generation after processing results for a batch experiment, which can include shiny graphs and videos. Part of default pipeline.

  • Stage5 - Perform graph generation for comparing controllers AFTER graph generation for batch experiments has been run. Not part of default pipeline.

Default: [1, 2, 3, 4]

--exp-range

Set the experiment numbers from the batch to run, average, generate intra-experiment graphs from, or generate inter-experiment graphs from (0 based). Specified in the form min_exp_num:max_exp_num (closed interval/inclusive). If omitted, runs, averages, and generates intra-experiment and inter-experiment performance measure graphs for all experiments in the batch (default behavior).

This is useful to re-run part of a batch experiment in HPC environments if SIERRA gets killed before it finishes running all experiments in the batch, or to redo a single experiment with real robots which failed for some reason.

Tip

Used by stage {2,3,4}; can be omitted otherwise. If omitted: N/A.

--platform-vc

For applicable --platforms, enable visual capturing of run-time data during stage 2. This data can be frames (i.e., .png files), or rendering videos, depending on the platform. If the captured data was frames, then SIERRA can render the captured frames into videos during stage 4. If the selected --platform does not support visual capture, then this option has no effect. See Platform Visual Capture for full details.

Tip

Used by stage {1,4}; can be omitted otherwise. If omitted: N/A.

Default: False

--processing-serial

If TRUE, then results processing/graph generation will be performed serially, rather than using parallellism where possible.

Tip

Used by stage {3,4}; can be omitted otherwise. If omitted: N/A.

Default: False

--n-runs

The # of experimental runs that will be executed and their results processed to form the result of a single experiment within a batch.

If --platform is a simulator and --exec-env is something other than hpc.local then this may be be used to determine the concurrency of experimental runs.

Tip

Used by stage {1,2}; can be omitted otherwise. If omitted: N/A.

--skip-collate

Specify that no collation of data across experiments within a batch (stage 4) or across runs within an experiment (stage 3) should be performed. Useful if collation takes a long time and multiple types of stage 4 outputs are desired. Collation is generally idempotent unless you change the stage3 options (YMMV).

Tip

Used by stage {3,4}; can be omitted otherwise. If omitted: N/A.

Default: False

--plot-log-xscale

Place the set of X values used to generate intra- and inter-experiment graphs into the logarithmic space. Mainly useful when the batch criteria involves large system sizes, so that the plots are more readable.

Tip

Applicable graphs:

Tip

Used by stage {4,5}; can be omitted otherwise. If omitted: N/A.

Default: False

--plot-enumerated-xscale

Instead of using the values generated by a given batch criteria for the X values, use an enumerated list[0, …, len(X value) - 1]. Mainly useful when the batch criteria involves large system sizes, so that the plots are more readable.

Tip

Applicable graphs:

Tip

Used by stage {4,5}; can be omitted otherwise. If omitted: N/A.

Default: False

--plot-log-yscale

Place the set of Y values used to generate intra - and inter-experiment graphs into the logarithmic space. Mainly useful when the batch criteria involves large system sizes, so that the plots are more readable.

Tip

Applicable graphs:

Tip

Used by stage {4,5}; can be omitted otherwise. If omitted: N/A.

Default: False

--plot-regression-lines

For all 2D generated scatterplots, plot a linear regression line and the equation of the line to the legend.

Tip

Applicable graphs:

Tip

Used by stage {4,5}; can be omitted otherwise. If omitted: N/A.

--plot-primary-axis

This option allows you to override the primary axis, which is normally is computed based on the batch criteria.

For example, in a bivariate batch criteria composed of

  • Population Size on the X axis (rows)

  • Another batch criteria which does not affect system size (columns)

Metrics will be calculated by computing across .csv rows and projecting down the columns by default, since system size will only vary within a row. Passing a value of 1 to this option will override this calculation, which can be useful in bivariate batch criteria in which you are interested in the effect of the OTHER non-size criteria on various performance measures.

0=criteria of interest varies across rows.

1=criteria of interest varies across columns.

This option only affects generating graphs from bivariate batch criteria.

Tip

Applicable graphs:

Tip

Used by stage {4,5}; can be omitted otherwise. If omitted: N/A.

--plot-large-text

This option specifies that the title, X/Y axis labels/tick labels should be larger than the SIERRA default. This is useful when generating graphs suitable for two column paper format where the default text size for rendered graphs will be too small to see easily. The SIERRA defaults are generally fine for the one column/journal paper format.

Tip

Used by stage {4,5}; can be omitted otherwise. If omitted: N/A.

Default: False

--plot-transpose-graphs

Transpose the X, Y axes in generated graphs. Useful as a general way to tweak graphs for best use of space within a paper.

Changed in version 1.2.20: Renamed from --transpose-graphs to make its relation to other plotting options clearer.

Tip

Applicable graphs:

Tip

Used by stage {4,5}; can be omitted otherwise. If omitted: N/A.

Default: False

Stage1: Generating Experiments

None for the moment.

Stage2: Running Experiments

None for the moment.

Stage3: Processing Experiment Results

usage: SIERRA [--df-skip-verify] [--storage-medium {storage.csv}]
              [--dist-stats {none,all,conf95,bw}]
              [--processing-mem-limit PROCESSING_MEM_LIMIT]
              [--df-homogenize DF_HOMOGENIZE]

Stage3: General options for eprocessing experiment results

--df-skip-verify

SIERRA generally assumes/relies on all dataframes with the same name having the same # of columns which are of equivalent length across Experimental Runs (different columns within a dataframe can of course have different lengths). This is strictly verified during stage 3 by default.

If passed, then the verification step will be skipped during experimental results processing, and outputs will be averaged directly.

If not all the corresponding CSV files in all experiments generated the same # rows, then SIERRA will (probably) crash during experiments exist and/or have the stage4. Verification can take a long time with large # of runs and/or dataframes per experiment.

Tip

Used by stage {3}; can be omitted otherwise. If omitted: N/A.

Default: False

--storage-medium

Possible choices: storage.csv

Specify the storage medium for Experimental Run outputs, so that SIERRA can select an appropriate plugin to read them. Any plugin under plugins/storage can be used, but the ones that come with SIERRA are:

  • storage.csv - Experimental run outputs are stored in a per-run directory as one or more CSV files.

Regardless of the value of this option, SIERRA always generates CSV files as it runs and averages outputs, generates graphs, etc.

Tip

Used by stage {3}; can be omitted otherwise. If omitted: N/A.

Default: “storage.csv”

--dist-stats

Possible choices: none, all, conf95, bw

Specify what kinds of statistics, if any, should be calculated on the distribution of experimental data during stage 3 for inclusion on graphs during stage 4:

  • none - Only calculate and show raw mean on graphs.

  • conf95 - Calculate standard deviation of experimental distribution and show 95% confidence interval on relevant graphs.

  • bw - Calculate statistics necessary to show box and whisker plots around each point in the graph. :class: ~sierra.core.graphs.summary_line_graph.SummaryLineGraph only).

  • all - Generate all possible statistics, and plot all possible statistics on graphs.

Tip

Applicable graphs:

Tip

Used by stage {3,4}; can be omitted otherwise. If omitted: N/A.

Default: “none”

--processing-mem-limit

Specify, as a percent in [0, 100], how much memory SIERRA should try to limit itself to using. This is useful on systems with limited memory, or on systems which are shared with other users without per-user memory restrictions.

Tip

Used by stage {3,4}; can be omitted otherwise. If omitted: N/A.

Default: 90

--df-homogenize

SIERRA generally assumes/relies on all dataframes with the same name having the same # of columns which are of equivalent length across: term: Experimental Runs < Experimental Run > (different columns within a dataframe can of course have different lengths). This is checked during stage 3 unless --df-skip-verify is passed. If strict verification is skipped, then SIERRA provides the following options when processing dataframes during stage {3, 4} to to homogenize them:

  • none - Don’t do anything. This may or may not produce crashes during stage 4, depending on what you are doing.

  • pad - Project last valid value in columns which are too short down the column to make it match those which are longer.

    Note that this may result in invalid data/graphs if the filled columns are intervallic, interval average, or cumulative average data. If the data is a cumulative count of something, then this policy will have no ill effects.

  • zero - Same as pad, but always fill with zeroes.

Homogenization is performed just before writing dataframes to the specified storage medium. Useful with real robot experiments if the number of datapoints captured per-robot is slightly different, depending on when they started executing relative to when the experiment started.

Default: “none”

Stage4: Deliverable Generation

usage: SIERRA [--exp-graphs {intra,inter,all,none}] [--project-no-LN]
              [--project-no-HM] [--models-enable]
              [--render-cmd-opts RENDER_CMD_OPTS] [--project-imagizing]
              [--project-rendering] [--bc-rendering]

Stage4: General options for generating graphs

--exp-graphs

Possible choices: intra, inter, all, none

Specify which types of graphs should be generated from experimental results:

  • intra - Generate intra-experiment graphs from the results of a single experiment within a batch, for each experiment in the batch(this can take a long time with large batch experiments). If any intra-experiment models are defined and enabled, those are run and the results placed on appropriate graphs.

  • inter - Generate inter-experiment graphs _across_ the results of all experiments in a batch. These are very fast to generate, regardless of batch experiment size. If any inter-experiment models are defined and enabled, those are run and the results placed on appropriate graphs.

  • all - Generate all types of graphs.

  • none - Skip graph generation; provided to skip graph generation if only video outputs are desired.

Tip

Used by stage {4}; can be omitted otherwise. If omitted: N/A.

Default: “all”

--project-no-LN

Specify that the intra-experiment and inter-experiment linegraphs defined in project YAML configuration should not be generated. Useful if you are working on something which results in the generation of other types of graphs, and the generation of those linegraphs is not currently needed only slows down your development cycle.

Model linegraphs are still generated, if applicable.

Default: False

--project-no-HM

Specify that the intra-experiment heatmaps defined in project YAML configuration should not be generated. Useful if:

  • You are working on something which results in the generation of other types of graphs, and the generation of heatmaps only slows down your development cycle.

  • You are working on stage5 comparison graphs for bivariate batch criteria, and re-generating many heatmaps during stage4 is taking too long.

Model heatmaps are still generated, if applicable.

New in version 1.2.20.

Default: False

Models

--models-enable

Enable running of all models; otherwise, no models are run, even if they appear in the project config file. The logic behind having models disabled by default is that most users won’t have them.

Default: False

Stage4: Rendering (see also stage1 rendering options)

--render-cmd-opts

Specify the: program: ffmpeg options to appear between the specification of the input image files and the specification of the output file. The default is suitable for use with ARGoS frame grabbing set to a frames size of 1600x1200 to output a reasonable quality video.

Tip

Used by stage {4}; can be omitted otherwise. If omitted: N/A.

Default: “-r 10 -s:v 800x600 -c:v libx264 -crf 25 -filter:v scale=-2:956 -pix_fmt yuv420p”

--project-imagizing

Enable generation of image files from CSV files captured during stage 2 and averaged during stage 3 for each experiment. See Project Rendering for details and restrictions.

Tip

Used by stage {3,4}; can be omitted otherwise. If omitted: N/A.

Default: False

--project-rendering

Enable generation of videos from imagized CSV files created as a result of --project-imagizing. See Project Rendering for details.

Tip

Used by stage {4}; can be omitted otherwise. If omitted: N/A.

Default: False

--bc-rendering

Enable generation of videos from generated graphs, such as heatmaps. Bivariate batch criteria only.

Tip

Used by stage {4}; can be omitted otherwise. If omitted: N/A.

Default: False

Stage5: Comparing Controllers

usage: SIERRA [--controllers-list CONTROLLERS_LIST]
              [--controllers-legend CONTROLLERS_LEGEND]
              [--scenarios-list SCENARIOS_LIST]
              [--scenarios-legend SCENARIOS_LEGEND] [--scenario-comparison]
              [--controller-comparison]
              [--comparison-type {LNraw,HMraw,HMdiff,HMscale,SUraw,SUscale,SUdiff}]
              [--bc-univar] [--bc-bivar]

Stage5: General options for controller comparison

--controllers-list

Comma separated list of controllers to compare within --sierra-root.

The first controller in this list will be used for as the controller of primary interest if --comparison-type is passed.

Tip

Used by stage {5}; can be omitted otherwise. If omitted: N/A.

--controllers-legend

Comma separated list of names to use on the legend for the generated comparison graphs, specified in the same order as the --controllers-list.

Tip

Used by stage {5}; can be omitted otherwise. If omitted: the raw controller names will be used.

--scenarios-list

Comma separated list of scenarios to compare --controller across within --sierra-root.

Tip

Used by stage {5}; can be omitted otherwise. If omitted: N/A.

--scenarios-legend

Comma separated list of names to use on the legend for the generated inter-scenario controller comparison graphs(if applicable), specified in the same order as the --scenarios-list.

Tip

Used by stage {5}; can be omitted otherwise. If omitted: the raw scenario names will be used.

--scenario-comparison

Perform a comparison of --controller across --scenarios-list (univariate batch criteria only).

Tip

Used by stage {5}; can be omitted otherwise. Either --scenario-comparison or --controller-comparison must be passed.

Default: False

--controller-comparison

Perform a comparison of --controllers-list across all scenarios at least one controller has been run on.

Tip

Used by stage {5}; can be omitted otherwise. Either --scenario-comparison or --controller-comparison must be passed.

Default: False

--comparison-type

Possible choices: LNraw, HMraw, HMdiff, HMscale, SUraw, SUscale, SUdiff

Specify how controller comparisons should be performed.

If the batch criteria is univariate, the options are:

  • LNraw - Output raw 1D performance measures using a single SummaryLineGraph for each measure, with all --controllers-list controllers shown on the same graph.

If the batch criteria is bivariate, the options are:

  • LNraw - Output raw performance measures as a set of :class:`~sierra.core.graphs.summary_line_graph.SummaryLineGraph`s, where each line graph is constructed from the i-th row/column for the 2D dataframe for the performance results for all controllers.

    Note

    SIERRA cannot currently plot statistics on the linegraphs built from slices of the 2D CSVs/heatmaps generated during stage4, because statistics generation is limited to stage3. This limitation may be removed in a future release.

  • HMraw - Output raw 2D performance measures as a set of dual heatmaps comparing all controllers against the controller of primary interest(one per pair).

  • HMdiff - Subtract the performance measure of the controller of primary interest against all other controllers, pairwise, outputting one 2D heatmap per comparison.

  • HMscale - Scale controller performance measures against those of the controller of primary interest by dividing, outputing one 2D heatmap per comparison.

  • SUraw - Output raw 3D performance measures as a single, stacked 3D surface plots comparing all controllers(identical plots, but viewed from different angles).

  • SUscale - Scale controller performance measures against those of the controller of primary interest by dividing. This results in a single stacked 3D surface plots comparing all controllers(identical plots, but viewed from different angles).

  • SUdiff - Subtract the performance measure of the controller of primary interest from each controller(including the primary). This results in a set single stacked 3D surface plots comparing all controllers(identical plots, but viewed from different angles), in which the controller of primary interest forms an(X, Y) plane at Z=0.

For all comparison types, --controllers-legend is used if passed for legend.

Tip

Used by stage {5}; can be omitted otherwise. If omitted: N/A.

--bc-univar

Specify that the batch criteria is univariate. This cannot be deduced from the command line --batch-criteria argument in all cases because we are comparing controllers across scenarios, and each scenario (potentially) has a different batch criteria definition, which will result in (potentially) erroneous comparisons if we don’t re-generate the batch criteria for each scenaro we compare controllers within.

Tip

Used by stage {5}; can be omitted otherwise. If omitted: N/A.

Default: False

--bc-bivar

Specify that the batch criteria is bivariate. This cannot be deduced from the command line --batch-criteria argument in all cases because we are comparing controllers across scenarios, and each scenario(potentially) has a different batch criteria definition, which will result in (potentially) erroneous comparisons if we don’t re-generate the batch criteria for each scenaro we compare controllers in .

Tip

Used by stage {5}; can be omitted otherwise. If omitted: N/A.

Default: False

ARGoS Platform

These options are enabled if --platform=platform.argos is passed.

Stage1: Generating Experiments

usage: SIERRA [--exp-setup EXP_SETUP] [--physics-engine-type2D {dynamics2d}]
              [--physics-engine-type3D {dynamics3d}]
              [--physics-n-engines {1,2,4,6,8,12,16,24}]
              [--physics-iter-per-tick PHYSICS_ITER_PER_TICK]
              [--physics-spatial-hash2D]
              [--camera-config {overhead,sw,sw+interp}] [--with-robot-rab]
              [--with-robot-leds] [--with-robot-battery] [--n-robots N_ROBOTS]

Stage1: Experiment generation

--exp-setup

Defines experiment run length, Ticks per second for the experiment (<experiment> tag), # of datapoints to capture/capture interval for each simulation. See Experiment Setup for a full description.

Tip

Used by stage {1}; can be omitted otherwise. If omitted: N/A.

Default: “exp_setup.T5000.K5.N50”

Stage1: Configuring ARGoS physics engines

--physics-engine-type2D

Possible choices: dynamics2d

The type of 2D physics engine to use for managing spatial extents within the arena, choosing one of the types that ARGoS supports. The precise 2D areas (if any) within the arena which will be controlled by 2D physics engines is defined on a per --project basis.

Tip

Used by stage {1}; can be omitted otherwise. If omitted: N/A.

Default: “dynamics2d”

--physics-engine-type3D

Possible choices: dynamics3d

The type of 3D physics engine to use for managing 3D volumetric extents within the arena, choosing one of the types that ARGoS supports. The precise 3D volumes (if any) within the arena which will be controlled by 3D physics engines is defined on a per --project basis.

Tip

Used by stage {1}; can be omitted otherwise. If omitted: N/A.

Default: “dynamics3d”

--physics-n-engines

Possible choices: 1, 2, 4, 6, 8, 12, 16, 24

# of physics engines to use during simulation (yay ARGoS!). If N > 1, the engines will be tiled in a uniform grid within the arena (X and Y spacing may not be the same depending on dimensions and how many engines are chosen, however), extending upward in Z to the height specified by --scenario (i.e., forming a set of “silos” rather that equal volumetric extents).

If 2D and 3D physics engines are mixed, then half of the specified # of engines will be allocated among all arena extents cumulatively managed by each type of engine. For example, if 4 engines are used, with 1/3 of the arena managed by 2D engines and 2/3 by 3D, then 2 2D engines will manage 1/3 of the arena, and 2 3D engines will manage the other 2/3 of the arena.

If --exec-env is something other than hpc.local then the # physics engines will be computed from the HPC environment, and the cmdline value (if any) will be ignored.

Important

When using multiple physics engines, always make sure that # engines / arena dimension (X AND Y dimensions) is always a rational number. That is,

  • 24 engines in a 12x12 arena will be fine, because 12/24=0.5, which can be represented reasonably well in floating point.

  • 24 engines in a 16x16 arena will not be fine, because 16/24=0.666667, which will very likely result in rounding errors and ARGoS being unable to initialize the space because it can’t place arena walls.

This is enforced by SIERRA.

Tip

Used by stage {1}; can be omitted otherwise. If omitted: N/A.

--physics-iter-per-tick

The # of iterations all physics engines should perform per Tick each time the controller loops are run (the # of ticks per second for controller control loops is set via --exp-setup).

Tip

Used by stage {1}; can be omitted otherwise. If omitted: N/A.

Default: 10

--physics-spatial-hash2D

Specify that each 2D physics engine should use a spatial hash (only applies to dynamics2d engine type).

Default: False

Stage1: Rendering (see also stage4 rendering options)

--camera-config

Possible choices: overhead, sw, sw+interp

Select the camera configuration for simulation. Ignored unless --platform-vc is passed. Valid values are:

  • overhead - Use a single overhead camera at the center of the aren looking straight down at an appropriate height to see the whole arena.

  • sw - Use the ARGoS camera configuration (12 cameras), cycling through them periodically throughout simulation without interpolation.

  • sw+interp - Same as sw, but with interpolation between cameras.

Tip

Used by stage {1}; can be omitted otherwise. If omitted: N/A.

Default: “overhead”

Stage1: Configuring robots

--with-robot-rab

If passed, do not remove the Range and Bearing (RAB) sensor, actuator, and medium XML definitions from --template-input-file before generating experimental inputs. Otherwise, the following XML tags are removed if they exist:

  • .//media/range_and_bearing

  • .//actuators/range_and_bearing

  • .//sensors/range_and_bearing

Tip

Used by stage {1}; can be omitted otherwise. If omitted: N/A.

Default: False

--with-robot-leds

If passed, do not remove the robot LED actuator XML definitions from the --template-input-file before generating experimental inputs. Otherwise, the following XML tags are removed if they exist:

  • .//actuators/leds

  • .//medium/leds

  • .//sensors/colored_blob_omnidirectional_camera

Tip

Used by stage {1}; can be omitted otherwise. If omitted: N/A.

Default: False

--with-robot-battery

If passed, do not remove the robot battery sensor XML definitions from --template-input-file before generating experimental inputs. Otherwise, the following XML tags are removed if they exist:

  • .//entity/*/battery

  • .//sensors/battery

Tip

Used by stage {1}; can be omitted otherwise. If omitted: N/A.

Default: False

--n-robots

The # robots that should be used in the simulation. Can be used to override batch criteria, or to supplement experiments that do not set it so that manual modification of input file is unneccesary.

Tip

Used by stage {1}; can be omitted otherwise. If omitted: N/A.

Stage2: Running Experiments

usage: SIERRA [--exec-jobs-per-node EXEC_JOBS_PER_NODE] [--exec-devnull]
              [--exec-no-devnull] [--exec-resume] [--exec-strict]

HPC options

For platforms which are simulators (and can therefore be run in HPC environments).

--exec-jobs-per-node

Specify the maximum number of parallel jobs to run per allocated node. By default this is computed from the selected HPC environment for maximum throughput given the desired --n-runs and CPUs per allocated node. However, for some environments being able to override the computed default can be useful.

Tip

Used by stage {2}; can be omitted otherwise. If omitted: N/A.

--exec-devnull

Redirect ALL output from simulations to /dev/null. Useful for platform where you can’t disable all INFO messages at compile time, and don’t want to have to grep through lots of redundant stdout files to see if there were any errors.

Tip

Used by stage {1,2}; can be omitted otherwise. If omitted: N/A.

Default: True

--exec-no-devnull

Don’t redirect ALL output from simulations to /dev/null. Useful for platform where you can’t disable all INFO messages at compile time, and don’t want to have to grep through lots of redundant stdout files to see if there were any errors.

Tip

Used by stage {1,2}; can be omitted otherwise. If omitted: N/A.

Default: True

--exec-resume

Resume a batch experiment that was killed/stopped/etc last time SIERRA was run.

Tip

Used by stage {2}; can be omitted otherwise. If omitted: N/A.

Default: False

--exec-strict

If passed, then if any experimental commands fail during stage 2 SIERRA will exit, rather than try to keep going and execute the rest of the experiments.

Useful for:

  • “Correctness by construction” experiments, where you know if SIERRA doesn’t crash and it makes it to the end of your batch experiment then none of the individual experiments crashed.

  • CI pipelines

Default: False

ROS1+Gazebo Platform

These options are enabled if --platform=platform.ros1gazebo is passed.

Stage1: Generating Experiments

usage: SIERRA [--exp-setup EXP_SETUP] [--robot ROBOT]
              [--robot-positions ROBOT_POSITIONS [ROBOT_POSITIONS ...]]
              [--physics-engine-type {ode,bullet,dart,simbody}]
              [--physics-iter-per-tick PHYSICS_ITER_PER_TICK]
              [--physics-n-threads PHYSICS_N_THREADS]
              [--physics-ec-threadpool PHYSICS_EC_THREADPOOL]

Stage1: Experiment generation

--exp-setup

Defines experiment run length, ticks per second for the experiment, # of datapoints to capture/capture interval for each simulation. See Experiment Setup for a full description.

Tip

Used by stage {1}; can be omitted otherwise. If omitted: N/A.

Default: “exp_setup.T1000.K5.N50”

--robot

The key name of the robot model, which must be present in the appropriate section of main.yaml for the Project. See Main Configuration for details.

Tip

Used by stage {1}; can be omitted otherwise. If omitted: N/A.

Stage1: Experiment setup

--robot-positions

A list of space-separated “X,Y,Z” tuples (no quotes) passed on the command line as valid starting positions for the robots within the world.

Tip

Used by stage {1}; can be omitted otherwise. If omitted: effective arena dimensions must be given as part of --scenario.

Default: []

Stage1: Configuring Gazebo physics engines

--physics-engine-type

Possible choices: ode, bullet, dart, simbody

The type of 3D physics engine to use for managing spatial extents within the arena, choosing one of the types that Gazebo supports. A single engine instance is used to manage all physics in the arena.

Tip

Used by stage {1}; can be omitted otherwise. If omitted: N/A.

Default: “ode”

--physics-iter-per-tick

The # of iterations all physics engines should perform per tick each time the controller loops are run (the # of ticks per second for controller control loops is set via --exp-setup).

Tip

Used by stage {1}; can be omitted otherwise. If omitted: N/A.

Default: 1000

--physics-n-threads

Gazebo can group non-interacting entities into computational “islands” and do the physics updates for those islands in parallel each timestep (islands) are recomputed after each timestep). Gazebo can also parallelize the computation of velocity/position updates with the computation of resolving collisions (i.e., the timestep impulse results in one entity “inside” another). You can assign multiple threads to a pool for cumulative use for these two parallelization methods (threads will be allocated evenly between them). The point at which adding more threads will start to DECREASE performance depends on the complexity of your world, the number and type of robots in it, etc., so don’t just set this parameter to the # of cores for your machine as a default.

From the Gazebo Parallel Physics Report, setting the pool size to the # robots/# joint trees in your simulation usually gives good results, as long as you have more cores available than you allocate to this pool (Gazebo has other threads too).

This only applies if --physics-engine-type=ode.

A value of 0=no threads.

Tip

Used by stage {1}; can be omitted otherwise. If omitted: N/A.

Default: 0

--physics-ec-threadpool

Gazebo can parallelize the computation of velocity/position updates with the computation of resolving collisions (i.e., the timestep impulse results in one entity “inside” another). You can assign multiple threads to a pool for cumulative use for this purpose. The point at which adding more threads will start to DECREASE performance depends on the complexity of your world, the number and type of robots in it, etc., so don’t just set this parameter to the # of cores for your machine as a default.

From the Gazebo Parallel Physics Report, setting the pool size to the # robots/#joint trees in your simulation usually gives good results, as long as you have more cores than than you allocate to physics (Gazebo has other threads too).

This only applies if --physics-engine-type=ode.

A value of 0=no threads.

Tip

Used by stage {1}; can be omitted otherwise. If omitted: N/A.

Default: 0

Stage2: Running Experiments

usage: SIERRA [--exec-jobs-per-node EXEC_JOBS_PER_NODE] [--exec-devnull]
              [--exec-no-devnull] [--exec-resume] [--exec-strict]

HPC options

For platforms which are simulators (and can therefore be run in HPC environments).

--exec-jobs-per-node

Specify the maximum number of parallel jobs to run per allocated node. By default this is computed from the selected HPC environment for maximum throughput given the desired --n-runs and CPUs per allocated node. However, for some environments being able to override the computed default can be useful.

Tip

Used by stage {2}; can be omitted otherwise. If omitted: N/A.

--exec-devnull

Redirect ALL output from simulations to /dev/null. Useful for platform where you can’t disable all INFO messages at compile time, and don’t want to have to grep through lots of redundant stdout files to see if there were any errors.

Tip

Used by stage {1,2}; can be omitted otherwise. If omitted: N/A.

Default: True

--exec-no-devnull

Don’t redirect ALL output from simulations to /dev/null. Useful for platform where you can’t disable all INFO messages at compile time, and don’t want to have to grep through lots of redundant stdout files to see if there were any errors.

Tip

Used by stage {1,2}; can be omitted otherwise. If omitted: N/A.

Default: True

--exec-resume

Resume a batch experiment that was killed/stopped/etc last time SIERRA was run.

Tip

Used by stage {2}; can be omitted otherwise. If omitted: N/A.

Default: False

--exec-strict

If passed, then if any experimental commands fail during stage 2 SIERRA will exit, rather than try to keep going and execute the rest of the experiments.

Useful for:

  • “Correctness by construction” experiments, where you know if SIERRA doesn’t crash and it makes it to the end of your batch experiment then none of the individual experiments crashed.

  • CI pipelines

Default: False

ROS1+Robot Platform

These options are enabled if --platform=platform.ros1robot is passed.

Stage1: Generating Experiments

usage: SIERRA [--exp-setup EXP_SETUP] [--robot ROBOT] [--skip-sync]

Stage1: Experiment generation

--exp-setup

Defines experiment run length, ticks per second for the experiment, # of datapoints to capture/capture interval for each simulation. See Experiment Setup for a full description.

Tip

Used by stage {1}; can be omitted otherwise. If omitted: N/A.

Default: “exp_setup.T1000.K5.N50”

--robot

The key name of the robot model, which must be present in the appropriate section of main.yaml for the Project. See Main Configuration for details.

Tip

Used by stage {1}; can be omitted otherwise. If omitted: N/A.

Stage1: Experiment generation

--skip-sync

If passed, then the generated experiment will not be synced to robots. This is useful when:

  • You are developing your Project and just want to check locally if the experiment is being generated properly.

  • You have a lot of robots and/or the network connection from the SIERRA host machine to the robots is slow, and copying the experiment multiple times as you tweak parameters takes a long time.

Tip

Used by stage {1}; can be omitted otherwise. If omitted: N/A.

Default: False

Stage2: Running Experiments

usage: SIERRA [--exec-inter-run-pause SECONDS] [--exec-resume]

Stage2: Experiment executionFor running real robot experiments

--exec-inter-run-pause

How long to pause between Experimental Runs, giving you time to reset the environment, move robots, etc.

Tip

Used by stage {2}; can be omitted otherwise. If omitted: N/A.

Default: 60

--exec-resume

Resume a batch experiment that was killed/stopped/etc last time SIERRA was run.

Tip

Used by stage {2}; can be omitted otherwise. If omitted: N/A.

Default: False