Glossary#

SIERRA uses a number of common research-related terms, perhaps in different ways than you are used to. They are defined here to help demystify why SIERRA works/is designed the way it is, and to help you find your way around.

ARGoS#

A state-of-the-art multi-physics engine robotics simulator which SIERRA supports as an Engine. The ARGoS website is at https://www.argos-sim.info/index.php.

Batch Criteria#

See Batch Criteria for a thorough explanation of batch criteria, but the short version is that they are the core of SIERRA--how to get it to DO stuff for you as you create and define an experiment which investigates a variable (or multiple variables) across a range of values, which when instantiated gives you a batch of experiments to run.

Batch Experiment#

A set of Experiments each defined by expdef changes generated by the selected Batch Criteria to a template .argos file passed to SIERRA during stage 1 via --expdef-template.

For example, for the Population Size batch criteria, each experiment is defined by a single expdef change to the provided .argos file: the number of robots in the swarm. Depending on the specifics you set for the range of sizes you are interested in, several experiments will be generated from the template .argos file, each differing from the template in the configured swarm size.

Batch Summary Data#

A csv/arrow/etc file created by SIERRA during stage 4 (if inter-experiment graph generation is to be run). Created from a Collated Output Data file by taking the last row; this usually corresponds to things like:

  • Steady-state behavior

  • Cumulative counts of something

These flavor of measures are usually what you are after. However, if you want something different you can specify it in a per-graph manner; see Graph Generation for more info. You can also capture transient behaviors by creating Collated Output Data files and thus Batch Summary Data files from captured Experimental Run outputs over short stretches of time--SIERRA does not know the difference.

Collated Output Data#

A csv/arrow/etc file created by SIERRA during stage 3/4 via Data Collation. See Intra-Experiment Data Collation.

Controller Category#

A semantic label attached to a set of controllers which are similar in some way. For example, if you have two controllers which use the same type of memory (say it's a "last N objects seen" memory), you could create a LastN category, and then define controllers within it, e.g., LastN.Ring and LastN.DecayRing for two controllers which have a ringbuffer of remembered objects and a decaying ringbuffer of remembered objects (i.e., an object is forgotten after some period of time even if it is not forced out of the ringbuffer by seeing a new object). See configuring your project.

Data Collation#

The process of taking a set of user-specified columns from a set of files and putting them into a single file. Comes in three flavors:

This can be visualized as follows:

../../_images/data-collation.png

For use in product generation and running models. See Intra-Experiment Data Collation for more information.

Engine#

The executor for experiments: either via a simulator of some kind, or a run-time framework for deploying code to one or more real robots.

Important

In SIERRA terminology, engine != OS. A given OS such has linux might support multiple engines like ARGoS, ROS, etc, while a different OS like OSX might support only ARGoS.

Experiment#

A single datapoint within a Batch Experiment; that is a single value of the thing that you are interested in varying across some range of experiments to see what happens (or doesn't happen).

Experimental Run#

Meaning is Engine dependent.

For simulator-based engines, such as ARGoS, each run is a simulation run. For real-world engines such as ROS1+Robot, it is a run with physical robots.

The number of runs which will be run by SIERRA in stage 2 and averaged together by SIERRA in stage 3 is controlled by --n-runs.

All runs in within an Experiment are identical, with the exception of:

  • Different values for the expdef changes resulting from the different experiments they are part of, as defined by the batch criteria generating the batch experiment.

  • Different random seeds

Gazebo#

A state-of-the-art robotics simulator with many features including .urdf robot models, realistic 3D rendering, and more. The Gazebo website is at https://gazebosim.org.

Graph Category#

A semantic label attached to a set of graphs which are similar. For example, if you want to generate graphs about the different ways that robots allocate tasks, you might create a LN_task_alloc label, so that you can enable/disable all task allocation related graphs for one or more controllers easily when configuring your project.

Imagize#

The process of taking a Raw Output Data or Processed Output Data file and turning it into a PNG/JPG file.

Imagizing#

The process of turning a text file of some kind (e.g., CSV, .gml) into an image.

Inter-Batch Data#

A CSV file created by SIERRA during stage 5. An inter-batch CSV is created by "collating" columns from a Batch Summary Data present in multiple Batch Experiments into a single CSV. Used during stage 5.

Model#

A python implementation of a theoretical model of some kind. Can use empirical data from simulations/real robot experiments, or not, as needed. Intended to generate predictions of something which can then be plotted against empirical results for comparison.

Plugin#

A python package/module living in a directory on SIERRA_PLUGIN_PATH which contains functionality to extend SIERRA without modifying its core (i.e., customization of different parts of the pipeline). Plugins come in several flavors, all of which are handled equivalently by SIERRA:

  • Engine plugins - Plugins which correspond to different Engines.

  • Storage plugins - Plugins which correspond to different experiment output formats, and are used in translating said formats into the internal representation used by SIERRA.

  • Execution environment plugins - Plugins which correspond to different run-time environments in which experiments are executed.

  • Experiment definition plugins - Plugins which correspond to different file input formats which are used by SIERRA to create runnable experiments from a template input file.

  • Data processing plugins - Plugins which correspond to different ways to process Raw Output Data files.

  • Project plugins - Plugins which correspond to different Projects.

  • Comparator plugins - Plugins which correspond to different way to compare generated Products during stage 5.

Prefect#

Python-based scheduling framework for defining and executing workflows. The website is at https://www.prefect.io.

Processed Output Data#

A csv/arrow/etc file generated as from averaging files from multiple Experimental Runs. It will (probably) contain a set of columns of representing outputs of interest, with rows corresponding to values captured throughout the run (i.e., a time series).

Product#

Something which is of interest/value generated duration stage 4. Probably a file of some kind, like a graphical image or video.

Project#

The SIERRA plugin allowing you to use your code on the engine of your choice within the SIERRA framework. A project is mainly a set of .yaml configuration files, project-specific Batch Criteria, and some utility classes which allow SIERRA to translate the --scenario argument passed on the cmdline to sets of expdef changes as part of Experiment definition.

Important

You can't share projects across Engines, so if you want to be able to use your project on ROS and ARGoS (for example), you will need to create 2 separate projects with shared python code imported into each as needed.

Specified via --project on the cmdline.

Raw Output Data#

A csv/arrow/etc file generated as an output from a single Experimental Run. It will (probably) contain a set of columns of representing outputs of interest, with rows corresponding to values captured throughout the run.

ROS1#

You know it. You either love it or hate it, but you can't escape it. The ROS website is at https://wiki.ros.org.

ROS1+Gazebo#

A Engine supported by SIERRA to use ROS1 with the Gazebo simulator.

ROS1+Robot#

A Engine supported by SIERRA using ROS1 and any robot which supports ROS.

Tick#

A timestep. In SIERRA, physics engines will perform some number of iterations per tick, and after that the controllers for all agents will be run.

This Page