Pipeline Stages: Deep Dive

This page dives deeps into the high level design of each of the SIERRA pipeline stages. You don't need to know any of this in order to use SIERRA successfully, but it doesn't hurt.

Stage 1

To begin, take note of the following terminology:

• There are \(i\) Experiments in a Batch Experiment.

• There are \(j\) experimental runs in an experiment (for a total of \(i\times{j}\) executable experimental runs).

• The "Generator Factory" abstractly refers to the various parts of the code that map cmdline arguments from strings into Generators. Generators are objects which can make modications to expdef files: removing elements, adding elements, and modifying element attributes. See Experiment Definition (--expdef) for for details.

Below is the execution flow of the first phase of stage 1: scaffolding the batch experiment. The outputs of this phase include:

• A unique directory for each experiment in the batch experiment containing all inputs.

• A modified expdef template file unique to each experiment in the batch which contains modifications to the original --expdef-template from the --batch-criteria This file (probably) cannot be actually run by the selected engine (i.e., it is not well formed, yet).

Below is the execution flow of the second phase of stage 1: generating each experiment within the parent batch experiment, using the generated per-experiment template from the previous phase. The outputs from this phase include:

• A unique directory for each Experimental Run in each experiment containing all inputs. These are files which can actually be run by the selected engine.

• A modified expdef template file unique to each experimental run in the experiment containing all modifications that SIERRA was directed to make through a combination of:

  • --controller

  • --scenario

  • --batch-criteria

  • --engine

  • --project

  • controllers.yaml