Data Decompression

When dealing with Projects which produce huge amounts of data, it is easy to blow out allocated storage with uncompressed data if you run lots of Batch Experiments. Thus, data for such projects is often compressed after being written (by the Engine, Project, or some other source). Before it can be processed, data for a given batch experiment needs to be decompressed; that's where this plugin comes in.

This plugin processes at the file level for each Experimental Run. All matching archive types are decompressed. Currently supports:

• .tar.gz

Ordering Considerations

Must precede any plugin which reads raw outputs if those outputs were previously compressed with Data Compression.

Usage

This plugin can be selected by adding proc.decompress to the list passed to --proc.

Cmdline Interface

No additional cmdline options.