compare command

  1. Usage
    1. Compare columns
    2. Column specs
    3. Sorting
    4. Limting results
    5. Specifying runs
    6. Filtering by operation and label
    7. Filtering by run status
    8. Batch runs
  2. Options

Usage

guild compare [OPTIONS] [RUN...]

Compare run results.

Guild Compare is a console based application that displays a table of runs with their current accuracy and loss. The application will continue to run until you exit it by pressing q (for quit).

Guild Compare supports a number of commands. Commands are activated by pressing a letter. To view the list of commands, press ?.

Guild Compare does not automatically update to display the latest available data. If you want to update the list of runs and their status, press r (for refresh).

You may alternative use this command to generate CSV output for run. Use the ‑‑csv option to print data to standard output instead of running as an application. You can redirect this output to a file using:

guild compare ‑‑csv > RUNS.csv

Compare columns

Guild Compare shows columns for each run based on the columns defined for each run operation. Additional columns may be specified using the ‑‑columns option, which must be a comma separated list of column specs. See below for column spec details.

If multiple columns have the same name, they are merged into a single column. Cell values are merged by taking the first non-null value in the list of cells with the common name from left-to-right.

By default, columns always contain run ID, model, operation, started, time, label, status, and the set of columns defined for each displayed operation. You can skip the core columns by with ‑‑skip‑core and skip the operation columns with ‑‑skip‑op‑cols.

Column specs

Each column specified in COLUMNS must be a valid column spec. A column spec is the name of a run flag or scalar key. Flag names must be preceded by an equals sign = to differentiate them from scalar keys.

For example, to include the flag epochs as a column, use ‑‑columns =epochs.

If a scalar is specified, it may be preceded by a qualifier of min, max, first, last, avg, total, or count to indicate the type of scalar value. For example, to include the highest logged value for accuracy, use ‑‑columns "max accuray".

By default last is assumed, so that the last logged value for the specified scalar is used.

A scalar spec may additionally contain the key word step to indicate that the step associated with the scalar is used. For example, to include the step of the last accuracy value, use ‑‑columns "accuracy step". Step may be used with scalar qualifiers. For example, to include the value and associated step of the lowest loss, use ‑‑columns "min loss, min loss step".

Column specs may contain an alternative column heading using the keyword as in the format COL as HEADING. Headings that contain spaces must be quoted.

For example, to include the scalar val_loss with name validation loss, use ‑‑columns val_loss as 'validation loss'.

You may include run attributes as column specs by preceding the run attribute name with a period .. For example, to include the stopped attribute, use ‑‑columns .stopped. This is useful when using ‑‑skip‑core.

Sorting

Use ‑‑min and ‑‑max to sort results by a particular column. ‑‑min sorts in ascending order and ‑‑max sorts in descending order.

When specifying COLUMN, use the column name as displayed in the table output. If the column name contains spaces, quote the value.

By default, runs are sorted by start time in ascending order - i.e. the most recent runs are listed first.

Limting results

To limit the results to the top N runs, use ‑‑top.

Specifying runs

You may use one or more RUN arguments to indicate which runs apply to the command. RUN may be a run ID, a run ID prefix, or a one-based index corresponding to a run returned by the list command.

Indexes may also be specified in ranges in the form START:END where START is the start index and END is the end index. Either START or END may be omitted. If START is omitted, all runs up to END are selected. If END id omitted, all runs from START on are selected. If both START and END are omitted (i.e. the : char is used by itself) all runs are selected.

If a RUN argument is not specified, : is assumed (all runs are selected).

Filtering by operation and label

Runs may be filtered by operation using ‑‑operation. A run is only included if any part of its full operation name, including the package and model name, matches the value.

Use ‑‑label to only include runs with labels matching a specified value.

‑‑operation and ‑‑label may be used multiple times to expand the runs that are included.

Use ‑‑unlabeled to only include runs without labels. This option may not be used with ‑‑label.

Use ‑‑marked to only include marked runs.

Filtering by run status

Runs may also be filtered by specifying one or more status filters: ‑‑running, ‑‑completed, ‑‑error, and ‑‑terminated. These may be used together to include runs that match any of the filters. For example to only include runs that were either terminated or exited with an error, use ‑‑terminated ‑‑error, or the short form ‑ET.

Status filters are applied before RUN indexes are resolved. For example, a run index of 1 is the latest run that matches the status filters.

Batch runs

By default, batch runs are not included in comparisons. To include batch runs, specify ‑‑include‑batch.

Options

-o, --operation VAL

Include runs with operations matching VAL.

-l, --label VAL

Include runs with labels matching VAL.

-u, --unlabeled

Include only runs without labels.

-M, --marked

Include only marked runs.

-U, --unmarked

Include only unmarked runs.

-R, --running

Include only runs that are still running.

-C, --completed

Include only completed runs.

-E, --error

Include only runs that exited with an error.

-T, --terminated

Include only runs terminated by the user.

-P, --pending

Include only pending runs.

-c, --cols COLUMNS

Additional columns to compare. Cannot be used with --strict-columns.

-cc, --strict-cols COLUMNS

Columns to compare. Cannot be used with --columns.

-n, --skip-op-cols

Don't show operation columns.

-r, --skip-core

Don't show core columns.

-t, --top N

Only show the top N runs.

-m, --min COLUMN

Show the lowest values for COLUMN first.

-x, --max COLUMN

Show the highest values for COLUMN first.

--table

Generate comparison data as a table.

--csv

Generate comparison data as a CSV file.

--include-batch

Include batch runs.

--print-scalars

Show available scalars and exit.

--help Show command help and exit.
Guild AI version 0.6.5.dev3