Compare run results.
Guild Compare is a console based application that displays a table
of runs with their current accuracy and loss. The application will
continue to run until you exit it by pressing
q (for quit).
Guild Compare supports a number of commands. Commands are
activated by pressing a letter. To view the list of commands,
Guild Compare does not automatically update to display the latest
available data. If you want to update the list of runs and their
r (for refresh).
You may alternative use this command to generate CSV output for
run. Use the
‑‑csv option to print data to standard output
instead of running as an application. You can redirect this output
to a file using:
guild compare ‑‑csv > RUNS.csv
Guild Compare shows columns for each run based on the columns
defined for each run operation. Additional columns may be
specified using the
‑‑columns option, which must be a comma
separated list of column specs. See below for column spec details.
If multiple columns have the same name, they are merged into a single column. Cell values are merged by taking the first non-null value in the list of cells with the common name from left-to-right.
By default, columns always contain run ID, model, operation,
started, time, label, status, and the set of columns defined for
each displayed operation. You can skip the core columns by with
‑‑skip‑core and skip the operation columns with
Each column specified in
COLUMNS must be a valid column spec. A
column spec is the name of a run flag or scalar key. Flag names
must be preceded by an equals sign
= to differentiate them
from scalar keys.
For example, to include the flag
epochs as a column, use
If a scalar is specified, it may be preceded by a qualifier of
indicate the type of scalar value. For example, to include the
highest logged value for
last is assumed, so that the last logged value for
the specified scalar is used.
A scalar spec may additionally contain the key word
indicate that the step associated with the scalar is used. For
example, to include the step of the last
accuracy value, use
‑‑columns "accuracy step". Step may be used with scalar
qualifiers. For example, to include the value and associated step
of the lowest loss, use
‑‑columns "min loss, min loss step".
Column specs may contain an alternative column heading using the
as in the format
COL as HEADING. Headings that
contain spaces must be quoted.
For example, to include the scalar
val_loss with name
validation loss, use
‑‑columns val_loss as 'validation
You may include run attributes as column specs by preceding the
run attribute name with a period
.. For example, to include
stopped attribute, use
‑‑columns .stopped. This is
useful when using
‑‑max to sort results by a particular
‑‑min sorts in ascending order and
‑‑max sorts in
COLUMN, use the column name as displayed in the
table output. If the column name contains spaces, quote the value.
By default, runs are sorted by start time in ascending order - i.e. the most recent runs are listed first.
To limit the results to the top
N runs, use
You may use one or more
RUN arguments to indicate which runs
apply to the command.
RUN may be a run ID, a run ID prefix, or a
one-based index corresponding to a run returned by the list
Indexes may also be specified in ranges in the form
START is the start index and
END is the end
END may be omitted. If
omitted, all runs up to
END are selected. If
END id omitted,
all runs from
START on are selected. If both
are omitted (i.e. the
: char is used by itself) all runs are
RUN argument is not specified,
: is assumed (all runs
Filtering by operation and label
Runs may be filtered by operation using
‑‑operation. A run is
only included if any part of its full operation name, including
the package and model name, matches the value.
‑‑label to only include runs with labels matching a
‑‑label may be used multiple times to expand
the runs that are included.
‑‑unlabeled to only include runs without labels. This option
may not be used with
‑‑marked to only include marked runs.
Filtering by run status
Runs may also be filtered by specifying one or more status
‑‑terminated. These may be used together to include runs that
match any of the filters. For example to only include runs that
were either terminated or exited with an error, use
‑‑error, or the short form
Status filters are applied before
RUN indexes are resolved. For
example, a run index of
1 is the latest run that matches the
By default, batch runs are not included in comparisons. To include
batch runs, specify
|-o, --operation VAL||
Include runs with operations matching
|-l, --label VAL||
Include runs with labels matching
Include only runs without labels.
Include only marked runs.
Include only unmarked runs.
Include only runs that are still running.
Include only completed runs.
Include only runs that exited with an error.
Include only runs terminated by the user.
Include only pending runs.
|-c, --cols COLUMNS||
Additional columns to compare. Cannot be used with --strict-columns.
|-cc, --strict-cols COLUMNS||
Columns to compare. Cannot be used with --columns.
Don't show operation columns.
Don't show core columns.
|-t, --top N||
Only show the top N runs.
|-m, --min COLUMN||
Show the lowest values for COLUMN first.
|-x, --max COLUMN||
Show the highest values for COLUMN first.
Generate comparison data as a table.
Generate comparison data as a CSV file.
Include batch runs.
Show available scalars and exit.
|--help||Show command help and exit.|